[FEAT] Add support for Voyage AI embedder (#1401)

* add support for voyageai embedder

* remove unneeded import

* linting

* Add ENV examples
Update how chunks are processed for Voyage
use correct langchain import
Add data handling

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
This commit is contained in:
Sean Hatfield
2024-05-19 11:20:23 -07:00
committed by GitHub
parent ecd5d3cb8a
commit 5bf4b4db58
12 changed files with 150 additions and 16 deletions

View File

@@ -121,6 +121,10 @@ JWT_SECRET="my-random-string-for-seeding" # Please generate random string at lea
# COHERE_API_KEY=
# EMBEDDING_MODEL_PREF='embed-english-v3.0'
# EMBEDDING_ENGINE='voyageai'
# VOYAGEAI_API_KEY=
# EMBEDDING_MODEL_PREF='voyage-large-2-instruct'
###########################################
######## Vector Database Selection ########
###########################################