* Improve DMR support
- Autodetect models installed
- Grab all models from hub.docker to show available
- UI to handle render,search, install, and management of models
- Support functionality for chat, stream, and agentic calls
* forgot files
* fix loader circle being too large
fix tooltip width command
adjust location of docker installer open for web platform
* adjust imports
* remove create workspace step for onboarding
* remove unused image
* workspace creation into dedicated useEffect + use translated workspace name
* dev tag
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Migrate Astra to class (#4722)
migrate astra to class
* Migrate LanceDB to class (#4721)
migrate lancedb to class
* Migrate Pinecone to class (#4726)
migrate pinecone to class
* Migrate Zilliz to class (#4729)
migrate zilliz to class
* Migrate Weaviate to class (#4728)
migrate weaviate to class
* Migrate Qdrant to class (#4727)
migrate qdrant to class
* Migrate Milvus to class (#4725)
migrate milvus to class
* Migrate Chroma to class (#4723)
migrate chroma to class
* Migrate Chroma Cloud to class (#4724)
* migrate chroma to class
* migrate chroma cloud to class
* move limits to class field
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Migrate PGVector to class (#4730)
* migrate pgvector to class
* patch pgvector test
* convert connectionString, tableName, and validateConnection to static methods
* move instance properties to class fields
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Refactor Zilliz Cloud vector DB provider (#4749)
simplify zilliz implementation by using milvus as base class
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* VectorDatabase base class (#4738)
create generic VectorDatabase base class
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Extend VectorDatabase base class to all providers (#4755)
extend VectorDatabase base class to all providers
* patch lancedb import
* breakout name and add generic logger
* dev tag build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Refactor non-local LLM Provider, Vector Database, and Embedding Engine privacy information to use their policy URLs instead of descriptions
* Update LLM Provider, Embedding Engine, and Vector Database sections to include privacy policy links
* fix broken links, lint
* Update AstraDB privacy policy URL in onboarding flow
* Refactor AnythingLLM Privacy & Data page to show managed provider privacy policy URLs
* Update Mistral privacy policy URLs in onboarding flow for consistency
* Abstract privacy policies of providers into a reusable component | Refactor Privacy & Data Handling Step of onboarding flow to focus on solely rendering that step | Move provider privacy policy maps into constants.js
* Remove commented-out code for third-party provider privacy policies in Privacy and Data Handling component
* Update privacy policy descriptions for consistency by adding periods at the end of sentences in ProviderPrivacy component and constants.js
* rescope constants for providers
* extract default to external function, add loading state
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* refactor localWhisper to use new custom FFMPEGWrapper class
* stub tests in github actions
* add back wavefile conversion to 16khz 32f to fix docker builds
* use afterEach for cleanup in ffmpeg tests
* remove unused FFMPEG_PATH env check
* use spawnSync for ffmpeg to capture and log output
* lint
* revert removal of try/catch around validateAudioFile for more helpful error msgs
* use readFileSync instead of createReadStream for less overhead
* change import to require for fix-path and stub import in tests
* refactor to singleton to preserve ffmpeg path
dev build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Add local ESLint configuration and disable rules to allow for errorless state
* Remove unnecessary ESLint disable comments in AuthContext and usePromptInputStorage for cleaner code.
* Update eslint-plugin-react-hooks
* Configure prettier to work with eslint
* Removed trailing commas from eslint config
* Prettier to source code
* add a v2 lint script
* put back eslint-disable comments
* fix eslinter and prettier application
always apply --fix since we --write prettier, otherwise it fails
* precaution dev build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Refactor LLMPerformanceMonitor to use options object for measureStream parameters
* Refactor invocations of `measureStream` to use options arguments
* Change invocation of `measureStream` in anthropic provider to use options argument
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add model tag to chatCompletion
* add modelTag `model` to async streaming
keeps default arguments for prompt token calculation where applied via explict arg
* fix HF default arg
* render all performance metrics as available for backward compatibility
add `timestamp` to both sync/async chat methods
* extract metrics string to function
* implement cohere agent support
* run yarn lint
* moderize Cohere
add supported langchain method
redo streaming since it was not working
looping of agent calls was not functioning
* change default model to real model tag
add case statement for model tag
* remove debug
* update default
* only whitelist known labels
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Enhance OllamaEmbedder to support authentication by adding an authorization token in headers for client initialization.
* Add optional Auth Token input for Ollama embedding options
* move info elements
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* add refresh user functionality
* prettier
* add eslint disable comment for exhaustive-deps warning in AuthContext to stop nagging about navigate func
* remove unused imports and fix typo
* handle unsafe parse of undefined for in-session user deleted
* Refactor refreshUser function to handle errors and return structured response. Update AuthProvider to manage user data based on success status.
* Remove console error logging from promise catch in System model for cleaner error handling.
* change status from 404 to 400 and valid to success
* Refactor error handling in AuthProvider's refreshUser logic to remove redundant catch block and streamline user session management on failure.
* prettier
* reorder clauses - return errors
* refactor
account for all user modes
dev build
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* iterate over all pages in paperless-ngx data connector
* add error handling and data validation
* refactor to handle edge cases and null values
* catch edge case to prevent infinite loop
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* Refactor Azure OpenAI integration to use OpenAI SDK and the v1 API | Enable streaming for Azure Open AI basic inference provider
* Add info tooltip to inform user about 'Model Type' form field
* Add 'model_type_tooltip' key to multiple language translations
* Validate AZURE_OPENAI_ENDPOINT in provider construction
* remove unused import, update error handler, rescope URL utils
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* test base image bump
* patch frontend compilation due to Vite QEMU exception
* migrate all runners to arm64 to avoid QEMU slowdown
* add back layer cleanup
* remove layer cache fix
* add refresh user functionality
* prettier
* add eslint disable comment for exhaustive-deps warning in AuthContext to stop nagging about navigate func
* remove unused imports and fix typo
* handle unsafe parse of undefined for in-session user deleted
---------
Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
* fix astra not showing similarity score in citations ui
* fix qdrant not showing similarity score in citations ui
* fix weaviate not showing similarity score in citations ui
