eliott/anything-llm
1
0
Fork 0
mirror of https://github.com/Mintplex-Labs/anything-llm synced 2026-04-25 17:15:37 +02:00
Code Issues Packages Projects Releases Wiki Activity

Microsoft Foundry Local LLM provider & agent provider (#4435)

Browse Source 
* add microsoft foundry local llm and agent providers

* minor change to fix early stop token + overloading of context window
always use user defined window _unless_ it is larger than the models real contenxt window
cache the context windows when we can from the API (0.7.*)+
Unload model forcefully on model change to prevent resource hogging

* add back token preference since some models have very large windows and can crash a machine
normalize cases

---------

Co-authored-by: Timothy Carambat <rambat1010@gmail.com>
This commit is contained in:
Sean Hatfield
2025-10-01 20:04:13 -07:00
committed by GitHub
parent 8f0f9df4fc
commit 599a3fd8b8
17 changed files with 626 additions and 3 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
server/.env.example
View File

@@ -143,6 +143,11 @@ SIG_SALT='salt' # Please generate random string at least 32 chars long.
# MOONSHOT_AI_API_KEY='your-moonshot-api-key-here'
# MOONSHOT_AI_MODEL_PREF='moonshot-v1-32k'
# LLM_PROVIDER='foundry'
# FOUNDRY_BASE_PATH='http://127.0.0.1:55776'
# FOUNDRY_MODEL_PREF='phi-3.5-mini'
# FOUNDRY_MODEL_TOKEN_LIMIT=4096
###########################################
######## Embedding API SElECTION ##########
###########################################