mirror of
https://github.com/Mintplex-Labs/anything-llm
synced 2026-04-25 17:15:37 +02:00
Enable real-time agent tool call streaming for all providers (#4279)
* WIP agentic tool call streaming - OpenAI - Anthropic - Azure OpenAI * WIP rest of providers EXCLUDES Bedrock and GenericOpenAI * patch untooled complete/streaming to use chatCallback provider from provider class and not assume OpenAI client struct example: Ollama * modify ollama to function with its own overrides normalize completion/stream outputs across providers/untooled * dev build * fix message sanization for anthropic agent streaming * wip fix anthropic agentic streaming sanitization * patch gemini, webgenui, generic aibitat providers + disable providers unable to test * refactor anthropic aibitat provider for empty message and tool call formatting * Add frontend missing prop check update Azure for streaming support update Gemini to streamting support on gemini-* models generic OpenAI disable streaming verify localAI support verify NVIDIA Nim support * DPAIS, remove temp from call, support streaming' * remove 0 temp to remove possibility of bad temp error/500s/400s * Patch condition where model is non-streamable and no tools are present or called resulting in the provider `handleFunctionCallChat` being called - which returns a string. This would then fail in Untooled.complete since response would be a string and not the expected `response.choices?.[0]?.message` Modified this line to handle both conditions for stream/non-streaming and tool presence or lack thereof * Allow generic Openai to be streamable since using untooled it should work fine honor disabled streaming for provider where that concern may apply for regular chats * rename function and more gemini-specific function to gemini provider * add comments for readability .complete on azure should be non-streaming as this is the sync response * migrate CometAPI, but disable as we cannot test --------- Co-authored-by: shatfield4 <seanhatfield5@gmail.com>
This commit is contained in:
2
.github/workflows/dev-build.yaml
vendored
2
.github/workflows/dev-build.yaml
vendored
@@ -6,7 +6,7 @@ concurrency:
|
||||
|
||||
on:
|
||||
push:
|
||||
branches: ['3999-chromium-flags'] # put your current branch to create a build. Core team only.
|
||||
branches: ['agentic-streaming'] # put your current branch to create a build. Core team only.
|
||||
paths-ignore:
|
||||
- '**.md'
|
||||
- 'cloud-deployments/*'
|
||||
|
||||
Reference in New Issue
Block a user