diff --git a/packages/docs/cloud/share-with-your-team/custom-llm-provider.mdx b/packages/docs/cloud/share-with-your-team/custom-llm-provider.mdx index b060cf5d..d190ad4e 100644 --- a/packages/docs/cloud/share-with-your-team/custom-llm-provider.mdx +++ b/packages/docs/cloud/share-with-your-team/custom-llm-provider.mdx @@ -68,63 +68,64 @@ The JSON must include `id`, `name`, `npm`, `env`, `doc`, and `models`. `api` is 3. Under `Cloud providers`, click `Import`. 4. Reload the workspace when OpenWork asks. -### Functional example: Ollama (Qwen3 8B) +### Functional example: an LLM gateway (Infron) -This setup a local [Ollama](https://docs.ollama.com/api/openai-compatibility) instance running `qwen3:8b`. -Ollama's OpenAI-compatible endpoint requires an API key but ignores the value, so paste anything (for example `ollama`) into the `API key / credential` field when creating the provider. +An LLM gateway is one OpenAI-compatible endpoint that fans out to many model providers. If you want to run the routing yourself, [LiteLLM](https://infron.ai/docs/frameworks-and-integrations/litellm) is a good starting point. [Infron](https://infron.ai) is the hosted option: one API key gets you every model in its [marketplace](https://infron.ai/models) with automatic provider fallbacks and a single invoice, so adding a new LLM to OpenWork is just another entry under `models`. - - ![Config to add Ollama as a custom provider](/images/ollama-custom-provider.png) - +Grab a key from the [API Keys dashboard](https://infron.ai/dashboard/apiKeys). If you don't have an account yet, sign up at [infron.ai/login](https://infron.ai/login) and their [quickstart](https://infron.ai/docs/overview/quickstart/text) walks you through your first request. + +Paste the key into `API key / credential` and use this JSON: ```json { - "id": "ollama", - "name": "Ollama", + "id": "infron", + "name": "Infron", "npm": "@ai-sdk/openai-compatible", "env": [ - "OLLAMA_API_KEY" + "INFRON_API_KEY" ], - "doc": "https://docs.ollama.com/api/openai-compatibility", - "api": "http://localhost:11434/v1", + "doc": "https://infron.ai/docs/frameworks-and-integrations/openwork", + "api": "https://llm.onerouter.pro/v1", "models": [ { - "id": "qwen3:8b", - "name": "Qwen3 8B", + "id": "deepseek/deepseek-v3.2", + "name": "DeepSeek V3.2", "attachment": false, "reasoning": true, "tool_call": true, "structured_output": true, "temperature": true, - "release_date": "2025-04-28", - "last_updated": "2025-04-28", + "release_date": "2025-09-29", + "last_updated": "2025-09-29", "open_weights": true, "limit": { - "context": 32768, - "input": 32768, + "context": 128000, + "input": 128000, "output": 8192 }, "modalities": { - "input": [ - "text" - ], - "output": [ - "text" - ] + "input": ["text"], + "output": ["text"] } } ] } ``` +Once imported, Infron models show up in the Chat model picker alongside everything else: + - ![Ollama added as a provider](/images/ollama-added-as-provider.png) + ![DeepSeek V3.2 via Infron in the OpenWork Chat model picker](/images/infron-model-picker.png) -Pull the model first with `ollama pull qwen3:8b`, then make sure the Ollama server is reachable at `http://localhost:11434` with `ollama serve` +Pick the model and it's live in the session footer: -Import it in your desktop, change the provider name and voila! You have an fully local LLM running in your machine. + + ![Infron ยท DeepSeek V3.2 selected in an OpenWork session](/images/infron-model-active.png) + + +Add more entries under `models` to expose other routes the gateway supports (e.g. `openai/gpt-5.4`, `google/gemini-2.5-flash`). ## When to use a cloud provider -We recommend using Cloud-bsaed provider when the setup is meant to be shared across an org or team. Otherwise, doing it directly in desktop is easier. +Use a Cloud provider when the setup is meant to be shared across an org or team. For solo use, configuring it directly in the desktop app is simpler. diff --git a/packages/docs/images/infron-model-active.png b/packages/docs/images/infron-model-active.png new file mode 100644 index 00000000..d4c604c7 Binary files /dev/null and b/packages/docs/images/infron-model-active.png differ diff --git a/packages/docs/images/infron-model-picker.png b/packages/docs/images/infron-model-picker.png new file mode 100644 index 00000000..7e233e0c Binary files /dev/null and b/packages/docs/images/infron-model-picker.png differ