docs: swap Ollama example for Infron LLM gateway in custom LLM provider doc (#1477)

* docs: swap Ollama functional example for Infron LLM gateway

Introduces the LLM gateway concept (single OpenAI-compatible endpoint,
unified billing, fallbacks), replaces the Ollama walkthrough with an
Infron config, and links out to Infron's signup + quickstart per their
reciprocal doc in their OpenWork page.

* docs: frame Infron as hosted gateway vs LiteLLM starting point

Link to Infron's LiteLLM integration page as the canonical open-source
gateway reference, and position Infron as the OpenRouter-style hosted
gateway that sits on top: one key, unified billing, provider fallbacks.

* docs: drop OpenRouter comparison, pitch Infron directly for dev ergonomics

* docs: drop em dashes, loosen Infron pitch to devrel voice

* docs: add Infron model picker + active session screenshots
This commit is contained in:
Jan Carbonell
2026-04-17 12:20:32 -07:00
committed by GitHub
parent 85ab73bcbe
commit 591dd2f6da
3 changed files with 28 additions and 27 deletions

View File

@@ -68,63 +68,64 @@ The JSON must include `id`, `name`, `npm`, `env`, `doc`, and `models`. `api` is
3. Under `Cloud providers`, click `Import`.
4. Reload the workspace when OpenWork asks.
### Functional example: Ollama (Qwen3 8B)
### Functional example: an LLM gateway (Infron)
This setup a local [Ollama](https://docs.ollama.com/api/openai-compatibility) instance running `qwen3:8b`.
Ollama's OpenAI-compatible endpoint requires an API key but ignores the value, so paste anything (for example `ollama`) into the `API key / credential` field when creating the provider.
An LLM gateway is one OpenAI-compatible endpoint that fans out to many model providers. If you want to run the routing yourself, [LiteLLM](https://infron.ai/docs/frameworks-and-integrations/litellm) is a good starting point. [Infron](https://infron.ai) is the hosted option: one API key gets you every model in its [marketplace](https://infron.ai/models) with automatic provider fallbacks and a single invoice, so adding a new LLM to OpenWork is just another entry under `models`.
<Frame>
![Config to add Ollama as a custom provider](/images/ollama-custom-provider.png)
</Frame>
Grab a key from the [API Keys dashboard](https://infron.ai/dashboard/apiKeys). If you don't have an account yet, sign up at [infron.ai/login](https://infron.ai/login) and their [quickstart](https://infron.ai/docs/overview/quickstart/text) walks you through your first request.
Paste the key into `API key / credential` and use this JSON:
```json
{
"id": "ollama",
"name": "Ollama",
"id": "infron",
"name": "Infron",
"npm": "@ai-sdk/openai-compatible",
"env": [
"OLLAMA_API_KEY"
"INFRON_API_KEY"
],
"doc": "https://docs.ollama.com/api/openai-compatibility",
"api": "http://localhost:11434/v1",
"doc": "https://infron.ai/docs/frameworks-and-integrations/openwork",
"api": "https://llm.onerouter.pro/v1",
"models": [
{
"id": "qwen3:8b",
"name": "Qwen3 8B",
"id": "deepseek/deepseek-v3.2",
"name": "DeepSeek V3.2",
"attachment": false,
"reasoning": true,
"tool_call": true,
"structured_output": true,
"temperature": true,
"release_date": "2025-04-28",
"last_updated": "2025-04-28",
"release_date": "2025-09-29",
"last_updated": "2025-09-29",
"open_weights": true,
"limit": {
"context": 32768,
"input": 32768,
"context": 128000,
"input": 128000,
"output": 8192
},
"modalities": {
"input": [
"text"
],
"output": [
"text"
]
"input": ["text"],
"output": ["text"]
}
}
]
}
```
Once imported, Infron models show up in the Chat model picker alongside everything else:
<Frame>
![Ollama added as a provider](/images/ollama-added-as-provider.png)
![DeepSeek V3.2 via Infron in the OpenWork Chat model picker](/images/infron-model-picker.png)
</Frame>
Pull the model first with `ollama pull qwen3:8b`, then make sure the Ollama server is reachable at `http://localhost:11434` with `ollama serve`
Pick the model and it's live in the session footer:
Import it in your desktop, change the provider name and voila! You have an fully local LLM running in your machine.
<Frame>
![Infron · DeepSeek V3.2 selected in an OpenWork session](/images/infron-model-active.png)
</Frame>
Add more entries under `models` to expose other routes the gateway supports (e.g. `openai/gpt-5.4`, `google/gemini-2.5-flash`).
## When to use a cloud provider
We recommend using Cloud-bsaed provider when the setup is meant to be shared across an org or team. Otherwise, doing it directly in desktop is easier.
Use a Cloud provider when the setup is meant to be shared across an org or team. For solo use, configuring it directly in the desktop app is simpler.

Binary file not shown.

After

Width:  |  Height:  |  Size: 384 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 487 KiB