mirror of
https://github.com/different-ai/openwork
synced 2026-04-25 17:15:34 +02:00
docs: swap Ollama example for Infron LLM gateway in custom LLM provider doc (#1477)
* docs: swap Ollama functional example for Infron LLM gateway Introduces the LLM gateway concept (single OpenAI-compatible endpoint, unified billing, fallbacks), replaces the Ollama walkthrough with an Infron config, and links out to Infron's signup + quickstart per their reciprocal doc in their OpenWork page. * docs: frame Infron as hosted gateway vs LiteLLM starting point Link to Infron's LiteLLM integration page as the canonical open-source gateway reference, and position Infron as the OpenRouter-style hosted gateway that sits on top: one key, unified billing, provider fallbacks. * docs: drop OpenRouter comparison, pitch Infron directly for dev ergonomics * docs: drop em dashes, loosen Infron pitch to devrel voice * docs: add Infron model picker + active session screenshots
This commit is contained in:
@@ -68,63 +68,64 @@ The JSON must include `id`, `name`, `npm`, `env`, `doc`, and `models`. `api` is
|
||||
3. Under `Cloud providers`, click `Import`.
|
||||
4. Reload the workspace when OpenWork asks.
|
||||
|
||||
### Functional example: Ollama (Qwen3 8B)
|
||||
### Functional example: an LLM gateway (Infron)
|
||||
|
||||
This setup a local [Ollama](https://docs.ollama.com/api/openai-compatibility) instance running `qwen3:8b`.
|
||||
Ollama's OpenAI-compatible endpoint requires an API key but ignores the value, so paste anything (for example `ollama`) into the `API key / credential` field when creating the provider.
|
||||
An LLM gateway is one OpenAI-compatible endpoint that fans out to many model providers. If you want to run the routing yourself, [LiteLLM](https://infron.ai/docs/frameworks-and-integrations/litellm) is a good starting point. [Infron](https://infron.ai) is the hosted option: one API key gets you every model in its [marketplace](https://infron.ai/models) with automatic provider fallbacks and a single invoice, so adding a new LLM to OpenWork is just another entry under `models`.
|
||||
|
||||
<Frame>
|
||||

|
||||
</Frame>
|
||||
Grab a key from the [API Keys dashboard](https://infron.ai/dashboard/apiKeys). If you don't have an account yet, sign up at [infron.ai/login](https://infron.ai/login) and their [quickstart](https://infron.ai/docs/overview/quickstart/text) walks you through your first request.
|
||||
|
||||
Paste the key into `API key / credential` and use this JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "ollama",
|
||||
"name": "Ollama",
|
||||
"id": "infron",
|
||||
"name": "Infron",
|
||||
"npm": "@ai-sdk/openai-compatible",
|
||||
"env": [
|
||||
"OLLAMA_API_KEY"
|
||||
"INFRON_API_KEY"
|
||||
],
|
||||
"doc": "https://docs.ollama.com/api/openai-compatibility",
|
||||
"api": "http://localhost:11434/v1",
|
||||
"doc": "https://infron.ai/docs/frameworks-and-integrations/openwork",
|
||||
"api": "https://llm.onerouter.pro/v1",
|
||||
"models": [
|
||||
{
|
||||
"id": "qwen3:8b",
|
||||
"name": "Qwen3 8B",
|
||||
"id": "deepseek/deepseek-v3.2",
|
||||
"name": "DeepSeek V3.2",
|
||||
"attachment": false,
|
||||
"reasoning": true,
|
||||
"tool_call": true,
|
||||
"structured_output": true,
|
||||
"temperature": true,
|
||||
"release_date": "2025-04-28",
|
||||
"last_updated": "2025-04-28",
|
||||
"release_date": "2025-09-29",
|
||||
"last_updated": "2025-09-29",
|
||||
"open_weights": true,
|
||||
"limit": {
|
||||
"context": 32768,
|
||||
"input": 32768,
|
||||
"context": 128000,
|
||||
"input": 128000,
|
||||
"output": 8192
|
||||
},
|
||||
"modalities": {
|
||||
"input": [
|
||||
"text"
|
||||
],
|
||||
"output": [
|
||||
"text"
|
||||
]
|
||||
"input": ["text"],
|
||||
"output": ["text"]
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Once imported, Infron models show up in the Chat model picker alongside everything else:
|
||||
|
||||
<Frame>
|
||||

|
||||

|
||||
</Frame>
|
||||
|
||||
Pull the model first with `ollama pull qwen3:8b`, then make sure the Ollama server is reachable at `http://localhost:11434` with `ollama serve`
|
||||
Pick the model and it's live in the session footer:
|
||||
|
||||
Import it in your desktop, change the provider name and voila! You have an fully local LLM running in your machine.
|
||||
<Frame>
|
||||

|
||||
</Frame>
|
||||
|
||||
Add more entries under `models` to expose other routes the gateway supports (e.g. `openai/gpt-5.4`, `google/gemini-2.5-flash`).
|
||||
|
||||
## When to use a cloud provider
|
||||
|
||||
We recommend using Cloud-bsaed provider when the setup is meant to be shared across an org or team. Otherwise, doing it directly in desktop is easier.
|
||||
Use a Cloud provider when the setup is meant to be shared across an org or team. For solo use, configuring it directly in the desktop app is simpler.
|
||||
|
||||
BIN
packages/docs/images/infron-model-active.png
Normal file
BIN
packages/docs/images/infron-model-active.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 384 KiB |
BIN
packages/docs/images/infron-model-picker.png
Normal file
BIN
packages/docs/images/infron-model-picker.png
Normal file
Binary file not shown.
|
After Width: | Height: | Size: 487 KiB |
Reference in New Issue
Block a user