---
title: Alternative backends
---

# Alternative backends

import { Callout, Steps } from 'nextra/components';
import Ort from '../../components/Ort';

Since [ONNX Runtime](https://onnxruntime.ai/) is written in C++, linking troubles often arise when attempting to use it in a Rust project--especially with WASM. `v2.0.0-rc.12` of <Ort/> introduced support for **alternative backends**: ONNX runtimes that aren't *the* ONNX Runtime.

As the Rust ML scene has evolved, many exciting new inference engines supporting ONNX models have popped up, like 🤗 Hugging Face's [`candle`](https://github.com/huggingface/candle), [Burn](https://github.com/tracel-ai/burn), and [`tract`](https://github.com/sonos/tract). These libraries, being written in pure Rust, play much nicer when it comes to linking, and often support any platform Rust's standard library does (which is a lot more than ONNX Runtime!) They're also, of course, memory safe and 🦀blazingly🔥fast🚀

<Ort/> alternative backends are simply wrappers that implements the ONNX Runtime C API over another crate. Because they implement the same exact API as ONNX Runtime, using them in <Ort/> is as simple as adding one line of code!

## Using an alternative backend

<Callout type='warning'>
    Alternative backends are experimental, and are constantly changing and growing -- use them at your own risk!

    We may not be able to provide the same level of support for different backends as we do with ONNX Runtime.
</Callout>

<Steps>

### Install the alternative backend
We'll use [`ort-tract`](/backends/tract) for this example.

```toml filename="Cargo.toml"
[dependencies]
ort-tract = "0.3.0+0.22"
...
```

### Enable the `alternative-backend` feature
This instructs <Ort/> to not try to download/link to ONNX Runtime.

```toml filename="Cargo.toml"
[dependencies.ort]
version = "=2.0.0-rc.12"
default-features = false # Disables the `download-binaries` feature since we don't need it
features = [
    "std",
    "ndarray",
    "alternative-backend"
]
```

### Initialize the backend
Use [`ort::set_api`](https://docs.rs/ort/latest/ort/fn.set_api.html) to use the crate's API implementation (replacing `ort_tract` with whichever backend crate you choose to use):

```rs
fn main() {
    // This should run as early in your application as possible - before you ever use `ort`!
    ort::set_api(ort_tract::api());
}
```

### Done!
<Callout type='info'>
    Be sure to check each backend's docs page to see which APIs are and are not supported.
</Callout>

</Steps>

## Available backends
<Ort/> currently has the following backends:

- [`ort-candle`](/backends/candle), based on [🤗 Hugging Face `candle`](https://github.com/huggingface/candle)
    - 🔷 **Supports**: CPU, CUDA (though not available via `ort-candle` right now), WebAssembly
    - ⚠️ Limited operator support; though most transformer models have good support.
- [`ort-tract`](/backends/tract), based on [`tract`](https://github.com/sonos/tract)
    - 🔷 **Supports**: CPU, WebAssembly
    - ✅ [Great operator support](https://github.com/sonos/tract?tab=readme-ov-file#onnx)
- [`ort-web`](/backends/web) runs ONNX Runtime in the web
    - 🔷 **Supports**: WebAssembly (with WebGL & WebGPU backends!)
    - ✅ Great operator support - *it's the full ONNX Runtime!*
