mirror of
https://github.com/pykeio/ort
synced 2026-04-25 16:34:55 +02:00
69 lines
1.8 KiB
Plaintext
69 lines
1.8 KiB
Plaintext
import { Steps } from 'nextra/components';
|
|
import Ort from '../../components/Ort';
|
|
|
|
# `ort-candle`
|
|
`ort-candle` is an [alternative backend](/backends) for <Ort/> based on [🤗 Hugging Face `candle`](https://github.com/huggingface/candle).
|
|
|
|
## Supported APIs
|
|
- ✅ `ort::init`
|
|
- 🔷 `ort::environment::EnvironmentBuilder`
|
|
- `EnvironmentBuilder::commit`
|
|
- 🔷 `ort::memory::Allocator`
|
|
- `Allocator::default`
|
|
- `Allocator::memory_info`
|
|
- ✅ `ort::memory::MemoryInfo`
|
|
- 🔷 `ort::session::Session`
|
|
- `Session::builder`
|
|
- `Session::allocator`
|
|
- `Session::run`
|
|
- `Session::run_with_options`
|
|
- 🔷 `ort::session::builder::SessionBuilder`
|
|
- `SessionBuilder::new`
|
|
- `SessionBuilder::commit_from_file`
|
|
- `SessionBuilder::commit_from_memory`
|
|
- `SessionBuilder::commit_from_memory_directly`
|
|
- `SessionBuilder::commit_from_url`
|
|
- ✅ `ort::value::DynValue`, `ort::value::DynValueRef`, `ort::value::DynValueRefMut`
|
|
- Only `Tensor` types are supported.
|
|
- ✅ `ort::value::Tensor`, `TensorRef`, `TensorRefMut`, etc.
|
|
- ✅ `ort::value::ValueType`
|
|
|
|
## Usage
|
|
|
|
<Steps>
|
|
|
|
### Install `ort-candle`
|
|
```toml filename="Cargo.toml"
|
|
[dependencies]
|
|
ort-candle = "0.3.0+0.9.2"
|
|
...
|
|
```
|
|
|
|
### Enable the `alternative-backend` feature
|
|
This instructs <Ort/> to not try to download/link to ONNX Runtime.
|
|
|
|
```toml filename="Cargo.toml"
|
|
[dependencies.ort]
|
|
version = "=2.0.0-rc.12"
|
|
default-features = false # Disables the `download-binaries` feature since we don't need it
|
|
features = [
|
|
"std",
|
|
"ndarray",
|
|
"alternative-backend"
|
|
]
|
|
```
|
|
|
|
### Initialize the backend
|
|
Use [`ort::set_api`](https://docs.rs/ort/latest/ort/fn.set_api.html) to use the crate's API implementation.
|
|
|
|
```rs filename="main.rs"
|
|
fn main() {
|
|
// This should run as early in your application as possible - before you ever use `ort`!
|
|
ort::set_api(ort_candle::api());
|
|
}
|
|
```
|
|
|
|
### Done!
|
|
|
|
</Steps>
|