Breaking because `extract_tensor_*` now returns `&[i64]` for dimensions, and `dtype()` and `memory_info()` also return references.
Each tensor extract call not only had multiple FFI calls to determine the `ValueType`, but also had to determine `MemoryInfo` to ensure the data was CPU-accessible. Since neither the data type or memory location can *change* for a given value, it doesn't make sense to compute this on each extract call; it's better to compute it once, when we create the `Value` (and we often already have the types created by this time, so little FFI is actually required).
This should make `extract_tensor_raw` zero-alloc, most benefitting usages of `IoBinding`/`OutputSelector`. This does mean usages of `Value` without ever extracting said value (like HF Transformers hidden state outputs which go ignored) incur slightly more overhead, but the tradeoff of having less overhead at extraction time seems worth it.
Many of the ORT APIs are hardcoded to actually *never* return a failure status, despite returning `*mut OrtStatus`, probably for consistency (which is appreciated!)
For `ort`, this means almost every API requires a ? or unwrap, even if it will never fail. Notably, this includes all functions on `MemoryInfo`.
This commit makes many of these functions return `T` instead of `Result<T>`.
Additionally, I found that all of the `downcast` methods on `Value` just panic if the type is not downcastable - that's bad! Now a nice `Display` is implemented for `ValueType`, and the downcast functions actually return errors. Extraction methods also have better error messages.
Also in this commit: checking if memory is CPU accessible the *correct* way instead of matching AllocationDevices.
Shaves off the `thiserror` dependency and should improve compile times slightly.
Unfortunately this does mean we can't match on `Error` anymore, though I'm not sure if that was ever useful to begin with.
when auto-importing, rust-analyzer seems to only do absolute imports on odd-numbered days and Tuesdays, falling back to crate:: imports otherwise. this commit adds some consistency.
aka The Cleanening, part 2
- Add clearer documentation and examples for more things.
- Rework string tensors by introducing `PrimitiveTensorElementType` for primitive (i.e. f32) types, and again re-implementing `IntoTensorElementType` for `String`. This allows string tensors to be used via `Tensor<String>` instead of exclusively via `DynTensor`. Additionally, string tensors no longer require an `Allocator` to be created (which didn't make sense, since string data in Rust can only ever be stored on the CPU anyway). This also now applies to `Map`s, since their data also needed to be on the CPU anyway. (`Sequence`s are currently unaffected because I think a custom allocator could be useful for them?)
- Rework the `IoBinding` interface, and add an example clarifying the intended usage of it (ref #209). Thanks to AAce from the pyke Discord for pointing out the mutability issue in the old interface, which should be addressed now.
- Refactor `OperatorDomain::add` from the slightly-nicer-looking-but-more-confusing `fn<T>(t: T)` to just `fn<T>()` to further enforce the fact that `Operator`s are zero-sized.
- Maps can now have `String` keys.
- Remove some unused errors.
