feat: many a feature

I did that thing again!

Features in this commit:
- `ThreadManager` allows you to define custom thread creation functions for environments & sessions.
- Sessions can now opt-out of using the environment's global thread pool.
- Implemented the safe `ShapeInferenceContext` wrapper for custom operators.
- Prepacked weights allow the CPU execution provider to share one allocation for identical weights between sessions.
- Customize workload type to prioritize efficiency; useful for background tasks.
- Configurable per-session log identifiers
- Dynamic dimension overrides

Breaking changes:
- `EnvironmentGlobalThreadPoolOptions` is now `GlobalThreadPoolOptions` and uses the builder pattern instead of exposed struct fields.
This commit is contained in:
Carson M.
2024-11-18 19:38:59 -06:00
parent 7819d56bf9
commit 87577ef396
12 changed files with 536 additions and 54 deletions

View File

@@ -30,6 +30,14 @@ impl Operator for CustomOpOne {
fn outputs() -> Vec<OperatorOutput> {
vec![OperatorOutput::required(TensorElementType::Float32)]
}
fn get_infer_shape_function() -> Option<Box<ort::operator::InferShapeFn>> {
Some(Box::new(|ctx| {
let inputs = ctx.inputs();
ctx.set_output(0, &inputs[0])?;
Ok(())
}))
}
}
impl Kernel for CustomOpOneKernel {