Let it die, let it die! You shall die
Instead of creating an environment that lives throughout the duration of the process, we now hold the environment options we commit, and create the environment from those options (or grab the current env) whenever we need it. When all holders of the environment are dropped, the environment is dropped as well.
Previously, we held `Environment` as a static variable. Statics are never dropped, but ONNX Runtime's own destructors assumes that the environment is long gone by the time the process exits, which was not the case in `ort`! This led to issues like #441 and the dumb `0003-leak-logger-mutex.patch` from `ort-artifacts`.
Works on my machine, not sure why it doesn't in CI. Maybe due to the different # of cores (cause I'm not sure why 4 inter + 2 intra threads = 4 in the global test...)
The global test uses the exact same API so it should be fine to remove the session test.
I did that thing again!
Features in this commit:
- `ThreadManager` allows you to define custom thread creation functions for environments & sessions.
- Sessions can now opt-out of using the environment's global thread pool.
- Implemented the safe `ShapeInferenceContext` wrapper for custom operators.
- Prepacked weights allow the CPU execution provider to share one allocation for identical weights between sessions.
- Customize workload type to prioritize efficiency; useful for background tasks.
- Configurable per-session log identifiers
- Dynamic dimension overrides
Breaking changes:
- `EnvironmentGlobalThreadPoolOptions` is now `GlobalThreadPoolOptions` and uses the builder pattern instead of exposed struct fields.
