[FFI] Add ref-qualified strict ObjectRef casts (#639)

This PR refactors strict ObjectRef casting to use the Any TypeTraits
strict-check path and adds strict throwing APIs.

Main benefit:
- `Any` and `ObjectRef` now have consistent strict `as` / `as_or_throw`
APIs: both rely on the same `TypeTraits` strict-check semantics, both
return optional values for probe-style `as<T>()`, and both provide
throwing `as_or_throw<T>()` variants for required casts.
- ObjectRef casts can now support richer compatibility checks through
`TypeTraits<ObjectRefType>::CheckAnyStrict`, instead of being limited to
the target ref's canonical `ContainerType` instance check.
- `ObjectRefType::ContainerType` now has an explicit
`_type_container_is_exact` invariant. Ordinary refs inherit `true` from
`ObjectRef`, while richer parameterized refs such as `Array<T>`,
`Map<K,V>`, `Tuple<...>`, and object-backed `Variant<...>` opt out
because their accepted values are determined by `TypeTraits`, not only
by the backing container. `GetRef` is guarded to only work for
exact-container refs.

API behavior:
- `ObjectRef::as<ObjectRefType>() const&` returns
`std::optional<ObjectRefType>`. For a non-null source, it succeeds only
when `TypeTraits<ObjectRefType>::CheckAnyStrict` accepts the object
through a compact temporary `TVMFFIAny` view. On success, it returns a
ref that shares the original object pointer. On type mismatch, it
returns `std::nullopt`.
- `ObjectRef::as<ObjectRefType>() &&` has the same strict-check and
`std::nullopt` behavior, but moves the object pointer into the returned
ref on success and clears the source ref.
- For null `ObjectRef` sources, `ObjectRef::as<ObjectRefType>()` returns
a successful null ref when the target ref type is nullable, and returns
`std::nullopt` for non-nullable target refs. The null path is explicit
and is not treated as a globally cold failure path.
- `ObjectRef::as_or_throw<ObjectRefType>() const&` and `&&` are the
throwing forms of the same strict ObjectRef cast. They return
`ObjectRefType` on success, preserve/move the object pointer according
to the receiver qualifier, return a null ref for nullable null targets,
and throw `TypeError` for non-nullable null or type-mismatch cases.
- `Any::as_or_throw<T>() const&` and `&&` are strict reinterpretation
helpers for `Any`. They call the corresponding strict `as<T>()` path,
return `T` on success, and throw `TypeError` on mismatch. They do not
run fallback conversions; use `cast<T>()` when conversion is intended.
- Optional-returning `as` APIs intentionally do not use success/failure
prediction annotations because `std::nullopt` can be a normal probe
result. Throwing APIs mark only the final failure/throw path as cold,
and `ObjectRef::as_or_throw` marks the strict-check success path as
likely because mismatch throws.

Implementation notes:
- `ObjectRef::as<T>()` no longer relies on `T::ContainerType` as the
complete runtime compatibility test. It piggy-backs on Any `TypeTraits`
strict checks, so ref types with richer strict compatibility rules work
consistently with `Any`.
- The temporary `TVMFFIAny` setup is deliberately kept inline in the
non-null ObjectRef paths. This preserves explicit null behavior and lets
`as_or_throw` call `TypeTraits<T>::GetMismatchTypeInfo` for richer
diagnostics instead of reducing to `as<T>()` and losing the synthesized
Any view.
- The compact temporary Any view is expected to optimize away on hot
paths; GCC/Clang assembly probes confirmed direct object-header
loads/type-index comparisons for the checked ObjectRef paths.

Changes:
- Add ref-qualified `ObjectRef::as<T>()` and
`ObjectRef::as_or_throw<T>()` overloads for const and rvalue receivers.
- Add `Any::as_or_throw<T>()` const/rvalue helpers.
- Keep ObjectRef null handling explicit while using temporary
`TVMFFIAny` views for rich `TypeTraits` checks and mismatch messages.
- Add `_type_container_is_exact` and guard `GetRef` so it is only used
for refs whose `ContainerType` is an exact acceptance predicate.
- Add focused tests for const and move variants of `as<T>()` and
`as_or_throw<T>()`, plus compile-time coverage for exact/non-exact
container-ref flags.
- Update `TVM_FFI_UNSAFE_ASSUME` lowering to be more robust for GCC.
13 files changed
tree: ad093a56045bda04da997ded4879f73276dd576b
  1. .agents/
  2. .claude/
  3. .github/
  4. 3rdparty/
  5. addons/
  6. cmake/
  7. docs/
  8. examples/
  9. include/
  10. licenses/
  11. python/
  12. rust/
  13. src/
  14. tests/
  15. .asf.yaml
  16. .clang-format
  17. .clang-tidy
  18. .cmake-format.json
  19. .gitignore
  20. .gitmodules
  21. .markdownlint-cli2.yaml
  22. .pre-commit-config.yaml
  23. .yamllint.yaml
  24. AGENTS.md
  25. CLAUDE.md
  26. CMakeLists.txt
  27. CONTRIBUTING.md
  28. KEYS
  29. LICENSE
  30. NOTICE
  31. pyproject.toml
  32. README.md
README.md

TVM FFI: Open ABI and FFI for Machine Learning Systems

📚 Documentation | 🚀 Quickstart

Apache TVM FFI is an open ABI and FFI for machine learning systems. It is a minimal, framework-agnostic, yet flexible open convention with the following systems in mind:

  • Kernel libraries - ship one wheel to support multiple frameworks, Python versions, and different languages. [FlashInfer]
  • Kernel DSLs - reusable open ABI for JIT and AOT kernel exposure frameworks and runtimes. [TileLang][cuteDSL]
  • Frameworks and runtimes - a uniform extension point for ABI-compliant libraries and DSLs. [PyTorch][JAX][PaddlePaddle][NumPy/CuPy]
  • ML infrastructure - out-of-box bindings and interop across languages. [Python][C++][Rust][XGrammar]
  • Coding agents - a unified mechanism for shipping generated code in production.

Features

  • Stable, minimal C ABI designed for kernels, DSLs, and runtime extensibility.
  • Zero-copy interop across PyTorch, JAX, and CuPy using DLPack protocol.
  • Compact value and call convention covering common data types for ultra low-overhead ML applications.
  • Multi-language support out of the box: Python, C++, and Rust (with a path towards more languages).

These enable broad interoperability across frameworks, libraries, DSLs, and agents; the ability to ship one wheel for multiple frameworks and Python versions (including free-threaded Python); and consistent infrastructure across environments.

Getting Started

Install TVM-FFI with pip, uv or from source:

pip install apache-tvm-ffi
pip install torch-c-dlpack-ext  # compatibility package for torch <= 2.9

Status and Release Versioning

C ABI stability is our top priority.

Status: RFC Main features are complete and ABI stable. We recognize potential needs for evolution to ensure it works best for the machine learning systems community, and would like to work together with the community for such evolution. We plan to stay in the RFC stage for three months from the v0.1.0 release.

Releases during the RFC stage will be 0.X.Y, where bumps in X indicate C ABI-breaking changes and Y indicates other changes. We anticipate the RFC stage will last for three months, then we will start following Semantic Versioning (major.minor.patch) going forward.

Documentation

Our documentation site includes:

Get Started

Guides

Concepts

Packaging

Developer Manual