Status: pre-RFC binding-specific design proposal. The current Ruby binding remains one gem and one Magnus native extension.
This document applies the shared extension architecture to Ruby. It incorporates the version-locked shared runtime alternative from the earlier Ruby design and aligns that native model with Python and Node.js. The shared compatibility contract is canonical for native ABI, configuration, lifetime, and loader rules; this document defines Ruby deltas.
The current Ruby binding provides a useful migration base but not a native extension seam:
bindings/ruby builds one opendal_ruby cdylib and one opendal gem.OpenDal::Operator.new(scheme, options) is blocking-only and constructs through the compiled core registry.Operator.from_uri, Operator.via_iter, or an async operator.Operator#middleware uses Ruby duck typing, but an independently built native middleware still cannot access the wrapped Rust Operator in another DSO.RuntimeError.The dynamic design must not describe proposed methods or guarantees as current behavior.
The proposed release family is:
opendal-runtime provides the shared native runtime opendal owns `require "opendal"` and the Ruby adapter opendal-service-s3 provides S3 registration and native artifacts opendal-service-hdfs provides libhdfs-backed HDFS lazily opendal-layer-timeout provides Timeout opendal-layer-foyer provides Foyer
The opendal gem declares its required_runtime_protocol. The opendal-runtime gem exposes its minimum and current protocol levels for the binding to check. Each native service/layer gem requires an exact opendal-runtime release and embeds that OpenDAL version in its bootstrap metadata. Installing the base gem resolves the runtime dependency:
gem install opendal
Applications select the main binding and extensions in their Gemfile; Bundler resolves opendal-runtime:
gem "opendal", "= <opendal-release>" gem "opendal-service-s3" gem "opendal-layer-timeout" gem "opendal-layer-foyer"
Each extension gem contains a Ruby registration stub, a JSON manifest, and gem metadata mapping its canonical service or layer ID to that stub:
require "opendal/runtime" OpenDal::Runtime.register_manifest( File.expand_path("../../../opendal-extension.json", __dir__) )
The expected require paths are:
require "opendal" require "opendal/services/s3" require "opendal/layers/timeout" require "opendal/layers/foyer"
Requiring an extension reads and registers metadata but does not activate its native library. The first service/layer construction performs native loading and the exact OpenDAL version check.
Construction of an unregistered scheme must resolve the one matching registration stub from installed gem metadata. The resolver reports duplicate claims, caches results and deterministic failures, handles aliases deterministically, and never requires every native extension at startup. Installing dependencies alone is not treated as registration.
Explicit require remains the preferred deterministic registration path. An application that wants to detect native dependency failures during controlled startup must also construct or explicitly probe the service/layer, because registration alone intentionally performs no native load.
Operator.new remains the compatibility constructor:
require "opendal" require "opendal/services/s3" op = OpenDal::Operator.new("s3", { "bucket" => "photos", "region" => "us-east-1", })
The binding can add URI and explicit registry construction as additive methods:
op = OpenDal::Operator.via_iter("s3", { "bucket" => "photos", "region" => "us-east-1", }) op = OpenDal::Operator.from_uri( "s3://photos/archive?region=us-east-1", {"endpoint" => "https://s3.example.com"} )
After those methods exist, Operator.new delegates to via_iter. Scheme strings remain canonical so third-party services do not require edits to a base enum.
The service gem receives the original URI and explicit string options. S3, WebDAV, HDFS, and third-party gems retain their own configurator behavior, validation, credentials, and redaction.
Typed Ruby configuration objects and hashes convert to the shared ConfigValue grammar. The base adapter rejects symbols or objects without a declared conversion, cyclic containers, oversized values, unknown fields, and numeric overflow before calling package code. Native factories never retain Ruby objects.
New code uses OpenDal::Layers and Operator#layer:
require "opendal/layers/throttle" require "opendal/layers/timeout" limit = OpenDal::Layers::Throttle.new(10 * 1024, 10 * 1024 * 1024) timeout = OpenDal::Layers::Timeout.new(60, 10) layered = op.layer(limit).layer(timeout)
The Ruby object wraps a runtime-owned native LayerHandle, not a package-local Rust object exposed through Magnus. Operator#layer returns a new operator and preserves native service and context hooks.
The current names remain compatibility adapters:
Operator#middleware(value) delegates to Operator#layer(value).OpenDal::Middleware::* aliases the corresponding OpenDal::Layers::* classes during a deprecation period. The layer classes preserve the current positional constructors so the aliases do not change existing calls.apply_to remains a Ruby decorator and must not be described as equivalent to an arbitrary native layer.Timeout values remain finite, non-negative seconds at the Ruby interface. The adapter uses the current Duration::try_from_secs_f64 rule, which rounds to the nearest nanosecond with ties to even, and then emits SignedDuration. It rejects values outside SignedDuration's i64-seconds range instead of saturating them.
Throttle accepts only positive integer bandwidth and burst values in the supported u32 range. Ruby validation must reject invalid values before the native constructor can assert.
Later layer calls are outer layers. The binding preserves the canonical Timeout/Retry order and rejects the known unsafe composition when it can observe both layer IDs.
Ruby operations remain blocking in the initial design. A layer such as Foyer still needs asynchronous native initialization:
require "opendal/layers/foyer" cache = OpenDal::Layers::Foyer.build( memory_capacity: 64 << 20, storage_path: "/var/cache/opendal" ) cached = op.layer(cache)
Foyer.build submits the async factory to the shared runtime and waits while releasing the GVL. It must use the runtime's Tokio instance, clean up partial resources on failure, and return only after it owns a valid LayerHandle.
The package must not start a private Tokio runtime or hold Ruby values inside its native future. A future Ruby async interface can adapt the same runtime future without changing the extension interface.
One Ruby layer object preserves one native sharing identity:
The first version does not define Marshal support for operators or live layer handles. A future declarative recipe format must reconstruct new native state rather than claiming to serialize cache contents, limiter history, a JVM, or a Tokio runtime.
The runtime should expose Ruby exception classes for extension lifecycle failures:
OpenDal::ExtensionNotInstalled OpenDal::ExtensionLoadError OpenDal::ExtensionIncompatible OpenDal::ExtensionConflict OpenDal::LayerInitializationError
Normal OpenDAL error kinds should also map to stable Ruby exception classes rather than losing all structure in RuntimeError. Compatibility aliases or a common superclass can preserve existing rescue behavior.
Errors include package ID, scheme/layer ID, and construction operation. They do not include credentials or unredacted option hashes.
The current native-gem matrix is best effort. Dynamic extensions should not claim broader binary coverage until runtime plus adapter artifacts pass an installation test on that platform.
The first design does not promise Ractor shareability. Runtime registries and native handles may be process-global Rust state, but they must not retain Ractor-local Ruby objects.
Blocking operations and layer initialization release the GVL only through well-defined base-adapter helpers. Package code must not call Ruby from shared runtime worker threads.
Runtime, JVM, connection-pool, and Foyer state is unsupported after fork unless a package later defines explicit reinitialization behavior.
NativeRuntime and an extension registry inside the current gem.via_iter, from_uri, layer, OpenDal::Layers, and structured errors without removing current methods.require "opendal" without optional extensions.Operator.new and middleware compatibility behavior.from_uri is implemented.