docs: correct the table duplicate rule and point at the binding section Four review nits, all documentation. The no-shadowing rule was credited to DataFusion. It is not DataFusion's: a `SchemaProvider` decides for itself whether a duplicate replaces or refuses, and only the in-memory one a session starts with refuses. `with_extensions` does not ask — it settles the question during resolve, which is what buys one rule for every destination and a refusal while a failure is still free, and which makes a bundle stricter than `register_table` against a provider that would have replaced. Said once in the collisions section and pointed at from the field docstring, the Rust resolver and the test that pins it. `udtfs` pointed at the two-phase section for a claim whose canonical home is the binding section this PR added; `functions.md` and `table-providers.md` already point there. The `Raises:` entry for a declared table did not mention two declarations resolving to one, and a rewrap had left "registering a" alone on a line. The user guide now says what a caller meets: a bundle cannot take a table name they already used. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
diff --git a/crates/core/src/context.rs b/crates/core/src/context.rs index 5065a65..9a92c60 100644 --- a/crates/core/src/context.rs +++ b/crates/core/src/context.rs
@@ -1853,9 +1853,11 @@ /// The fallible half. Each provider is imported against `slf` — the handle /// carrying the completed codec chains, not the context the components hook /// was given — and each name is resolved to the schema that will hold it. - /// A name already taken is refused here, because DataFusion refuses a - /// duplicate registration rather than replacing it, and a refusal is much - /// more useful before anything has been written. + /// A name already taken is refused here rather than left to the insert. + /// Whether a duplicate replaces or refuses is the `SchemaProvider`'s own + /// call — the in-memory one refuses — and asking it would mean asking at + /// commit time, once refusing costs something. Deciding here buys one rule + /// for every destination and a refusal while a failure is still free. /// /// Two declarations landing on one destination are refused here too. Both /// checks are against the *resolved* reference rather than the declared
diff --git a/docs/source/extension-guide/bundles.md b/docs/source/extension-guide/bundles.md index 6922acb..11dc278 100644 --- a/docs/source/extension-guide/bundles.md +++ b/docs/source/extension-guide/bundles.md
@@ -346,9 +346,18 @@ than replace, so two bundles contributing one each is the normal case and there is nothing to refuse. See {doc}`other-components`. -Tables go the other way. DataFusion refuses a duplicate table registration -rather than replacing it, so a declared table name that is *already* on the -session is an error too — a table cannot shadow one the way a function can. +Tables go the other way: a declared table name that is *already* on the session +is an error too, so a table cannot shadow one the way a function can. + +Strictly, whether a duplicate registration replaces or refuses is the +`SchemaProvider`'s own call, and the in-memory one a session starts with +refuses. `with_extensions` does not ask. It settles the question during resolve +and refuses whatever the destination would have done, which buys two things a +per-provider answer could not: the same rule wherever your table lands, and the +refusal arriving while a failure still costs nothing. A `SchemaProvider` that +would have replaced is therefore stricter through a bundle than through +{py:meth}`~datafusion.context.SessionContext.register_table`, which asks it +directly. Deregister the old table first if replacing is what you meant. A table collides on where it lands rather than on how it was spelled. A name is lowercased when it is parsed and filled out from the session's default catalog
diff --git a/docs/source/user-guide/extensions.md b/docs/source/user-guide/extensions.md index e758bd5..7b75b64 100644 --- a/docs/source/user-guide/extensions.md +++ b/docs/source/user-guide/extensions.md
@@ -80,7 +80,11 @@ `with_extensions` returns a context; use the returned one. It shares everything else with the context you called it on, so tables you registered -before the call are still there. +before the call are still there — and a library shipping a table under a name +you have already used cannot replace it. The call fails and writes nothing, so +drop yours with {py:meth}`~datafusion.context.SessionContext.deregister_table` +first, or install onto a context that does not hold it. See +{ref}`extension_bundles_collisions`. ## Using more than one library
diff --git a/python/datafusion/context.py b/python/datafusion/context.py index e7c446e..f46deb1 100644 --- a/python/datafusion/context.py +++ b/python/datafusion/context.py
@@ -2199,9 +2199,8 @@ session as it was. Declared tables install first and everything else after the planner is bound; all of them are visible on every handle sharing this session. A hook that *mutates* the context it is handed — - registering a - table, say — is not rolled back, which is why bundle objects must be - configuration-only. + registering a table, say — is not rolled back, which is why bundle + objects must be configuration-only. A call that installs optimizer rules rebuilds the session state, which drops the session's prepared statements — see @@ -2239,7 +2238,8 @@ and does not name the bundle, because by then the declaration has already been accepted as the right shape. Exception: If a declared table cannot be resolved — the name is - taken, the schema unknown, or the value not a table. + already registered, two declarations resolve to one table, the + schema is unknown, or the value is not a table. Examples: The returned handle is a different object sharing one session, and
diff --git a/python/datafusion/extensions.py b/python/datafusion/extensions.py index 0181d0d..8e89c5c 100644 --- a/python/datafusion/extensions.py +++ b/python/datafusion/extensions.py
@@ -356,7 +356,7 @@ capsule getter with the session, and the context a bundle is handed has not had this call's codecs installed yet — so a wrapper built inside the hook would be bound to the wrong chains. The host wraps these against the - finished context instead. See :ref:`extension_bundles_two_phases`. + finished context instead. See :ref:`extension_bundles_binding`. Collides by name like :py:attr:`udfs`. """ @@ -373,11 +373,12 @@ Bound to the finished context for the same reason as :py:attr:`udtfs`. - A name that is **already registered** is an error, here and in - ``register_table`` alike — DataFusion refuses a duplicate table rather than - replacing it, so unlike a function a table cannot shadow one. Two - declarations collide when they resolve to one table, not when they match as - strings: see :ref:`extension_bundles_collisions`. + A name that is **already registered** is an error, so unlike a function a + table cannot shadow one. Two declarations collide when they resolve to one + table, not when they match as strings. Both rules hold wherever the table + lands, even where + :py:meth:`~datafusion.context.SessionContext.register_table` would have + replaced: see :ref:`extension_bundles_collisions`. """ physical_optimizer_rules: tuple[PhysicalOptimizerRuleExportable, ...] = _components(
diff --git a/python/tests/test_context.py b/python/tests/test_context.py index 99e6902..b160b5c 100644 --- a/python/tests/test_context.py +++ b/python/tests/test_context.py
@@ -1793,9 +1793,10 @@ def test_with_extensions_rejects_a_table_name_the_session_holds(ctx): """A table cannot shadow one, the way a function can. - DataFusion refuses a duplicate registration rather than replacing it, so - this is its rule rather than a policy chosen here — and catching it during - resolution is what keeps the rest of the call from being written first. + The destination's own policy is not consulted: refusing during resolution + is what keeps the rest of the call from being written first, and asking a + ``SchemaProvider`` would mean asking at commit time, once refusing costs + something. """ ctx.from_pydict({"a": [1]}, name="events") provider = ctx.from_pydict({"a": [2]}).into_view()