)]}'
{
  "commit": "0a9ca68ade8bbe06aaf67928bb8edc65670baa24",
  "tree": "8c9a9acdca0b95cfad42961c19ffe43adc5d3063",
  "parents": [
    "baec559b0a7c85934338d6da80ffbe538004f4d4"
  ],
  "author": {
    "name": "Tim Saucer",
    "email": "timsaucer@gmail.com",
    "time": "Thu May 28 17:09:33 2026 -0400"
  },
  "committer": {
    "name": "GitHub",
    "email": "noreply@github.com",
    "time": "Thu May 28 17:09:33 2026 -0400"
  },
  "message": "docs: user guide + runnable examples for distributing expressions (#1547)\n\n* docs: user guide page + runnable examples for distributing expressions\n\nWraps up the Expr-pickle work with the user-facing material:\n\n* docs/source/user-guide/io/distributing_work.rst — new user guide\n  page covering the multiprocessing, Ray, and datafusion-distributed\n  patterns. Includes the Security section that is the canonical home\n  for the cloudpickle / pickle.loads threat model.\n* docs/source/user-guide/io/index.rst — toctree entry.\n* examples/multiprocessing_pickle_expr.py — runnable example: a\n  Pool.map of a closure-capturing UDF across processes, with worker\n  context registration in the initializer.\n* examples/ray_pickle_expr.py — Ray actor analogue.\n* examples/datafusion-ffi-example/python/tests/_test_pickle_strict_ffi.py\n  — exercises the strict-mode refusal end to end against an FFI\n  capsule scalar UDF (kept under the FFI example crate because the\n  test needs that crate\u0027s compiled artifacts).\n* examples/README.md — index entries for the new files.\n\nAlso tightens three docstrings that previously duplicated the\nsecurity warning so they point at the canonical Security section\ninstead:\n\n* PythonLogicalCodec::with_python_udf_inlining (rustdoc): one-line\n  summary plus a relative pointer to distributing_work.rst and the\n  upstream Python pickle module security warning.\n* SessionContext.with_python_udf_inlining: one-sentence summary plus\n  :doc: link to the user guide.\n* datafusion.ipc module docstring: cross-reference to the user guide\n  for the full pattern.\n\nThe crate-level codec.rs module rustdoc also updates \"pure-Python\nscalar UDFs\" to \"scalar / aggregate / window UDFs\" now that all three\nare covered.\n\nCo-Authored-By: Claude Opus 4.7 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* docs: document Python-version and import portability caveats for inline UDFs\n\nReviewer feedback on the Expr-pickle PRs (#1544) asked that the\ncloudpickle portability caveats be discoverable on the user-facing\npage, not only in docstrings. The distributing_work.rst page is the\ndesignated canonical home for the distribution story, so add them here:\n\n* New \u0027Portability requirements for inline Python UDFs\u0027 subsection\n  covering the matching-Python-minor-version requirement and the\n  by-value vs by-reference import-capture rule (imported modules must\n  be importable on the worker).\n* Qualify the \u0027fully portable\u0027 Python-UDF bullet to point at the new\n  requirements.\n* Cross-reference the new subsection from the closure-capture note.\n\nCo-Authored-By: Claude Opus 4.7 (1M context) \u003cnoreply@anthropic.com\u003e\n\n* docs: restore version-byte and cloudpickle-cache rustdoc wording\n\nTwo codec.rs docstrings were reworded in PR4 in ways that dropped\ninformation:\n\n* try_encode_python_scalar_udf: restore the `DFPYUDF` family prefix +\n  version byte description of the payload framing (PR4 had collapsed it\n  to `DFPYUDF1` prefix, dropping the version-byte mention).\n* cloudpickle cached-handle comment: restore \"The encode/decode helpers\n  above\" wording.\n\n* docs: fix reversed tuple order in multiprocessing example docstring\n\nThe \u0027Worker layout\u0027 docstring described tasks as `(expr, label)` but\nthe code builds and unpacks them as `(label, expr)`. Correct the doc\nto match.\n\n* Respond to first batch of reviewer comments\n\n* docs: relocate and restructure distributing-work guide\n\nMove the page from user-guide/io/ to the top level of user-guide/ — distributing work is a runtime/operational concern, not a file-format topic, and the shorter \"Distributing work\" title fits the sidebar cleanly.\n\nRestructure the body to lead with the practical worker-setup pattern instead of the four-slot SessionContext taxonomy. The taxonomy survives at the bottom as a reference subsection; the worker-init example and portability rules now reach the reader before they need it. Also addresses reviewer NIT: wrap the `if __name__ \u003d\u003d \"__main__\":` guidance in a `.. note::` admonition and link to the Python multiprocessing docs.\n\nAdd a header paragraph to each runnable example pointing to the user-guide page so a reader who jumps straight to the example gets the surrounding context.\n\nCo-Authored-By: Claude Opus 4.7 \u003cnoreply@anthropic.com\u003e\n\n---------\n\nCo-authored-by: Claude Opus 4.7 (1M context) \u003cnoreply@anthropic.com\u003e",
  "tree_diff": [
    {
      "type": "modify",
      "old_id": "b1b9f99dccbb8d770d07b22deb9444bb6333febc",
      "old_mode": 33188,
      "old_path": "crates/core/src/codec.rs",
      "new_id": "a6ea6671c8ee9346b21809fb7285c12b98d1f14e",
      "new_mode": 33188,
      "new_path": "crates/core/src/codec.rs"
    },
    {
      "type": "modify",
      "old_id": "0007cc41a954db873a0f12b7fcde11c7842ff995",
      "old_mode": 33188,
      "old_path": "docs/source/index.rst",
      "new_id": "7edb69807a15b457b3666af4826bf94e0762e9f5",
      "new_mode": 33188,
      "new_path": "docs/source/index.rst"
    },
    {
      "type": "add",
      "old_id": "0000000000000000000000000000000000000000",
      "old_mode": 0,
      "old_path": "/dev/null",
      "new_id": "03b5ca0b9e89c3448f853840e3d269ce704d6af0",
      "new_mode": 33188,
      "new_path": "docs/source/user-guide/distributing-work.rst"
    },
    {
      "type": "modify",
      "old_id": "3024c782f1afe6d63f49b9459e2a47304240e1b2",
      "old_mode": 33188,
      "old_path": "examples/README.md",
      "new_id": "e0e3056d9ec4211fc70ccdc294521de80fc6599a",
      "new_mode": 33188,
      "new_path": "examples/README.md"
    },
    {
      "type": "add",
      "old_id": "0000000000000000000000000000000000000000",
      "old_mode": 0,
      "old_path": "/dev/null",
      "new_id": "67c0b245a5dbea6f7bcf928d228f76482ad93bba",
      "new_mode": 33188,
      "new_path": "examples/datafusion-ffi-example/python/tests/_test_pickle_strict_ffi.py"
    },
    {
      "type": "add",
      "old_id": "0000000000000000000000000000000000000000",
      "old_mode": 0,
      "old_path": "/dev/null",
      "new_id": "73a99c2db0862f2463b2067c4df13e9e2fed9d30",
      "new_mode": 33188,
      "new_path": "examples/multiprocessing_pickle_expr.py"
    },
    {
      "type": "add",
      "old_id": "0000000000000000000000000000000000000000",
      "old_mode": 0,
      "old_path": "/dev/null",
      "new_id": "04cea463dcd277747172c148f6bfef746c259532",
      "new_mode": 33188,
      "new_path": "examples/ray_pickle_expr.py"
    }
  ]
}