)]}'
{
  "log": [
    {
      "commit": "7ccacb14c006cee191fcbb48e248ca4d1d291bd1",
      "tree": "e0f065a7ce8be7c5d99e1f250b627bff4f34cece",
      "parents": [
        "31d7d0343917fb1d8e711b7747597c488f6113c1"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 29 01:10:53 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 29 01:10:53 2026 -0700"
      },
      "message": "[SPARK-58380] Update Spark 4.1/4.0 examples to use 4.1.3 and 4.0.4 docker images\n\n### What changes were proposed in this pull request?\n\nThis PR aims to update Spark 4.1/4.0 examples to use 4.1.3 and 4.0.4 docker images\n\n- [v4.1.3](https://github.com/apache/spark/releases/tag/v4.1.3) (2026-07-11)\n- [v4.0.4](https://github.com/apache/spark/releases/tag/v4.0.4) (2026-07-12)\n\n### Why are the changes needed?\n\nTo recommend the latest Apache Spark maintenance releases which bring the latest bug fixes.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo, this is an example-only change.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #776 from dongjoon-hyun/SPARK-58380.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "31d7d0343917fb1d8e711b7747597c488f6113c1",
      "tree": "3e3a09b1517da1e69ca41cda951fd2edeebcbab6",
      "parents": [
        "618273e3017c66ab57ee58e274e058456fa066b3"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri Jul 24 13:19:20 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri Jul 24 13:19:20 2026 -0700"
      },
      "message": "[SPARK-58331] Use Java-friendly `sparkVersion` API instead of `labels` override in `SparkAppDriverConf`\n\n### What changes were proposed in this pull request?\n\nThis PR aims to replace the Scala-typed `labels()` override of `SparkAppDriverConf` with the Java-friendly `sparkVersion()` API which was introduced via SPARK-56736 in Apache Spark 4.2.0.\n\n- https://github.com/apache/spark/pull/55697\n\n### Why are the changes needed?\n\nTo use Java-friendly APIs instead of Scala APIs.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo. The generated driver labels are identical. The existing `testLabels` test passes without modification.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #774 from dongjoon-hyun/SPARK-58331.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "618273e3017c66ab57ee58e274e058456fa066b3",
      "tree": "52559c0230f7bf3dcb0f60483b984194d33cb683",
      "parents": [
        "e588fca22efdd104756086337e85fe9aacb0a503"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 15:24:08 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 15:24:08 2026 -0700"
      },
      "message": "[SPARK-58306] Use `ubuntu-26.04(-arm)?` in GitHub Action jobs\n\n### What changes were proposed in this pull request?\n\nThis PR aims to use `ubuntu-26.04` and `ubuntu-26.04-arm` runner images in all GitHub Action jobs.\n\n- `ubuntu-slim` → `ubuntu-26.04` (This is better because `ubuntu-slim` is flaky in these days due to 15-min timeout)\n- `ubuntu-latest` → `ubuntu-26.04`\n- `ubuntu-24.04-arm` → `ubuntu-26.04-arm`\n\n### Why are the changes needed?\n\nFor Apache Spark K8s Operator v1.1.0, this PR aims the following:\n\n1. **Consistent environment across the build matrix** — Previously, the x86 job used `ubuntu-slim` (24.04-based) while the arm job used `ubuntu-24.04-arm`, so the two architectures ran on different image types. Now both architectures are tested on the same Ubuntu 26.04 environment, eliminating variables other than the architecture itself.\n\n2. **Longer support window** — The 26.04 runner image will be supported longer than the 24.04-based images, buying time until the next LTS migration.\n\n3. **Early validation on the latest kernel and system libraries** — The operator is built and tested on top of the newest glibc, kernel, and Docker versions, so issues on modern environments are caught in CI first.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo. This is an infra-only change.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #773 from dongjoon-hyun/SPARK-58306.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "e588fca22efdd104756086337e85fe9aacb0a503",
      "tree": "25c30bba54f276c6d923d7e434384c5e37f91bf5",
      "parents": [
        "e47af1a2d438faf45cadb391193b30ee152c41d0"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 15:22:27 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 15:22:27 2026 -0700"
      },
      "message": "[SPARK-58304] Simplify examples to show only feature-specific configurations\n\n### What changes were proposed in this pull request?\n\nThis PR aims to simplify the example YAML files so that each file contains only the\nconfigurations relevant to the feature its name describes.\n\n- Remove the `spark.dynamicAllocation.*` boilerplate from all examples except `pi.yaml`.\n- Remove the generic `applicationTolerations` boilerplate (`resourceRetainPolicy`,\n  `ttlAfterStopMillis`) where it is not the subject of the example.\n- Remove a few unrelated configurations (`spark.eventLog.*` from `stream-word-count.yaml`,\n  `spark.logConf` from `dfs-read-write.yaml`).\n\nThe examples referenced by the e2e tests keep their `applicationTolerations` unchanged.\n\n### Why are the changes needed?\n\nThe repeated boilerplate obscured the actual point of each example. Now each example shows only\nthe feature-specific configurations, making it easier to read, copy, and maintain.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #772 from dongjoon-hyun/SPARK-58304.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "e47af1a2d438faf45cadb391193b30ee152c41d0",
      "tree": "95628c347d5200f6ad5354fb2b16f490f50fff7c",
      "parents": [
        "4e05aeb34d53f13f0feadd443b8da038f9497558"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 15:21:33 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 15:21:33 2026 -0700"
      },
      "message": "[SPARK-58303] Declare least-privilege `permissions` in GitHub Actions workflows\n\n### What changes were proposed in this pull request?\n\nThis PR aims to declare least-privilege `permissions: contents: read` in the four GitHub Actions workflows that did not declare any token permissions: `build_and_test.yml`, `build_main.yml`, `publish_snapshot_chart.yml`, and `publish_snapshot_dockerhub.yml`. The other workflows already declare job-level permissions.\n\n### Why are the changes needed?\n\nWithout an explicit `permissions` block, workflows run with the default `GITHUB_TOKEN` permissions, which are broader than needed. These workflows only read via `GITHUB_TOKEN`; publishing uses dedicated secrets.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #771 from dongjoon-hyun/SPARK-58303.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "4e05aeb34d53f13f0feadd443b8da038f9497558",
      "tree": "f3582de8fa6ce701800fc9295077031ea9c0f3be",
      "parents": [
        "8d602bcedbd047ee8927993bb1b591a51b60d6ce"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 14:28:37 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 14:28:37 2026 -0700"
      },
      "message": "[SPARK-58301] Upgrade `Apache DataFusion Comet` to 0.17.1\n\n### What changes were proposed in this pull request?\n\nThis PR upgrades Apache DataFusion Comet to `0.17.1` in the `pi-with-comet` example.\n\n### Why are the changes needed?\n\nTo use the latest Comet release (`0.17.1`, released on 2026-07-17).\n- https://github.com/apache/datafusion-comet/releases/tag/0.17.1\n- https://datafusion.apache.org/comet/changelog/0.17.1.html\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #770 from dongjoon-hyun/SPARK-58301.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "8d602bcedbd047ee8927993bb1b591a51b60d6ce",
      "tree": "3b71aabb8ee72ef39413686100c0215f3e3afd84",
      "parents": [
        "ebfeb5e84d4743b4fbd9b6d342384e1cbba73576"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 14:17:37 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 14:17:37 2026 -0700"
      },
      "message": "[SPARK-58299] Upgrade `Iceberg` example to use Spark 4.1.2\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `Iceberg` example to use Spark 4.1.2 in the `spark-connect-server-iceberg.yaml` example.\n\n### Why are the changes needed?\n\nTo provide an up-to-date example.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nManual review.\n\n```\n$ kubectl apply -f examples/spark-connect-server-iceberg.yaml\nsparkapplication.spark.apache.org/spark-connect-server-iceberg created\n```\n\n```\n$ kubectl port-forward spark-connect-server-iceberg-0-driver 15002\n```\n\n```\n$ bin/spark-connect-shell --remote sc://localhost:15002\n...\n$ bin/spark-connect-shell --remote sc://localhost:15002\nWARNING: Using incubator modules: jdk.incubator.vector\n26/07/23 14:09:02 INFO BaseAllocator: Debug mode disabled. Enable with the VM option -Darrow.memory.debug.allocator\u003dtrue.\n26/07/23 14:09:02 INFO DefaultAllocationManagerOption: allocation manager type not specified, using netty as the default type\n26/07/23 14:09:02 INFO CheckAllocator: Using DefaultAllocationManager at memory/netty/DefaultAllocationManagerFactory.class\nCompiling (synthetic)/ammonite/predef/ArgsPredef.sc\nCompiling /Users/dongjoon/APACHE/spark-releases/spark-4.2.0-bin-hadoop3/(console)\nWelcome to\n      ____              __\n     / __/__  ___ _____/ /__\n    _\\ \\/ _ \\/ _ `/ __/  \u0027_/\n   /___/ .__/\\_,_/_/ /_/\\_\\   version 4.2.0\n      /_/\n\nType in expressions to have them evaluated.\nSpark connect server version 4.1.2.\nSpark session available as \u0027spark\u0027.\nscala\u003e\nsql(\"\"\"CREATE TABLE taxis(vendor_id bigint, trip_id bigint, trip_distance float, fare_amount double, store_and_fwd_flag string) PARTITIONED BY (vendor_id);\"\"\").show()\nUsing Spark\u0027s default log4j profile: org/apache/spark/log4j2-defaults.properties\n\n++\n||\n++\n++\n\nscala\u003e\nscala\u003e\nsql(\"\"\"INSERT INTO taxis VALUES (1, 1000371, 1.8, 15.32, \u0027N\u0027), (2, 1000372, 2.5, 22.15, \u0027N\u0027), (2, 1000373, 0.9, 9.01, \u0027N\u0027), (1, 1000374, 8.4, 42.13, \u0027Y\u0027);\"\"\").show()\n++\n||\n++\n++\n\nscala\u003e sql(\"SELECT * FROM taxis\").show(false)\n+---------+-------+-------------+-----------+------------------+\n|vendor_id|trip_id|trip_distance|fare_amount|store_and_fwd_flag|\n+---------+-------+-------------+-----------+------------------+\n|1        |1000374|8.4          |42.13      |Y                 |\n|1        |1000371|1.8          |15.32      |N                 |\n|2        |1000372|2.5          |22.15      |N                 |\n|2        |1000373|0.9          |9.01       |N                 |\n+---------+-------+-------------+-----------+------------------+\n\nscala\u003e sql(\"SELECT * FROM taxis.history\").show(false)\n+-----------------------+-------------------+---------+-------------------+\n|made_current_at        |snapshot_id        |parent_id|is_current_ancestor|\n+-----------------------+-------------------+---------+-------------------+\n|2026-07-23 21:09:24.528|5673557555333823555|NULL     |true               |\n+-----------------------+-------------------+---------+-------------------+\n\nscala\u003e sql(\"SELECT * FROM taxis VERSION AS OF 5673557555333823555\").show(false)\n+---------+-------+-------------+-----------+------------------+\n|vendor_id|trip_id|trip_distance|fare_amount|store_and_fwd_flag|\n+---------+-------+-------------+-----------+------------------+\n|1        |1000374|8.4          |42.13      |Y                 |\n|1        |1000371|1.8          |15.32      |N                 |\n|2        |1000372|2.5          |22.15      |N                 |\n|2        |1000373|0.9          |9.01       |N                 |\n+---------+-------+-------------+-----------+------------------+\n```\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #769 from dongjoon-hyun/SPARK-58299.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "ebfeb5e84d4743b4fbd9b6d342384e1cbba73576",
      "tree": "621e767ad7fbce2ee99bb7308024d6ba45c178f5",
      "parents": [
        "81a748853bda9d8190de82b01cc309cdaac4dab2"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 01:32:10 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 01:32:10 2026 -0700"
      },
      "message": "[SPARK-58287] Add `dev/bump_maintenance_version.py` to bump maintenance version with dev postfix\n\n### What changes were proposed in this pull request?\n\nThis PR aims to add `dev/bump_maintenance_version.py` to bump the maintenance version for\nthe next development cycle. It reads the operator version from `build.gradle` and the Helm\nchart version from `Chart.yaml`, and works whether or not `./dev/prepare_release.py` has\nalready removed the suffixes.\n\n- `1.1.0-SNAPSHOT` or `1.1.0` -\u003e `1.1.1-SNAPSHOT`\n- `1.9.0-dev` or `1.9.0` -\u003e `1.9.1-dev`\n\n### Why are the changes needed?\n\nTo simplify the release process like `dev/prepare_release.py`.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo. This is a dev-only change.\n\n### How was this patch tested?\n\nManually.\n\n```\n$ ./dev/bump_maintenance_version.py\n```\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #768 from dongjoon-hyun/SPARK-58287.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "81a748853bda9d8190de82b01cc309cdaac4dab2",
      "tree": "61cf796cf5d946d38ec31db9901825aee6ffed4e",
      "parents": [
        "24be49ccdd5f1bfce373406c8faf55b411dfd733"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 01:24:03 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 01:24:03 2026 -0700"
      },
      "message": "[SPARK-58289] Add `dev/prepare_release.py` to automate release version preparation\n\n### What changes were proposed in this pull request?\n\nThis PR aims to add a `dev/prepare_release.py` script that removes the `-SNAPSHOT` and `-dev` suffixes from version strings (e.g. `1.0.1-SNAPSHOT` -\u003e `1.0.1`, `1.9.0-dev` -\u003e `1.9.0`).\n\n### Why are the changes needed?\n\nTo replace repetitive and error-prone manual edits (14 version strings across 8 files) with a single command:\n\n```bash\n./dev/prepare_release.py\n```\n\n### Does this PR introduce _any_ user-facing change?\n\nNo. This is a developer-only script.\n\n### How was this patch tested?\n\nManually verified.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #767 from dongjoon-hyun/SPARK-58289.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "24be49ccdd5f1bfce373406c8faf55b411dfd733",
      "tree": "dd949355b15d8dbc8ae264fae8e636c557e7a757",
      "parents": [
        "04b1b9f5ecca98e17c559cc5f0dd13fdc0824ef9"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 00:32:17 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 00:32:17 2026 -0700"
      },
      "message": "[SPARK-58288] Bump version to 1.1.0-SNAPSHOT in `main` branch\n\n### What changes were proposed in this pull request?\n\nThis PR aims to bump version to `1.1.0-SNAPSHOT` in `main` branch.\n\n### Why are the changes needed?\n\n`branch-1.0` is created. We need to distinguish `main` branch from `branch-1.0`.\n- https://github.com/apache/spark-kubernetes-operator/tree/branch-1.0\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: `Claude Fable 5`\n\nCloses #766 from dongjoon-hyun/SPARK-58288.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "04b1b9f5ecca98e17c559cc5f0dd13fdc0824ef9",
      "tree": "c34f6eb349a4bb0cf8e4d1233f3a0c6089ba9ff1",
      "parents": [
        "746602722928ce7963d4de37997ea82852682d6b"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 00:03:13 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 23 00:03:13 2026 -0700"
      },
      "message": "[SPARK-58284] Remove duplicated NetworkPolicy creation in favor of `NetworkPolicyFeatureStep` of Spark 4.2\n\n### What changes were proposed in this pull request?\n\nThis PR aims to remove the operator-side `NetworkPolicy` creation from `SparkAppResourceSpec` in favor of Apache Spark 4.2.0\u0027s built-in `NetworkPolicyFeatureStep`.\n\n- Remove `buildNetworkPolicy` and its invocation from `SparkAppResourceSpec`.\n- Remove the corresponding `testNetworkPolicy` test case and adjust the pre-resource count assertion.\n- Keep the `networkpolicies` RBAC rule in the Helm chart because the operator still needs it to create the `NetworkPolicy` generated by Spark\u0027s `NetworkPolicyFeatureStep`.\n\n### Why are the changes needed?\n\nThe operator has created a `NetworkPolicy` per `SparkApplication` since SPARK-55085. After SPARK-58194 upgraded the Spark dependency to 4.2.0, Spark\u0027s `KubernetesDriverBuilder` also generates a functionally identical `NetworkPolicy` via `NetworkPolicyFeatureStep` (SPARK-55653). As a result, every `SparkApplication` ends up with two duplicated policies.\n\n- https://github.com/apache/spark/pull/54442\n\n```\n$ kubectl get networkpolicy\nNAME                  POD-SELECTOR                                  AGE\npi-0-network-policy   spark-app-selector\u003dpi-0,spark-role\u003dexecutor   41s\npi-0-policy           spark-app-selector\u003dpi-0,spark-role\u003dexecutor   40s\n```\n\nBoth policies have the same pod selector and ingress rule (allowing traffic only from the pods of the same Spark application). Delegating to Spark\u0027s `NetworkPolicyFeatureStep` removes the duplication and avoids maintaining the same logic in two places.\n\n### Does this PR introduce _any_ user-facing change?\n\nYes, only one `NetworkPolicy` (`\u003cappId\u003e-policy`, created by Spark) is deployed per `SparkApplication` instead of two. The security behavior is unchanged because the two policies were functionally identical.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #764 from dongjoon-hyun/SPARK-58284.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "746602722928ce7963d4de37997ea82852682d6b",
      "tree": "a4d9fcbe0cb8778581aa4f2a47b9f573d872d896",
      "parents": [
        "e5bd0c6058046fa821974b36284e34ab2b73ecda"
      ],
      "author": {
        "name": "Chandni Singh",
        "email": "csingh@cloudera.com",
        "time": "Wed Jul 22 23:05:15 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 22 23:05:15 2026 -0700"
      },
      "message": "[SPARK-58160] Add Gateway API HTTPRoute/GRPCRoute support for driver endpoints\n\n### What changes were proposed in this pull request?\nAdds opt-in native support for Kubernetes Gateway API v1 routes on SparkApplication, parallel to the existing driverServiceIngressList. Two new optional lists on ApplicationSpec — driverHttpRouteList and driverGrpcRouteList — each pair a Service (fronting the driver pod) with an HTTPRoute or GRPCRoute. When the user omits the route rules, a default rule targeting the Service\u0027s first port is emitted.\n\nNo behavior change when unset. Not tied to a specific Gateway controller — Istio, Envoy Gateway, and Cilium are all GA on HTTPRoute/GRPCRoute v1.\n\nIncludes:\n- DriverHttpRouteSpec / DriverGrpcRouteSpec CRD types\n- DriverGatewayRouteUtils in spark-submission-worker\n- Wiring through SparkAppSubmissionWorker / SparkAppResourceSpec\n- Helm RBAC for gateway.networking.k8s.io httproutes/grpcroutes\n- Regenerated v1 CRD\n- Unit tests for default and user-provided rule shapes\n\n### Why are the changes needed?\nKubernetes Gateway API v1 is the successor to Ingress and is GA across major implementations (Istio, Envoy Gateway, Cilium, etc.). Today SparkApplication only supports Ingress via driverServiceIngressList; users on Gateway-based clusters have to create HTTPRoute / GRPCRoute objects out-of-band, which don\u0027t share the driver pod\u0027s lifecycle and can leak on cleanup. Native support lets the operator emit and GC these routes alongside the driver, on par with the existing Ingress path, and enables exposing Spark Connect (gRPC) endpoints — something Ingress can\u0027t cleanly do.\n\n### Does this PR introduce _any_ user-facing change?\nYes. Adds two optional fields on SparkApplication.spec:\n\n- driverHttpRouteList — emits a Service + Gateway API v1 HTTPRoute per entry for the driver.\n- driverGrpcRouteList — emits a Service + Gateway API v1 GRPCRoute per entry.\n\nBoth are opt-in and parallel to the existing driverServiceIngressList. The emitted resources are owned by the driver pod, so they GC automatically. If a route list is set, httpRouteSpec.parentRefs / grpcRouteSpec.parentRefs and the service/route metadata names are required — otherwise the app moves to Failed with a message naming the offending field. No behavior change when both fields are unset.\n\n### How was this patch tested?\nUnit tests and on an internal dev cluster\n\n### Was this patch authored or co-authored using generative AI tooling?\nYes.\nGenerated-by: Claude Code (Claude Opus 4.7)\n\nCloses #763 from otterc/http_grpc_routes.\n\nAuthored-by: Chandni Singh \u003ccsingh@cloudera.com\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "e5bd0c6058046fa821974b36284e34ab2b73ecda",
      "tree": "65965e9ef5e75e3ddc3bdc329962a2d96e525bb9",
      "parents": [
        "77e8a3925070574234956ea4cb0dfe86480037f8"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri Jul 17 09:58:56 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri Jul 17 09:58:56 2026 -0700"
      },
      "message": "[SPARK-58197] Upgrade `Java Operator SDK` to 5.5.0\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `Java Operator SDK` to 5.5.0.\n\n### Why are the changes needed?\n\nTo bring the latest improvements and bug fixes of Java Operator SDK.\n\n- https://github.com/operator-framework/java-operator-sdk/releases/tag/v5.5.0\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #762 from dongjoon-hyun/SPARK-58197.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "77e8a3925070574234956ea4cb0dfe86480037f8",
      "tree": "5a3453e40fef15586c3662e962a970d90d33761f",
      "parents": [
        "8a6c7569cd819e9cf7d780918fc52e65e844ac87"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri Jul 17 08:27:02 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri Jul 17 08:27:02 2026 -0700"
      },
      "message": "[SPARK-58194] Upgrade docker images to Apache Spark 4.2.0\n\n### What changes were proposed in this pull request?\n\nThis PR aims to use Apache Spark 4.2.0 docker images instead of `4.1.2` and `4.2.0-preview5`.\n\n- Rename `pi-preview*` examples and the `pi-preview` E2E test to `pi-java25*` with `apache/spark:4.2.0-scala-java25` image.\n- Update all examples, E2E tests, benchmark script, and docs from `4.1.2` to `4.2.0`.\n- Update the default Spark cluster image of `SparkClusterResourceSpec` from `apache/spark:4.1.2` to `apache/spark:4.2.0`.\n\nNote that `pi-with-comet.yaml` stays on `4.1.2` because `Apache DataFusion Comet` does not provide a Spark 4.2 artifact yet, like `pi-with-gluten.yaml` stays on `4.0.3`.\n\n### Why are the changes needed?\n\nSince Apache Spark 4.2.0 is released, the examples and tests should use the official release images instead of the preview and old versions. The build dependency is already on Spark 4.2.0 in `gradle/libs.versions.toml`.\n\n### Does this PR introduce _any_ user-facing change?\n\nYes, the default Spark cluster image becomes `apache/spark:4.2.0`, and the examples are updated. However, this is a consistent behavior change with the already-upgraded Spark 4.2.0 dependency.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #761 from dongjoon-hyun/SPARK-58194.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "8a6c7569cd819e9cf7d780918fc52e65e844ac87",
      "tree": "6fde092f31678b7f7652f35d9600f60faafafe6b",
      "parents": [
        "1d123acc388aec3a9e7c462aea90997942c1c262"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 10:02:20 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 10:02:20 2026 -0700"
      },
      "message": "[SPARK-58147] Upgrade SpotBugs to 4.10.3\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `SpotBugs` to 4.10.3.\n\nIn addition, `gradle/verification-metadata.xml` is regenerated from scratch by following the\n`AGENTS.md` guide, which replaces the SpotBugs 4.10.2 checksums with 4.10.3 ones (including\nits new transitive dependency `Saxon-HE` 12.10) and removes the stale entries of\n`log4j` 2.26.0 and `junit-bom` 6.1.0.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes.\n\n- https://github.com/spotbugs/spotbugs/releases/tag/4.10.3\n\n### Does this PR introduce _any_ user-facing change?\n\nNo. This is a build-time analysis tool update.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #760 from dongjoon-hyun/SPARK-58147.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "1d123acc388aec3a9e7c462aea90997942c1c262",
      "tree": "4ae667929d85fd5f69f1977a16729f110cb0e8b1",
      "parents": [
        "54721859dfdde28432c62cfb9836b65fd5e4b55c"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 09:45:22 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 09:45:22 2026 -0700"
      },
      "message": "[SPARK-58146] Upgrade `log4j` to 2.26.1\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `Apache Log4j` to 2.26.1.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes.\n\n- https://logging.apache.org/log4j/2.x/release-notes.html#release-notes-2-26-1\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #759 from dongjoon-hyun/SPARK-58146.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "54721859dfdde28432c62cfb9836b65fd5e4b55c",
      "tree": "220a75d1447478a13518989b87484459b46b0217",
      "parents": [
        "e795e2dccf7b05b7d95b76509b0115305365774f"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 09:18:01 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 09:18:01 2026 -0700"
      },
      "message": "[SPARK-58144] Add `verification-metadata.xml` regeneration guide to `AGENTS.md` and fix `README.md`\n\n### What changes were proposed in this pull request?\n\nThis PR aims to\n- Add a `gradle/verification-metadata.xml` regeneration guide to **`AGENTS.md`** and\n- Fix the existing guide in **`README.md`** accordingly.\n\nPreviously, since `--write-verification-metadata` is append-only, the guides now instruct to delete the existing file first and restore the ASF license header after regeneration.\n\n`gradle/verification-metadata.xml` is also regenerated by following the new guide,\nremoving 237 lines of stale entries.\n\n### Why are the changes needed?\n\nThe previous guide kept stale checksums of removed or upgraded dependencies forever.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nManually verified that `./gradlew build` passes with the regenerated metadata.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #757 from dongjoon-hyun/SPARK-58144.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "e795e2dccf7b05b7d95b76509b0115305365774f",
      "tree": "066aafa5aa9102ada6da934564a63d56b5c0949c",
      "parents": [
        "15ddd375bd59b877a97e353f9c794d55030a6ea9"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 09:01:40 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 09:01:40 2026 -0700"
      },
      "message": "[SPARK-58145] Upgrade `Apache Celeborn` to 0.6.3\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade Apache Celeborn to 0.6.3 in the `word-count-celeborn` example.\n\n### Why are the changes needed?\n\nApache Celeborn 0.6.3 is the latest version with bug fixes and improvements.\n\n- https://celeborn.apache.org/community/release_notes/release_note_0.6.3\n\n### Does this PR introduce _any_ user-facing change?\n\nNo, this is an example-only change.\n\n### How was this patch tested?\n\nManual review.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #758 from dongjoon-hyun/SPARK-58145.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "15ddd375bd59b877a97e353f9c794d55030a6ea9",
      "tree": "ed6029436ab38b51dbfe19196e09f29462e99455",
      "parents": [
        "42279b1e268dc57afddac3b55b0f36c945fa5376"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 08:14:41 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 08:14:41 2026 -0700"
      },
      "message": "[SPARK-58143] Upgrade `spotbugs-gradle-plugin` to 6.5.9\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `spotbugs-gradle-plugin` to 6.5.9. In addition, `gradle/verification-metadata.xml` is regenerated accordingly.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes:\n\n- https://github.com/spotbugs/spotbugs-gradle-plugin/releases/tag/6.5.9\n\n### Does this PR introduce _any_ user-facing change?\n\nNo, this is a build-time dependency change.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #756 from dongjoon-hyun/SPARK-58143.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "42279b1e268dc57afddac3b55b0f36c945fa5376",
      "tree": "4ee780ecbee06c65caa97a4fbe2793fd8262d5e2",
      "parents": [
        "b8e4f36c3084ade19c5dd37a98fe7b5eee6c1b2a"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 07:52:15 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 07:52:15 2026 -0700"
      },
      "message": "[SPARK-58142] Upgrade JUnit to 6.1.2\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade JUnit to 6.1.2.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes.\n\n- https://junit.org/release-notes/#release-notes-6.1.2\n\n### Does this PR introduce _any_ user-facing change?\n\nNo, this is a test-only dependency change.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #755 from dongjoon-hyun/SPARK-58142.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "b8e4f36c3084ade19c5dd37a98fe7b5eee6c1b2a",
      "tree": "7a3357c9d90b3d5bbcaa8563b9120a6d9fe5371e",
      "parents": [
        "383f8bc9c3eceb6a2137d11d32659e614e781e54"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 07:32:58 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 15 07:32:58 2026 -0700"
      },
      "message": "[SPARK-58140] Upgrade Spark to 4.2.0\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade Apache Spark to 4.2.0.\n\n### Why are the changes needed?\n\nApache Spark 4.2.0 is released officially.\n- https://github.com/apache/spark/releases/tag/v4.2.0\n- https://spark.apache.org/docs/4.2.0/\n- https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.13/4.2.0/\n\nThis PR also excludes the `leveldbjni` variants, too.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo behavior change.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #754 from dongjoon-hyun/SPARK-58140.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "383f8bc9c3eceb6a2137d11d32659e614e781e54",
      "tree": "5a1747a423ed6447f55a2cbbfcb891da1ff33b32",
      "parents": [
        "64696971d813201583f8c8e26dc8bf6ff1fd3bcc"
      ],
      "author": {
        "name": "yalin.sahin",
        "email": "yalin.sahin@tradition.ch",
        "time": "Mon Jul 13 09:15:52 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jul 13 09:15:52 2026 -0700"
      },
      "message": "[SPARK-58101] Remove misleading `spark.submit.deployMode\u003dcluster` from `spark-connect-server-with-spark-cluster.yaml`\n\n### What changes were proposed in this pull request?\n\nRemove the `spark.submit.deployMode: \"cluster\"` line from `examples/spark-connect-server-with-spark-cluster.yaml`.\n\n### Why are the changes needed?\n\nThe example declares cluster deploy mode for a `SparkApplication` whose main class is `org.apache.spark.sql.connect.service.SparkConnectServer`, but the setting is silently ignored: the operator launches the main class inside the driver pod it creates, so the server effectively runs in client mode regardless (verified on-cluster with operator 0.9.0 / chart 1.7.0, Spark 4.1.2 — runtime behavior is identical with `deployMode` set to `cluster` and `client`). Upstream `SparkSubmit` also explicitly rejects cluster deploy mode for the Connect server (\"Cluster deploy mode is not applicable to Spark Connect server\"). The example therefore implies a capability that doesn\u0027t exist, and a user copying it silently gets client mode.\n\nNote this is distinct from `examples/submit-spark-connect-server.sh` (SPARK-51382), which submits via the standalone REST API on port 6066 and bypasses the SparkSubmit-side check; that example is not touched.\n\nSee https://issues.apache.org/jira/browse/SPARK-58101.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo functional change — documentation/example fix only. The example now reflects the actual runtime behavior.\n\n### How was this patch tested?\n\nManually verified on a minikube cluster with spark-kubernetes-operator 0.9.0 (Helm chart 1.7.0) and Spark 4.1.2 attached to a standalone `SparkCluster` master: applying the example with `deployMode` set to `cluster` and to `client` produces identical behavior (Connect server runs in the operator-managed driver pod in client mode in both cases).\n\n### Was this patch authored or co-authored using generative AI tooling?\nGenerated-by: Claude Code (Claude Fable 5)\n\nCloses #753 from yalindogusahin/SPARK-58101.\n\nAuthored-by: yalin.sahin \u003cyalin.sahin@tradition.ch\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "64696971d813201583f8c8e26dc8bf6ff1fd3bcc",
      "tree": "f4f6d9a24f4c032f61f0dde059b82881d7d9b93a",
      "parents": [
        "d3b7eb51091b137317e7e932fe3e0fcc164a7926"
      ],
      "author": {
        "name": "Qi Tan",
        "email": "16416018+TQJADE@users.noreply.github.com",
        "time": "Mon Jul 13 09:02:22 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jul 13 09:02:22 2026 -0700"
      },
      "message": "[SPARK-57868] Requeue after driver not found because of the stale cache\n\n### What changes were proposed in this pull request?\nWhen the operator observes a SparkApplication and the driver pod is absent from the informer (secondary-resource) cache, it previously concluded immediately that the driver had been removed and transitioned the application to Failed (driverUnexpectedRemoved, message \"Driver removed…\").\n\n  This PR makes that determination robust against transient informer-cache staleness:\n\n  - Grace period before verification. If the driver pod is missing from the cache but the application\u0027s current state is younger than spark.kubernetes.operator.reconciler.missingDriverGracePeriodSeconds (default 60), the operator defers the verdict and requeues, giving the informer time to catch up (e.g. right after pod creation or a watch reconnect).\n  - Authoritative live lookup. Once past the grace period, the operator performs a direct, cache-bypassing lookup against the API server via the new SparkAppContext.getDriverPodFromApi(). Only if the API server also reports no driver pod is the application marked driverUnexpectedRemoved. If the pod is present, the informer is treated as stale and the live pod is used for the current reconcile.\n  - Short-lived requeue cadence. Deferrals requeue after spark.kubernetes.operator.reconciler.defaultRequeueIntervalSeconds\n  (default 15), distinct from the coarse steady-state reconciler.intervalSeconds (default 120), so the transient condition is re-checked promptly rather than waiting for the next periodic reconcile.\n  - API-failure handling. If the live lookup itself throws (API server unreachable), the removal verdict is deferred and the reconcile requeues rather than failing the application; the condition self-heals when the API server recovers.\n\n  Two new dynamically-overridable config options are added in SparkOperatorConf, documented in docs/config_properties.md.\n\n### Why are the changes needed?\nAn informer cache can briefly diverge from API-server state. The driver pod appears \"missing\" from the cache even though it is healthy on the API server. The previous logic treated a cache miss as ground truth and failed the application.\n\n### Does this PR introduce _any_ user-facing change?\nyes\n\n### How was this patch tested?\nunit test\n\n### Was this patch authored or co-authored using generative AI tooling?\nYes. Generated-by: Claude Code Opus 4.8 (1M context)\n\nCloses #749 from TQJADE/requeue.\n\nAuthored-by: Qi Tan \u003c16416018+TQJADE@users.noreply.github.com\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "d3b7eb51091b137317e7e932fe3e0fcc164a7926",
      "tree": "4cce249be05923ec97f0beff87cd026a0c49e866",
      "parents": [
        "5bdc47f22814f601647f27b6c92be7771e2b5d75"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri Jul 10 13:46:58 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri Jul 10 13:46:58 2026 -0700"
      },
      "message": "[SPARK-58090] Add `doc.crds.dev` link to Spark custom resources documentation\n\n### What changes were proposed in this pull request?\n\nThis PR aims to add a `doc.crds.dev` link to `docs/spark_custom_resources.md` so that users can browse the full `SparkApplication` and `SparkCluster` CRD schemas on the web.\n\n- https://doc.crds.dev/github.com/apache/spark-kubernetes-operator\n\n### Why are the changes needed?\n\n`doc.crds.dev` is a community service that indexes CRDs from public git repositories and renders all `spec` and `status` fields with types and descriptions in a browsable form. It already indexes this repository, so linking it from the custom resources documentation lets users explore the full CRD schemas without cloning the repository or reading the raw CRD YAML files.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo. This is a documentation-only change.\n\n### How was this patch tested?\n\nManual review. Verified that the link resolves and shows the `SparkApplication` and `SparkCluster` CRDs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #752 from dongjoon-hyun/SPARK-58090.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "5bdc47f22814f601647f27b6c92be7771e2b5d75",
      "tree": "090b8510898d3a1875f4887c71abc9b44b94bb74",
      "parents": [
        "1127bf9fee08b55251401b86c847e6646b520c98"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 08 10:00:34 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 08 10:00:34 2026 -0700"
      },
      "message": "[SPARK-58039] Upgrade `Netty` to 4.2.16.Final\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `Netty` to 4.2.16.Final. In addition, `gradle/verification-metadata.xml` is regenerated accordingly.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes:\n\n- https://netty.io/news/2026/07/06/4-2-16-Final.html\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #750 from dongjoon-hyun/SPARK-58039.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "1127bf9fee08b55251401b86c847e6646b520c98",
      "tree": "d28829ddae1ce6e57ab1b06f18235af03305d803",
      "parents": [
        "d2023062171d6f2163cdfd92343fe43a4bdeabfc"
      ],
      "author": {
        "name": "Zhou JIANG",
        "email": "jiangzho@umich.edu",
        "time": "Tue Jul 07 17:03:44 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jul 07 17:03:44 2026 -0700"
      },
      "message": "[SPARK-57543] Make ConfigOption dynamic override opt-in and filter dynamic config refresh\n\n### What changes were proposed in this pull request?\n\nThis PR makes runtime dynamic config override opt-in and enforces it centrally when dynamic config is refreshed:\n\n1. `ConfigOption`\n    - `enableDynamicOverride` now defaults to `false` — an option must explicitly opt in via `.enableDynamicOverride(true)` to be overridable at runtime.\n    - Each option registers itself (keyed by its config key) into a static registry on construction, exposing `dynamicOverrideEnabledKeys()` as the allow-list of overridable keys. `enableDynamicOverride` on each option remains the single source of truth.\n2. `SparkOperatorConfManager.refresh(Map)` now filters the incoming ConfigMap data against `ConfigOption.dynamicOverrideEnabledKeys()`, applying only overridable keys and logging any dropped (unknown or non-overridable) keys at WARN. `getValue(String)` and `getAll()` therefore no longer surface override values for keys that are never honored.\n3. `SparkOperatorConf` options now set `enableDynamicOverride`explicitly; this also fixes docs for `LOG_CONF` and `PERIODIC_GC_INTERVAL_SECONDS`, which are only read at startup and are non-overridable.\n\nThe existing per-option check in `ConfigOption.resolveValue()` is kept as defense-in-depth.\n\n### Why are the changes needed?\n\nPreviously `refresh()` stored the entire ConfigMap into the override map, and `enableDynamicOverride` defaulted to `true`. As a result, an override value could be stored (and surfaced via `getValue`/`getAll`/the `LOG_CONF` dump) for keys that are not meant to be hot-reloaded, and any new option was overridable by default. Defaulting to opt-in and dropping non-allow-listed keys at refresh time makes the override surface explicit, safer, and self-documenting, and avoids silently ignored config updates.\n\n### Does this PR introduce _any_ user-facing change?\n\nYes. Dynamic config updates for keys that are not explicitly marked `enableDynamicOverride(true)` are now ignored (and logged at WARN) instead of being stored. In particular `spark.logConf` and `spark.kubernetes.operator.periodicGC.intervalSeconds` are no longer dynamically overridable (both are only read at startup). The generated config documentation\u0027s \"Allow Hot Reloading\" column reflects each option\u0027s flag.\n\n### How was this patch tested?\n\nUpdated `ConfigOptionTest` and `SparkOperatorConfManagerTest`, and added `SparkOperatorConfManagerTest#testRefreshDropsUnknownAndNonOverridableKeys` covering the allow-list filtering. Existing `SparkOperatorConfigMapReconcilerTest` continues to exercise the dynamic-config refresh path with an overridable key.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nCo-Authored-By: Claude Opus 4.8\n\nCloses #713 from jiangzho/conf_wl.\n\nAuthored-by: Zhou JIANG \u003cjiangzho@umich.edu\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "d2023062171d6f2163cdfd92343fe43a4bdeabfc",
      "tree": "7f75f0f070b45f30bed401a86dc9ba5b44a412e7",
      "parents": [
        "ccdeff0a592d2cad6ada258fcfc6b6443197bb7b"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jul 06 12:08:44 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jul 06 12:08:44 2026 -0700"
      },
      "message": "[SPARK-57969] Use virtual thread for periodic GC scheduler\n\n### What changes were proposed in this pull request?\n\nThis PR aims to use a virtual thread factory for the periodic GC scheduler thread in `SparkOperator`.\n\n### Why are the changes needed?\n\nThe scheduled task (`SparkOperator::runSystemGc`) is short and periodic, which is an ideal fit for a virtual thread. Virtual threads are always daemon threads, so the existing daemon semantics are preserved.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #748 from dongjoon-hyun/SPARK-57969.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "ccdeff0a592d2cad6ada258fcfc6b6443197bb7b",
      "tree": "b274a005aaa0c4894fec4c718fbff92d7fa97d4f",
      "parents": [
        "f3551e102cfff92233d8662b3ab4d180f550ba8b"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jul 06 11:54:33 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jul 06 11:54:33 2026 -0700"
      },
      "message": "[SPARK-57968] Use virtual threads for `SentinelManager` scheduled executor\n\n### What changes were proposed in this pull request?\n\nThis PR aims to use virtual threads for the `SentinelManager` scheduled executor by providing a virtual thread factory.\n\n### Why are the changes needed?\n\n- The sentinel health-check workers spend most of their time on Kubernetes API calls (network I/O), which is an ideal workload for virtual threads.\n\n- In addition, the worker threads now have meaningful names (`sentinel-0`, `sentinel-1`, ...) instead of the anonymous default (`pool-N-thread-M`), which helps thread-dump analysis.\n\n- Note on behavior change: the workers change from non-daemon platform threads to daemon virtual threads (virtual threads are always daemon). This does not affect JVM liveness in practice because the operator process is kept alive by the `com.sun.net.httpserver` dispatcher thread (non-daemon) and the Java Operator SDK threads.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #747 from dongjoon-hyun/SPARK-57968.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "f3551e102cfff92233d8662b3ab4d180f550ba8b",
      "tree": "4deb79d441f7ac7b3982a7c6ff07a93dd7148a14",
      "parents": [
        "dc4dea995f0c70a8c321172ac08925207ca9a6b7"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sun Jul 05 21:30:17 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sun Jul 05 21:30:17 2026 -0700"
      },
      "message": "[SPARK-57950] Support `spark.redaction.regex` and redact sensitive values in operator logs\n\n### What changes were proposed in this pull request?\n\nThis PR aims to redact sensitive configuration values in operator logs.\n\n- Add a new config option, `spark.redaction.regex`, reusing Apache Spark\u0027s redaction key and default pattern (`(?i)secret|password|token|access[.]?key`).\n- Add `Utils.redactSensitiveInfo` which replaces a value with `*********(redacted)` when its key or value matches the pattern.\n- Apply redaction to the configuration logging on startup (`spark.logConf`).\n- Escape pipes in `DocTable` cells and regenerate `docs/config_properties.md`.\n\n### Why are the changes needed?\n\nWhen `spark.logConf` is enabled, the operator prints all configuration properties without filtering, exposing sensitive values like `*.password` or `spark.hadoop.fs.s3a.access.key` in plain text.\n\n### Does this PR introduce _any_ user-facing change?\n\nYes, sensitive configuration values are now printed as `*********(redacted)`, and the pattern is customizable via `spark.redaction.regex`.\n\n### How was this patch tested?\n\nPass the CIs with the newly added test cases.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #746 from dongjoon-hyun/SPARK-57950.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "dc4dea995f0c70a8c321172ac08925207ca9a6b7",
      "tree": "27b8ae9e7081de3480fb4833147cdc352eccd200",
      "parents": [
        "73097e3f5bb2f3b9eb43f16d4eff9dbcb3c76edb"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sun Jul 05 17:57:16 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sun Jul 05 17:57:16 2026 -0700"
      },
      "message": "[SPARK-57943] MetricsSystem should not expose sink property values\n\n### What changes were proposed in this pull request?\n\nThis PR aims to prevent metrics sink properties from being logged in `MetricsSystem`.\n\n- `registerSinks()` logs only the sink names instead of the entire `sinkPropertiesMap`.\n- The error log on sink creation failure reports only the sink class name, not the properties.\n\n### Why are the changes needed?\n\nSink properties can contain sensitive values such as credentials, which should not be leaked into operator logs.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo, except the log messages.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #745 from dongjoon-hyun/SPARK-57943.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "73097e3f5bb2f3b9eb43f16d4eff9dbcb3c76edb",
      "tree": "0322f514e27f8796cf9369c2904e5523c9d6755e",
      "parents": [
        "d7151c0d5965f7102b153064b72b73c6e12b4fb2"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sun Jul 05 17:56:20 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sun Jul 05 17:56:20 2026 -0700"
      },
      "message": "[SPARK-57942] Replace wildcard RBAC verbs with explicit verb list in Helm chart\n\n### What changes were proposed in this pull request?\n\nThis PR aims to replace the wildcard `verbs: [\u0027*\u0027]` in the Helm chart RBAC rules\n(`operator-rbac.yaml` and `workload-rbac.yaml`) with the explicit verb list\n`get, list, watch, create, update, patch, delete`.\n\n### Why are the changes needed?\n\nThis addresses CIS Kubernetes Benchmark 5.1.3 (minimize wildcard use in Roles and\nClusterRoles). The operator uses informers (`get`/`list`/`watch`), server-side apply\n(`create`/`patch`), status updates and sentinel replacement (`update`), and resource\ncleanup (`delete`). Unused verbs such as `deletecollection` are no longer granted.\n\n### Does this PR introduce _any_ user-facing change?\n\nYes. The generated (Cluster)Roles enumerate verbs explicitly instead of `*`; permissions\nsuch as `deletecollection` are no longer granted. No operator behavior change is expected.\n\n### How was this patch tested?\n\nPass the CIs. Also manually verified with `helm lint --strict` and `helm template`\n(default values and all conditional branches).\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #744 from dongjoon-hyun/SPARK-57942.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "d7151c0d5965f7102b153064b72b73c6e12b4fb2",
      "tree": "c8963b0b3e5d95f61a5c8e7c6c08c57916cd2770",
      "parents": [
        "cac534c62797a30aa4183b0838fa5a99a459bc6e"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sun Jul 05 17:55:11 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sun Jul 05 17:55:11 2026 -0700"
      },
      "message": "[SPARK-57940] Enable `readOnlyRootFilesystem` for operator container in Helm chart\n\n### What changes were proposed in this pull request?\n\nThis PR aims to enable `readOnlyRootFilesystem` for the operator container in the Helm chart.\n\nSince the operator writes temporary files under `java.io.tmpdir` (SPARK-57918), an `emptyDir`\nvolume is mounted at `/tmp`, following the existing `logs-volume` pattern.\n\n### Why are the changes needed?\n\nA read-only root filesystem prevents tampering with the container image at runtime and\ncompletes the CIS Benchmark hardening on top of the existing restricted Pod Security\nStandard compliance.\n\n### Does this PR introduce _any_ user-facing change?\n\nYes. The operator container root filesystem becomes read-only by default; writable paths are\nprovided via `emptyDir` volumes (`/tmp`, `/opt/spark-operator/logs`). Users can override\n`operatorDeployment.operatorPod.operatorContainer.securityContext` if needed.\n\n### How was this patch tested?\n\nPass the CIs. Also manually verified with `helm lint --strict` and `helm template`.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #743 from dongjoon-hyun/SPARK-57940.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "cac534c62797a30aa4183b0838fa5a99a459bc6e",
      "tree": "a07b373c1d7185e39c7bbec21f6f7da8a96f9f52",
      "parents": [
        "17410b055b8425bad4b503d3b290a25ce1062978"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sun Jul 05 12:16:59 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sun Jul 05 12:16:59 2026 -0700"
      },
      "message": "[SPARK-57934] Support opt-in `NetworkPolicy` for operator pod in Helm chart\n\n### What changes were proposed in this pull request?\n\nThis PR aims to support an opt-in `NetworkPolicy` for the operator pod in the Helm chart.\n\nWhen `operatorDeployment.networkPolicy.enable` is `true` (default: `false`), all ingress to the\noperator pod is denied except the health probe port (open to any source, for kubelet probes) and\nthe metrics port, which is reachable only from the peers listed in\n`operatorDeployment.networkPolicy.metricsIngress`.\n\n```yaml\noperatorDeployment:\n  networkPolicy:\n    enable: true\n    metricsIngress:\n      - namespaceSelector:\n          matchLabels:\n            kubernetes.io/metadata.name: \"monitoring\"\n```\n\n### Why are the changes needed?\n\nThe operator serves unauthenticated plain-HTTP probe and metrics endpoints, and the chart\nprovided no way to restrict in-cluster access to them.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo, because this is disabled by default.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #742 from dongjoon-hyun/SPARK-57934.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "17410b055b8425bad4b503d3b290a25ce1062978",
      "tree": "9d4d03b196eadf3c2bb4733a2c33b7fa35dcc6ab",
      "parents": [
        "b418dd1646dae6996aa6c4299ddeaa363e22ac86"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 02 19:05:39 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 02 19:05:39 2026 -0700"
      },
      "message": "[SPARK-57919] Add `gradle/verification-metadata.xml` for dependency verification\n\n### What changes were proposed in this pull request?\n\nThis PR aims to add `gradle/verification-metadata.xml` to enable Gradle dependency verification.\n\nThe file was generated with the following command and covers all resolved artifacts with\n`sha512` checksums. The regeneration step is documented in `README.md`.\n\n```bash\n./gradlew --write-verification-metadata sha512 build\n```\n\n### Why are the changes needed?\n\nGradle verifies the checksum of every resolved artifact against this file and fails the build\non any mismatch, protecting the build from tampered dependencies.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #741 from dongjoon-hyun/SPARK-57919.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "b418dd1646dae6996aa6c4299ddeaa363e22ac86",
      "tree": "5ef492d3242c4c96a3c1f8f131bc8817f3a9f29d",
      "parents": [
        "4077c8d26f7c94e79aa6273f4d1a95334dd79261"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 02 17:32:13 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 02 17:32:13 2026 -0700"
      },
      "message": "[SPARK-57917] Fix workload RoleBinding name in operator namespace\n\n### What changes were proposed in this pull request?\n\nThis PR fixes the workload `RoleBinding` name in `workload-rbac.yaml` to use\n`$workloadResources.roleBinding.name` instead of `$workloadResources.serviceAccount.name`\nwhen `workloadResources.namespaces.data` is empty, consistent with the per-namespace branch.\n\n### Why are the changes needed?\n\nThe `RoleBinding` was named `spark` (the service account name) instead of the intended\n`spark-workload-rolebinding`.\n\n### Does this PR introduce _any_ user-facing change?\n\nYes. The `RoleBinding` in the operator namespace is now named `spark-workload-rolebinding`\ninstead of `spark`.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #739 from dongjoon-hyun/dongjoon/nervous-leavitt-747dd2.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "4077c8d26f7c94e79aa6273f4d1a95334dd79261",
      "tree": "a568aa199d88e7e651b2229bd0f4eda3d06d5925",
      "parents": [
        "78f26109c8e366570fb51d38560eba707a88e423"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 02 17:30:40 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 02 17:30:40 2026 -0700"
      },
      "message": "[SPARK-57918] Use `Files.createTempFile` instead of `File.createTempFile` to ensure owner-only temp file\n\n### What changes were proposed in this pull request?\n\nThis PR aims to replace `File.createTempFile` with `Files.createTempFile` in\n`SparkAppResourceSpecFactory` to guarantee that the pod template temp file is\ncreated with owner-only permissions.\n\n### Why are the changes needed?\n\n`SparkAppResourceSpecFactory.createLocalFileForPodTemplateSpec` writes the driver/executor\n`PodTemplateSpec` (which may contain sensitive information such as environment variables)\nto a local temporary file. `File.createTempFile` creates the file with permissions subject\nto the process `umask` (e.g., `0644`), which may allow other users on the same host to read it.\n`java.nio.file.Files.createTempFile` guarantees owner-only permissions (`0600`) on\nPOSIX file systems.\n\nBefore:\n\n```\n-rw-r--r--  ... \u003cuid\u003e-driver-XXXX.json\n```\n\nAfter:\n\n```\n-rw-------  ... \u003cuid\u003e-driver-XXXX.json\n```\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #740 from dongjoon-hyun/SPARK-57918.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "78f26109c8e366570fb51d38560eba707a88e423",
      "tree": "16efe0c98937fb5ef1258ac2e640a3942501016f",
      "parents": [
        "6cbc6917545d84134f377733f3ea9e4dc597281c"
      ],
      "author": {
        "name": "Tan Qi",
        "email": "16416018+TQJADE@users.noreply.github.com",
        "time": "Thu Jul 02 16:00:33 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 02 16:00:33 2026 -0700"
      },
      "message": "[SPARK-57906] Upgrade `java-operator-sdk` to 5.4.0\n\n### What changes were proposed in this pull request?\n\nUpdate operator-sdk version\n\n### Why are the changes needed?\n\ninclude features \u0026 bug fix from https://github.com/operator-framework/java-operator-sdk/releases/tag/v5.4.0\nhttps://github.com/operator-framework/java-operator-sdk/releases/tag/v5.4.0\n### Does this PR introduce _any_ user-facing change?\n\nno\n\n### How was this patch tested?\n\nunit test\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nno\n\nCloses #735 from TQJADE/patch-1.\n\nAuthored-by: Tan Qi \u003c16416018+TQJADE@users.noreply.github.com\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "6cbc6917545d84134f377733f3ea9e4dc597281c",
      "tree": "6cde5d48288cbf98ebdcdab6cc20c1e4de8d8472",
      "parents": [
        "7ca6320b4f754b7d1a86c9fb215d966983332e06"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 02 11:56:01 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 02 11:56:01 2026 -0700"
      },
      "message": "[SPARK-57888] Upgrade JUnit to 6.1.1\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade JUnit to 6.1.1.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes from the JUnit 6.1.x line.\n\n- https://docs.junit.org/6.1.1/release-notes/\n\n### Does this PR introduce _any_ user-facing change?\n\nNo, this is a test-only dependency change.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #738 from dongjoon-hyun/SPARK-57888.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "7ca6320b4f754b7d1a86c9fb215d966983332e06",
      "tree": "9e61ef6dbc32aec1d062aedf72be9d50f326346c",
      "parents": [
        "d9401c5259b1a05b86a614a3ddb6708ad0ee8cbb"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 02 11:54:56 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 02 11:54:56 2026 -0700"
      },
      "message": "[SPARK-57887] Upgrade `spotbugs-gradle-plugin` to 6.5.8\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `spotbugs-gradle-plugin` to 6.5.8.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes.\n\n- https://github.com/spotbugs/spotbugs-gradle-plugin/releases/tag/6.5.8\n- https://github.com/spotbugs/spotbugs-gradle-plugin/releases/tag/6.5.7\n\n### Does this PR introduce _any_ user-facing change?\n\nNo, this is a build-time dependency change.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nThis patch had conflicts when merged, resolved by\nCommitter: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n\nCloses #737 from dongjoon-hyun/SPARK-57887.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "d9401c5259b1a05b86a614a3ddb6708ad0ee8cbb",
      "tree": "f70eca7514d378216a2f78c558c2bf8afa4a3f47",
      "parents": [
        "a434e94e273e93a400edbeb2a52da90c3055857d"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 02 11:53:27 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jul 02 11:53:27 2026 -0700"
      },
      "message": "[SPARK-57886] Upgrade `spotless-plugin-gradle` to 8.8.0\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `spotless-plugin-gradle` to 8.8.0.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes and improvements.\n\n- https://github.com/diffplug/spotless/releases/tag/gradle%2F8.8.0\n\n### Does this PR introduce _any_ user-facing change?\n\nNo. This is a dev-only change.\n\n### How was this patch tested?\n\nPass the CI\n\nCloses #736 from dongjoon-hyun/SPARK-57886.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "a434e94e273e93a400edbeb2a52da90c3055857d",
      "tree": "7212e535decdc7684204157d80bb637d6e1c7656",
      "parents": [
        "8d524902b26f424543da72bed1aa7afe281c6123"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 23:40:18 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 23:40:18 2026 -0700"
      },
      "message": "[SPARK-57789] Update and bundle `LICENSE-binary` and `licenses-binary` to describe all bundled libraries\n\n### What changes were proposed in this pull request?\n\nThis PR updates `LICENSE-binary` to describe all third-party libraries bundled in the operator\nshadow jar, adds the `licenses-binary/` directory with the license texts of the non-Apache\ncomponents, and bundles them into the jar like Apache Hadoop\u0027s `hadoop-client-runtime`.\n\n- `LICENSE-binary`: rewritten in Apache Spark\u0027s `LICENSE-binary` format, listing the 170 bundled\n  artifacts grouped by license. The list was verified against the actual shadow jar contents.\n- `licenses-binary/`: adds the 11 license texts for the non-Apache-licensed components.\n- `spark-operator/build.gradle`: the shadow jar now ships `LICENSE-binary`/`NOTICE-binary` as\n  `META-INF/LICENSE`/`META-INF/NOTICE` plus `META-INF/licenses-binary/`, instead of the\n  source-release `LICENSE`/`NOTICE`.\n- `.github/.licenserc.yaml`: skips the header check for `licenses-binary/**`.\n\n### Why are the changes needed?\n\n`LICENSE-binary` was a copy of `LICENSE` and did not enumerate any bundled dependency. Per the\n[ASF licensing policy](https://www.apache.org/legal/resolved.html), a convenience binary must account for all bundled licenses and include the non-Apache license texts.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\nManually verified that `META-INF/LICENSE` in the shadow jar is identical to `LICENSE-binary`, and\nthat every listed artifact actually has classes in the shadow jar (relocation-aware comparison\nagainst the original jars from Maven Central).\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #721 from dongjoon-hyun/SPARK-57789.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "8d524902b26f424543da72bed1aa7afe281c6123",
      "tree": "65875df4e209b0d8cee0915977855c8bf715a5ab",
      "parents": [
        "9c521a6d6486f5e9377a1550b06b8a3af6526367"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 22:57:51 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 22:57:51 2026 -0700"
      },
      "message": "[SPARK-57875] Exclude Hadoop `filecache` classes from shadow jar\n\n### What changes were proposed in this pull request?\n\nThis PR excludes the Hadoop `filecache` package (`org/apache/hadoop/filecache/**`) from the\noperator shadow jar by adding a class-path filter to the `shadowJar` block in\n`spark-operator/build.gradle`.\n\n### Why are the changes needed?\n\nThe operator never uses Hadoop `DistributedCache`. While the `minimize` step already drops the\nunreferenced `DistributedCache` class, the package directory entry and `package-info.class`\n(from `hadoop-client-api`) still remain in the shadow jar. This change removes those leftover\nentries explicitly, consistent with the existing MapReduce exclusions.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #734 from dongjoon-hyun/SPARK-57875.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "9c521a6d6486f5e9377a1550b06b8a3af6526367",
      "tree": "c0adab4f5390947c846e034592fd5fba1515704a",
      "parents": [
        "7d73a2d12fe59d42598f6a5446bc19655c607610"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 22:20:37 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 22:20:37 2026 -0700"
      },
      "message": "[SPARK-57874] Exclude Hadoop MapReduce resources from shadow jar\n\n### What changes were proposed in this pull request?\n\nThis PR excludes the remaining Hadoop MapReduce resource files from the operator shadow jar.\n\n- `mapred-default.xml`\n- `META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider`\n\n### Why are the changes needed?\n\nSPARK-57871 excluded the Hadoop MapReduce classes, but these two resource files remain.\nThey are dead resources because the classes that use them (`JobConf`, `YarnClientProtocolProvider`,\n`LocalClientProtocolProvider`) are already excluded. This shrinks the shadow jar by ~21KB.\n\n- https://github.com/apache/spark-kubernetes-operator/pull/732\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #733 from dongjoon-hyun/SPARK-57874.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "7d73a2d12fe59d42598f6a5446bc19655c607610",
      "tree": "b9e6138e5f349537d0c0a9ad61feb1a7a10093d0",
      "parents": [
        "3444e976f226525247d5e243e1323c353a113e13"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 17:44:35 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 17:44:35 2026 -0700"
      },
      "message": "[SPARK-57871] Exclude Hadoop MapReduce classes from shadow jar\n\n### What changes were proposed in this pull request?\n\nThis PR excludes Hadoop MapReduce classes (`org/apache/hadoop/mapred/**` and\n`org/apache/hadoop/mapreduce/**`) from the operator shadow jar by adding\nclass-path filters to the `shadowJar` block in `spark-operator/build.gradle`.\n\n### Why are the changes needed?\n\nThe operator manages Spark applications on Kubernetes and never runs the MapReduce\nframework. The bundled MapReduce classes are dead weight in the runtime classpath\nand shaded jar. Removing them drops 403 classes (~1.7 MB uncompressed) that\noriginate from `hadoop-client-api`, shrinking the shadow jar by ~774 KB.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Fable 5\n\nCloses #732 from dongjoon-hyun/SPARK-57871.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "3444e976f226525247d5e243e1323c353a113e13",
      "tree": "2304ef0f8552a897324e93bf3b67c05ffa3a5449",
      "parents": [
        "d2118d6192cf05fec55998048dd5b951925e7077"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 10:53:15 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 10:53:15 2026 -0700"
      },
      "message": "[SPARK-57859] Exclude `com.esotericsoftware` dependency\n\n### What changes were proposed in this pull request?\n\nThis PR excludes the transitive `com.esotericsoftware` dependency from the shaded operator jar.\n\n### Why are the changes needed?\n\n`com.esotericsoftware:kryo-shaded` is pulled in via `spark-core` (directly and through `com.twitter:chill`) as a serializer. The operator only builds driver/submission specs and never runs it, like the other already-excluded runtime codecs and serializers (`org.lz4`, `org.xerial.snappy`, `com.github.luben`, `org.roaringbitmap`, `com.ning`).\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #731 from dongjoon-hyun/SPARK-57859.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "d2118d6192cf05fec55998048dd5b951925e7077",
      "tree": "189754df48a710ed7f98bd49d63e48aa4a3bce7f",
      "parents": [
        "751592199df68eba993a533394d0317ab12284a7"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 09:04:54 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 09:04:54 2026 -0700"
      },
      "message": "[SPARK-57857] Exclude `com.ning` dependency\n\n### What changes were proposed in this pull request?\n\nThis PR excludes the transitive `com.ning:compress-lzf` dependency from the shaded operator jar.\n\n### Why are the changes needed?\n\n`compress-lzf` is pulled in via `spark-core` as a shuffle/broadcast compression codec. The operator only builds driver/submission specs and never runs it, like the other already-excluded runtime codecs (`org.lz4`, `org.xerial.snappy`, `com.github.luben`, `org.roaringbitmap`).\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #730 from dongjoon-hyun/SPARK-57857.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "751592199df68eba993a533394d0317ab12284a7",
      "tree": "b182a0e2eeaa99a2a5c623ed58866bb88a71115e",
      "parents": [
        "7420db7ac1043971a97d836809d1490516299cf5"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 08:46:30 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 08:46:30 2026 -0700"
      },
      "message": "[SPARK-57856] Authenticate `minikube` GitHub API requests to avoid CI rate limit failures\n\n### What changes were proposed in this pull request?\n\nThis PR adds `GITHUB_TOKEN` to the `medyagh/setup-minikube` steps in\n`.github/workflows/build_and_test.yml` (the `k8s-integration-tests` and\n`helm-tests` jobs) so that minikube authenticates its GitHub API requests.\n\n### Why are the changes needed?\n\nWhen starting minikube with a pinned `kubernetes-version` (e.g. `1.36.0`),\nminikube resolves the exact version by querying the GitHub API\n(`api.github.com/repos/kubernetes/kubernetes/releases/tags/v1.36.0`).\nWithout a token this request is unauthenticated and limited to 60 requests\nper hour per runner IP. Because these jobs fan out across a large matrix\n(`max-parallel: 20`), the shared limit is exhausted and the jobs fail with:\n\n- https://github.com/apache/spark-kubernetes-operator/actions/runs/28527974981/job/84569529838\n\n```\nExiting due to K8S_FAIL_CONNECT: error fetching Kubernetes version list from GitHub:\nGET https://api.github.com/repos/kubernetes/kubernetes/releases/tags/v1.36.0:\n403 API rate limit exceeded for 13.71.231.42.\n\n(But here\u0027s the good news: Authenticated requests get a higher rate limit.\nCheck out the documentation for more details.) [rate reset in 41m27s]\n```\n\nProviding `GITHUB_TOKEN` switches minikube to authenticated requests, raising\nthe limit to 1,000 requests/hour per repository and removing the flaky failure.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo. This is a CI-only change.\n\n### How was this patch tested?\n\nExisting CI. The `k8s-integration-tests` and `helm-tests` jobs run minikube\nsetup and pass without hitting the GitHub API rate limit.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #729 from dongjoon-hyun/SPARK-57856.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "7420db7ac1043971a97d836809d1490516299cf5",
      "tree": "f99212bed164aad673da274a7ab0cf6f1cc2886f",
      "parents": [
        "8ad18867ed92cb36e110235830a1367dc7823266"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 08:16:04 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 08:16:04 2026 -0700"
      },
      "message": "[SPARK-57845] Exclude `org.roaringbitmap` dependency\n\n### What changes were proposed in this pull request?\n\nThis PR excludes the transitive `org.roaringbitmap:RoaringBitmap` dependency from the shaded operator jar by adding `exclude group: \"org.roaringbitmap\"` to the Spark dependency declarations in the `spark-operator` and `spark-submission-worker` modules.\n\n### Why are the changes needed?\n\n`RoaringBitmap` is pulled in transitively via `org.apache.spark:spark-network-common`, where Spark uses it for shuffle map status tracking (`HighlyCompressedMapStatus`). The operator only builds driver/submission specs and does not run the shuffle/executor runtime, so this library is unused. It belongs to the same category of shuffle/storage runtime libraries that are already excluded (`org.lz4`, `org.rocksdb`, `org.fusesource.leveldbjni`, `org.xerial.snappy`, `com.github.luben`). Removing it keeps the shaded jar lean and reduces the dependency surface.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #728 from dongjoon-hyun/SPARK-57845.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "8ad18867ed92cb36e110235830a1367dc7823266",
      "tree": "6b3593341eaaec80fd9f749f0f77f28b049f6c76",
      "parents": [
        "5ce9584d3ceb2ea480dd563feacddb8afcf82e15"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 08:15:22 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jul 01 08:15:22 2026 -0700"
      },
      "message": "[SPARK-57844] Exclude Hadoop YARN classes from shadow jar\n\n### What changes were proposed in this pull request?\n\nThis PR excludes Hadoop YARN classes (`org/apache/hadoop/yarn/**`) from the operator shadow jar by adding a class-path filter to the `shadowJar` block in `spark-operator/build.gradle`.\n\nNote that the individual `org.apache.hadoop:hadoop-yarn-api`, `hadoop-yarn-client`, and `hadoop-yarn-common` artifacts are not present in the dependency graph. Spark\u0027s `spark-core` / `spark-kubernetes` pull in the shaded `hadoop-client-api` uber-jar, which bundles the YARN classes internally. Therefore these classes are filtered at packaging time rather than excluded as Maven modules. This follows the existing precedent of excluding Hadoop `webapps/**` (HDFS/YARN UI).\n\n### Why are the changes needed?\n\nThe operator manages Spark applications on Kubernetes and never uses YARN. The bundled YARN classes are dead weight in the runtime classpath and shaded jar. Removing them drops 2,766 classes (~20 MB uncompressed) that originate from `hadoop-client-api`.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #727 from dongjoon-hyun/SPARK-57844.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "5ce9584d3ceb2ea480dd563feacddb8afcf82e15",
      "tree": "cd1e504bd48618da181c9c0caedb847848b025e8",
      "parents": [
        "891f3ebfdfe84d2ad1793d0b83350608289d010a"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 30 22:22:24 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 30 22:22:24 2026 -0700"
      },
      "message": "[SPARK-57801] Upgrade `fabric8` to 7.8.0\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `fabric8` (Kubernetes Client) to 7.8.0.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes and improvements.\n- https://github.com/fabric8io/kubernetes-client/releases/tag/v7.8.0\n  - https://github.com/fabric8io/kubernetes-client/issues/7907\n  - https://github.com/fabric8io/kubernetes-client/issues/7847\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #726 from dongjoon-hyun/SPARK-57801.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "891f3ebfdfe84d2ad1793d0b83350608289d010a",
      "tree": "75b3bd11eced5bff506bfc58d24b9df1ff787552",
      "parents": [
        "aa702a7c21326b1b75b73da48de447d3610c7a15"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 30 22:00:46 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 30 22:00:46 2026 -0700"
      },
      "message": "[SPARK-57799] Exclude `net.razorvine` dependency\n\n### What changes were proposed in this pull request?\n\nThis PR excludes the `net.razorvine` dependency (transitively pulled in by Spark) from `spark-submission-worker` (`spark-kubernetes`) and `spark-operator` (`spark-core`).\n\n### Why are the changes needed?\n\n`net.razorvine:pickle` is a PySpark serialization library that the operator never uses. Excluding it removes an unnecessary dependency from the runtime classpath and shaded jar.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nThis patch had conflicts when merged, resolved by\nCommitter: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n\nCloses #725 from dongjoon-hyun/SPARK-57799.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "aa702a7c21326b1b75b73da48de447d3610c7a15",
      "tree": "7910f416cb00f69f7c1c2e25da7d820392052417",
      "parents": [
        "63fd0880647fb5344e192b982ef0bb91d4343ca9"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 30 21:59:03 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 30 21:59:03 2026 -0700"
      },
      "message": "[SPARK-57798] Exclude `net.sf.py4j` dependency\n\n### What changes were proposed in this pull request?\n\nThis PR excludes the transitive `net.sf.py4j:py4j` dependency from the operator shadow jar.\n\n### Why are the changes needed?\n\n`py4j` is PySpark\u0027s Python-to-JVM gateway, pulled in transitively by `spark-core`. It only runs inside the Spark driver pod, never in the operator JVM, yet all 71 `py4j/` classes were bundled into the operator fat jar as dead weight.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #724 from dongjoon-hyun/SPARK-57798.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "63fd0880647fb5344e192b982ef0bb91d4343ca9",
      "tree": "b5b2c5aade7d4ae27bc718704d739c3c8f359856",
      "parents": [
        "5526266cc8cb5f26b3dfbd268e861e7bc253f570"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 30 19:55:14 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 30 19:55:14 2026 -0700"
      },
      "message": "[SPARK-57796] Exclude `org.checkerframework` dependency\n\n### What changes were proposed in this pull request?\n\nThis PR excludes the `org.checkerframework` dependency (transitively pulled in by Spark via Guava) from the Spark dependency configurations in both `spark-submission-worker` (`spark-kubernetes`) and `spark-operator` (`spark-core`).\n\n### Why are the changes needed?\n\n`org.checkerframework:checker-qual` is a compile-time annotation-only artifact that is not needed at runtime. Excluding it removes an unnecessary dependency from the operator\u0027s runtime classpath and shaded jar.\n\nBefore this change, `checker-qual` appeared in the runtime classpath:\n\n```\n$ ./gradlew -q :spark-operator:dependencies --configuration runtimeClasspath | grep checker\n     |    |                   +--- org.checkerframework:checker-qual:3.33.0\n```\n\nAfter this change, it is no longer present.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs and verify that `org.checkerframework:checker-qual` is removed from the runtime classpath:\n\n```\n$ ./gradlew -q :spark-operator:dependencies --configuration runtimeClasspath | grep -ic checker\n0\n```\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #723 from dongjoon-hyun/SPARK-57796.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "5526266cc8cb5f26b3dfbd268e861e7bc253f570",
      "tree": "eaef96129153b70ed984c12c8625fcbba87f3dd6",
      "parents": [
        "9ef651f2fc136b4ae597ccd894d3f90152e78add"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 30 17:43:28 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 30 17:43:28 2026 -0700"
      },
      "message": "[SPARK-57794] Exclude 3rd party Maven metadata from shadow jar\n\n### What changes were proposed in this pull request?\n\nThis PR excludes 3rd party Maven metadata (`META-INF/maven/**`) from the `spark-operator` shadow jar.\n\n### Why are the changes needed?\n\n`minimize()` strips unreachable classes but leaves the matching `pom.properties`/`pom.xml` behind, so orphaned Maven metadata remains for fully-removed dependencies. These files are build-time only and serve no runtime purpose.\n\nFor the record, we provide SBOM for trace-ability.\n- #332\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nManual test.\n\n**BEFORE**\n```\n$ gradle build -x test\n$ jar tvf spark-operator/build/libs/spark-kubernetes-operator-1.0.0-SNAPSHOT-all.jar | grep maven | wc -l\n     696\n```\n\n**AFTER**\n```\n$ gradle build -x test\n$ jar tvf spark-operator/build/libs/spark-kubernetes-operator-1.0.0-SNAPSHOT-all.jar | grep maven | wc -l\n       0\n```\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #722 from dongjoon-hyun/SPARK-57794.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "9ef651f2fc136b4ae597ccd894d3f90152e78add",
      "tree": "85163289de2d594c3d8848b0316dbe4f27c326c9",
      "parents": [
        "5283a09f751f00b3a8f04a6ce04c1ec31647cf38"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 30 15:39:58 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 30 15:39:58 2026 -0700"
      },
      "message": "[SPARK-57788] Prevent Build CI cancellation between different commits in `main` branch\n\n### What changes were proposed in this pull request?\n\nThis PR scopes the `Build` workflow `concurrency.group` by commit SHA on `main`, while keeping the existing branch-level grouping for all other branches:\n\n```yaml\ngroup: ${{ github.workflow }}-${{ github.ref }}${{ github.ref \u003d\u003d \u0027refs/heads/main\u0027 \u0026\u0026 format(\u0027-{0}\u0027, github.sha) || \u0027\u0027 }}\n```\n\n### Why are the changes needed?\n\nWith the branch-only key, all `Build` runs on a branch shared one concurrency group, and `cancel-in-progress: true` made unrelated runs cancel each other. Re-running an older commit\u0027s failed jobs on `main` cancelled the newest commit\u0027s in-progress CI.\n\nAppending the commit SHA only on `main` ensures different commits no longer cancel each other there, while feature/PR branches keep the original behavior where a new push cancels the previous in-progress run on the same branch.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nManual review.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #720 from dongjoon-hyun/SPARK-57788.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "5283a09f751f00b3a8f04a6ce04c1ec31647cf38",
      "tree": "016c713fe4c81000580d8dd85bd87eaf244d319a",
      "parents": [
        "ea1b8be113511118b2281e0e32e9947b932f18ea"
      ],
      "author": {
        "name": "Attila Mészáros",
        "email": "a_meszaros@apple.com",
        "time": "Tue Jun 30 13:49:05 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 30 13:49:05 2026 -0700"
      },
      "message": "[SPARK-57466] Support `s.k.o.dynamicConfig.source` to use mounted-`ConfigMap`\n\n### What changes were proposed in this pull request?\n\n  This PR adds a second, mount-based source for dynamic configuration overrides and makes the dynamic-config mechanism pluggable, while retaining the existing ConfigMap-informer behavior for backwards compatibility.\n\n  A new config option `spark.kubernetes.operator.dynamicConfig.source` selects the source:\n\n  - **`configMap`** (default) — the existing behavior: a dedicated `Operator` runs `SparkOperatorConfigMapReconciler`, watching a ConfigMap via a Kubernetes informer. Requires RBAC to read ConfigMaps.\n  - **`file`** — a new `DynamicConfigMonitor` periodically reloads a properties file mounted from a ConfigMap volume, requiring no extra RBAC.\n\n  Both paths funnel through the same `SparkOperatorConfManager.refresh(...)` + watched-namespace update logic, so runtime behavior is identical once overrides are applied.\n\n  Key changes:\n\n  - **New `DynamicConfigMonitor`** (`config/DynamicConfigMonitor.java`) — a single-thread scheduled reader of a properties file with `start()` / `stop()` / `isRunning()` lifecycle and change detection against the last loaded\n  snapshot.\n  - **`SparkOperator`** — `registerDynamicConfig()` dispatches on `dynamicConfig.source`: the `file` source returns a `DynamicConfigMonitor`; the `configMap` source registers the second informer `Operator` (restored\n  `registerSparkOperatorConfMonitor()` / `overrideConfigMonitorConfigs()`). The operator still tracks a `List\u003cOperator\u003e` plus an optional monitor.\n  - **`SparkOperatorConf`** — added `dynamicConfig.source`, `dynamicConfig.filePath`, `dynamicConfig.reloadIntervalSeconds`; retained `dynamicConfig.selector` and `dynamicConfig.reconcilerParallelism`.\n  - **Probes** (`HealthProbe`, `ReadinessProbe`, `ProbeService`, `ProbeUtil`) — health/readiness cover all registered operators (so the informer operator is checked under `configMap`) and additionally the monitor\u0027s running\n  state under `file`.\n  - **Helm chart**:\n    - `dynamicConfig.source` added to `values.yaml` (default `configMap`).\n    - The ConfigMap `Role`/`RoleBinding` is created only when `enable \u0026\u0026 source \u003d\u003d configMap`.\n    - The dynamic-config volume and mount are added only when `enable \u0026\u0026 source \u003d\u003d file`.\n    - The dynamic ConfigMap renders a single `spark-operator-dynamic.properties` key for the `file` source, or raw per-key data for the `configMap` source.\n\n  ### Why are the changes needed?\n\n  The informer-based source requires the operator to hold cluster/namespace RBAC on ConfigMaps and runs a second JOSDK `Operator` purely to watch one ConfigMap. The mount-based source removes that RBAC requirement and the extra\n  informer by letting the kubelet project ConfigMap changes onto a volume that the operator polls. Rather than forcing a migration, this PR offers the mount-based approach as an opt-in alternative while keeping the informer as\n  the default, so existing deployments are unaffected.\n\n  ### Does this PR introduce _any_ user-facing change?\n\n  Yes, additive and backwards-compatible:\n\n  - New `spark.kubernetes.operator.dynamicConfig.source` option (default `configMap`). With the default, behavior is unchanged from the released operator.\n  - New `dynamicConfig.filePath` and `dynamicConfig.reloadIntervalSeconds` options, used only by the `file` source.\n  - New `dynamicConfig.source` Helm value (default `configMap`). Opting into `file` switches the chart to mount the ConfigMap as a volume (rendered under a single `spark-operator-dynamic.properties` key) and drops the ConfigMap\n  RBAC `Role`/`RoleBinding`.\n\n  No change for users who do not set `dynamicConfig.source`.\n\n  ### How was this patch tested?\n\n  - Unit tests for both sources:\n    - `configMap` — `SparkOperatorConfigMapReconcilerTest` and the configMap-source case in `SparkOperatorTest` (asserts a second `Operator` is registered and no `DynamicConfigMonitor`).\n    - `file` — `DynamicConfigMonitorTest` (initial load, no-op reload when unchanged, refresh + namespace update on change, `isRunning()` lifecycle) and the file-source case in `SparkOperatorTest`.\n  - Updated `HealthProbeTest`, `ReadinessProbeTest`, `ProbeServiceTest` for the `List\u003cOperator\u003e` + monitor signatures.\n  - e2e `watched-namespaces` chainsaw test runs against the `file` source (`tests/e2e/helm/dynamic-config-values{,-2}.yaml` set `source: file`); ConfigMaps use the `spark-operator-dynamic.properties` layout and the propagation\n  wait was raised to account for kubelet volume-sync plus the reload interval.\n\n  ### Was this patch authored or co-authored using generative AI tooling?\n\n  Generated-by: Claude Code (Claude Opus 4.8)\n\nCloses #707 from csviri/dynamic-config-map-name-informer-mount.\n\nAuthored-by: Attila Mészáros \u003ca_meszaros@apple.com\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "ea1b8be113511118b2281e0e32e9947b932f18ea",
      "tree": "ebf300535be9e5617dc70e1e8f37b0a04a701992",
      "parents": [
        "9d5f1c2143859f5ba1e9dfb0fff0151f0bd8000f"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jun 29 14:38:53 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jun 29 14:38:53 2026 -0700"
      },
      "message": "[SPARK-57757] Upgrade `Gradle` to 9.6.1\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade Gradle to `9.6.1`.\n\n### Why are the changes needed?\n\nTo use the latest version of Gradle (`9.6.1`, released on 2026-06-26).\n- https://github.com/gradle/gradle/releases/tag/v9.6.1\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #719 from dongjoon-hyun/SPARK-57757.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "9d5f1c2143859f5ba1e9dfb0fff0151f0bd8000f",
      "tree": "a1c236af7fda212f921da0c66feb23660b55ef87",
      "parents": [
        "8813726a4d7b8f1669ceb5c744e9845420e430d4"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jun 25 18:25:47 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jun 25 18:25:47 2026 -0700"
      },
      "message": "[SPARK-57706] Add one-zone `Spark Connect Server` example\n\n### What changes were proposed in this pull request?\n\nThis PR aims to add one-zone `Spark Connect Server` example.\n\n### Why are the changes needed?\n\nOne-zone deployment is recommended to reduce the network cost among pods.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nManual review.\n\n```\n$ kubectl get pod -l spark.operator/spark-app-name\u003dspark-connect-server-usw2-az2\nNAME                                     READY   STATUS    RESTARTS   AGE\nspark-connect-server-exec-1              1/1     Running   0          87s\nspark-connect-server-exec-2              1/1     Running   0          87s\nspark-connect-server-exec-3              1/1     Running   0          87s\nspark-connect-server-usw2-az2-0-driver   1/1     Running   0          93s\n```\n\n```\n$ kubectl get pod -l spark.operator/spark-app-name\u003dspark-connect-server-usw2-az2 -oyaml | grep zone-id\n      topology.k8s.aws/zone-id: usw2-az2\n      topology.k8s.aws/zone-id: usw2-az2\n      topology.k8s.aws/zone-id: usw2-az2\n      topology.k8s.aws/zone-id: usw2-az2\n```\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nNo.\n\nCloses #718 from dongjoon-hyun/SPARK-57706.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "8813726a4d7b8f1669ceb5c744e9845420e430d4",
      "tree": "b7566b9e3d5a04f03f8219ffc9b43b056b8cbc9c",
      "parents": [
        "4779d71ae93d72ca04e6d58807ff0c81ad042410"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jun 22 21:53:34 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jun 22 21:53:34 2026 -0700"
      },
      "message": "[SPARK-57630] Fix `publish_snapshot_chart` GitHub Actions job\n\n### What changes were proposed in this pull request?\n\nThis PR aims to fix `publish_snapshot_chart` GitHub Actions job by updating `burnett01/rsync-deployments` to an ASF-approved commit hash.\n\n### Why are the changes needed?\n\n`publish_snapshot_chart` is broken since last month because the old hash is outdated on the ASF `approved_patterns.yml` allowlist.\n- https://github.com/apache/spark-kubernetes-operator/actions/workflows/publish_snapshot_chart.yml\n  - https://github.com/apache/spark-kubernetes-operator/actions/runs/27994804602\n\n    \u003e The action burnett01/rsync-deployments7659d600d8bdd035bb5cdfba1d4bd0dd4a307ca6 is not allowed in apache/spark-kubernetes-operator because ...\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nCI-only change. Verified the new hash is in ASF `approved_patterns.yml`.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Claude Opus 4.8)\n\nCloses #717 from dongjoon-hyun/SPARK-57630.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "4779d71ae93d72ca04e6d58807ff0c81ad042410",
      "tree": "657cb208014eb7c0381276b7faedaa7b4e178625",
      "parents": [
        "8d539ccdf7990c58bce6207d6512649b74ea55b5"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jun 22 21:51:10 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jun 22 21:51:10 2026 -0700"
      },
      "message": "[SPARK-57629] Upgrade `spotless-plugin-gradle` to 8.7.0\n\n### What changes were proposed in this pull request?\n\nThis PR upgrades `spotless-plugin-gradle` to `8.7.0`.\n\n### Why are the changes needed?\n\nTo use the latest release (`8.7.0`, released on 2026-06-16).\n- https://github.com/diffplug/spotless/releases/tag/gradle/8.7.0\n  - https://github.com/diffplug/spotless/pull/2962\n  - https://github.com/diffplug/spotless/issues/2940\n  - https://github.com/diffplug/spotless/pull/2973\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #716 from dongjoon-hyun/SPARK-57629.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "8d539ccdf7990c58bce6207d6512649b74ea55b5",
      "tree": "8d8f83f0a6241033fae365a255afecdd01ab096b",
      "parents": [
        "ed1f0148f4f6aedc9386c7487f18e0febe8a9009"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jun 22 21:50:11 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jun 22 21:50:11 2026 -0700"
      },
      "message": "[SPARK-57628] Upgrade `Gradle` to 9.6.0\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade Gradle to `9.6.0`.\n\n### Why are the changes needed?\n\nTo use the latest version of Gradle (`9.6.0`, released on 2026-06-18).\n- https://github.com/gradle/gradle/releases/tag/v9.6.0\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #715 from dongjoon-hyun/SPARK-57628.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "ed1f0148f4f6aedc9386c7487f18e0febe8a9009",
      "tree": "6ab2967ca5be022751b9467c8c61c92759fb72ae",
      "parents": [
        "b03e313617f249ff314fdf95b78080db409629e7"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jun 22 08:04:24 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon Jun 22 08:04:24 2026 -0700"
      },
      "message": "[SPARK-57612] Upgrade `Apache DataFusion Comet` to 0.17.0\n\n### What changes were proposed in this pull request?\n\nThis PR upgrades Apache DataFusion Comet from `0.16.0` to `0.17.0` in the `pi-with-comet` example.\n\n### Why are the changes needed?\n\nTo use the latest Comet release (`0.17.0`, released on 2026-06-20).\n- https://github.com/apache/datafusion-comet/releases/tag/0.17.0\n- https://datafusion.apache.org/comet/changelog/0.17.0.html\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #714 from dongjoon-hyun/SPARK-57612.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "b03e313617f249ff314fdf95b78080db409629e7",
      "tree": "e30c40b676525dbae16eb32e34d39b16e09a5a18",
      "parents": [
        "bc15bc0d9f5cdd70d14519f51233c6c3cb65e5ba"
      ],
      "author": {
        "name": "Zhou JIANG",
        "email": "jiangzho@umich.edu",
        "time": "Thu Jun 18 09:54:42 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jun 18 09:54:42 2026 -0700"
      },
      "message": "[SPARK-57497] Fix NPE in AppInitStep restart backoff check when stateTransitionHistory is null\n\n### What changes were proposed in this pull request?\n\n  In AppInitStep.reconcile(), when a previous attempt summary exists, the restart backoff\n  check now handles the case where previousAttemptSummary.getStateTransitionHistory() is\n  null (non-trim mode). Instead of dereferencing it directly (causing an NPE), the code\n  falls back to the main status.getStateTransitionHistory() and selects the state entered\n  immediately before the current initializing state — the stopping state that triggered the\n  restart — using NavigableMap.lowerEntry().\n\n  A null guard for getRestartConfig() was also added to the same block.\n\n### Why are the changes needed?\n\n  In non-trim mode (TRIM_ATTEMPT_STATE_TRANSITION_HISTORY\u003dfalse),\n  ApplicationStatus.terminateOrRestart() sets previousAttemptSummary to the old\n  currentAttemptSummary, which always carries stateTransitionHistory \u003d null. The previous\n  code called .getStateTransitionHistory().get(...) unconditionally, throwing a\n  NullPointerException on every restart attempt in non-trim mode. This caused the operator\n  to fail to re-initialize the driver after a restart.\n\n### Does this PR introduce any user-facing change?\n\n  No functional behavior change for users running in trim mode (the default). In non-trim\n  mode, restart backoff is now correctly enforced instead of crashing with an NPE.\n\n### How was this patch tested?\n\n  Two unit tests were added to AppInitStepTest:\n\n  - nonTrimModeRestartBackoffElapsedProceedsToDriverCreation: verifies that when the backoff has elapsed, the app reaches DriverRequested state without throwing an NPE (non-trim mode, previousAttemptSummary present with null stateTransitionHistory).\n  - nonTrimModeRestartBackoffActiveRequeuesWithDelay: verifies that when the backoff has not yet elapsed, the reconcile returns a delayed requeue without throwing an NPE, and the state remains ScheduledToRestart.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\n  Generated-by: Claude Sonnet 4.6\n\nCloses #711 from jiangzho/corner_npe.\n\nAuthored-by: Zhou JIANG \u003cjiangzho@umich.edu\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "bc15bc0d9f5cdd70d14519f51233c6c3cb65e5ba",
      "tree": "f3bd0ece457b61c477ffa316608e9b88dc25b028",
      "parents": [
        "4949266ad2ec537a973df94f94f09ddfd511f65b"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jun 17 10:52:19 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed Jun 17 10:52:19 2026 -0700"
      },
      "message": "[SPARK-57508] Upgrade `java-operator-sdk` to 5.3.5\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `java-operator-sdk` to 5.3.5.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes and improvements from `java-operator-sdk` 5.3.5:\n- https://github.com/operator-framework/java-operator-sdk/releases/tag/v5.3.5\n  - https://github.com/operator-framework/java-operator-sdk/pull/3383\n  - https://github.com/operator-framework/java-operator-sdk/pull/3414\n  - https://github.com/operator-framework/java-operator-sdk/pull/3365\n  - https://github.com/operator-framework/java-operator-sdk/pull/3382\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #712 from dongjoon-hyun/SPARK-57508.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "4949266ad2ec537a973df94f94f09ddfd511f65b",
      "tree": "a3b2981edd947b5934654e9367b757cf4264c354",
      "parents": [
        "4a00f012a5301b31eb82f891eac078ddba04aa2d"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 16 15:08:30 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 16 15:08:30 2026 -0700"
      },
      "message": "[SPARK-57495] Default `driver/master/worker` container `allowPrivilegeEscalation` to false if unset\n\n### What changes were proposed in this pull request?\n\nAdd a `SecurityContextDecorator` that sets each container\u0027s `securityContext.allowPrivilegeEscalation` to `false` when it is unset, and apply it to the operator-created Pods:\n\n- driver Pod (`SparkApplication`) in `SparkAppResourceSpecFactory`\n- master/worker `StatefulSet`s (`SparkCluster`) in `SparkClusterResourceSpecFactory`\n\nIt only fills in the value when null, so a value the user set explicitly is preserved. Executor Pods (created by the Spark driver, not the operator) are out of scope.\n\n### Why are the changes needed?\n\nAn unset `allowPrivilegeEscalation` is treated as `true` by Kubernetes, which violates the Pod Security Standards \"restricted\" profile. Defaulting it to `false` hardens operator-created Pods by default.\n\nIn addition, this is consistent with Apache Spark 4.3.0 behavior.\n- https://github.com/apache/spark/pull/56549\n\n### Does this PR introduce _any_ user-facing change?\n\nYes. Driver/master/worker containers now get `securityContext.allowPrivilegeEscalation: false` when the user did not set it. An explicitly set value (e.g. `true`) is kept.\n\n### How was this patch tested?\n\nAdded `SecurityContextDecoratorTest` (unset, partially-set, explicit-value-preserved, init containers, `StatefulSet`, non-Pod no-op). `gradle build` passes.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Opus 4.8)\n\nCloses #710 from dongjoon-hyun/SPARK-57495.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "4a00f012a5301b31eb82f891eac078ddba04aa2d",
      "tree": "1b63dc9466990933c0e8218ffce5e136d7d7dcf0",
      "parents": [
        "c5bc180122f7852353f9174993fca7eaf75b137e"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 16 15:03:36 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 16 15:03:36 2026 -0700"
      },
      "message": "[SPARK-57494] Add `Pod Security Admission` section to `SECURITY.md`\n\n### What changes were proposed in this pull request?\n\nAdd a **Pod Security Admission** section to `SECURITY.md` documenting that the security posture of user-supplied pod templates is enforced by the Kubernetes API server, not the operator:\n\n- Administrators set the [Pod Security Admission](https://kubernetes.io/docs/concepts/security/pod-security-admission/) level on workload namespaces; it applies to every pod the operator launches.\n- The pod template is the only place to set a pod\u0027s security context, so meeting a stricter level requires supplying one. This covers `SparkApplication` (`.spec.driverSpec/executorSpec.podTemplateSpec`) and `SparkCluster` (`.spec.masterSpec/workerSpec.statefulSetSpec.template`).\n\n### Why are the changes needed?\n\nThe operator passes pod templates through unchanged, which is what lets users satisfy stricter Pod Security Admission levels. Documenting this clarifies that pod security is a Kubernetes responsibility, not a missing operator control.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo. Documentation only.\n\n### How was this patch tested?\n\nDocumentation-only change; no code is affected.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Claude Opus 4.8)\n\nCloses #709 from dongjoon-hyun/SPARK-57494.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "c5bc180122f7852353f9174993fca7eaf75b137e",
      "tree": "b660ac081bc71cffcf2a6bb123b602789b7f89cd",
      "parents": [
        "e875263375e5cd5bac7e160d8d7b913b9b9c28e6"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 16 09:09:55 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 16 09:09:55 2026 -0700"
      },
      "message": "[SPARK-57492] Use K8s Garbage Collection to delete executors only when restart is disabled\n\n### What changes were proposed in this pull request?\n\nThis PR applies the `spark.kubernetes.executor.deleteOnTermination\u003dfalse` override from\nSPARK-55352 only when the application does not restart, i.e. `RestartPolicy` is `Never` or\nunset, in addition to the existing static-allocation condition.\n\n### Why are the changes needed?\n\nWhen `RestartPolicy\u003dAlways` (like `spark-thrift-server.yaml`), the driver pod can be recreated while old executor pods linger, so they accumulate instead of being garbage collected. Such applications must keep the default behavior which delete executors on termination explicitly.\n\nhttps://github.com/apache/spark-kubernetes-operator/blob/e875263375e5cd5bac7e160d8d7b913b9b9c28e6/examples/spark-thrift-server.yaml#L34\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs with the updated `useGarbageCollectionToDeleteExecutors` unit test.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Claude Opus 4.8)\n\nCloses #708 from dongjoon-hyun/SPARK-57492.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "e875263375e5cd5bac7e160d8d7b913b9b9c28e6",
      "tree": "55914bc67f4f3422f4744597d596fa8c20653033",
      "parents": [
        "ee1b768d55050376b3e6347a6b2fbd9ab0eebf8d"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sun Jun 14 20:33:14 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sun Jun 14 20:33:14 2026 -0700"
      },
      "message": "[SPARK-57450] Add JWSFilter-enabled Spark Cluster example\n\n### What changes were proposed in this pull request?\n\nThis PR adds `examples/cluster-with-jws-filter.yaml`, a copy of `examples/cluster.yaml`\nthat enables `org.apache.spark.ui.JWSFilter` via the following `sparkConf` entries:\n\n```yaml\nspark.ui.filters: \"org.apache.spark.ui.JWSFilter\"\nspark.master.rest.filters: \"org.apache.spark.ui.JWSFilter\"\nspark.org.apache.spark.ui.JWSFilter.param.secretKey: \"VmlzaXQgaHR0cHM6Ly9zcGFyay5hcGFjaGUub3JnIHRvIGRvd25sb2FkIEFwYWNoZSBTcGFyay4\u003d\"\n```\n\n### Why are the changes needed?\n\nTo provide an example for securing the Spark UI and Master REST endpoint with `JWSFilter`.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nManually test after launching the example Spark cluster and do the port-forward.\n\n**Without Authorization Header**\n```\n$ curl http://localhost:6066/v1/submissions/readyz\n\u003chtml\u003e\n\u003chead\u003e\n\u003cmeta http-equiv\u003d\"Content-Type\" content\u003d\"text/html;charset\u003dISO-8859-1\"/\u003e\n\u003ctitle\u003eError 403 Authorization header is missing.\u003c/title\u003e\n\u003c/head\u003e\n\u003cbody\u003e\u003ch2\u003eHTTP ERROR 403 Authorization header is missing.\u003c/h2\u003e\n\u003ctable\u003e\n\u003ctr\u003e\u003cth\u003eURI:\u003c/th\u003e\u003ctd\u003e/v1/submissions/readyz\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003cth\u003eSTATUS:\u003c/th\u003e\u003ctd\u003e403\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003cth\u003eMESSAGE:\u003c/th\u003e\u003ctd\u003eAuthorization header is missing.\u003c/td\u003e\u003c/tr\u003e\n\u003ctr\u003e\u003cth\u003eSERVLET:\u003c/th\u003e\u003ctd\u003eorg.apache.spark.deploy.rest.StandaloneReadyzRequestServlet-2e828cb1\u003c/td\u003e\u003c/tr\u003e\n\u003c/table\u003e\n\n\u003c/body\u003e\n\u003c/html\u003e\n```\n\n**With Authorization Header**\n```\n$ curl -H \"Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.e30.4EKWlOkobpaAPR0J4BE0cPQ-ZD1tRQKLZp1vtE7upPw\" http://localhost:6066/v1/submissions/readyz\n{\n  \"action\" : \"ReadyzResponse\",\n  \"message\" : \"\",\n  \"serverSparkVersion\" : \"4.1.2\",\n  \"success\" : true\n}%\n```\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Opus 4.8)\n\nCloses #706 from dongjoon-hyun/SPARK-57450.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "ee1b768d55050376b3e6347a6b2fbd9ab0eebf8d",
      "tree": "ca21cfd3e7eb2a818e8d89805452a55be3c8a550",
      "parents": [
        "a5902468f3cff4a708d83c59e7bc060b0643d6f2"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sat Jun 13 21:56:03 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sat Jun 13 21:56:03 2026 -0700"
      },
      "message": "[SPARK-57433] Upgrade `spotbugs-tool` to 4.10.2\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `spotbugs-tool` to `4.10.2`.\n\n### Why are the changes needed?\n\nTo bring the latest improvements and bug fixes:\n\n- https://github.com/spotbugs/spotbugs/releases/tag/4.10.2\n- https://github.com/spotbugs/spotbugs/releases/tag/4.10.1\n- https://github.com/spotbugs/spotbugs/releases/tag/4.10.0\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CI.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Claude Opus 4.8)\n\nThis patch had conflicts when merged, resolved by\nCommitter: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n\nCloses #705 from dongjoon-hyun/SPARK-57433.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "a5902468f3cff4a708d83c59e7bc060b0643d6f2",
      "tree": "8b5fc4442c5021bacdf905d5f8e162fa42058da5",
      "parents": [
        "94c9ea02e001bd96dfb665b9f8133c16441069d9"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sat Jun 13 21:29:12 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Sat Jun 13 21:29:12 2026 -0700"
      },
      "message": "[SPARK-57432] Upgrade `spotbugs-gradle-plugin` to 6.5.6\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `spotbugs-gradle-plugin` to 6.5.6.\n\n### Why are the changes needed?\n\nTo keep the build tooling up to date by using the latest version of the plugin.\n- https://github.com/spotbugs/spotbugs-gradle-plugin/releases/tag/6.5.6\n\n### Does this PR introduce _any_ user-facing change?\n\nNo. This is a build/test-only dependency change.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Opus 4.8)\n\nCloses #704 from dongjoon-hyun/SPARK-57432.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "94c9ea02e001bd96dfb665b9f8133c16441069d9",
      "tree": "98b68fc1f6cec67ab54f69747dc9860b132730d3",
      "parents": [
        "4a26f3b26743440fae33223b87f6104a8ed11caa"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jun 11 09:54:56 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jun 11 09:54:56 2026 -0700"
      },
      "message": "[SPARK-57391] Use Apache Spark 4.0.3 in examples\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade Apache Spark to `4.0.3` in examples.\n\n### Why are the changes needed?\n\nTo use the latest Apache Spark 4.0.x.\n\n- https://spark.apache.org/news/spark-4-0-3-released.html\n- https://github.com/apache/spark/releases/tag/v4.0.3\n- [Docker Hub: apache/spark:4.0.3](https://hub.docker.com/r/apache/spark/tags?name\u003d4.0.3)\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nManual review.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Fable 5)\n\nCloses #703 from dongjoon-hyun/SPARK-57391.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "4a26f3b26743440fae33223b87f6104a8ed11caa",
      "tree": "9a277477b1cd5b2f0e364d58696a40536716461c",
      "parents": [
        "13a3cd243bc5c394a46331aa6b78e52ae31f6087"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 09 11:48:45 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue Jun 09 11:48:45 2026 -0700"
      },
      "message": "[SPARK-57350] Enable `spark.kubernetes.executor.useDriverPodIP` by default\n\n### What changes were proposed in this pull request?\n\nThis PR sets `spark.kubernetes.executor.useDriverPodIP` to `true` by default via\n`setIfMissing` in `SparkAppSubmissionWorker#buildDriverConf`, next to the other\noperator-injected defaults. Since it uses `setIfMissing`, an explicit value in the\napplication spec is always respected.\n\n### Why are the changes needed?\n\n`spark.kubernetes.executor.useDriverPodIP` (added in Spark 4.1.0 via SPARK-53944) makes\nexecutor pods connect to the driver using the driver pod IP instead of the driver\n`Service` hostname, bypassing Kubernetes DNS. The operator fully manages the driver pod\nlifecycle, and in cluster mode the driver is not restarted (the application terminates if\nit dies), so the pod IP is stable for the application\u0027s lifetime. Enabling it by default\navoids DNS propagation delays/failures on executor startup and reduces CoreDNS load. It is\na no-op for Spark versions older than 4.1.0.\n- https://github.com/apache/spark/pull/52650\n\n### Does this PR introduce _any_ user-facing change?\n\nYes. When `spark.kubernetes.executor.useDriverPodIP` is unset, the operator now defaults\nit to `true` (previously Spark\u0027s default of `false`). Applications that set it explicitly\nare unaffected.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Claude Opus 4.8)\n\nCloses #702 from dongjoon-hyun/SPARK-57350.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "13a3cd243bc5c394a46331aa6b78e52ae31f6087",
      "tree": "345bf89c9e018f2ef0cd9a9783f2b740e0789484",
      "parents": [
        "ef978e0279cb90ba1ff46b8427e16dd8cbf31fd0"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri Jun 05 08:57:37 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri Jun 05 08:57:37 2026 -0700"
      },
      "message": "[SPARK-57280] Add `SECURITY.md` for scan-agent discoverability\n\n### What changes were proposed in this pull request?\n\nThis PR aims to add `SECURITY.md` for scan-agent discoverability.\n\n### Why are the changes needed?\n\nLike Apache Spark main repository, we had better provide this.\n- https://github.com/apache/spark/blob/master/SECURITY.md\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nManual review.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nNo.\n\nCloses #701 from dongjoon-hyun/SPARK-57280.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "ef978e0279cb90ba1ff46b8427e16dd8cbf31fd0",
      "tree": "9f3e4509c124f13cb27e408dbcd11ee67778b6b7",
      "parents": [
        "87fdd88259d8c25e1c50ab961436bf7297b2085b"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jun 04 16:29:48 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu Jun 04 16:29:48 2026 -0700"
      },
      "message": "[SPARK-57272] Upgrade `Netty` to 4.2.15.Final\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `Netty` to 4.2.15.Final.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes:\n\n- https://netty.io/news/2026/06/01/4-2-15-Final.html\n  - [CVE-2026-48059](https://github.com/netty/netty/security/advisories/GHSA-h2qv-fj59-j46j): memory exhaustion in io.netty:netty-codec-haproxy (high).\n  - [CVE-2026-47691](https://github.com/netty/netty/security/advisories/GHSA-5pvg-856g-cp85): DNS cache poisoning in io.netty:netty-resolver-dns (high).\n  - [CVE-2026-XXXXX](https://github.com/netty/netty/security/advisories/GHSA-563q-j3cm-6jxm): DDoS in io.netty:netty-codec-http2.\n  - [CVE-2026-XXXXX](https://github.com/netty/netty/security/advisories/GHSA-5w86-c3rq-vjj7): memory exhaustion in io.netty:netty-codec-redis (high).\n  - [CVE-2026-44250](https://github.com/netty/netty/security/advisories/GHSA-3244-j874-rhc2): memory exhaustion in io.netty:netty-codec-redis (high).\n  - [CVE-2026-44890](https://github.com/netty/netty/security/advisories/GHSA-6ghj-frrj-jjj3): memory exhaustion in io.netty:netty-codec-redis (high).\n  - [CVE-2026-XXXXX](https://github.com/netty/netty/security/advisories/GHSA-cq4q-cv5g-r8q5): information disclosure and denial of service in io.netty:netty-codec-classes-quic.\n  - [CVE-2026-44249](https://github.com/netty/netty/security/advisories/GHSA-3qp7-7mw8-wx86): IPv6 subnet filter bypass in io.netty:netty-handler (high).\n  - [CVE-2026-XXXXX](https://github.com/netty/netty/security/advisories/GHSA-hvcg-qmg6-jm4c): request smuggling in io.netty:netty-codec-http.\n  - [CVE-2026-44892](https://github.com/netty/netty/security/advisories/GHSA-c2rx-5r8w-8xr2): memory exhaustion in io.netty:netty-codec-http3 (high).\n  - [CVE-2026-44893](https://github.com/netty/netty/security/advisories/GHSA-cc37-9q2j-3hfv): memory leak in io.netty:netty-codec-haproxy (high).\n  - [CVE-2026-44894](https://github.com/netty/netty/security/advisories/GHSA-cmm3-54f8-px4j): traffic amplification in io.netty:netty-codec-classes-quic (high).\n  - [CVE-2026-XXXXX](https://github.com/netty/netty/security/advisories/GHSA-c653-97m9-rcg9): TLS hostname verification accidentally disabled in io.netty:netty-handler (high).\n  - [CVE-2026-45673](https://github.com/netty/netty/security/advisories/GHSA-xmv7-r254-6q78): DNS cache poisoning in io.netty:netty-resolver-dns.\n  - [CVE-2026-45416](https://github.com/netty/netty/security/advisories/GHSA-x4gw-5cx5-pgmh): excessive memory usage from SNIHandler in io.netty:netty-handler (high).\n  - [CVE-2026-45536](https://github.com/netty/netty/security/advisories/GHSA-w573-9ffj-6ff9): file descriptor leak in io.netty:netty-transport-native-epoll and io.netty:netty-transport-native-kqueue.\n  - [CVE-2026-45674](https://github.com/netty/netty/security/advisories/GHSA-676x-f7gg-47vc): DNS cache poisoning in io.netty:netty-resolver-dns (high).\n  - [CVE-2026-46340](https://github.com/netty/netty/security/advisories/GHSA-5xrh-qmmq-w6ch): memory exhaustion in io.netty:netty-transport-sctp (high).\n  - [CVE-2026-47244](https://github.com/netty/netty/security/advisories/GHSA-5x3r-wrvg-rp6q): denial of service in io.netty:netty-codec-http2.\n  - [CVE-2026-48006](https://github.com/netty/netty/security/advisories/GHSA-6jv9-x5w9-2ccm): memory exhaustion in io.netty:netty-codec-redis (high).\n  - [CVE-2026-48748](https://github.com/netty/netty/security/advisories/GHSA-4grm-h2qv-h6w6): memory exhaustion in io.netty:netty-codec-http3 (high).\n  - [CVE-2026-48043](https://github.com/netty/netty/security/advisories/GHSA-c2gf-v879-257j): memory exhaustion in io.netty:netty-codec-http2.\n  - Fix race in io.netty.channel.uring.IoUringIoHandler.wakeup [#16836](https://github.com/netty/netty/pull/16836)\n  - HTTP/2: Parse request-target path like Vert.x [#16810](https://github.com/netty/netty/pull/16810)\n  - ChannelInitializer: correct misleading comment on exceptionCaught route [#16853](https://github.com/netty/netty/pull/16853)\n  - FlowControlHandler: Suppress duplicate channelReadComplete after draining queue [#16837](https://github.com/netty/netty/pull/16837)\n  - Pass maxAllocation to Brotli and Zstd decoders [#16844](https://github.com/netty/netty/pull/16844)\n  - Add maxWindowLog parameter to ZstdDecoder to bound memory allocation [#16850](https://github.com/netty/netty/pull/16850)\n  - MQTT: Reject malformed no-payload packets with non-zero Remaining Length [#16890](https://github.com/netty/netty/pull/16890)\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8\n\nCloses #700 from dongjoon-hyun/SPARK-57272.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "87fdd88259d8c25e1c50ab961436bf7297b2085b",
      "tree": "cf66f86b3cf0db789b85d0e765a60220c4d4da3f",
      "parents": [
        "56ff3e01f7159cbddac432e70a3dc6aed7ab6864"
      ],
      "author": {
        "name": "Liang-Chi Hsieh",
        "email": "liangchi.hsieh@databricks.com",
        "time": "Fri May 29 18:42:52 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 29 18:42:52 2026 -0700"
      },
      "message": "[SPARK-57157] Harden fork CI status workflows against run/check-run race and pagination\n\n### What changes were proposed in this pull request?\n\nPort the hardening fixes from Apache Spark (SPARK-57154, SPARK-57155) to this\nrepository\u0027s `notify_test_workflow.yml` and `update_build_status.yml`, which were\nintroduced here as a copy of Spark\u0027s fork-based CI status mechanism (SPARK-57153).\n\n`notify_test_workflow.yml`:\n\n1. When listing the fork\u0027s workflow runs, instead of blindly taking the most\n   recent run (`workflow_runs[0]`) and throwing if its `head_sha` does not match\n   the PR head SHA, retry (up to 3 times, 3s apart) looking for the run whose\n   `head_sha` matches the PR head SHA. The listing endpoint orders by most recent,\n   so the run for the just-pushed SHA may not be registered yet and a stale run\n   from a previous push could be returned.\n\n2. When resolving the `Run / License Check` check-run id (used only to render a\n   Check-run view link instead of the Actions view, see SPARK-37879), a missing\n   check-run no longer throws. The check-run materializes later than the workflow\n   run, especially when the matrix is queued, so this is now best-effort: if it\n   cannot be found, the `Build` check is still created pointing at the Actions\n   run URL.\n\n`update_build_status.yml`:\n\n3. List a commit\u0027s check-runs with `github.paginate(..., per_page: 100)` instead\n   of a single un-paginated request, matching `notify_test_workflow.yml`. The\n   default page size is 30, so the target `Build` check could fall off the first\n   page on a SHA that accumulates more check-runs than that.\n\n4. Wrap `JSON.parse(cr.output.text)` in try/catch and `continue` on failure, so a\n   `Build` check with empty or malformed output text does not abort the whole\n   scheduled run and block updates for every PR queued behind it.\n\n### Why are the changes needed?\n\nThe race conditions previously left a PR with no `Build` check at all, and the\nscheduled updater only syncs existing checks, so the PR had no status reported\nuntil the next push. The pagination and parsing issues silently block status\nupdates, leaving PRs stuck in `queued`.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo. CI infrastructure only.\n\n### How was this patch tested?\n\nStatic verification: the embedded `actions/github-script` bodies pass\n`node --check`, and the workflow YAML parses.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Claude Opus 4.8)\n\nCloses #699 from viirya/SPARK-57157-harden-ci-status.\n\nAuthored-by: Liang-Chi Hsieh \u003cliangchi.hsieh@databricks.com\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "56ff3e01f7159cbddac432e70a3dc6aed7ab6864",
      "tree": "f0b8fd61d623b245775a9a7b0e5ad8b3d6143413",
      "parents": [
        "40eea1318a0459730ffcbe5bd1aa9ccb4c0c45a5"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 29 12:38:10 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 29 12:38:10 2026 -0700"
      },
      "message": "[SPARK-57153] Run CI in forked repositories and report build status to PRs\n\n### What changes were proposed in this pull request?\n\nMake `build_and_test.yml` a reusable (`workflow_call`) workflow and add Apache\nSpark\u0027s fork-based CI status mechanism:\n\n- `build_main.yml` — runs on `push` to all branches and calls `build_and_test.yml`;\n  it also carries the `cancel-in-progress` concurrency group (moved out of\n  `build_and_test.yml`, since the triggers now live here).\n- `notify_test_workflow.yml` — on `pull_request_target`, creates a `Build` check\n  on the PR that points to the fork\u0027s run.\n- `update_build_status.yml` — every 15 minutes, syncs the `Build` check from the\n  fork\u0027s run.\n- Add `images/workflow-enable-button.png` used by the notify message, and repoint\n  the README badge to `build_main.yml`.\n\nFlow:\n\n```mermaid\nflowchart TD\n    subgraph fork[\"Forked repo (contributor)\"]\n        P([push to any branch]) --\u003e BM[\"build_main.yml — job: Run\"]\n        BM --\u003e|workflow_call| BAT[\"build_and_test.yml (reusable):\u003cbr/\u003eLicense Check, build/test matrix, K8s and Helm tests\"]\n        BAT --\u003e CR[\"check runs:\u003cbr/\u003eRun / License Check, ...\"]\n    end\n\n    subgraph up[\"Upstream repo (apache/spark-kubernetes-operator)\"]\n        PRT([pull_request_target]) --\u003e N[\"notify_test_workflow.yml\"]\n        SCH([schedule: every 15 min]) --\u003e U[\"update_build_status.yml\"]\n        N --\u003e|create| B([\"Build check on PR\"])\n        U --\u003e|\"PATCH status / conclusion\"| B\n    end\n\n    BM -.-\u003e|\"① find run (id: build_main.yml)\"| N\n    CR -.-\u003e|\"② link check-run view\"| N\n    BM -.-\u003e|\"③ poll run status\"| U\n```\n\n### Why are the changes needed?\n\nLike the Apache Spark main repository, run the heavy test matrix in contributors\u0027\nforks and mirror only a single `Build` status check upstream, instead of running\nthe matrix in the upstream repo on every PR. This matches the Apache Spark main\nrepository and is especially valuable here, where the matrix spans multi-arch\nbuild/test across JDK 21 / 25 / 26 plus dozens of Kubernetes integration and Helm\njobs.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo. CI and documentation only.\n\n### How was this patch tested?\n\nStatic verification of the cross-workflow wiring: `build_main.yml`\u0027s `Run` job and\n`build_and_test.yml`\u0027s `License Check` job produce the `Run / License Check`\ncheck-run name that `notify_test_workflow.yml` probes for; the shared `Build` check\nname; and the `run_id` passed via the check output text from `notify` to `update`.\nAll four workflow YAMLs parse cleanly. Full behavior takes effect only after merge\nto `main`, since the `pull_request_target` / `schedule` workflows and the\n`main`-pinned image URL resolve from the default branch.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Claude Opus 4.8)\n\nCloses #697 from dongjoon-hyun/SPARK-57153.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "40eea1318a0459730ffcbe5bd1aa9ccb4c0c45a5",
      "tree": "ce282b3ec37778fe11cde36488b5fc6c97b0efa1",
      "parents": [
        "2f3b2339b711abdaa3c03872cbe6da973d186de6"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 29 10:35:39 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 29 10:35:39 2026 -0700"
      },
      "message": "[SPARK-57147] Add `AGENTS.md` and symlink `CLAUDE.md` to it\n\n### What changes were proposed in this pull request?\n\nThis PR adds `AGENTS.md`, a contributor/agent guide for the repository, and `CLAUDE.md` as a\nsymlink to it.\n\n### Why are the changes needed?\n\nTo give contributors and AI agents one accurate reference for the build, test, style, and\ncontribution conventions. The symlink keeps `CLAUDE.md` and `AGENTS.md` in sync without duplication.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nDocumentation-only change. Verified that `CLAUDE.md` resolves to `AGENTS.md`.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nYes. Generated-by: Claude Code (Claude Opus 4.8)\n\nCloses #696 from dongjoon-hyun/SPARK-57147.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "2f3b2339b711abdaa3c03872cbe6da973d186de6",
      "tree": "0941e8a4622331f7b76822478dd35ff9981232c1",
      "parents": [
        "4ced1de381ac921ef851f48ede398ab0e9acfb9b"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 27 17:22:21 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 27 17:22:21 2026 -0700"
      },
      "message": "[SPARK-57114] Upgrade `spotless-plugin-gradle` to 8.6.0\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `spotless-plugin-gradle` to 8.6.0.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes.\n- https://github.com/diffplug/spotless/releases/tag/gradle%2F8.6.0\n  - https://github.com/diffplug/spotless/issues/2948\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code Opus\n\nCloses #694 from dongjoon-hyun/SPARK-57114.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "4ced1de381ac921ef851f48ede398ab0e9acfb9b",
      "tree": "549530e13d1f162662a807fd4517a3b2a4eb38dc",
      "parents": [
        "9de9156bda9e42e307af91fc9be98d4f245d95b1"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 27 11:19:55 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 27 11:19:55 2026 -0700"
      },
      "message": "[SPARK-57107] Pin `apache/skywalking-eyes` to `v0.8.0` commit SHA\n\n### What changes were proposed in this pull request?\n\nThis PR aims to pin `apache/skywalking-eyes` to the `v0.8.0` commit SHA in `.github/workflows/build_and_test.yml`.\n\n```yaml\n- uses: apache/skywalking-eyese9f91c35e4d4ae4420f722aa6598c4a13cc69093\n```\n\n### Why are the changes needed?\n\nTo improve supply-chain security by pinning the third-party GitHub Action to an immutable commit SHA instead of the mutable `main` branch reference.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7\n\nCloses #693 from dongjoon-hyun/SPARK-57107.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "9de9156bda9e42e307af91fc9be98d4f245d95b1",
      "tree": "90e9384b1a8e6beaff5293e7bca3d4d78fafaaa8",
      "parents": [
        "0b6869e007c97c4c482c4cb7ef0e44827615c06f"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue May 26 11:01:38 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue May 26 11:01:38 2026 -0700"
      },
      "message": "[SPARK-57078] Upgrade `Dropwizard` metrics to 4.2.39\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `Dropwizard` metrics to 4.2.39.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes including the official Java 25 support.\n\n- https://github.com/dropwizard/metrics/milestone/109?closed\u003d1\n  - https://github.com/dropwizard/metrics/pull/4964\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7\n\nCloses #690 from dongjoon-hyun/SPARK-57078.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "0b6869e007c97c4c482c4cb7ef0e44827615c06f",
      "tree": "725cb1e0ba63c4cb4d64a02abab654ff83afee5f",
      "parents": [
        "3a45bd3f47b50cad49ec3c9946c16fedd8a87da2"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue May 26 09:31:30 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Tue May 26 09:31:30 2026 -0700"
      },
      "message": "[SPARK-57079] Add `*.pyc` and `*.pyo` to `.gitignore`\n\n### What changes were proposed in this pull request?\n\nThis PR aims to add `*.pyc` and `*.pyo` to `.gitignore` to exclude Python bytecode files from being tracked by Git.\n\n### Why are the changes needed?\n\nLike Apache Spark main repository, to prevent accidental commits of Python compiled bytecode files (`*.pyc`, `*.pyo`) that may be generated when running Python scripts in this repository (e.g., `dev/spark_jira_utils.py`).\n\nhttps://github.com/apache/spark/blob/30bdf0b39fa6b28ea149ddd43ecd962b2b3c0cd2/.gitignore#L6-L7\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nManual review of `.gitignore`.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7\n\nCloses #691 from dongjoon-hyun/SPARK-57079.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "3a45bd3f47b50cad49ec3c9946c16fedd8a87da2",
      "tree": "76cb5d29eb6d84c7acb64aad41d4ffd583036428",
      "parents": [
        "40718f0b25ff11533673b5f3becf2def88417566"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 22 15:42:36 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 22 15:42:36 2026 -0700"
      },
      "message": "[SPARK-57017] Upgrade `Netty` to 4.2.14.Final\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `Netty` to 4.2.14.Final.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes and improvements.\n\n- https://netty.io/news/2026/05/20/4-2-14-Final.html\n  - https://github.com/netty/netty/pull/16747\n  - https://github.com/netty/netty/pull/16759\n  - https://github.com/netty/netty/pull/16767\n  - https://github.com/netty/netty/pull/16781\n  - https://github.com/netty/netty/pull/16788\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7\n\nCloses #689 from dongjoon-hyun/SPARK-57017.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "40718f0b25ff11533673b5f3becf2def88417566",
      "tree": "d62fa52c6eef434dc63d88f36e4fa2802a92278b",
      "parents": [
        "a57ff62b1b9e88926035af0785f43637c11d4e86"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 22 14:32:56 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 22 14:32:56 2026 -0700"
      },
      "message": "[SPARK-57016] Upgrade `spotbugs-gradle-plugin` to 6.5.5\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `spotbugs-gradle-plugin` from 6.5.4 to 6.5.5.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes:\n- https://github.com/spotbugs/spotbugs-gradle-plugin/releases/tag/6.5.5 (2026-05-21)\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7\n\nCloses #688 from dongjoon-hyun/SPARK-57016.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "a57ff62b1b9e88926035af0785f43637c11d4e86",
      "tree": "d03d4783ca6925ad4492d2c010be9990af7dacdf",
      "parents": [
        "5860bb05dcffefb0a7e429686b3af0130c6120f8"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 22 14:32:28 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 22 14:32:28 2026 -0700"
      },
      "message": "[SPARK-57015] Upgrade `slf4j` to 2.0.18\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `slf4j` from 2.0.17 to 2.0.18.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes from SLF4J 2.0.18.\n- https://www.slf4j.org/news.html#2.0.18 (2026-05-12)\n- https://github.com/qos-ch/slf4j/milestone/6\n  - https://github.com/qos-ch/slf4j/issues/471\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7\n\nCloses #687 from dongjoon-hyun/SPARK-57015.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "5860bb05dcffefb0a7e429686b3af0130c6120f8",
      "tree": "4c7fa449aa5671cae82b3f8d8e240b99c841276e",
      "parents": [
        "f0510e225323f652d9846d6b927ab04e4f94f63e"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 22 13:28:15 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 22 13:28:15 2026 -0700"
      },
      "message": "[SPARK-57012] Drop Spark 3.5 Support\n\n### What changes were proposed in this pull request?\n\nThis PR removes Apache Spark 3.5 related contents and code.\n\n### Why are the changes needed?\n\nAccording to the Apache Spark EOL policy, we drop Spark 3.5 support. Spark 3.5.x reached **the planned EOL on 2026-04-12**. The only exception is the extension to the main repository which is irrelevant to `Apache Spark K8s Operator` as mentioned in the website.\n\n- https://spark.apache.org/versioning-policy.html\n\n\u003cimg width\u003d\"981\" height\u003d\"348\" alt\u003d\"Screenshot 2026-05-22 at 13 03 12\" src\u003d\"https://github.com/user-attachments/assets/602f66d1-20ab-4b99-a046-86aa57d9f95f\" /\u003e\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7\n\nCloses #686 from dongjoon-hyun/SPARK-57012.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "f0510e225323f652d9846d6b927ab04e4f94f63e",
      "tree": "1564448df5b7775a347ea544d11d212f520916a2",
      "parents": [
        "98028e9a203ca1e12e75a789e2a10342fc32da90"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 22 10:02:42 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Fri May 22 10:02:42 2026 -0700"
      },
      "message": "[SPARK-57006] Use Spark 4.1.2 in docs, examples, tests, and default values\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade Apache Spark from `4.1.1` to `4.1.2`.\n\n### Why are the changes needed?\n\nTo use the latest Apache Spark.\n\n- https://spark.apache.org/news/spark-4-1-2-released.html\n- https://github.com/apache/spark/releases/tag/v4.1.2\n- [Docker Hub: apache/spark:4.1.2](https://hub.docker.com/r/apache/spark/tags?name\u003d4.1.2)\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7\n\nCloses #685 from dongjoon-hyun/SPARK-57006.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "98028e9a203ca1e12e75a789e2a10342fc32da90",
      "tree": "1e66452224457b7110c437d9f5f067239635b5ad",
      "parents": [
        "b0f472ba0d7f60698f0362e9e18450c694fea27f"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu May 21 10:51:17 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu May 21 10:51:17 2026 -0700"
      },
      "message": "[SPARK-56987] Upgrade `JUnit` to 6.1.0\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `JUnit` from `6.0.3` to `6.1.0`.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes and improvements.\n\n- https://docs.junit.org/6.1.0/release-notes/ (2026-05-19)\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7\n\nCloses #683 from dongjoon-hyun/SPARK-56987.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "b0f472ba0d7f60698f0362e9e18450c694fea27f",
      "tree": "507ed10a5baa64c9ed513cdfc0a61e06b47e2b57",
      "parents": [
        "7edd4aa14648bb0de9f672eb45a60743c3d1a937"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu May 21 10:50:11 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu May 21 10:50:11 2026 -0700"
      },
      "message": "[SPARK-56986] Upgrade `java-operator-sdk` to 5.3.4\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `java-operator-sdk` to 5.3.4.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes and improvements from `java-operator-sdk` 5.3.4:\n- https://github.com/operator-framework/java-operator-sdk/releases/tag/v5.3.4\n  - Revert breaking constructor change in `InformerConfiguration` to restore compatibility.\n  - Read-cache-after-write aware `list` and `byIndex`.\n  - Use `onList()` for ghost resource cleanup.\n  - Handle empty YAML configuration files containing only comments.\n  - Invoke `ConfigurationServiceOverrider` only once.\n  - Restore JUnit5 compatibility for `LocallyRunOperatorExtension`.\n  - Fix flaky integration tests (`WorkflowAllFeatureIT`, `MultipleReconcilerSameTypeIT`).\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7\n\nCloses #682 from dongjoon-hyun/SPARK-56986.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "7edd4aa14648bb0de9f672eb45a60743c3d1a937",
      "tree": "d826d2f1627d2c5b5c0c801fbe3b60ff1f53c668",
      "parents": [
        "31e9e6f56ad025df9cdb51cdf3fc945a97ea337b"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu May 21 10:19:27 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu May 21 10:19:27 2026 -0700"
      },
      "message": "[SPARK-56988] Add `dev/spark_jira_utils.py`\n\n### What changes were proposed in this pull request?\n\nThis PR aims to add `dev/spark_jira_utils.py` by extracting the JIRA-related helpers out of `dev/create_jira_and_branch.py`, mirroring Apache Spark\u0027s [dev/spark_jira_utils.py](https://github.com/apache/spark/blob/master/dev/spark_jira_utils.py).\n\n`dev/create_jira_and_branch.py` is updated to import from this module and passes `\"Kubernetes\"` as the component.\n\n### Why are the changes needed?\n\nTo keep the script aligned with Apache Spark\u0027s `dev/spark_jira_utils.py` structure and to allow reuse of the JIRA helpers from future dev scripts in this repository.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nManual test.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7\n\nCloses #684 from dongjoon-hyun/SPARK-56988.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "31e9e6f56ad025df9cdb51cdf3fc945a97ea337b",
      "tree": "8bf37d3b314769313f22bfa5b5025251982e2a51",
      "parents": [
        "c77f70b91a2d446d8a6a3d4196db21f98aad98b6"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu May 21 09:03:38 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu May 21 09:03:38 2026 -0700"
      },
      "message": "[SPARK-56985] Upgrade `Iceberg` example to use Iceberg 1.11.0\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `Iceberg` to 1.11.0 in the `spark-connect-server-iceberg.yaml` example.\n\n### Why are the changes needed?\n\nTo use the latest Iceberg release.\n- https://iceberg.apache.org/releases/#1110-release\n- https://github.com/apache/iceberg/releases/tag/apache-iceberg-1.11.0 (2026-05-19)\n  - https://github.com/apache/iceberg/pull/14400\n  - https://github.com/apache/iceberg/pull/14099\n  - https://github.com/apache/iceberg/pull/14155\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nManual review.\n\n```\n$ kubectl apply -f examples/spark-connect-server-iceberg.yaml\nsparkapplication.spark.apache.org/spark-connect-server-iceberg created\n\n$ kubectl get sparkapp\nNAME                           AGE   CURRENT STATE\nspark-connect-server-iceberg   46s   RunningHealthy\n\n$ kybectl get pod\nNAME                                         READY   STATUS    RESTARTS      AGE\nspark-connect-server-iceberg-0-driver        1/1     Running   0             47s\nspark-connect-server-iceberg-exec-1          1/1     Running   0             11s\nspark-connect-server-iceberg-exec-2          1/1     Running   0             11s\nspark-connect-server-iceberg-exec-3          1/1     Running   0             11s\nspark-kubernetes-operator-86bfb88655-w5fs8   1/1     Running   7 (42h ago)   6d19h\n\n$ kubectl logs -f spark-connect-server-iceberg-0-driver\n...\norg.apache.iceberg#iceberg-spark-runtime-4.0_2.13 added as a dependency\n...\n        found org.apache.iceberg#iceberg-spark-runtime-4.0_2.13;1.11.0 in central\n...\ndownloading https://repo1.maven.org/maven2/org/apache/iceberg/iceberg-spark-runtime-4.0_2.13/1.11.0/iceberg-spark-runtime-4.0_2.13-1.11.0.jar ...\n        [SUCCESSFUL ] org.apache.iceberg#iceberg-spark-runtime-4.0_2.13;1.11.0!iceberg-spark-runtime-4.0_2.13.jar (834ms)\n...\n        org.apache.iceberg#iceberg-spark-runtime-4.0_2.13;1.11.0 from central in [default]\n...\nFiles file:///tmp/.ivy2.5.2/jars/org.apache.iceberg_iceberg-spark-runtime-4.0_2.13-1.11.0.jar from /tmp/.ivy2.5.2/jars/org.apache.iceberg_iceberg-spark-runtime-4.0_2.13-1.11.0.jar to /opt/spark/work-dir/org.apache.iceberg_iceberg-spark-runtime-4.0_2.13-1.11.0.jar\n...\n26/05/21 14:37:11 INFO SparkConnectServer: Starting Spark session.\n26/05/21 14:37:11 INFO SparkContext: Running Spark version 4.0.2\n26/05/21 14:37:11 INFO SparkContext: OS info Linux, 6.6.119-0-virt, aarch64\n26/05/21 14:37:11 INFO SparkContext: Java version 17.0.17\n...\n26/05/21 14:37:17 INFO Utils: Successfully started service \u0027org.apache.spark.sql.connect.service.SparkConnectService$\u0027 on port 15002.\n26/05/21 14:37:17 INFO SparkConnectServer: Spark Connect server started at: 0:0:0:0:0:0:0:0:15002\n```\n\n```\n$ kubectl port-forward spark-connect-server-iceberg-0-driver 15002\n```\n\n```\n$ bin/spark-connect-shell --remote sc://localhost:15002\n...\nWelcome to\n      ____              __\n     / __/__  ___ _____/ /__\n    _\\ \\/ _ \\/ _ `/ __/  \u0027_/\n   /___/ .__/\\_,_/_/ /_/\\_\\   version 4.0.2\n      /_/\n\nType in expressions to have them evaluated.\nSpark connect server version 4.0.2.\nSpark session available as \u0027spark\u0027.\n\nscala\u003e\nsql(\"\"\"CREATE TABLE taxis(vendor_id bigint, trip_id bigint, trip_distance float, fare_amount double, store_and_fwd_flag string) PARTITIONED BY (vendor_id);\"\"\").show()\nUsing Spark\u0027s default log4j profile: org/apache/spark/log4j2-defaults.properties\n++\n||\n++\n++\n\nscala\u003e\nsql(\"\"\"INSERT INTO taxis VALUES (1, 1000371, 1.8, 15.32, \u0027N\u0027), (2, 1000372, 2.5, 22.15, \u0027N\u0027), (2, 1000373, 0.9, 9.01, \u0027N\u0027), (1, 1000374, 8.4, 42.13, \u0027Y\u0027);\"\"\").show()\n++\n||\n++\n++\n\nscala\u003e sql(\"SELECT * FROM taxis\").show(false)\n+---------+-------+-------------+-----------+------------------+\n|vendor_id|trip_id|trip_distance|fare_amount|store_and_fwd_flag|\n+---------+-------+-------------+-----------+------------------+\n|1        |1000374|8.4          |42.13      |Y                 |\n|1        |1000371|1.8          |15.32      |N                 |\n|2        |1000373|0.9          |9.01       |N                 |\n|2        |1000372|2.5          |22.15      |N                 |\n+---------+-------+-------------+-----------+------------------+\n\nscala\u003e sql(\"SELECT * FROM taxis.history\").show(false)\n+-----------------------+-------------------+---------+-------------------+\n|made_current_at        |snapshot_id        |parent_id|is_current_ancestor|\n+-----------------------+-------------------+---------+-------------------+\n|2026-05-21 14:55:46.629|4314466511351517368|NULL     |true               |\n+-----------------------+-------------------+---------+-------------------+\n\nscala\u003e sql(\"SELECT * FROM taxis VERSION AS OF 4314466511351517368\").show(false)\n+---------+-------+-------------+-----------+------------------+\n|vendor_id|trip_id|trip_distance|fare_amount|store_and_fwd_flag|\n+---------+-------+-------------+-----------+------------------+\n|1        |1000374|8.4          |42.13      |Y                 |\n|1        |1000371|1.8          |15.32      |N                 |\n|2        |1000373|0.9          |9.01       |N                 |\n|2        |1000372|2.5          |22.15      |N                 |\n+---------+-------+-------------+-----------+------------------+\n```\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7\n\nCloses #681 from dongjoon-hyun/SPARK-56985.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "c77f70b91a2d446d8a6a3d4196db21f98aad98b6",
      "tree": "1efe444e7d7db4b407088f054e837269605d722a",
      "parents": [
        "9c034a424b4dc5448807466298e3ec96d6c9c232"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon May 18 16:05:45 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon May 18 16:05:45 2026 -0700"
      },
      "message": "[SPARK-56930] Upgrade `spotless-plugin-gradle` to 8.5.1\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade `spotless-plugin-gradle` to 8.5.1.\n\n### Why are the changes needed?\n\nTo bring the latest bug fixes and improvements.\n- https://github.com/diffplug/spotless/releases/tag/gradle%2F8.5.1 (2025-05-15)\n- https://github.com/diffplug/spotless/releases/tag/gradle%2F8.5.0 (2026-05-14)\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Opus 4.7)\n\nCloses #680 from dongjoon-hyun/SPARK-56930.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "9c034a424b4dc5448807466298e3ec96d6c9c232",
      "tree": "ebab047f947d688af1244f35beb3267d5fe30129",
      "parents": [
        "2ebaea7d4a542420cf5f3557e140b3500273851c"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 13 15:34:11 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 13 15:34:11 2026 -0700"
      },
      "message": "[SPARK-56855] Upgrade `kubernetes-client` to 7.6.0 and update CRDs for K8s 1.36\n\n### What changes were proposed in this pull request?\n\nThis PR upgrades `kubernetes-client` to `7.7.0` and updates the Helm chart CRDs to match the newly generated output.\n\n### Why are the changes needed?\n\n`v7.7.0` is the first release to support K8s 1.36 (Haru) officially.\n- https://github.com/fabric8io/kubernetes-client/releases/tag/v7.7.0 (2026-05-12)\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: `Claude Code`\n\nCloses #678 from dongjoon-hyun/SPARK-56855.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "2ebaea7d4a542420cf5f3557e140b3500273851c",
      "tree": "9fbe7433658e449f167ee9dc5bf31cb9611eb5b6",
      "parents": [
        "025d22b492491b424cbd2c66b5d210639eb917c3"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 13 13:34:26 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 13 13:34:26 2026 -0700"
      },
      "message": "[SPARK-56852] Upgrade `Gradle` to 9.5.1\n\n### What changes were proposed in this pull request?\n\nThis PR upgrades `Gradle` to 9.5.1.\n\n### Why are the changes needed?\n\nTo bring the latest Gradle bug fixes:\n- https://docs.gradle.org/9.5.1/release-notes.html (2026-05-12)\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs and check the log.\n\n```\nWelcome to Gradle 9.5.1!\nHere are the highlights of this release:\n - Task provenance in reports and failure messages\n - Type-safe accessors for precompiled Kotlin Settings plugins\nFor more details see https://docs.gradle.org/9.5.1/release-notes.html\n```\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Claude Opus 4.7)\n\nCloses #677 from dongjoon-hyun/SPARK-56852.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "025d22b492491b424cbd2c66b5d210639eb917c3",
      "tree": "e64d7a330f3c42a8dfec60733832f448254465bf",
      "parents": [
        "c35ec123263ab29803c8a4b535c31c9dac97dc34"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 13 13:33:16 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 13 13:33:16 2026 -0700"
      },
      "message": "[SPARK-56851] Remove `v1alpha1` version of SparkApp and SparkCluster CRDs\n\n### What changes were proposed in this pull request?\n\nThis PR removes the `v1alpha1` schema from `SparkApplication` and `SparkCluster` CRDs.\n\n### Why are the changes needed?\n\nApache Spark K8s Operator moved away from `v1alpha1` over one year. Apache Spark K8s Operator v1.0.0 will enforce `v1` more by dropping `v1alpha1` support officially.\n- `v1beta1` is kept for migration.\n  - #219 (2025-05-21)\n- `v1` has been the storage version since SPARK-52606 (`0.5.0`)\n  - #261 (2026-06-28)\n\nTo slim down the CRDs ahead of the `1.0.0` release. Per Kubernetes API conventions, `Alpha` versions are for short-lived testing clusters with no long-term support.\n\n### Does this PR introduce _any_ user-facing change?\n\nYes. `apiVersion: spark.apache.org/v1alpha1` is no longer accepted. Users should switch to `v1` (or temporarily `v1beta1`) — all three schemas were identical.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (claude-opus-4-7)\n\nCloses #676 from dongjoon-hyun/SPARK-56851.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "c35ec123263ab29803c8a4b535c31c9dac97dc34",
      "tree": "9d9730b72915d644565c249ebaccbd115d696f0c",
      "parents": [
        "1a6cb2c50fe662023601feb11a08a11e048ecea2"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 13 13:32:29 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 13 13:32:29 2026 -0700"
      },
      "message": "[SPARK-56850] Add Spark `4.2.0-preview5` K8s integration tests\n\n### What changes were proposed in this pull request?\n\nThis PR adds a Chainsaw-based e2e integration test group `pi-preview` that validates [`examples/pi-preview.yaml`](examples/pi-preview.yaml) end-to-end, following the same pattern as `pi-with-comet` and `pi-with-gluten`.\n\n### Why are the changes needed?\n\nTo improve the test coverage.\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7 (1M context)\n\nCloses #675 from dongjoon-hyun/SPARK-56850.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "1a6cb2c50fe662023601feb11a08a11e048ecea2",
      "tree": "6a3671197c544cc2f221b8e9593925f7b5a00f34",
      "parents": [
        "8ec164af96010faaf29098f385bc586ff4708260"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 13 10:22:36 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 13 10:22:36 2026 -0700"
      },
      "message": "[SPARK-56849] Use Spark 4.1.1 in `Apache DataFusion Comet` example\n\n### What changes were proposed in this pull request?\n\nThis PR aims to make `tests/e2e/pi-with-comet` use Apache Spark 4.1.1 by updating `examples/pi-with-comet.yaml`.\n\n### Why are the changes needed?\n\nTo test the latest Apache Spark release. Recently, `Apache DataFusion Comet` 0.16 starts to support Apache Spark 4.1.x with a known limitation.\n- https://datafusion.apache.org/comet/user-guide/latest/compatibility/spark-versions.html#spark-4-1\n\n\u003cimg width\u003d\"946\" height\u003d\"288\" alt\u003d\"Screenshot 2026-05-13 at 09 33 13\" src\u003d\"https://github.com/user-attachments/assets/84c689d3-2062-42d7-8b48-365690101ec8\" /\u003e\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.7\n\nCloses #674 from dongjoon-hyun/SPARK-56849.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "8ec164af96010faaf29098f385bc586ff4708260",
      "tree": "2f37f00da42422f1029fb31a8e3e57185eff88a1",
      "parents": [
        "d4984b6c6801dfde49051d6ba81045d82746c66b"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 13 08:41:21 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Wed May 13 08:41:21 2026 -0700"
      },
      "message": "[SPARK-56848] Upgrade `Apache DataFusion Comet` to 0.16.0\n\n### What changes were proposed in this pull request?\n\nThis PR aims to upgrade Apache DataFusion Comet to 0.16.0 in `examples/pi-with-comet.yaml`.\n\n### Why are the changes needed?\n\nTo use the latest version of Apache DataFusion Comet 0.16.0.\n- https://github.com/apache/datafusion-comet/releases/tag/0.16.0 (2026-05-06)\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs because we have an integration test.\n- #652\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Opus 4.7)\n\nCloses #673 from dongjoon-hyun/SPARK-56848.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "d4984b6c6801dfde49051d6ba81045d82746c66b",
      "tree": "4da3a35bdbf3ca2834469bdfd46debdd989098a4",
      "parents": [
        "9b12cd2e23ec8715c7d3068942e04ffda01de3ae"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon May 11 09:52:38 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Mon May 11 09:52:38 2026 -0700"
      },
      "message": "[SPARK-56824] Bump version to 1.0.0-SNAPSHOT in `main` branch\n\n### What changes were proposed in this pull request?\n\nThis PR aims to bump version to `1.0.0-SNAPSHOT` in `main` branch.\n\n### Why are the changes needed?\n\n`branch-0.9` is created. We need to distinguish `main` branch from `branch-0.9`.\n- https://github.com/apache/spark-kubernetes-operator/tree/branch-0.9\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nPass the CIs.\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: `Claude Opus 4.7 (1M context)`\n\nCloses #671 from dongjoon-hyun/SPARK-56824.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "9b12cd2e23ec8715c7d3068942e04ffda01de3ae",
      "tree": "3506d69b02394d92c2c63def451955c742fa54d3",
      "parents": [
        "a4c228eab356c6a54160113c1609dffc330c0715"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu May 07 15:33:18 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu May 07 15:33:18 2026 -0700"
      },
      "message": "[SPARK-56789] Add `ExecutorResizePlugin` example\n\n### What changes were proposed in this pull request?\n\nThis PR aims to add a new example that show Apache Spark 4.2.0-preview5\u0027s `ExecutorResizePlugin`.\n\n- apache/spark#54215\n\n### Why are the changes needed?\n\nTo provide an out-of-the-box `SparkApplication` example that demonstrates the `ExecutorResizePlugin` along with its related configurations (`spark.kubernetes.executor.resizeInterval`, `spark.kubernetes.executor.resizeThreshold`).\n\n### Does this PR introduce _any_ user-facing change?\n\nNo, this is an example-only addition.\n\n### How was this patch tested?\n\nManual review.\n\n```\n$ kubectl apply -f examples/pi-preview-with-executor-resize-plugin.yaml\n\n$ kubectl logs -f pi-preview-with-executor-resize-plugin-0-driver | grep Plugin\n26/05/07 21:31:59 INFO DriverPluginContainer: Initialized driver component for plugin org.apache.spark.scheduler.cluster.k8s.ExecutorResizePlugin.\n26/05/07 21:32:59 INFO ExecutorResizeDriverPlugin: Increase executor 1 container memory from 1476395008 to 1624034508 as usage 505622528 exceeded threshold.\n26/05/07 21:33:59 INFO ExecutorResizeDriverPlugin: Increase executor 1 container memory from 1624034508 to 1786437958 as usage 506228736 exceeded threshold.\n```\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Opus 4.7)\n\nCloses #669 from dongjoon-hyun/SPARK-56789.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    },
    {
      "commit": "a4c228eab356c6a54160113c1609dffc330c0715",
      "tree": "d56337339a79b534bf1780cff4e1d3d3bfbbe7c1",
      "parents": [
        "50a7d03020ce0e4a7973dddd55dba94ac533a9e5"
      ],
      "author": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu May 07 13:50:30 2026 -0700"
      },
      "committer": {
        "name": "Dongjoon Hyun",
        "email": "dongjoon@apache.org",
        "time": "Thu May 07 13:50:30 2026 -0700"
      },
      "message": "[SPARK-56787] Add `Deployment`-based SparkApp example\n\n### What changes were proposed in this pull request?\n\nAdd `examples/pi-preview-deployment.yaml`, a SparkPi example that uses `spark.kubernetes.allocation.pods.allocator\u003ddeployment`.\n\n### Why are the changes needed?\n\nTo provide a ready-to-apply example demonstrating the `deployment`-based pod allocator alongside the upcoming `4.2.0-preview5` release.\n\n- apache/spark#52867\n\n### Does this PR introduce _any_ user-facing change?\n\nNo.\n\n### How was this patch tested?\n\nManually tested.\n\n```\n$ kubectl apply -f examples/pi-preview-deployment.yaml\nsparkapplication.spark.apache.org/pi-preview-deployment created\n\n$ kubectl get sparkapp\nNAME                    AGE   CURRENT STATE\npi-preview-deployment   3s    DriverRequested\n\n$ kubectl get sparkapp\nNAME                    AGE   CURRENT STATE\npi-preview-deployment   11s   RunningHealthy\n\n$ kubectl get deploy\nNAME                                READY   UP-TO-DATE   AVAILABLE   AGE\nspark-d-pi-preview-deployment-0-0   3/3     3            3           16s\nspark-kubernetes-operator           1/1     1            1           3d4h\n\n$ kubectl get pod\nNAME                                                 READY   STATUS    RESTARTS      AGE\npi-preview-deployment-0-driver                       1/1     Running   0             23s\nspark-d-pi-preview-deployment-0-0-7cc95db6bb-8dzln   1/1     Running   0             20s\nspark-d-pi-preview-deployment-0-0-7cc95db6bb-hhfqn   1/1     Running   0             20s\nspark-d-pi-preview-deployment-0-0-7cc95db6bb-pglpb   1/1     Running   0             20s\nspark-kubernetes-operator-584498648c-pxfpc           1/1     Running   4 (46h ago)   3d4h\n```\n\n### Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Opus 4.7)\n\nCloses #668 from dongjoon-hyun/SPARK-56787.\n\nAuthored-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\nSigned-off-by: Dongjoon Hyun \u003cdongjoon@apache.org\u003e\n"
    }
  ],
  "next": "50a7d03020ce0e4a7973dddd55dba94ac533a9e5"
}