)]}'
{
  "log": [
    {
      "commit": "bc234453011e55138767f1b0ad3de9d4f77f93e8",
      "tree": "33a9687973bd3864727ba78c0b40adc144b4679e",
      "parents": [
        "722b8f023847a72319539977ee13122baa07773f"
      ],
      "author": {
        "name": "Gabor Roczei",
        "email": "1918366+roczei@users.noreply.github.com",
        "time": "Tue Oct 20 18:53:15 2026 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Oct 20 12:53:15 2026 -0400"
      },
      "message": "[LIVY-1090] Upgrade SLF4J 1.7.36 -\u003e 2.0.17 (log4j-slf4j-impl -\u003e log4j-slf4j2-impl), replace jetty-runner\n\n## What changes were proposed in this pull request?\n\nSLF4J 2.0.x replaced the 1.x StaticLoggerBinder lookup with a ServiceLoader-based\nSLF4JServiceProvider, so the Log4j 2 binding must be swapped from log4j-slf4j-impl\n(SLF4J 1.x) to log4j-slf4j2-impl (SLF4J 2.x).\n\n- pom.xml: slf4j.version 1.7.36 -\u003e 2.0.17\n- Swap log4j-slf4j-impl -\u003e log4j-slf4j2-impl in the binding declarations:\n  root dependencyManagement, core, repl (provided), rsc (provided)\n- thriftserver/client: exclude log4j-slf4j-impl from hive-jdbc and hive-beeline\n  (hive-jdbc:3.0.0 and hive-beeline -\u003e hive-common still bring log4j-slf4j-impl:2.10.0)\n  and declare log4j-slf4j2-impl explicitly\n- integration-test: additionally exclude log4j-slf4j2-impl from the test-scoped\n  hive-jdbc (next to the existing log4j-slf4j-impl exclusion)\n- thriftserver/server: replace the jetty-runner uber-jar (which bundles a shaded\n  SLF4J 1.x that shadowed slf4j-api 2.x and caused a \"Failed to load class\n  org.slf4j.impl.StaticLoggerBinder\" NOP fallback at server startup) with the\n  individual jetty-server, jetty-servlet and jetty-webapp modules, and drop the\n  now-obsolete jetty-util/jetty-xml/jetty-webapp excludeArtifactIds that only\n  existed to avoid duplicating jetty-runner\u0027s bundled copies. The existing\n  log4j-slf4j-impl exclusion on hive-jdbc is kept.\n- Root pom: manage jetty-webapp, jetty-xml and jetty-client at ${jetty.version} in\n  dependencyManagement so all Jetty modules resolve to the same version as\n  livy-server, overriding Hive\u0027s older Jetty (LIVY-543). The jetty-webapp declaration\n  in thriftserver/server now inherits its version from the root pom.\n\nNo Log4j bump needed (already on 2.26.0, \u003e\u003d the 2.23.0 minimum for log4j-slf4j2-impl).\nLivy has no org.slf4j.impl / StaticLoggerBinder usage, so no source changes are required.\n\nJIRA: https://issues.apache.org/jira/browse/LIVY-1090\n\n## How was this patch tested?\n\nUnit tests.\n- core/src/test/scala/org/apache/livy/LoggingSuite.scala: verifies a single\n  SLF4JServiceProvider is registered and resolves to Log4j 2 (org.apache.logging.slf4j),\n  and that messages logged through the Logging trait are routed to Log4j 2. Passes 2/2\n  on Scala 2.13 and test-compiles under Scala 2.12.\n- thriftserver/server/src/test/scala/org/apache/livy/thriftserver/TestThriftServerLogging.scala:\n  asserts exactly one org.slf4j.LoggerFactory is on the classpath (fails if a shaded\n  SLF4J such as jetty-runner is reintroduced) and that SLF4J binds to Log4j 2 rather\n  than falling back to the NOP logger.\n\nAlso verified via mvn dependency:tree on core, repl, rsc, server, and thriftserver/server:\nslf4j-api resolves to 2.0.17, log4j-slf4j2-impl is present, and no log4j-slf4j-impl\nremains on the compile/runtime classpath (including under hive-jdbc and hive-beeline).\nStarted livy-server locally and confirmed the StaticLoggerBinder NOP warning is gone and\nlogging works.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Claude Opus 4.8)"
    },
    {
      "commit": "722b8f023847a72319539977ee13122baa07773f",
      "tree": "cc4f6cd37db3844c15fd72d8d96d6400aafa318b",
      "parents": [
        "fc5f2387be5682f880503ad98b4fcec3bfa190aa"
      ],
      "author": {
        "name": "idzikovsky",
        "email": "23353239+idzikovsky@users.noreply.github.com",
        "time": "Tue Oct 06 19:22:36 2026 +0300"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Oct 06 12:22:36 2026 -0400"
      },
      "message": "[LIVY-1088] Update of frontend libs to resolve unnecessary CVE scanner warnings\n\n## What changes were proposed in this pull request?\n\nhttps://github.com/apache/livy/issues/560\n\nhttps://issues.apache.org/jira/browse/LIVY-1088 - Update of frontend libs to resolve unnecessary CVE scanner warnings\n\nSince it is not a good practice to accept PRs with minified lib updates from anyone in community, I will provide all links to the libraries I\u0027ve updated so maintainers should be able to verify checksums, or alternatively it should be OK for you to update libraries by yourself, and just cherry-pick this commit to update HTML templates work with Bootstrap 4:\n\nhttps://github.com/apache/livy/pull/563/changes/16637dbddd88629c86eef07cd50f4e0471546df3\nhttps://github.com/apache/livy/pull/563/changes/c66a65af427d6a887df6e2ddef8dd2e7d5b27a4d\n\nI took jQuery from here:\n- https://releases.jquery.com/jquery/\n- https://blog.jquery.com/2023/08/28/jquery-3-7-1-released-reliable-table-row-dimensions/\n\nBootstrap from here:\n- https://getbootstrap.com/docs/4.6/getting-started/download/\n- https://github.com/twbs/bootstrap/releases/download/v4.6.2/bootstrap-4.6.2-dist.zip\n\nAnd dataTables from this link:\nhttps://cdn.datatables.net/1.13.11/\n\n\n## How was this patch tested?\n\nManual sanity check of the Livy UI after all those updates.\n\nAlso, I\u0027ve built Livy with running unit tests.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes.\n\nGenerated-by: GitHub Copilot CLI (claude-opus-5.5)"
    },
    {
      "commit": "fc5f2387be5682f880503ad98b4fcec3bfa190aa",
      "tree": "cd74c3bf1b925420c505eb3592ef08027393c997",
      "parents": [
        "48a143a9e234ef9e6dc2a04e12a71e956a5b9a5c"
      ],
      "author": {
        "name": "Gabor Roczei",
        "email": "1918366+roczei@users.noreply.github.com",
        "time": "Mon Oct 05 21:42:16 2026 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Oct 05 15:42:16 2026 -0400"
      },
      "message": "[LIVY-1089] Upgrade Mockito 1.10.19 -\u003e 3.4.6 (mockito-all -\u003e mockito-core)\n\n## What changes were proposed in this pull request?\n\nmockito-all:1.10.19 is an uber-jar that bundles Hamcrest 1.x which\nconflicts with the Hamcrest 2.x pulled in by the ScalaTest 3.2.9 /\nscalatestplus mockito-3-4 upgrade: the bundled LocalizedMatcher does\nnot implement Matcher.describeMismatch(Object, Description) and fails\nto compile. This switches to mockito-core (which does not bundle\nHamcrest) and aligns the version with the mockito-3-4_2.13:3.2.9.0\nbinding.\n\n- pom.xml: mockito-all -\u003e mockito-core, version 1.10.19 -\u003e 3.4.6\n- Replace removed org.mockito.Matchers with org.mockito.ArgumentMatchers\n- Replace removed anyObject() with any()\n- Rewrite the custom ArgumentMatcher[Path] to the Mockito 2+ shape\n  (matches(T) instead of matches(Object); drop the removed describeTo),\n  dropping the now-unused Hamcrest Description and internal Equals imports\n\nNote: in Mockito 3.4.6 org.mockito.Matchers still exists only as a\ndeprecated subclass of ArgumentMatchers and declares no members of its\nown, so Scala cannot import its inherited statics (e.g.\n\"import org.mockito.Matchers.any\" fails to compile). All Scala specs\nmust import from org.mockito.ArgumentMatchers directly.\n\nUpdated specs: HttpClientSpec, ScalaJobHandleTest, BatchSessionSpec,\nInteractiveSessionServletSpec, InteractiveSessionSpec,\nFileSystemStateStoreSpec, SparkYarnAppSpec.\n\nJIRA: https://issues.apache.org/jira/browse/LIVY-1089\n\n## How was this patch tested?\n\nCompiled the server module and its reactor dependencies with\n`mvn -pl server -am test-compile`, which succeeds (exit 0) and confirms\nall migrated specs now compile under Mockito 3.4.6 - including\nSparkYarnAppSpec, whose `import org.mockito.Matchers.any` would\notherwise fail to compile.\n\nAlso compiled the thriftserver modules with the profile enabled via\n`mvn -Pthriftserver -pl thriftserver/server,thriftserver/session,thriftserver/client -am test-compile`,\nwhich succeeds (exit 0) - its tests use org.mockito.Mockito mock/when,\nwhich are unaffected by the migration.\n\nExisting unit tests exercise the migrated mocks.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Claude Opus 4.8)"
    },
    {
      "commit": "48a143a9e234ef9e6dc2a04e12a71e956a5b9a5c",
      "tree": "8f2097059a69530b138ae85468f75e92957aa7aa",
      "parents": [
        "c7af4370a61ea3472ac96a63dc3e1b7bdce25445"
      ],
      "author": {
        "name": "Soumyadeep Roy",
        "email": "soumyadeeplogin@gmail.com",
        "time": "Thu Sep 24 00:32:25 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Sep 23 15:02:25 2026 -0400"
      },
      "message": "[LIVY-1084] Prefer spark container exit status over pod phase when sidecars are enabled\n\n## What changes were proposed in this pull request?\n\nWhen `livy.server.kubernetes.spark.sidecar.enabled` is `true`, `SparkKubernetesApp.KubernetesAppReport.getDriverState` returned the raw pod phase whenever the phase wasn\u0027t `\"running\"`, before ever consulting the spark container\u0027s own termination status. A sidecar container that fails or gets OOMKilled after the spark driver container has already exited successfully can flip the pod phase to `\"Failed\"`, which Livy then reports as the terminal application state even though the actual Spark job succeeded.\n\nThis PR changes `getDriverState` so that, when sidecars are enabled, it always consults `getTerminalState` (the spark container\u0027s own exit code) first, and only falls back to the pod phase if the spark container cannot be found in the pod\u0027s container statuses. It also guards `getTerminalState` against `getContainerStatuses()` returning `null`, which happens while the pod is still in the `Pending` phase, to avoid an NPE.\n\n## Why are the changes needed?\n\nTo report the correct terminal application state (succeeded/failed) based on the actual Spark driver\u0027s exit code, rather than being misled by an unrelated sidecar container\u0027s failure flipping the pod phase.\n\n## Does this PR introduce _any_ user-facing change?\n\nYes, but only when `livy.server.kubernetes.spark.sidecar.enabled` is `true` (the default). `/sessions/:id/state` and `/batches/:id/state` will now report `succeeded` for a pod whose spark container exited 0 even if a sidecar container\u0027s failure left the pod phase as `Failed`.\n\n## How was this patch tested?\n\nAdded a unit test to `SparkKubernetesAppSpec` (`should prefer spark container exit status over pod phase when sidecars are enabled`) covering: sidecar failure with a successfully-terminated spark container, a failed spark container, an indeterminate/not-yet-terminated spark container (falls back to phase), a pending pod with null container statuses (no NPE, falls back to phase), and sidecars disabled (phase is authoritative).\n\nRan `mvn -pl server -am test -Dtest\u003dSparkKubernetesAppSpec -DwildcardSuites\u003dorg.apache.livy.utils.SparkKubernetesAppSpec`; all 12 tests pass.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes.\n\nGenerated-by: Claude Code (Sonnet 5, Anthropic), used to implement the\nexit-status-priority fix in `getDriverState` and the corresponding `SparkKubernetesAppSpec`\ntest cases, under human review. Please refer to the\n[ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html)\nfor details."
    },
    {
      "commit": "c7af4370a61ea3472ac96a63dc3e1b7bdce25445",
      "tree": "12ea5e4406741de578a3de768b4fc40152e3ea3a",
      "parents": [
        "4b53b4c34c63556b7a54b11001237a1b2d41cde7"
      ],
      "author": {
        "name": "Gyorgy Gal",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Wed Sep 23 16:12:52 2026 +0000"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Sep 23 12:12:52 2026 -0400"
      },
      "message": "[LIVY-1085][TEST] Skip remaining R test paths when -DskipRTests is specified\n\n## What changes were proposed in this pull request?\n\nTwo R-dependent tests (`SharedSessionSpec`, `InteractiveSessionSpec`) ran a\nreal `sparkr` interpreter even with `-DskipRTests\u003dtrue`, unlike every other\nR test in the suite. Guard both with the same `sys.props` pattern already\nused by `SparkRInterpreterSpec`/`SparkRSessionSpec`/`Batch`IT/`InteractiveIT`,\nand make `InteractiveSessionSpec`\u0027s downstream `execution_count` expectation\nconditional on the flag, since it depends on the skipped R statement\nhaving incremented the session\u0027s shared statement counter.\n\n## How was this patch tested?\n\nVerified via `mvn test` runs of both specs with `-DskipRTests\u003dtrue` (R test cancels,\nrest pass) and the default `-DskipRTests\u003dfalse` on a machine with R installed.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (claude-sonnet-5)"
    },
    {
      "commit": "4b53b4c34c63556b7a54b11001237a1b2d41cde7",
      "tree": "16e19fb4075742aba8f990f1c2c9a21728afcd35",
      "parents": [
        "7205db3a983461eef2b9a5820fd36e44d58f2f2f"
      ],
      "author": {
        "name": "Soumyadeep Roy",
        "email": "soumyadeeplogin@gmail.com",
        "time": "Wed Sep 23 00:53:48 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Sep 22 15:23:48 2026 -0400"
      },
      "message": "[LIVY-1073] Add SessionManager refresh and /recovery operator endpoint\n\n## What changes were proposed in this pull request?\n\nAdds a `SessionManager` refresh capability and a `/recovery` operator endpoint on `LivyServer`, allowing an operator to trigger a manual re-sync of in-memory session state against the recovery/session-state store without restarting the Livy server process.\n\nThe `/recovery/*` endpoints are implemented as a standalone `RecoveryServlet`, restricted to `livy.superusers`. If `livy.server.recovery.mode` is `off` (no state store configured), the endpoints return `409` rather than a silently-empty result. The three endpoints (`/recovery/sessions/refresh`, `/recovery/batches/refresh`, `/recovery/refresh`) are documented in `docs/rest-api.md`.\n\n## How was this patch tested?\n\nUnit tests added in `SessionManagerSpec` covering the refresh path, and `RecoveryServletSpec` covering the servlet\u0027s authorization (403 vs 200) and recovery-mode-off (409) behavior.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes.\n\nGenerated-by: Claude Code (Sonnet 5, Anthropic), used to scan/plan the extraction of local patches for upstreaming and to draft/iterate on the code and tests in this PR under human review. Please refer to the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) for details.\n\nCo-authored-by: Venugopal Allenki \u003cvallenki@salesforce.com\u003e"
    },
    {
      "commit": "7205db3a983461eef2b9a5820fd36e44d58f2f2f",
      "tree": "b63dd6b85ee54105a3b72edd752c6c18d10b2799",
      "parents": [
        "7f24ceb9e4361af2fd9bcfc04527b41a33bfb6ce"
      ],
      "author": {
        "name": "nileshrathi345",
        "email": "113672051+nileshrathi345@users.noreply.github.com",
        "time": "Tue Sep 22 21:03:46 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Sep 22 11:33:46 2026 -0400"
      },
      "message": "[LIVY-1065][TEST] Pin Spark driver to loopback in remaining macOS tests\n\nExtend the LIVY-1065 macOS test fixes by binding Spark\u0027s driver host and bind address to TestUtils.TEST_BIND_HOST in test fixtures that start Spark directly (REPL interpreter/session specs, RSC client, Scala API client, interactive server tests, Thrift server JDBC sessions, and MiniCluster).\n\n## What changes were proposed in this pull request?\n\nThis is a follow-up to [#538](https://github.com/apache/livy/pull/538)\n\n[PR #538](https://github.com/apache/livy/pull/538) introduced `TestUtils.TEST_BIND_HOST `and pinned Livy, RSC, YARN mini-cluster, and several unit-test services to loopback (`127.0.0.1`) so tests pass on macOS. However, some unit tests still start Spark directly without setting `spark.driver.host` and `spark.driver.bindAddress`. On macOS, Spark\u0027s driver can advertise the machine\u0027s LAN IP (from `InetAddress.getLocalHost()`), which peer processes on the same host cannot reach, causing connect timeouts and stage failures (e.g. `Failed to connect to /192.168.x.x:...`).\n\nThis PR extends the LIVY-1065 fix by:\n\n- **`client-common/TestUtils.java`**: Add `SPARK_DRIVER_HOST` and `SPARK_DRIVER_BIND_ADDRESS` constants so Spark driver config keys are not duplicated as raw strings across modules (same pattern as `TEST_BIND_HOST`).\n\n- **`Unit-test fixtures:`** Set `spark.driver.host` and `spark.driver.bindAddress` to `TestUtils.TEST_BIND_HOST `in fixtures that create Spark sessions/drivers directly:\n1. `repl`: `BaseSessionSpec` ,`SessionSpec`, `ReplDriverSuite`, `PythonInterpreterSpec`, `ScalaInterpreterSpec`, `SQLInterpreterSpec`, `SparkRInterpreterSpec`\n2. `rsc`: `TestSparkClient`\n3. `scala-api`: `ScalaClientTest`\n4. `server`: `BaseInteractiveServletSpec`, `InteractiveSessionSpec`\n5. `thriftserver`: `ThriftServerBaseTest` (inject driver host/bind via JDBC session conf in `jdbcUri()`)\n\n- **`integration-test/MiniCluster.scala`**: Use `TestUtils.SPARK_DRIVER_HOST` instead of a hardcoded `\"spark.driver.host\"` key string (value remains `TEST_BIND_HOST` from [#538](https://github.com/apache/livy/pull/538)).\n\nTogether with [#538](https://github.com/apache/livy/pull/538), this ensures both Livy/RSC services and Spark driver processes started from tests bind/advertise a locally reachable loopback address on macOS. Linux CI behavior is unchanged.\n\nJIRA: https://issues.apache.org/jira/browse/LIVY-1086\n\n## How was this patch tested?\nTested locally on macOS (aarch64, JDK 17):\n`$ mvn -Pthriftserver -B verify\n...\n\n```\n[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 0, approved: 1 licenses.\n[INFO] ------------------------------------------------------------------------\n[INFO] Reactor Summary for Livy Project Parent POM 1.0.0-SNAPSHOT:\n[INFO] \n[INFO] Livy Project Parent POM ............................ SUCCESS [  3.706 s]\n[INFO] livy-api ........................................... SUCCESS [  5.891 s]\n[INFO] livy-client-common ................................. SUCCESS [  2.877 s]\n[INFO] livy-test-lib ...................................... SUCCESS [  3.400 s]\n[INFO] multi-scala-project-root ........................... SUCCESS [  1.027 s]\n[INFO] livy-core-parent ................................... SUCCESS [  0.295 s]\n[INFO] livy-core_2.13 ..................................... SUCCESS [  1.394 s]\n[INFO] livy-rsc ........................................... SUCCESS [ 29.710 s]\n[INFO] livy-repl-parent ................................... SUCCESS [  3.695 s]\n[INFO] livy-repl_2.13 ..................................... SUCCESS [01:29 min]\n[INFO] livy-server ........................................ SUCCESS [03:33 min]\n[INFO] livy-thriftserver-session .......................... SUCCESS [ 14.393 s]\n[INFO] livy-thriftserver .................................. SUCCESS [05:34 min]\n[INFO] livy-assembly ...................................... SUCCESS [  4.162 s]\n[INFO] livy-client-http ................................... SUCCESS [ 12.459 s]\n[INFO] livy-scala-api-parent .............................. SUCCESS [  1.066 s]\n[INFO] livy-scala-api_2.13 ................................ SUCCESS [ 20.797 s]\n[INFO] livy-integration-test .............................. SUCCESS [24:52 min]\n[INFO] livy-coverage-report ............................... SUCCESS [  3.216 s]\n[INFO] livy-examples ...................................... SUCCESS [  4.174 s]\n[INFO] livy-python-api .................................... SUCCESS [  2.423 s]\n[INFO] livy-beeline ....................................... SUCCESS [  2.968 s]\n[INFO] ------------------------------------------------------------------------\n[INFO] BUILD SUCCESS\n[INFO] ------------------------------------------------------------------------\n[INFO] Total time:  37:27 min\n[INFO] Finished at: 2026-09-17T19:24:42+05:30\n[INFO] ------------------------------------------------------------------------\n\n```\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Cursor AI (Opus 4.8)"
    },
    {
      "commit": "7f24ceb9e4361af2fd9bcfc04527b41a33bfb6ce",
      "tree": "8f9891dd3430910f8c12612aa091b07e5ab15cda",
      "parents": [
        "4f6ee668f7af8e771b921acf7dd88dbf77d36bb8"
      ],
      "author": {
        "name": "ashokkumarrathore",
        "email": "30638343+ashokkumarrathore@users.noreply.github.com",
        "time": "Tue Sep 22 20:56:25 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Sep 22 11:26:25 2026 -0400"
      },
      "message": "[LIVY-1079] Support multi-tenant (multi-namespace) Kubernetes clusters\n\n## What changes were proposed in this pull request?\n\nRelated issue: https://github.com/apache/incubator-livy/issues/461\nJIRA: https://issues.apache.org/jira/browse/LIVY-1079\n\nLivy on Kubernetes previously assumed a single namespace and listed/killed applications across all namespaces (`inAnyNamespace`). In a multi-tenant cluster the Livy service account is typically scoped to a subset of namespaces, so cluster-wide calls fail. This PR makes the namespace a first-class, per-session property:\n\n- `SparkApp.getNamespace` resolves the target namespace from the session Spark conf (`spark.kubernetes.namespace`), falling back to `$SPARK_CONF_DIR`/`$SPARK_HOME/conf/spark-defaults.conf` and finally the `default` namespace.\n- The namespace is threaded through `SparkApp.create` and persisted in the batch and interactive recovery metadata, so a recovered session keeps the namespace it was created with. Pre-upgrade metadata (no namespace field) is treated as \"unknown\" and adopts the driver pod\u0027s real namespace once discovered.\n- `SparkKubernetesApp` scopes every client call with `.inNamespace(...)` and tracks the set of namespaces it has seen in a thread-safe set, so leaked-application cleanup iterates only over namespaces Livy actually has access to and isolates per-namespace failures.\n- Bumps `kubernetes-client` to 6.8.1 and `netty` to 4.1.108.Final for security fixes.\n\n## How was this patch tested?\n\n- Added `SparkAppSpec` covering the namespace-resolution precedence chain (including malformed-conf fallback), and a `BatchSession` recovery test asserting the namespace round-trips through recovery metadata; updated existing recovery-metadata fixtures for the new field.\n- Manually tested by deploying the updated server in a k8s cluster with multiple namespaces and running a job.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nAuthor: Ashok Kumar \u003cashokkumarrathore@gmail.com\u003e\nGenerated-by: Claude Opus 4.8"
    },
    {
      "commit": "4f6ee668f7af8e771b921acf7dd88dbf77d36bb8",
      "tree": "a8064812ff51079bfa0392daa4a0aadcfa979660",
      "parents": [
        "35fa3dcae5da72e4959e9eb46ea91a023ddb6bd7"
      ],
      "author": {
        "name": "Soumyadeep Roy",
        "email": "soumyadeeplogin@gmail.com",
        "time": "Sat Sep 19 00:50:27 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Fri Sep 18 15:20:27 2026 -0400"
      },
      "message": "[LIVY-1082] Add config to disable Kubernetes driver pod log polling\n\n## What changes were proposed in this pull request?\n\n`SparkKubernetesApp` currently fetches the driver pod\u0027s Kubernetes log (`pods/\u003cdriver\u003e/log`) on every poll cycle to populate live log lines in `/sessions/:id/log` and `/batches/:id/log` responses. On large clusters, or when driver logs are already collected externally (e.g. via a centralized log pipeline / Spark History Server), this is wasted request volume against the Kubernetes API server, and can be a source of transient errors during pod lifecycle transitions.\n\nThis PR adds a new config, `livy.server.kubernetes.driver-log-polling.enabled` (default `true`, preserving existing behavior), that when set to `false` skips the `pods/\u003cdriver\u003e/log` fetch while Livy continues to poll pod state and diagnostics normally.\n\n## Why are the changes needed?\n\nTo let operators opt out of the driver log poll when it\u0027s unnecessary, reducing Kubernetes API server load and avoiding transient log-fetch errors during pod lifecycle transitions, without losing pod state/diagnostics monitoring.\n\n## Does this PR introduce _any_ user-facing change?\n\nYes. A new optional config `livy.server.kubernetes.driver-log-polling.enabled` is added (default `true`). When explicitly set to `false`, `/sessions/:id/log` and `/batches/:id/log` will no longer include live Kubernetes driver log lines; pod state and diagnostics continue to be polled and reported as before.\n\n## How was this patch tested?\n\nAdded/updated unit tests in `SparkKubernetesAppSpec`:\n- `should enable driver log polling by default`\n- `resolveDriverAppLog` - `should skip fetching the log when driver log polling is disabled`\n- `resolveDriverAppLog` - `should fetch the log when driver log polling is enabled`\n\nRan `mvn -pl server -am test -Dtest\u003dSparkKubernetesAppSpec -DwildcardSuites\u003dorg.apache.livy.utils.SparkKubernetesAppSpec`; all 14 tests in the suite pass.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes, this patch was co-authored using Claude Code (Anthropic)."
    },
    {
      "commit": "35fa3dcae5da72e4959e9eb46ea91a023ddb6bd7",
      "tree": "ea09540b4bb88bc6f37b6bfc7de14294d194364d",
      "parents": [
        "628a6f6f1e84a3c5c72d7d892f33ccd5e62c6759"
      ],
      "author": {
        "name": "idzikovsky",
        "email": "23353239+idzikovsky@users.noreply.github.com",
        "time": "Wed Sep 16 19:10:37 2026 +0300"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Sep 16 12:10:37 2026 -0400"
      },
      "message": "[LIVY-1080] Add missing livy.server.launch.kerberos.* properties to default livy.conf\n\n## What changes were proposed in this pull request?\n\nThis properties allow to configure kinit thread and not documented anywhere except for the code:\nhttps://github.com/apache/incubator-livy/blob/86fc823/server/src/main/scala/org/apache/livy/LivyConf.scala#L106-L109\nhttps://github.com/apache/incubator-livy/blob/86fc823/server/src/main/scala/org/apache/livy/server/LivyServer.scala#L106-L141\n\n## How was this patch tested?\n\nManual\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nNo"
    },
    {
      "commit": "628a6f6f1e84a3c5c72d7d892f33ccd5e62c6759",
      "tree": "f9a699562b6780f7fdaeb89465e0d8e82d2ccc7d",
      "parents": [
        "e52d1d1639f7cb39372fc59f97fee524a0049fff"
      ],
      "author": {
        "name": "samuhasavak5",
        "email": "samuhasavak5@gmail.com",
        "time": "Tue Sep 15 21:07:24 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Sep 15 11:37:24 2026 -0400"
      },
      "message": "[LIVY-1070] Gate session metrics behind livy.server.session-metrics.enabled\n\n## What changes were proposed in this pull request?\n\nThis PR adds an opt-in configuration flag for LIVY-1070 session monitoring gauges (follow-up to #543).\n\nProblem: PR #543 always registers `LivySessionMetrics` at server startup. Operators and downstream distributions need per-deployment control — session gauges should be disabled by default and enabled only when monitoring is desired.\n\nSolution: Introduce `livy.server.session-metrics.enabled` (default `false`). When `true`, `LivySessionMetrics.register()` runs at startup (same behavior as #543). When `false`, the `/metrics/metrics` servlet remains available but `livy.sessions.*` gauges are not published.\n\n### Changes\n\nFile | Change\n-- | --\nLivyConf.scala | Add `SESSION_METRICS_ENABLED` config entry\nLivyServer.scala | Gate `LivySessionMetrics.register()` behind config flag\nLivySessionMetricsSpec.scala | Unit tests (+1 case: default disabled)\ndocs/rest-api.md | Document opt-in behavior for `/metrics/metrics`\nconf/livy.conf.template | Commented example config\n\nNo changes to `LivySessionMetrics.scala` — the 18 gauges implementation remains as merged in #543.\n\n### Configuration\nDefault: opt-in disabled\nlivy.server.session-metrics.enabled\u003dfalse\n\nEnable session gauges (same 18 gauges as #543)\nlivy.server.session-metrics.enabled\u003dtrue\n\n\n### Behavior when enabled (`livy.server.session-metrics.enabled\u003dtrue`)\n\nSame 18 gauges as #543:\n\nOverall (3)\n\n* `livy.sessions.total`\n* `livy.sessions.active.total`\n* `livy.sessions.terminal.total`\n\nInteractive (8)\n\n* `livy.sessions.interactive.total`\n* `livy.sessions.interactive.{idle,busy,starting,shutting_down,dead,error,killed}`\n\nBatch (7)\n\n* `livy.sessions.batch.total`\n* `livy.sessions.batch.{starting,running,success,dead,error,killed}`\n\n### Design notes\n\n* Minimal follow-up to #543 — only gates existing gauge registration\n* Default `false` — opt-in per deployment\n* No REST API or session lifecycle behavior changes\n* Idempotent registration unchanged from #543 — skips gauge names already present in the registry\n* Error-safe callbacks unchanged from #543 — gauge `getValue` returns `0` on exception\n* HA note — gauge values reflect the local Livy server instance; in HA deployments only the leader holds active sessions\n\n### Compatibility\n\n* No new endpoints; uses existing `/metrics` AdminServlet\n* Behavior change from #543: session gauges are no longer always-on; set `livy.server.session-metrics.enabled\u003dtrue` to enable them\n* When enabled, gauge names and semantics are identical to #543\n\nJIRA: https://issues.apache.org/jira/browse/LIVY-1070\n\n## How was this patch tested?\n\n### Build\nmvn package -Pspark3 -Pscala-2.12 -pl server -am\n-s /tmp/livy-mvn-central-settings.xml -DskipTests\n\n\nResult: BUILD SUCCESS\n\n### Unit tests\n\nmvn test -Pspark3 -Pscala-2.12 -pl server\n-s /tmp/livy-mvn-central-settings.xml\n-Dsuites\u003dorg.apache.livy.server.LivySessionMetricsSpec\n\n\nResult: 9/9 tests passed\n\nTest | Coverage\n-- | --\nDefault session metrics disabled in LivyConf | Opt-in default\nAll 18 gauge registrations | Registration\nDuplicate registration guard | Idempotency\nZero sessions | Empty state\nInteractive sessions by state | State counting\nBatch sessions by state (incl. succeeded alias) | State counting\nBatch succeeded alias | Edge case\nOverall totals (total, active, terminal) | Aggregation\nException fallback returns 0 | Error handling\n\n### Code coverage\n\nJaCoCo agent enabled during test run (`server/target/jacoco/main.exec` generated).\n\nNo UI changes in this PR.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes, this was co-authored using Cursor to help generate the new test case and documentation."
    },
    {
      "commit": "e52d1d1639f7cb39372fc59f97fee524a0049fff",
      "tree": "0713f5b0aa9f6ea797f67d911697ade2469fb922",
      "parents": [
        "094581b3553d013893512b6ce4377b61ebce8c13"
      ],
      "author": {
        "name": "Gyorgy Gal",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Tue Sep 15 14:01:27 2026 +0000"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Sep 15 10:01:27 2026 -0400"
      },
      "message": "[LIVY-1072][FOLLOWUP] Fix Scala 2.13 compile error in SparkKubernetesApp\n\n## What changes were proposed in this pull request?\n\n`KubernetesExtensions.getApplicationReport` failed to compile under\nScala 2.13 (the default build): `client.pods...list.getItems.asScala`\nreturns `scala.collection.mutable.Buffer`, but the enclosing `val\nexecutors: Seq[Pod]` requires `scala.collection.immutable.Seq` (the 2.13\ndefault for the unqualified `Seq` alias). Scala 2.12 allowed the\nimplicit widening; 2.13 does not.\n\nAdds `.toSeq`, matching the same conversion already used a few lines\nabove for the driver-pod lookup.\n\n## How was this patch tested?\n\n`mvn install -pl core/scala-2.13,server -am -DskipTests` failed with\nthis exact type-mismatch error before the fix and succeeds after it.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Sonnet 5)"
    },
    {
      "commit": "094581b3553d013893512b6ce4377b61ebce8c13",
      "tree": "e59561fc00c014ff92e08c27fa5789c218449036",
      "parents": [
        "34549272ba3bbdab105445b8f6466604467124bc"
      ],
      "author": {
        "name": "Gabor Roczei",
        "email": "1918366+roczei@users.noreply.github.com",
        "time": "Mon Sep 14 23:49:15 2026 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Sep 14 17:49:15 2026 -0400"
      },
      "message": "[LIVY-1041] Add Spark 4 support and make it the default build\n\n## What changes were proposed in this pull request?\n\nThis PR adds support for building and running Apache Livy against\nApache Spark 4.x (pinned to 4.1.2). Spark 4 is now the default build\n(`mvn package` with no extra profiles); Spark 3.x remains available via\n`-Pspark3 -Pscala-2.12`.\n\nSpark 4 introduced breaking changes that required widespread updates:\nJDK 17 minimum, Scala 2.13 only, Netty 4.2.x, a rewritten\nScala REPL, and json4s 4.0.7.\n\n**Build / POM changes**\n- pom.xml: root properties default to Spark 4.1.2, Scala 2.13.17,\n  Hadoop 3.4.1, and `java.version\u003d17` so `mvn package` works without\n  profile flags. The existing `spark3` and `scala-2.12` profiles\n  override those defaults for legacy Spark 3 builds; dedicated\n  `spark4` / `scala-2.13` profiles are no longer needed. Per Gyorgy\u0027s\n  suggestion on [LIVY-1069](https://lists.apache.org/thread/xmb0hlqtmh0po2o75c025047x26lor9c),\n  the latest supported Spark/Scala versions are now the default rather\n  than requiring explicit `-P` flags.\n- JDK handling: maven-enforcer follows the active profile — JDK 17 for the default\n  Spark 4 build, JDK 8 minimum when `-Pspark3` sets `java.version\u003d1.8`.\n- maven-shade-plugin 3.5.0 -\u003e 3.6.2 (bundles an ASM version that\n  understands Java 22 bytecode, needed to shade Jackson 2.18.2\u0027s\n  `META-INF/versions/22` multi-release classes).\n- repl/pom.xml: added `json4s-jackson-core` to the shade includes\n  (json4s 4.0.7 split `JsonMethods` into a separate artifact; missing\n  it caused a NoClassDefFoundError in PythonInterpreter at runtime).\n  Also pulls `livy-test-lib` in as a test dependency so the repl tests\n  can use the new shared `ScalaVersionAware` trait.\n- New Scala 2.13 module wrapper POMs: `core/scala-2.13`,\n  `repl/scala-2.13`, `scala-api/scala-2.13`.\n- README.md: documented default Spark 4 build and Spark 3 profile\n  usage; Spark 4 Python compatibility — supported Python versions for\n  Spark 4.1 are 3.10 – 3.14.\n\n**Spark 4 / Scala 2.13 source-level fixes**\n- `repl/scala-2.13/SparkInterpreter.scala`: new implementation ported\n  to the Spark 4 SparkILoop (`shell.ILoop`, `PrintWriter`,\n  `createInterpreter(settings)`, operations via `sparkILoop.intp`,\n  `ReplCompletion`). To make extra JARs supplied via\n  `spark.jars.packages` visible to `import ...` we feed those JARs to\n  the Scala compiler through the `-classpath` argument at Settings\n  construction time, instead of via a post-init\n  `IMain.addUrlsToClassPath` call. In Scala 2.13 the latter does not\n  reliably register URLs with the compiler\u0027s `platform.classPath`, so\n  `import org.codehaus.plexus.util._` fails with `object plexus is not\n  a member of package org.codehaus`. A new `collectUserJarsClasspath()`\n  helper walks the context classloader chain to Spark\u0027s\n  `MutableURLClassLoader`, filters out `livy-*` and the wrong-version\n  `scala-reflect` (same rules as the old `addUrls` path), and joins the\n  resulting file paths with `File.pathSeparator`. If the chain has no\n  `MutableURLClassLoader` we log a warning and skip the extra\n  `-classpath` entry rather than passing an empty argument.\n\n- `AbstractSparkInterpreter.scala`:\n  * `parseError` skips the leading caret line so `ename` lands on the\n    \"error: ...\" message on both Scala 2.12 and 2.13 (2.13 prints the\n    caret pointer before the message). Implemented with\n    `lines.indexWhere(_.trim !\u003d \"^\") ... lines.patch(idx, Nil, 1)`.\n  * New `isEffectivelyEmpty` helper strips block/line comments and\n    returns success for comment-only inputs (Scala 2.12 accepted them\n    silently; 2.13 rejects them as compile errors).\n- `repl/Session.scala`: SparkR `setJobGroup` match now accepts `\"4\"`\n  alongside `\"2\" | \"3\"`.\n- `server/interactive/InteractiveSession.scala`: `datanucleusJars` and\n  `mergeHiveSiteAndHiveDeps` now take an extra `scalaVersion`\n  parameter, adds `case 3 | 4` to the major-version match and replaces\n  the hard-coded `assembly/target/scala-2.12/jars` with\n  `assembly/target/scala-$scalaVersion/jars`.\n- `server/batch/BatchSession.scala`: pass `--verbose` to spark-submit\n  under `LivyConf.TEST_MODE` so tests can grep the resolved arguments\n  out of the child\u0027s log.\n- `rsc/driver/SparkEntries.java`: new reflection-based\n  `hiveClassesArePresent` helper that probes\n  `org.apache.spark.sql.classic.SparkSession$` (Spark 4) first and\n  falls back to `org.apache.spark.sql.SparkSession$` (Spark 3).\n- `utils/SparkProcessBuilder.scala`: new `verbose(v: Boolean)` mutator\n  and an internal `_verbose` flag that appends `--verbose` to\n  `spark-submit` when set (used by `BatchSession`).\n- `utils/LivySparkUtils.scala`: `(4, 0)` and `(4, 1)` -\u003e `\"2.13\"`\n  entries in `sparkScalaVersionMap`; `MAX_VERSION` bumped from\n  `(3, 6)` to `(4, 2)`.\n- `repl/SparkRInterpreter.scala`: log a warning on Spark 4 that\n  SparkR was deprecated in SPARK-49347 and may be removed in a future\n  Spark release.\n- `scala-api/scalaapi/package.scala`: inline comment documenting that\n  `Duration.isFinite` is a parameterless def in Scala 2.13 (took an\n  empty parameter list in 2.12); dropping the parens compiles on both.\n- Thriftserver Scala/json4s 4.x updates:\n  * `session/Get{Columns,Functions,Schemas,Tables}Job.java`:\n    `scala.collection.JavaConversions.seqAsJavaList` (removed in Scala\n    2.13) replaced with\n    `scala.collection.JavaConverters.seqAsJavaListConverter(...).asJava()`.\n  * `types/DataTypeUtils.scala`: json4s 4.x `parse(input, useBigDecimal)`\n    requires an `AsJsonInput[T]`; switch to the single-arg String\n    overload `parse(sparkJson)`.\n  * `LivyExecuteStatementOperation.scala`: explicit `res.toSeq` since\n    Scala 2.13 no longer widens a mutable `Buffer` to `Seq` implicitly.\n  * `ThriftServerSuites.scala`: `LIVY-571` assertion now also accepts\n    the Spark 4 error string `[SCHEMA_NOT_FOUND] The schema\n    `spark_catalog`.`invalid_database` cannot be found`.\n- `InteractiveSessionServlet.scala`: `logs.asJava` -\u003e `logs.toSeq.asJava`\n  so Scala 2.13 picks the `Seq[A]` -\u003e `java.util.List` `asJava`\n  overload instead of erroring on ambiguity.\n\n**Thriftserver libthrift pre-0.16 / 0.16 compatibility**\nThe Livy Thrift binary CLI service was written against libthrift 0.9.3\n(the version pinned in the root pom.xml). Spark 3 keeps a compatible\npre-0.16 libthrift on the runtime classpath (e.g. 0.12.0 in Spark 3.3),\nbut Spark 4 pulls in libthrift 0.16.0 transitively through spark-hive,\nand 0.16 removed four `TThreadPoolServer.Args` builder methods that\nLivy relied on: `requestTimeout`, `requestTimeoutUnit`,\n`beBackoffSlotLength` and `beBackoffSlotLengthUnit`. Compiling against\n0.9.3 but running against 0.16.0 made the mini-cluster Thrift server\nabort during startup with a `NoSuchMethodError` on\n`TThreadPoolServer$Args.requestTimeout(int)`, which in turn hung\nJdbcIT on the SASL handshake. `ThriftBinaryCLIService` now invokes\nthose four builders reflectively via a small `applyOptionalArg`\nhelper: they run unchanged on Spark 3 classpaths and are silently\nskipped on Spark 4, where they no longer exist. The functional loss\non Spark 4 is limited to the login-timeout / backoff knobs, which\nhave no equivalent in the newer libthrift API.\n\n**Test fix: Scala 2.13 REPL output shape changes in InteractiveIT**\nThe Scala 2.13 REPL prints results differently from Scala 2.12 in\nfour ways that `InteractiveIT` was too strict about; the default\nSpark 4 build (Scala 2.13 only) trips all four. Fixed in place:\n- Value results: `val res0: Int \u003d 2` (2.13) vs. `res0: Int \u003d 2`\n  (2.12). Six `verifyResult` regexes (across `basic interactive\n  session`, `user jars are properly imported ...` and `recover\n  interactive session`) now accept an optional `val ` prefix.\n- Compile errors: `error: not found: value abcde` (2.13) vs.\n  `\u003cconsole\u003e:12: error: not found: value abcde` (2.12). The\n  `\u003csource\u003e:\u003cline\u003e: ` location marker is now optional in the\n  `verifyError` regex.\n- Class definitions: `class Item` (2.13) vs. `defined class Item`\n  (2.12). The `case class Item(i: Int)` check in `user jars are\n  properly imported ...` now accepts an optional `defined ` prefix.\n- Deprecation-warning header: Scala 2.13 prints a\n  `warning: 1 deprecation ...` line above the value binding when the\n  expression uses a deprecated API. `SparkContext.parallelize` is\n  deprecated on Spark 4, so\n  `val rdd \u003d sc.parallelize(Array.fill(10){...})` prints the warning\n  before the `val rdd: ...` line. The `verifyResult` regex now uses\n  `(?s)(?:warning:.*\\n)?(?:val )?rdd.*` to accept the warning prefix\n  and the DOTALL semantics needed to span the newline.\nBoth forms match on the default Scala 2.13 build and `-Pscala-2.12`.\n\n**Test fix: Spark 4 SQLContext / DataFrame runtime class in InteractiveIT**\nSpark 4 moved the runtime `SQLContext` and `DataFrame` implementations\ninto `org.apache.spark.sql.classic`; only the compile-time aliases\nremain at `org.apache.spark.sql.SQLContext` / `org.apache.spark.sql.DataFrame`.\nThe Scala 2.13 REPL surfaces the runtime class name, so Spark 4\nprints `sql: org.apache.spark.sql.classic.SQLContext \u003d ...` and\n`val df: org.apache.spark.sql.classic.DataFrame \u003d ...`, where Spark 3\nprints the alias form. Two `InteractiveIT` regexes now allow an\noptional `classic.` package qualifier so the same expectations match\nboth the default Spark 4 build and `-Pspark3`:\n- `sql: org.apache.spark.sql.(?:classic\\.)?SQLContext \u003d ...`\n- `(?:val )?df: org.apache.spark.sql.(?:classic\\.)?DataFrame`\nThe old `Pattern.quote(\"df: org.apache.spark.sql.DataFrame\")` literal\nwas too strict for Spark 4 and caused the `basic interactive session`\ncase to fail.\n\n**Cross-version test helpers: `ScalaVersionAware` trait**\nNew `test-lib/.../ScalaVersionAware` trait exposes the small\nScala-2.12 vs 2.13 REPL-output differences as reusable fragments:\n- `optionalValPrefix` (`\"val \"` on 2.13, `\"\"` on 2.12) for exact-match\n  expected strings.\n- `optionalValPrefixRegex` / `optionalDefinedPrefixRegex` /\n  `optionalWarningPrefixRegex` regex fragments for the corresponding\n  `verifyResult` regex assertions.\nDetection uses the runtime `scala.util.Properties.versionNumberString`\nso the same test class works regardless of which Scala the artifact\nwas compiled against. Mixed into `BaseInterpreterSpec`,\n`BaseSessionSpec` and `InteractiveIT`. Downstream specs\n(`ScalaInterpreterSpec`, `SharedSessionSpec`, `SparkSessionSpec`,\n`InteractiveIT`) rewrote their expected-value strings against these\nfragments; some hard-`equal` assertions were relaxed to\n`include`/`fullyMatch regex` where the 2.13 REPL adds a deprecation\nbanner ahead of the result or renumbers `resN` slots differently.\n\n**Server/unit tests: cross-Scala tolerance in `InteractiveSessionSpec`**\nThe `should get scala version` case now decomposes the JSON result by\nhand and asserts `text/plain` equals either `res0: Int \u003d 3\\n` or\n`val res0: Int \u003d 3\\n`, plus separate `status` / `execution_count`\nassertions. Straight equality with a decomposed Map broke on the\nSpark 4 driver where the 2.13 REPL adds the `val ` prefix.\n\n**Unit tests: `LivySparkUtilsSuite`**\nAdded `4.0.0` and `4.1.2` to both the supported-version list and the\n`defaultSparkScalaVersion` -\u003e `\"2.13\"` expectations.\n\n**scala-api tests**\n- `ScalaClientTestUtils.scala`: `context.sc.parallelize(buffer, ...)`\n  -\u003e `context.sc.parallelize(buffer.toSeq, ...)` (2.13 no longer\n  widens a mutable `ArrayBuffer` to `Seq` implicitly for the\n  `parallelize` signature).\n- `ScalaJobHandleTest.scala`: `Duration.Undefined` -\u003e `Duration.Inf`\n  (2.13\u0027s `Await.ready` rejects `Undefined` with \"Cannot wait for\n  Undefined duration of time\"); assertion switched from\n  `verify(mockJobHandle, times(1)).get()` to\n  `verify(mockJobHandle, atLeastOnce()).isDone`, because 2.13\u0027s\n  `Await.ready` short-circuits when `isCompleted` is already true and\n  does not call the underlying `ready(atMost)` -- so `.get()` is not\n  observed on 2.13. `isDone` is exercised by both versions.\n\n**Repl tests: `SparkInterpreterSpec` move**\n`SparkInterpreterSpec.scala` moved from `repl/scala-2.12/src/test/...`\ninto the shared `repl/src/test/scala/...` tree (it was byte-identical\nbetween scala-2.12 and scala-2.13). A new `should skip leading caret\nlines in Scala 2.13 error format.` test case was added for the caret-\nfirst `parseError` branch, so the shared spec runs three cases under\nboth the default Scala 2.13 build and `-Pscala-2.12`.\n\n**CI matrix**\n- `.github/workflows/unit-tests.yaml` and `integration-tests.yaml`\n  converted the flat `maven_profile` x `jdk_path` matrix to an\n  explicit `include:` list. The default spark4 build (no extra Maven\n  profiles) runs on JDK 17; Spark 3 matrix entries use\n  `-Pscala-2.12 -Pspark3`.\n\n## How was this patch tested?\n\n- Unit tests: `mvn verify -Pthriftserver` passes all 19 modules on JDK\n  17 (macOS aarch64) with the default Spark 4 build. Verified modules\n  include livy-rsc (40 tests), livy-repl_2.13 (87 tests), livy-server\n  (205 tests), livy-client-http (17 tests), livy-scala-api_2.13\n  (17 tests), and the thriftserver integration suite (27 tests:\n  HttpThriftServerSuite + BinaryThriftServerSuite). The\n  `SparkInterpreterSpec` move added one new caret-first `parseError`\n  test case; the shared spec now has 3 cases and runs identically\n  under both the default Scala 2.13 build and `-Pscala-2.12`.\n- Integration tests (`mvn integration-test -Pspark3 -Pscala-2.12 -pl\n  :livy-integration-test`) pass.\n- GitHub Actions CI runs both spark3 and spark4 matrix entries for\n  unit and integration test workflows.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Opus 4.7)"
    },
    {
      "commit": "34549272ba3bbdab105445b8f6466604467124bc",
      "tree": "ce6e83f9df23e9df4eca329ed85494cf534fb3c2",
      "parents": [
        "ea09c8c065a434d23ec13d59e84d39ddfad3779b"
      ],
      "author": {
        "name": "Soumyadeep Roy",
        "email": "soumyadeeplogin@gmail.com",
        "time": "Wed Sep 09 03:24:33 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Sep 08 17:54:33 2026 -0400"
      },
      "message": "[LIVY-1072] Reduce Kubernetes API load in SparkKubernetesApp polling\n\n## What changes were proposed in this pull request?\n\n`SparkKubernetesApp.getApplicationReport` previously issued a single LIST returning every pod tagged with the session\u0027s app tag. On large jobs this deserializes hundreds of executor pods into memory on every poll cycle, even though application state is derived only from the driver pod.\n\nTwo changes:\n- Narrow the driver lookup to `spark-role\u003ddriver` at the API server so the LIST returns a single pod.\n- Gate the separate executor LIST behind a new config flag `livy.server.kubernetes.executor-tracking.enabled` (default `true`, so existing deployments see no behavior change). Operators running large clusters can disable it to skip the executor LIST entirely. When disabled, per-executor Grafana/Loki log URLs and per-executor entries in session diagnostics are omitted; driver state tracking is unaffected.\n\nAdds two tests: default-enabled invariant, and diagnostics graceful degradation when the executor list is empty.\n\n## How was this patch tested?\n\nExisting and newly added unit tests in `SparkKubernetesAppSpec`, plus manual validation against a live Kubernetes cluster running Spark jobs with hundreds of executors.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes. Generated-by: Claude Code (Sonnet 5, Anthropic), used to scan/plan the extraction of local UIP patches for upstreaming and to draft/iterate on the code and tests in this PR under human review. Please refer to the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) for details.\n\n## Note on JIRA\n\nFiled as [LIVY-1072](https://issues.apache.org/jira/browse/LIVY-1072)."
    },
    {
      "commit": "ea09c8c065a434d23ec13d59e84d39ddfad3779b",
      "tree": "740f53b579d594bd050dfec80516ea96ce134e66",
      "parents": [
        "9baff1f5b355fd831ae0809bc3821aa7da8eabe8"
      ],
      "author": {
        "name": "Soumyadeep Roy",
        "email": "soumyadeeplogin@gmail.com",
        "time": "Wed Sep 09 02:51:54 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Sep 08 17:21:54 2026 -0400"
      },
      "message": "[LIVY-1071] Add driver.address.use-hostname RSC config flag\n\n## What changes were proposed in this pull request?\n\nAdds an opt-in `RSCConf` key, `driver.address.use-hostname` (default `false`), that makes `ContextLauncher` resolve the RSC driver\u0027s address from the hostname reported in `RemoteDriverAddress` instead of the socket\u0027s IP. Existing IP-based behavior is unchanged when the flag is left at its default.\n\nThis is useful for Kubernetes + service-mesh (Istio) deployments where pod IPs are not stable/routable but hostnames are.\n\nOriginally authored by Venugopal Allenki, extracted and rebased onto current `master` for upstreaming.\n\n## How was this patch tested?\n\nManually verified in a Kubernetes + Istio deployment with the flag enabled and disabled.\n\nAdded `TestContextLauncher` with unit tests covering both resolution paths (default socket-IP path, and hostname path when the flag is enabled) against a new `ContextLauncher.resolveDriverAddress` method extracted from `RegistrationHandler.handle()` for direct testability, since the handler itself is a private inner class only exercisable via a full RPC handshake. Also documented the new flag in `conf/livy.conf.template`.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes. Generated-by: Claude Code (Sonnet 5, Anthropic), used to scan/plan the extraction of local UIP patches for upstreaming and to draft/iterate on the code and tests in this PR under human review. Please refer to the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) for details.\n\n## Note on JIRA\n\nFiled as [LIVY-1071](https://issues.apache.org/jira/browse/LIVY-1071)."
    },
    {
      "commit": "9baff1f5b355fd831ae0809bc3821aa7da8eabe8",
      "tree": "ca8b45902b0327316ff221fe5dcb7b59ced0657b",
      "parents": [
        "e958c4556e8c05616371016a67a12471aca53b09"
      ],
      "author": {
        "name": "Soumyadeep Roy",
        "email": "soumyadeeplogin@gmail.com",
        "time": "Wed Sep 09 01:32:00 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Sep 08 16:02:00 2026 -0400"
      },
      "message": "[LIVY-1077] Bump jetty.version to latest 9.4.x (9.4.58.v20250814)\n\n## What changes were proposed in this pull request?\n\nBumps `jetty.version` in `pom.xml` from `9.4.56.v20240826` to `9.4.58.v20250814` -- the latest version actually published on the Jetty 9.4.x branch.\n\nThe current `jetty-http`/`jetty-server`/`jetty-security` version is affected by **CVE-2026-2332** / [GHSA-355h-qmc2-wpwf](https://github.com/jetty/jetty.project/security/advisories/GHSA-355h-qmc2-wpwf) -- an HTTP/1.1 request smuggling issue via malformed chunk-extension quoted-string parsing in Jetty\u0027s chunked transfer-encoding parser (CVSS 7.4 High, CWE-444).\n\nChecked against the advisory (OSV) and Maven Central directly: the actual fix for this CVE on the 9.4.x branch is version `9.4.60`, but `9.4.59`/`9.4.60` have never been published to Maven Central -- the upstream advisory itself still reads \"No patches yet\" for that branch as of this writing. The only published fixes are on Jetty 10.x/11.x/12.x, all of which require Java 11+ (12.x requires Java 17+ and moves `javax.servlet` -\u003e `jakarta.servlet`) -- a real migration, not a version-property bump, and out of scope for this PR.\n\nThis bumps to the latest available 9.4.x release as defense-in-depth, picking up every other 9.4.x security fix since 9.4.56. **CVE-2026-2332 itself remains open** against `jetty-http` pending an upstream Jetty 9.4.x patch release, or a separately-scoped move to Jetty 10+/Java 11+.\n\n## How was this patch tested?\n\n`mvn dependency:tree` resolves cleanly to `9.4.58.v20250814` across every module that pulls in `jetty.version` (`server`, `assembly`, `client-http`, `thriftserver`), with no dependency conflicts. `mvn compile` on those modules succeeds with no errors. This is a version-property change only, no code paths are touched, so no new unit tests are needed.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes. Generated-by: Claude Code (Sonnet 5, Anthropic), used to research the CVE against Maven Central/OSV/GHSA data, draft this change, and file the companion JIRA ticket, under human review. Please refer to the [ASF Generative Tooling Guidance](https://www.apache.org/legal/generative-tooling.html) for details.\n\n## Note on JIRA\n\nFiled as [LIVY-1077](https://issues.apache.org/jira/browse/LIVY-1077)."
    },
    {
      "commit": "e958c4556e8c05616371016a67a12471aca53b09",
      "tree": "06c4a465f1b9e7bd266ab0551c5e14bda618aa88",
      "parents": [
        "92ccea357d8ea2f8d8ecdcf8408b9de129e88adf"
      ],
      "author": {
        "name": "nileshrathi345",
        "email": "113672051+nileshrathi345@users.noreply.github.com",
        "time": "Fri Aug 28 19:00:40 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Fri Aug 28 09:30:40 2026 -0400"
      },
      "message": "[LIVY-1076] Python 3.12 to 3.14 compatibility fixes\n\n## What changes were proposed in this pull request?\n\nThis PR resolves [LIVY-1076](https://issues.apache.org/jira/browse/LIVY-1076) by fixing Python 3.12+ compatibility issues in `livy-python-api`.\n\nChanges:\n- **client.py**: Replace deprecated `ConfigParser.readfp()` with `read_file()` (removed in Python 3.12).\n- **job_handle.py**: Replace deprecated `notifyAll()` with `notify_all()`.\n- **job_context.py**: Replace removed `abc.abstractproperty` with `@property` + `@abstractmethod` on all abstract properties (required for Python 3.12+).\n- **setup.py**: Add PyPI classifiers for Python 3.8 through 3.14 to reflect supported versions.\n\nThese changes address the deprecation warnings reported in the Jira ticket and ensure the Python client library runs cleanly through Python 3.14, aligning with Spark 4.1’s supported Python range.\n\nJira: https://issues.apache.org/jira/browse/LIVY-1076\n\n## How was this patch tested?\n\n- Ran livy-python-api unit tests locally:\n `python -m pytest`\n- Verified 12 tests passed with no deprecation warnings on Python 3.12.8:\n`\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d 12 passed in 0.83s \u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d\u003d`\n- Also verified on Python 3.8.19, 3.9.18, and 3.14.0 — all 12 tests passed.\n- Ran stricter check with deprecations treated as errors:\n`python -m pytest -W error::DeprecationWarning`\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes. Cursor AI was used to run the local test verification."
    },
    {
      "commit": "92ccea357d8ea2f8d8ecdcf8408b9de129e88adf",
      "tree": "b192e9fefb73f55c9f35666e95395811ca30f9f3",
      "parents": [
        "f7f7d408a1d3dcd389783694f0c7ee6056e31886"
      ],
      "author": {
        "name": "samuhasavak5",
        "email": "samuhasavak5@gmail.com",
        "time": "Tue Aug 25 20:25:39 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Aug 25 10:55:39 2026 -0400"
      },
      "message": "[LIVY-1070] Add LivySessionMetrics Codahale gauges for session monitoring\n\n## What changes were proposed in this pull request?\n\nThis PR adds server-side session monitoring gauges for LIVY-1070.\n\nProblem: Livy exposes session state via the REST API (/sessions, /batches) but does not publish session counts as Codahale metrics on the existing /metrics endpoint. External monitoring systems must poll the REST API to observe session distribution.\n\nSolution: Introduce LivySessionMetrics, which registers session count gauges into Livy\u0027s MetricRegistry at server startup. Gauges are derived from InteractiveSessionManager and BatchSessionManager and exposed alongside existing Livy metrics via the AdminServlet at /metrics.\n\nMetrics registered (18 gauges)\n\nOverall (3)\n\n    livy.sessions.total\n    livy.sessions.active.total\n    livy.sessions.terminal.total\n\nInteractive (8)\n\n    livy.sessions.interactive.total\n    livy.sessions.interactive.{idle,busy,starting,shutting_down,dead,error,killed}\n\nBatch (7)\n\n    livy.sessions.batch.total\n    livy.sessions.batch.{starting,running,success,dead,error,killed}\n\nDesign notes\n\n    Additive only — no REST API or session lifecycle behavior changes\n    Idempotent registration — skips gauge names already present in the registry\n    Error-safe callbacks — gauge getValue returns 0 on exception\n    State normalization — handles case and hyphen/underscore variants (e.g. shuttingdown, succeeded)\n    HA note — gauge values reflect the local Livy server instance; in HA deployments only the leader holds active sessions\n\nCompatibility\n\n    No new endpoints; uses existing /metrics AdminServlet\n    Backward compatible — new gauges appear alongside existing metrics\n\n## How was this patch tested?\n\nBuild\nmvn package -Pspark3 -Pscala-2.12 -pl server -am \\\n-s /tmp/livy-mvn-central-settings.xml -DskipTests\n\nResult: BUILD SUCCESS\nUnit tests\nmvn test -Pspark3 -Pscala-2.12 -pl server \\\n-s /tmp/livy-mvn-central-settings.xml \\\n-Dsuites\u003dorg.apache.livy.server.LivySessionMetricsSpec\n\nResult: 8/8 tests passed\n\nTest | Coverage -- | -- All 18 gauge registrations | Registration Duplicate registration guard | Idempotency Zero sessions | Empty state Interactive sessions by state | State counting Batch sessions by state (incl. succeeded alias) | State counting Batch succeeded alias | Edge case Overall totals (total, active, terminal) | Aggregation Exception fallback returns 0 | Error handling\nCode coverage\n\nJaCoCo agent enabled during test run (server/target/jacoco/main.exec generated).\n\nNo UI changes in this PR.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes, this was co-authored using Cursor to help generate the new test cases."
    },
    {
      "commit": "f7f7d408a1d3dcd389783694f0c7ee6056e31886",
      "tree": "ecb56923b8c053303a998f7ebb5d15eb8596cdbb",
      "parents": [
        "e3101e8c3eb20b05361f134b718ce10a65da6359"
      ],
      "author": {
        "name": "nileshrathi345",
        "email": "113672051+nileshrathi345@users.noreply.github.com",
        "time": "Tue Aug 25 20:01:41 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Aug 25 10:31:41 2026 -0400"
      },
      "message": "[LIVY-979] Create a helm-based integration environment for debugging Livy on Kubernetes\n\n## What changes were proposed in this pull request?\n\nAdds a Helm chart under dev/helmchart/ for running Livy on a local Kubernetes cluster (e.g. Docker Desktop) for development and debugging, without cloud dependencies.\n\n**Included:**\n\n- Helm chart for Livy (StatefulSet), Spark History Server, RBAC, Services, PVC/StorageClass, and optional ingress + TLS\n- Optional subcharts: ingress-nginx and cert-manager (gated via values)\n- `values.yaml`aligned with Livy 1.0.0-SNAPSHOT, Spark 3.5.6, Scala 2.12\n- Docs: `README.md` (install, smoke tests, optional Grafana/Loki), `Docker.md` (image build steps)\n- `test/validate.sh` — ASF license header checks, version consistency, helm lint / helm template\n- ASF license headers on chart source files; `.rat-excludes` updated only for generated `Chart.lock`\n\n**Template fixes found during local testing:**\n\n- Ingress `apiVersion` rendering (trim directive)\n- cert-manager / ClusterIssuer / Certificate resources only when cert-manager is enabled\n- Spark History Server startup fix (`HADOOP_USER_NAME\u003droot` for local Hadoop user resolution)\n\nThis PR is **dev/ops tooling only** — no changes to Livy server runtime code.\n\nJIRA: [LIVY-979](https://issues.apache.org/jira/browse/LIVY-979)\n\n## How was this patch tested?\n\n**Automated / local chart checks:**\n\n- `dev/helmchart/test/validate.sh `— license headers, version keys, helm dependency build, helm lint, helm template\n\n**Manual testing (Docker Desktop Kubernetes, namespace livy-dev)**\n\n- Built local images per `Docker.md` (`livy-local/spark-py:v3.5.6`, `livy-local/livy:spark3.5.6`)\n- Deployed with `helm install `/ `helm upgrade`\n- Verified pods: `livycluster-0`, ingress-nginx, cert-manager, `spark-history-0`\n- **REST via ingress/TLS**: `GET /livy/version` → 200\n- **Interactive session**: created pyspark session, ran `1+1`, `sc.parallelize(...).count()`, `spark.range(5).collect()` — all succeeded\n- Spark driver and executor pods reached Running on Kubernetes\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes, co-authored by Cursor AI\n\nNote: This PR builds on the original Helm chart work contributed by @askhatri\n\nCo-authored-by: Asif Khatri \u003c123077165+askhatri@users.noreply.github.com\u003e"
    },
    {
      "commit": "e3101e8c3eb20b05361f134b718ce10a65da6359",
      "tree": "a56a6cb48cf26bc997df9fd0ea52c776a9cf085c",
      "parents": [
        "1dcb2fd5430ee3647cc4e83508b15db3817adf40"
      ],
      "author": {
        "name": "nileshrathi345",
        "email": "113672051+nileshrathi345@users.noreply.github.com",
        "time": "Tue Aug 25 02:20:00 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Aug 24 16:50:00 2026 -0400"
      },
      "message": "[LIVY-1063] Fix resource leak in Kubernetes service account token refresh\n\n## What changes were proposed in this pull request?\n\nReuse the existing `KubernetesClient` during service account token refresh instead of\nrecreating `DefaultKubernetesClient` every 5 minutes without closing the old client.\n\nJIRA: https://issues.apache.org/jira/browse/LIVY-1063\n\n## How was this patch tested?\n\n- Existing unit tests: `mvn -pl server -am -DskipITs test`\n- Manual review: refresh loop only calls `config.setOauthToken()`; no new client creation\n- Manual testing: deployed on Kubernetes with in-cluster SA auth;\n  verified \"Refreshed Kubernetes service account token.\" logs every ~5 minutes and\n  native/HTTP client thread count remains stable over several refresh cycles\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes, used Cursor AI to write down the test case."
    },
    {
      "commit": "1dcb2fd5430ee3647cc4e83508b15db3817adf40",
      "tree": "5a22e11dddd06c153d5e6e69ff258e74f4c6dc50",
      "parents": [
        "501e8226cbb36d6ae00d00bb208e778c9328ff72"
      ],
      "author": {
        "name": "Gyorgy Gal",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Mon Aug 24 20:19:06 2026 +0000"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Aug 24 16:19:06 2026 -0400"
      },
      "message": "[LIVY-1075] Migrate from pytest-runner to direct pytest invocation\n\n## What changes were proposed in this pull request?\n\nsetuptools \u003e\u003d 58 removed the tests_require keyword and pytest-runner\u0027s python setup.py test hook. Replace with direct pytest invocation:\n\n- setup.py: remove tests_require, setup_requires, the livy-tests package declaration, and the dead Python 2.7 futures extras_require shim\n- pom.xml: replace `python setup.py test` with `python -m pytest`\n- setup.cfg: remove defunct [aliases] block; add pythonpath \u003d src/main/python so pytest can resolve the livy package without a prior install step\n- Dockerfile: remove pytest-runner from pip install list\n\n## How was this patch tested?\n\nTested in a forked repository.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code"
    },
    {
      "commit": "501e8226cbb36d6ae00d00bb208e778c9328ff72",
      "tree": "7cd3d339f966e518abbfe4b90e6c278586c52f41",
      "parents": [
        "3d2a94d7507dc8dab0bd8c8b2da42a9903764346"
      ],
      "author": {
        "name": "Gyorgy Gal",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Mon Aug 24 14:16:46 2026 +0000"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Aug 24 10:16:46 2026 -0400"
      },
      "message": "[LIVY-1074] Update build-ci-image workflow, Dockerfile\n\nThe *build-ci-image* GitHub workflow has stopped working because\nexternal actions can not rely on tags anymore, using tags prevents the\nGitHub action from launching. This change fixes the broken workflow by\nreferencing commit hashes instead of tags.\n\nAlso included a small change to the Dockerfile to trigger the workflow\nonce merged.\n\nMany thanks for the help with the investigation to Arnav Balyan.\n\nCo-authored-by: Arnav Balyan \u003c60175178+ArnavBalyan@users.noreply.github.com\u003e"
    },
    {
      "commit": "3d2a94d7507dc8dab0bd8c8b2da42a9903764346",
      "tree": "7b080263f135f94e9795604f3a45e1ab37c0d4e3",
      "parents": [
        "d175722d1a24631499db99786940b3fe1f93cce4"
      ],
      "author": {
        "name": "Gabor Roczei",
        "email": "1918366+roczei@users.noreply.github.com",
        "time": "Thu Aug 20 17:06:07 2026 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Thu Aug 20 11:06:07 2026 -0400"
      },
      "message": "[LIVY-1067] Bump CI Python to 3.11.11 and fix python-api PEP 440 version\n\n## What changes were proposed in this pull request?\n\nPrepares the CI environment and the python-api package metadata for the\nupcoming Spark 4 support commit. Both changes are also useful on the\nexisting Spark 3 build: the PEP 440 fix unblocks `pip3 install\nlivy-python-api` on modern pip (\u003e\u003d 24), and the Python bump keeps the\nCI image on a version supported by both Spark 3.5 and Spark 4.1.\n\n**CI Python bumped to 3.11.11**\ndev/docker/livy-dev-base/Dockerfile pyenv Python bumped from 3.9.21 to\n3.11.11, and .github/workflows/integration-tests.yaml pins\n`pyenv global 3.11.11` explicitly. Rationale (per each Spark release\u0027s\npython/setup.py): Spark 4.1 declares `python_requires\u003d\"\u003e\u003d3.10\"` with\nclassifiers listing 3.10/3.11/3.12/3.13/3.14, while Spark 3.5 declares\n`python_requires\u003d\"\u003e\u003d3.8\"` with classifiers listing 3.8/3.9/3.10/3.11.\nThe intersection of the two supported ranges is 3.10 and 3.11; we pick\nthe higher one (3.11.11) so the same image serves both matrix profiles.\nThe old 3.9.21 was below the Spark 4.1 lower bound. Both the Dockerfile\nand the workflow now document this choice inline so future bumps stay\nin sync.\n\nThe CI Docker image (`ghcr.io/${owner}/livy-ci:latest`) had to be\nrebuilt and pushed manually for the PR CI to pick up the new Python\nbefore this PR is merged: `.github/workflows/build-ci-image.yaml` only\nfires on push to master, so branch/PR runs would otherwise still pull\nthe stale image cached from the previous Dockerfile. Commands used\n(from macOS Apple Silicon, cross-built for linux/amd64 to match the\nGitHub Actions runners):\n\n    gh auth refresh --scopes write:packages,read:packages\n    gh auth token | docker login ghcr.io -u \u003cowner\u003e --password-stdin\n    docker buildx build --platform linux/amd64 \\\n      -t ghcr.io/\u003cowner\u003e/livy-ci:latest --push \\\n      dev/docker/livy-dev-base\n\nAfter this PR merges to master, the workflow will republish the image\non any future Dockerfile change automatically, so this manual step is\nonly needed for the bootstrap run.\n\n**python-api/setup.py PEP 440 version fix**\nVersion bumped from `1.0.0-SNAPSHOT` (Maven-style, not PEP 440\ncompliant) to `1.0.0.dev0` (the canonical Python \"pre-release under\nactive development\" form). Modern pip (\u003e\u003d 24) refuses to parse the old\nvalue with `Invalid version: \u00271.0.0-SNAPSHOT\u0027`, which surfaced as\n`WARNING: Error parsing dependencies of livy-python-api` on every\n`pip3 install` and blocked the `validate Python-API requests`\nintegration test from resolving its dependencies. The Maven POM\nversion (`python-api/pom.xml`) stays `1.0.0-SNAPSHOT` -- Maven and pip\nhave separate versioning conventions and only the pip-visible metadata\nneeds to change.\n\n## How was this patch tested?\n\n- `pip3 install ./python-api` no longer emits\n  `WARNING: Error parsing dependencies of livy-python-api: Invalid\n  version: \u00271.0.0-SNAPSHOT\u0027`; the package installs cleanly on pip 24+.\n- Rebuilt the CI Docker image locally with the pyenv 3.11.11 bump and\n  confirmed `python3 --version` reports `Python 3.11.11` inside the\n  container.\n- GitHub Actions Integration Tests workflow runs `pyenv global 3.11.11`\n  successfully on the -Pspark3 -Pscala-2.12 matrix entry (this commit\n  does not yet add any Spark 4 entry).\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Opus 4.7)"
    },
    {
      "commit": "d175722d1a24631499db99786940b3fe1f93cce4",
      "tree": "423efd444f7355a4b8a7c3b4edafa73102e60fd4",
      "parents": [
        "4b3d90f8ded807f2789c5a8f1645c616ca0d83e2"
      ],
      "author": {
        "name": "Gabor Roczei",
        "email": "1918366+roczei@users.noreply.github.com",
        "time": "Thu Aug 20 17:03:00 2026 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Thu Aug 20 11:03:00 2026 -0400"
      },
      "message": "[LIVY-1066] Upgrade scalatest to 3.2.9 and scalatra to 2.8.4\n\n## What changes were proposed in this pull request?\n\nUpgrade scalatest 3.0.8 -\u003e 3.2.9 and scalatra 2.6.5 -\u003e 2.8.4. Both\nupgrades are prerequisites for Spark 4 / Scala 2.13 support (parent\nJIRA LIVY-1041) and are split into a dedicated commit to keep test\nmigrations separate from core Spark 4 source changes for easier review.\n\nScalatest 3.0.8 -\u003e 3.2.9:\n- Migrate FunSuite/FunSpec/FunSpecLike/FlatSpec test classes to their\n  3.2 successors AnyFunSuite/AnyFunSpec/AnyFunSpecLike/AnyFlatSpec.\n- Move `org.scalatest.Matchers` to\n  `org.scalatest.matchers.should.Matchers`.\n- Add `org.scalatestplus:mockito-3-4_${scala.binary.version}:3.2.9.0`,\n  since scalatest 3.2 moved `MockitoSugar.mock` into a separate\n  scalatestplus artifact.\n\nScalatra 2.6.5 -\u003e 2.8.4:\n- Bump `metrics.version` 3.1.0 -\u003e 4.2.19: scalatra 2.8.x\u0027s\n  metrics-servlets pulls in Dropwizard metrics 4.x, and keeping\n  metrics-core / metrics-healthchecks at 3.1.0 causes a\n  NoClassDefFoundError for HealthCheckFilter at runtime.\n\n## How was this patch tested?\n\n- Unit tests: `mvn verify -Pspark3 -Pscala-2.12 -Pthriftserver` passes\n  on JDK 8/17 for all modules with the migrated ScalaTest 3.2 test\n  suites (matching what was verified as part of the parent LIVY-1041\n  branch).\n- The metrics 4.2.19 bump is tested by the integration test suite\n  (`mvn integration-test -Pspark3 -Pscala-2.12 -pl\n  :livy-integration-test`), which previously failed with\n  `NoClassDefFoundError: com/codahale/metrics/servlets/HealthCheckFilter`\n  on MiniCluster startup and now passes.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Opus 4.7)"
    },
    {
      "commit": "4b3d90f8ded807f2789c5a8f1645c616ca0d83e2",
      "tree": "9dbc371e906a6e261f6757ad997443374ff762cf",
      "parents": [
        "d0821740848c2b2d3f9bdf084b3b457665a25984"
      ],
      "author": {
        "name": "Gabor Roczei",
        "email": "1918366+roczei@users.noreply.github.com",
        "time": "Tue Aug 18 23:48:11 2026 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Aug 18 17:48:11 2026 -0400"
      },
      "message": "[LIVY-1065] Ensure all tests pass on macOS\n\n## What changes were proposed in this pull request?\n\nThe Livy unit-test and integration-test suites do not pass on macOS\ntoday. The root cause is that several Livy and Spark services bind\nto the LAN IP returned by InetAddress.getLocalHost, but on macOS\nthe canonical hostname routinely resolves to a loopback-only\naddress (or the LAN IP is otherwise unreachable from a peer process\non the same box). As a result the mini-cluster YARN containers and\nJDBC clients on the same host cannot reach the server side, and\ntests time out with \"RSCClient instance stopped\" or hang on\n\"Initial job has not accepted any resources\".\n\nThe changes below are the minimum set required to turn every Livy\ntest suite green on macOS developer machine -- both the\nunit-test and integration-test runs, on the -Pspark3 profile. Linux\nCI hosts already resolve the primary hostname to a routable\naddress, so 127.0.0.1 is equally valid there.\n\nTo avoid hardcoding \"127.0.0.1\" at every call site, a shared\nTestUtils.TEST_BIND_HOST constant is introduced in client-common\n(a @Private Java helper class already on the compile classpath of\nevery affected module) and referenced from every test-scope bind,\nso future changes can flip a single location.\n\nWhat changed (each item is required for at least one macOS test\nsuite to go green; together they take the whole build to zero\nfailures on macOS):\n\n- client-common/TestUtils.java: add the TEST_BIND_HOST \u003d \"127.0.0.1\"\n  constant with a Javadoc pointing at LIVY-1065.\n\n- integration-test MiniCluster.scala, MiniLivyMain: set\n  livy.server.host, livy.server.thrift.bind.host and\n  livy.rsc.rpc.server.address to TEST_BIND_HOST. The RSC key is\n  referenced through the typed RSCConf.Entry.RPC_SERVER_ADDRESS.key()\n  rather than the raw string, matching the rest of the patch.\n  Without these the RSC driver inside the YARN AM container cannot\n  reach the Livy server on macOS and JdbcIT / InteractiveIT fail\n  with \"RSCClient instance stopped\".\n\n- integration-test MiniCluster.scala, MiniCluster.deploy(): set\n  spark.driver.host, spark.yarn.appMasterEnv.SPARK_LOCAL_IP and\n  spark.executorEnv.SPARK_LOCAL_IP to TEST_BIND_HOST in the\n  generated spark-defaults.conf, and set SPARK_LOCAL_IP\u003dTEST_BIND_HOST\n  in the child process environment. SPARK_LOCAL_IP is what the YARN\n  AM driver actually reads on macOS; spark.driver.host on the\n  submit side alone is not honoured in cluster deploy mode.\n  Without these the executor never registers with the driver on\n  macOS and JdbcIT hangs on \"Initial job has not accepted any\n  resources\".\n\n- integration-test MiniCluster.scala, MiniCluster.deploy(): also\n  propagate the host shell PATH into the YARN AM and executor\n  containers (spark.yarn.appMasterEnv.PATH /\n  spark.executorEnv.PATH), so RRunner and PythonRunner can resolve\n  Rscript / python on macOS -- YARN\u0027s default sanitised PATH omits\n  /opt/homebrew/bin, which is where Homebrew installs the R and\n  Python interpreters that macOS developers use, and without it\n  RRunner and PythonRunner fail with `error\u003d2, No such file or\n  directory` in the SparkR / PySpark integration tests.\n\n- Unit-test fixtures: set RSCConf.Entry.RPC_SERVER_ADDRESS to\n  TEST_BIND_HOST in every fixture that spins up an in-process RSC\n  server -- BaseSessionSpec, ReplDriverSuite, TestSparkClient,\n  TestRpc, ScalaClientTest, BaseInteractiveServletSpec,\n  InteractiveSessionSpec, ThriftServerBaseTest, ThriftSessionTest.\n  Each of these RSC-hosting unit-test classes hangs or fails on\n  macOS without the loopback pin (the child driver JVM times out\n  contacting the server on the machine\u0027s routable IP).\n\n- client-http HttpClientSpec.scala, LivyConnectionSpec.scala: change\n  the WebServer bind host from \"0.0.0.0\" / InetAddress.getLocalHost\n  to TEST_BIND_HOST, and switch the URIs the tests build to use\n  server.host so the client and server agree. Drop the now-unused\n  InetAddress import. Fixes the two HTTP-client suites\u0027 timeouts\n  on macOS.\n\n- thriftserver ThriftServerBaseTest.scala: use the typed\n  RSCConf.Entry.RPC_SERVER_ADDRESS.key() rather than the raw\n  string, matching every other Scala test file in the patch.\n\n## How was this patch tested?\n\n- `mvn -Pthriftserver -Pspark3 -Pscala-2.12 -B verify` locally on\n  macOS: all unit-test suites (including livy-repl_2.12,\n  livy-server, livy-thriftserver, livy-thriftserver-session,\n  livy-client-http, livy-scala-api_2.12) pass with these changes;\n  without them the same suites fail on macOS with \"RSCClient\n  instance stopped\" / connect timeouts to the LAN IP.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Code (Opus 4.7)"
    },
    {
      "commit": "d0821740848c2b2d3f9bdf084b3b457665a25984",
      "tree": "8157fc47babaea2c47c83d9b9ce0da7ae16a4074",
      "parents": [
        "4ef810c7fc0c2af2dcb58bfc96c91d55bc0d59d7"
      ],
      "author": {
        "name": "idzikovsky",
        "email": "23353239+idzikovsky@users.noreply.github.com",
        "time": "Thu Aug 06 04:38:26 2026 +0300"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Aug 05 21:38:26 2026 -0400"
      },
      "message": "[LIVY-1064] Initial changes to provide Scala 2.13 compatibility\n\n## What changes were proposed in this pull request?\n\nI\u0027ve stared working on Spark 4 compatibility in my local fork. I don\u0027t think that all changes that I\u0027ve made is applicable to be pushed to the upstream project, but this change I\u0027m confident about.\n\nIn this fix I mainly updated usage of collections to be compatible with new redesigned Scala 2.13 collections (so as with older Scala 2.11/2.12 collections).\n\nAlso, I replaced invocation of `ServletContext.initParameters()` with `ServletContext.setInitParameter()`, because the old wrapper is not compatible with a new `jakarta.servlet.ServletContext` interface (Spark 4 migrated from javax to jakarta), while `ServletContext.setInitParameter()` is version-independent.\n\n## How was this patch tested?\n\nI run unit tests and manually tested that I\u0027m able to create a Session.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nGenerated-by: Claude Opus 4.8."
    },
    {
      "commit": "4ef810c7fc0c2af2dcb58bfc96c91d55bc0d59d7",
      "tree": "391c515e303e9fc9a529a4ede1543dda7344ba11",
      "parents": [
        "a676d999c7101ef4bb12c863536ff244e10cf420"
      ],
      "author": {
        "name": "nileshrathi345",
        "email": "113672051+nileshrathi345@users.noreply.github.com",
        "time": "Tue Aug 04 20:38:03 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Aug 04 11:08:03 2026 -0400"
      },
      "message": "[LIVY-866] Optimize Yarn GetApplications Query to prevent additional load on Yarn and Livy\n\n## What changes were proposed in this pull request?\n\n**Problem**\nLivy was calling YarnClient.getApplications(SPARK), which asks the ResourceManager for all Spark applications and then filters client-side by application tag. On clusters with many Spark apps, this produces large RM responses, adds unnecessary load on YARN, and can contribute to RM memory pressure (especially during session startup polling and leaked-app GC).\n\n**Solution**\nUse a tag-filtered GetApplicationsRequest so the RM returns only matching applications:\n- **getAppIdFromTag()** — when resolving a session’s YARN app ID by tag, query with the lowercase session tag instead of scanning all Spark apps.\n- **leakedAppsGCThread** — when cleaning up leaked apps, batch all pending leaked tags into one filtered request per GC cycle instead of fetching the full Spark app list.\n\nJira: https://issues.apache.org/jira/browse/LIVY-866\n\n## How was this patch tested?\n\nUpdated and ran SparkYarnAppSpec, including: \n- \"should get App Id\" — verifies tag-based app ID lookup uses the filtered getApplications request.\n- \"should delete leak app when timeout\" — verifies the leaked-app GC path uses the filtered request.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes, this was co-authored using Cursor to help understanding of  solution for **leakedAppsGCThread**"
    },
    {
      "commit": "a676d999c7101ef4bb12c863536ff244e10cf420",
      "tree": "d0241b9548f768a79ae38f7b056db2d315a0f4a0",
      "parents": [
        "9e18a86524cdc7724946ba169f1769997153d3b8"
      ],
      "author": {
        "name": "nileshrathi345",
        "email": "113672051+nileshrathi345@users.noreply.github.com",
        "time": "Tue Aug 04 19:37:31 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Aug 04 10:07:31 2026 -0400"
      },
      "message": "[LIVY-1060] Livy Spark sessions fail to connect to Hive Metastore over SSL with keystore password error\n\n## What changes were proposed in this pull request?\n\nResolves Hive Metastore SSL credential handling for Spark sessions/jobs submitted by Livy.\n\n- Added `enrichHiveMetastoreSslConf` in `SparkApp.prepareSparkConf` to read `hive.metastore.keystore.password` and `hive.metastore.truststore.password` from Livy’s local credential provider (`livy.hadoop.security.credential.provider.path`) and inject them as `spark.hadoop.*` properties for the Spark app.\n- Ensures YARN/K8s driver and executors get explicit HMS SSL passwords without needing access to the Livy host’s local JCEKS file.\n- Preserves the existing credential provider path in Spark conf (fix for interactive session creation regression from blanking the path).\n\nJIRA: https://issues.apache.org/jira/browse/LIVY-1060\n\n## How was this patch tested?\n\n- Added `SparkAppSpec` unit tests for no-op, successful HMS SSL resolution, unrelated credentials, and unreadable provider path cases.\n- Verified credential resolution from a local JCEKS test store and no-op paths when provider is missing, unreadable, or has no HMS aliases.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nNo"
    },
    {
      "commit": "9e18a86524cdc7724946ba169f1769997153d3b8",
      "tree": "5f47ebdb04222123069199cd5e72cea64602ff6c",
      "parents": [
        "228b889750caf12529ea90fab1ca98f075fcc8d3"
      ],
      "author": {
        "name": "samuhasavak5",
        "email": "samuhasavak5@gmail.com",
        "time": "Tue Jul 28 21:06:28 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Jul 28 11:36:28 2026 -0400"
      },
      "message": "[LIVY-1061] Add livy.server.app-name-prefix for interactive session YARN app names\n\n## What changes were proposed in this pull request?\n\nThis PR adds a new Livy server configuration property, livy.server.app-name-prefix, to allow cluster administrators to customize the default YARN application name for interactive Spark sessions.\n\nProblem: When a client creates an interactive session via POST /sessions without specifying a name, Livy sets spark.app.name to the hardcoded value livy-session-\u003cid\u003e. In HA or multi-cluster environments, this makes it difficult to distinguish Livy-managed applications in the YARN Resource Manager UI.\n\nSolution:\n\nAdd LivyConf.SERVER_APP_NAME_PREFIX (livy.server.app-name-prefix, default: livy-session)\nUse the configured prefix when building the default spark.app.name in InteractiveSession.create()\nDocument the new property in conf/livy.conf.template\nAdd unit tests for default and custom prefix behavior\nScope: Interactive sessions only. Batch sessions already support per-request naming via the REST API name field and are unchanged.\n\nBackward compatibility: When the property is unset, behavior remains livy-session-\u003cid\u003e.\n\nExample:\n**livy.conf**\n```\nlivy.server.app-name-prefix \u003d my-cluster\n```\nCreating a session without a name yields YARN application name: my-cluster-3\n\nJIRA: [LIVY-1061](https://issues.apache.org/jira/browse/LIVY-1061)\n\n## How was this patch tested?\n\n- Unit tests: Added tests in InteractiveSessionSpec for:\n  -Default prefix (livy-session) when config is not set\n  -Custom prefix (e.g. my-cluster) when livy.server.app-name-prefix is configured\n- Build: mvn package -Pspark3 -Pscala-2.12 — livy-server and dependent modules built successfully\n- Manual verification (recommended): Set livy.server.app-name-prefix in livy.conf, create an interactive session without name, and confirm the YARN RM UI shows \u003cprefix\u003e-\u003csession-id\u003e\n\nNo UI changes in this PR.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nYes, this was co-authored using Cursor to help generate the new test cases."
    },
    {
      "commit": "228b889750caf12529ea90fab1ca98f075fcc8d3",
      "tree": "6f0af2de365540f89aa30628315d7724bd2496b8",
      "parents": [
        "1aa25ed384064da37942c78a421e7dec189bd6f4"
      ],
      "author": {
        "name": "nileshrathi345",
        "email": "113672051+nileshrathi345@users.noreply.github.com",
        "time": "Fri Jul 10 00:51:12 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Thu Jul 09 15:21:12 2026 -0400"
      },
      "message": "[LIVY-1059][FOLLOWUP] Fix missing validation condition and add unit test\n\n## What changes were proposed in this pull request?\n\nThis is a follow-up fix for [LIVY-1059](https://github.com/apache/livy/pull/529).\n\nWhen a client sends `\"queue\": \"\"` in the session create JSON, Livy was treating it as a valid queue value instead of falling back to the server default `livy.spark.yarn.queue`.\n\nThis change treats an empty queue string the same as a missing queue, so the default YARN queue from Livy config is used when the client passes \"\".\n\nUpdated in both batch and interactive session creation:\n`request.queue.filterNot(_.isEmpty).orElse(livyConf.sparkYarnQueue())`\n\n## How was this patch tested?\n\n- Added unit test in `BatchSessionSpec` for `queue \u003d Some(\"\")` and verified the default queue is passed to spark-submit\n- Added unit test in `InteractiveSessionSpec` for `queue \u003d Some(\"\")` and verified the session uses the default queue from `LivyConf`\n- Existing tests for `queue \u003d None` and user-provided queue values still pass"
    },
    {
      "commit": "1aa25ed384064da37942c78a421e7dec189bd6f4",
      "tree": "cd2c3506c6bba05aec44bd9c986c0364cbddf79b",
      "parents": [
        "e5accb9830627959b829c3025d3dc907d3e70e1b"
      ],
      "author": {
        "name": "nileshrathi345",
        "email": "113672051+nileshrathi345@users.noreply.github.com",
        "time": "Tue Jul 07 03:30:21 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Jul 06 18:00:21 2026 -0400"
      },
      "message": "[LIVY-1059] Add livy.spark.yarn.queue configuration to set default YARN queue\n\n## What changes were proposed in this pull request?\n\nThis PR introduces support for configuring a default YARN queue at the Livy server level via a new configuration property: `livy.spark.yarn.queue`.\n\nPreviously, if a client did not explicitly specify a YARN queue parameter (`queue`) in their session creation payload, the session would fall back to the global Hadoop cluster\u0027s default YARN queue. This change allows cluster administrators to isolate Livy-generated workloads into a specific default queue without requiring end-users to pass it manually in every API request.\n\nWhen a client provides `queue` in the session creation request, that value takes priority over the server default.\n\n## How was this patch tested?\n\n- Added unit tests in `BatchSessionSpec` that verify `--queue` is passed to spark-submit when using the LivyConf default or a user-provided queue\n- Added unit tests in `InteractiveSessionSpec` that verify the resolved queue is stored on the session\n- Manual API validation against a YARN cluster"
    },
    {
      "commit": "e5accb9830627959b829c3025d3dc907d3e70e1b",
      "tree": "aff7f5634af59344f53149025a19a411b6bd2290",
      "parents": [
        "320df286d117184f489d1e728a48b9d31eced486"
      ],
      "author": {
        "name": "Gabor Roczei",
        "email": "roczei@cloudera.com",
        "time": "Wed Jun 17 18:19:12 2026 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Jun 17 12:19:12 2026 -0400"
      },
      "message": "[LIVY-1040] Migrate logging framework from Reload4j to Log4j2\n\n## What changes were proposed in this pull request?\n\nReplaced the reload4j dependency with the Log4j2 library across all modules.\n\nCo-authored by Peter Toth and Gyorgy Gal:\n\n- Added log4j-api, log4j-core, log4j-slf4j-impl and log4j-1.2-api (backward-compat bridge) to dependencies\n- Removed ch.qos.reload4j:reload4j from all module poms\n- Migrated all log4j.properties test config files to log4j2.properties using the new Log4j2 property-file syntax (rootLogger, appender.* keys)\n\n## How was this patch tested?\n\nIt has been manually tested with the existing unit tests.\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nNo\n\nCo-authored-by: Peter Toth \u003c7253827+peter-toth@users.noreply.github.com\u003e\nCo-authored-by: Gyorgy Gal \u003c27883675+gyogal@users.noreply.github.com\u003e"
    },
    {
      "commit": "320df286d117184f489d1e728a48b9d31eced486",
      "tree": "e3ef07306d9261ee840e37fc03cfd661448856e0",
      "parents": [
        "ce48d4f3d13c45f0eef4ee09a71530167896b6d0"
      ],
      "author": {
        "name": "Gabor Roczei",
        "email": "roczei@cloudera.com",
        "time": "Wed Jun 17 17:41:13 2026 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Jun 17 11:41:13 2026 -0400"
      },
      "message": "[LIVY-1049] Remove the \u0027-incubating\u0027 designator\n\n## What changes were proposed in this pull request?\n\nAs Apache Livy has graduated to a Top-Level Project (TLP), this commit removes the \u0027-incubating\u0027 suffix from all files, build configurations and documentation.\n\nSee announcement: https://news.apache.org/foundation/entry/the-apache-software-foundation-announces-new-top-level-project-4\n\n## How was this patch tested?\n\nManually verified\n\n## Was this patch authored or co-authored using generative AI tooling?\n\nNo"
    },
    {
      "commit": "ce48d4f3d13c45f0eef4ee09a71530167896b6d0",
      "tree": "373ce4e5e9fc58866ae412591f35fbd1998061a3",
      "parents": [
        "4457bdc15e2403ab2f6ab429b9af509c9a48d0ef"
      ],
      "author": {
        "name": "Gabor Roczei",
        "email": "roczei@cloudera.com",
        "time": "Wed Jun 10 21:03:02 2026 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Thu Jun 11 00:33:02 2026 +0530"
      },
      "message": "[LIVY-1042] Add SSL/TLS support for ZooKeeper connection\n\n* [LIVY-1042] Add SSL/TLS support for ZooKeeper connection\n\nIntroduced an optional TLS encryption for the Curator based ZooKeeper\nclient used during session recovery.\n\nChanges by Asif Khatri:\n\n- Added five LivyConf entries (LIVY_ZK_CLIENT_SECURE, LIVY_ZK_CLIENT_SOCKET,\n  LIVY_ZK_KEYSTORE_PASS, LIVY_ZK_TRUSTSTORE_FILE, LIVY_ZK_TRUSTSTORE_PASS)\n  and document them in livy.conf.template.\n- Added ZooKeeperManager.createZKClientConfig that assembles a ZKClientConfig\n  from keystore/truststore settings; applied to the CuratorFramework builder\n  only when livy.server.zk.client.secure\u003dtrue.\n\nTest improvements:\n\n- Extracted mockCurator() helper in ZooKeeperStateStoreSpec to eliminate\n  repeated CuratorFramework mock setup boilerplate.\n- Added an \"SSL config\" describe block with withSslMock\n  helper (SslTestFixture) and tests covering SSL property mapping and\n  both the secure and non-secure construction paths.\n\nWas this patch authored or co-authored using generative AI tooling?\n\nYes, this was co-authored using Cursor to help generate the new test cases.\n\nGenerated-by: Cursor 3.5.17\n\nCo-authored-by: Asif Khatri \u003c123077165+askhatri@users.noreply.github.com\u003e\n\n* Implemented Arnav\u0027s suggestion\n\nWhen livy.server.zk.client.secure\u003dtrue, the following are required: livy.keystore,\nlivy.server.zk.ssl.keyStore.password livy.server.zk.ssl.truststore.location, livy.server.zk.ssl.truststore.password.\n\n---------\n\nCo-authored-by: Asif Khatri \u003c123077165+askhatri@users.noreply.github.com\u003e"
    },
    {
      "commit": "4457bdc15e2403ab2f6ab429b9af509c9a48d0ef",
      "tree": "d34ef8253d8a5a6bd1502e5ff811521082fb8a26",
      "parents": [
        "d0e846e7b8625506e0aab72349ef1b6f5998122c"
      ],
      "author": {
        "name": "Arnav Balyan",
        "email": "60175178+ArnavBalyan@users.noreply.github.com",
        "time": "Tue Jun 09 21:53:59 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Jun 09 12:23:59 2026 -0400"
      },
      "message": "[LIVY-1056] Update Livy readme to reflect post graduation status\n\n## What changes were proposed in this pull request?\n\n - Livy has graduated, the repo has now been formally renamed\n - Updating the readme to point to updated links.\n\n## How was this patch tested?\n\n - CI"
    },
    {
      "commit": "d0e846e7b8625506e0aab72349ef1b6f5998122c",
      "tree": "8f7f92ef07bc72963c6a85a809b48c916804e96b",
      "parents": [
        "45e96b669ed98d2930664b57af164cc2dda8990b"
      ],
      "author": {
        "name": "Gabor Roczei",
        "email": "roczei@cloudera.com",
        "time": "Mon Jun 08 10:07:53 2026 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Jun 08 13:37:53 2026 +0530"
      },
      "message": "[LIVY-1050] Remove the DISCLAIMER file from the Livy repo\n\nRemove the DISCLAIMER file as Apache Livy has graduated to a Top-Level Project (TLP).\nhttps://news.apache.org/foundation/entry/the-apache-software-foundation-announces-new-top-level-project-4"
    },
    {
      "commit": "45e96b669ed98d2930664b57af164cc2dda8990b",
      "tree": "0a1192791e36c1f412866edf7d1ae5892ef2643d",
      "parents": [
        "fc4202ea31722979566f7f730f141420828e4d6b"
      ],
      "author": {
        "name": "Gabor Roczei",
        "email": "roczei@cloudera.com",
        "time": "Fri May 29 16:11:02 2026 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Fri May 29 16:11:02 2026 +0200"
      },
      "message": "[LIVY-1047] Update PR template to align with ASF Generative Tooling Guidance (#524)\n\n## What changes were proposed in this pull request?\nThis adds a section to the PR template asking \"Was this patch authored or co-authored using generative AI tooling?\" to align with the ASF Generative Tooling Guidance: https://www.apache.org/legal/generative-tooling.html\n\nThis was created based on the Spark template with minor adjustments to ensure the formatting matches the previous two sections: https://github.com/apache/spark/blob/master/.github/PULL_REQUEST_TEMPLATE\n\n## How was this patch tested?\nI tried it out in my own git repository."
    },
    {
      "commit": "fc4202ea31722979566f7f730f141420828e4d6b",
      "tree": "3b8eb171815b74d258a461a51f457fccd509d8ff",
      "parents": [
        "717bf7478233fd0f92f55bfe18d6816723697e78"
      ],
      "author": {
        "name": "Arnav Balyan",
        "email": "60175178+ArnavBalyan@users.noreply.github.com",
        "time": "Tue May 19 20:00:13 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue May 19 10:30:13 2026 -0400"
      },
      "message": "[LIVY-1046] Avoid decoding user info twice in HTTP client\n\n## What changes were proposed in this pull request?\n\n - The HTTP client decodes the user info in url twice. \n - Java `URI` returns percent decoded output which is passed to .decode() again. Any password sent with special characters is wrongly decoded.\n - Remove the second decode call and ensure `:` inside the password is not treated as a delimiter\n - Closes LIVY-1046\n\n## How was this patch tested?\n\n - UT"
    },
    {
      "commit": "717bf7478233fd0f92f55bfe18d6816723697e78",
      "tree": "8741d3b2498b118c3981633577d5a75caa8225cb",
      "parents": [
        "70dbf978cee856f086d28f4f777244ec8ea0c593"
      ],
      "author": {
        "name": "Arnav Balyan",
        "email": "60175178+ArnavBalyan@users.noreply.github.com",
        "time": "Mon Apr 27 19:24:32 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Apr 27 09:54:32 2026 -0400"
      },
      "message": "[LIVY-1043] Track ZK connection transitions in Livy\n\n## What changes were proposed in this pull request?\n\n - Currently ZKManager does not observe connection transitions. When underlying connection changes, Livy is not aware of it, only ZK side logs show up.\n - Additionally register to ConnectionStateListener, which adds/listens to Livy owned logs for the connection changes.\n - Same events can be used for recovery in the future.\n\n## How was this patch tested?\n\n - Unit tests"
    },
    {
      "commit": "70dbf978cee856f086d28f4f777244ec8ea0c593",
      "tree": "c21f903804dcf33f801603c6be6e14d0551cd4ae",
      "parents": [
        "fd21e1dbd03636ec152780bb7caf1029eb2d6df4"
      ],
      "author": {
        "name": "Arnav Balyan",
        "email": "60175178+ArnavBalyan@users.noreply.github.com",
        "time": "Thu Apr 09 08:23:24 2026 -0700"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Thu Apr 09 11:23:24 2026 -0400"
      },
      "message": "[LIVY-1039] Redact sensitive credentials/tokens from log output\n\n## What changes were proposed in this pull request?\n\n - Logs in several places are logging sensitive data.\n - Ensure log line does not display this in plain text, credentials should not be logged \n\n## How was this patch tested?\n\n - Existing UTs"
    },
    {
      "commit": "fd21e1dbd03636ec152780bb7caf1029eb2d6df4",
      "tree": "ee413ce4fec0c7e9d3ac475ea5dcd582e51c8890",
      "parents": [
        "0a32275c3717d7486f54c97ef6ecfb3be6b16c66"
      ],
      "author": {
        "name": "Arnav Balyan",
        "email": "60175178+ArnavBalyan@users.noreply.github.com",
        "time": "Thu Mar 26 22:34:35 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Thu Mar 26 13:04:35 2026 -0400"
      },
      "message": "[LIVY-1038] Provide configs for ZK SASL authentication\n\n## What changes were proposed in this pull request?\n\n - Allow configuring configs for ZK SASL authentication\n - Users can enable auth, and pass in custom login context\n - Closes #487 \n \n## How was this patch tested?\n\n - UT, setup a KDC, ZK and tested Livy auth to zk"
    },
    {
      "commit": "0a32275c3717d7486f54c97ef6ecfb3be6b16c66",
      "tree": "4fa980a247454bec2e4ba54f11050389e1725337",
      "parents": [
        "5da3da4e9704c7f2a9dbdfb6e50da4f13a99118e"
      ],
      "author": {
        "name": "Arnav Balyan",
        "email": "60175178+ArnavBalyan@users.noreply.github.com",
        "time": "Mon Mar 02 22:21:31 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Mar 02 11:51:31 2026 -0500"
      },
      "message": "[LIVY-789] Ensure Spark data files are available in interactive session\n\n## What changes were proposed in this pull request?\n\n - Ensures the CWD for interactive sessions is set to Spark root directory for data files to be available. For interactive sessions this bug causes --files argument to not work since the shell does not have access to the staging directory where files are made available. \n - Deprecated unused Spark 1.x code.\n - Adds UTs to ensure files are available when set for interactive sessions.\n\n## How was this patch tested?\n\n - UT"
    },
    {
      "commit": "5da3da4e9704c7f2a9dbdfb6e50da4f13a99118e",
      "tree": "5a8eff0da415f3ef06fb171d2961a0693d209220",
      "parents": [
        "b284a868ed8c81a8af73801f605f8a8b6fbf4f49"
      ],
      "author": {
        "name": "György Gál",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Thu Feb 26 11:28:34 2026 -0500"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Thu Feb 26 11:28:34 2026 -0500"
      },
      "message": "[LIVY-1037] Fix typo in add-exports causing warning on launch\n\n** What changes were proposed in this pull request?\n\nIn the \u0027livy-server\u0027 startup script there is a typo in the \u0027add-exports\u0027\nlist causing a warning to be logged on startup (\u0027ALL_UNNAMED\u0027 instead of\n\u0027ALL-UNNAMED\u0027).\n\n** How was this patch tested?\n\nTested by running `livy-server` locally."
    },
    {
      "commit": "b284a868ed8c81a8af73801f605f8a8b6fbf4f49",
      "tree": "567ecfa81ee3c36358208cd275330281be799e2d",
      "parents": [
        "b141e0aa8b9ff54d360254ab7ed4f8f3ef684207"
      ],
      "author": {
        "name": "Takanobu Asanuma",
        "email": "tasanuma@apache.org",
        "time": "Wed Feb 25 04:12:20 2026 +0900"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Feb 24 14:12:20 2026 -0500"
      },
      "message": "[LIVY-1016] Improve exception handling for Spark YARN Job Startup Failures\n\n## What changes were proposed in this pull request?\n\nWhen jobs fail at startup, such as those specifying an undefined queue, are executed multiple times, the Livy Thrift Server gets stuck.\n\n**Cause of the Problem**  \n\n- When a problematic query is submitted, the `getAppIdFromTag` method throws an `IllegalStateException(\"spark-submit start failed\")`.\n  - https://github.com/apache/incubator-livy/blob/d6822aaa04df87fb17e7e061d39cbc05cedae5c3/server/src/main/scala/org/apache/livy/utils/SparkYarnApp.scala#L196\n- Subsequently, a `yarn application kill` is executed in the `kill()` method, but the above `IllegalStateException` is not caught and slips through, resulting in `yarnAppMonitorThread.interrupt()` not being executed.\n  - https://github.com/apache/incubator-livy/blob/d6822aaa04df87fb17e7e061d39cbc05cedae5c3/server/src/main/scala/org/apache/livy/utils/SparkYarnApp.scala#L153-L160\n- If the `yarnAppMonitorThread` is not stopped, it seems that the Livy Thrift Server eventually hangs.\n\n**Proposed Fix:** Change the exception capture target to `NonFatal` to catch all exceptions.\n\n## How was this patch tested?\n\n- unit test (I added a new unit test in SparkYarnAppSpec.)\n"
    },
    {
      "commit": "b141e0aa8b9ff54d360254ab7ed4f8f3ef684207",
      "tree": "621959b0b28e09548a2e3bec421ec7f1eb33ca07",
      "parents": [
        "01523b89073523611000c6ac577d5d9f12e9084e"
      ],
      "author": {
        "name": "György Gál",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Sat Feb 21 18:59:06 2026 -0500"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Sat Feb 21 18:59:06 2026 -0500"
      },
      "message": "[LIVY-1036] Run both unit tests and integration tests on new PRs\n\n## What changes were proposed in this pull request?\n\nCurrently only integration tests are run on new PRs. Unit tests could also be\nlaunched to ensure code quality.\n\n## How was this patch tested?\n\n- Tested on Github PR."
    },
    {
      "commit": "01523b89073523611000c6ac577d5d9f12e9084e",
      "tree": "11514489a27f15d7b8c2330e5c7d447c6981b83d",
      "parents": [
        "147fc5e920ab8367c6b0a1bb62343f1e55ba25f1"
      ],
      "author": {
        "name": "György Gál",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Sat Feb 21 18:54:56 2026 -0500"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Sat Feb 21 18:54:56 2026 -0500"
      },
      "message": "[LIVY-1031][DOC] README build instructions are out-of-date (#510)\n\n## What changes were proposed in this pull request?\n\n * The project\u0027s `README.md` states that `spark3` and `spark-2.12` are\n   default profiles, however this is not the case.\n * The profile `thriftserver` is added to `README.md`.\n\n## How was this patch tested?\n\nTested with unit test and integration test workflows."
    },
    {
      "commit": "147fc5e920ab8367c6b0a1bb62343f1e55ba25f1",
      "tree": "e68d7a63a5b6a40ad7cf58bc4bf1a18f092ada67",
      "parents": [
        "d432b4c7a990a7899491108fc911139ce61278c3"
      ],
      "author": {
        "name": "Gyorgy Gal",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Fri Jan 02 15:27:39 2026 -0500"
      },
      "committer": {
        "name": "György Gál",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Sat Feb 21 18:52:59 2026 -0500"
      },
      "message": "Update Copyrights in NOTICE and footer for 2026\n"
    },
    {
      "commit": "d432b4c7a990a7899491108fc911139ce61278c3",
      "tree": "831b0f046aac627b04d11979e11b496ea5a42923",
      "parents": [
        "e79ebe379af3cc2f1d4dce52f0b79b37c9e7939e"
      ],
      "author": {
        "name": "Gyorgy Gal",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Mon Dec 01 15:00:25 2025 -0500"
      },
      "committer": {
        "name": "György Gál",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Sat Feb 21 18:46:46 2026 -0500"
      },
      "message": "Update Copyrights in NOTICE and footer\n"
    },
    {
      "commit": "e79ebe379af3cc2f1d4dce52f0b79b37c9e7939e",
      "tree": "3ea7ae01f4d23c76a6c0f26baf1baa83fedbac8e",
      "parents": [
        "27c8603ca9252e5359902deffba867cb750083fa"
      ],
      "author": {
        "name": "Gyorgy Gal",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Mon Dec 01 14:59:49 2025 -0500"
      },
      "committer": {
        "name": "György Gál",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Sat Feb 21 18:32:26 2026 -0500"
      },
      "message": "Update directory configuration\n"
    },
    {
      "commit": "27c8603ca9252e5359902deffba867cb750083fa",
      "tree": "1cc49beb330174930c25704bd74e30f33d6fecb9",
      "parents": [
        "db99f620ca6d5adef59040e83971df191b9054e4"
      ],
      "author": {
        "name": "Gabor Roczei",
        "email": "roczei@gmail.com",
        "time": "Thu Nov 13 15:41:08 2025 -0500"
      },
      "committer": {
        "name": "György Gál",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Sat Feb 21 09:53:32 2026 -0500"
      },
      "message": "Update pre-defined run configuration\n"
    },
    {
      "commit": "db99f620ca6d5adef59040e83971df191b9054e4",
      "tree": "bdbca36480fce2c127d0c7404179d70b84874ef2",
      "parents": [
        "c97fd58eeaa8b52a26836afd9916e4bb6dd58640"
      ],
      "author": {
        "name": "Arnav Balyan",
        "email": "60175178+ArnavBalyan@users.noreply.github.com",
        "time": "Fri Feb 20 08:56:12 2026 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Thu Feb 19 22:26:12 2026 -0500"
      },
      "message": "update (#511)\n\n"
    },
    {
      "commit": "c97fd58eeaa8b52a26836afd9916e4bb6dd58640",
      "tree": "6a02c218923b807b67d89ee21b9ce6eec9e111e6",
      "parents": [
        "674ac00ab28c4a259cb5e9b3ccf9afe706b9aad1"
      ],
      "author": {
        "name": "Edward Capriolo",
        "email": "edlinuxguru@gmail.com",
        "time": "Wed Jan 21 14:31:00 2026 -0500"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Jan 21 14:31:00 2026 -0500"
      },
      "message": "LIVY-1032 include owasp plugin (#507) (edwardcapriolo via lmccay)\n\n"
    },
    {
      "commit": "674ac00ab28c4a259cb5e9b3ccf9afe706b9aad1",
      "tree": "b8752d44efecb991dd164ae0f30e8a7a05ba12cd",
      "parents": [
        "284467b98b8f086cb3cf9a6813873141bced5dc2"
      ],
      "author": {
        "name": "Edward Capriolo",
        "email": "edlinuxguru@gmail.com",
        "time": "Wed Jan 21 14:29:41 2026 -0500"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Jan 21 14:29:41 2026 -0500"
      },
      "message": "LIVY-1030 cleanup of context and clients (#506) (edwardcapriolo via lmccay)\n\n"
    },
    {
      "commit": "284467b98b8f086cb3cf9a6813873141bced5dc2",
      "tree": "27ce57ed08f262da838750945363263d5e617667",
      "parents": [
        "22923a1f1ba8908c1bd16154b46bf4177172bab9"
      ],
      "author": {
        "name": "Edward Capriolo",
        "email": "edlinuxguru@gmail.com",
        "time": "Fri Jan 16 10:24:14 2026 -0500"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Fri Jan 16 10:24:14 2026 -0500"
      },
      "message": "LIVY-1029 modernize versions (#505)\n\n"
    },
    {
      "commit": "22923a1f1ba8908c1bd16154b46bf4177172bab9",
      "tree": "4f2447fc932c4df64db5d163e7a7557868180df8",
      "parents": [
        "26ccdebff41af59822ac0039a41c00c3ee530dda"
      ],
      "author": {
        "name": "Damon P. Cortesi",
        "email": "d.lifehacker@gmail.com",
        "time": "Fri Dec 12 08:48:57 2025 -0800"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Fri Dec 12 11:48:57 2025 -0500"
      },
      "message": "[LIVY-981] Exclude test and provided scopes from THIRD-PARTY license generation\n\n## What changes were proposed in this pull request?\n\n- Added `test,provided` to `excludedScopes` property for license maven plugin.\n- https://issues.apache.org/jira/browse/LIVY-981\n\n## How was this patch tested?\n\n- Generated licenses manually, identified test-scope license removed from generated file"
    },
    {
      "commit": "26ccdebff41af59822ac0039a41c00c3ee530dda",
      "tree": "838c4203283a1177e4832fb18bef269587f5c87d",
      "parents": [
        "4c2549ac92159f8e7ab886c9a836632ec5633281"
      ],
      "author": {
        "name": "György Gál",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Sat Dec 06 22:53:01 2025 -0500"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Sat Dec 06 22:53:01 2025 -0500"
      },
      "message": "[LIVY-1026] Fix Content-Types for nosniff header\n\n## What changes were proposed in this pull request?\n\nIn LIVY-785 the `X-Content-Type-Options: nosniff` HTTP header was added,\nhowever this breaks transferring some files where the Content-Type is\nnot correctly specified.\n\n## How was this patch tested?\n\nTested by running the server locally."
    },
    {
      "commit": "4c2549ac92159f8e7ab886c9a836632ec5633281",
      "tree": "eed043453bfd52fa8da437461258e463aa24715c",
      "parents": [
        "5b4e3480c51298d02242eb427717d97d91eae011"
      ],
      "author": {
        "name": "Arnav Balyan",
        "email": "60175178+ArnavBalyan@users.noreply.github.com",
        "time": "Fri Dec 05 01:53:23 2025 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Thu Dec 04 15:23:23 2025 -0500"
      },
      "message": "[LIVY-1025][TEST] Clean up Python2InterpreterSpec post Python deprecation\n\n## What changes were proposed in this pull request?\n\n * Cleans up Python2InterpreterSpec since python 2 is deprecated.\n * Also removed the Python2SessionSpec counterpart\n\n## How was this patch tested?\n\n * CI"
    },
    {
      "commit": "5b4e3480c51298d02242eb427717d97d91eae011",
      "tree": "49416626a889cfb34dd7a5616ae704c239847e59",
      "parents": [
        "49a36164784f00a652df3d86f4eaf9652fb50f13"
      ],
      "author": {
        "name": "Arnav Balyan",
        "email": "60175178+ArnavBalyan@users.noreply.github.com",
        "time": "Thu Dec 04 20:34:38 2025 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Thu Dec 04 10:04:38 2025 -0500"
      },
      "message": "[LIVY-1024] Upgrade Livy to Python3\n\n## What changes were proposed in this pull request?\n\n * Python2 was deprecated a while back, most systems/environments don\u0027t support it anymore\n * Bump Livy to Python3, remove Python2 support since Spark 2 has also now been deprecated\n * Closes LIVY-1024\n\n## How was this patch tested?\n\n * CI / Locally tested the python3 files manually"
    },
    {
      "commit": "49a36164784f00a652df3d86f4eaf9652fb50f13",
      "tree": "db5d839d6c76d7d4e47e45038702dff633089cd9",
      "parents": [
        "0efa2d7e508c6d955cebe48a5159ffc3202e57a1"
      ],
      "author": {
        "name": "Edward Capriolo",
        "email": "edlinuxguru@gmail.com",
        "time": "Wed Dec 03 15:10:16 2025 -0500"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Dec 03 15:10:16 2025 -0500"
      },
      "message": "[LIVY-1023] Missing double quote causes server not to start\n\n## What changes were proposed in this pull request?\n\nbin/livy-server doesnt work at all.\n\n## How was this patch tested?\n\n```\nedward@fedora:~/incubator-livy$ bin/livy-server \nWARNING: Unknown module: ALL_UNNAMED specified to --add-exports\nlog4j:WARN No appenders could be found for logger (org.apache.livy.server.AccessManager).\nlog4j:WARN Please initialize the log4j system properly.\nlog4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.\nException in thread \"main\" java.lang.IllegalArgumentException: Livy requires the SPARK_HOME environment variable\n\tat org.apache.livy.utils.LivySparkUtils$$anonfun$1.apply(LivySparkUtils.scala:55)\n\tat org.apache.livy.utils.LivySparkUtils$$anonfun$1.apply(LivySparkUtils.scala:55)\n\tat scala.Option.getOrElse(Option.scala:121)\n\tat org.apache.livy.utils.LivySparkUtils$.testSparkHome(LivySparkUtils.scala:54)\n\tat org.apache.livy.server.LivyServer.start(LivyServer.scala:79)\n\tat org.apache.livy.server.LivyServer$.main(LivyServer.scala:439)\n\tat org.apache.livy.server.LivyServer.main(LivyServer.scala)\nedward@fedora:~/incubator-livy$ git status\n```"
    },
    {
      "commit": "0efa2d7e508c6d955cebe48a5159ffc3202e57a1",
      "tree": "9a6ba6720c6c3df35489401c360b0399d31c181b",
      "parents": [
        "acb11e11ebaf36c5792e0b51d90421964c57c88d"
      ],
      "author": {
        "name": "Arnav Balyan",
        "email": "60175178+ArnavBalyan@users.noreply.github.com",
        "time": "Tue Dec 02 01:27:42 2025 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Dec 01 14:57:42 2025 -0500"
      },
      "message": "[LIVY-1022][TEST] Update NM Disk utilization percentage for UT\n\n## What changes were proposed in this pull request?\n\n * Currently Livy UT for `JobApiIT.scala` fails due to disk being 92% which causes the node to be lost.\n * Updated this to 100% for the UT to ensure the tests pass even if the disk utilization is over 90%\n * Closes LIVY-1022\n\n## How was this patch tested?\n\n * UT fails on master without this change\n * This PR should allow UT to go through"
    },
    {
      "commit": "acb11e11ebaf36c5792e0b51d90421964c57c88d",
      "tree": "2a6daa199175e47a0224be51221b1d0b97430e96",
      "parents": [
        "c7d1c26435b72dc64620a632853837d2fa57d991"
      ],
      "author": {
        "name": "Arnav Balyan",
        "email": "60175178+ArnavBalyan@users.noreply.github.com",
        "time": "Fri Nov 14 07:11:09 2025 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Thu Nov 13 20:41:09 2025 -0500"
      },
      "message": "[LIVY-1019] Deprecate Spark 2 support\n\n## What changes were proposed in this pull request?\n\n * Spark 2.x is at end of life and no longer supported, newer Spark versions (Spark 4) are now available.\n * Deprecate Spark 2.x\n * Also offboard Scala 2.11 support\n\n## How was this patch tested?\n\n * CI and local builds are successful"
    },
    {
      "commit": "c7d1c26435b72dc64620a632853837d2fa57d991",
      "tree": "87c6969c74ce04872f4f0a0e8e1179547ff9b24e",
      "parents": [
        "47db26909ca5e681a8c53653683dd33c5240d079"
      ],
      "author": {
        "name": "György Gál",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Sun Nov 09 18:13:25 2025 -0500"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Sun Nov 09 18:13:25 2025 -0500"
      },
      "message": "Update main to 0.10.0-incubating-SNAPSHOT\n\n"
    },
    {
      "commit": "47db26909ca5e681a8c53653683dd33c5240d079",
      "tree": "0db270f71e8269c5a89320078e44a9ea9780f3bd",
      "parents": [
        "1fb8f67e68912f271a0b5f0756c5302646bcdc01"
      ],
      "author": {
        "name": "Damon P. Cortesi",
        "email": "d.lifehacker@gmail.com",
        "time": "Mon Nov 03 10:36:45 2025 -0800"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Nov 03 13:36:45 2025 -0500"
      },
      "message": "[LIVY-985] Remove unused Cat X licenses\n\n## What changes were proposed in this pull request?\n\nExclude unnecessary Cat X licenses where possible.\n\n## How was this patch tested?\n\nBuilding project during release process."
    },
    {
      "commit": "1fb8f67e68912f271a0b5f0756c5302646bcdc01",
      "tree": "c0a681b5d560a62ea661a3ed257c68b0295a1559",
      "parents": [
        "f3e23ed0b34da9d44ec06c724d5d03c7cf15aebe"
      ],
      "author": {
        "name": "Damon P. Cortesi",
        "email": "d.lifehacker@gmail.com",
        "time": "Mon Oct 27 10:23:53 2025 -0700"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Oct 27 13:23:53 2025 -0400"
      },
      "message": "[LIVY-980] Update release script with modern commands\n\n## What changes were proposed in this pull request?\n\nDuring the previous release, several changes had to be made to the release script to get it to work. This PR includes those changes.\n\n## How was this patch tested?\n\nManually during the release process."
    },
    {
      "commit": "f3e23ed0b34da9d44ec06c724d5d03c7cf15aebe",
      "tree": "d318ea5eea3a660fbef0344a766e3c8ab99be92c",
      "parents": [
        "370a71561dcf60a860d3b97ef20810b95bf90980"
      ],
      "author": {
        "name": "György Gál",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Wed Oct 22 10:29:20 2025 -0400"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Oct 22 10:29:20 2025 -0400"
      },
      "message": "[LIVY-1017] Support Java 17, set up JDK 17 based tests\n\n## What changes were proposed in this pull request?\n\nEnable Livy to run with Java 17, set up unit test and integration test runs with JDK 17.\n\n## How was this patch tested?\n\nUnit and integration tests."
    },
    {
      "commit": "370a71561dcf60a860d3b97ef20810b95bf90980",
      "tree": "3d0608caa65621afa8f5b2edde09ba9aebb4db4d",
      "parents": [
        "710c0784dae73aa7c5a6ce0bac264bcfdefd7681"
      ],
      "author": {
        "name": "Arnav Balyan",
        "email": "60175178+ArnavBalyan@users.noreply.github.com",
        "time": "Tue Oct 07 20:00:17 2025 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Oct 07 10:30:17 2025 -0400"
      },
      "message": "[LIVY-1018] Extract field comments from Spark schema metadata and set in Datatypeutils\n\n## What changes were proposed in this pull request?\n\n * During schema fetch, ensure we can extract and persist the comments.\n * Column comments extracted from Spark metadata for complexToDataType and schemaFromSparkJson\n * Closes Extract comment metadata for complexToDataType and schemaFromSparkJson #483\n\n## How was this patch tested?\n\nCI"
    },
    {
      "commit": "710c0784dae73aa7c5a6ce0bac264bcfdefd7681",
      "tree": "5544eceb0409d2dc0dcc8c3a2dcf5ea46bf79380",
      "parents": [
        "c28f2fce58e1fc2e69b1c7470bd9c02d8cb196a6"
      ],
      "author": {
        "name": "György Gál",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Tue Aug 12 19:38:36 2025 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Aug 12 19:38:36 2025 +0200"
      },
      "message": "[LIVY-1010] Add support for Spark 3.5.6\n\n## What changes were proposed in this pull request?\n\nKeep Spark 3 support up to date with the latest Spark 3 release, upgrade dependencies.\n\n## How was this patch tested?\n\nUnit and integration tests."
    },
    {
      "commit": "c28f2fce58e1fc2e69b1c7470bd9c02d8cb196a6",
      "tree": "9e490f59afc17d0d02ade0151b3da00354c47a2d",
      "parents": [
        "adb10848e259fbd9687198b137a8e141abcdbc73"
      ],
      "author": {
        "name": "nileshrathi345",
        "email": "113672051+nileshrathi345@users.noreply.github.com",
        "time": "Mon Aug 11 17:52:41 2025 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Aug 11 14:22:41 2025 +0200"
      },
      "message": "[LIVY-1009] Add support for global TTL for sessions\n\n## What changes were proposed in this pull request?\n\nAdded two new Livy configuration parameters as below:\n\n```\nEnabled to check whether TTL Livy sessions should be stopped.\nlivy.server.session.ttl-check \u003d false\n\nTime in milliseconds on how long Livy will wait before TTL is an inactive session.\nNote that the inactive session could be busy running jobs.\nlivy.server.session.ttl \u003d 8h\n```\n\n## How was this patch tested?\n\nRead LIVY-1009 for details."
    },
    {
      "commit": "adb10848e259fbd9687198b137a8e141abcdbc73",
      "tree": "5428d562d0b8cb850b70c1dffa411a69fe374339",
      "parents": [
        "d6822aaa04df87fb17e7e061d39cbc05cedae5c3"
      ],
      "author": {
        "name": "Yi Wei",
        "email": "yiwei.in.cyber@gmail.com",
        "time": "Sat Jun 14 02:11:04 2025 +0800"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Fri Jun 13 20:11:04 2025 +0200"
      },
      "message": "[MINOR][DOC] Fix the missing fields in the Batch model\n\n"
    },
    {
      "commit": "d6822aaa04df87fb17e7e061d39cbc05cedae5c3",
      "tree": "21fbbdaec5293ce380d7b85706185ec89fb7768c",
      "parents": [
        "df691033b11440837c2a06fe624510bdbf815fd5"
      ],
      "author": {
        "name": "idzikovsky",
        "email": "23353239+idzikovsky@users.noreply.github.com",
        "time": "Tue May 20 13:26:32 2025 +0300"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue May 20 12:26:32 2025 +0200"
      },
      "message": "[MINOR][DOC] Fix cursor parameter documentation for completion endpoint\n\n"
    },
    {
      "commit": "df691033b11440837c2a06fe624510bdbf815fd5",
      "tree": "98cb133f6994815b4e643f7508d02e3a20a4cbda",
      "parents": [
        "f580e71b5805c0b2ed2b93005fce9bc6396354e7"
      ],
      "author": {
        "name": "Surendra Singh Lilhore",
        "email": "45159337+surendralilhore@users.noreply.github.com",
        "time": "Wed Apr 23 19:15:58 2025 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Apr 23 15:45:58 2025 +0200"
      },
      "message": "[LIVY-1012] Use SslContextFactory.Server() instead of SslContextFactory\n\n## What changes were proposed in this pull request?\n\nUse SslContextFactory.Server() instead of SslContextFactory() when constructing the factory.\n\n## How was this patch tested?\n\nWith Jetty 9.4.50, we should call SslContextFactory.Server(), instead of SslContextFactory(), to create SslContextFactory. Otherwise we get the following error when using a KeyStore with multiple certificates in it.\n\n```\nException in thread \"main\" java.lang.IllegalStateException: KeyStores with multiple certificates are not supported on the base class org.eclipse.jetty.util.ssl.SslContextFactory. (Use org.eclipse.jetty.util.ssl.\nSslContextFactory$Server or org.eclipse.jetty.util.ssl.SslContextFactory$Client instead)\nat org.eclipse.jetty.util.ssl.SslContextFactory.newSniX509ExtendedKeyManager(SslContextFactory.java:1289)\n```"
    },
    {
      "commit": "f580e71b5805c0b2ed2b93005fce9bc6396354e7",
      "tree": "5b35bda2cc78d2f0fa329c9c7b58080107ad12e3",
      "parents": [
        "826ec291fa83c26fd715b5f043629af2a48014cb"
      ],
      "author": {
        "name": "nileshrathi345",
        "email": "113672051+nileshrathi345@users.noreply.github.com",
        "time": "Mon Apr 14 19:25:39 2025 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Apr 14 15:55:39 2025 +0200"
      },
      "message": "[LIVY-785] Enabled adding security related HTTP headers\n\n## What changes were proposed in this pull request?\n\nThis change introduces a new configuration option `livy.server.security-headers.enabled`.\nWhen this property is set to true, the following security headers are added to HTTP\nresponses by default:\n\n * X-XSS-Protection\n * X-Frame-Options\n * X-Content-Type-Options\n * Strict-Transport-Security\n * Content-Security-Policy\n\nAlso, adds content type information to all responses as required when using content type option nosniff\n\n## How was this patch tested?\n\nTested manually"
    },
    {
      "commit": "826ec291fa83c26fd715b5f043629af2a48014cb",
      "tree": "1afbbbf7a3ead7e149091de56401cbe13d73b584",
      "parents": [
        "1f6bd7ab55db3e5b08ff133d554b32db83a23890"
      ],
      "author": {
        "name": "György Gál",
        "email": "27883675+gyogal@users.noreply.github.com",
        "time": "Wed Apr 02 13:23:42 2025 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Apr 02 13:23:42 2025 +0200"
      },
      "message": "[LIVY-1011] Upgrade CI image, fix Spark 3 tests\n\n## What changes were proposed in this pull request?\n\nThe CI environment is broken due to library version conflicts between\nthe image and the checkout action.\n\nThe livy-ci Docker image needs to be upgraded to a more recent version\nof Ubuntu, ensuring that the unit tests and integration tests all pass.\n\nDue to a bug in Livy\u0027s GitHub workflows, tests had only been executed\nwith Spark 2. Now that this is fixed, we also need to make updates to\nsome of the Python-based tests to work with Python 3 because Spark 3.2\ndoes not support Python 2.\n\nTwo R-based integration tests are ignored in the Spark 2 integration\ntest runs because SparkR 2 does not work with R 4, and the last version\nof Ubuntu with R 3.6 is Ubuntu 20. If we want to re-enable these tests,\nwe may need to build R 3.6 for Ubuntu 24.\n\n## How was this patch tested?\n\nCI and unit test runs in a private fork of the repo."
    },
    {
      "commit": "1f6bd7ab55db3e5b08ff133d554b32db83a23890",
      "tree": "e430bcc25002260f94664c9c87cbcd1c87e8c6a4",
      "parents": [
        "6097af1cdd536ebbe1d7eacb1513a440a5fa2784"
      ],
      "author": {
        "name": "Moritz Meister",
        "email": "8422705+moritzmeister@users.noreply.github.com",
        "time": "Tue Nov 12 14:33:33 2024 +0100"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Nov 12 14:33:33 2024 +0100"
      },
      "message": "[LIVY-863] Missing JVM class imports for Spark3\n\n## What changes were proposed in this pull request?\r\n\r\nDescription of the problem: https://issues.apache.org/jira/browse/LIVY-863\r\n\r\nThe proposed fix consists of adding the missing imports that upstream Spark has when it\u0027s initiating the Java Gateway.\r\nSee the imports of Spark here: https://github.com/apache/spark/blob/87bf6b0ea4ca0618c8604895d05037edce8b7cb0/python/pyspark/java_gateway.py#L153\r\n\r\nAs far as I am aware, the java_import() does not fail or fails silently if the imported class does not exist.\r\nBut I might need to add some code to account for different Spark versions, looking for some guidance on this.\r\n\r\n## How was this patch tested?\r\n\r\nTested with both valid and \"bogus\" imports by reviewers."
    },
    {
      "commit": "6097af1cdd536ebbe1d7eacb1513a440a5fa2784",
      "tree": "4e501b70a0217871dc591c45e2ccbc851838982f",
      "parents": [
        "6dcb29422f5a43e396a6a2006819284e491de169"
      ],
      "author": {
        "name": "wangdengshan",
        "email": "wangdengshan01@163.com",
        "time": "Fri Sep 13 16:10:30 2024 +0800"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Fri Sep 13 10:10:30 2024 +0200"
      },
      "message": "[LIVY-1003][RSC] Interactive session - Setting large value of rsc.server.connect.timeout blocks other tasks\n\n## What changes were proposed in this pull request?\r\n\r\nThe main adjustment here is the thread pool used when creating and closing sessions asynchronously. Scala\u0027s default thread pool size is limited, which will cause the waiting thread to be blocked.\r\n\r\nhttps://issues.apache.org/jira/browse/LIVY-1003\r\n\r\n## How was this patch tested?\r\n\r\nHow to reproduce:\r\n1. Set `livy.rsc.server.connect.timeout` to something high like 24h.\r\n2. Create enough interactive livy sessions in YARN so that they are queued in ACCEPTED state. The number of sessions that are stuck in ACCEPTED state should be equal to global execution context [thread pool size](https://docs.scala-lang.org/overviews/core/futures.html#the-global-execution-context) (Runtime.availableProcessors)\r\n3. Try to delete a session using DELETE /sessions/{sessionId}\r\nand it should not be hang until one of the sessions is no longer stuck in ACCEPTED state.\r\n"
    },
    {
      "commit": "6dcb29422f5a43e396a6a2006819284e491de169",
      "tree": "695bbae4ef42409745138a0e1e9199885a04e3c9",
      "parents": [
        "b089dd68277f5b97bd939787f05a953694302fba"
      ],
      "author": {
        "name": "Asif Khatri",
        "email": "123077165+askhatri@users.noreply.github.com",
        "time": "Mon Aug 12 16:56:29 2024 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Aug 12 13:26:29 2024 +0200"
      },
      "message": "[LIVY-1007] Livy should not spawn one thread per job to track the job on Kubernetes\n\n## What changes were proposed in this pull request?\r\n\r\nInstead of spawning a monitor thread for every session, create a centralised thread to monitor all Kubernetes apps.\r\n\r\nJIRA link: https://issues.apache.org/jira/browse/LIVY-1007\r\n\r\n## How was this patch tested?\r\n\r\n**Unit Tests:** The patch has been verified through comprehensive unit tests.\r\n**Manual Testing:** Conducted manual testing using Kubernetes on Docker Desktop.\r\n\r\nEnvironment: Helm charts. For detailed instructions on testing using Helm charts, please refer to the documentation available at [livycluster](https://github.com/askhatri/livycluster).\r\n\r\nCo-authored-by: Asif Khatri \u003casif.khatri@cloudera.com\u003e"
    },
    {
      "commit": "b089dd68277f5b97bd939787f05a953694302fba",
      "tree": "6b60e5e070b85075676a600e7c4db0c8276b32e0",
      "parents": [
        "03ceb4ae1f764b7299998ffafffda0a18d8a1602"
      ],
      "author": {
        "name": "Asif Khatri",
        "email": "123077165+askhatri@users.noreply.github.com",
        "time": "Wed Jul 10 14:35:17 2024 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Jul 10 11:05:17 2024 +0200"
      },
      "message": "[LIVY-702] Submit Spark apps to Kubernetes (#451)\n\nThis pull request (PR) is the foundational PR for adding Kubernetes support in Apache Livy, originally found here (#249). This update includes a newer version of the Kubernetes client and adds code to display the Spark UI.\r\n\r\n## Summary of the Proposed Changes\r\n\r\nThis PR introduces a method to submit Spark applications to a Kubernetes cluster. The key points covered include:\r\n\r\n * Submitting batch sessions\r\n * Submitting interactive sessions\r\n * Monitoring sessions, collecting logs, and gathering diagnostic information\r\n * Restoring session monitoring after restarts\r\n * Garbage collection (GC) of created Kubernetes resources\r\n\r\nJIRA link: https://issues.apache.org/jira/browse/LIVY-702\r\n\r\n## How was this patch tested?\r\n\r\n * Unit Tests: The patch has been verified through comprehensive unit tests.\r\n * Manual Testing: Conducted manual testing using Kubernetes on Docker Desktop.\r\n    *  Environment: Helm charts.\r\n\r\nFor detailed instructions on testing using Helm charts, please refer to the documentation available at https://github.com/askhatri/livycluster\r\n\r\nCo-authored-by: Asif Khatri \u003casif.khatri@cloudera.com\u003e\r\nCo-authored-by: Alex Sasnouskikh \u003cjahstreetlove@gmail.com\u003e"
    },
    {
      "commit": "03ceb4ae1f764b7299998ffafffda0a18d8a1602",
      "tree": "0664807170a704bd6c200462eee23745005aa1d9",
      "parents": [
        "21ca618e13137ff60553e91284e7b6f26b571819"
      ],
      "author": {
        "name": "Javi Roman",
        "email": "jroman.espinar@gmail.com",
        "time": "Wed Jun 26 10:21:03 2024 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Jun 26 10:21:03 2024 +0200"
      },
      "message": "[LIVY-997][DOC] Add file .sdkmanrc to .gitignore (#444)\n\n## What changes were proposed in this pull request?\r\n\r\nAdd new ignore pattern to .gitignore file for .sdkmanrc file."
    },
    {
      "commit": "21ca618e13137ff60553e91284e7b6f26b571819",
      "tree": "de6e927752761b3123c9ee4d7cc1517603bf067b",
      "parents": [
        "78ddd595953711aeed2a40b76408b58faba971f1"
      ],
      "author": {
        "name": "RajshekharMuchandi",
        "email": "rsmuchandi@gmail.com",
        "time": "Tue Jun 25 19:54:52 2024 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Jun 25 16:24:52 2024 +0200"
      },
      "message": "[LIVY-991][SERVER] Facing issues with the Livy UI Driver link (#437)\n\nAdded conditional check on finished state to set driverlogUrl to null\r\n\r\nCo-authored-by: Rajshekhar Muchandi \u003crajshekhar.muchandi@cloudera.com\u003e"
    },
    {
      "commit": "78ddd595953711aeed2a40b76408b58faba971f1",
      "tree": "15058b089e7dd1f3135fcfa654e4227bfbc8314e",
      "parents": [
        "728fcf0ae9b94b7690c8a3f1eaa4d05507bfbea6"
      ],
      "author": {
        "name": "RajshekharMuchandi",
        "email": "rsmuchandi@gmail.com",
        "time": "Tue Jun 25 19:41:02 2024 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Jun 25 16:11:02 2024 +0200"
      },
      "message": "[LIVY-992][SERVER] In livy logs, user is not able to differentiate between interactive session and batch session (#435)\n\nChanged log info to provide session type details\r\n\r\nCo-authored-by: Rajshekhar Muchandi \u003crajshekhar.muchandi@cloudera.com\u003e"
    },
    {
      "commit": "728fcf0ae9b94b7690c8a3f1eaa4d05507bfbea6",
      "tree": "d95b8fc88f7967479343a0d786ba3c923ca81ea8",
      "parents": [
        "f615f272e9130d02170024832ea308516b907195"
      ],
      "author": {
        "name": "Asif Khatri",
        "email": "123077165+askhatri@users.noreply.github.com",
        "time": "Mon Jun 10 13:46:06 2024 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Jun 10 10:16:06 2024 +0200"
      },
      "message": "[LIVY-998][THRIFT] Support connecting to existing sessions using session name via Thrift Server\n\nCo-authored-by: Asif Khatri \u003casif.khatri@cloudera.com\u003e"
    },
    {
      "commit": "f615f272e9130d02170024832ea308516b907195",
      "tree": "807b5bb3a2469e2c4f4a63d034fdcc6d9dd75700",
      "parents": [
        "96b1eb617b18302ae842d0ebed298e5ec61429f5"
      ],
      "author": {
        "name": "RajshekharMuchandi",
        "email": "rsmuchandi@gmail.com",
        "time": "Tue Mar 12 20:57:17 2024 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Mar 12 16:27:17 2024 +0100"
      },
      "message": "[LIVY-977][SERVER][CONF] Livy can not be started if HDFS is still in safe mode (#440)\n\n## What changes were proposed in this pull request?\r\n\r\nHDFS safe mode is checked when livy session is created. If safe mode is ON, then IllegalStateException is thrown after max retry attempts (configurable) with safe mode interval (configurable) checks are done. If safe mode is OFF, then livy will be able to create session.\r\n\r\nhttps://issues.apache.org/jira/browse/LIVY-977\r\n\r\n## How was this patch tested?\r\n\r\nAdded unit test cases to validate code changes. Also, done manual testing in CDP cluster by creating livy sessions with HDFS safe mode check ON/OFF."
    },
    {
      "commit": "96b1eb617b18302ae842d0ebed298e5ec61429f5",
      "tree": "e7ba69990db8eac6b57dc1e00e67a176652c1dcc",
      "parents": [
        "86fc823893ee96d4effaa4f2b8ef6603cea9d77a"
      ],
      "author": {
        "name": "jianzhenwu",
        "email": "117174379+jianzhenwu@users.noreply.github.com",
        "time": "Fri Jan 26 07:56:55 2024 +0800"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Thu Jan 25 18:56:55 2024 -0500"
      },
      "message": "[LIVY-995][REPL] JsonParseException is thrown when closing Livy session when using python profile (#439)\n\n* [LIVY-995][REPL] JsonParseException is thrown when closing Livy session when using python profile"
    },
    {
      "commit": "86fc823893ee96d4effaa4f2b8ef6603cea9d77a",
      "tree": "e74aac11924532634df6736b03fd7da70515fb28",
      "parents": [
        "024efa549bb490b6682ac6bd7fa5bf8b556b0d0c"
      ],
      "author": {
        "name": "Asif Khatri",
        "email": "123077165+askhatri@users.noreply.github.com",
        "time": "Mon Oct 30 19:28:53 2023 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Mon Oct 30 14:58:53 2023 +0100"
      },
      "message": "[LIVY-989] Livy core support for interactive session idleTimeout (#426)\n\n## What changes were proposed in this pull request?\r\n\r\nCurrently, a Livy interactive session has a field called ttl, which kills the session if it has been idle for a given amount of time. However, here is the expected behavior:\r\n\r\n * ttl: kills the session if it has been active for a certain duration, irrespective of idleness.\r\n * idleTimeout: kills the session if it has been idle for the given duration. (The current TTL behaves in this manner.)\r\n\r\nJIRA: https://issues.apache.org/jira/browse/LIVY-989\r\n\r\n## How was this patch tested?\r\n\r\nTested manually by creating interactive session with idle time and TTL."
    },
    {
      "commit": "024efa549bb490b6682ac6bd7fa5bf8b556b0d0c",
      "tree": "e9b2e5234c29944261d97878f129a0f6762f47dc",
      "parents": [
        "8b2e29fe9fd42c20395c63e1571f2492f005162b"
      ],
      "author": {
        "name": "mvanderlee",
        "email": "918128+mvanderlee@users.noreply.github.com",
        "time": "Tue Oct 24 22:41:22 2023 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Oct 24 13:41:22 2023 -0700"
      },
      "message": "Fix testing with Python3 (#424)\n\nFix print statement in tests to be Python 2 and Python 3 compatible. #408"
    },
    {
      "commit": "8b2e29fe9fd42c20395c63e1571f2492f005162b",
      "tree": "ae26dd0faf0a540509e6070c61d276fdd886e1fa",
      "parents": [
        "bf30e3bd3105f72a6a7fd2a276a16dabe8bdf104"
      ],
      "author": {
        "name": "Marco Gaido",
        "email": "mgaido@fbk.eu",
        "time": "Thu Sep 07 15:56:45 2023 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Thu Sep 07 21:56:45 2023 +0800"
      },
      "message": "[DEV] Support python3 for merge_livy_pr script (#418)\n\n"
    },
    {
      "commit": "bf30e3bd3105f72a6a7fd2a276a16dabe8bdf104",
      "tree": "78ea49696aa3f34d47b3ddf6094ce17ebc7d825f",
      "parents": [
        "5dccc479c6087112f048a7e5cff0723855ef14e9"
      ],
      "author": {
        "name": "jianzhen.wu",
        "email": "jianzhen.wu@shopee.com",
        "time": "Tue Sep 05 11:57:36 2023 +0200"
      },
      "committer": {
        "name": "Marco Gaido",
        "email": "mgaido@fbk.eu",
        "time": "Tue Sep 05 11:57:36 2023 +0200"
      },
      "message": "[LIVY-987] NPE when waiting for thrift session to start timeout.\n\n## What changes were proposed in this pull request?\nFix NPE when waiting for thrift session to start timeout.\nhttps://issues.apache.org/jira/browse/LIVY-987\n\n## How was this patch tested?\nmanual tests by beeline and set timeout to 10s.\n\n0: jdbc:hive2://username:passwordthrift-server\u003e select 123;\n\nRSC client is executing SQL query: select 123, statementId \u003d 681bc017-8f37-4665-a575-da355db77254, session \u003d SessionHandle [c17f1729-6ee1-4260-b82b-aebec3b08e14]\n\nLivy session has not yet started. Please wait for it to be ready...\n\nError: java.util.concurrent.TimeoutException: Futures timed out after [10000 milliseconds] (state\u003d,code\u003d0)\n\n0: jdbc:hive2://username:passwordthrift-server\u003e Closing: 0: jdbc:hive2://username:passwordthrift-server\n\nPlease review https://livy.incubator.apache.org/community/ before opening a pull request.\n\nAuthor: jianzhen.wu \u003cjianzhen.wu@shopee.com\u003e\nAuthor: jianzhenwu \u003c117174379+jianzhenwu@users.noreply.github.com\u003e\n\nCloses #416 from jianzhenwu/LIVY-987.\n"
    },
    {
      "commit": "5dccc479c6087112f048a7e5cff0723855ef14e9",
      "tree": "52addb86bb080ee43d0b17c3ac1c712ae028a7cb",
      "parents": [
        "4cee47df13348b6a3daedc5215d0cec0f98c83ee"
      ],
      "author": {
        "name": "Andras Beni",
        "email": "anna.andras.beni@gmail.com",
        "time": "Fri Aug 18 13:57:43 2023 +0200"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Fri Aug 18 13:57:43 2023 +0200"
      },
      "message": "[LIVY-793] Make keystore type configurable (#395)\n\n## What changes were proposed in this pull request?\r\n\r\nThis change introduces configuration parameter livy.keystore.type.\r\nThe default value is JKS which is equivalent to current functionality.\r\n\r\n## How was this patch tested?\r\n\r\nThis change was tested by running existing tests and manually verifying functionality using non-JKS keystore."
    },
    {
      "commit": "4cee47df13348b6a3daedc5215d0cec0f98c83ee",
      "tree": "42e50aa6c6279511dd584bd30925ed3f0cb27529",
      "parents": [
        "074d6a284281f5d08d4572f97216716696b1ca9b"
      ],
      "author": {
        "name": "Asif Khatri",
        "email": "123077165+askhatri@users.noreply.github.com",
        "time": "Fri Aug 18 17:23:06 2023 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Fri Aug 18 13:53:06 2023 +0200"
      },
      "message": "[LIVY-986][SERVER] Adding null pointer check for SessionInfo (#412)\n\nWe need to add a null check for SessionInfo in InteractiveSessionServlet.scala to avoid an exception that might occur during a Livy upgrade."
    },
    {
      "commit": "074d6a284281f5d08d4572f97216716696b1ca9b",
      "tree": "8b46a58c41c43d40f394b31e871baf1083cb343a",
      "parents": [
        "b175ba1bd1210e2d1db61bdf19a3dbfc1374dd8c"
      ],
      "author": {
        "name": "lmccay",
        "email": "lmccay@apache.org",
        "time": "Sun Aug 13 15:56:15 2023 -0400"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Sun Aug 13 15:56:15 2023 -0400"
      },
      "message": "Livy 984 (#410)\n\nLIVY-984 - Update Copyrights in NOTICE and footer"
    },
    {
      "commit": "b175ba1bd1210e2d1db61bdf19a3dbfc1374dd8c",
      "tree": "ab6084c026286f16af7b1beaf2bba94629174f22",
      "parents": [
        "713cb3e7938863969d5de26b0d5a20d8990ff584"
      ],
      "author": {
        "name": "Asif Khatri",
        "email": "123077165+askhatri@users.noreply.github.com",
        "time": "Sat Jul 22 09:10:56 2023 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Sat Jul 22 11:40:56 2023 +0800"
      },
      "message": "LIVY-978: Update the application tag generation logic in Livy (#409)\n\nCo-authored-by: Asif Khatri \u003casif.khatri@cloudera.com\u003e"
    },
    {
      "commit": "713cb3e7938863969d5de26b0d5a20d8990ff584",
      "tree": "36c97dbb74ad72c4838cb4cb7cef3b4621a592eb",
      "parents": [
        "314f2def423954a77eb75f72e9bf7f4e51a6f7f4"
      ],
      "author": {
        "name": "Damon P. Cortesi",
        "email": "d.lifehacker@gmail.com",
        "time": "Wed Jul 19 23:07:25 2023 +0000"
      },
      "committer": {
        "name": "Damon P. Cortesi",
        "email": "d.lifehacker@gmail.com",
        "time": "Wed Jul 19 23:07:25 2023 +0000"
      },
      "message": "Update main to 0.9.0-incubating-SNAPSHOT\n"
    },
    {
      "commit": "314f2def423954a77eb75f72e9bf7f4e51a6f7f4",
      "tree": "91925a3b085ec5336b9bc111038791930fb568ff",
      "parents": [
        "c3dd6457c45662de13be72919532ed0ef1b57ab3"
      ],
      "author": {
        "name": "Sumit Kumar",
        "email": "ksumit@users.noreply.github.com",
        "time": "Fri Jun 16 17:03:57 2023 -0700"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Fri Jun 16 17:03:57 2023 -0700"
      },
      "message": "[LIVY-969] Create docker based integration environment for local debugging (#407)\n\n* [LIVY-969] Create docker based integration environment for local\r\ndebugging\r\n* Add missing newlines, move customization steps towards the end in Dockerfile so we can reuse previous layers and avoid recreating it for every image build\r\n* Revert changes in module list, still update the scala-maven-plugin version to avoid issues compiling against 2.12\r\n* Upgrade spark3 version to spark 3.2.3\r\n* Fix Dockerfile path in ci image build pipeline, \r\n* Update instructions to use livy-ci container image name\r\n\r\n---------\r\n\r\nCo-authored-by: Sumit Kumar \u003cksumit@apache.org\u003e"
    },
    {
      "commit": "c3dd6457c45662de13be72919532ed0ef1b57ab3",
      "tree": "bdb40e7a75ae767a5a6e65d2870632665690023b",
      "parents": [
        "cb484ea66d80381ba28d6b3bdc84d1828084c84d"
      ],
      "author": {
        "name": "Sumit Kumar",
        "email": "ksumit@users.noreply.github.com",
        "time": "Tue May 16 11:09:49 2023 -0700"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue May 16 11:09:49 2023 -0700"
      },
      "message": " [LIVY-593] Add flags to support hadoop, spark and scala version specific builds (#405)\n\nEnable flags to build livy against specific hadoop, spark and scala versions. Currently supports:\r\n1. scala 2.11 and scala 2.12 with Spark 2 and Hadoop 2\r\n2. scala 2.12 with Spark 3 and hadoop 2 (Spark 3 doesn\u0027t come with scala 2.11 builds)\r\n\r\nCo-authored-by: Sumit Kumar \u003cksumit@apache.org\u003e"
    },
    {
      "commit": "cb484ea66d80381ba28d6b3bdc84d1828084c84d",
      "tree": "b6a6d6c2a2d8a9edf4cda162cc3aa016308587c3",
      "parents": [
        "7a6b194583946d618b827293368544c1f2abb578"
      ],
      "author": {
        "name": "Asif Khatri",
        "email": "123077165+askhatri@users.noreply.github.com",
        "time": "Tue May 09 13:11:28 2023 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue May 09 09:41:28 2023 +0200"
      },
      "message": "[LIVY-975][SERVER] Fix compatibility issue related to a change on the Spark side (#403)\n\n## What changes were proposed in this pull request?\r\n\r\nRecent change in spark required us to fix compatibility issue related to a change on the Spark side.\r\n\r\n## How was this patch tested?\r\n\r\nVerified manually by creating interactive / batch sessions via REST API call in a local Yarn cluster. Also, we have updated the unit tests."
    },
    {
      "commit": "7a6b194583946d618b827293368544c1f2abb578",
      "tree": "1ce617a128d235fd756abd6b6864a23430c5a732",
      "parents": [
        "12505dd88fac4c2b9da92a020205ffb72524a9c8"
      ],
      "author": {
        "name": "Asif Khatri",
        "email": "123077165+askhatri@users.noreply.github.com",
        "time": "Tue May 09 12:42:59 2023 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue May 09 09:12:59 2023 +0200"
      },
      "message": "LIVY-974: Remove verbose output on Livy UI error pages (#404)\n\n## What changes were proposed in this pull request?\r\n\r\nOn error, the Livy UI shows verbose output on error pages including the Jetty version number. This could be considered as a security vulnerability. We can make it configurable and avoid sending server version details. The Jetty version is there in every response header as well.\r\n\r\n## How was this patch tested?\r\n\r\nVerified manually by calling get sessions via REST API call in a local Yarn cluster."
    },
    {
      "commit": "12505dd88fac4c2b9da92a020205ffb72524a9c8",
      "tree": "26670d5d38ea8b18d7f7f9be7b78b0b9004012e9",
      "parents": [
        "80c6303b5bc7ac5d07281b0366dc2c4a2d7f082a"
      ],
      "author": {
        "name": "Damon P. Cortesi",
        "email": "d.lifehacker@gmail.com",
        "time": "Wed Apr 26 10:14:29 2023 -0700"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Apr 26 10:14:29 2023 -0700"
      },
      "message": "[LIVY-878] Replace log4j with reload4j (#392)\n\n* Replace log4j with reload4j along with relevant exclusions in dependencies\r\n* Bump surefire to fix build\r\n* Swap slf4j-api with slf4j-reload4j"
    },
    {
      "commit": "80c6303b5bc7ac5d07281b0366dc2c4a2d7f082a",
      "tree": "b3ea69b3029aab16b23f17240b5da7912c739d32",
      "parents": [
        "830a27e16efa9d1e6265988d7d71dc1de6ed2b39"
      ],
      "author": {
        "name": "Asif Khatri",
        "email": "123077165+askhatri@users.noreply.github.com",
        "time": "Tue Apr 18 17:45:55 2023 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Apr 18 14:15:55 2023 +0200"
      },
      "message": "[LIVY-967][SERVER] Return session information with livy sessions APIs (#383)\n\n## What changes were proposed in this pull request?\r\n\r\nCurrently livy GET /Sessions doesn\u0027t return fields like driver-executor memory, spark configuration, etc. Ideally a session response should return all the values set in session request api call(POST /sessions).\r\n\r\nJIRA: https://issues.apache.org/jira/browse/LIVY-967\r\n\r\n## How was this patch tested?\r\n\r\nVerified manually by creating interactive session via REST API call in a local Yarn cluster. Also, we have updated the unit tests."
    },
    {
      "commit": "830a27e16efa9d1e6265988d7d71dc1de6ed2b39",
      "tree": "753435d856963fa4eb53acc498d56d606a2d61bb",
      "parents": [
        "e4c2676c161edf585bf9bb543a9277aaf92283a5"
      ],
      "author": {
        "name": "RajshekharMuchandi",
        "email": "rsmuchandi@gmail.com",
        "time": "Wed Apr 12 13:26:39 2023 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Wed Apr 12 09:56:39 2023 +0200"
      },
      "message": "[LIVY-973][RSC] Always close RSCClient\u0027s EventLoopGroup (#398)\n\n## What changes were proposed in this pull request?\r\n\r\nAdded try-catch block to jobs\u0027 setFailure() and log entries if there is any exception. eventLoopGroup.shutdownGracefully(); statement will be executed after exception is caught."
    },
    {
      "commit": "e4c2676c161edf585bf9bb543a9277aaf92283a5",
      "tree": "896153335779d78c31a9f0b629b84ff61d1911c7",
      "parents": [
        "64521ce806c8263cd243e221982b026edd9cdf28"
      ],
      "author": {
        "name": "Asif Khatri",
        "email": "123077165+askhatri@users.noreply.github.com",
        "time": "Tue Apr 11 17:44:33 2023 +0530"
      },
      "committer": {
        "name": "GitHub",
        "email": "noreply@github.com",
        "time": "Tue Apr 11 14:14:33 2023 +0200"
      },
      "message": "[LIVY-970][SERVER] Ordering and pagination support in GET /statement (#389)\n\n## What changes were proposed in this pull request?\r\n\r\nGET /sessions/id/statements returns a list of statements run in a session. However the ordering of the statements are older ones first. Livy could expose a parameter (something like orderBy\u003dlatest) that will order the list of statements as latest first.\r\nJIRA: https://issues.apache.org/jira/browse/LIVY-970\r\n\r\n## How was this patch tested?\r\n\r\nVerified manually by creating interactive session and statements via REST API call in a local Yarn cluster."
    }
  ],
  "next": "64521ce806c8263cd243e221982b026edd9cdf28"
}
