Google Git
Sign in
apache / spark-kubernetes-operator / 83be95338c26bb7114325fdc785fbc87af066a8a
commit83be95338c26bb7114325fdc785fbc87af066a8a[log] [tgz]
authorDongjoon Hyun <dongjoon@apache.org>Sat Aug 17 11:47:41 2024 -0700
committerDongjoon Hyun <dongjoon@apache.org>Sat Aug 17 11:47:41 2024 -0700
treea4552e7c65824d8dbce6cc2f8814dd88a23b8624
parentc862e2cba715117aef67402b0ca41765006aa473 [diff]
Revert "[SPARK-49272] Update `Dockerfile` to use `alpine/java:17-jdk` instead of `gradle:8.9.0-jdk17-jammy`"

This reverts commit c862e2cba715117aef67402b0ca41765006aa473.
2 files changed
tree: a4552e7c65824d8dbce6cc2f8814dd88a23b8624
  1. .github/
  2. build-tools/
  3. config/
  4. dev/
  5. examples/
  6. gradle/
  7. spark-operator/
  8. spark-operator-api/
  9. spark-submission-worker/
  10. .asf.yaml
  11. .gitignore
  12. build.gradle
  13. gradle.properties
  14. gradlew
  15. LICENSE
  16. README.md
  17. settings.gradle
README.md

Apache Spark K8s Operator

Apache Spark K8s Operator is a subproject of Apache Spark and aims to extend K8s resource manager to manage Apache Spark applications via Operator Pattern.

GitHub Actions Build

Building Spark K8s Operator

Spark K8s Operator is built using Gradle. To build, run:

$ ./gradlew build -x test

Running Tests

$ ./gradlew build

Build Docker Image

$ docker build --build-arg APP_VERSION=0.1.0 -t spark-kubernetes-operator:0.1.0 -f build-tools/docker/Dockerfile  .

Install Helm Chart

$ ./gradlew spark-operator-api:relocateGeneratedCRD

$ helm install spark-kubernetes-operator --create-namespace -f build-tools/helm/spark-kubernetes-operator/values.yaml build-tools/helm/spark-kubernetes-operator/

Run Spark Pi App

$ kubectl apply -f examples/pi.yaml

$ kubectl get sparkapp
NAME   CURRENT STATE      AGE
pi     ResourceReleased   4m10s

$ kubectl delete sparkapp/pi

Contributing

Please review the Contribution to Spark guide for information on how to get started contributing to the project.