Google Git Google Git
Sign in
apache/spark/refs/heads/master/./dev
tree: 1ca4873e19a29886bbf1e68e1412b6e180fa221a [search]
  1. create-release/
  2. deps/
  3. spark-test-image/
  4. spark-test-image-util/
  5. sparktestsupport/
  6. .rat-excludes
  7. .scalafmt.conf
  8. change-scala-version.sh
  9. check-license
  10. check-protos.py
  11. check_pyspark_custom_errors.py
  12. checkstyle-suppressions.xml
  13. checkstyle.xml
  14. connect-gen-protos.sh
  15. connect-jvm-client-mima-check
  16. create_jira_and_branch.py
  17. create_spark_jira.py
  18. dispatch-if-changed.sh
  19. eslint.js
  20. free_disk_space
  21. free_disk_space_container
  22. gen-protos.sh
  23. generate_srs_registry.py
  24. is-changed.py
  25. java-file-header
  26. lint-java
  27. lint-js
  28. lint-python
  29. lint-r
  30. lint-r.R
  31. lint-scala
  32. make-distribution.sh
  33. merge_spark_pr.py
  34. mima
  35. next_version_candidates.py
  36. package-lock.json
  37. package.json
  38. pip-sanity-check.py
  39. pr_merge_status.py
  40. protobuf-breaking-changes-check.sh
  41. py-cleanup
  42. README.md
  43. reformat-python
  44. requirements.txt
  45. run-pip-tests
  46. run-tests
  47. run-tests.py
  48. sbt-checkstyle
  49. scalafmt
  50. scalastyle
  51. spark_jira_utils.py
  52. spark_merge_footer.py
  53. streaming-gen-protos.sh
  54. structured_logging_style.py
  55. test-dependencies.sh
dev/README.md

Spark Developer Scripts

This directory contains scripts useful to developers when packaging, testing, or committing to Spark.

Many of these scripts require Apache credentials to work correctly.

Testing sparktestsupport

The tests in sparktestsupport/tests/, which include the doctests of every sparktestsupport module, run automatically in CI and in dev/run-tests. To run them directly:

python3 -m unittest discover -s dev/sparktestsupport/tests -t dev

Keep short usage examples as doctests. Put regression cases, module-graph invariants, and tests requiring setup or mocks in the test files.