Google Git
Sign in
apache / dolphinscheduler / refs/tags/3.1.6
tagc999ef4e44646c42a65ab7e035e2cef732ad908a
taggerzhuangchong <zhuang.kerwin@gmail.com>Tue May 09 21:46:46 2023 +0800
object7780fdc2f537aa6978d76ee62f2088f730b5969d 
[maven-release-plugin] copy for tag 3.1.6
commit7780fdc2f537aa6978d76ee62f2088f730b5969d[log] [tgz]
authorzhuangchong <zhuang.kerwin@gmail.com>Tue May 09 21:46:46 2023 +0800
committerzhuangchong <zhuang.kerwin@gmail.com>Tue May 09 21:46:46 2023 +0800
treeb008ef6530483e144c41b69ac4a04bc9cff84202
parentbe2157f5dad448234dfcd4e0bf3a6e5102a7d591 [diff]
[maven-release-plugin] prepare release 3.1.6
89 files changed
tree: b008ef6530483e144c41b69ac4a04bc9cff84202
  1. .github/
  2. .mvn/
  3. deploy/
  4. docs/
  5. dolphinscheduler-alert/
  6. dolphinscheduler-api/
  7. dolphinscheduler-api-test/
  8. dolphinscheduler-bom/
  9. dolphinscheduler-common/
  10. dolphinscheduler-dao/
  11. dolphinscheduler-data-quality/
  12. dolphinscheduler-datasource-plugin/
  13. dolphinscheduler-dist/
  14. dolphinscheduler-e2e/
  15. dolphinscheduler-master/
  16. dolphinscheduler-meter/
  17. dolphinscheduler-microbench/
  18. dolphinscheduler-python/
  19. dolphinscheduler-registry/
  20. dolphinscheduler-remote/
  21. dolphinscheduler-scheduler-plugin/
  22. dolphinscheduler-service/
  23. dolphinscheduler-spi/
  24. dolphinscheduler-standalone-server/
  25. dolphinscheduler-task-plugin/
  26. dolphinscheduler-tools/
  27. dolphinscheduler-ui/
  28. dolphinscheduler-worker/
  29. images/
  30. licenses/
  31. script/
  32. style/
  33. tools/
  34. .asf.yaml
  35. .dlc.json
  36. .flake8
  37. .gitattributes
  38. .gitignore
  39. .gitmodules
  40. .licenserc.yaml
  41. CONTRIBUTING.md
  42. LICENSE
  43. lombok.config
  44. mvnw
  45. mvnw.cmd
  46. NOTICE
  47. pom.xml
  48. README.md
  49. README_zh_CN.md
README.md

Apache Dolphinscheduler

License codecov Quality Gate Status Twitter Follow Slack Status CN doc

About

Apache DolphinScheduler is the modern data orchestration platform. Agile to create high performance workflow with low-code. It is also provided powerful user interface, dedicated to solving complex task dependencies in the data pipeline and providing various types of jobs available out of the box

The key features for DolphinScheduler are as follows:

  • Easy to deploy, provide four ways to deploy which including Standalone, Cluster, Docker and Kubernetes.
  • Easy to use, workflow can be created and managed by four ways, which including Web UI, Python SDK, Yaml file and Open API
  • Highly reliable and high availability, decentralized architecture with multi-master and multi-worker, native supports horizontal scaling.
  • High performance, its performance is N times faster than other orchestration platform and it can support tens of millions of tasks per day
  • Cloud Native, DolphinScheduler supports orchestrating multi-cloud/data center workflow, and supports custom task type
  • Versioning both workflow and workflow instance(including tasks)
  • Various state control of workflow and task, support pause/stop/recover them in any time
  • Multi-tenancy support
  • Others like backfill support(Web UI native), permission control including project, resource and data source

QuickStart

StabilityAccessibilityFeaturesScalability
Decentralized multi-master and multi-workerVisualization of workflow key information, such as task status, task type, retry times, task operation machine information, visual variables, and so on at a glance.  Support pause, recover operationSupport customized task types
support HAVisualization of all workflow operations, dragging tasks to draw DAGs, configuring data sources and resources. At the same time, for third-party systems, provide API mode operations.Users on DolphinScheduler can achieve many-to-one or one-to-one mapping relationship through tenants and Hadoop users, which is very important for scheduling large data jobs.The scheduler supports distributed scheduling, and the overall scheduling capability will increase linearly with the scale of the cluster. Master and Worker support dynamic adjustment.
Overload processing: By using the task queue mechanism, the number of schedulable tasks on a single machine can be flexibly configured. Machine jam can be avoided with high tolerance to numbers of tasks cached in task queue.One-click deploymentSupport traditional shell tasks, and big data platform task scheduling: MR, Spark, SQL (MySQL, PostgreSQL, hive, spark SQL), Python, Procedure, Sub_Process

User Interface Screenshots

  • Homepage: Project and workflow overview, including the latest workflow instance and task instance status statistics. home

  • Workflow Definition: Create and manage workflow by drag and drop, easy to build and maintain complex workflow, support bulk of tasks out of box. workflow-definition

  • Workflow Tree View: Abstract tree structure could clearer understanding of the relationship between tasks workflow-tree

  • Data source: Manage support multiple external data sources, provide unified data access capabilities for such as MySQL, PostgreSQL, Hive, Trino, etc. data-source

  • Monitor: View the status of the master, worker and database in real time, including server resource usage and load, do quick health check without logging in to the server. monitor

Suggestions & Bug Reports

Follow this guide to report your suggestions or bugs.

Contributing

The community welcomes everyone to contribute, please refer to this page to find out more: How to contribute, find the good first issue in here if you are new to DolphinScheduler.

Community

Welcome to join the Apache DolphinScheduler community by:

Landscapes