tag | e38075e805f76d6b23e4e05add795062e2d09e1e | |
---|---|---|
tagger | Kevin Su <pingsutw@apache.org> | Mon Apr 25 08:35:44 2022 +0800 |
object | 1f61aa9336b418ee9f6fc919cae0be6b5e946721 |
Submarine 0.7.0 release
commit | 1f61aa9336b418ee9f6fc919cae0be6b5e946721 | [log] [tgz] |
---|---|---|
author | featherchen <garychen0975321741@gmail.com> | Tue Apr 19 14:47:08 2022 +0800 |
committer | Kevin <pingsutw@apache.org> | Wed Apr 20 17:24:22 2022 +0000 |
tree | 6f832e4c275160921eab77e9174737c19cba1c79 | |
parent | 9a9c7100dfa95c5126c15aec82092bb899565dac [diff] |
SUBMARINE-1257. Show submarine's paper in Submarine github repo and website ### What is this PR for? <!-- A few sentences describing the overall goals of the pull request's commits. First time? Check out the contributing guide - https://submarine.apache.org/contribution/contributions.html --> Show [submarine's paper](https://dl.acm.org/doi/abs/10.1145/3517207.3526984) in Submarine GitHub repo and website ### What type of PR is it? Improvement ### Todos ### What is the Jira issue? <!-- * Open an issue on Jira https://issues.apache.org/jira/browse/SUBMARINE/ * Put link here, and add [SUBMARINE-*Jira number*] in PR title, eg. `SUBMARINE-23. PR title` --> https://issues.apache.org/jira/browse/SUBMARINE-1257 ### How should this be tested? <!-- * First time? Setup Travis CI as described on https://submarine.apache.org/contribution/contributions.html#continuous-integration * Strongly recommended: add automated unit tests for any new or changed behavior * Outline any manual steps to test the PR here. --> No need. ### Screenshots (if appropriate) ![Screenshot from 2022-04-19 14-39-40](https://user-images.githubusercontent.com/57944334/163941662-5f131668-e71c-4647-8157-c30b9851b3c2.png) ### Questions: * Do the license files need updating? No * Are there breaking changes for older versions? No * Does this need new documentation? No Author: featherchen <garychen0975321741@gmail.com> Signed-off-by: Kevin <pingsutw@apache.org> Closes #938 from featherchen/SUBMARINE-1257 and squashes the following commits: 525c8a37 [featherchen] Add README 5f8ea6c5 [featherchen] SUBMARINE-1257. Show submarine's paper in Submarine github repo and website b840749c [featherchen] SUBMARINE-1257
Apache Submarine (Submarine for short) is an End-to-End Machine Learning Platform to allow data scientists to create end-to-end machine learning workflows. On Submarine, data scientists can finish each stage in the ML model lifecycle, including data exploration, data pipeline creation, model training, serving, and monitoring.
Some open-source and commercial projects are trying to build an end-to-end ML platform. What's the vision of Submarine?
Theodore Levitt once said:
“People don’t want to buy a quarter-inch drill. They want a quarter-inch hole.”
experiment
on prem or cloud via easy-to-use UI/API/SDK.experiment
and dependencies of environment
.As mentioned above, Submarine attempts to provide Data-Scientist-friendly UI to make data scientists have a good user experience. Here're some examples.
# New a submarine client of the submarine server submarine_client = submarine.ExperimentClient(host='http://localhost:8080') # The experiment's environment, could be Docker image or Conda environment based environment = EnvironmentSpec(image='apache/submarine:tf-dist-mnist-test-1.0') # Specify the experiment's name, framework it's using, namespace it will run in, # the entry point. It can also accept environment variables. etc. # For PyTorch job, the framework should be 'Pytorch'. experiment_meta = ExperimentMeta(name='mnist-dist', namespace='default', framework='Tensorflow', cmd='python /var/tf_dist_mnist/dist_mnist.py --train_steps=100') # 1 PS task of 2 cpu, 1GB ps_spec = ExperimentTaskSpec(resources='cpu=2,memory=1024M', replicas=1) # 1 Worker task worker_spec = ExperimentTaskSpec(resources='cpu=2,memory=1024M', replicas=1) # Wrap up the meta, environment and task specs into an experiment. # For PyTorch job, the specs would be "Master" and "Worker". experiment_spec = ExperimentSpec(meta=experiment_meta, environment=environment, spec={'Ps':ps_spec, 'Worker': worker_spec}) # Submit the experiment to submarine server experiment = submarine_client.create_experiment(experiment_spec=experiment_spec) # Get the experiment ID id = experiment['experimentId']
submarine_client.get_experiment(id)
submarine_client.wait_for_finish(id)
submarine_client.get_log(id)
submarine_client.list_experiments(status='running')
For a quick-start, see Submarine On K8s
(Available on 0.7.0, see Roadmap)
If you want to know more about Submarine's architecture, components, requirements and design doc, they can be found on Architecture-and-requirement
Detailed design documentation, implementation notes can be found at: Implementation notes
Read the Apache Submarine Community Guide
How to contribute Contributing Guide
Issue Tracking: https://issues.apache.org/jira/projects/SUBMARINE
What to know more about what's coming for Submarine? Please check the roadmap out: https://cwiki.apache.org/confluence/display/SUBMARINE/Roadmap
Apache submarine: a unified machine learning platform made simple at EuroMLSys '22
The Apache Submarine project is licensed under the Apache 2.0 License. See the LICENSE file for details.