commit | b0ea9583d24418653ef341681d68fedb40219dc6 | [log] [tgz] |
---|---|---|
author | kobe860219 <kobe860219@gmail.com> | Tue Apr 13 00:12:50 2021 +0800 |
committer | Liu Xun <liuxun@apache.org> | Thu Apr 22 09:35:35 2021 +0000 |
tree | 304b1696e8e9b654e21f7555fc26ab0674dfbe30 | |
parent | 2e6c387d461dfa4a066a50640327bcf5f55e43a8 [diff] |
SUBMARINE-722. [WEB] Add template page on workbench ### What is this PR for? Complete template page on workbench. This pr is a UI for users to register new pre-defined experiment template. Now user could register a pre-defined experiment template, then run experiments quickly with different hyper-parameters. ### What type of PR is it? [Feature] ### Todos * [ ] - templateIT ### What is the Jira issue? https://issues.apache.org/jira/browse/SUBMARINE-722 ### How should this be tested? https://github.com/kobe860219/submarine/runs/2325676029 ### Screenshots (if appropriate) <img width="1440" alt="截圖 2021-04-13 上午12 18 14" src="https://user-images.githubusercontent.com/48027290/114427696-eebf4900-9bed-11eb-8790-0ec75a0e347f.png"> <img width="1440" alt="截圖 2021-04-13 上午12 18 23" src="https://user-images.githubusercontent.com/48027290/114427715-f252d000-9bed-11eb-9fec-51b5ffaf0d4d.png"> <img width="1440" alt="截圖 2021-04-13 上午12 18 38" src="https://user-images.githubusercontent.com/48027290/114427720-f383fd00-9bed-11eb-83af-e144914674da.png"> <img width="1440" alt="截圖 2021-04-13 上午12 18 47" src="https://user-images.githubusercontent.com/48027290/114427728-f41c9380-9bed-11eb-9d4d-0d9b82da4bc0.png"> <img width="1440" alt="截圖 2021-04-13 上午12 19 03" src="https://user-images.githubusercontent.com/48027290/114427732-f54dc080-9bed-11eb-82bc-09e7d24f02e3.png"> <img width="1440" alt="截圖 2021-04-13 上午12 19 13" src="https://user-images.githubusercontent.com/48027290/114427736-f5e65700-9bed-11eb-80c2-8700aa0458bb.png"> ### Questions: * Does the licenses files need update? No * Is there breaking changes for older versions? No * Does this needs documentation? No Author: kobe860219 <kobe860219@gmail.com> Signed-off-by: Liu Xun <liuxun@apache.org> Closes #555 from kobe860219/SUBMARINE-722 and squashes the following commits: 48da7b6 [kobe860219] Add image column fc9e695 [kobe860219] Template page done 20569a4 [kobe860219] Init form 6536789 [kobe860219] Delete done a4cf16d [kobe860219] Implement template info page. ce0b758 [kobe860219] Init template page
Apache Submarine (Submarine for short) is an End-to-End Machine Learning Platform to allow data scientists to create end-to-end machine learning workflows. On Submarine, data scientists can finish each stage in the ML model lifecycle, including data exploration, data pipeline creation, model training, serving, and monitoring.
Some open-source and commercial projects are trying to build an end-to-end ML platform. What's the vision of Submarine?
Theodore Levitt once said:
“People don’t want to buy a quarter-inch drill. They want a quarter-inch hole.”
experiment
on prem or cloud via easy-to-use UI/API/SDK.experiment
and dependencies of environment
As mentioned above, Submarine attempts to provide Data-Scientist-friendly UI to make data scientists have a good user experience. Here're some examples.
# New a submarine client of the submarine server submarine_client = submarine.ExperimentClient(host='http://localhost:8080') # The experiment's environment, could be Docker image or Conda environment based environment = EnvironmentSpec(image='apache/submarine:tf-dist-mnist-test-1.0') # Specify the experiment's name, framework it's using, namespace it will run in, # the entry point. It can also accept environment variables. etc. # For PyTorch job, the framework should be 'Pytorch'. experiment_meta = ExperimentMeta(name='mnist-dist', namespace='default', framework='Tensorflow', cmd='python /var/tf_dist_mnist/dist_mnist.py --train_steps=100') # 1 PS task of 2 cpu, 1GB ps_spec = ExperimentTaskSpec(resources='cpu=2,memory=1024M', replicas=1) # 1 Worker task worker_spec = ExperimentTaskSpec(resources='cpu=2,memory=1024M', replicas=1) # Wrap up the meta, environment and task specs into an experiment. # For PyTorch job, the specs would be "Master" and "Worker". experiment_spec = ExperimentSpec(meta=experiment_meta, environment=environment, spec={'Ps':ps_spec, 'Worker': worker_spec}) # Submit the experiment to submarine server experiment = submarine_client.create_experiment(experiment_spec=experiment_spec) # Get the experiment ID id = experiment['experimentId']
submarine_client.get_experiment(id)
submarine_client.wait_for_finish(id)
submarine_client.get_log(id)
submarine_client.list_experiments(status='running')
For a quick-start, see Submarine On K8s
(Available on 0.6.0, see Roadmap)
If you want to know more about Submarine's architecture, components, requirements and design doc, they can be found on Architecture-and-requirement
Detailed design documentation, implementation notes can be found at: Implementation notes
Read the Apache Submarine Community Guide
How to contribute Contributing Guide
Issue Tracking: https://issues.apache.org/jira/projects/SUBMARINE
What to know more about what's coming for Submarine? Please check the roadmap out: https://cwiki.apache.org/confluence/display/SUBMARINE/Roadmap
The Apache Submarine project is licensed under the Apache 2.0 License. See the LICENSE file for details.