This guide describes how to use Jupyter notebook in Submarine to launch and manage Jupyter notebooks.
We recommend using Web UI to manage notebooks.
Notebooks can be started from the Web UI. You can click the “Notebook” tab in the
left-hand panel to manage your notebooks.
To create a new notebook server, click “New Notebook”. You should see a form for entering
details of your new notebook server.
If you’re not sure which environment you need, please choose the environment “notebook-env”
for the new notebook.
You should see your new notebook server. Click the name of your notebook server to connect to it.
The environment “notebook-env” includes Submarine Python SDK which can talk to Submarine Server to
create experiments, as the example below:
from __future__ import print_function import submarine from submarine.experiment.models.environment_spec import EnvironmentSpec from submarine.experiment.models.experiment_spec import ExperimentSpec from submarine.experiment.models.experiment_task_spec import ExperimentTaskSpec from submarine.experiment.models.experiment_meta import ExperimentMeta from submarine.experiment.models.code_spec import CodeSpec # Create Submarine Client submarine_client = submarine.ExperimentClient() # Define TensorFlow experiment spec environment = EnvironmentSpec(image='apache/submarine:tf-dist-mnist-test-1.0') experiment_meta = ExperimentMeta(name='mnist-dist', namespace='default', framework='Tensorflow', cmd='python /var/tf_dist_mnist/dist_mnist.py --train_steps=100', env_vars={'ENV1': 'ENV1'}) worker_spec = ExperimentTaskSpec(resources='cpu=1,memory=1024M', replicas=1) ps_spec = ExperimentTaskSpec(resources='cpu=1,memory=1024M', replicas=1) code_spec = CodeSpec(sync_mode='git', url='https://github.com/apache/submarine.git') experiment_spec = ExperimentSpec(meta=experiment_meta, environment=environment, code=code_spec, spec={'Ps' : ps_spec,'Worker': worker_spec}) # Create experiment experiment = submarine_client.create_experiment(experiment_spec=experiment_spec)
You can create a new notebook, paste the above code and run it. Or, you can find the notebook submarine_experiment_sdk.ipynb
inside the launched notebook session. You can open it, try it out.
After experiment submitted to Submarine server, you can find the experiment jobs on the UI.