title: PySubmarine Tracking

It helps developers use submarine's internal data caching, data exchange, and task tracking capabilities to more efficiently improve the development and execution of machine learning productivity

  • Allow data scientist to track distributed ML experiemnt
  • Support store ML parameters and metrics in Submarine-server
  • Support hdfs, S3 and mysql (Currently we only support mysql)

Quickstart

  1. Start mini-submarine

  2. Start Mysql server in mini-submarine

  3. Uncomment the log_param and log_metric in mnist_distributed.py

  4. Start Submarine experiment (e.g., run_submarine_mnist_tony.sh)

Functions

submarine.get_tracking_uri()

return the tracking URI.

submarine.set_tracking_uri(URI)

set the tracking URI. You can also set the SUBMARINE_TRACKING_URI environment variable to have Submarine find a URI from there. The URI should be database connection string.

Parameters

  • URI - Submarine record data to Mysql server. The database URL is expected in the format <dialect>+<driver>://<username>:<password>@<host>:<port>/<database>. By default it's mysql+pymysql://submarine:password@localhost:3306/submarine. More detail : SQLAlchemy docs

submarine.log_metric(key, value, step=0)

logs a single key-value metric. The value must always be a number.

Parameters

  • key - Metric name (string).
  • value - Metric value (float).
  • step - A single integer step at which to log the specified Metrics, by default it's 0.

submarine.log_param(key, value)

logs a single key-value parameter. The key and value are both strings.

Parameters

  • key - Parameter name (string).
  • value - Parameter value (string).