Building and Updating MXNet Documentation

The website is hosted at https://mxnet.apache.org/. https://mxnet.io redirects to this site and advised to use links with https://mxnet.apache.org/ instead of https://mxnet.io/.

Website & Documentation Contributions

Detailed information on website development, continuous integration, and proposals for future projects can be found on the MXNet Wiki.

The website is built using Jekyll. You may run your own version of the static website by following the instructions on the wiki.

Each language documentation is built in a modular way, so that if you are a contributor to Julia, for example, you only need Julia-related tools to build it. Each language API has a section on installation and building along with how to build the docs locally.

You can also use the project's CI tools to emulate any changes with Docker. You can use these tools to install dependencies and run the parts of the build you want to test.

Refer to the MXNet Developer Wiki for instructions on building the docs locally.

If you plan to contribute changes to the documentation or website, please submit a pull request. Contributions are welcome!

Python Docs

MXNet's Python documentation is built with Sphinx and a variety of plugins including pandoc, and recommonmark.

More information on the dependencies can be found in the CI folder's installation scripts.

You can run just the Python docs by following the instructions in the Python API guide.

Other API Docs

The docs are hosted on the website in each language API's section. You can find installation and build instructions there.

How to Build the MXNet Website for Development and QA

conda or miniconda is recommended.

If you only need to make changes to tutorials or other pages that are not generated from one of the API source code folders, then you can use a basic Python pip or conda installation. But if you want edit the API source and have the reference API docs update, you also need to build MXNet from source. Refer to the build from source instructions for this requirement.

Ubuntu Setup

As this is maintained for CI, Ubuntu is recommended. Refer to ubuntu_doc.sh for the latest install script.

Caveat for Rendering Outputs

Note that without a GPU you will not be able to generate the docs with the outputs in the tutorials.

GPU setup

To run the full build, including tests of all tutorials, you will need at least two GPUs. Distributed training is a key feature of MXNet, so multiple GPUs are required for running through every tutorial.

CPU-only setup

In the environment.yml file:

  • Change mxnet-cu92 to mxnet.

macOS setup

In the environment.yml file:

  • Change mxnet-cu92 to mxnet. (There is no CUDA package for mac anyway.)

Windows Setup

If you have a GPU and have installed CUDA 9.2 you can leave the MXNet dependency alone. Otherwise, in the environment.yml file:

  • Change mxnet-cu92 to mxnet.

Install recommended software:

  • git bash
  • Be sure to install Conda in PATH
  • Install make from a git bash terminal with Admin rights
  • Restart terminals after installations to make sure PATH is set.
    • The choco, make, and conda commands should work in git bash.

Conda environment setup

Run the following commands from the project root (new-docs) to setup the environment.

conda env create -f environment.yml
source activate mxnet-docs

Build the docs

  • Change directories to new-docs/python.

To build without GPUs and without testing the notebooks (faster):

make EVAL=0

To build with testing the notebooks (requires GPU):

make

The build docs will be available at build/_build/html.

Each build may take a few minutes even without evaluation. To accelerate it, we can use one of the following ways:

  1. open build/conf.py, add the folders you want to skip into exclude_patterns, such as exclude_patterns = ['templates', 'api', 'develop', 'blog'].
  2. move the files into a different folder, such as mv api /tmp/, and then make clean.

Check results

To run a server to see the website:

  1. Start a http server: cd build/_build/html; python -m http.server
  2. For viewing a remote machine, ssh to your machine with port forwarding: ssh -L8000:localhost:8000 your_machine
  3. Open http://localhost:8000 in your local machine

Run tutorials

In addition to view the built html pages, you can run the Jupyter notebook from a remote machine.

  1. Install notedown plugin: pip install https://github.com/mli/notedown/tarball/master in remote server
  2. Start Jupyter notebook jupyter notebook --NotebookApp.contents_manager_class='notedown.NotedownContentsManager' in remote server
  3. ssh to your machine with port forwarding: ssh -L8888:localhost:8888 your_machine
  4. Open http://localhost:8888 in your local machine and run the md files directly

Optionally, one can run the following to launch the notedown plugin automatically when starting jupyter notebook.

  1. Generate the jupyter configure file ~/.jupyter/jupyter_notebook_config.py if it is not existing by run jupyter notebook --generate-config
  2. Add c.NotebookApp.contents_manager_class = 'notedown.NotedownContentsManager' to ~/.jupyter/jupyter_notebook_config.py
  3. Simply run jupyter notebook

Troubleshooting

Dependencies and the setup steps for this website are changing often. Here are some troubleshooting tips.

  • You might need to update the environment for the latest modules.
conda env update -f environment.yml

The -W Sphinx option enforces “warnings as errors”. This will help you debug your builds and get them through CI. CI will not let a PR through if it breaks the website. Refer to the MXNet Developer wiki's documentation guide for troubleshooting tips.

Production Website Deployment Process

Apache Jenkins MXNet website building job is used to build MXNet website.

The Jenkins docs build job will fetch MXNet repository, build MXNet website and push all static files to host repository.

The host repo is hooked with Apache gitbox to host website.

Processes for Running the Docs Build Jobs

This information is maintained on the MXNet Wiki.

Other Docs Build Processes

  • Perl API docs are maintained separately at metacpan.

Troubleshooting

  • If C++ code has been changed, remove the previous results to trigger the rebuild for all pages. To do this, run make clean_docs.
  • If C++ code fails to build, run make clean.
  • If CSS or javascript are changed, clear the cache in the browser with a forced refresh.
  • If search doesn't work, run make clean and then make docs.