The Pulsar website and documentation

This README is basically the meta-documentation for the Pulsar website and documentation. Here you'll find instructions on running the site locally

Tools

This website is built using a wide range of tools. The most important among them:

• Jekyll is the static site generator
• CSS is compiled using Sass
• Swagger is used to create a JSON definition of Pulsar's admin REST API
• Doxygen is used to generate the C++ API docs
• pdoc is used to generate the Python API docs
• Javadoc is used to generate the Java API docs
• protoc-gen-doc is used to generate a JSON definition of Pulsar's Protocol Buffers interface (which is in turn used to auto-generate Protobuf docs)

Requirements and setup

Note: at the moment, running the site locally may work on Linux but the site setup has been built with MacOS in mind. We will provide better cross-platform support in the near future.

To build and run the site locally, you need to have Ruby 2.4.1 installed and set as the Ruby version here in the site directory. You can install and set the Ruby version using rvm:

$ cd site
$ rvm install .
$ rvm use .

Then you can install all Ruby dependencies, as well as Doxygen:

$ make setup

Running the site locally

To run the site locally:

$ make serve

This will start up Jekyll in live mode. As you make changes to the site or to the Sass files, the site will auto-update (and the browser will auto-refresh). There is typically a time lag of roughly 1.5 seconds for the refresh.

Building and publishing the site

Building and publishing the website is handled for you automatically at the CI level. No action is required on your part. If you push changes to the website to the master branch (via pull request), this will trigger a CI job that handles everything for you, unless you need to build one of the following locally:

• The Java, Python, or C++ API docs
• The Protobuf docs
• The REST API docs

If you'd like to view changes to those three things, you will need to build those parts of the site locally. Instructions for each are below.

Java API docs

To re-generate the Java API docs, you'll need to have the javadoc CLI tool installed and on your PATH (javadoc comes as part of most Java distributions). Then you need to run:

$ make javadoc

Python API docs

To re-generate the Python API docs, you'll need to have the pdoc and pygments libraries installed, which you can install by running:

$ make python_setup

To generate the Python docs:

$ make python_doc_gen

C++ API docs

To generate C++ API docs, you'll first need to install Doxygen. To do so:

$ make doxygen_install

Then generate the docs:

$ make cpp_doc_gen

Protobuf docs

To re-generate the Protobuf docs, you'll need to install the protoc-gen-doc tool:

$ make protobuf_setup

To re-generate the Protobuf docs:

$ make protobuf_doc_gen

REST API docs

The REST API docs are built using a JSON definition generated by Maven‘s Swagger plugin. This definition is automatically generated when Maven builds Pulsar. In the rare case when you do need to re-generate this JSON definition, run mvn package -P swagger to build all of Pulsar. Then you’ll need to copy the new definition into the site/_data folder using this command:

$ make swagger_definition_copy

Checking links

The htmltest tool is installed on MacOS when you run the make setup command. You can check links by running make linkcheck_macos. At the moment, linkchecking is performed ad hoc but may eventually be applied at the CI level.

Contributing

We are very much open to your contributions! Pulsar is a big and ever-changing project, and it takes a lot of combined effort to keep the website looking good and the documentation up to date. Just fork the project, make changes locally, and submit a pull request to this repository.