website2/docs/extending-heron-scheduler.md

id: extending-heron-scheduler title: Implementing a Custom Scheduler sidebar_label: Custom Scheduler

To run a Heron topology, you’ll need to set up a scheduler that is responsible for topology management. Note: one scheduler is managing only one topology, for the purpose of better isolation. Heron currently supports the following schedulers out of the box:

If you‘d like to run Heron on a not-yet-supported system, such as Amazon ECS, you can create your own scheduler using Heron’s spi, as detailed in the sections below.

Java is currently the only supported language for custom schedulers. This may change in the future.

Java Setup

In order to create a custom scheduler, you need to import the heron-spi library into your project.

Maven

<dependency>
  <groupId>org.apache.heron</groupId>
  <artifactId>heron-spi</artifactId>
  <version>{{% heronVersion %}}</version>
</dependency>

Gradle

dependencies {
  compile group: "org.apache.heron", name: "heron-spi", version: "{{% heronVersion %}}"
}

Interfaces

Creating a custom scheduler involves implementing each of the following Java interfaces:

Interface	Role	Examples
`IPacking`	Defines the algorithm used to generate physical plan for a topology.	RoundRobin
`ILauncher`	Defines how the scheduler is launched	Aurora, local
`IScheduler`	Defines the scheduler object used to construct topologies	local
`IUploader`	Uploads the topology to a shared location accessible to the runtime environment of the topology	local HDFS S3

Heron provides a number of built-in implementations out of box.

Running the Scheduler

To run the a custom scheduler, the implementation of the interfaces above must be specified in the config. By default, the heron-cli looks for configurations under ${HERON_HOME}/conf/. The location can be overridden using option --config-path. Below is an example showing the command for topology submission:

$ heron submit [cluster-name-storing-your-new-config]/[role]/[env] \
    --config-path [config-folder-path-storing-your-new-config] \
    /path/to/topology/my-topology.jar \
    biz.acme.topologies.MyTopology

The implementation for each of the interfaces listed above must be on Heron's classpath.