This document summarizes the information relevant to Gearpump committers and contributors. It includes information about the development processes and policies as well as the tools we use to facilitate those.
Table of Contents
If you are reading this document then you are interested in contributing to the Gearpump project -- many thanks for that! All contributions are welcome: ideas, documentation, code, patches, bug reports, feature requests, etc.
This section explains how to make a contribution.
To report a bug you should open an issue in our issue tracker that summarizes the bug. Set the form field “Issue type” to “Bug”. If you have not used the issue tracker before you will need to register an account (free), log in, and then click on the red “Create Issue” button in the top navigation bar.
In order to help us understand and fix the bug it would be great if you could provide us with:
Feel free to search the issue tracker for existing issues (aka tickets) that already describe the problem; if there is such a ticket please add your information as a comment.
If you want to provide a patch along with your bug report: That is great! In this case please send us a pull request as described in section Create a pull request below. You can also opt to attach a patch file to the issue ticket, but we prefer pull requests because they are easier to work with.
To request a new feature you should open an issue in our issue tracker and summarize the desired functionality. Set the form field “Issue type” to “New feature”. If you have not used the issue tracker before you will need to register an account (free), log in, and then click on the red “Create Issue” button in the top navigation bar.
Before you set out to contribute code we recommend that you familiarize yourself with the Gearpump codebase and corresponding development document at Gearpump website.
If you are interested in contributing code to Gearpump but do not know where to begin: In this case you should browse our issue tracker for open issues and tasks.
Contributions to the Gearpump codebase should be sent as GitHub pull requests. See section Create a pull request below for details. If there is any problem with the pull request we can iterate on it using the commenting features of GitHub.
For small patches, feel free to submit pull requests directly for those patches.
For larger code contributions, please use the following process. The idea behind this process is to prevent any wasted work and catch design issues early on.
Before working on code contribution, you need to prepare your development environment.
To work on a code contribution, following process is suggested:
git branch branch_issueId git checkout branch_issueId
Work on the branch, make changes, then push to your forked Gearpump repo.
git push origin branch_issueId
When there are changes in upstream/master, rebase your work on upstream/master.
git checkout branch_issueId git fetch upstream git rebase -i upstream/master
You can also use this when you want to squash(merge) multiple commits into one. git rebase -i will pop up a window, which allows you to squash(merge) multiple commits into one commit. For example I might have 12 commits in my branch. git rebase -i upstream/master opens a nice editor where you can mark some commits to be squashed(merged) into prior commits, and make 1 big commit (or several) out of it. In this way, I can tidy up what will be committed to the project master's history since otherwise my commit messages are like “not working” or “got it working” or “more fix”.
If there is conflict, resolve the conflict, and then
git rebase --continue
After the code is successfully rebased, a window will pop up to edit the commit log, edit it then save and exit. 4. After rebase, now you have a clean log history. push to your remote working branch
git push origin branch_issueId
If commits have already been pushed to your forked repository on GitHub, you will have to “git push --force” to overwrite them with squashed commits.
git push origin -f branch_issueId
Committer will review your code periodically. When there is any comment/feedback from committer(s), it‘s contributor’s duty to update the pull request correspondingly.
When the merge is done by committer, you can optionally delete your PR branch.
Though without a development environment setup, you can still contribute to Gearpump by reporting ideas, documentations, bugs and feature requests. It is highly recommended to set up a development environment to make any code contribution.
If you just want to study Gearpump source code, it is optional to perform following steps. But, if you plan to contribute to Gearpump's code base, it is necessary to perform following steps:
Fork https://github.com/apache/incubator-gearpump to your own repo at https://github.com/your_username/incubator-gearpump.
Clone the forked repo at your computer.
git clone https://github.com/your_username/incubator-gearpump cd incubator-gearpump
git remote add upstream https://github.com/apache/incubator-gearpump.git
git pull --rebase upstream master git push origin master
Another way to do this is to
git checkout master git fetch upstream git rebase upstream/master
No development work should ever be done in the forked master.
To make a compilation of Gearpump, you can execute:
sbt compile pack
To build a Gearpump package, you can execute following commands:
## The target package path: output/target/gearpump-${version}.zip sbt clean +assembly +packArchiveZip
After the build, there will be a package file gearpump-${version}.zip generated under output/target/ folder.
NOTE: The build requires network connection. If you are behind a proxy, make sure you have set the proxy in your env before running the build commands.
For Windows:
set HTTP_PROXY=http://host:port set HTTPS_PROXY= http://host:port
For Linux:
export HTTP_PROXY=http://host:port export HTTPS_PROXY= http://host:port
Unit tests and Integration tests are an essential part of code contributions.
To run unit test, you can run
sbt test
Gearpump has an integration test system which is based on Docker. Please check the instructions.
To build Gearpump document use
## the 2nd argument indicates whether to build API doc or not ## for release, we need to use '1' to build it (time consuming) docs/build_doc.sh 2.11 0
IDE environment can be set up on either Windows, Linux and Mac platform. You can choose the one you prefer. The IDE setup guide can be found at Gearpump website.
It is highly recommended to perform package build before IDE setup.
Gearpump follows the standard Scala style, just like other Scala projects. This style is:
Mainly, we follow the Spark Scala style.
We allow some exceptions: e.g. allowing using !, ? for Akka to send actor message.
Before submitting a PR, you should always run style check first:
## Run style check for compile, test, and integration test. sbt scalastyle test:scalastyle it:scalastyle
TBD
TBD
Documentation contributions are very welcome!
You can contribute documentation by pull request, same as code contribution. Main directory is docs/, and you can refer to docs/README.md for how to build / test documentation.
NOTE: this documentation is the release documentation for Gearpump, not the website documentation at Apache website, which is maintained by another repository.
This section applies to committers only.
It‘s committer’s duty to review pull requests from contributors.
Any PR ready to merge shall have at least one +1(s) and no -1(s) from other than the one who authored this PR. And any merge shall wait another 24 hours after the first +1 received to wait for potential comments. Only committer has the right to perform PR merge to Apache upstream.
All merges should be done using the dev-tools/merge_gearpump_pr.py script. To use this script, you will need to add a git remote called “apache” at https://git-wip-us.apache.org/repos/asf/incubator-gearpump.git, as well as one called “apache-github” at git://github.com/apache/incubator-gearpump. For the “apache” repo, you can authenticate using your ASF username and password.
The script is fairly self explanatory and walks you through following steps and options interactively.
If you want to amend a commit before merging – which should be used for trivial touch-ups – then simply let the script wait at the point where it asks you if you want to push to Apache. Then, in a separate window, modify the code and push a commit. Run “git rebase -i HEAD~2” and “squash” your new commit. Edit the commit message just after to remove your commit message. You can verify the result is one change with “git log”. Then resume the script in the other window. Also, please remember to set Assignee on JIRAs where applicable when they are resolved. The script can't do this automatically.
Follow release process for making a release.