| <!DOCTYPE HTML> |
| <html lang="en-US"> |
| <head> |
| <meta charset="UTF-8"> |
| <title>Developing Calcite</title> |
| <meta name="viewport" content="width=device-width,initial-scale=1"> |
| <meta name="generator" content="Jekyll v3.7.3"> |
| <link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900"> |
| <link rel="stylesheet" href="/css/screen.css"> |
| <link rel="icon" type="image/x-icon" href="/favicon.ico"> |
| <!--[if lt IE 9]> |
| <script src="/js/html5shiv.min.js"></script> |
| <script src="/js/respond.min.js"></script> |
| <![endif]--> |
| </head> |
| |
| |
| <body class="wrap"> |
| <header role="banner"> |
| <div class="grid"> |
| <div class="unit center-on-mobiles"> |
| <h1> |
| <a href="/"> |
| <span class="sr-only">Apache Calcite</span> |
| <img src="/img/logo.svg" alt="Calcite Logo"> |
| </a> |
| </h1> |
| </div> |
| <nav class="main-nav"> |
| <ul> |
| <li class=""> |
| <a href="/">Home</a> |
| </li> |
| <li class=""> |
| <a href="/downloads/">Download</a> |
| </li> |
| <li class=""> |
| <a href="/community/">Community</a> |
| </li> |
| <li class="current"> |
| <a href="/develop/">Develop</a> |
| </li> |
| <li class=""> |
| <a href="/news/">News</a> |
| </li> |
| <li class=""> |
| <a href="/docs/">Docs</a> |
| </li> |
| </ul> |
| |
| </nav> |
| </div> |
| </header> |
| |
| |
| <section class="standalone"> |
| <div class="grid"> |
| |
| <div class="unit whole"> |
| <article> |
| <h1>Developing Calcite</h1> |
| <!-- |
| |
| --> |
| |
| <p>Want to help add a feature or fix a bug?</p> |
| |
| <ul id="markdown-toc"> |
| <li><a href="#source-code" id="markdown-toc-source-code">Source code</a></li> |
| <li><a href="#download-source-build-and-run-tests" id="markdown-toc-download-source-build-and-run-tests">Download source, build, and run tests</a></li> |
| <li><a href="#contributing" id="markdown-toc-contributing">Contributing</a></li> |
| <li><a href="#null-safety" id="markdown-toc-null-safety">Null safety</a></li> |
| <li><a href="#continuous-integration-testing" id="markdown-toc-continuous-integration-testing">Continuous Integration Testing</a></li> |
| <li><a href="#getting-started" id="markdown-toc-getting-started">Getting started</a></li> |
| </ul> |
| |
| <h2 id="source-code">Source code</h2> |
| |
| <p>You can get the source code by |
| <a href="/downloads">downloading a release</a> |
| or from source control.</p> |
| |
| <p>Calcite uses git for version control. The canonical source is in |
| <a href="https://gitbox.apache.org/repos/asf/calcite.git">Apache</a>, |
| but most people find the |
| <a href="https://github.com/apache/calcite">Github mirror</a> more |
| user-friendly.</p> |
| |
| <h2 id="download-source-build-and-run-tests">Download source, build, and run tests</h2> |
| |
| <p>Prerequisites are Git, |
| and Java (JDK 8u220 or later, 11 preferred) on your path.</p> |
| |
| <p>Note: early OpenJDK 1.8 versions (e.g. versions before 1.8u202) are known to have issues with |
| producing bytecode for type annotations (see <a href="https://bugs.openjdk.java.net/browse/JDK-8187805">JDK-8187805</a>, |
| <a href="https://bugs.openjdk.java.net/browse/JDK-8187805">JDK-8187805</a>, |
| <a href="https://bugs.openjdk.java.net/browse/JDK-8210273">JDK-8210273</a>, |
| <a href="https://bugs.openjdk.java.net/browse/JDK-8160928">JDK-8160928</a>, |
| <a href="https://bugs.openjdk.java.net/browse/JDK-8144185">JDK-8144185</a> ), so make sure you use up to date Java.</p> |
| |
| <p>Create a local copy of the Git repository, <code class="highlighter-rouge">cd</code> to its root directory, |
| then build using Gradle:</p> |
| |
| <figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nv">$ </span>git clone git://github.com/apache/calcite.git |
| <span class="nv">$ </span><span class="nb">cd </span>calcite |
| <span class="nv">$ </span>./gradlew build</code></pre></figure> |
| |
| <p>The HOWTO describes how to |
| <a href="/docs/howto.html#building-from-a-source-distribution">build from a source distribution</a>, |
| <a href="/docs/howto.html#setting-up-an-ide-for-contributing">set up an IDE for contributing</a>, |
| <a href="/docs/howto.html#running-tests">run more or fewer tests</a> and |
| <a href="/docs/howto.html#running-integration-tests">run integration tests</a>.</p> |
| |
| <h2 id="contributing">Contributing</h2> |
| |
| <p>We welcome contributions.</p> |
| |
| <p>If you are planning to make a large contribution, talk to us first! It |
| helps to agree on the general approach. Log a |
| <a href="https://issues.apache.org/jira/browse/CALCITE">JIRA case</a> for your |
| proposed feature or start a discussion on the dev list.</p> |
| |
| <p>Before opening up a new JIRA case, have a look in the existing issues. |
| The feature or bug that you plan to work on may already be there.</p> |
| |
| <p>If a new issue needs to be created, it is important to provide a |
| concise and meaningful summary line. It should imply what the end user |
| was trying to do, in which component, and what symptoms were seen. |
| If it’s not clear what the desired behavior is, rephrase: e.g., |
| “Validator closes model file” to “Validator should not close model file”.</p> |
| |
| <p>Contributors to the case should feel free to rephrase and clarify the |
| summary line. If you remove information while clarifying, put it in |
| the description of the case.</p> |
| |
| <p>Design discussions may happen in various places (email threads, |
| github reviews) but the JIRA case is the canonical place for those |
| discussions. Link to them or summarize them in the case.</p> |
| |
| <p>When implementing a case, especially a new feature, make sure |
| the case includes a functional specification of the change. For instance, |
| “Add a IF NOT EXISTS clause to the CREATE TABLE command; the command is |
| a no-op if the table already exists.” Update the description if |
| the specification changes during design discussions or implementation.</p> |
| |
| <p>When implementing a feature or fixing a bug, endeavor to create |
| the jira case before you start work on the code. This gives others |
| the opportunity to shape the feature before you have gone too far down |
| (what the reviewer considers to be) the wrong path.</p> |
| |
| <p>The best place to ask for feedback related to an issue is the developers list. |
| Please avoid tagging specific people in the JIRA case asking for feedback. |
| This discourages other contributors to participate in the discussion and |
| provide valuable feedback.</p> |
| |
| <p>If there is a regression that seems to be related with a particular commit, |
| feel free to tag the respective contributor(s) in the discussion.</p> |
| |
| <p>If you are going to take on the issue right away assign it to yourself. |
| To assign issues to yourself you have to be registered in JIRA as a contributor. |
| In order to do that, send an email to the developers list |
| and provide your JIRA username.</p> |
| |
| <p>If you are committed to fixing the issue before the upcoming release set |
| the fix version accordingly (e.g., 1.20.0), otherwise leave it as blank.</p> |
| |
| <p>If you pick up an existing issue, mark it ‘in progress’, and when it’s |
| finished flag it with ‘pull-request-available’.</p> |
| |
| <p>If for any reason you decide that an issue cannot go into the ongoing |
| release, reset the fix version to blank.</p> |
| |
| <p>During a release, the release manager will update the issues that were |
| not completed for the current release to the next release.</p> |
| |
| <p>There are cases where the JIRA issue may be solved in the discussion |
| (or some other reason) without necessitating a change. In such cases, |
| the contributor(s) involved in the discussion should:</p> |
| <ul> |
| <li>resolve the issue (do not close it);</li> |
| <li>select the appropriate resolution cause (“Duplicate”, “Invalid”, “Won’t fix”, etc.);</li> |
| <li>add a comment with the reasoning if that’s not obvious.</li> |
| </ul> |
| |
| <p>Fork the GitHub repository, and create a branch for your feature.</p> |
| |
| <p>Develop your feature and test cases, and make sure that |
| <code class="highlighter-rouge">./gradlew build</code> succeeds. (Run extra tests if your change warrants it.)</p> |
| |
| <p>Commit your change to your branch, and use a comment that starts with |
| the JIRA case number, like this:</p> |
| |
| <figure class="highlight"><pre><code class="language-text" data-lang="text">[CALCITE-345] AssertionError in RexToLixTranslator comparing to date literal (FirstName LastName)</code></pre></figure> |
| |
| <p>If your change had multiple commits, use <code class="highlighter-rouge">git rebase -i master</code> to |
| squash them into a single commit, and to bring your code up to date |
| with the latest on the main line.</p> |
| |
| <p>In order to keep the commit history clean and uniform, you should |
| respect the following guidelines.</p> |
| <ul> |
| <li>Read the messages of previous commits, and follow their style.</li> |
| <li>The first line of the commit message must be a concise and useful |
| description of the change.</li> |
| <li>The message is often, but not always, the same as the JIRA subject. |
| If the JIRA subject is not clear, change it (perhaps move the original |
| subject to the description of the JIRA case, if it clarifies).</li> |
| <li>Start with a capital letter.</li> |
| <li>Do not finish with a period.</li> |
| <li>Use imperative mood (“Add a handler …”) rather than past tense |
| (“Added a handler …”) or present tense (“Adds a handler …”).</li> |
| <li>If possible, describe the user-visible behavior that you changed |
| (“FooCommand now creates directory if it does not exist”), rather than |
| the implementation (“Add handler for FileNotFound”).</li> |
| <li>If you are fixing a bug, it is sufficient to describe the bug |
| (“NullPointerException if user is unknown”) and people will correctly |
| surmise that the purpose of your change is to fix the bug.</li> |
| <li>If you are not a committer, add your name in parentheses at the end |
| of the message.</li> |
| </ul> |
| |
| <p>Then push your commit(s) to GitHub, and create a pull request from |
| your branch to the calcite master branch. Update the JIRA case |
| to reference your pull request, and a committer will review your |
| changes.</p> |
| |
| <p>The pull request may need to be updated (after its submission) for three main |
| reasons:</p> |
| <ol> |
| <li>you identified a problem after the submission of the pull request;</li> |
| <li>the reviewer requested further changes;</li> |
| <li>the Travis CI build failed and the failure is not caused by your changes.</li> |
| </ol> |
| |
| <p>In order to update the pull request, you need to commit the changes in your |
| branch and then push the commit(s) to GitHub. You are encouraged to use regular |
| (non-rebased) commits on top of previously existing ones.</p> |
| |
| <p>When pushing the changes to GitHub, you should refrain from using the <code class="highlighter-rouge">--force</code> |
| parameter and its alternatives. You may choose to force push your changes under |
| certain conditions:</p> |
| <ul> |
| <li>the pull request has been submitted less than 10 minutes ago and there is no |
| pending discussion (in the PR and/or in JIRA) concerning it;</li> |
| <li>a reviewer has explicitly asked you to perform some modifications that |
| require the use of the <code class="highlighter-rouge">--force</code> option.</li> |
| </ul> |
| |
| <p>In the special case, that the Travis CI build failed and the failure is not |
| caused by your changes create an empty commit (<code class="highlighter-rouge">git commit --allow-empty</code>) and |
| push it.</p> |
| |
| <h2 id="null-safety">Null safety</h2> |
| |
| <p>Apache Calcite uses the Checker Framework to avoid unexpected <code class="highlighter-rouge">NullPointerExceptions</code>. |
| You might find a detailed documentation at https://checkerframework.org/</p> |
| |
| <p>Note: only main code is verified for now, so nullness annotation is not enforced in test code.</p> |
| |
| <p>To execute the Checker Framework locally please use the following command:</p> |
| |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>./gradlew -PenableCheckerframework :linq4j:classes :core:classes |
| </code></pre></div></div> |
| |
| <p>Here’s a small introduction to null-safe programming:</p> |
| |
| <ul> |
| <li>By default, parameters, return values and fields are non-nullable, so refrain from using <code class="highlighter-rouge">@NonNull</code></li> |
| <li>Local variables infer nullness from the expression, so you can write <code class="highlighter-rouge">Object v = ...</code> instead of <code class="highlighter-rouge">@Nullable Object v = ...</code></li> |
| <li> |
| <p>Avoid the use of <code class="highlighter-rouge">javax.annotation.*</code> annotations. The annotations from <code class="highlighter-rouge">jsr305</code> do not support cases like <code class="highlighter-rouge">List<@Nullable String></code> |
| so it is better to stick with <code class="highlighter-rouge">org.checkerframework.checker.nullness.qual.Nullable</code>. |
| Unfortunately, Guava (as of <code class="highlighter-rouge">29-jre</code>) has <strong>both</strong> <code class="highlighter-rouge">jsr305</code> and <code class="highlighter-rouge">checker-qual</code> dependencies at the same time, |
| so you might want to configure your IDE to exclude <code class="highlighter-rouge">javax.annotation.*</code> annotations from code completion.</p> |
| </li> |
| <li> |
| <p>The Checker Framework verifies code method by method. That means, it can’t account for method execution order. |
| That is why <code class="highlighter-rouge">@Nullable</code> fields should be verified in each method where they are used. |
| If you split logic into multiple methods, you might want verify null once, then pass it via non-nullable parameters. |
| For fields that start as null and become non-null later, use <code class="highlighter-rouge">@MonotonicNonNull</code>. |
| For fields that have already been checked against null, use <code class="highlighter-rouge">@RequiresNonNull</code>.</p> |
| </li> |
| <li> |
| <p>If you are absolutely sure the value is non-null, you might use <code class="highlighter-rouge">org.apache.calcite.linq4j.Nullness.castNonNull(T)</code>. |
| The intention behind <code class="highlighter-rouge">castNonNull</code> is like <code class="highlighter-rouge">trustMeThisIsNeverNullHoweverTheVerifierCantTellYet(...)</code></p> |
| </li> |
| <li> |
| <p>If the expression is nullable, however, you need to pass it to a non-null method, use <code class="highlighter-rouge">Objects.requireNonNull</code>. |
| It allows to have a better error message that includes context information.</p> |
| </li> |
| <li> |
| <p>The Checker Framework comes with an annotated JDK, however, there might be invalid annotations. |
| In that cases, stub files can be placed to <code class="highlighter-rouge">/src/main/config/checkerframework</code> to override the annotations. |
| It is important the files have <code class="highlighter-rouge">.astub</code> extension otherwise they will be ignored.</p> |
| </li> |
| <li> |
| <p>In array types, a type annotation appears immediately before the type component (either the array or the array component) it refers to. |
| This is explained in the <a href="https://docs.oracle.com/javase/specs/jls/se8/html/jls-9.html#jls-9.7.4">Java Language Specification</a>.</p> |
| |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> String nonNullable; |
| @Nullable String nullable; |
| |
| java.lang.@Nullable String fullyQualifiedNullable; |
| |
| // array and elements: non-nullable |
| String[] x; |
| |
| // array: nullable, elements: non-nullable |
| String @Nullable [] x; |
| |
| // array: non-nullable, elements: nullable |
| @Nullable String[] x; |
| |
| // array: nullable, elements: nullable |
| @Nullable String @Nullable [] x; |
| |
| // arrays: nullable, elements: nullable |
| // x: non-nullable |
| // x[0]: non-nullable |
| // x[0][0]: nullable |
| @Nullable String[][] x; |
| |
| // x: nullable |
| // x[0]: non-nullable |
| // x[0][0]: non-nullable |
| String @Nullable [][] x; |
| |
| // x: non-nullable |
| // x[0]: nullable |
| // x[0][0]: non-nullable |
| String[] @Nullable [] x; |
| </code></pre></div> </div> |
| </li> |
| <li> |
| <p>By default, generic parameters can be both nullable and non-nullable:</p> |
| |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> class Holder<T> { // can be both nullable |
| final T value; |
| T get() { |
| return value; // works |
| } |
| int hashCode() { |
| return value.hashCode(); // error here since T can be nullable |
| } |
| </code></pre></div> </div> |
| </li> |
| <li> |
| <p>However, default bounds are non-nullable, so if you write <code class="highlighter-rouge"><T extends Number></code>, |
| then it is the same as <code class="highlighter-rouge"><T extends @NonNull Number></code>.</p> |
| |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> class Holder<T extends Number> { // note how this T never permits nulls |
| final T value; |
| Holder(T value) { |
| this.value = value; |
| } |
| static <T> Holder<T> empty() { |
| return new Holder<>(null); // fails since T must be non-nullable |
| } |
| </code></pre></div> </div> |
| </li> |
| <li> |
| <p>If you need “either nullable or non-nullable <code class="highlighter-rouge">Number</code>”, then use <code class="highlighter-rouge"><T extends @Nullable Number></code>,</p> |
| </li> |
| <li> |
| <p>If you need to ensure the type is <strong>always</strong> nullable, then use <code class="highlighter-rouge"><@Nullable T></code> as follows:</p> |
| |
| <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> class Holder<@Nullable T> { // note how this requires T to always be nullable |
| protected T get() { // Default implementation. |
| // Default implementation returns null, so it requires that T must always be nullable |
| return null; |
| } |
| static void useHolder() { |
| // T is declared as <@Nullable T>, so Holder<String> would not compile |
| Holder<@Nullable String> holder = ...; |
| String value = holder.get(); |
| } |
| </code></pre></div> </div> |
| </li> |
| </ul> |
| |
| <h2 id="continuous-integration-testing">Continuous Integration Testing</h2> |
| |
| <p>Calcite has a collection of Jenkins jobs on ASF-hosted infrastructure. |
| They are all organized in a single view and available at |
| <a href="https://builds.apache.org/view/A-D/view/Calcite/">https://builds.apache.org/view/A-D/view/Calcite/</a>.</p> |
| |
| <h2 id="getting-started">Getting started</h2> |
| |
| <p>Calcite is a community, so the first step to joining the project is to introduce yourself. |
| Join the <a href="https://mail-archives.apache.org/mod_mbox/calcite-dev/">developers list</a> |
| and send an email.</p> |
| |
| <p>If you have the chance to attend a <a href="https://www.meetup.com/Apache-Calcite/">meetup</a>, |
| or meet <a href="https://calcite.apache.org/develop/#project-members">members of the community</a> |
| at a conference, that’s also great.</p> |
| |
| <p>Choose an initial task to work on. It should be something really simple, |
| such as a bug fix or a <a href="https://issues.apache.org/jira/issues/?jql=labels%20%3D%20newbie%20%26%20project%20%3D%20Calcite%20%26%20status%20%3D%20Open">Jira task that we have labeled |
| “newbie”</a>. |
| Follow the <a href="#contributing">contributing guidelines</a> to get your change committed.</p> |
| |
| <p>After you have made several useful contributions we may |
| <a href="https://community.apache.org/contributors/">invite you to become a committer</a>. |
| We value all contributions that help to build a vibrant community, not just code. |
| You can contribute by testing the code, helping verify a release, |
| writing documentation or the web site, |
| or just by answering questions on the list.</p> |
| |
| </article> |
| </div> |
| |
| <div class="clear"></div> |
| |
| </div> |
| </section> |
| |
| |
| <footer role="contentinfo"> |
| <div id="poweredby"> |
| <a href="http://www.apache.org/"> |
| <span class="sr-only">Apache</span> |
| <img src="/img/feather.png" width="190" height="77" alt="Apache Logo"></a> |
| </div> |
| <div id="copyright"> |
| <p>The contents of this website are Copyright © 2021 |
| <a href="https://www.apache.org/">Apache Software Foundation</a> |
| under the terms of |
| the <a href="https://www.apache.org/licenses/"> |
| Apache License v2</a>. Apache Calcite and its logo are |
| trademarks of the Apache Software Foundation.</p> |
| </div> |
| </footer> |
| |
| <script> |
| var anchorForId = function (id) { |
| var anchor = document.createElement("a"); |
| anchor.className = "header-link"; |
| anchor.href = "#" + id; |
| anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>"; |
| anchor.title = "Permalink"; |
| return anchor; |
| }; |
| |
| var linkifyAnchors = function (level, containingElement) { |
| var headers = containingElement.getElementsByTagName("h" + level); |
| for (var h = 0; h < headers.length; h++) { |
| var header = headers[h]; |
| |
| if (typeof header.id !== "undefined" && header.id !== "") { |
| header.appendChild(anchorForId(header.id)); |
| } |
| } |
| }; |
| |
| document.onreadystatechange = function () { |
| if (this.readyState === "complete") { |
| var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0]; |
| if (!contentBlock) { |
| return; |
| } |
| for (var level = 1; level <= 6; level++) { |
| linkifyAnchors(level, contentBlock); |
| } |
| } |
| }; |
| </script> |
| |
| |
| </body> |
| </html> |