Google Git
Sign in
apache / orc / asf-site / . / develop / index.html
blob: de7a7cc3db98b1e2cb2ecb6e49549891c0e8bff6 [file] [log] [blame]
<!DOCTYPE HTML>
<html lang="en-US">
<head>
<meta charset="UTF-8">
<title>Developing</title>
<meta name="viewport" content="width=device-width,initial-scale=1">
<meta name="generator" content="Jekyll v4.3.4">
<link rel="stylesheet" href="//fonts.googleapis.com/css?family=Lato:300,300italic,400,400italic,700,700italic,900">
<link rel="stylesheet" href="/css/screen.css">
<link rel="icon" type="image/x-icon" href="/favicon.ico">
<!--[if lt IE 9]>
<script src="/js/html5shiv.min.js"></script>
<script src="/js/respond.min.js"></script>
<![endif]-->
<!-- Matomo -->
<script>
var _paq = window._paq = window._paq || [];
/* tracker methods like "setCustomDimension" should be called before "trackPageView" */
_paq.push(["setDoNotTrack", true]);
_paq.push(["disableCookies"]);
_paq.push(['trackPageView']);
_paq.push(['enableLinkTracking']);
(function() {
var u="https://analytics.apache.org/";
_paq.push(['setTrackerUrl', u+'matomo.php']);
_paq.push(['setSiteId', '68']);
var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
})();
</script>
<!-- End Matomo Code -->
</head>
<body class="wrap">
<header role="banner">
<nav class="mobile-nav show-on-mobiles">
<ul>
<li class="">
<a href="/">Home</a>
</li>
<li class="">
<a href="/releases/"><span class="show-on-mobiles">Rel</span>
<span class="hide-on-mobiles">Releases</span></a>
</li>
<li class="">
<a href="/docs/"><span class="show-on-mobiles">Doc</span>
<span class="hide-on-mobiles">Documentation</span></a>
</li>
<li class="">
<a href="/talks/"><span class="show-on-mobiles">Talk</span>
<span class="hide-on-mobiles">Talks</span></a>
</li>
<li class="">
<a href="/news/">News</a>
</li>
<li class="current">
<a href="/develop/"><span class="show-on-mobiles">Dev</span>
<span class="hide-on-mobiles">Develop</span></a>
</li>
<li class="">
<a href="/help/">Help</a>
</li>
</ul>
</nav>
<div class="grid">
<div class="unit one-quarter center-on-mobiles">
<h1>
<a href="/">
<span class="sr-only">Apache ORC</span>
<img src="/img/logo.png" width="249" height="101" alt="ORC Logo">
</a>
</h1>
</div>
<nav class="main-nav unit three-quarters hide-on-mobiles">
<ul>
<li class="">
<a href="/">Home</a>
</li>
<li class="">
<a href="/releases/"><span class="show-on-mobiles">Rel</span>
<span class="hide-on-mobiles">Releases</span></a>
</li>
<li class="">
<a href="/docs/"><span class="show-on-mobiles">Doc</span>
<span class="hide-on-mobiles">Documentation</span></a>
</li>
<li class="">
<a href="/talks/"><span class="show-on-mobiles">Talk</span>
<span class="hide-on-mobiles">Talks</span></a>
</li>
<li class="">
<a href="/news/">News</a>
</li>
<li class="current">
<a href="/develop/"><span class="show-on-mobiles">Dev</span>
<span class="hide-on-mobiles">Develop</span></a>
</li>
<li class="">
<a href="/help/">Help</a>
</li>
</ul>
</nav>
</div>
</header>
<section class="standalone">
<div class="grid">
<div class="unit whole">
<article>
<h1>Developing</h1>
<p>Information about the ORC project that is most important for
developers working on the project.</p>
<p>The <a href="/specification">ORC format specification</a> defines the format
to promote compatibility between implementations.</p>
<h2 id="development-community">Development community</h2>
<p>We have committers from many different companies. The full
list of <a href="committers">ORC committers</a> is available.</p>
<h2 id="mailing-lists">Mailing Lists</h2>
<p>The most important communication mechanism for the project are its
mailing lists. The mailing lists have the advantage that they are
publicly archived and work well asynchronously across timezones.</p>
<p>Beside the user mailing list, there are several development mailing
lists for ORC:</p>
<ul>
<li><a href="mailto:dev@orc.apache.org">dev@orc.apache.org</a> - Development discussions
with archive <a href="https://mail-archives.apache.org/mod_mbox/orc-dev/">here</a></li>
<li><a href="mailto:issues@orc.apache.org">issues@orc.apache.org</a> - Bug tracking
with archive <a href="https://mail-archives.apache.org/mod_mbox/orc-issues/">here</a></li>
<li><a href="mailto:commits@orc.apache.org">commits@orc.apache.org</a> - Git tracking
with archive <a href="https://mail-archives.apache.org/mod_mbox/orc-commits/">here</a></li>
</ul>
<p>You can subscribe to the lists by sending email to
<em>list</em>-subscribe@orc.apache.org and unsubscribe by sending email to
<em>list</em>-unsubscribe@orc.apache.org.</p>
<h2 id="bug-reports">Bug reports</h2>
<p>Each code change requires a <a href="https://issues.apache.org/jira/browse/ORC">jira</a> to track the
discussion of the change.</p>
<h2 id="design">Design</h2>
<p>Some code changes provide <a href="design">design/additional documentation</a>.</p>
<h2 id="source-code">Source code</h2>
<p>ORC uses git for version control. Get the source code and configure it
to fetch the pull requests also:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git clone -o apache git@github.com:apache/orc.git
% cd orc
% git config --add remote.apache.fetch '+refs/pull/*/head:refs/remotes/apache/pr/*'
</code></pre></div></div>
<p>Pull requests will be named “apache/pr/999” for pull request 999.</p>
<p>If you are a committer, you need to register your Github id with Apache:</p>
<ul>
<li>Visit <a href="https://id.apache.org">id.apache.org</a> and add your Github id.</li>
<li>Visit <a href="https://gitbox.apache.org/setup/">Apache gitbox</a> and get authorization for the projects that use Apache gitbox, which includes ORC.</li>
</ul>
<p>Once this is done, your Github account will be placed in the ORC github team,
so you’ll be able to close PRs, etc.</p>
<p>The important branches are:</p>
<ul>
<li><a href="https://github.com/apache/orc/tree/main">main</a> -
The main branch for all development</li>
<li>branch-X.Y - The release branches</li>
<li><a href="https://github.com/apache/orc/tree/asf-site">asf-site</a> -
The generated html pages that are deployed as https://orc.apache.org/</li>
</ul>
<p>Releases are tagged as “rel/release-X.Y.Z”. Apache’s git repository
guarantees that tags in the “rel/*” namespace are never deleted or
changed.</p>
<p>Please check our <a href="coding">coding guidelines</a>.</p>
<h2 id="website-shortcuts">Website shortcuts</h2>
<p>We’ve added several shortcuts to various relevant pages:</p>
<p>From our website, you can use:</p>
<ul>
<li><a href="https://orc.apache.org/bugs">/bugs</a> to jump to our bug database</li>
<li><a href="https://orc.apache.org/downloads">/downloads</a> to jump to our downloads page</li>
<li><a href="https://orc.apache.org/releases">/releases</a> to jump to our releases page</li>
<li><a href="https://orc.apache.org/src">/src</a> to jump to our source code</li>
<li><a href="https://orc.apache.org/web-src">/web-src</a> to jump to our site source code</li>
</ul>
<h2 id="reviews">Reviews</h2>
<p>ORC uses Commit-Then-Review, so patches can be committed without a
committer’s review. However, most changes should be reviewed first.</p>
<h2 id="tests">Tests</h2>
<p>In principle, new pull requests had better provide test coverage for newly proposed code path.
In addition, newly added test cases are supposed to fail without the proposed main code.
In other words, reviewers need to check these two things during the review manually.
The following is useful during Java code review and tests.</p>
<h3 id="running-individual-java-test-file">Running Individual Java Test File</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% cd java
% ./mvnw test -pl core -Dtest=TestRecordReaderImpl
</code></pre></div></div>
<h3 id="running-multiple-java-test-files">Running Multiple Java Test Files</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% cd java
% ./mvnw test -pl core -Dtest='Test*Reader*'
</code></pre></div></div>
<h3 id="running-individual-java-test-case">Running Individual Java Test Case</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% cd java
$ ./mvnw test -pl core -Dtest=TestRecordReaderImpl#testFindColumn
</code></pre></div></div>
<h3 id="running-multiple-java-test-cases">Running Multiple Java Test Cases</h3>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% cd java
% ./mvnw test -pl core -Dtest='TestRecordReaderImpl#testFind*'
</code></pre></div></div>
<h2 id="approving-a-pull-request">Approving a pull request</h2>
<p>It is recommended to use an interactive script to merge pull requests, which can be automatically associated with the JIRA.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% ./dev/merge_orc_pr.py
</code></pre></div></div>
<p>Fetch the current state of the project:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git fetch apache
</code></pre></div></div>
<p>Switch to the branch:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git checkout apache/pr/999
</code></pre></div></div>
<p>You’ll want to rebase it and make it a single commit by squashing
the commits into a single commit.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git rebase -i apache/main
</code></pre></div></div>
<p>Update the commit message to sign it using your GPG key and close the
pull request:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git commit --amend -s -S
</code></pre></div></div>
<p>Ensure the first line of the commit starts with the jira number
(eg. ORC-123) and includes a description of what was changed. Also add
a line such as “Fixes #999”, which asks the Apache infrastructure to
close pull request 999. If you wish you close a pull request without
claiming to have fixed the problem, the form “Closes #999” also works.</p>
<p>Finally, push the result to Apache:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% git push apache HEAD:main
</code></pre></div></div>
<h2 id="creating-a-gpg-key">Creating a GPG key</h2>
<p>When you become a committer, you should create a 4096 bit GPG key.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% gpg --full-gen-key
</code></pre></div></div>
<p>Use 4096 bits and your Apache email address. Once it is created,
you’ll need to get your key fingerprint. Avoid using the short
fingerprint (eg. 3D0C92B9), because it is possible to generate fake
keys that have the same short fingerprint as the real key.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>% gpg --list-secret-keys --keyid-format LONG
</code></pre></div></div>
<p>Your key fingerprint is the string after “rsa4096/”. Example output
for the key with fingerprint 1209E7F13D0C92B9 looks like:</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>/Users/owen/.gnupg/pubring.gpg
------------------------------
sec rsa4096/1209E7F13D0C92B9 2010-02-23 [SC]
47660BC98BC433F01E5C90581209E7F13D0C92B9
uid [ultimate] Owen O'Malley &lt;omalley@apache.org&gt;
</code></pre></div></div>
<p>Now publish your public key to one of the public keyservers. I usually use
hkp://pgp.mit.edu, although any of them will work.</p>
<div class="language-plaintext highlighter-rouge"><div class="highlight"><pre class="highlight"><code>gpg --send-key &lt;your key fingerprint&gt;
</code></pre></div></div>
<p>Next, you need to update the <a href="https://id.apache.org">Apache account
database</a> with your new key. Login to add your
new key’s fingerprint and your github id. It is also good to update
your github profile with your GPG public key as well at
<a href="https://github.com/settings/keys">https://github.com/settings/keys</a>.</p>
<p>After you’ve created your key, it is good to get someone in the ORC
community to sign it for you. Contact someone directly or send email
to dev@orc.apache.org asking for someone to sign it.</p>
<h2 id="making-a-release">Making a Release</h2>
<p>The release process for ORC is driven by a Release Manager. They should
discuss their intention to start the process on the dev list and then
follow the steps of <a href="make-release">how to release ORC</a>.</p>
<h2 id="dist-directory">Dist Directory</h2>
<p>Apache expects the projects to manage their current release artifact
distribution using subversion. It should be limited to the latest
release in each of the active release branches.</p>
<p>The ORC dist directory is managed via svn in
<a href="https://dist.apache.org/repos/dist/release/orc">https://dist.apache.org/repos/dist/release/orc</a>.
The release artifacts are pushed to many mirrors. Files in the dist
directory are available forever via the <a href="https://archive.apache.org/dist/orc/">Apache dist
archive</a>.</p>
<h2 id="bylaws">Bylaws</h2>
<p>ORC has a set of <a href="bylaws">bylaws</a> that describe the rules for the different
votes within our project.</p>
</article>
</div>
<div class="clear"></div>
</div>
</section>
<footer role="contentinfo">
<p style="margin-left: 20px; margin-right; 20px; text-align: center">The contents of this website are &copy;&nbsp;2025
<a href="https://www.apache.org/">Apache Software Foundation</a>
under the terms of the <a
href="https://www.apache.org/licenses/LICENSE-2.0.html">
Apache&nbsp;License&nbsp;v2</a>. Apache ORC and its logo are trademarks
of the Apache Software Foundation.</p>
</footer>
<script>
var anchorForId = function (id) {
var anchor = document.createElement("a");
anchor.className = "header-link";
anchor.href = "#" + id;
anchor.innerHTML = "<span class=\"sr-only\">Permalink</span><i class=\"fa fa-link\"></i>";
anchor.title = "Permalink";
return anchor;
};
var linkifyAnchors = function (level, containingElement) {
var headers = containingElement.getElementsByTagName("h" + level);
for (var h = 0; h < headers.length; h++) {
var header = headers[h];
if (typeof header.id !== "undefined" && header.id !== "") {
header.appendChild(anchorForId(header.id));
}
}
};
document.onreadystatechange = function () {
if (this.readyState === "complete") {
var contentBlock = document.getElementsByClassName("docs")[0] || document.getElementsByClassName("news")[0];
if (!contentBlock) {
return;
}
for (var level = 1; level <= 6; level++) {
linkifyAnchors(level, contentBlock);
}
}
};
</script>
</body>
</html>