blob: f9e697908cb6d2b6bfd7e9214b4bd20863f2c4ad [file] [log] [blame]
<!DOCTYPE html>
<html lang="en-US">
<head>
<meta charset="UTF-8">
<meta http-equiv="X-UA-Compatible" content="IE=edge">
<meta name="viewport" content="width=device-width, initial-scale=1">
<!-- The above meta tags *must* come first in the head; any other head content must come *after* these tags -->
<title>A Native Go Library for Apache Arrow | Apache Arrow</title>
<!-- Begin Jekyll SEO tag v2.8.0 -->
<meta name="generator" content="Jekyll v4.4.1" />
<meta property="og:title" content="A Native Go Library for Apache Arrow" />
<meta name="author" content="pmc" />
<meta property="og:locale" content="en_US" />
<meta name="description" content="Since launching in early 2016, Apache Arrow has been growing fast. We have made nine major releases through the efforts of over 120 distinct contributors. The project’s scope has also expanded. We began by focusing on the development of the standardized in-memory columnar data format, which now serves as a pillar of the project. Since then, we have been growing into a more general cross-language platform for in-memory data analysis through new additions to the project like the Plasma shared memory object store. A primary goal of the project is to enable data system developers to process and move data fast. So far, we officially have developed native Arrow implementations in C++, Java, and JavaScript. We have created binding layers for the C++ libraries in C (using the GLib libraries) and Python. We have also seen efforts to develop interfaces to the Arrow C++ libraries in Go, Lua, Ruby, and Rust. While binding layers serve many purposes, there can be benefits to native implementations, and so we’ve been keen to see future work on native implementations in growing systems languages like Go and Rust. This past October, engineers Stuart Carnie, Nathaniel Cook, and Chris Goller, employees of InfluxData, began developing a native [Go language implementation of the Apache Arrow in-memory columnar format for use in Go-based database systems like InfluxDB. We are excited to announce that InfluxData has donated this native Go implementation to the Apache Arrow project, where it will continue to be developed. This work features low-level integration with the Go runtime and native support for SIMD instruction sets. We are looking forward to working more closely with the Go community on solving in-memory analytics and data interoperability problems. One of the mantras in The Apache Software Foundation is &quot;Community over Code&quot;. By building an open and collaborative development community across many programming language ecosystems, we will be able to development better and longer-lived solutions to the systems problems faced by data developers. We are excited for what the future holds for the Apache Arrow project. Adding first-class support for a popular systems programming language like Go is an important step along the way. We welcome others from the Go community to get involved in the project. We also welcome others who wish to explore building Arrow support for other programming languages not yet represented. Learn more at https://arrow.apache.org and join the mailing list dev@arrow.apache.org." />
<meta property="og:description" content="Since launching in early 2016, Apache Arrow has been growing fast. We have made nine major releases through the efforts of over 120 distinct contributors. The project’s scope has also expanded. We began by focusing on the development of the standardized in-memory columnar data format, which now serves as a pillar of the project. Since then, we have been growing into a more general cross-language platform for in-memory data analysis through new additions to the project like the Plasma shared memory object store. A primary goal of the project is to enable data system developers to process and move data fast. So far, we officially have developed native Arrow implementations in C++, Java, and JavaScript. We have created binding layers for the C++ libraries in C (using the GLib libraries) and Python. We have also seen efforts to develop interfaces to the Arrow C++ libraries in Go, Lua, Ruby, and Rust. While binding layers serve many purposes, there can be benefits to native implementations, and so we’ve been keen to see future work on native implementations in growing systems languages like Go and Rust. This past October, engineers Stuart Carnie, Nathaniel Cook, and Chris Goller, employees of InfluxData, began developing a native [Go language implementation of the Apache Arrow in-memory columnar format for use in Go-based database systems like InfluxDB. We are excited to announce that InfluxData has donated this native Go implementation to the Apache Arrow project, where it will continue to be developed. This work features low-level integration with the Go runtime and native support for SIMD instruction sets. We are looking forward to working more closely with the Go community on solving in-memory analytics and data interoperability problems. One of the mantras in The Apache Software Foundation is &quot;Community over Code&quot;. By building an open and collaborative development community across many programming language ecosystems, we will be able to development better and longer-lived solutions to the systems problems faced by data developers. We are excited for what the future holds for the Apache Arrow project. Adding first-class support for a popular systems programming language like Go is an important step along the way. We welcome others from the Go community to get involved in the project. We also welcome others who wish to explore building Arrow support for other programming languages not yet represented. Learn more at https://arrow.apache.org and join the mailing list dev@arrow.apache.org." />
<link rel="canonical" href="https://arrow.apache.org/blog/2018/03/22/go-code-donation/" />
<meta property="og:url" content="https://arrow.apache.org/blog/2018/03/22/go-code-donation/" />
<meta property="og:site_name" content="Apache Arrow" />
<meta property="og:image" content="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" />
<meta property="og:type" content="article" />
<meta property="article:published_time" content="2018-03-22T00:00:00-04:00" />
<meta name="twitter:card" content="summary_large_image" />
<meta property="twitter:image" content="https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png" />
<meta property="twitter:title" content="A Native Go Library for Apache Arrow" />
<script type="application/ld+json">
{"@context":"https://schema.org","@type":"BlogPosting","author":{"@type":"Person","name":"pmc"},"dateModified":"2018-03-22T00:00:00-04:00","datePublished":"2018-03-22T00:00:00-04:00","description":"Since launching in early 2016, Apache Arrow has been growing fast. We have made nine major releases through the efforts of over 120 distinct contributors. The project’s scope has also expanded. We began by focusing on the development of the standardized in-memory columnar data format, which now serves as a pillar of the project. Since then, we have been growing into a more general cross-language platform for in-memory data analysis through new additions to the project like the Plasma shared memory object store. A primary goal of the project is to enable data system developers to process and move data fast. So far, we officially have developed native Arrow implementations in C++, Java, and JavaScript. We have created binding layers for the C++ libraries in C (using the GLib libraries) and Python. We have also seen efforts to develop interfaces to the Arrow C++ libraries in Go, Lua, Ruby, and Rust. While binding layers serve many purposes, there can be benefits to native implementations, and so we’ve been keen to see future work on native implementations in growing systems languages like Go and Rust. This past October, engineers Stuart Carnie, Nathaniel Cook, and Chris Goller, employees of InfluxData, began developing a native [Go language implementation of the Apache Arrow in-memory columnar format for use in Go-based database systems like InfluxDB. We are excited to announce that InfluxData has donated this native Go implementation to the Apache Arrow project, where it will continue to be developed. This work features low-level integration with the Go runtime and native support for SIMD instruction sets. We are looking forward to working more closely with the Go community on solving in-memory analytics and data interoperability problems. One of the mantras in The Apache Software Foundation is &quot;Community over Code&quot;. By building an open and collaborative development community across many programming language ecosystems, we will be able to development better and longer-lived solutions to the systems problems faced by data developers. We are excited for what the future holds for the Apache Arrow project. Adding first-class support for a popular systems programming language like Go is an important step along the way. We welcome others from the Go community to get involved in the project. We also welcome others who wish to explore building Arrow support for other programming languages not yet represented. Learn more at https://arrow.apache.org and join the mailing list dev@arrow.apache.org.","headline":"A Native Go Library for Apache Arrow","image":"https://arrow.apache.org/img/arrow-logo_horizontal_black-txt_white-bg.png","mainEntityOfPage":{"@type":"WebPage","@id":"https://arrow.apache.org/blog/2018/03/22/go-code-donation/"},"publisher":{"@type":"Organization","logo":{"@type":"ImageObject","url":"https://arrow.apache.org/img/logo.png"},"name":"pmc"},"url":"https://arrow.apache.org/blog/2018/03/22/go-code-donation/"}</script>
<!-- End Jekyll SEO tag -->
<!-- favicons -->
<link rel="icon" type="image/png" sizes="16x16" href="/img/favicon-16x16.png" id="light1">
<link rel="icon" type="image/png" sizes="32x32" href="/img/favicon-32x32.png" id="light2">
<link rel="apple-touch-icon" type="image/png" sizes="180x180" href="/img/apple-touch-icon.png" id="light3">
<link rel="apple-touch-icon" type="image/png" sizes="120x120" href="/img/apple-touch-icon-120x120.png" id="light4">
<link rel="apple-touch-icon" type="image/png" sizes="76x76" href="/img/apple-touch-icon-76x76.png" id="light5">
<link rel="apple-touch-icon" type="image/png" sizes="60x60" href="/img/apple-touch-icon-60x60.png" id="light6">
<!-- dark mode favicons -->
<link rel="icon" type="image/png" sizes="16x16" href="/img/favicon-16x16-dark.png" id="dark1">
<link rel="icon" type="image/png" sizes="32x32" href="/img/favicon-32x32-dark.png" id="dark2">
<link rel="apple-touch-icon" type="image/png" sizes="180x180" href="/img/apple-touch-icon-dark.png" id="dark3">
<link rel="apple-touch-icon" type="image/png" sizes="120x120" href="/img/apple-touch-icon-120x120-dark.png" id="dark4">
<link rel="apple-touch-icon" type="image/png" sizes="76x76" href="/img/apple-touch-icon-76x76-dark.png" id="dark5">
<link rel="apple-touch-icon" type="image/png" sizes="60x60" href="/img/apple-touch-icon-60x60-dark.png" id="dark6">
<script>
// Switch to the dark-mode favicons if prefers-color-scheme: dark
function onUpdate() {
light1 = document.querySelector('link#light1');
light2 = document.querySelector('link#light2');
light3 = document.querySelector('link#light3');
light4 = document.querySelector('link#light4');
light5 = document.querySelector('link#light5');
light6 = document.querySelector('link#light6');
dark1 = document.querySelector('link#dark1');
dark2 = document.querySelector('link#dark2');
dark3 = document.querySelector('link#dark3');
dark4 = document.querySelector('link#dark4');
dark5 = document.querySelector('link#dark5');
dark6 = document.querySelector('link#dark6');
if (matcher.matches) {
light1.remove();
light2.remove();
light3.remove();
light4.remove();
light5.remove();
light6.remove();
document.head.append(dark1);
document.head.append(dark2);
document.head.append(dark3);
document.head.append(dark4);
document.head.append(dark5);
document.head.append(dark6);
} else {
dark1.remove();
dark2.remove();
dark3.remove();
dark4.remove();
dark5.remove();
dark6.remove();
document.head.append(light1);
document.head.append(light2);
document.head.append(light3);
document.head.append(light4);
document.head.append(light5);
document.head.append(light6);
}
}
matcher = window.matchMedia('(prefers-color-scheme: dark)');
matcher.addListener(onUpdate);
onUpdate();
</script>
<link href="/css/main.css" rel="stylesheet">
<link href="/css/syntax.css" rel="stylesheet">
<script src="/javascript/main.js"></script>
<!-- Matomo -->
<script>
var _paq = window._paq = window._paq || [];
/* tracker methods like "setCustomDimension" should be called before "trackPageView" */
/* We explicitly disable cookie tracking to avoid privacy issues */
_paq.push(['disableCookies']);
_paq.push(['trackPageView']);
_paq.push(['enableLinkTracking']);
(function() {
var u="https://analytics.apache.org/";
_paq.push(['setTrackerUrl', u+'matomo.php']);
_paq.push(['setSiteId', '20']);
var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0];
g.async=true; g.src=u+'matomo.js'; s.parentNode.insertBefore(g,s);
})();
</script>
<!-- End Matomo Code -->
<link type="application/atom+xml" rel="alternate" href="https://arrow.apache.org/feed.xml" title="Apache Arrow" />
</head>
<body class="wrap">
<header>
<nav class="navbar navbar-expand-md navbar-dark bg-dark">
<a class="navbar-brand no-padding" href="/"><img src="/img/arrow-inverse-300px.png" height="40px"></a>
<button class="navbar-toggler ml-auto" type="button" data-toggle="collapse" data-target="#arrow-navbar" aria-controls="arrow-navbar" aria-expanded="false" aria-label="Toggle navigation">
<span class="navbar-toggler-icon"></span>
</button>
<!-- Collect the nav links, forms, and other content for toggling -->
<div class="collapse navbar-collapse justify-content-end" id="arrow-navbar">
<ul class="nav navbar-nav">
<li class="nav-item"><a class="nav-link" href="/overview/" role="button" aria-haspopup="true" aria-expanded="false">Overview</a></li>
<li class="nav-item"><a class="nav-link" href="/faq/" role="button" aria-haspopup="true" aria-expanded="false">FAQ</a></li>
<li class="nav-item"><a class="nav-link" href="/blog" role="button" aria-haspopup="true" aria-expanded="false">Blog</a></li>
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" href="#" id="navbarDropdownGetArrow" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
Get Arrow
</a>
<div class="dropdown-menu" aria-labelledby="navbarDropdownGetArrow">
<a class="dropdown-item" href="/install/">Install</a>
<a class="dropdown-item" href="/release/">Releases</a>
</div>
</li>
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" href="#" id="navbarDropdownDocumentation" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
Docs
</a>
<div class="dropdown-menu" aria-labelledby="navbarDropdownDocumentation">
<a class="dropdown-item" href="/docs">Project Docs</a>
<a class="dropdown-item" href="/docs/format/Columnar.html">Format</a>
<hr>
<a class="dropdown-item" href="/docs/c_glib">C GLib</a>
<a class="dropdown-item" href="/docs/cpp">C++</a>
<a class="dropdown-item" href="https://github.com/apache/arrow/blob/main/csharp/README.md" target="_blank" rel="noopener">C#</a>
<a class="dropdown-item" href="https://godoc.org/github.com/apache/arrow/go/arrow" target="_blank" rel="noopener">Go</a>
<a class="dropdown-item" href="/docs/java">Java</a>
<a class="dropdown-item" href="/docs/js">JavaScript</a>
<a class="dropdown-item" href="/julia/">Julia</a>
<a class="dropdown-item" href="https://github.com/apache/arrow/blob/main/matlab/README.md" target="_blank" rel="noopener">MATLAB</a>
<a class="dropdown-item" href="/docs/python">Python</a>
<a class="dropdown-item" href="/docs/r">R</a>
<a class="dropdown-item" href="https://github.com/apache/arrow/blob/main/ruby/README.md" target="_blank" rel="noopener">Ruby</a>
<a class="dropdown-item" href="https://docs.rs/arrow/latest" target="_blank" rel="noopener">Rust</a>
<a class="dropdown-item" href="/swift">Swift</a>
</div>
</li>
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" href="#" id="navbarDropdownSource" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
Source
</a>
<div class="dropdown-menu" aria-labelledby="navbarDropdownSource">
<a class="dropdown-item" href="https://github.com/apache/arrow" target="_blank" rel="noopener">Main Repo</a>
<hr>
<a class="dropdown-item" href="https://github.com/apache/arrow/tree/main/c_glib" target="_blank" rel="noopener">C GLib</a>
<a class="dropdown-item" href="https://github.com/apache/arrow/tree/main/cpp" target="_blank" rel="noopener">C++</a>
<a class="dropdown-item" href="https://github.com/apache/arrow/tree/main/csharp" target="_blank" rel="noopener">C#</a>
<a class="dropdown-item" href="https://github.com/apache/arrow-go" target="_blank" rel="noopener">Go</a>
<a class="dropdown-item" href="https://github.com/apache/arrow-java" target="_blank" rel="noopener">Java</a>
<a class="dropdown-item" href="https://github.com/apache/arrow-js" target="_blank" rel="noopener">JavaScript</a>
<a class="dropdown-item" href="https://github.com/apache/arrow-julia" target="_blank" rel="noopener">Julia</a>
<a class="dropdown-item" href="https://github.com/apache/arrow/tree/main/matlab" target="_blank" rel="noopener">MATLAB</a>
<a class="dropdown-item" href="https://github.com/apache/arrow/tree/main/python" target="_blank" rel="noopener">Python</a>
<a class="dropdown-item" href="https://github.com/apache/arrow/tree/main/r" target="_blank" rel="noopener">R</a>
<a class="dropdown-item" href="https://github.com/apache/arrow/tree/main/ruby" target="_blank" rel="noopener">Ruby</a>
<a class="dropdown-item" href="https://github.com/apache/arrow-rs" target="_blank" rel="noopener">Rust</a>
<a class="dropdown-item" href="https://github.com/apache/arrow-swift" target="_blank" rel="noopener">Swift</a>
</div>
</li>
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" href="#" id="navbarDropdownSubprojects" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
Subprojects
</a>
<div class="dropdown-menu" aria-labelledby="navbarDropdownSubprojects">
<a class="dropdown-item" href="/adbc">ADBC</a>
<a class="dropdown-item" href="/docs/format/Flight.html">Arrow Flight</a>
<a class="dropdown-item" href="/docs/format/FlightSql.html">Arrow Flight SQL</a>
<a class="dropdown-item" href="https://datafusion.apache.org" target="_blank" rel="noopener">DataFusion</a>
<a class="dropdown-item" href="/nanoarrow">nanoarrow</a>
</div>
</li>
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" href="#" id="navbarDropdownCommunity" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
Community
</a>
<div class="dropdown-menu" aria-labelledby="navbarDropdownCommunity">
<a class="dropdown-item" href="/community/">Communication</a>
<a class="dropdown-item" href="/docs/developers/index.html">Contributing</a>
<a class="dropdown-item" href="https://github.com/apache/arrow/issues" target="_blank" rel="noopener">Issue Tracker</a>
<a class="dropdown-item" href="/committers/">Governance</a>
<a class="dropdown-item" href="/use_cases/">Use Cases</a>
<a class="dropdown-item" href="/powered_by/">Powered By</a>
<a class="dropdown-item" href="/visual_identity/">Visual Identity</a>
<a class="dropdown-item" href="/security/">Security</a>
<a class="dropdown-item" href="https://www.apache.org/foundation/policies/conduct.html" target="_blank" rel="noopener">Code of Conduct</a>
</div>
</li>
<li class="nav-item dropdown">
<a class="nav-link dropdown-toggle" href="#" id="navbarDropdownASF" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false">
ASF Links
</a>
<div class="dropdown-menu dropdown-menu-right" aria-labelledby="navbarDropdownASF">
<a class="dropdown-item" href="https://www.apache.org/" target="_blank" rel="noopener">ASF Website</a>
<a class="dropdown-item" href="https://www.apache.org/licenses/" target="_blank" rel="noopener">License</a>
<a class="dropdown-item" href="https://www.apache.org/foundation/sponsorship.html" target="_blank" rel="noopener">Donate</a>
<a class="dropdown-item" href="https://www.apache.org/foundation/thanks.html" target="_blank" rel="noopener">Thanks</a>
<a class="dropdown-item" href="https://www.apache.org/security/" target="_blank" rel="noopener">Security</a>
</div>
</li>
</ul>
</div>
<!-- /.navbar-collapse -->
</nav>
</header>
<div class="container p-4 pt-5">
<div class="col-md-8 mx-auto">
<main role="main" class="pb-5">
<h1>
A Native Go Library for Apache Arrow
</h1>
<hr class="mt-4 mb-3">
<p class="mb-4 pb-1">
<span class="badge badge-secondary">Published</span>
<span class="published mr-3">
22 Mar 2018
</span>
<br>
<span class="badge badge-secondary">By</span>
<a class="mr-3" href="https://arrow.apache.org">The Apache Arrow PMC (pmc) </a>
</p>
<!--
-->
<p>Since launching in early 2016, Apache Arrow has been growing fast. We have made
nine major releases through the efforts of over 120 distinct contributors. The
project’s scope has also expanded. We began by focusing on the development of
the standardized in-memory columnar data format, which now serves as a pillar
of the project. Since then, we have been growing into a more general
cross-language platform for in-memory data analysis through new additions to
the project like the <a href="http://arrow.apache.org/blog/2017/08/16/0.6.0-release/">Plasma shared memory object store</a>. A primary goal of
the project is to enable data system developers to process and move data fast.</p>
<p>So far, we officially have developed native Arrow implementations in C++, Java,
and JavaScript. We have created binding layers for the C++ libraries in C
(using the GLib libraries) and Python. We have also seen efforts to develop
interfaces to the Arrow C++ libraries in Go, Lua, Ruby, and Rust. While binding
layers serve many purposes, there can be benefits to native implementations,
and so we’ve been keen to see future work on native implementations in growing
systems languages like Go and Rust.</p>
<p>This past October, engineers <a href="https://github.com/stuartcarnie" target="_blank" rel="noopener">Stuart Carnie</a>, <a href="https://github.com/nathanielc" target="_blank" rel="noopener">Nathaniel Cook</a>, and
<a href="https://github.com/goller" target="_blank" rel="noopener">Chris Goller</a>, employees of <a href="https://influxdata.com" target="_blank" rel="noopener">InfluxData</a>, began developing a native [Go
language implementation of the <a href="https://github.com/influxdata/arrow" target="_blank" rel="noopener">Apache Arrow</a> in-memory columnar format for
use in Go-based database systems like InfluxDB. We are excited to announce that
InfluxData has <a href="https://www.businesswire.com/news/home/20180322005393/en/InfluxData-Announces-Language-Implementation-Contribution-Apache-Arrow" target="_blank" rel="noopener">donated this native Go implementation to the Apache Arrow
project</a>, where it will continue to be developed. This work features
low-level integration with the Go runtime and native support for SIMD
instruction sets. We are looking forward to working more closely with the Go
community on solving in-memory analytics and data interoperability problems.</p>
<div align="center">
<img src="/img/native_go_implementation.png" alt="Apache Arrow implementations and bindings" width="60%" class="img-responsive">
</div>
<p>One of the mantras in <a href="https://www.apache.org" target="_blank" rel="noopener">The Apache Software Foundation</a> is "Community over
Code". By building an open and collaborative development community across many
programming language ecosystems, we will be able to development better and
longer-lived solutions to the systems problems faced by data developers.</p>
<p>We are excited for what the future holds for the Apache Arrow project. Adding
first-class support for a popular systems programming language like Go is an
important step along the way. We welcome others from the Go community to get
involved in the project. We also welcome others who wish to explore building
Arrow support for other programming languages not yet represented. Learn more
at <a href="https://arrow.apache.org">https://arrow.apache.org</a> and join the mailing list
<a href="https://lists.apache.org/list.html?dev@arrow.apache.org" target="_blank" rel="noopener">dev@arrow.apache.org</a>.</p>
</main>
</div>
<hr>
<footer class="footer">
<div class="row">
<div class="col-md-9">
<p>Apache Arrow, Arrow, Apache, the Apache logo, and the Apache Arrow project logo are either registered trademarks or trademarks of The Apache Software Foundation in the United States and other countries.</p>
<p>© 2016-2025 The Apache Software Foundation</p>
</div>
<div class="col-md-3">
<a class="d-sm-none d-md-inline pr-2" href="https://www.apache.org/events/current-event.html" target="_blank" rel="noopener">
<img src="https://www.apache.org/events/current-event-234x60.png">
</a>
</div>
</div>
</footer>
</div>
</body>
</html>