Apache Iceberg - Go

Clone this repo:
  1. b4babc8 feat (cli): Add Support CreateTableOpt with table properties, partition spec & sort order (#475) by Dao Thanh Tung · 8 days ago main
  2. 1289d19 feat(table/updates): add stubs for the remove schemas & remove partition specs table updates (#491) by Tobias Pütz · 9 days ago
  3. 59e4b22 fix(schema): Fix bug in Schema.HighestFieldId (#494) by Alex Nagel · 9 days ago
  4. 27dc0f3 build(deps): bump the gomod_updates group with 16 updates (#490) by dependabot[bot] · 12 days ago
  5. 2f0b76a feat(table/updates): add unmarshal for table updates & requirements (#488) by Tobias Pütz · 2 weeks ago

Iceberg Golang

Go Reference

iceberg is a Golang implementation of the Iceberg table spec.

Build From Source

Prerequisites

  • Go 1.23 or later

Build

$ git clone https://github.com/apache/iceberg-go.git
$ cd iceberg-go/cmd/iceberg && go build .

Feature Support / Roadmap

FileSystem Support

Filesystem TypeSupported
S3X
Google Cloud StorageX
Azure Blob StorageX
Local FilesystemX

Metadata

OperationSupported
Get SchemaX
Get SnapshotsX
Get Sort OrdersX
Get Partition SpecsX
Get ManifestsX
Create New ManifestsX
Plan Scanx
Plan Scan for Snapshotx

Catalog Support

OperationRESTHiveGlueSQL
Load TableXXX
List TablesXXX
Create TableXXX
Register TableXX
Update Current SnapshotXXX
Create New SnapshotXXX
Rename TableXXX
Drop TableXXX
Alter TableXXX
Check Table ExistsXXX
Set Table PropertiesXXX
List NamespacesXXX
Create NamespaceXXX
Check Namespace ExistsXXX
Drop NamespaceXXX
Update Namespace PropertiesXXX
Create ViewXX
Load ViewX
List ViewXX
Drop ViewXX
Check View ExistsXX

Read/Write Data Support

  • Data can currently be read as an Arrow Table or as a stream of Arrow record batches.

Supported Write Operations

As long as the FileSystem is supported and the Catalog supports altering the table, the following tracks the current write support:

OperationSupported
Append StreamX
Append Data FilesX
Rewrite Files
Rewrite manifests
Overwrite Files
Write Pos Delete
Write Eq Delete
Row Delta

CLI Usage

Run go build ./cmd/iceberg from the root of this repository to build the CLI executable, alternately you can run go install github.com/apache/iceberg-go/cmd/iceberg to install it to the bin directory of your GOPATH.

The iceberg CLI usage is very similar to pyiceberg CLI
You can pass the catalog URI with --uri argument.

Example: You can start the Iceberg REST API docker image which runs on default in port 8181

docker pull apache/iceberg-rest-fixture:latest
docker run -p 8181:8181 apache/iceberg-rest-fixture:latest

and run the iceberg CLI pointing to the REST API server.

 ./iceberg --uri http://0.0.0.0:8181 list
┌─────┐
| IDs |
| --- |
└─────┘

Create Namespace

./iceberg --uri http://0.0.0.0:8181 create namespace taxitrips

List Namespace

 ./iceberg --uri http://0.0.0.0:8181 list
┌───────────┐
| IDs       |
| --------- |
| taxitrips |
└───────────┘


Get in Touch