0.2.0 RC1
fix(catalog): add rest integration write test (#352)

Includes a few fixes that were found when writing the test
3 files changed
tree: f8682ef4a47be82a945082b90ebcfb37ffdbe7ab
  1. .github/
  2. catalog/
  3. cmd/
  4. config/
  5. dev/
  6. internal/
  7. io/
  8. table/
  9. utils/
  10. .asf.yaml
  11. .gitattributes
  12. .gitignore
  13. .golangci.yml
  14. .pre-commit-config.yaml
  15. errors.go
  16. exprs.go
  17. exprs_test.go
  18. go.mod
  19. go.sum
  20. LICENSE
  21. literals.go
  22. literals_test.go
  23. manifest.go
  24. manifest_test.go
  25. name_mapping.go
  26. name_mapping_test.go
  27. NOTICE
  28. operation_string.go
  29. partitions.go
  30. partitions_test.go
  31. predicates.go
  32. README.md
  33. schema.go
  34. schema_conversions.go
  35. schema_test.go
  36. transforms.go
  37. transforms_test.go
  38. types.go
  39. types_test.go
  40. utils.go
  41. visitors.go
  42. visitors_test.go
README.md

Iceberg Golang

Go Reference

iceberg is a Golang implementation of the Iceberg table spec.

Build From Source

Prerequisites

  • Go 1.23 or later

Build

$ git clone https://github.com/apache/iceberg-go.git
$ cd iceberg-go/cmd/iceberg && go build .

Feature Support / Roadmap

FileSystem Support

Filesystem TypeSupported
S3X
Google Cloud StorageX
Azure Blob Storage
Local FilesystemX

Metadata

OperationSupported
Get SchemaX
Get SnapshotsX
Get Sort OrdersX
Get Partition SpecsX
Get ManifestsX
Create New ManifestsX
Plan Scanx
Plan Scan for Snapshotx

Catalog Support

OperationRESTHiveDynamoDBGlueSQL
Load TableXXX
List TablesXXX
Create TableXXX
Update Current SnapshotX
Create New SnapshotX
Rename TableXXX
Drop TableXXX
Alter TableXX
Set Table PropertiesXX
Create NamespaceXXX
Check Namespace ExistsXXX
Drop NamespaceXXX
Set Namespace PropertiesXXX
List ViewX
Drop ViewX
Check View ExistsX

Read/Write Data Support

  • Data can currently be read as an Arrow Table or as a stream of Arrow record batches.

Supported Write Operations

As long as the FileSystem is supported and the Catalog supports altering the table, the following tracks the current write support:

OperationSupported
Append Stream
Append Data FilesX
Rewrite Files
Rewrite manifests
Overwrite Files
Write Pos Delete
Write Eq Delete
Row Delta

CLI Usage

Run go build ./cmd/iceberg from the root of this repository to build the CLI executable, alternately you can run go install github.com/apache/iceberg-go/cmd/iceberg to install it to the bin directory of your GOPATH.

The iceberg CLI usage is very similar to pyiceberg CLI
You can pass the catalog URI with --uri argument.

Example: You can start the Iceberg REST API docker image which runs on default in port 8181

docker pull apache/iceberg-rest-fixture:latest
docker run -p 8181:8181 apache/iceberg-rest-fixture:latest

and run the iceberg CLI pointing to the REST API server.

 ./iceberg --uri http://0.0.0.0:8181 list
┌─────┐
| IDs |
| --- |
└─────┘

Create Namespace

./iceberg --uri http://0.0.0.0:8181 create namespace taxitrips

List Namespace

 ./iceberg --uri http://0.0.0.0:8181 list
┌───────────┐
| IDs       |
| --------- |
| taxitrips |
└───────────┘


Get in Touch