commit | 97c62cb19a4eb3a938ec81496102d3bcfb011d64 | [log] [tgz] |
---|---|---|
author | Robert Stupp <snazy@snazy.de> | Thu Oct 02 03:53:03 2025 +0200 |
committer | GitHub <noreply@github.com> | Wed Oct 01 21:53:03 2025 -0400 |
tree | 99a45e1e1c307f159a1209d336a577adb542930d | |
parent | 395f7bdc23edd55bc3c33095f258f3026a92478e [diff] |
NoSQL: minor change to polaris-core for paging (#2740) Non-functional change for existing code, adds a `Page` factory function.
Apache Polaris™ is an open-source, fully-featured catalog for Apache Iceberg™. It implements Iceberg's REST API, enabling seamless multi-engine interoperability across a wide range of platforms, including Apache Doris™, Apache Flink®, Apache Spark™, Dremio® OSS, StarRocks, and Trino.
Documentation is available at https://polaris.apache.org. The REST OpenAPI specifications are available here: Polaris management API doc and Polaris Catalog API doc.
Subscribe to the dev mailing list to join discussions via email or browse the archives. Check out the CONTRIBUTING guide for contribution guidelines.
Click here for a quick overview of Polaris.
Click here for the quickstart experience, which will help you set up a Polaris instance locally or on any supported cloud provider.
Apache Polaris is organized into the following modules:
polaris-core
- The main Polaris entity definitions and core business logicpolaris-api-management-model
- Polaris Management API model classespolaris-api-management-service
- Polaris Management API service classespolaris-api-iceberg-service
- The Iceberg REST service classespolaris-api-catalog-service
- The Polaris Catalog API service classespolaris-admin
- The Polaris Admin Tool; mainly for bootstrapping persistencepolaris-runtime-defaults
- The runtime configuration defaultspolaris-distribution
- The Polaris distributionpolaris-server
- The Polaris Quarkus Serverpolaris-runtime-service
- The package containing the Polaris service.polaris-runtime-spark-tests
- Integration tests for the Polaris Spark pluginpolaris-runtime-test-common
- Test utilitiespolaris-eclipselink
- The Eclipselink implementation of the MetaStoreManager interface (deprecated for removal)polaris-relational-jdbc
- The JDBC implementation of BasePersistence to be used via AtomicMetaStoreManagerpolaris-extensions-federation-hadoop
- The Hadoop federation extensionpolaris-extensions-federation-hive
- The Hive federation extensionagregated-license-report
- Generates the aggregated license reportpolaris-bom
- The Bill of Materials (BOM) for Polarispolaris-build-logic
- Establishes consistent build logicpolaris-tests
- Normative integration tests for reuse in downstream projectspolaris-config-docs-annotations
- Annotations for documentation generatorpolaris-config-docs-generator
- Generates Polaris reference docspolaris-config-docs-site
- The configuration documentation sitepolaris-container-spec-helper
- Helper for container specificationspolaris-immutables
- Predefined Immutables configuration & annotations for Polarispolaris-minio-testcontainer
- Minio test containerpolaris-misc-types
- Miscellaneous types for Polarispolaris-version
- Versioning for PolarisIn addition to modules, there are:
Apache Polaris is built using Gradle with Java 21+ and Docker 27+.
./gradlew build
- To build and run tests. Make sure Docker is running, as the integration tests depend on it../gradlew assemble
- To skip tests../gradlew check
- To run all checks, including unit tests and integration tests../gradlew run
- To run the Polaris server locally; the server is reachable at localhost:8181. This is also suitable for running regression tests, or for connecting with Spark. Set your own credentials by specifying system property ./gradlew run -Dpolaris.bootstrap.credentials=POLARIS,root,secret
where:POLARIS
is the realmroot
is the CLIENT_IDsecret
is the CLIENT_SECRETPOLARIS,root,s3cr3t
./regtests/run_spark_sql.sh
- To connect from Spark SQL. Here are some example commands to run in the Spark SQL shell:create database db1; show databases; create table db1.table1 (id int, name string); insert into db1.table1 values (1, 'a'); select * from db1.table1;
env POLARIS_HOST=localhost ./regtests/run.sh
- To run regression tests locally, see more options here.To streamline the developer experience, especially for common setup and build tasks, a root-level Makefile is available. This Makefile acts as a convenient wrapper around various Gradle commands and other tooling, simplifying interactions. While Gradle remains the primary build system, the Makefile provides concise shortcuts for frequent operations like:
make build-server, make build-admin
make minikube-start-cluster, make minikube-cleanup
make helm-doc-generate, make helm-unittest
make install-dependencies-brew
make client-lint, make client-regenerate
To see available commands:
make help
For example, to build the Polaris server and its container image, you can simply run:
make build-server
./gradlew \ :polaris-server:assemble \ :polaris-server:quarkusAppPartsBuild --rerun \ -Dquarkus.container-image.build=true
docker run -p 8181:8181 -p 8182:8182 apache/polaris:latest
- To run the image.The Polaris codebase contains some docker compose examples to quickly get started with Polaris, using different configurations. Check the ./getting-started
directory for more information.
helm/polaris
for more information.Polaris Servers can be configured using a variety of ways. Please see the Configuration Guide for more information.
Default configuration values can be found in runtime/defaults/src/main/resources/application.properties
.
site/bin/run-hugo-in-docker.sh
site/
for more information.Build scans of CI builds from a branch or tag in the apache/polaris
repository on GitHub publish build scans to the ASF Develocity instance at develocity.apache.org, if the workflow runs have access to the Apache organization-level secret DEVELOCITY_ACCESS_KEY
.
Build scans of local developer builds publish build scans only if the Gradle command line option --scan
is used. Those build scans are published to Gradle‘s public Develocity instance (see advanced configuration options below). Note that build scans on Gradle’s public Develocity instance are publicly accessible to anyone. You have to accept Gradle‘s terms of service to publish to the Gradle’s public Develocity instance.
CI builds originating from pull requests against the apache/polaris
GitHub repository are published to Gradle's public Develocity instance.
Other CI build scans do only publish build scans to the Gradle's public Develocity instance, if the environment variable GRADLE_TOS_ACCEPTED
is set to true
. By setting this variable you agree to the Gradle's terms of service, because accepting these ToS is your personal decision. You can configure this environment variable for your GitHub repository in the GitHub repository settings under Secrets
> Secrets and variables
> Actions
> choose the Variables
tab > New repository variable
.
Advanced configuration options for publishing build scans (only local and non-apache/polaris
repository CI):
DEVELOCITY_PROJECT_ID
. The project ID defaults to the GitHub repository owner/name, for example octocat/polaris
.DEVELOCITY_SERVER
if build scans should be published to another than Gradle's public Develocity instance.DEVELOCITY_ACCESS_KEY
.Apache Polaris is under the Apache License Version 2.0. See the LICENSE.
Apache Polaris™ is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.
Apache®, Apache Polaris™, Apache Iceberg™, Apache Spark™ are either registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries.