Getting Started with Apache Polaris, Relational JDBC, Postgres and Spark SQL

This example requires jq to be installed on your machine.

1. If such an image is not already present, build the Polaris image with support for JDBC persistence and the Postgres JDBC driver:

   ./gradlew \
      :polaris-server:assemble \
      :polaris-server:quarkusAppPartsBuild --rerun \
      :polaris-admin:assemble \
      :polaris-admin:quarkusAppPartsBuild --rerun \
      -Dquarkus.container-image.build=true
   

2. Start the docker compose group by running the following command from the root of the repository:

   export QUARKUS_DATASOURCE_JDBC_URL=jdbc:postgresql://postgres:5432/POLARIS
   export QUARKUS_DATASOURCE_USERNAME=postgres
   export QUARKUS_DATASOURCE_PASSWORD=postgres
   export ASSETS_PATH=$(pwd)/getting-started/assets/
   export CLIENT_ID=root
   export CLIENT_SECRET=s3cr3t
   docker compose -f getting-started/jdbc/docker-compose-bootstrap-db.yml -f getting-started/assets/postgres/docker-compose-postgres.yml -f getting-started/jdbc/docker-compose.yml up
   

3. Using spark-sql: attach to the running spark-sql container:

   docker attach $(docker ps -q --filter name=spark-sql)
   

   You may not see Spark's prompt immediately, type ENTER to see it. A few commands that you can try:

   CREATE NAMESPACE polaris.ns1;
   USE polaris.ns1;
   CREATE TABLE table1 (id int, name string);
   INSERT INTO table1 VALUES (1, 'a');
   SELECT * FROM table1;
   

4. To access Polaris from the host machine, first request an access token:

   export POLARIS_TOKEN=$(curl -s http://localhost:8181/api/catalog/v1/oauth/tokens \
      --user root:s3cr3t \
      -d 'grant_type=client_credentials' \
      -d 'scope=PRINCIPAL_ROLE:ALL' | jq -r .access_token)
   

5. Then, use the access token in the Authorization header when accessing Polaris:

   curl -v http://localhost:8181/api/management/v1/principal-roles -H "Authorization: Bearer $POLARIS_TOKEN"
   curl -v http://localhost:8181/api/management/v1/catalogs/quickstart_catalog -H "Authorization: Bearer $POLARIS_TOKEN"
   

6. Using Trino CLI: To access the Trino CLI, run this command:

docker exec -it jdbc-trino-1 trino

Note, jdbc-trino-1 is the name of the Docker container.

Example Trino queries:

SHOW CATALOGS;
SHOW SCHEMAS FROM iceberg;
SHOW TABLES FROM iceberg.information_schema;
DESCRIBE iceberg.information_schema.tables;

CREATE SCHEMA iceberg.tpch;
CREATE TABLE iceberg.tpch.test_polaris AS SELECT 1 x;
SELECT * FROM iceberg.tpch.test_polaris;