The Polaris Policy framework empowers organizations to centrally define, manage, and enforce fine-grained governance, lifecycle, and operational rules across all data resources in the catalog.
With the policy API, you can:
A policy in Apache Polaris is a structured entity that defines rules governing actions on specified resources under predefined conditions. Each policy contains:
Polaris supports several predefined system policy types (prefixed with system.):
| Policy Type | Purpose | JSON-Schema | Applies To |
|---|---|---|---|
system.data-compaction | Defines rules for data file compaction operations | data-compaction/2025-02-03.json | Iceberg table, namespace, catalog |
system.metadata-compaction | Defines rules for metadata file compaction operations | metadata-compaction/2025-02-03.json | Iceberg table, namespace, catalog |
system.orphan-file-removal | Defines rules for removing orphaned files | orphan-file-removal/2025-02-03.json | Iceberg table, namespace, catalog |
system.snapshot-expiry | Defines rules for snapshot expiration | snapshot-expiry/2025-02-03.json | Iceberg table, namespace, catalog |
Support for additional predefined system policy types and custom policy type definitions is in progress. For more details, please refer to the roadmap.
The entity hierarchy in Polaris is structured as follows:
Catalog
|
Namespace
|
+-----------+----------+
| | |
Iceberg Iceberg Generic
Table View Table
Policies can be attached at any level, and inheritance flows from catalog down to namespace, then to tables and views.
Policies can be inheritable or non-inheritable:
The inheritance follows an override mechanism:
{{< alert important >}} Because an override completely replaces the same policy type at higher levels, only one instance of a given policy type can be attached to (and therefore affect) a resource. {{< /alert >}}
To create a policy, you need to provide a name, type, and optionally a description and content:
POST /polaris/v1/{prefix}/namespaces/{namespace}/policies { "name": "compaction-policy", "type": "system.data-compaction", "description": "Policy for optimizing table storage", "content": "{\"version\": \"2025-02-03\", \"enable\": true, \"config\": {\"target_file_size_bytes\": 134217728}}" }
The policy content is validated against a schema specific to its type. Here are a few policy content examples:
{ "version": "2025-02-03", "enable": true, "config": { "target_file_size_bytes": 134217728, "compaction_strategy": "bin-pack", "max-concurrent-file-group-rewrites": 5 } }
{ "version": "2025-02-03", "enable": true, "max_orphan_file_age_in_days": 30, "locations": ["s3://my-bucket/my-table-location"], "config": { "prefix_mismatch_mode": "ignore" } }
Policies can be attached to different resource levels:
Example of attaching a policy to a table:
PUT /polaris/v1/{prefix}/namespaces/{namespace}/policies/{policy-name}/mappings { "target": { "type": "table-like", "path": ["NS1", "NS2", "test_table_1"] } }
For inheritable policies, only one policy of a given type can be attached to a resource. For non-inheritable policies, multiple policies of the same type can be attached.
A user can view applicable policies on a resource (e.g., table, namespace, or catalog) as long as they have read permission on that resource.
Here is an example to find all policies that apply to a specific resource (including inherited policies):
GET /polaris/v1/catalog/applicable-policies?namespace=finance%1Fquarterly&target-name=transactions
Sample response:
{ "policies": [ { "name": "snapshot-expiry-policy", "type": "system.snapshot-expiry", "appliedAt": "namespace", "content": { "version": "2025-02-03", "enable": true, "config": { "min_snapshot_to_keep": 1, "max_snapshot_age_days": 2, "max_ref_age_days": 3 } } }, { "name": "compaction-policy", "type": "system.data-compaction", "appliedAt": "catalog", "content": { "version": "2025-02-03", "enable": true, "config": { "target_file_size_bytes": 134217728 } } } ] }
For the complete and up-to-date API specification, see the policy-api.yaml.