blob: 537f3c42df16b0e7e1481b4892ccacb3e906f8fa [file] [view]
---
id: admin-api-schemas
title: Manage Schemas
sidebar_label: "Schemas"
description: Learn how to manage schemas using Pulsar CLI and admin APIs.
---
````mdx-code-block
import Tabs from '@theme/Tabs';
import TabItem from '@theme/TabItem';
````
:::tip
This page only shows **some frequently used operations**.
- For the latest and complete information about `Pulsar admin`, including commands, flags, descriptions, and more, see [Pulsar admin docs](/reference/#/@pulsar:version_reference@/pulsar-admin/).
- For the latest and complete information about `REST API`, including parameters, responses, samples, and more, see {@inject: rest:REST:/} API doc.
- For the latest and complete information about `Java admin API`, including classes, methods, descriptions, and more, see [Java admin API doc](@pulsar:javadoc:admin@/).
:::
## Manage schema
### Upload a schema
To upload (register) a new schema for a topic, you can use one of the following methods.
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
Use the `upload` subcommand.
```bash
pulsar-admin schemas upload --filename <schema-definition-file> <topic-name>
```
The `schema-definition-file` is in JSON format.
```json
{
"type": "<schema-type>",
"schema": "<an-utf8-encoded-string-of-schema-definition-data>",
"properties": {} // the properties associated with the schema
}
```
</TabItem>
<TabItem value="REST API">
Send a `POST` request to the endpoint documented here: {@inject: endpoint|POST|/admin/v2/schemas/:tenant/:namespace/:topic/schema|operation/SchemasResource_postSchema?version=@pulsar:version_number@}
Below is an example with CURL with a payload stored on the `schema.json` file, Pulsar broker running on `localhost` and the topic `my-tenant/my-ns/my-topic`:
```bash
curl -X POST -H 'Content-Type: application/json' -d @schema.json http://localhost:8080/admin/v2/schemas/my-tenant/my-ns/my-topic/schema
```
The post payload is in JSON format.
```json
{
"type": "<schema-type>",
"schema": "<an-utf8-encoded-string-of-schema-definition-data>",
"properties": {} // the properties associated with the schema
}
```
</TabItem>
<TabItem value="Java">
The method on `PulsarAdmin` client is:
```java
void createSchema(String topic, PostSchemaPayload schemaPayload)
```
Here is an example of `PostSchemaPayload`:
```java
PulsarAdmin admin = …;
PostSchemaPayload payload = new PostSchemaPayload();
payload.setType("INT8");
payload.setSchema("");
admin.createSchema("my-tenant/my-ns/my-topic", payload);
```
If the schema is a **primitive** schema, the `schema` field must be blank.
If the schema is a **struct** schema, this field must be a JSON string of the Avro schema definition.
</TabItem>
</Tabs>
````
The payload includes the following fields:
| Field | Description |
|--------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `type` | <li>Allowed values for primitive-type schemas are listed on the following page: [Primitive types](schema-understand.md#primitive-type)</li><li>Allowed values for struct-type schemas are **AVRO**, **PROTOBUF**, **PROTOBUF_NATIVE** and **JSON**.</li> |
| `schema` | The schema definition data, which is encoded in UTF 8 charset. <li>If the schema type is **AVRO**, **PROTOBUF** or **JSON** schema, this field should be an <a href="https://avro.apache.org/docs/1.11.1/specification/" target="blank">Avro schema definition</a> in JSON format.</li><li>If the schema type is **PROTOBUF_NATIVE** schema, this field should contain a Protobuf descriptor. </li><li>Otherwise, this field should be blank.</li> |
| `properties` | The additional properties associated with the schema. |
The following is an example for a JSON schema.
**Example**
```json
{
"type": "JSON",
"schema": "{\"type\":\"record\",\"name\":\"User\",\"namespace\":\"com.foo\",\"fields\":[{\"name\":\"file1\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"file2\",\"type\":[\"null\",\"string\"],\"default\":null},{\"name\":\"file3\",\"type\":[\"string\",\"null\"],\"default\":\"dfdf\"}]}",
"properties": {}
}
```
### Get the latest schema
To get the latest schema for a topic, you can use one of the following methods.
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
Use the `get` subcommand.
```bash
pulsar-admin schemas get <topic-name>
```
Example output:
```json
{
"version": 0,
"type": "String",
"timestamp": 0,
"data": "string",
"properties": {
"property1": "string",
"property2": "string"
}
}
```
</TabItem>
<TabItem value="REST API">
Send a `GET` request to this endpoint: {@inject: endpoint|GET|/admin/v2/schemas/:tenant/:namespace/:topic/schema|operation/SchemasResource_getSchema?version=@pulsar:version_number@}
Here is an example of a response, which is returned in JSON format.
```json
{
"version": "<the-version-number-of-the-schema>",
"type": "<the-schema-type>",
"timestamp": "<the-creation-timestamp-of-the-version-of-the-schema>",
"data": "<an-utf8-encoded-string-of-schema-definition-data>",
"properties": {} // the properties associated with the schema
}
```
</TabItem>
<TabItem value="Java">
```java
SchemaInfo createSchema(String topic)
```
Here is an example of `SchemaInfo`:
```java
PulsarAdmin admin = …;
SchemaInfo si = admin.getSchema("my-tenant/my-ns/my-topic");
```
</TabItem>
</Tabs>
````
### Get a specific schema
To get a specific version of a schema, you can use one of the following methods.
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
Use the `get` subcommand.
```bash
pulsar-admin schemas get <topic-name> --version <version>
```
</TabItem>
<TabItem value="REST API">
Send a `GET` request to a schema endpoint: {@inject: endpoint|GET|/admin/v2/schemas/:tenant/:namespace/:topic/schema/:version|operation/SchemasResource_getSchema?version=@pulsar:version_number@}
Here is an example of a response, which is returned in JSON format.
```json
{
"version": "<the-version-number-of-the-schema>",
"type": "<the-schema-type>",
"timestamp": "<the-creation-timestamp-of-the-version-of-the-schema>",
"data": "<an-utf8-encoded-string-of-schema-definition-data>",
"properties": {} // the properties associated with the schema
}
```
</TabItem>
<TabItem value="Java">
```java
SchemaInfo createSchema(String topic, long version)
```
Here is an example of `SchemaInfo`:
```java
PulsarAdmin admin = …;
SchemaInfo si = admin.getSchema("my-tenant/my-ns/my-topic", 1L);
```
</TabItem>
</Tabs>
````
### Extract a schema
To extract (provide) a schema via a topic, use the following method.
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"}]}>
<TabItem value="Admin CLI">
Use the `extract` subcommand.
```bash
pulsar-admin schemas extract --classname <class-name> --jar <absolute-jar-path> --type <type-name>
```
</TabItem>
</Tabs>
````
### Delete a schema
:::note
In any case, the `delete` action deletes **all versions** of a schema registered for a topic.
:::
To delete a schema for a topic, you can use one of the following methods.
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
Use the `delete` subcommand.
```bash
pulsar-admin schemas delete <topic-name>
```
</TabItem>
<TabItem value="REST API">
Send a `DELETE` request to a schema endpoint: {@inject: endpoint|DELETE|/admin/v2/schemas/:tenant/:namespace/:topic/schema|operation/SchemasResource_deleteSchema?version=@pulsar:version_number@}
Here is an example of a response returned in JSON format.
```json
{
"version": "<the-latest-version-number-of-the-schema>",
}
```
</TabItem>
<TabItem value="Java">
```java
void deleteSchema(String topic)
```
Here is an example of deleting a schema.
```java
PulsarAdmin admin = …;
admin.deleteSchema("my-tenant/my-ns/my-topic");
```
</TabItem>
</Tabs>
````
## Manage schema AutoUpdate
### Enable schema AutoUpdate
To enable/enforce schema auto-update at the namespace level, you can use one of the following methods.
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
Use the `set-is-allow-auto-update-schema` subcommand.
```bash
bin/pulsar-admin namespaces set-is-allow-auto-update-schema --enable tenant/namespace
```
</TabItem>
<TabItem value="REST API">
Send a `POST` request to a namespace endpoint: {@inject: endpoint|POST|/admin/v2/namespaces/:tenant/:namespace/isAllowAutoUpdateSchema|operation/Namespaces_setIsAllowAutoUpdateSchema?version=@pulsar:version_number@}
The post payload is in JSON format.
```json
{
“isAllowAutoUpdateSchema”: “true”
}
```
</TabItem>
<TabItem value="Java">
Here is an example to enable schema auto-update for a tenant/namespace.
```java
admin.namespaces().setIsAllowAutoUpdateSchema("my-namspace", true);
```
</TabItem>
</Tabs>
````
### Disable schema AutoUpdate
:::note
When schema auto-update is disabled, you can only [register a new schema](#upload-a-schema).
:::
To disable schema auto-update at the **namespace** level, you can use one of the following commands.
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
Use the `set-is-allow-auto-update-schema` subcommand.
```bash
bin/pulsar-admin namespaces set-is-allow-auto-update-schema --disable tenant/namespace
```
</TabItem>
<TabItem value="REST API">
Send a `POST` request to a namespace endpoint: {@inject: endpoint|POST|/admin/v2/namespaces/:tenant/:namespace/isAllowAutoUpdateSchema|operation/Namespaces_setIsAllowAutoUpdateSchema?version=@pulsar:version_number@}
The post payload is in JSON format.
```json
{
“isAllowAutoUpdateSchema”: “false”
}
```
</TabItem>
<TabItem value="Java">
Here is an example to enable schema auto-unpdate of a tenant/namespace.
```java
admin.namespaces().setIsAllowAutoUpdateSchema("my-namspace", false);
```
</TabItem>
</Tabs>
````
## Manage schema validation enforcement
### Enable schema validation enforcement
To enforce schema validation enforcement at the **cluster** level, you can configure `isSchemaValidationEnforced` to `true` in the `conf/broker.conf` file.
To enable schema validation enforcement at the **namespace** level, you can use one of the following commands.
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
Use the `set-schema-validation-enforce` subcommand.
```bash
bin/pulsar-admin namespaces set-schema-validation-enforce --enable tenant/namespace
```
</TabItem>
<TabItem value="REST API">
Send a `POST` request to a namespace endpoint: {@inject: endpoint|POST|/admin/v2/namespaces/:tenant/:namespace/schemaValidationEnforced|operation/Namespaces_setSchemaValidationEnforced?version=@pulsar:version_number@}
The post payload is in JSON format.
```json
{
“schemaValidationEnforced”: “true”
}
```
</TabItem>
<TabItem value="Java">
Here is an example to enable schema validation enforcement for a tenant/namespace.
```java
admin.namespaces().setSchemaValidationEnforced("my-namspace", true);
```
</TabItem>
</Tabs>
````
### Disable schema validation enforcement
To disable schema validation enforcement at the **namespace** level, you can use one of the following commands.
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
Use the `set-schema-validation-enforce` subcommand.
```bash
bin/pulsar-admin namespaces set-schema-validation-enforce --disable tenant/namespace
```
</TabItem>
<TabItem value="REST API">
Send a `POST` request to a namespace endpoint: {@inject: endpoint|POST|/admin/v2/namespaces/:tenant/:namespace/schemaValidationEnforced|operation/Namespaces_setSchemaValidationEnforced?version=@pulsar:version_number@}
The post payload is in JSON format.
```json
{
“schemaValidationEnforced”: “false”
}
```
</TabItem>
<TabItem value="Java">
Here is an example to enable schema validation enforcement for a tenant/namespace.
```java
admin.namespaces().setSchemaValidationEnforced("my-namspace", false);
```
</TabItem>
</Tabs>
````
## Manage schema compatibility strategy
The [schema compatibility check strategy](schema-understand.md#schema-compatibility-check-strategy) configured at different levels has priority: topic level > namespace level > cluster level. In other words:
* If you set the strategy at both topic and namespace levels, the topic-level strategy is used.
* If you set the strategy at both namespace and cluster levels, the namespace-level strategy is used.
### Set schema compatibility strategy
#### Set topic-level schema compatibility strategy
To set a schema compatibility check strategy at the topic level, you can use one of the following methods.
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
Use the [`pulsar-admin topicPolicies set-schema-compatibility-strategy`](/reference/#/@pulsar:version_reference@/pulsar-admin/topicPolicies?id=set-schema-compatibility-strategy) command.
```shell
pulsar-admin topicPolicies set-schema-compatibility-strategy <strategy> <topicName>
```
</TabItem>
<TabItem value="REST API">
Send a `PUT` request to this endpoint: {@inject: endpoint|PUT|/admin/v2/topics/:tenant/:namespace/:topic|operation/PersistentTopics_setSchemaCompatibilityStrategy?version=@pulsar:version_number@}
</TabItem>
<TabItem value="Java">
```java
void setSchemaCompatibilityStrategy(String topic, SchemaCompatibilityStrategy strategy)
```
Here is an example of setting a schema compatibility check strategy at the topic level.
```java
PulsarAdmin admin = …;
admin.topicPolicies().setSchemaCompatibilityStrategy("my-tenant/my-ns/my-topic", SchemaCompatibilityStrategy.ALWAYS_INCOMPATIBLE);
```
</TabItem>
</Tabs>
````
#### Set namespace-level schema compatibility strategy
To set schema compatibility check strategy at the namespace level, you can use one of the following methods.
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
Use the [`pulsar-admin namespaces set-schema-compatibility-strategy`](/reference/#/@pulsar:version_reference@/pulsar-admin/namespaces?id=set-schema-compatibility-strategy) command.
```shell
pulsar-admin namespaces set-schema-compatibility-strategy options
```
</TabItem>
<TabItem value="REST API">
Send a `PUT` request to this endpoint: {@inject: endpoint|PUT|/admin/v2/namespaces/:tenant/:namespace/schemaCompatibilityStrategy|operation/Namespaces_setSchemaCompatibilityStrategy?version=@pulsar:version_number@}
</TabItem>
<TabItem value="Java">
Use the [`setSchemaCompatibilityStrategy`](@pulsar:javadoc:admin@/) method.
```java
admin.namespaces().setSchemaCompatibilityStrategy("test", SchemaCompatibilityStrategy.FULL);
```
</TabItem>
</Tabs>
````
#### Set cluster-level schema compatibility strategy
To set schema compatibility check strategy at the **cluster** level, set `schemaCompatibilityStrategy` in the `conf/broker.conf` file.
The following is an example:
```conf
schemaCompatibilityStrategy=ALWAYS_INCOMPATIBLE
```
### Get schema compatibility strategy
#### Get topic-level schema compatibility strategy
To get the topic-level schema compatibility check strategy, you can use one of the following methods.
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
Use the [`pulsar-admin topicPolicies get-schema-compatibility-strategy`](/reference/#/@pulsar:version_reference@/pulsar-admin/topicPolicies?id=get-schema-compatibility-strategy) command.
```shell
pulsar-admin topicPolicies get-schema-compatibility-strategy <topicName>
```
</TabItem>
<TabItem value="REST API">
Send a `GET` request to this endpoint: {@inject: endpoint|GET|/admin/v2/topics/:tenant/:namespace/:topic/schemaCompatibilityStrategy|operation/PersistentTopics_getSchemaCompatibilityStrategy?version=@pulsar:version_number@}
</TabItem>
<TabItem value="Java">
```java
SchemaCompatibilityStrategy getSchemaCompatibilityStrategy(String topic, boolean applied)
```
Here is an example of getting the topic-level schema compatibility check strategy.
```java
PulsarAdmin admin = …;
// get the current applied schema compatibility strategy
admin.topicPolicies().getSchemaCompatibilityStrategy("my-tenant/my-ns/my-topic", true);
// only get the schema compatibility strategy from topic policies
admin.topicPolicies().getSchemaCompatibilityStrategy("my-tenant/my-ns/my-topic", false);
```
</TabItem>
</Tabs>
````
#### Get namespace-level schema compatibility strategy
You can get schema compatibility check strategy at namespace level using one of the following methods.
````mdx-code-block
<Tabs groupId="api-choice"
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java","value":"Java"}]}>
<TabItem value="Admin CLI">
Use the [`pulsar-admin namespaces get-schema-compatibility-strategy`](/reference/#/@pulsar:version_reference@/pulsar-admin/namespaces?id=get-schema-compatibility-strategy) command.
```shell
pulsar-admin namespaces get-schema-compatibility-strategy options
```
</TabItem>
<TabItem value="REST API">
Send a `GET` request to this endpoint: {@inject: endpoint|GET|/admin/v2/namespaces/:tenant/:namespace/schemaCompatibilityStrategy|operation/Namespaces_getSchemaCompatibilityStrategy?version=@pulsar:version_number@}
</TabItem>
<TabItem value="Java">
Use the [`getSchemaCompatibilityStrategy`](@pulsar:javadoc:admin@/) method.
```java
admin.namespaces().getSchemaCompatibilityStrategy("test", SchemaCompatibilityStrategy.FULL);
```
</TabItem>
</Tabs>
````