pip/pip-373.md

# PIP-373: Add a topic's system prop that indicates whether users have published TXN messages in before.

# Background knowledge

**Transaction Buffer**
- It maintains `aborted` messages indexes and `ongoing` TXN messages indexes in memory, we call them `Two Indexes` in the following sections.
- It helps to filter out the messages that have been aborted or are still binding with an `on-going` TXN, to avoid consuming aborted messages or TXN in-progress messages.

**Transaction Buffer initializes when a topic is loading up**
- It reads messages that contain TXN state from the original topic, to re-build `Two Indexes` into memory.

**Transaction Buffer closing when a topic is unloading**
- It takes a snapshot for `Two Indexes` to rebuild the states faster next loading up.

# Motivation

- Since TXN is a feature that across multiple namespaces, it can be enabled or disabled at the cluster level.
- Transaction Buffer will try to re-build `Two Indexes` for every topic if you enable TXN.
- If you have a huge number of topics in a namespace, the task that re-build `Two Indexes` costs huge resources(CPU, Memory).

We'd better skip Transaction Buffer re-building `Two Indexes` if the topic it is related to does not contain TXN messages to save resources usage.

# Goals

Skip Transaction Buffer re-building `Two Indexes` if the topic it is related to does not contain TXN messages to save resources usage.

## In Scope

This PIP only focuses on improving the scenario in which users have never published TXN messages on a topic.

## Out of Scope

This PIP does not focus on the scenario that follows.
- Published TXN messages.
- Consumed all TXN messages that were sent before.
- Transaction Buffer keeps re-building `Two Indexes` even if there are no TXN messages in the topic anymore, and it costs resources.

# Detailed Design

## Design & Implementation Details

- Add a topic-level system property named `__contains_txn_messages`, the default value is `false`, and it will be set to `true` when the first TXN message is publishing.
- Transaction Buffer skips re-building `Two Indexes` if the property is `false`.

## Public-facing Changes

The topic property `__contains_txn_messages` becomes to a system property, it can never be used by users anymore.

### Public API
- You will get a `NotAllowedException` when you try to set/remove a topic property named `__contains_txn_messages` by the API `pulsar-admin topics update-properties/remove-properties`.
- The value of the property `__contains_txn_messages` can be queried by `pulsar-admin topics get-properties`.


### Metrics
| Name                                       | Description                                         | Attributes | Units|
|--------------------------------------------|-----------------------------------------------------| --- | --- |
| `pulsar_broker_using_txn_topics_count` | Counter. The number of topics contains TXN messages. | cluster | - |

# Backward & Forward Compatibility

## Upgrade

There are `3` scenarios that the topic's property `__contains_txn_messages` is when users try to upgrade.
- `__contains_txn_messages` is empty: broker initializes it by confirming whether there is TXN messages or not.
- **(Highlight)** `__contains_txn_messages` is not empty and is not typed `boolean`, rollback to the original behavior that always re-building `Two Indexes`, but you can never modify it anymore.
- **(Highlight)** `__contains_txn_messages` is not empty and is typed `boolean`, but it is a users' property, broker assumed that it is a system property.

## Downgrade / Rollback

You can downgrade or roll back gracefully.

## Pulsar Geo-Replication Upgrade & Downgrade/Rollback Considerations

The PIP does not affect Geo-Replication.

# Links
* Mailing List discussion thread: https://lists.apache.org/thread/7mblhyvsrw5zybo0gs5512xg8f9sm67v
* Mailing List voting thread: https://lists.apache.org/thread/s6z5gcjyw081cxf9pwz361r8dt2k8gvl