pip/pip-284.md

# PIP-284: Migrate topic policies implementation to use TableView

# Background knowledge

Apache Pulsar introduced the topic-level policies by [PIP-39](pip-39.md).
It uses the reader API to read messages and relies on compaction to implement table-like logic in the memory.

[PIP-104](https://github.com/apache/pulsar/issues/12356) Introduced new consumer type `TableView` to support the same logic by
Public Stable API.

# Motivation

Due to a lot of problems caused by the complex topic policy logic is as follows. we can use a new stable `TableView` API instead of
previous one.

- https://github.com/apache/pulsar/pull/20763
- https://github.com/apache/pulsar/pull/20613
- https://github.com/apache/pulsar/pull/19746
- etc...

# Goals

## In Scope

- Reduce complex logic.
- Reuse the public stable API to improve the project quality.

## Out of Scope

none.

# High Level Design

```mermaid
sequenceDiagram
    participant client
    participant topic_policy_service
    participant system_event_topic
    client ->> topic_policy_service: Hi, what's the topic policy of `persistent://public/default/test`
    loop policy update event fetching
        topic_policy_service ->> system_event_topic: Do you have new events?;
        alt new policy event
            system_event_topic ->> topic_policy_service: Yes, Sure. here you are.
        else no policy event
            system_event_topic ->> topic_policy_service: No, Please wait for a moment.
        end;
    end
    Note right of topic_policy_service: Update the local cache table by keeping the latest one;
    topic_policy_service -->> client: Here you are.
```

# Detailed Design

## Design & Implementation Details

The new implementation will continue reuse the previous system topic `namespace/__change_event` to store the topic policy data.
and it will change the reading logic from the raw reader to TableView to enhance the robustness.

## Public-facing Changes

none.

### Public API

none.

### Binary protocol

none.

### Configuration

none.

### CLI

none.

### Metrics

none.

# Monitoring

none.

# Security Considerations

none.

# Backward & Forward Compatibility

Without backward compatibility since we will reuse the same system topic `namesapce/__change_event`.

## Revert

none.

## Upgrade

none.

# Alternatives

none.

# General Notes

none.

# Links

none.

* Mailing List discussion thread:  https://lists.apache.org/thread/9v00sfpfxjpm775vgltkjoxwnllsgskg
* Mailing List voting thread: https://lists.apache.org/thread/gx43mzh88xp5ttz2gqghfqpz1yq51k60