Apache Pulsar introduced the topic-level policies by PIP-39. It uses the reader API to read messages and relies on compaction to implement table-like logic in the memory.
PIP-104 Introduced new consumer type TableView
to support the same logic by Public Stable API.
Due to a lot of problems caused by the complex topic policy logic is as follows. we can use a new stable TableView
API instead of previous one.
none.
sequenceDiagram participant client participant topic_policy_service participant system_event_topic client ->> topic_policy_service: Hi, what's the topic policy of `persistent://public/default/test` loop policy update event fetching topic_policy_service ->> system_event_topic: Do you have new events?; alt new policy event system_event_topic ->> topic_policy_service: Yes, Sure. here you are. else no policy event system_event_topic ->> topic_policy_service: No, Please wait for a moment. end; end Note right of topic_policy_service: Update the local cache table by keeping the latest one; topic_policy_service -->> client: Here you are.
The new implementation will continue reuse the previous system topic namespace/__change_event
to store the topic policy data. and it will change the reading logic from the raw reader to TableView to enhance the robustness.
none.
none.
none.
none.
none.
none.
none.
none.
Without backward compatibility since we will reuse the same system topic namesapce/__change_event
.
none.
none.
none.
none.
none.