[feat][client] PCIP-5 Support Pull Consumer (#24) ### Motivation Our organization is currently planning a migration from RocketMQ to Pulsar. To facilitate this transition, we aim to implement a standardized abstraction layer for MQ clients that encapsulates implementation details of specific messaging systems. This abstraction layer will allow seamless engine replacement while maintaining consistent client interfaces. However, two critical compatibility issues hinder the unification of message fetching patterns between RocketMQ/Kafka and Pulsar: **1. Interface Disparity:** - Pulsar currently lacks native support for offset-based message fetching (fetch/pull paradigm) that allows specifying exact message positions and batch sizes. **2. Positioning Mechanism Mismatch:** - RocketMQ/Kafka: Utilize monotonically increasing numerical offsets for message positioning and acknowledgment - Pulsar: Relies on composite MessageID (ledgerId + entryId + batchIndex) for message identification. Therefore, I propose to contribute a customized feature implementation to the Pulsar contributor repository that leverages existing Pulsar APIs to achieve functionality similar to RocketMQ's pull consumer. This implementation could serve companies like ours with standardized client requirements, allowing them to reuse this solution. ### Modifications 1. Upgrade Pulsar version to 4.0.8(features include PIP-415 a new API to get the message id by offset) #23 2. Implement RocketMQ-like pull consumer functionality using existing Pulsar APIs.
Pulsar java contrib is to provide a non-core code maintenance repository to collect plugin implementations, personalized features, experimental features, and best practices from users.
Plugin Contribution Guide lists the core interfaces in Pulsar that can be implemented by contributors, and provides implementation guidelines for each type of interface.
Plugin Implementation List lists the implemented plugins. Users can select the ones they need for reuse.
Personalization Features lists the customized features and experimental features that require modification to the Pulsar source code.
Best Practices lists the best practices for each function summarized by community contributions.
This project follows the terms of Apache License 2.0. You can format the code by mvn spotless:apply and generate license headers by mvn license:format. Please note that the code formatted by Spotless may still not meet the formatting requirements. Please run mvn checkstyle:check for inspection.
pulsar-java-contrib is actively in development. If you have some common use cases for plugins, please contact us and we‘ll be happy to support. Please open an issue to share your idea or suggestion. PRs are always welcome and greatly appreciated, but for larger functional changes a pre-coding introduction can be helpful to ensure this is the correct place and that active or conflicting efforts don’t exist.
Compared with pulsar , pulsar-java-contrib is faster, and there is no need to send emails for DISCUSSION and VOTE. it can be summarized as follows: