commit | 8465c55036eb715663aeca95277106255fee2842 | [log] [tgz] |
---|---|---|
author | wuYin <wuyinpost@gmail.com> | Thu Nov 05 16:47:20 2020 +0800 |
committer | GitHub <noreply@github.com> | Thu Nov 05 16:47:20 2020 +0800 |
tree | 9861f60220f51c44a5cfe5b3ed2066d47048a737 | |
parent | 793ea91e37ead175ccc6d61a62310fd8d3f968d1 [diff] |
Fix reconsume broken while using non-FQDN topics (#386) ### Issue Retry policy not effective with non-FQDN topic. - reproduction ```go client, _ := pulsar.NewClient(pulsar.ClientOptions{URL: "pulsar://localhost:6650"}) consumer, _ := client.Subscribe(pulsar.ConsumerOptions{ Topic: "topic-01", SubscriptionName: "my-sub", RetryEnable: true, DLQ: &pulsar.DLQPolicy{MaxDeliveries: 2}, }) msg, _ := consumer.Receive(context.Background()) consumer.ReconsumeLater(msg, 5*time.Second) ``` - logs ``` RN[0000] consumer of topic [persistent://public/default/topic-01] not exist unexpectedly topic="[topic-01 persistent://public/default/my-sub-RETRY]" ``` ### Cause For MultiTopicConsumer `consumers` map filed: - key: user provided topic, maybe non-FQDN. - value: consumer instance. `ReconsumeLater` using msg's FQDN topic as key to find `consumer` in `consumers`, if mismatch with non-FQDN topic, this invoke will be ignored, lead to Retry policy not effective. ### Modifications - Normalize user provided topics as FQDN topics before initializing consumers. - Add non-FQDN topic consumption case in Retry policy tests. ### Verifying this change - [x] Make sure that the change passes the CI checks.
A Go client library for the Apache Pulsar project.
This projects is developing a pure-Go client library for Pulsar that does not depend on the C++ Pulsar library.
Once feature parity and stability are reached, this will supersede the current CGo based library.
Check the Projects page at https://github.com/apache/pulsar-client-go/projects for tracking the status and the progress.
Import the client library:
import "github.com/apache/pulsar-client-go/pulsar"
Create a Producer:
client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
})
defer client.Close()
producer, err := client.CreateProducer(pulsar.ProducerOptions{
Topic: "my-topic",
})
_, err = producer.Send(context.Background(), &pulsar.ProducerMessage{
Payload: []byte("hello"),
})
defer producer.Close()
if err != nil {
fmt.Println("Failed to publish message", err)
}
fmt.Println("Published message")
Create a Consumer:
client, err := pulsar.NewClient(pulsar.ClientOptions{
URL: "pulsar://localhost:6650",
})
defer client.Close()
consumer, err := client.Subscribe(pulsar.ConsumerOptions{
Topic: "my-topic",
SubscriptionName: "my-sub",
Type: pulsar.Shared,
})
defer consumer.Close()
msg, err := consumer.Receive(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Printf("Received message msgId: %#v -- content: '%s'\n",
msg.ID(), string(msg.Payload()))
Create a Reader:
client, err := pulsar.NewClient(pulsar.ClientOptions{URL: "pulsar://localhost:6650"})
if err != nil {
log.Fatal(err)
}
defer client.Close()
reader, err := client.CreateReader(pulsar.ReaderOptions{
Topic: "topic-1",
StartMessageID: pulsar.EarliestMessageID(),
})
if err != nil {
log.Fatal(err)
}
defer reader.Close()
for reader.HasNext() {
msg, err := reader.Next(context.Background())
if err != nil {
log.Fatal(err)
}
fmt.Printf("Received message msgId: %#v -- content: '%s'\n",
msg.ID(), string(msg.Payload()))
}
Contributions are welcomed and greatly appreciated. See CONTRIBUTING.md for details on submitting patches and the contribution workflow.
Name | Scope | |||
---|---|---|---|---|
users@pulsar.apache.org | User-related discussions | Subscribe | Unsubscribe | Archives |
dev@pulsar.apache.org | Development-related discussions | Subscribe | Unsubscribe | Archives |
Pulsar slack channel #dev-go
at https://apache-pulsar.slack.com/
You can self-register at https://apache-pulsar.herokuapp.com/
Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0