Google Git
Sign in
apache / pulsar / refs/tags/v2.7.2
tag289f894b9e0079b1ff6acd14c2ce536220020eb4
taggerEnrico Olivelli <eolivelli@apache.org>Mon May 10 11:40:20 2021 +0200
object7bf14b5ac049d71c7ff74bbe758cb41aaffeb0af 
Release v2.7.2
commit7bf14b5ac049d71c7ff74bbe758cb41aaffeb0af[log] [tgz]
authorfeynmanlin <feynmanlin@tencent.com>Tue May 04 02:14:21 2021 +0800
committerGitHub <noreply@github.com>Mon May 03 11:14:21 2021 -0700
tree12f4211d63eab436a2019d54d10f7f47d89177c2
parenta695aee1bd538ea5fa13405fdf89a22c0809fcba [diff]
Fix CPU 100% when deleting namespace (#10337) (#10454)

When deleting the namespace, the namespace Policies will be marked as deleted.
This will trigger topic's `onPoliciesUpdate`
However, in onPoliciesUpdate, the data of the Policies node on zk will be read, such as: `checkReplicationAndRetryOnFailure`
Due to the deletion of the namespace, the zk node may no longer exist at this time.
Failure to read data will trigger infinite retries.
https://github.com/apache/pulsar/blob/e970c2947aff9231202ab72bdbad047d85c55633/pulsar-broker/src/main/java/org/apache/pulsar/broker/service/persistent/PersistentTopic.java#L1175-L1193

If there are many topics, there will be a short-term CPU spike

![image](https://user-images.githubusercontent.com/9758905/115834541-ebc32480-a447-11eb-887a-95c4a3d1adf1.png)
2 files changed
tree: 12f4211d63eab436a2019d54d10f7f47d89177c2
  1. .github/
  2. .test-infra/
  3. .travis/
  4. bin/
  5. bouncy-castle/
  6. build/
  7. buildtools/
  8. conf/
  9. dashboard/
  10. deployment/
  11. dev/
  12. distribution/
  13. docker/
  14. docker-compose/
  15. jclouds-shaded/
  16. kafka-connect-avro-converter-shaded/
  17. managed-ledger/
  18. node_modules/
  19. protobuf/
  20. protobuf-shaded/
  21. pulsar-broker/
  22. pulsar-broker-auth-athenz/
  23. pulsar-broker-auth-sasl/
  24. pulsar-broker-common/
  25. pulsar-broker-shaded/
  26. pulsar-client/
  27. pulsar-client-1x-base/
  28. pulsar-client-admin/
  29. pulsar-client-admin-shaded/
  30. pulsar-client-all/
  31. pulsar-client-api/
  32. pulsar-client-auth-athenz/
  33. pulsar-client-auth-sasl/
  34. pulsar-client-cpp/
  35. pulsar-client-messagecrypto-bc/
  36. pulsar-client-shaded/
  37. pulsar-client-tools/
  38. pulsar-client-tools-test/
  39. pulsar-common/
  40. pulsar-config-validation/
  41. pulsar-discovery-service/
  42. pulsar-function-go/
  43. pulsar-functions/
  44. pulsar-io/
  45. pulsar-metadata/
  46. pulsar-proxy/
  47. pulsar-sql/
  48. pulsar-testclient/
  49. pulsar-transaction/
  50. pulsar-websocket/
  51. pulsar-zookeeper/
  52. pulsar-zookeeper-utils/
  53. site2/
  54. src/
  55. testmocks/
  56. tests/
  57. tiered-storage/
  58. .asf.yaml
  59. .gitignore
  60. .travis.yml
  61. CONTRIBUTING.md
  62. CONTRIBUTORS.md
  63. faq.md
  64. LICENSE
  65. NOTICE
  66. pom.xml
  67. README.md
README.md

logo

Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API.

Learn more about Pulsar at https://pulsar.apache.org

Main features

  • Horizontally scalable (Millions of independent topics and millions of messages published per second)
  • Strong ordering and consistency guarantees
  • Low latency durable storage
  • Topic and queue semantics
  • Load balancer
  • Designed for being deployed as a hosted service:
    • Multi-tenant
    • Authentication
    • Authorization
    • Quotas
    • Support mixing very different workloads
    • Optional hardware isolation
  • Keeps track of consumer cursor position
  • REST API for provisioning, admin and stats
  • Geo replication
  • Transparent handling of partitioned topics
  • Transparent batching of messages

Repositories

This repository is the main repository of Apache Pulsar. Pulsar PMC also maintains other repositories for components in the Pulsar ecosystem, including connectors, adapters, and other language clients.

Helm Chart

Ecosystem

Clients

Dashboard & Management Tools

Documentation

CI/CD

Build Pulsar

Requirements:

  • Java JDK 1.8 or Java JDK 11
  • Maven 3.3.9+

Compile and install:

$ mvn install -DskipTests

Minimal build (This skips most of external connectors and tiered storage handlers)

mvn install -Pcore-modules

Run Unit Tests:

$ mvn test

Run Individual Unit Test:

$ cd module-name (e.g: pulsar-client)
$ mvn test -Dtest=unit-test-name (e.g: ConsumerBuilderImplTest)

Run Selected Test packages:

$ cd module-name (e.g: pulsar-broker)
$ mvn test -pl module-name -Dinclude=org/apache/pulsar/**/*.java

Start standalone Pulsar service:

$ bin/pulsar standalone

Check https://pulsar.apache.org for documentation and examples.

Setting up your IDE

Apache Pulsar is using lombok so you have to ensure your IDE setup with required plugins.

Intellij

To configure annotation processing in IntelliJ:

  1. Open Annotation Processors Settings dialog box by going to Settings -> Build, Execution, Deployment -> Compiler -> Annotation Processors.

  2. Select the following buttons:

    1. “Enable annotation processing”
    2. “Obtain processors from project classpath”
    3. “Store generated sources relative to: Module content root”

  3. Set the generated source directories to be equal to the Maven directories:

    1. Set “Production sources directory:” to “target/generated-sources/annotations”.
    2. Set “Test sources directory:” to “target/generated-test-sources/test-annotations”.

  4. Click “OK”.

  5. Install the lombok plugin in intellij.

Eclipse

Follow the instructions here to configure your Eclipse setup.

Build Pulsar docs

Refer to the docs README.

Contact

Mailing lists
NameScope
users@pulsar.apache.orgUser-related discussionsSubscribeUnsubscribeArchives
dev@pulsar.apache.orgDevelopment-related discussionsSubscribeUnsubscribeArchives
Slack

Pulsar slack channel at https://apache-pulsar.slack.com/

You can self-register at https://apache-pulsar.herokuapp.com/

License

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

Crypto Notice

This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.

The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.

The following provides more details on the included cryptographic software: Pulsar uses the SSL library from Bouncy Castle written by http://www.bouncycastle.org.