| commit | 13416ea03d0d36f0e87954a0450994bf3b46fec9 | [log] [tgz] |
|---|---|---|
| author | Yunze Xu <xyzinfernity@163.com> | Thu Sep 02 22:07:11 2021 +0800 |
| committer | chenhang <chenhang@apache.org> | Sat Sep 04 08:15:22 2021 +0800 |
| tree | 6605014e8bb6b15ad5fcd4fcfbbae982b7e2c04a | |
| parent | a2d2757bc4e67be2012846bc767c7ad4dc2314c7 [diff] |
Fix incorrect connect timeout (#11889) Fixes #10721 ### Motivation Currently the connection timer stops after the TCP connection is established immediately (the state becomes `TcpConnected`). However, the connect phase should also include sending the `CommandConnect` request and receiving the `CommandConnected` response from broker successfully. For example, in a case like #10721 described, if the Pulsar broker received a SIGSTOP signal and became pending, the TCP connection can be established but no response could be received from a stopped broker. Then the client would hang forever. ### Modifications Stop the connection timer only after the `ClientConnection`'s state becomes `Ready`, which means the client has received the `CommandConnected` response successfully. It's also consistent with Java client's implementation, see https://github.com/apache/pulsar/blob/235e678a56d0284e68b45e46706b6237d7c6d5f9/pulsar-client/src/main/java/org/apache/pulsar/client/impl/ClientCnx.java#L329-L330 ### Verifying this change - [ ] Make sure that the change passes the CI checks. It's hard to simulate the scenario in unit test. But we can simply reproduce it in local env. 1. Run a pulsar standalone and send `SIGSTOP` signal by `pkill -SIGSTOP -f pulsar`. 2. Run any C++/Python client with this fix to connect to the standalone. ``` >>> import pulsar >>> c = pulsar.Client('pulsar://localhost:6650') >>> c.create_producer('xxx') 2021-09-02 11:54:38.828 INFO [0x10d72ae00] ClientConnection:181 | [<none> -> pulsar://localhost:6650] Create ClientConnection, timeout=10000 2021-09-02 11:54:38.828 INFO [0x10d72ae00] ConnectionPool:96 | Created connection for pulsar://localhost:6650 2021-09-02 11:54:38.969 INFO [0x700003136000] ClientConnection:367 | [127.0.0.1:62784 -> 127.0.0.1:6650] Connected to broker 2021-09-02 11:54:48.973 ERROR [0x700003136000] ClientConnection:532 | [127.0.0.1:62784 -> 127.0.0.1:6650] Connection was not established in 10000 ms, close the socket 2021-09-02 11:54:48.973 ERROR [0x700003136000] ClientConnection:572 | [127.0.0.1:62784 -> 127.0.0.1:6650] Read failed: Operation canceled 2021-09-02 11:54:48.973 INFO [0x700003136000] ClientConnection:1495 | [127.0.0.1:62784 -> 127.0.0.1:6650] Connection closed 2021-09-02 11:54:48.973 ERROR [0x700003136000] ClientImpl:188 | Error Checking/Getting Partition Metadata while creating producer on persistent://public/default/xxx -- ConnectError Traceback (most recent call last): # ... _pulsar.ConnectError: Pulsar error: ConnectError ``` we can see after 10 seconds (the default connect timeout), it failed. (cherry picked from commit cdd0f41f90ee0a8a03201adf759af056b065a547)
Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API.
Learn more about Pulsar at https://pulsar.apache.org
This repository is the main repository of Apache Pulsar. Pulsar PMC also maintains other repositories for components in the Pulsar ecosystem, including connectors, adapters, and other language clients.
Requirements:
Compile and install:
$ mvn install -DskipTests
mvn install -Pcore-modules,-main
Run Unit Tests:
$ mvn test
Run Individual Unit Test:
$ cd module-name (e.g: pulsar-client) $ mvn test -Dtest=unit-test-name (e.g: ConsumerBuilderImplTest)
Run Selected Test packages:
$ cd module-name (e.g: pulsar-broker) $ mvn test -pl module-name -Dinclude=org/apache/pulsar/**/*.java
Start standalone Pulsar service:
$ bin/pulsar standalone
Check https://pulsar.apache.org for documentation and examples.
Docker images must be built with Java 8 for branch-2.7 or previous branches because of issue 8445. Java 11 is the recommended JDK version in master/branch-2.8.
This builds the docker images apachepulsar/pulsar-all:latest and apachepulsar/pulsar:latest.
mvn clean install -DskipTests mvn package -Pdocker,-main -am -pl docker/pulsar-all -DskipTests
After the images are built, they can be tagged and pushed to your custom repository. Here's an example of a bash script that tags the docker images with the current version and git revision and pushes them to localhost:32000/apachepulsar.
image_repo_and_project=localhost:32000/apachepulsar pulsar_version=$(mvn initialize help:evaluate -Dexpression=project.version -pl . -q -DforceStdout) gitrev=$(git rev-parse HEAD | colrm 10) tag="${pulsar_version}-${gitrev}" echo "Using tag $tag" docker tag apachepulsar/pulsar-all:latest ${image_repo_and_project}/pulsar-all:$tag docker push ${image_repo_and_project}/pulsar-all:$tag docker tag apachepulsar/pulsar:latest ${image_repo_and_project}/pulsar:$tag docker push ${image_repo_and_project}/pulsar:$tag
Apache Pulsar is using lombok so you have to ensure your IDE setup with required plugins.
Open Project Settings.
Click File -> Project Structure -> Project Settings -> Project.
Select the JDK version.
From the JDK version drop-down list, select Download JDK... or choose an existing recent Java 11 JDK version.
In the download dialog, select version 11. You can pick a version from many vendors. Unless you have a specific preference, choose AdoptOpenJDK (Hotspot).
Open Maven Importing Settings dialog by going to Settings -> Build, Execution, Deployment -> Build Tools -> Maven -> Importing.
Choose Use Project JDK for JDK for Importer setting. This uses the Java 11 JDK for running Maven when importing the project to IntelliJ. Some of the configuration in the Maven build is conditional based on the JDK version. Incorrect configuration gets chosen when the “JDK for Importer” isn't the same as the “Project JDK”.
Validate that the JRE setting in Maven -> Runner dialog is set to Use Project JDK.
Open Annotation Processors Settings dialog box by going to Settings -> Build, Execution, Deployment -> Compiler -> Annotation Processors.
Select the following buttons:
Set the generated source directories to be equal to the Maven directories:
Click OK.
Install the lombok plugin in intellij.
Open Code Style Settings dialog box by going to Settings -> Editor -> Code Style.
Click on the :gear: symbol -> Import scheme -> Intellij IDEA code style XML
Pick the file ${pulsar_dir}/src/idea-code-style.xml
On the dialog box that opens, click OK.
Ensure the scheme you just created is selected in Scheme dropdown then click OK.
Install the Checkstyle-IDEA plugin.
Open Checkstyle Settings dialog box by going to Settings -> Tools -> Checkstyle.
Set Checkstyle version to 8.37.
Set Scan scope to Only Java sources (including tests).
Click + button in the Configuration section to open a dialog to choose the checkfile file.
Activate the configuration you just added by toggling the corresponding box.
Click OK.
When working on the Pulsar core modules in IntelliJ, reduce the number of active projects in IntelliJ to speed up IDE actions and reduce unrelated IDE warnings.
Run the “Generate Sources and Update Folders For All Projects” action from the Maven UI toolbar. You can also find the action by the name in the IntelliJ “Search Everywhere” window that gets activated by pressing the Shift key twice. Running the action takes about 10 minutes for all projects. This is faster when the “core-modules” profile is the only active profile.
In the case of compilation errors with missing Protobuf classes, ensure to run the “Generate Sources and Update Folders For All Projects” action.
All of the Pulsar source code doesn't compile properly in IntelliJ and there are compilation errors.
mvn test -Dtest=TestClassName command.Follow the instructions here to configure your Eclipse setup.
Refer to the docs README.
| Name | Scope | |||
|---|---|---|---|---|
| users@pulsar.apache.org | User-related discussions | Subscribe | Unsubscribe | Archives |
| dev@pulsar.apache.org | Development-related discussions | Subscribe | Unsubscribe | Archives |
Pulsar slack channel at https://apache-pulsar.slack.com/
You can self-register at https://apache-pulsar.herokuapp.com/
Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0
This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.
The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.
The following provides more details on the included cryptographic software: Pulsar uses the SSL library from Bouncy Castle written by http://www.bouncycastle.org.