Remove bouncy castle shaded module to avoid bring error of verifySingleJar (#7453)
### Motivation
shade bouncy castle will cause some signature errors, this PR tries to remove the bouncy castle shaded module.
Here is the related error stack:
```
10:01:34.257 [pulsar-client-io-33-1] ERROR org.apache.pulsar.client.impl.ConsumerImpl - MessageCryptoBc may not included in the jar. e:
java.lang.SecurityException: JCE cannot authenticate the provider BC
at javax.crypto.Cipher.getInstance(Cipher.java:657) ~[?:1.8.0_121]
at javax.crypto.Cipher.getInstance(Cipher.java:596) ~[?:1.8.0_121]
at org.apache.pulsar.client.impl.crypto.MessageCryptoBc.<init>(MessageCryptoBc.java:147) ~[classes/:?]
at org.apache.pulsar.client.impl.ConsumerImpl.<init>(ConsumerImpl.java:270) ~[classes/:?]
at org.apache.pulsar.client.impl.ConsumerImpl.newConsumerImpl(ConsumerImpl.java:209) ~[classes/:?]
at org.apache.pulsar.client.impl.PulsarClientImpl.lambda$doSingleTopicSubscribeAsync$5(PulsarClientImpl.java:364) ~[classes/:?]
at java.util.concurrent.CompletableFuture.uniAccept(CompletableFuture.java:656) ~[?:1.8.0_131]
at java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:632) ~[?:1.8.0_131]
...
Caused by: java.util.jar.JarException: file:/Users/jia/.m2/repository/org/apache/pulsar/bouncy-castle-bc-shaded/2.7.0-SNAPSHOT/bouncy-castle-bc-shaded-2.7.0-SNAPSHOT.jar has unsigned entries - org/bouncycastle/cert/AttributeCertificateHolder.class
at javax.crypto.JarVerifier.verifySingleJar(JarVerifier.java:500) ~[?:1.8.0_121]
at javax.crypto.JarVerifier.verifyJars(JarVerifier.java:361) ~[?:1.8.0_121]
at javax.crypto.JarVerifier.verify(JarVerifier.java:289) ~[?:1.8.0_121]
at javax.crypto.JceSecurity.verifyProviderJar(JceSecurity.java:159) ~[?:1.8.0_121]
at javax.crypto.JceSecurity.getVerificationResult(JceSecurity.java:185) ~[?:1.8.0_121]
at javax.crypto.Cipher.getInstance(Cipher.java:653) ~[?:1.8.0_121]
```
### Modifications
- Remove bouncy castle shaded module, avoid package bouncy castle into a dependency jar.
- enhance test case to identify this error.
### Verifying this change
ut passed.
* remove dep of bc-shaded from other module
* remove bc-shaded module
* enhance testECDSAEncryption and testRSAEncryption to cover error case
* fix license check
* remove bc-shaded module
* build a jar in jar to avoid break bc signature
* use new bc dependency by classifier in maven
* build pulsar-all docker image instead of pull from dockerhub in integration tests
* remove nar
* fix licence, fix error brings in #7640
* add bc when broker/client is referenced in pom
* add missing bc reference in pom
* change ci back to not build docker image
diff --git a/distribution/server/pom.xml b/distribution/server/pom.xml
index 381ac6a..66af231 100644
--- a/distribution/server/pom.xml
+++ b/distribution/server/pom.xml
@@ -42,6 +42,14 @@
<dependency>
<groupId>${project.groupId}</groupId>
+ <artifactId>bouncy-castle-bc</artifactId>
+ <version>${project.version}</version>
+ <classifier>pkg</classifier>
+ <optional>true</optional>
+ </dependency>
+
+ <dependency>
+ <groupId>${project.groupId}</groupId>
<artifactId>pulsar-discovery-service</artifactId>
<version>${project.version}</version>
</dependency>
@@ -146,7 +154,7 @@
<groupId>io.grpc</groupId>
<artifactId>*</artifactId>
</exclusion>
- <!-- bouncy-castle-bc-shaded already contains this-->
+ <!-- bouncy-castle-bc already contains this-->
<exclusion>
<groupId>org.bouncycastle</groupId>
<artifactId>*</artifactId>
diff --git a/distribution/server/src/assemble/LICENSE.bin.txt b/distribution/server/src/assemble/LICENSE.bin.txt
index 899b8e4..0f0ace0 100644
--- a/distribution/server/src/assemble/LICENSE.bin.txt
+++ b/distribution/server/src/assemble/LICENSE.bin.txt
@@ -583,6 +583,12 @@
* Jcip -- licenses/LICENSE-jcip.txt
- net.jcip-jcip-annotations-1.0.jar
+Bouncy Castle License
+ * Bouncy Castle -- licenses/LICENSE-bouncycastle.txt
+ - org.bouncycastle-bcpkix-jdk15on-1.66.jar
+ - org.bouncycastle-bcprov-ext-jdk15on-1.66.jar
+ - org.bouncycastle-bcprov-jdk15on-1.66.jar
+
------------------------
Additionaly, Netty includes code with the following licenses:
diff --git a/tests/integration/pom.xml b/tests/integration/pom.xml
index 295339c..3219d3a 100644
--- a/tests/integration/pom.xml
+++ b/tests/integration/pom.xml
@@ -56,6 +56,13 @@
</dependency>
<dependency>
<groupId>org.apache.pulsar</groupId>
+ <artifactId>bouncy-castle-bc</artifactId>
+ <version>${project.version}</version>
+ <classifier>pkg</classifier>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-admin</artifactId>
<version>${project.version}</version>
<scope>test</scope>
diff --git a/tests/pulsar-kafka-compat-client-test/pom.xml b/tests/pulsar-kafka-compat-client-test/pom.xml
index e77cb8d..d0adddc 100644
--- a/tests/pulsar-kafka-compat-client-test/pom.xml
+++ b/tests/pulsar-kafka-compat-client-test/pom.xml
@@ -64,6 +64,13 @@
</dependency>
<dependency>
<groupId>org.apache.pulsar</groupId>
+ <artifactId>bouncy-castle-bc</artifactId>
+ <version>${project.version}</version>
+ <classifier>pkg</classifier>
+ <scope>test</scope>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-client-kafka</artifactId>
<version>${project.version}</version>
<scope>test</scope>
diff --git a/tests/pulsar-storm-test/pom.xml b/tests/pulsar-storm-test/pom.xml
index a0fef6e..0a0d524 100644
--- a/tests/pulsar-storm-test/pom.xml
+++ b/tests/pulsar-storm-test/pom.xml
@@ -47,7 +47,7 @@
</exclusion>
</exclusions>
</dependency>
-
+
<dependency>
<groupId>org.apache.storm</groupId>
<artifactId>storm-server</artifactId>
@@ -67,6 +67,14 @@
<dependency>
<groupId>org.apache.pulsar</groupId>
+ <artifactId>bouncy-castle-bc</artifactId>
+ <version>${project.version}</version>
+ <classifier>pkg</classifier>
+ <scope>test</scope>
+ </dependency>
+
+ <dependency>
+ <groupId>org.apache.pulsar</groupId>
<artifactId>pulsar-broker</artifactId>
<version>${project.version}</version>
<scope>test</scope>
