feat: add adaptive gzip compression for gRPC responses Motivation: The previous PR always used Identity (no compression) for native gRPC responses to avoid CPU overhead on small messages. However, large messages benefit significantly from gzip compression. An adaptive approach that compresses only messages above a configurable threshold provides the best of both worlds. Modification: - Add AdaptiveGzip codec that only compresses messages above a configurable threshold (default 1024 bytes). Messages below the threshold pass through uncompressed while the grpc-encoding header still advertises gzip. - Add Codec.compressWithFlag() returning (ByteString, Boolean) to signal per-frame compression status accurately in the gRPC frame header (bit 0 of the 5-byte header). This fixes a correctness bug where streaming frames would incorrectly set the compression flag even for uncompressed data. - Add pekko.grpc.server.compression-threshold config (default 1024). Generated Scala handlers read config once at handler creation and pass the threshold as Int to avoid per-request config overhead. - Pre-compute NativeIdentityAdaptiveGzip negotiation result and cache the default AdaptiveGzip instance for zero per-request allocation on the common path. - Add ProtobufFrameSerializer.serializedDataSize() for efficient size checking without double serialization. - Add adaptive fast path in GrpcResponseHelpers for small messages using serializeDataFrame single-allocation path. Result: - Messages below threshold: zero compression CPU overhead, sent uncompressed with correct frame header flag. - Messages above threshold: gzip compressed with correct frame header flag. Per gRPC spec, per-frame compression flag tells clients whether each frame is compressed. - Default threshold of 1024 bytes aligns with industry consensus (Netty, Ktor, OneUptime gRPC guide) as the break-even point where gzip CPU cost is justified by bandwidth savings on protobuf data. - JIT/GC optimized: zero per-request allocation on common path via pre-computed negotiation result and cached AdaptiveGzip singleton. Tests: - runtime / Test / test - 152 tests passed - plugin-tester-scala / Test / testOnly ErrorReportingSpec - passed - 30 new AdaptiveGzip tests covering codec behavior, boundary conditions, compressWithFlag, streaming frame header correctness, round-trip encode/decode, instance caching, and negotiate integration References: Refs #739
Support for building streaming gRPC servers and clients on top of Apache Pekko Streams.
This library is meant to be used as a building block in projects using the Pekko toolkit.
This library is ready to be used in production, but API's and build system plugins are still expected to be improved and may change.
The API on both sides (Client and Server) is a simple Pekko Streams-based one.
The client side is currently implemented on top of io.grpc:grpc-netty-shaded, we plan to replace this by just io.grpc:grpc-core and Pekko HTTP.
As for performance, we are currently relying on the JVM TLS implementation, which is sufficient for many use cases, but is planned to be replaced with conscrypt or netty-tcnative.
gRPC is a schema-first RPC framework, where your protocol is declared in a protobuf definition, and requests and responses will be streamed over an HTTP/2 connection.
Based on a protobuf service definition, pekko-grpc can generate:
SourcesThe project is split up in a number of subprojects:
Additionally, ‘plugin-tester-java’ and ‘plugin-tester-scala’ contain an example project in Java and Scala respectively, with both sbt and Gradle configurations.
gradlew scripts that will install the right version of Gradle and run the gradle tasks using it.build.sbt in this directory)sbt compile compiles the main source for project default version of Scala (2.13)sbt +compile will compile for all supported versions of Scalasbt test will compile the code and run the unit testssbt testQuick similar to test but when repeated in shell mode will only run failing testssbt package will build the jarsruntime/target/scala-2.13/sbt publishLocal will push the jars to your local Apache Ivy repositorysbt publishM2 will push the jars to your local Apache Maven repositorysbt docs/paradox will build the docs (the ones describing the module features)sbt docs/paradoxBrowse does the same but will open the docs in your browser when completeindex.html file will appear in target/paradox/site/main/sbt unidoc will build the Javadocs for all the modules and load them to one place (may require Graphviz, see Prerequisites above)index.html file will appear in target/scala-2.13/unidoc/sbt sourceDistGenerate will generate source release to target/dist/sbt "set ThisBuild / version := \"1.0.0\"; sourceDistGenerate"version.sbt to the same directory that has the build.sbt containing something likeThisBuild / version := "1.0.0"plugin-tester-java dirmvn -Dpekko.grpc.project.version=<version> pekko-grpc:generate compileplugin-tester-scala dirmvn -Dpekko.grpc.project.version=<version> pekko-grpc:generate scala:compileplugin-tester-java dir./gradlew clean test -Dpekko.grpc.project.version=<version>plugin-tester-scala dir./gradlew clean test -Dpekko.grpc.project.version=<version>Pekko gRPC is Open Source and available under the Apache 2 License.