| <?xml version="1.0"?> |
| <!-- |
| ~ Licensed to the Apache Software Foundation (ASF) under one |
| ~ or more contributor license agreements. See the NOTICE file |
| ~ distributed with this work for additional information |
| ~ regarding copyright ownership. The ASF licenses this file |
| ~ to you under the Apache License, Version 2.0 (the |
| ~ "License"); you may not use this file except in compliance |
| ~ with the License. You may obtain a copy of the License at |
| ~ |
| ~ http://www.apache.org/licenses/LICENSE-2.0 |
| ~ |
| ~ Unless required by applicable law or agreed to in writing, |
| ~ software distributed under the License is distributed on an |
| ~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY |
| ~ KIND, either express or implied. See the License for the |
| ~ specific language governing permissions and limitations |
| ~ under the License. |
| --> |
| <document xmlns="http://maven.apache.org/XDOC/2.0" |
| xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" |
| xsi:schemaLocation="http://maven.apache.org/XDOC/2.0 http://maven.apache.org/xsd/xdoc-2.0.xsd"> |
| <properties> |
| <title>Streaming JSON Message Formatter</title> |
| </properties> |
| |
| <body> |
| <h1>Streaming JSON Message Formatter</h1> |
| |
| <section name="Overview"> |
| <p>Axis2 2.0.1 includes streaming message formatters for JSON responses. |
| These formatters wrap the transport OutputStream with a FlushingOutputStream |
| that pushes data to the HTTP layer every 64 KB (configurable), converting |
| a single buffered response into a stream of HTTP/2 DATA frames or HTTP/1.1 |
| chunked transfer encoding segments.</p> |
| |
| <p>Both GSON and Moshi variants are provided as drop-in replacements for |
| their respective base formatters. No service code changes are required.</p> |
| </section> |
| |
| <section name="Problem - Large HTTP Responses and Reverse Proxies"> |
| <p>When an Axis2 service returns a large JSON response (hundreds of MB), |
| the default formatter serializes the entire response into memory before |
| writing it to the wire. A reverse proxy (nginx, AWS ALB, or similar) may |
| reject the response due to body-size limits or buffering timeouts, |
| returning <code>502 Bad Gateway</code> to the client even though the |
| Axis2 service completed successfully.</p> |
| |
| <p>The streaming formatter eliminates this by flushing incrementally |
| during GSON/Moshi serialization. The proxy never sees the full response |
| body as a single buffer; it forwards chunks as they arrive.</p> |
| |
| <p><strong>Request path (client to server):</strong> The streaming |
| formatter operates on the <em>response</em> path only. For large |
| HTTP POST request bodies, note that neither |
| <a href="https://www.rfc-editor.org/rfc/rfc9110#section-8.6">HTTP/1.1 (RFC 9110)</a> |
| nor <a href="https://www.rfc-editor.org/rfc/rfc9113">HTTP/2 (RFC 9113)</a> |
| define a limit on request body size. HTTP/2 is actually better |
| suited for large requests because the body is sent as DATA frames |
| with flow control — the sender and receiver negotiate how much |
| data to send at a time, preventing buffer overflow. In practice, |
| size rejections come from infrastructure layers, not the HTTP |
| spec:</p> |
| |
| <ul> |
| <li><strong>Reverse proxies</strong> (CloudFlare, nginx) — |
| <code>client_max_body_size</code> or equivalent; increase |
| or remove the limit</li> |
| <li><strong>Load balancers</strong> (AWS ALB/NLB) — ALB imposes |
| no body size limit when routing to EC2/ECS targets. |
| The 1MB limit applies only to the ALB→Lambda integration |
| path (an AWS hard limit that cannot be increased). |
| NLB operates at TCP level and has no body size limit</li> |
| <li><strong>Web servers</strong> (Tomcat, WildFly) — |
| <code>maxPostSize</code> / <code>max-post-size</code>; |
| set to <code>-1</code> for unlimited</li> |
| </ul> |
| |
| <p>When a large POST is rejected, debug the specific error and |
| the infrastructure layer that imposed the limit — the fix is |
| usually a configuration change, not a code refactor. Breaking |
| requests into smaller payloads is a last resort when the |
| infrastructure limit cannot be changed.</p> |
| </section> |
| |
| <section name="Available Variants"> |
| <table> |
| <tr><th>JSON Library</th><th>Formatter Class</th><th>Replaces</th></tr> |
| <tr> |
| <td>GSON</td> |
| <td><code>org.apache.axis2.json.streaming.JSONStreamingMessageFormatter</code></td> |
| <td><code>org.apache.axis2.json.gson.JsonFormatter</code></td> |
| </tr> |
| <tr> |
| <td>Moshi</td> |
| <td><code>org.apache.axis2.json.streaming.MoshiStreamingMessageFormatter</code></td> |
| <td><code>org.apache.axis2.json.moshi.JsonFormatter</code></td> |
| </tr> |
| </table> |
| |
| <p>Both variants share the same <code>FlushingOutputStream</code> |
| implementation in the <code>org.apache.axis2.json.streaming</code> |
| package.</p> |
| </section> |
| |
| <section name="Configuration"> |
| |
| <subsection name="Global (axis2.xml)"> |
| <p>Replace the default JSON message formatter with the streaming variant. |
| The recommended configuration uses <code>FieldFilteringMessageFormatter</code>, |
| which wraps the Moshi streaming formatter and adds support for response |
| field selection via a <code>?fields=</code> query parameter:</p> |
| <source><![CDATA[<!-- Recommended: streaming + field filtering (wraps MoshiStreamingMessageFormatter) --> |
| <messageFormatter contentType="application/json" |
| class="org.apache.axis2.json.streaming.FieldFilteringMessageFormatter"/> |
| |
| <!-- OR: streaming only (Moshi, no field filtering) --> |
| <messageFormatter contentType="application/json" |
| class="org.apache.axis2.json.streaming.MoshiStreamingMessageFormatter"/> |
| |
| <!-- OR: streaming only (GSON variant) --> |
| <messageFormatter contentType="application/json" |
| class="org.apache.axis2.json.streaming.JSONStreamingMessageFormatter"/>]]></source> |
| <p>All JSON-RPC services in the deployment will use the streaming |
| formatter. Existing services require no code changes.</p> |
| </subsection> |
| |
| <subsection name="Field Selection (?fields=)"> |
| <p>When using <code>FieldFilteringMessageFormatter</code>, callers can |
| reduce response payload size by specifying which fields to include. |
| This is useful for AI agents (MCP tools), mobile clients, and API |
| consumers that need only a subset of the response.</p> |
| |
| <p><b>Flat filtering</b> — select top-level response fields:</p> |
| <source><![CDATA[GET /services/MyService?fields=status,result |
| # Response: {"response":{"status":"SUCCESS","result":0.0245}} |
| # (other top-level fields omitted)]]></source> |
| |
| <p><b>Multi-level dot-notation</b> — filter inside nested objects and |
| collections. This is the key feature for services that return large |
| nested data structures.</p> |
| |
| <p>Consider a service that returns a response POJO where the heavy |
| data lives inside a <code>Map<String, Object></code> — common |
| when the service parses JSON from a backend into Java Collections |
| rather than typed POJOs:</p> |
| |
| <source><![CDATA[// The response POJO — what the Axis2 service method returns |
| public class ServiceResponse { |
| private String status; |
| private long responseTimeMs; |
| private Map<String, Object> data; // parsed from backend JSON |
| } |
| |
| // At runtime, "data" contains: |
| // "records" -> List<Map<String, Object>> (100+ fields per element) |
| // "metadata" -> Map<String, Object> |
| // "diagnostics" -> Map<String, Object>]]></source> |
| |
| <p>The Java path from the response root to a record field maps |
| directly to the <code>?fields=</code> syntax:</p> |
| |
| <source><![CDATA[response.getData() // Map<String, Object> = "data" |
| .get("records") // List<Map> = "data.records" |
| .get(0).get("id") // Object = "data.records.id" |
| |
| ?fields=status,data.records.id,data.records.name |
| | | | | |
| | | | +-- 3rd level: key inside each List element |
| | | +------------- 2nd level: key inside the data Map |
| | +----------------------- 1st level: field on the response POJO |
| +------------------------------- top-level POJO field (no dots)]]></source> |
| |
| <p><b>Before</b> (5MB, 127 fields per record):</p> |
| <source><![CDATA[{"response": { |
| "status": "SUCCESS", |
| "data": { |
| "records": [ |
| {"id":"item-1", "name":"Widget A", ... 125 more fields ...}, |
| {"id":"item-2", "name":"Widget B", ... 125 more fields ...} |
| ], |
| "metadata": {...}, |
| "diagnostics": {...} |
| } |
| }}]]></source> |
| |
| <p><b>After</b> <code>?fields=status,data.records.id</code> (~150KB, 97% reduction):</p> |
| <source><![CDATA[{"response": { |
| "status": "SUCCESS", |
| "data": { |
| "records": [ |
| {"id":"item-1"}, |
| {"id":"item-2"} |
| ] |
| } |
| }}]]></source> |
| |
| <p>Filtering happens during serialization — excluded fields are never |
| serialized, never buffered, never written to the wire. The streaming |
| pipeline is preserved end-to-end. When no <code>fields</code> parameter |
| is present, the formatter delegates directly with zero overhead.</p> |
| |
| <p>Multi-level dot-notation works on POJOs, Maps, and Collections — |
| including <code>Map<String, Object></code> containing |
| <code>List<Map<String, Object>></code>. Both Moshi and GSON |
| formatters support the same filtering behavior.</p> |
| |
| <p><b>Competitive context:</b> No other Java or Python JSON web services |
| framework ships recursive multi-level field filtering that operates on |
| runtime Maps and Collections from a query parameter. The closest |
| comparable is gRPC FieldMask (Protobuf only) and GraphQL (requires a |
| different API architecture). Spring + Jackson, Django REST, and FastAPI |
| all require custom serializer code for nested field selection. |
| Axis2/Java achieves this with zero service-side code changes.</p> |
| |
| <p><b>Limitation:</b> Field names containing a literal dot character |
| cannot be selected, as the dot is always interpreted as a nesting |
| delimiter. The Axis2/C implementation supports single-level |
| dot-notation only; multi-level is an Axis2/Java extension.</p> |
| </subsection> |
| |
| <subsection name="Flush Interval Tuning (services.xml)"> |
| <p>The default flush interval is 64 KB. Override per-service:</p> |
| <source><![CDATA[<parameter name="streamingFlushIntervalBytes">131072</parameter>]]></source> |
| <p>Smaller values increase flush frequency (lower latency to first |
| byte, more HTTP frames). Larger values reduce flush overhead at the |
| cost of larger transport buffers before each flush.</p> |
| </subsection> |
| |
| </section> |
| |
| <section name="How It Works"> |
| <p>The streaming formatter is structurally identical to the standard |
| GSON/Moshi formatter with one difference: before creating the |
| <code>JsonWriter</code>, it wraps the transport <code>OutputStream</code> |
| with a <code>FlushingOutputStream</code>:</p> |
| |
| <source><![CDATA[// Inside writeTo(): |
| OutputStream flushingStream = new FlushingOutputStream(outputStream, flushInterval); |
| // ... GSON/Moshi serialization proceeds normally against the flushing stream]]></source> |
| |
| <p>During serialization, every time the accumulated bytes exceed the |
| flush interval, the <code>FlushingOutputStream</code> calls |
| <code>flush()</code> on the underlying transport stream. This triggers |
| the servlet container (Tomcat, WildFly/Undertow) to send the buffered |
| bytes as an HTTP/2 DATA frame or HTTP/1.1 chunk. The reverse proxy |
| receives and forwards each chunk independently.</p> |
| |
| <p>The three serialization paths (fault response, element response, |
| and object response) all benefit from flushing without any path-specific |
| changes.</p> |
| </section> |
| |
| <section name="Services That Benefit"> |
| <p>The streaming formatter applies to <strong>all</strong> Axis2 |
| JSON-RPC services in the deployment. Any service that returns a |
| large JSON response benefits transparently:</p> |
| |
| <ul> |
| <li><strong>BigDataH2Service</strong> — enterprise big data |
| processing with large record sets. The streaming formatter |
| prevents proxy rejections as response sizes grow into the |
| hundreds of MB range.</li> |
| <li><strong>FinancialBenchmarkService</strong> — portfolio |
| variance, Monte Carlo VaR, and scenario analysis. Responses |
| are typically small (1-10 KB) but the formatter operates |
| transparently with no overhead on small payloads.</li> |
| <li><strong>Any custom service</strong> — services deployed |
| as <code>.aar</code> archives benefit without code changes |
| once the formatter is configured in <code>axis2.xml</code>.</li> |
| </ul> |
| </section> |
| |
| <section name="Testing"> |
| <p>The streaming formatter has been tested on:</p> |
| <ul> |
| <li>WildFly 32 (local) — all services produce valid JSON</li> |
| <li>WildFly 32 behind a reverse proxy (HTTP/2 ALPN on port 8443) |
| — all services produce bit-identical results compared to the |
| non-streaming formatter</li> |
| </ul> |
| <p>To verify the formatter is active, enable DEBUG logging for |
| <code>org.apache.axis2.json.streaming</code> and look for:</p> |
| <source>MoshiStreamingMessageFormatter: using FlushingOutputStream with 65536 byte flush interval</source> |
| </section> |
| |
| <section name="Axis2/C Equivalent"> |
| <p><a href="https://axis.apache.org/axis2/c/core/">Axis2/C</a> |
| achieves the same streaming behavior natively through Apache httpd's |
| <code>mod_h2</code>. During JSON response generation, the C service |
| calls <code>ap_rflush(r)</code> periodically to flush the response |
| bucket brigade. This causes <code>mod_h2</code> to emit HTTP/2 DATA |
| frames incrementally — the same 64KB chunked pattern as the Java |
| formatter, with identical proxy behavior. Both implementations |
| produce matching HTTP/2 frame sequences for the same payload.</p> |
| </section> |
| |
| </body> |
| </document> |