blob: 4b4bc00539ee4c0999f806c296f1dc664c684e99 [file]
<?xml version="1.0"?>
<!--
~ Licensed to the Apache Software Foundation (ASF) under one
~ or more contributor license agreements. See the NOTICE file
~ distributed with this work for additional information
~ regarding copyright ownership. The ASF licenses this file
~ to you under the Apache License, Version 2.0 (the
~ "License"); you may not use this file except in compliance
~ with the License. You may obtain a copy of the License at
~
~ http://www.apache.org/licenses/LICENSE-2.0
~
~ Unless required by applicable law or agreed to in writing,
~ software distributed under the License is distributed on an
~ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
~ KIND, either express or implied. See the License for the
~ specific language governing permissions and limitations
~ under the License.
-->
<document xmlns="http://maven.apache.org/XDOC/2.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/XDOC/2.0 http://maven.apache.org/xsd/xdoc-2.0.xsd">
<properties>
<title>Streaming JSON Message Formatter</title>
</properties>
<body>
<h1>Streaming JSON Message Formatter</h1>
<section name="Overview">
<p>Axis2 2.0.1 includes streaming message formatters for JSON responses.
These formatters wrap the transport OutputStream with a FlushingOutputStream
that pushes data to the HTTP layer every 64 KB (configurable), converting
a single buffered response into a stream of HTTP/2 DATA frames or HTTP/1.1
chunked transfer encoding segments.</p>
<p>Both GSON and Moshi variants are provided as drop-in replacements for
their respective base formatters. No service code changes are required.</p>
</section>
<section name="Problem - Large HTTP Responses and Reverse Proxies">
<p>When an Axis2 service returns a large JSON response (hundreds of MB),
the default formatter serializes the entire response into memory before
writing it to the wire. A reverse proxy (nginx, AWS ALB, or similar) may
reject the response due to body-size limits or buffering timeouts,
returning <code>502 Bad Gateway</code> to the client even though the
Axis2 service completed successfully.</p>
<p>The streaming formatter eliminates this by flushing incrementally
during GSON/Moshi serialization. The proxy never sees the full response
body as a single buffer; it forwards chunks as they arrive.</p>
<p><strong>Request path (client to server):</strong> The streaming
formatter operates on the <em>response</em> path only. For large
HTTP POST request bodies, note that neither
<a href="https://www.rfc-editor.org/rfc/rfc9110#section-8.6">HTTP/1.1 (RFC 9110)</a>
nor <a href="https://www.rfc-editor.org/rfc/rfc9113">HTTP/2 (RFC 9113)</a>
define a limit on request body size. HTTP/2 is actually better
suited for large requests because the body is sent as DATA frames
with flow control — the sender and receiver negotiate how much
data to send at a time, preventing buffer overflow. In practice,
size rejections come from infrastructure layers, not the HTTP
spec:</p>
<ul>
<li><strong>Reverse proxies</strong> (CloudFlare, nginx) —
<code>client_max_body_size</code> or equivalent; increase
or remove the limit</li>
<li><strong>Load balancers</strong> (AWS ALB/NLB) — ALB imposes
no body size limit when routing to EC2/ECS targets.
The 1MB limit applies only to the ALB→Lambda integration
path (an AWS hard limit that cannot be increased).
NLB operates at TCP level and has no body size limit</li>
<li><strong>Web servers</strong> (Tomcat, WildFly) —
<code>maxPostSize</code> / <code>max-post-size</code>;
set to <code>-1</code> for unlimited</li>
</ul>
<p>When a large POST is rejected, debug the specific error and
the infrastructure layer that imposed the limit — the fix is
usually a configuration change, not a code refactor. Breaking
requests into smaller payloads is a last resort when the
infrastructure limit cannot be changed.</p>
</section>
<section name="Available Variants">
<table>
<tr><th>JSON Library</th><th>Formatter Class</th><th>Replaces</th></tr>
<tr>
<td>GSON</td>
<td><code>org.apache.axis2.json.streaming.JSONStreamingMessageFormatter</code></td>
<td><code>org.apache.axis2.json.gson.JsonFormatter</code></td>
</tr>
<tr>
<td>Moshi</td>
<td><code>org.apache.axis2.json.streaming.MoshiStreamingMessageFormatter</code></td>
<td><code>org.apache.axis2.json.moshi.JsonFormatter</code></td>
</tr>
</table>
<p>Both variants share the same <code>FlushingOutputStream</code>
implementation in the <code>org.apache.axis2.json.streaming</code>
package.</p>
</section>
<section name="Configuration">
<subsection name="Global (axis2.xml)">
<p>Replace the default JSON message formatter with the streaming variant.
The recommended configuration uses <code>FieldFilteringMessageFormatter</code>,
which wraps the Moshi streaming formatter and adds support for response
field selection via a <code>?fields=</code> query parameter:</p>
<source><![CDATA[<!-- Recommended: streaming + field filtering (wraps MoshiStreamingMessageFormatter) -->
<messageFormatter contentType="application/json"
class="org.apache.axis2.json.streaming.FieldFilteringMessageFormatter"/>
<!-- OR: streaming only (Moshi, no field filtering) -->
<messageFormatter contentType="application/json"
class="org.apache.axis2.json.streaming.MoshiStreamingMessageFormatter"/>
<!-- OR: streaming only (GSON variant) -->
<messageFormatter contentType="application/json"
class="org.apache.axis2.json.streaming.JSONStreamingMessageFormatter"/>]]></source>
<p>All JSON-RPC services in the deployment will use the streaming
formatter. Existing services require no code changes.</p>
</subsection>
<subsection name="Field Selection (?fields=)">
<p>When using <code>FieldFilteringMessageFormatter</code>, callers can
reduce response payload size by specifying which fields to include.
This is useful for AI agents (MCP tools), mobile clients, and API
consumers that need only a subset of the response.</p>
<p><b>Flat filtering</b> — select top-level response fields:</p>
<source><![CDATA[GET /services/MyService?fields=status,result
# Response: {"response":{"status":"SUCCESS","result":0.0245}}
# (other top-level fields omitted)]]></source>
<p><b>Multi-level dot-notation</b> — filter inside nested objects and
collections. This is the key feature for services that return large
nested data structures.</p>
<p>Consider a service that returns a response POJO where the heavy
data lives inside a <code>Map&lt;String, Object&gt;</code> — common
when the service parses JSON from a backend into Java Collections
rather than typed POJOs:</p>
<source><![CDATA[// The response POJO — what the Axis2 service method returns
public class ServiceResponse {
private String status;
private long responseTimeMs;
private Map<String, Object> data; // parsed from backend JSON
}
// At runtime, "data" contains:
// "records" -> List<Map<String, Object>> (100+ fields per element)
// "metadata" -> Map<String, Object>
// "diagnostics" -> Map<String, Object>]]></source>
<p>The Java path from the response root to a record field maps
directly to the <code>?fields=</code> syntax:</p>
<source><![CDATA[response.getData() // Map<String, Object> = "data"
.get("records") // List<Map> = "data.records"
.get(0).get("id") // Object = "data.records.id"
?fields=status,data.records.id,data.records.name
| | | |
| | | +-- 3rd level: key inside each List element
| | +------------- 2nd level: key inside the data Map
| +----------------------- 1st level: field on the response POJO
+------------------------------- top-level POJO field (no dots)]]></source>
<p><b>Before</b> (5MB, 127 fields per record):</p>
<source><![CDATA[{"response": {
"status": "SUCCESS",
"data": {
"records": [
{"id":"item-1", "name":"Widget A", ... 125 more fields ...},
{"id":"item-2", "name":"Widget B", ... 125 more fields ...}
],
"metadata": {...},
"diagnostics": {...}
}
}}]]></source>
<p><b>After</b> <code>?fields=status,data.records.id</code> (~150KB, 97% reduction):</p>
<source><![CDATA[{"response": {
"status": "SUCCESS",
"data": {
"records": [
{"id":"item-1"},
{"id":"item-2"}
]
}
}}]]></source>
<p>Filtering happens during serialization — excluded fields are never
serialized, never buffered, never written to the wire. The streaming
pipeline is preserved end-to-end. When no <code>fields</code> parameter
is present, the formatter delegates directly with zero overhead.</p>
<p>Multi-level dot-notation works on POJOs, Maps, and Collections —
including <code>Map&lt;String, Object&gt;</code> containing
<code>List&lt;Map&lt;String, Object&gt;&gt;</code>. Both Moshi and GSON
formatters support the same filtering behavior.</p>
<p><b>Competitive context:</b> No other Java or Python JSON web services
framework ships recursive multi-level field filtering that operates on
runtime Maps and Collections from a query parameter. The closest
comparable is gRPC FieldMask (Protobuf only) and GraphQL (requires a
different API architecture). Spring + Jackson, Django REST, and FastAPI
all require custom serializer code for nested field selection.
Axis2/Java achieves this with zero service-side code changes.</p>
<p><b>Limitation:</b> Field names containing a literal dot character
cannot be selected, as the dot is always interpreted as a nesting
delimiter. The Axis2/C implementation supports single-level
dot-notation only; multi-level is an Axis2/Java extension.</p>
</subsection>
<subsection name="Flush Interval Tuning (services.xml)">
<p>The default flush interval is 64 KB. Override per-service:</p>
<source><![CDATA[<parameter name="streamingFlushIntervalBytes">131072</parameter>]]></source>
<p>Smaller values increase flush frequency (lower latency to first
byte, more HTTP frames). Larger values reduce flush overhead at the
cost of larger transport buffers before each flush.</p>
</subsection>
</section>
<section name="How It Works">
<p>The streaming formatter is structurally identical to the standard
GSON/Moshi formatter with one difference: before creating the
<code>JsonWriter</code>, it wraps the transport <code>OutputStream</code>
with a <code>FlushingOutputStream</code>:</p>
<source><![CDATA[// Inside writeTo():
OutputStream flushingStream = new FlushingOutputStream(outputStream, flushInterval);
// ... GSON/Moshi serialization proceeds normally against the flushing stream]]></source>
<p>During serialization, every time the accumulated bytes exceed the
flush interval, the <code>FlushingOutputStream</code> calls
<code>flush()</code> on the underlying transport stream. This triggers
the servlet container (Tomcat, WildFly/Undertow) to send the buffered
bytes as an HTTP/2 DATA frame or HTTP/1.1 chunk. The reverse proxy
receives and forwards each chunk independently.</p>
<p>The three serialization paths (fault response, element response,
and object response) all benefit from flushing without any path-specific
changes.</p>
</section>
<section name="Services That Benefit">
<p>The streaming formatter applies to <strong>all</strong> Axis2
JSON-RPC services in the deployment. Any service that returns a
large JSON response benefits transparently:</p>
<ul>
<li><strong>BigDataH2Service</strong> — enterprise big data
processing with large record sets. The streaming formatter
prevents proxy rejections as response sizes grow into the
hundreds of MB range.</li>
<li><strong>FinancialBenchmarkService</strong> — portfolio
variance, Monte Carlo VaR, and scenario analysis. Responses
are typically small (1-10 KB) but the formatter operates
transparently with no overhead on small payloads.</li>
<li><strong>Any custom service</strong> — services deployed
as <code>.aar</code> archives benefit without code changes
once the formatter is configured in <code>axis2.xml</code>.</li>
</ul>
</section>
<section name="Testing">
<p>The streaming formatter has been tested on:</p>
<ul>
<li>WildFly 32 (local) — all services produce valid JSON</li>
<li>WildFly 32 behind a reverse proxy (HTTP/2 ALPN on port 8443)
— all services produce bit-identical results compared to the
non-streaming formatter</li>
</ul>
<p>To verify the formatter is active, enable DEBUG logging for
<code>org.apache.axis2.json.streaming</code> and look for:</p>
<source>MoshiStreamingMessageFormatter: using FlushingOutputStream with 65536 byte flush interval</source>
</section>
<section name="Axis2/C Equivalent">
<p><a href="https://axis.apache.org/axis2/c/core/">Axis2/C</a>
achieves the same streaming behavior natively through Apache httpd's
<code>mod_h2</code>. During JSON response generation, the C service
calls <code>ap_rflush(r)</code> periodically to flush the response
bucket brigade. This causes <code>mod_h2</code> to emit HTTP/2 DATA
frames incrementally — the same 64KB chunked pattern as the Java
formatter, with identical proxy behavior. Both implementations
produce matching HTTP/2 frame sequences for the same payload.</p>
</section>
</body>
</document>