HTTP/2 JSON Response Processing in Axis2/C

Overview

This document explains how Axis2/C processes JSON requests and generates HTTP/2 responses, based on extensive investigation of the curl “Binary output can mess up your terminal” issue.

CRITICAL WARNING: Never use Unicode emoji characters in logging statements - they break Apache's logging subsystem and prevent debug output from appearing.

Request Processing Flow

1. Initial Request Handling

HTTP/2 requests arrive at Apache HTTP Server with mod_http2 enabled
Apache routes requests to mod_axis2 based on URL patterns (e.g., /services/*)
The main entry point is axis2_apache2_worker_process_request() in apache2_worker.c

2. JSON Request Detection and Processing

File: src/core/transport/http/server/apache2/apache2_worker.c
Key Function: axis2_apache2_worker_process_request()
Lines: ~400-500 (request processor selection)

The worker determines the processing method based on Content-Type headers:

// HTTP/2 JSON interface pattern processing is selected when:
// - Content-Type: application/json
// - HTTP/2 protocol is detected
// - Request uses interface pattern (direct service method invocation)

3. JSON Response Generation

File: src/core/transport/http/server/apache2/axis2_apache2_request_processor_json_impl.c
Key Function: axis2_apache2_json_processor_parse_and_process_json()
Lines: ~880-940 (JSON response creation)

The JSON response is created using sprintf():

sprintf(json_response,
    "{\n"
    "  \"status\": \"success\",\n"
    "  \"message\": \"JSON request processed via interface pattern\",\n"
    "  \"service\": \"%s\",\n"
    "  \"timestamp\": \"%ld\",\n"
    "  \"request_size\": %d,\n"
    "  \"http2_optimized\": %s,\n"
    "  \"processing_mode\": \"interface_pattern\",\n"
    "  \"content_type\": \"%s\"\n"
    "}",
    service_path, timestamp, request_length,
    is_http2_optimized ? "true" : "false", content_type);

Critical Issue: sprintf() adds a null terminator (\0) at the end of the string.

4. Response Stream Processing

File: src/core/transport/http/server/apache2/apache2_worker.c
Key Lines: 1177-1178, 1389-1390, and 1520-1543

The actual response transmission happens through these steps:

// Step 1: Get response from output stream
body_string = axutil_stream_get_buffer(out_stream, env);
body_string_len = axutil_stream_get_len(out_stream, env);

// Step 2: Write to Apache response (lines 1520-1543)
if (body_string) {
    ap_rwrite(body_string, body_string_len, request);
}

The Null Byte Problem

Root Cause Analysis

After extensive investigation, we determined the null byte issue has multiple potential sources:

sprintf() null terminator: The JSON generation using sprintf() adds \0
Stream length calculation: axutil_stream_get_len() may include the null terminator
Apache HTTP/2 module: mod_http2 may add padding during HTTP/2 frame generation

Investigation Results

Multiple fix attempts were implemented but failed to resolve the issue:

Request Processor Fixes: Modified axis2_apache2_request_processor_json_impl.c to eliminate null bytes
Worker-Level Fixes: Added null byte scanning in apache2_worker.c:1520-1543
Stream Writing Fixes: Implemented clean buffer approaches with memcpy()

None of these fixes took effect, indicating the null byte is added after our Axis2/C code completes processing.

Final Determination

The null byte (0x00) at position 302 (end of JSON response) is added by Apache's HTTP/2 module during HTTP/2 DATA frame generation, not by Axis2/C code.

Evidence:

Issue only occurs with --http2 (HTTP/1.1 works fine)
Null byte persists despite comprehensive Axis2/C code fixes
HTTP/2 debug logs show DATA[length=302] which includes the null byte

HTTP Response Headers

Current Headers Set

The system correctly sets these HTTP response headers:

HTTP/2 202
access-control-allow-origin: *
access-control-allow-methods: GET, POST, PUT, DELETE, OPTIONS
access-control-allow-headers: Content-Type, Authorization, Accept
content-length: 302
content-type: application/json
date: Sun, 14 Dec 2025 22:04:06 GMT
server: Apache/2.4.64 (Unix) OpenSSL/3.5.3 Axis2C/1.7.0

Accept Header Implementation

The code attempts to set proper Accept headers in multiple locations:

Request Processor (axis2_apache2_request_processor_json_impl.c:246-250):

axutil_property_t* accept_prop = axutil_property_create(env);
axutil_property_set_value(accept_prop, env, axutil_strdup(env, "application/json"));
axis2_msg_ctx_set_property(out_msg_ctx, env, "Accept", accept_prop);

Apache Worker (apache2_worker.c:1100+):

request->content_type = "application/json";

However, the Accept header doesn‘t appear in the final HTTP response, suggesting it’s either:

Overridden by Apache configuration
Not properly propagated from message context to HTTP headers
Handled differently by HTTP/2 protocol implementation

Apache HTTP Server Issue

The Problem

Apache HTTP Server's mod_http2 module appears to add a null byte during HTTP/2 DATA frame generation. This causes curl to detect the response as binary content rather than text.

Technical Details

Affected Protocol: HTTP/2 only (HTTP/1.1 works correctly)
Symptom: curl displays “Binary output can mess up your terminal”
Cause: Null byte (0x00) at end of JSON response (position 302)
Apache Version: 2.4.64 with OpenSSL/3.5.3

Investigation Attempts

Multiple code-level fixes were attempted:

JSON generation without null terminators
Stream buffer cleaning with memcpy()
Aggressive null byte scanning and replacement
Response length truncation

Result: None effective, confirming the issue is in Apache HTTP/2 module.

Curl Workaround

Problem Command

curl -vk --http2 -H "Content-Type: application/json" \
     -d '{"action":"get_status"}' \
     https://localhost/services/CameraControlService/getStatus

Result: “Binary output can mess up your terminal” warning

Solution Commands

Option 1: Force Terminal Output

curl -vk --http2 -H "Content-Type: application/json" \
     -d '{"action":"get_status"}' \
     https://localhost/services/CameraControlService/getStatus \
     --output -

Option 2: Save to File

curl -vk --http2 -H "Content-Type: application/json" \
     -d '{"action":"get_status"}' \
     https://localhost/services/CameraControlService/getStatus \
     --output response.json

Option 3: Use HTTP/1.1 (if supported)

curl -vk --http1.1 -H "Content-Type: application/json" \
     -d '{"action":"get_status"}' \
     https://localhost/services/CameraControlService/getStatus

Note: Option 3 may not work if the service requires HTTP/2.

Code Path Summary

Definitive JSON Response Flow

Entry: apache2_worker.c:axis2_apache2_worker_process_request()
Processing: axis2_apache2_request_processor_json_impl.c:axis2_apache2_json_processor_parse_and_process_json()
JSON Generation: sprintf() creates JSON string with null terminator
Stream Creation: JSON written to axutil_stream_t via axutil_stream_write()
Buffer Retrieval: axutil_stream_get_buffer() and axutil_stream_get_len() in apache2_worker.c
Apache Output: ap_rwrite(body_string, body_string_len, request) at apache2_worker.c:1522
HTTP/2 Processing: Apache mod_http2 converts to HTTP/2 DATA frames
Issue Point: Null byte added during step 7 (outside Axis2/C control)

Recommendations

Short Term

Use curl workarounds (--output - or save to file)
Document the issue for API consumers

Long Term

Investigate Apache HTTP/2 module configuration options
Consider Apache HTTP Server upgrade
Monitor for Apache HTTP/2 module fixes
Evaluate alternative HTTP/2 implementations if critical

Code Improvements

Add explicit null byte detection logging for future debugging
Consider implementing response validation in development builds
Document the HTTP/2 vs HTTP/1.1 behavior differences

Conclusion

The curl “Binary output can mess up your terminal” issue is caused by Apache HTTP Server's mod_http2 module adding a null byte during HTTP/2 response processing. This occurs after Axis2/C completes its JSON response generation and is outside the control of Axis2/C code.

The JSON service functionality is correct - this is purely a presentation issue affecting curl's binary content detection when displaying HTTP/2 responses to the terminal.