HTTP GET Arrow Data in multipart/mixed: Python Server Example

This directory contains an example of a Python HTTP server that sends a multipart/mixed response to clients. The server:

1. Creates a list of record batches and populates it with synthesized data.
2. Listens for HTTP GET requests from clients.
3. Upon receiving a request, builds and sends an HTTP 200 multipart/mixed response containing:
   • A JSON part with metadata about the Arrow stream.
   • An Arrow stream part with the Arrow IPC stream of record batches.
   • A plain text part with a message containing timing information. This part is optional (included if ?include_footnotes is present in the URL).

To run this example:

pip install pyarrow
python server.py

[!NOTE]
This example uses Python‘s built-in http.server module. This allows us to implement chunked transfer encoding manually. Other servers may implement chunked transfer encoding automatically at the cost of an undesirable new layer of buffering. Arrow IPC streams already offer a natural way of chunking large amounts of tabular data. It’s not a general requirement, but in this example each chunk corresponds to one Arrow record batch.