Arrow Dissociated IPC Protocol Example

This directory contains a reference example implementation of the Arrow Dissociated IPC Protocol.

This protocol splits the Arrow Flatbuffers IPC metadata and the body buffers into separate streams to allow for utilizing shared memory, non-cpu device memory, or remote memory (RDMA) with Arrow formatted datasets.

This example utilizes libcudf and UCX to transfer Arrow data located on an NVIDIA GPU.

Building

You must have libcudf, libarrow, libarrow_flight, libarrow_cuda, and ucx accessible on your CMAKE_MODULE_PATH/CMAKE_PREFIX_PATH so that cmake can find them.

After that you can simply do the following:

$ cmake -S . -B build -DCMAKE_BUILD_TYPE=Release
$ cmake --build build

to build the arrow-cudf-flight mainprog.

Running

You can start the server by just running arrow-cudf-flight which will default to using 31337 as the Flight port and 127.0.0.1 for the host. Both of these can be changed via the -port and -address gflags respectively.

You can run the client by adding the -client option when running the command.