Profiling Cookbook

The section contains examples how to perform CPU profiling for Apache DataFusion on different operating systems.

MacOS

Building a flamegraph

Test:

CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph --root --unit-test datafusion  -- dataframe::tests::test_array_agg

Benchmark:

CARGO_PROFILE_RELEASE_DEBUG=true cargo flamegraph --root --bench sql_planner -- --bench

Open flamegraph.svg file with the browser

  • dtrace with DataFusion CLI
git clone https://github.com/brendangregg/FlameGraph.git /tmp/fg
cd datafusion-cli
CARGO_PROFILE_RELEASE_DEBUG=true cargo build --release
echo "select * from table;" >> test.sql
sudo dtrace -c './target/debug/datafusion-cli -f test.sql' -o out.stacks -n 'profile-997 /execname == "datafusion-cli"/ { @[ustack(100)] = count(); }'
/tmp/fg/FlameGraph/stackcollapse.pl out.stacks | /tmp/fg/FlameGraph/flamegraph.pl > flamegraph.svg

Open flamegraph.svg file with the browser

CPU profiling with XCode Instruments

Video: how to CPU profile DataFusion with XCode Instruments

Linux

Windows