To track progress on performance, we regularly run benchmarks derived from TPC-H and TPC-DS.
The benchmarking scripts are contained here.
Data generation scripts are available in the DataFusion Benchmarks GitHub repository.
Available benchmarking guides:
We also have many micro benchmarks that can be run from an IDE located here. These can also be run as a suite on a dedicated machine, see Micro Benchmarking on AWS EC2. Published results are in benchmarks/results/micro.
:hidden: benchmark-results/tpc-ds benchmarking_macos benchmarking_aws_ec2 benchmarking_micro_ec2 benchmarking_spark_sql_perf