| ================================================================================================ |
| Arrow Cache vs Default Cache |
| ================================================================================================ |
| |
| ================================================================================================ |
| Cache primitive types |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure |
| AMD EPYC 9V74 80-Core Processor |
| Cache 5M rows with primitives: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| --------------------------------------------------------------------------------------------------------------------------- |
| Default cache - write + read 1765 1794 41 2.8 353.0 1.0X |
| Default cache - write + read (uncompressed) 1154 1168 20 4.3 230.9 1.5X |
| Arrow cache - write + read 1279 1290 15 3.9 255.9 1.4X |
| Arrow cache - write + read (zstd level -1) 1736 1736 1 2.9 347.1 1.0X |
| Arrow cache - write + read (zstd level 1) 1795 1815 28 2.8 359.0 1.0X |
| Arrow cache - write + read (zstd level 3) 1858 1862 6 2.7 371.5 1.0X |
| |
| |
| ================================================================================================ |
| Cache then filter |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure |
| AMD EPYC 9V74 80-Core Processor |
| Cache 5M rows, then filter: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ------------------------------------------------------------------------------------------------------------------------ |
| Default cache - filter 1454 1478 33 3.4 290.9 1.0X |
| Default cache - filter (uncompressed) 1312 1315 4 3.8 262.5 1.1X |
| Arrow cache - filter 1391 1394 4 3.6 278.2 1.0X |
| Arrow cache - filter (zstd level -1) 1737 1751 21 2.9 347.4 0.8X |
| Arrow cache - filter (zstd level 1) 1808 1814 7 2.8 361.7 0.8X |
| Arrow cache - filter (zstd level 3) 1874 1874 0 2.7 374.8 0.8X |
| |
| |
| ================================================================================================ |
| Cache columnar input (Parquet) |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure |
| AMD EPYC 9V74 80-Core Processor |
| Cache 2M rows from Parquet: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ----------------------------------------------------------------------------------------------------------------------------- |
| Default cache - columnar input 1492 1498 8 1.3 746.1 1.0X |
| Default cache - columnar input (uncompressed) 1263 1264 2 1.6 631.4 1.2X |
| Arrow cache - columnar input 1318 1318 0 1.5 658.9 1.1X |
| Arrow cache - columnar input (zstd level -1) 1541 1542 1 1.3 770.7 1.0X |
| Arrow cache - columnar input (zstd level 1) 1556 1565 12 1.3 778.1 1.0X |
| Arrow cache - columnar input (zstd level 3) 1621 1625 5 1.2 810.5 0.9X |
| |
| |
| ================================================================================================ |
| Re-cache Arrow cached data (zero-copy test) |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure |
| AMD EPYC 9V74 80-Core Processor |
| Re-cache 2M rows (zero-copy): Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| -------------------------------------------------------------------------------------------------------------------------------- |
| Default cache - cache a cached DF 402 413 11 5.0 201.2 1.0X |
| Default cache - cache a cached DF (uncompressed) 208 225 28 9.6 103.9 1.9X |
| Arrow cache - cache a cached DF (zero-copy) 144 155 17 13.9 72.2 2.8X |
| Arrow cache - cache a cached DF (zstd level -1) 343 346 4 5.8 171.5 1.2X |
| Arrow cache - cache a cached DF (zstd level 1) 356 367 15 5.6 178.0 1.1X |
| Arrow cache - cache a cached DF (zstd level 3) 364 366 1 5.5 181.9 1.1X |
| |
| |
| ================================================================================================ |
| Cache with column pruning (select 1 of 20 columns) |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 17.0.19+10-LTS on Linux 6.17.0-1018-azure |
| AMD EPYC 9V74 80-Core Processor |
| Cache 5M rows, select 1 column: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ----------------------------------------------------------------------------------------------------------------------------- |
| Default cache - select 1 of 20 columns 10187 10276 126 0.5 2037.4 1.0X |
| Default cache - select 1 of 20 (uncompressed) 4115 4129 19 1.2 823.1 2.5X |
| Arrow cache - select 1 of 20 4927 4931 6 1.0 985.4 2.1X |
| Arrow cache - select 1 of 20 (zstd level -1) 9043 9098 77 0.6 1808.6 1.1X |
| Arrow cache - select 1 of 20 (zstd level 1) 9139 9216 109 0.5 1827.9 1.1X |
| Arrow cache - select 1 of 20 (zstd level 3) 9420 9453 47 0.5 1884.0 1.1X |
| |
| |
| |
