| ================================================================================================ |
| Dataset Benchmark |
| ================================================================================================ |
| |
| OpenJDK 64-Bit Server VM 17.0.16+8-LTS on Linux 6.11.0-1018-azure |
| AMD EPYC 7763 64-Core Processor |
| back-to-back map long: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ------------------------------------------------------------------------------------------------------------------------ |
| RDD 5898 5930 46 17.0 59.0 1.0X |
| DataFrame 1234 1271 53 81.1 12.3 4.8X |
| Dataset 1338 1351 19 74.8 13.4 4.4X |
| |
| OpenJDK 64-Bit Server VM 17.0.16+8-LTS on Linux 6.11.0-1018-azure |
| AMD EPYC 7763 64-Core Processor |
| back-to-back map: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ------------------------------------------------------------------------------------------------------------------------ |
| RDD 7320 7452 188 13.7 73.2 1.0X |
| DataFrame 2788 2803 21 35.9 27.9 2.6X |
| Dataset 7187 7220 46 13.9 71.9 1.0X |
| |
| OpenJDK 64-Bit Server VM 17.0.16+8-LTS on Linux 6.11.0-1018-azure |
| AMD EPYC 7763 64-Core Processor |
| back-to-back filter Long: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ------------------------------------------------------------------------------------------------------------------------ |
| RDD 4085 4191 150 24.5 40.8 1.0X |
| DataFrame 719 732 18 139.0 7.2 5.7X |
| Dataset 1592 1597 6 62.8 15.9 2.6X |
| |
| OpenJDK 64-Bit Server VM 17.0.16+8-LTS on Linux 6.11.0-1018-azure |
| AMD EPYC 7763 64-Core Processor |
| back-to-back filter: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ------------------------------------------------------------------------------------------------------------------------ |
| RDD 2012 2020 10 49.7 20.1 1.0X |
| DataFrame 119 133 12 837.5 1.2 16.9X |
| Dataset 2449 2452 4 40.8 24.5 0.8X |
| |
| OpenJDK 64-Bit Server VM 17.0.16+8-LTS on Linux 6.11.0-1018-azure |
| AMD EPYC 7763 64-Core Processor |
| aggregate: Best Time(ms) Avg Time(ms) Stdev(ms) Rate(M/s) Per Row(ns) Relative |
| ------------------------------------------------------------------------------------------------------------------------ |
| RDD sum 1404 1418 20 71.2 14.0 1.0X |
| DataFrame sum 70 84 12 1437.3 0.7 20.2X |
| Dataset sum using Aggregator 2046 2057 15 48.9 20.5 0.7X |
| Dataset complex Aggregator 5197 5229 45 19.2 52.0 0.3X |
| |
| |
