| <!-- |
| Licensed to the Apache Software Foundation (ASF) under one |
| or more contributor license agreements. See the NOTICE file |
| distributed with this work for additional information |
| regarding copyright ownership. The ASF licenses this file |
| to you under the Apache License, Version 2.0 (the |
| "License"); you may not use this file except in compliance |
| with the License. You may obtain a copy of the License at |
| |
| http://www.apache.org/licenses/LICENSE-2.0 |
| |
| Unless required by applicable law or agreed to in writing, software |
| distributed under the License is distributed on an "AS IS" BASIS, |
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| See the License for the specific language governing permissions and |
| limitations under the License. |
| --> |
| # HyperLogLogPlus |
| |
| ## Preliminaries - Distinct Value (cardinality) Estimation |
| |
| Calculating the number of distinct values (DV) in a specific finite-sized list of values using brute force (e.g. a HashSet) techniques is fairly straight forward provided |
| that linear scaling to your number of values fits within your memory constraints. For many cases, the simple calculation serves just fine. However, |
| in many other instances the simple linear approach simply will not scale as well as we would like, and a more |
| robust algorithm is desirable. Enter probabilistic distinct value estimation algorithms: k-minimum-values (KMV), LogLog, HyperLogLog (HLL), and most recently HyperLogLog++ (HLLP). |
| As data volume grows it becomes impractical to store all values in memory in order to reasonably calculate set cardinality. This is where probabilistic estimation algorithms |
| become extremely useful. Often, we are happy to trade a handful of memory and better performance in exchange for an inexact, |
| yet reasonably close approximation of the actual cardinality. |
| |
| |
| ## The Case For An Improvement |
| |
| HyperLogLogPlus is an algorithm introduced by Google that builds on the DV estimation capabilities of HLL and improves on accuracy as well as scalability. |
| HLLP not only works well with *extremely* large data sets (think *trillion*), but with smaller cardinality |
| sets as well. HyperLogLog is simply not as accurate on the lower end of the cardinality spectrum. In addition, HLLP provides the ability |
| to easily merge estimation sets. Although a data stream may not initially require high cardinality estimates as afforded by HLLP, it |
| will still work accurately and efficiently on the smaller volumes without the need to rewrite code if data volume begins to surpass memory |
| constraints. The paper describing the algorithm can be found here - https://research.google.com/pubs/pub40671.html |
| |
| ## Performance Metrics |
| |
| ### Overview |
| |
| Below is a table detailing the performance characteristics when running HLLP over multiple data set sizes. There are three primary variables |
| manipulated during the test: cardinality, sparse set precision, and normal set precision. Each of these values will have an impact on error rates, execution time, and total memory consumption. |
| |
| ### Choosing p, sp |
| There is a tradeoff when choosing values for p and sp. A higher value for sp means a lower error rate for instances when the sparse set is being used. However, a higher value for sp also requires |
| more memory, which means the algorithm will cut over to the dense representation more quickly. The Google paper (listed above) recommends settings of p=25, sp=14 for optimal memory use and |
| algorithm accuracy, and these are the defaults provided by the Apache Metron implementation in Stellar. You'll notice in the tests below, memory consumption tops out around 12KB |
| for this setting, regardless of cardinality size. |
| |
| ### Key |
| --- |
| ``` |
| card: cardinality<br> |
| sp: sparse precision value<br> |
| p: normal (dense) precision value<br> |
| err: error as a percent of the expected cardinality<br> |
| time: total time to add values to the hllp estimator and calculate a cardinality estimate<br> |
| size: size of the hllp set in bytes once all values have been added for the specified cardinality<br> |
| l=low, m=mid(based on percentile chosen), h=high, std=standard deviation<br> |
| ``` |
| |
| ### Table Cardinality 200-1000, step by 200 |
| --------------------------------------- |
| |
| #### Options Used |
| ``` |
| num trials: 5000 |
| card min: 200 |
| card max: 1000 |
| card step: 200 |
| card start: 200 |
| sp min: 4 |
| sp max: 32 |
| sp step: 4 |
| error percentile: 50.0 |
| time percentile: 50.0 |
| size percentile: 50.0 |
| format err as %: true |
| ``` |
| |
| |
| |card |sp |p |err l/m/h/std (% of actual) |time l/m/h/std (ms) |size l/m/h/std (b) | |
| |----------|----------|----------|----------------------------------------|----------------------------------------|------------------------------| |
| |200 |4 |4 |0.000 / 17.000 / 180.000 / 17.470 |0.070 / 0.118 / 10.092 / 0.204 |218 / 220 / 220 / 1 | |
| |200 |8 |4 |0.000 / 17.000 / 209.500 / 18.060 |0.071 / 0.085 / 8.786 / 0.128 |219 / 221 / 221 / 1 | |
| |200 |8 |8 |0.000 / 3.500 / 21.500 / 3.185 |0.082 / 0.085 / 8.375 / 0.119 |531 / 587 / 637 / 14 | |
| |200 |12 |4 |0.000 / 17.000 / 174.000 / 17.376 |0.070 / 0.085 / 8.708 / 0.128 |219 / 221 / 221 / 1 | |
| |200 |12 |8 |0.000 / 3.000 / 25.500 / 2.847 |0.086 / 0.119 / 8.984 / 0.151 |353 / 372 / 627 / 74 | |
| |200 |12 |12 |0.000 / 0.500 / 5.500 / 0.659 |0.082 / 0.096 / 17.208 / 0.243 |775 / 816 / 834 / 9 | |
| |200 |16 |4 |0.000 / 17.500 / 172.000 / 16.622 |0.068 / 0.075 / 9.553 / 0.137 |220 / 222 / 222 / 1 | |
| |200 |16 |8 |0.000 / 3.500 / 18.000 / 2.899 |0.095 / 0.118 / 0.663 / 0.031 |354 / 373 / 390 / 5 | |
| |200 |16 |12 |0.000 / 0.000 / 1.500 / 0.276 |0.085 / 0.108 / 13.977 / 0.199 |802 / 833 / 852 / 6 | |
| |200 |16 |16 |0.000 / 0.000 / 2.500 / 0.279 |0.079 / 0.085 / 0.271 / 0.017 |990 / 1027 / 1044 / 6 | |
| |200 |20 |4 |0.000 / 17.000 / 209.500 / 17.424 |0.068 / 0.072 / 0.250 / 0.014 |221 / 223 / 223 / 1 | |
| |200 |20 |8 |0.000 / 3.500 / 18.000 / 2.944 |0.096 / 0.106 / 13.581 / 0.192 |357 / 374 / 388 / 5 | |
| |200 |20 |12 |0.000 / 0.000 / 1.000 / 0.066 |0.084 / 0.088 / 1.096 / 0.019 |915 / 945 / 972 / 8 | |
| |200 |20 |16 |0.000 / 0.000 / 0.500 / 0.070 |0.083 / 0.087 / 10.272 / 0.145 |912 / 953 / 982 / 8 | |
| |200 |20 |20 |0.000 / 0.000 / 1.000 / 0.069 |0.079 / 0.081 / 0.210 / 0.017 |1028 / 1056 / 1058 / 2 | |
| |200 |24 |4 |0.000 / 17.500 / 136.000 / 16.589 |0.068 / 0.071 / 0.952 / 0.025 |221 / 223 / 223 / 1 | |
| |200 |24 |8 |0.000 / 3.500 / 18.000 / 2.955 |0.095 / 0.106 / 10.387 / 0.148 |353 / 373 / 390 / 5 | |
| |200 |24 |12 |0.000 / 0.000 / 0.500 / 0.016 |0.082 / 0.083 / 0.235 / 0.014 |1035 / 1048 / 1056 / 3 | |
| |200 |24 |16 |0.000 / 0.000 / 0.500 / 0.012 |0.082 / 0.084 / 10.904 / 0.154 |1038 / 1051 / 1059 / 3 | |
| |200 |24 |20 |0.000 / 0.000 / 0.500 / 0.010 |0.082 / 0.085 / 0.410 / 0.019 |1049 / 1064 / 1082 / 5 | |
| |200 |24 |24 |0.000 / 0.000 / 0.500 / 0.023 |0.079 / 0.084 / 11.926 / 0.168 |1247 / 1264 / 1275 / 4 | |
| |200 |28 |4 |0.000 / 17.500 / 229.000 / 17.294 |0.068 / 0.072 / 0.369 / 0.015 |222 / 224 / 224 / 1 | |
| |200 |28 |8 |83.000 / 83.000 / 84.000 / 0.118 |0.093 / 0.098 / 10.918 / 0.157 |272 / 276 / 278 / 1 | |
| |200 |28 |12 |0.000 / 0.000 / 0.500 / 0.016 |0.082 / 0.083 / 10.157 / 0.143 |1032 / 1067 / 1075 / 3 | |
| |200 |28 |16 |0.000 / 0.000 / 0.500 / 0.007 |0.082 / 0.083 / 0.617 / 0.020 |1058 / 1069 / 1077 / 3 | |
| |200 |28 |20 |0.000 / 0.000 / 0.500 / 0.010 |0.082 / 0.085 / 11.619 / 0.165 |1060 / 1071 / 1080 / 3 | |
| |200 |28 |24 |0.000 / 0.000 / 0.500 / 0.007 |0.083 / 0.085 / 0.516 / 0.022 |1065 / 1083 / 1100 / 4 | |
| |200 |28 |28 |0.000 / 0.000 / 0.500 / 0.019 |0.080 / 0.087 / 11.278 / 0.160 |1250 / 1267 / 1278 / 4 | |
| |200 |32 |4 |0.000 / 18.000 / 175.500 / 17.157 |0.068 / 0.078 / 0.223 / 0.019 |222 / 224 / 224 / 1 | |
| |200 |32 |8 |99.000 / 99.000 / 99.000 / 0.000 |0.093 / 0.107 / 11.695 / 0.167 |258 / 258 / 258 / 0 | |
| |200 |32 |12 |0.000 / 0.000 / 0.500 / 0.014 |0.081 / 0.084 / 8.906 / 0.126 |1060 / 1071 / 1075 / 2 | |
| |200 |32 |16 |0.000 / 0.000 / 0.500 / 0.010 |0.082 / 0.085 / 1.458 / 0.027 |1061 / 1073 / 1077 / 2 | |
| |200 |32 |20 |0.000 / 0.000 / 0.500 / 0.012 |0.082 / 0.084 / 10.499 / 0.148 |1065 / 1074 / 1079 / 2 | |
| |200 |32 |24 |0.000 / 0.000 / 0.500 / 0.012 |0.082 / 0.085 / 0.420 / 0.020 |1066 / 1075 / 1082 / 2 | |
| |200 |32 |28 |0.000 / 0.000 / 0.500 / 0.016 |0.082 / 0.086 / 8.753 / 0.124 |1076 / 1088 / 1102 / 4 | |
| |200 |32 |32 |0.000 / 0.000 / 0.500 / 0.010 |0.080 / 0.081 / 13.248 / 0.188 |1249 / 1268 / 1279 / 4 | |
| |400 |4 |4 |0.000 / 17.500 / 175.250 / 17.530 |0.136 / 0.142 / 0.591 / 0.035 |218 / 220 / 220 / 0 | |
| |400 |8 |4 |0.000 / 17.000 / 148.500 / 17.174 |0.136 / 0.156 / 12.882 / 0.241 |219 / 221 / 221 / 0 | |
| |400 |8 |8 |0.000 / 3.500 / 21.250 / 3.215 |0.159 / 0.186 / 10.396 / 0.151 |375 / 387 / 802 / 62 | |
| |400 |12 |4 |0.000 / 17.750 / 166.000 / 17.363 |0.135 / 0.142 / 11.607 / 0.165 |219 / 221 / 221 / 0 | |
| |400 |12 |8 |0.000 / 3.250 / 23.750 / 3.133 |0.166 / 0.179 / 8.713 / 0.130 |376 / 387 / 401 / 3 | |
| |400 |12 |12 |0.000 / 0.750 / 4.250 / 0.706 |0.159 / 0.163 / 10.537 / 0.189 |1334 / 1387 / 1434 / 14 | |
| |400 |16 |4 |0.000 / 17.125 / 163.250 / 17.208 |0.135 / 0.138 / 12.403 / 0.178 |220 / 222 / 222 / 0 | |
| |400 |16 |8 |0.000 / 3.250 / 23.500 / 3.126 |0.165 / 0.179 / 9.985 / 0.147 |378 / 388 / 401 / 3 | |
| |400 |16 |12 |0.000 / 0.250 / 1.500 / 0.195 |0.166 / 0.173 / 10.434 / 0.150 |1414 / 1446 / 1475 / 8 | |
| |400 |16 |16 |0.000 / 0.250 / 1.750 / 0.194 |0.160 / 0.177 / 1.300 / 0.048 |1761 / 1798 / 1825 / 8 | |
| |400 |20 |4 |0.000 / 17.750 / 270.750 / 17.592 |0.135 / 0.149 / 8.814 / 0.128 |221 / 223 / 223 / 0 | |
| |400 |20 |8 |0.000 / 3.500 / 23.000 / 3.103 |0.165 / 0.183 / 9.271 / 0.139 |379 / 389 / 398 / 3 | |
| |400 |20 |12 |0.000 / 0.000 / 0.500 / 0.066 |0.164 / 0.182 / 11.093 / 0.162 |1617 / 1652 / 1683 / 10 | |
| |400 |20 |16 |0.000 / 0.000 / 0.500 / 0.070 |0.166 / 0.170 / 8.676 / 0.128 |1624 / 1665 / 1706 / 10 | |
| |400 |20 |20 |0.000 / 0.000 / 0.750 / 0.069 |0.159 / 0.162 / 12.846 / 0.183 |1840 / 1853 / 1856 / 2 | |
| |400 |24 |4 |0.000 / 16.750 / 174.750 / 16.786 |0.135 / 0.145 / 8.586 / 0.128 |221 / 223 / 223 / 0 | |
| |400 |24 |8 |0.000 / 3.500 / 24.500 / 3.090 |0.165 / 0.179 / 11.344 / 0.164 |377 / 389 / 399 / 3 | |
| |400 |24 |12 |0.000 / 0.000 / 0.250 / 0.016 |0.164 / 0.183 / 10.708 / 0.159 |1828 / 1841 / 1852 / 4 | |
| |400 |24 |16 |0.000 / 0.000 / 0.250 / 0.017 |0.164 / 0.172 / 11.180 / 0.161 |1830 / 1844 / 1855 / 4 | |
| |400 |24 |20 |0.000 / 0.000 / 0.250 / 0.015 |0.166 / 0.170 / 9.589 / 0.140 |1843 / 1868 / 1890 / 6 | |
| |400 |24 |24 |0.000 / 0.000 / 0.250 / 0.022 |0.160 / 0.162 / 9.137 / 0.134 |2213 / 2232 / 2247 / 5 | |
| |400 |28 |4 |0.000 / 18.250 / 164.000 / 16.587 |0.135 / 0.144 / 14.520 / 0.209 |222 / 224 / 224 / 0 | |
| |400 |28 |8 |33.000 / 47.750 / 56.750 / 3.588 |0.164 / 0.168 / 1.305 / 0.052 |355 / 375 / 394 / 5 | |
| |400 |28 |12 |0.000 / 0.000 / 0.250 / 0.006 |0.165 / 0.178 / 11.264 / 0.167 |1837 / 1848 / 1854 / 3 | |
| |400 |28 |16 |0.000 / 0.000 / 0.250 / 0.014 |0.164 / 0.167 / 8.689 / 0.130 |1838 / 1850 / 1857 / 3 | |
| |400 |28 |20 |0.000 / 0.000 / 0.250 / 0.011 |0.164 / 0.167 / 10.679 / 0.159 |1839 / 1852 / 1863 / 3 | |
| |400 |28 |24 |0.000 / 0.000 / 0.250 / 0.011 |0.166 / 0.175 / 12.887 / 0.190 |1857 / 1875 / 1896 / 5 | |
| |400 |28 |28 |0.000 / 0.000 / 0.250 / 0.015 |0.160 / 0.196 / 1.478 / 0.071 |2217 / 2235 / 2252 / 5 | |
| |400 |32 |4 |0.000 / 17.500 / 183.750 / 19.308 |0.135 / 0.142 / 9.949 / 0.148 |222 / 224 / 224 / 0 | |
| |400 |32 |8 |46.500 / 56.250 / 63.250 / 2.194 |0.164 / 0.181 / 1.209 / 0.063 |349 / 370 / 390 / 5 | |
| |400 |32 |12 |0.000 / 0.000 / 0.250 / 0.012 |0.164 / 0.171 / 11.529 / 0.170 |1837 / 1848 / 1854 / 3 | |
| |400 |32 |16 |0.000 / 0.000 / 0.250 / 0.016 |0.164 / 0.167 / 10.276 / 0.154 |1836 / 1850 / 1856 / 3 | |
| |400 |32 |20 |0.000 / 0.000 / 0.250 / 0.015 |0.164 / 0.184 / 11.611 / 0.227 |1839 / 1851 / 1858 / 3 | |
| |400 |32 |24 |0.000 / 0.000 / 0.250 / 0.012 |0.165 / 0.171 / 9.313 / 0.143 |1842 / 1853 / 1862 / 3 | |
| |400 |32 |28 |0.000 / 0.000 / 0.250 / 0.013 |0.166 / 0.180 / 10.708 / 0.162 |1860 / 1877 / 1900 / 5 | |
| |400 |32 |32 |0.000 / 0.000 / 0.250 / 0.014 |0.160 / 0.166 / 1.205 / 0.057 |2217 / 2235 / 2249 / 5 | |
| |600 |4 |4 |0.000 / 17.333 / 219.667 / 17.222 |0.202 / 0.240 / 13.390 / 0.237 |218 / 220 / 220 / 0 | |
| |600 |8 |4 |0.000 / 17.417 / 231.333 / 17.340 |0.202 / 0.224 / 9.662 / 0.150 |220 / 221 / 221 / 0 | |
| |600 |8 |8 |0.000 / 3.500 / 24.333 / 3.240 |0.234 / 0.271 / 11.443 / 0.216 |382 / 392 / 403 / 3 | |
| |600 |12 |4 |0.000 / 17.167 / 188.167 / 17.324 |0.202 / 0.210 / 1.320 / 0.055 |219 / 221 / 221 / 0 | |
| |600 |12 |8 |0.000 / 3.500 / 23.833 / 3.243 |0.235 / 0.246 / 10.821 / 0.164 |382 / 392 / 403 / 3 | |
| |600 |12 |12 |0.000 / 0.667 / 4.500 / 0.694 |0.240 / 0.259 / 12.004 / 0.183 |2015 / 2101 / 2178 / 21 | |
| |600 |16 |4 |0.000 / 17.500 / 170.000 / 17.882 |0.202 / 0.218 / 9.208 / 0.140 |221 / 222 / 222 / 0 | |
| |600 |16 |8 |0.000 / 3.667 / 22.667 / 3.276 |0.234 / 0.251 / 12.301 / 0.237 |383 / 393 / 408 / 3 | |
| |600 |16 |12 |0.000 / 0.167 / 1.000 / 0.170 |0.250 / 0.284 / 12.000 / 0.180 |2199 / 2243 / 2282 / 12 | |
| |600 |16 |16 |0.000 / 0.167 / 1.167 / 0.171 |0.239 / 0.291 / 21.163 / 0.333 |2811 / 2859 / 2901 / 13 | |
| |600 |20 |4 |0.000 / 17.667 / 176.667 / 18.162 |0.202 / 0.223 / 16.405 / 0.275 |221 / 223 / 223 / 0 | |
| |600 |20 |8 |0.000 / 3.500 / 20.833 / 3.180 |0.234 / 0.243 / 5.621 / 0.104 |385 / 394 / 405 / 3 | |
| |600 |20 |12 |0.000 / 0.000 / 0.500 / 0.066 |0.246 / 0.281 / 19.279 / 0.390 |2541 / 2604 / 2663 / 15 | |
| |600 |20 |16 |0.000 / 0.000 / 0.667 / 0.071 |0.248 / 0.258 / 24.627 / 0.387 |2569 / 2627 / 2682 / 15 | |
| |600 |20 |20 |0.000 / 0.000 / 0.500 / 0.069 |0.239 / 0.246 / 13.606 / 0.248 |2938 / 2954 / 2972 / 3 | |
| |600 |24 |4 |0.000 / 17.333 / 210.333 / 18.157 |0.202 / 0.218 / 11.991 / 0.180 |222 / 223 / 223 / 0 | |
| |600 |24 |8 |0.000 / 3.667 / 21.167 / 3.184 |0.234 / 0.258 / 11.130 / 0.238 |385 / 394 / 406 / 3 | |
| |600 |24 |12 |0.000 / 0.000 / 0.167 / 0.017 |0.246 / 0.272 / 12.032 / 0.191 |2912 / 2935 / 2954 / 5 | |
| |600 |24 |16 |0.000 / 0.000 / 0.167 / 0.019 |0.246 / 0.268 / 19.358 / 0.282 |2916 / 2940 / 2959 / 6 | |
| |600 |24 |20 |0.000 / 0.000 / 0.167 / 0.015 |0.248 / 0.259 / 20.293 / 0.343 |2948 / 2980 / 3009 / 9 | |
| |600 |24 |24 |0.000 / 0.000 / 0.167 / 0.017 |0.239 / 0.252 / 11.721 / 0.230 |3586 / 3618 / 3643 / 7 | |
| |600 |28 |4 |0.000 / 17.667 / 182.667 / 17.309 |0.202 / 0.210 / 11.957 / 0.177 |223 / 224 / 224 / 0 | |
| |600 |28 |8 |10.333 / 28.000 / 42.833 / 3.687 |0.233 / 0.238 / 10.166 / 0.210 |377 / 392 / 403 / 3 | |
| |600 |28 |12 |0.000 / 0.000 / 0.333 / 0.013 |0.246 / 0.252 / 9.907 / 0.158 |3005 / 3024 / 3041 / 5 | |
| |600 |28 |16 |0.000 / 0.000 / 0.167 / 0.012 |0.246 / 0.256 / 18.673 / 0.273 |3007 / 3026 / 3045 / 5 | |
| |600 |28 |20 |0.000 / 0.000 / 0.167 / 0.011 |0.246 / 0.283 / 17.762 / 0.355 |3005 / 3029 / 3047 / 6 | |
| |600 |28 |24 |0.000 / 0.000 / 0.167 / 0.011 |0.248 / 0.254 / 8.821 / 0.138 |3035 / 3065 / 3098 / 8 | |
| |600 |28 |28 |0.000 / 0.000 / 0.167 / 0.011 |0.239 / 0.242 / 12.171 / 0.177 |3589 / 3621 / 3646 / 7 | |
| |600 |32 |4 |0.000 / 17.333 / 200.000 / 18.294 |0.202 / 0.205 / 9.036 / 0.138 |223 / 224 / 224 / 0 | |
| |600 |32 |8 |25.167 / 37.167 / 46.833 / 3.185 |0.232 / 0.236 / 9.061 / 0.144 |377 / 389 / 403 / 3 | |
| |600 |32 |12 |0.000 / 0.000 / 0.167 / 0.013 |0.246 / 0.256 / 11.119 / 0.169 |3028 / 3043 / 3055 / 4 | |
| |600 |32 |16 |0.000 / 0.000 / 0.167 / 0.010 |0.246 / 0.257 / 11.557 / 0.213 |3030 / 3045 / 3057 / 4 | |
| |600 |32 |20 |0.000 / 0.000 / 0.167 / 0.011 |0.246 / 0.266 / 20.353 / 0.295 |3030 / 3047 / 3071 / 4 | |
| |600 |32 |24 |0.000 / 0.000 / 0.167 / 0.012 |0.246 / 0.270 / 11.461 / 0.183 |3035 / 3050 / 3073 / 4 | |
| |600 |32 |28 |0.000 / 0.000 / 0.167 / 0.013 |0.248 / 0.260 / 11.755 / 0.242 |3060 / 3085 / 3108 / 7 | |
| |600 |32 |32 |0.000 / 0.000 / 0.167 / 0.012 |0.239 / 0.248 / 10.380 / 0.157 |3593 / 3621 / 3645 / 7 | |
| |800 |4 |4 |0.000 / 17.250 / 188.250 / 17.248 |0.269 / 0.280 / 10.818 / 0.212 |219 / 220 / 220 / 0 | |
| |800 |8 |4 |0.000 / 17.875 / 184.125 / 17.721 |0.270 / 0.298 / 16.706 / 0.252 |220 / 221 / 221 / 0 | |
| |800 |8 |8 |0.000 / 3.625 / 22.000 / 3.420 |0.303 / 0.315 / 12.489 / 0.243 |386 / 395 / 406 / 3 | |
| |800 |12 |4 |0.000 / 17.750 / 196.500 / 16.870 |0.269 / 0.289 / 13.533 / 0.235 |220 / 221 / 221 / 0 | |
| |800 |12 |8 |0.000 / 3.750 / 25.875 / 3.359 |0.303 / 0.330 / 18.306 / 0.365 |386 / 395 / 408 / 3 | |
| |800 |12 |12 |0.000 / 0.750 / 4.375 / 0.677 |0.322 / 0.357 / 20.131 / 0.350 |2449 / 2603 / 2694 / 27 | |
| |800 |16 |4 |0.000 / 17.750 / 195.250 / 17.455 |0.269 / 0.274 / 10.472 / 0.205 |221 / 222 / 222 / 0 | |
| |800 |16 |8 |0.000 / 3.750 / 20.375 / 3.357 |0.302 / 0.308 / 10.637 / 0.165 |386 / 396 / 409 / 3 | |
| |800 |16 |12 |0.000 / 0.125 / 1.375 / 0.170 |0.334 / 0.344 / 10.745 / 0.221 |2712 / 2836 / 2882 / 15 | |
| |800 |16 |16 |0.000 / 0.250 / 1.375 / 0.169 |0.321 / 0.334 / 13.979 / 0.252 |3390 / 3625 / 3668 / 19 | |
| |800 |20 |4 |0.000 / 17.750 / 220.750 / 17.953 |0.269 / 0.281 / 9.422 / 0.148 |222 / 223 / 223 / 0 | |
| |800 |20 |8 |0.000 / 3.750 / 21.875 / 3.305 |0.302 / 0.358 / 24.073 / 0.435 |387 / 397 / 408 / 3 | |
| |800 |20 |12 |0.000 / 0.000 / 0.500 / 0.067 |0.331 / 0.345 / 13.461 / 0.211 |3249 / 3304 / 3360 / 17 | |
| |800 |20 |16 |0.000 / 0.000 / 0.500 / 0.069 |0.333 / 0.349 / 8.271 / 0.146 |3190 / 3331 / 3392 / 17 | |
| |800 |20 |20 |0.000 / 0.000 / 0.375 / 0.069 |0.320 / 0.335 / 11.307 / 0.239 |3539 / 3752 / 3771 / 5 | |
| |800 |24 |4 |0.000 / 17.125 / 184.500 / 18.184 |0.269 / 0.283 / 1.541 / 0.079 |222 / 223 / 223 / 0 | |
| |800 |24 |8 |0.000 / 3.625 / 23.125 / 3.301 |0.302 / 0.317 / 10.845 / 0.169 |388 / 397 / 409 / 3 | |
| |800 |24 |12 |0.000 / 0.000 / 0.250 / 0.017 |0.330 / 0.352 / 10.221 / 0.244 |3703 / 3729 / 3749 / 6 | |
| |800 |24 |16 |0.000 / 0.000 / 0.250 / 0.018 |0.330 / 0.344 / 13.454 / 0.239 |3711 / 3734 / 3753 / 6 | |
| |800 |24 |20 |0.000 / 0.000 / 0.250 / 0.018 |0.333 / 0.348 / 16.987 / 0.374 |3751 / 3786 / 3826 / 10 | |
| |800 |24 |24 |0.000 / 0.000 / 0.250 / 0.018 |0.320 / 0.333 / 17.097 / 0.288 |4567 / 4605 / 4630 / 8 | |
| |800 |28 |4 |0.000 / 17.375 / 326.125 / 18.121 |0.269 / 0.274 / 9.600 / 0.149 |223 / 224 / 224 / 0 | |
| |800 |28 |8 |0.375 / 19.875 / 34.500 / 4.445 |0.302 / 0.313 / 10.025 / 0.181 |387 / 397 / 409 / 3 | |
| |800 |28 |12 |0.000 / 0.000 / 0.125 / 0.011 |0.330 / 0.343 / 11.347 / 0.216 |3797 / 3821 / 3839 / 6 | |
| |800 |28 |16 |0.000 / 0.000 / 0.125 / 0.010 |0.330 / 0.344 / 10.748 / 0.219 |3801 / 3823 / 3843 / 6 | |
| |800 |28 |20 |0.000 / 0.000 / 0.125 / 0.011 |0.330 / 0.355 / 15.268 / 0.314 |3802 / 3827 / 3845 / 6 | |
| |800 |28 |24 |0.000 / 0.000 / 0.125 / 0.011 |0.334 / 0.360 / 11.192 / 0.243 |3841 / 3874 / 3905 / 9 | |
| |800 |28 |28 |0.000 / 0.000 / 0.250 / 0.011 |0.321 / 0.334 / 1.757 / 0.095 |4575 / 4609 / 4635 / 8 | |
| |800 |32 |4 |0.000 / 17.125 / 161.000 / 18.370 |0.269 / 0.279 / 20.700 / 0.344 |223 / 224 / 224 / 0 | |
| |800 |32 |8 |10.500 / 27.750 / 39.500 / 3.883 |0.301 / 0.333 / 18.781 / 0.332 |385 / 395 / 406 / 3 | |
| |800 |32 |12 |0.000 / 0.000 / 0.250 / 0.011 |0.330 / 0.371 / 18.883 / 0.317 |3823 / 3840 / 3853 / 4 | |
| |800 |32 |16 |0.000 / 0.000 / 0.125 / 0.011 |0.330 / 0.353 / 9.319 / 0.195 |3824 / 3842 / 3855 / 4 | |
| |800 |32 |20 |0.000 / 0.000 / 0.125 / 0.012 |0.330 / 0.353 / 15.582 / 0.305 |3826 / 3843 / 3859 / 4 | |
| |800 |32 |24 |0.000 / 0.000 / 0.125 / 0.013 |0.330 / 0.349 / 11.346 / 0.234 |3830 / 3847 / 3862 / 5 | |
| |800 |32 |28 |0.000 / 0.000 / 0.250 / 0.013 |0.333 / 0.368 / 14.482 / 0.309 |3866 / 3894 / 3925 / 8 | |
| |800 |32 |32 |0.000 / 0.000 / 0.125 / 0.012 |0.320 / 0.335 / 17.771 / 0.295 |4576 / 4608 / 4633 / 8 | |
| |1000 |4 |4 |0.000 / 17.800 / 187.500 / 18.576 |0.337 / 0.374 / 10.384 / 0.179 |219 / 220 / 220 / 0 | |
| |1000 |8 |4 |0.000 / 17.600 / 181.100 / 18.035 |0.336 / 0.372 / 13.220 / 0.262 |220 / 221 / 221 / 0 | |
| |1000 |8 |8 |0.000 / 3.900 / 23.100 / 3.425 |0.371 / 0.398 / 9.495 / 0.175 |386 / 398 / 409 / 3 | |
| |1000 |12 |4 |0.000 / 17.600 / 178.900 / 17.559 |0.336 / 0.351 / 11.176 / 0.265 |220 / 221 / 221 / 0 | |
| |1000 |12 |8 |0.000 / 3.800 / 22.700 / 3.438 |0.371 / 0.392 / 10.942 / 0.188 |387 / 398 / 411 / 3 | |
| |1000 |12 |12 |0.000 / 0.800 / 4.200 / 0.684 |0.403 / 0.443 / 11.219 / 0.279 |2971 / 3242 / 3364 / 39 | |
| |1000 |16 |4 |0.000 / 17.600 / 184.300 / 17.148 |0.336 / 0.368 / 21.656 / 0.350 |221 / 222 / 222 / 0 | |
| |1000 |16 |8 |0.000 / 3.800 / 21.900 / 3.350 |0.369 / 0.385 / 15.840 / 0.277 |388 / 399 / 412 / 3 | |
| |1000 |16 |12 |0.000 / 0.200 / 1.200 / 0.174 |0.422 / 0.438 / 16.035 / 0.335 |3428 / 3530 / 3628 / 16 | |
| |1000 |16 |16 |0.000 / 0.200 / 1.200 / 0.173 |0.405 / 0.420 / 13.370 / 0.305 |4314 / 4484 / 4573 / 20 | |
| |1000 |20 |4 |0.000 / 17.600 / 223.000 / 16.968 |0.336 / 0.372 / 17.746 / 0.392 |222 / 223 / 223 / 0 | |
| |1000 |20 |8 |0.000 / 3.800 / 21.400 / 3.384 |0.369 / 0.419 / 16.486 / 0.257 |389 / 400 / 412 / 3 | |
| |1000 |20 |12 |0.000 / 0.000 / 0.400 / 0.068 |0.417 / 0.446 / 12.313 / 0.265 |3954 / 4099 / 4158 / 18 | |
| |1000 |20 |16 |0.000 / 0.000 / 0.500 / 0.069 |0.422 / 0.439 / 11.284 / 0.262 |4016 / 4134 / 4211 / 19 | |
| |1000 |20 |20 |0.000 / 0.000 / 0.600 / 0.069 |0.404 / 0.429 / 16.987 / 0.335 |4517 / 4641 / 4651 / 5 | |
| |1000 |24 |4 |0.000 / 17.600 / 226.700 / 17.761 |0.339 / 0.351 / 12.804 / 0.275 |222 / 223 / 223 / 0 | |
| |1000 |24 |8 |0.000 / 3.800 / 24.100 / 3.403 |0.371 / 0.406 / 15.710 / 0.298 |390 / 400 / 412 / 3 | |
| |1000 |24 |12 |0.000 / 0.000 / 0.200 / 0.018 |0.417 / 0.449 / 12.253 / 0.294 |4494 / 4614 / 4636 / 7 | |
| |1000 |24 |16 |0.000 / 0.000 / 0.100 / 0.018 |0.417 / 0.434 / 11.046 / 0.254 |4594 / 4620 / 4641 / 7 | |
| |1000 |24 |20 |0.000 / 0.000 / 0.200 / 0.019 |0.422 / 0.458 / 13.433 / 0.270 |4645 / 4682 / 4722 / 11 | |
| |1000 |24 |24 |0.000 / 0.000 / 0.200 / 0.016 |0.405 / 0.435 / 11.470 / 0.189 |5635 / 5672 / 5703 / 9 | |
| |1000 |28 |4 |0.000 / 17.200 / 148.500 / 17.046 |0.338 / 0.363 / 20.446 / 0.380 |224 / 224 / 224 / 0 | |
| |1000 |28 |8 |0.000 / 15.100 / 30.700 / 4.794 |0.371 / 0.389 / 18.272 / 0.359 |390 / 400 / 411 / 3 | |
| |1000 |28 |12 |0.000 / 0.000 / 0.100 / 0.013 |0.417 / 0.432 / 13.575 / 0.264 |4677 / 4700 / 4719 / 6 | |
| |1000 |28 |16 |0.000 / 0.000 / 0.200 / 0.013 |0.417 / 0.435 / 18.060 / 0.381 |4545 / 4702 / 4723 / 6 | |
| |1000 |28 |20 |0.000 / 0.000 / 0.200 / 0.013 |0.417 / 0.448 / 17.596 / 0.344 |4679 / 4707 / 4728 / 6 | |
| |1000 |28 |24 |0.000 / 0.000 / 0.200 / 0.012 |0.421 / 0.454 / 13.807 / 0.297 |4733 / 4764 / 4801 / 10 | |
| |1000 |28 |28 |0.000 / 0.000 / 0.200 / 0.013 |0.406 / 0.456 / 20.533 / 0.353 |5646 / 5675 / 5704 / 9 | |
| |1000 |32 |4 |0.000 / 18.150 / 177.600 / 19.537 |0.338 / 0.352 / 12.293 / 0.252 |223 / 224 / 224 / 0 | |
| |1000 |32 |8 |1.500 / 22.100 / 35.100 / 4.399 |0.371 / 0.407 / 17.782 / 0.386 |388 / 398 / 412 / 3 | |
| |1000 |32 |12 |0.000 / 0.000 / 0.200 / 0.011 |0.417 / 0.435 / 11.830 / 0.247 |4698 / 4717 / 4730 / 5 | |
| |1000 |32 |16 |0.000 / 0.000 / 0.200 / 0.013 |0.417 / 0.443 / 18.214 / 0.276 |4700 / 4719 / 4733 / 5 | |
| |1000 |32 |20 |0.000 / 0.000 / 0.200 / 0.012 |0.417 / 0.437 / 11.208 / 0.232 |4703 / 4720 / 4737 / 5 | |
| |1000 |32 |24 |0.000 / 0.000 / 0.200 / 0.013 |0.417 / 0.448 / 18.288 / 0.405 |4706 / 4724 / 4740 / 5 | |
| |1000 |32 |28 |0.000 / 0.000 / 0.200 / 0.011 |0.420 / 0.437 / 13.389 / 0.245 |4613 / 4782 / 4814 / 9 | |
| |1000 |32 |32 |0.000 / 0.000 / 0.100 / 0.012 |0.406 / 0.424 / 12.513 / 0.232 |5641 / 5675 / 5704 / 9 | |
| |
| |
| Table Cardinality 1500-5000, step by 500 |
| ---------------------------------------- |
| |
| #### Options Used |
| ``` |
| num trials: 5000 |
| card min: 1500 |
| card max: 5000 |
| card step: 500 |
| card start: 1500 |
| sp min: 4 |
| sp max: 32 |
| sp step: 4 |
| error percentile: 50.0 |
| time percentile: 50.0 |
| size percentile: 50.0 |
| format err as %: true |
| ``` |
| |
| |card |sp |p |err l/m/h/std (% of actual) |time l/m/h/std (ms) |size l/m/h/std (b) | |
| |----------|----------|----------|----------------------------------------|----------------------------------------|------------------------------| |
| |1500 |4 |4 |0.000 / 17.600 / 254.400 / 17.627 |0.501 / 0.612 / 19.629 / 0.593 |220 / 220 / 220 / 0 | |
| |1500 |8 |4 |0.000 / 17.600 / 158.133 / 16.980 |0.506 / 0.594 / 16.843 / 0.493 |221 / 221 / 221 / 0 | |
| |1500 |8 |8 |0.000 / 4.000 / 23.933 / 3.590 |0.533 / 0.567 / 11.059 / 0.259 |392 / 404 / 416 / 3 | |
| |1500 |12 |4 |0.000 / 18.067 / 208.733 / 17.934 |0.505 / 0.531 / 20.544 / 0.368 |221 / 221 / 221 / 0 | |
| |1500 |12 |8 |0.000 / 4.067 / 22.133 / 3.579 |0.529 / 0.576 / 22.421 / 0.453 |391 / 404 / 416 / 3 | |
| |1500 |12 |12 |0.000 / 0.800 / 4.133 / 0.707 |0.607 / 0.638 / 15.124 / 0.329 |4338 / 4489 / 4656 / 43 | |
| |1500 |16 |4 |0.000 / 17.200 / 167.333 / 17.918 |0.508 / 0.546 / 17.715 / 0.381 |221 / 222 / 222 / 0 | |
| |1500 |16 |8 |0.000 / 3.933 / 23.667 / 3.605 |0.530 / 0.564 / 18.635 / 0.407 |393 / 405 / 417 / 3 | |
| |1500 |16 |12 |0.000 / 0.200 / 1.067 / 0.166 |0.621 / 0.689 / 17.660 / 0.364 |5177 / 5271 / 5341 / 21 | |
| |1500 |16 |16 |0.000 / 0.200 / 0.933 / 0.165 |0.612 / 0.654 / 19.700 / 0.419 |6688 / 6775 / 6855 / 23 | |
| |1500 |20 |4 |0.000 / 17.600 / 231.333 / 17.757 |0.509 / 0.544 / 20.194 / 0.369 |223 / 223 / 223 / 0 | |
| |1500 |20 |8 |0.000 / 3.933 / 25.600 / 3.728 |0.529 / 0.570 / 15.702 / 0.372 |395 / 406 / 417 / 3 | |
| |1500 |20 |12 |0.000 / 0.067 / 0.333 / 0.047 |0.612 / 0.659 / 12.620 / 0.343 |6097 / 6191 / 6281 / 24 | |
| |1500 |20 |16 |0.000 / 0.067 / 0.400 / 0.047 |0.619 / 0.648 / 13.508 / 0.300 |6163 / 6244 / 6344 / 24 | |
| |1500 |20 |20 |0.000 / 0.067 / 0.333 / 0.047 |0.610 / 0.650 / 12.478 / 0.323 |7019 / 7048 / 7093 / 6 | |
| |1500 |24 |4 |0.000 / 17.467 / 156.333 / 18.009 |0.508 / 0.569 / 17.878 / 0.386 |223 / 223 / 223 / 0 | |
| |1500 |24 |8 |0.000 / 4.067 / 27.400 / 3.599 |0.529 / 0.564 / 21.583 / 0.397 |394 / 406 / 416 / 3 | |
| |1500 |24 |12 |0.000 / 0.000 / 0.133 / 0.017 |0.611 / 0.655 / 16.659 / 0.383 |6974 / 7008 / 7039 / 8 | |
| |1500 |24 |16 |0.000 / 0.000 / 0.133 / 0.017 |0.612 / 0.644 / 10.934 / 0.313 |6981 / 7016 / 7048 / 9 | |
| |1500 |24 |20 |0.000 / 0.000 / 0.133 / 0.017 |0.620 / 0.683 / 21.873 / 0.512 |7064 / 7115 / 7167 / 14 | |
| |1500 |24 |24 |0.000 / 0.000 / 0.133 / 0.017 |0.609 / 0.634 / 13.249 / 0.315 |8645 / 8684 / 8722 / 11 | |
| |1500 |28 |4 |0.000 / 17.467 / 157.067 / 17.312 |0.508 / 0.529 / 14.149 / 0.336 |224 / 224 / 224 / 0 | |
| |1500 |28 |8 |0.000 / 8.533 / 25.933 / 4.865 |0.529 / 0.561 / 15.511 / 0.380 |396 / 406 / 419 / 3 | |
| |1500 |28 |12 |0.000 / 0.000 / 0.133 / 0.012 |0.611 / 0.657 / 10.977 / 0.294 |7178 / 7209 / 7239 / 8 | |
| |1500 |28 |16 |0.000 / 0.000 / 0.133 / 0.013 |0.611 / 0.636 / 12.958 / 0.319 |7177 / 7211 / 7240 / 8 | |
| |1500 |28 |20 |0.000 / 0.000 / 0.133 / 0.012 |0.612 / 0.646 / 13.818 / 0.327 |7185 / 7218 / 7248 / 9 | |
| |1500 |28 |24 |0.000 / 0.000 / 0.133 / 0.012 |0.619 / 0.662 / 11.359 / 0.279 |7258 / 7305 / 7354 / 13 | |
| |1500 |28 |28 |0.000 / 0.000 / 0.133 / 0.012 |0.609 / 0.648 / 12.260 / 0.300 |8643 / 8687 / 8732 / 12 | |
| |1500 |32 |4 |0.000 / 17.267 / 175.933 / 18.415 |0.508 / 0.529 / 11.408 / 0.321 |224 / 224 / 224 / 0 | |
| |1500 |32 |8 |0.000 / 14.667 / 29.200 / 4.897 |0.529 / 0.563 / 11.784 / 0.278 |393 / 404 / 416 / 3 | |
| |1500 |32 |12 |0.000 / 0.000 / 0.133 / 0.012 |0.611 / 0.668 / 13.335 / 0.342 |7226 / 7252 / 7273 / 6 | |
| |1500 |32 |16 |0.000 / 0.000 / 0.133 / 0.012 |0.611 / 0.655 / 19.309 / 0.454 |7230 / 7254 / 7276 / 6 | |
| |1500 |32 |20 |0.000 / 0.000 / 0.133 / 0.012 |0.612 / 0.643 / 12.975 / 0.314 |7231 / 7256 / 7277 / 6 | |
| |1500 |32 |24 |0.000 / 0.000 / 0.067 / 0.013 |0.612 / 0.666 / 12.496 / 0.357 |7236 / 7262 / 7286 / 7 | |
| |1500 |32 |28 |0.000 / 0.000 / 0.133 / 0.013 |0.625 / 0.657 / 21.381 / 0.401 |7305 / 7347 / 7383 / 11 | |
| |1500 |32 |32 |0.000 / 0.000 / 0.133 / 0.012 |0.609 / 0.652 / 19.148 / 0.387 |8643 / 8687 / 8724 / 12 | |
| |2000 |4 |4 |0.050 / 17.950 / 200.300 / 17.701 |0.678 / 0.712 / 13.274 / 0.370 |220 / 220 / 220 / 0 | |
| |2000 |8 |4 |0.050 / 17.200 / 227.450 / 17.112 |0.678 / 0.706 / 12.428 / 0.357 |221 / 221 / 221 / 0 | |
| |2000 |8 |8 |0.000 / 4.150 / 28.100 / 3.687 |0.705 / 0.746 / 11.845 / 0.297 |397 / 408 / 418 / 3 | |
| |2000 |12 |4 |0.050 / 17.075 / 166.300 / 17.289 |0.679 / 0.746 / 19.810 / 0.436 |221 / 221 / 221 / 0 | |
| |2000 |12 |8 |0.000 / 4.050 / 26.000 / 3.697 |0.700 / 0.772 / 19.166 / 0.385 |396 / 408 / 419 / 3 | |
| |2000 |12 |12 |0.050 / 0.850 / 4.150 / 0.733 |0.818 / 0.904 / 20.237 / 0.460 |5109 / 5354 / 5633 / 103 | |
| |2000 |16 |4 |0.050 / 17.650 / 196.200 / 17.539 |0.678 / 0.747 / 17.211 / 0.435 |222 / 222 / 222 / 0 | |
| |2000 |16 |8 |0.000 / 4.100 / 23.650 / 3.720 |0.699 / 0.770 / 17.887 / 0.378 |397 / 409 / 420 / 3 | |
| |2000 |16 |12 |0.000 / 0.200 / 1.100 / 0.163 |0.835 / 0.915 / 24.825 / 0.515 |6456 / 6547 / 6647 / 23 | |
| |2000 |16 |16 |0.000 / 0.200 / 1.100 / 0.167 |0.820 / 0.906 / 15.149 / 0.427 |8177 / 8278 / 8384 / 27 | |
| |2000 |20 |4 |0.050 / 17.650 / 184.550 / 17.468 |0.678 / 0.742 / 19.468 / 0.449 |223 / 223 / 223 / 0 | |
| |2000 |20 |8 |0.000 / 4.150 / 27.500 / 3.693 |0.700 / 0.756 / 12.525 / 0.289 |399 / 410 / 420 / 3 | |
| |2000 |20 |12 |0.000 / 0.050 / 0.350 / 0.044 |0.830 / 0.921 / 19.963 / 0.553 |7551 / 7643 / 7742 / 23 | |
| |2000 |20 |16 |0.000 / 0.050 / 0.300 / 0.044 |0.840 / 0.916 / 16.698 / 0.448 |7615 / 7702 / 7789 / 23 | |
| |2000 |20 |20 |0.000 / 0.050 / 0.300 / 0.044 |0.824 / 0.909 / 23.619 / 0.600 |8602 / 8633 / 8652 / 7 | |
| |2000 |24 |4 |0.050 / 17.600 / 249.350 / 17.740 |0.675 / 0.754 / 18.418 / 0.431 |223 / 223 / 223 / 0 | |
| |2000 |24 |8 |0.000 / 4.150 / 24.700 / 3.663 |0.697 / 0.753 / 24.521 / 0.466 |398 / 410 / 421 / 3 | |
| |2000 |24 |12 |0.000 / 0.000 / 0.150 / 0.017 |0.825 / 0.907 / 16.320 / 0.373 |8559 / 8591 / 8621 / 8 | |
| |2000 |24 |16 |0.000 / 0.000 / 0.100 / 0.017 |0.826 / 0.876 / 20.177 / 0.403 |8567 / 8600 / 8630 / 9 | |
| |2000 |24 |20 |0.000 / 0.000 / 0.100 / 0.017 |0.836 / 0.891 / 18.865 / 0.429 |8670 / 8715 / 8762 / 13 | |
| |2000 |24 |24 |0.000 / 0.000 / 0.150 / 0.017 |0.823 / 0.874 / 13.270 / 0.398 |10491 / 10531 / 10569 / 11 | |
| |2000 |28 |4 |0.050 / 17.600 / 206.800 / 17.750 |0.675 / 0.726 / 18.063 / 0.424 |224 / 224 / 224 / 0 | |
| |2000 |28 |8 |0.000 / 6.050 / 23.850 / 4.359 |0.698 / 0.752 / 24.616 / 0.531 |400 / 411 / 421 / 3 | |
| |2000 |28 |12 |0.000 / 0.000 / 0.100 / 0.013 |0.826 / 0.879 / 16.497 / 0.458 |8600 / 8623 / 8641 / 6 | |
| |2000 |28 |16 |0.000 / 0.000 / 0.100 / 0.012 |0.826 / 0.898 / 16.948 / 0.417 |8602 / 8626 / 8645 / 6 | |
| |2000 |28 |20 |0.000 / 0.000 / 0.150 / 0.013 |0.826 / 0.876 / 21.015 / 0.435 |8609 / 8634 / 8656 / 6 | |
| |2000 |28 |24 |0.000 / 0.000 / 0.100 / 0.012 |0.836 / 0.878 / 19.773 / 0.462 |8704 / 8746 / 8791 / 12 | |
| |2000 |28 |28 |0.000 / 0.000 / 0.100 / 0.012 |0.823 / 0.877 / 14.429 / 0.366 |10489 / 10535 / 10573 / 11 | |
| |2000 |32 |4 |0.050 / 18.300 / 180.850 / 18.635 |0.676 / 0.723 / 18.331 / 0.431 |224 / 224 / 224 / 0 | |
| |2000 |32 |8 |0.000 / 10.900 / 26.400 / 5.122 |0.698 / 0.742 / 18.170 / 0.443 |397 / 409 / 420 / 3 | |
| |2000 |32 |12 |0.000 / 0.000 / 0.100 / 0.013 |0.825 / 0.869 / 12.927 / 0.376 |8603 / 8623 / 8642 / 6 | |
| |2000 |32 |16 |0.000 / 0.000 / 0.100 / 0.012 |0.826 / 0.876 / 12.052 / 0.376 |8601 / 8625 / 8644 / 6 | |
| |2000 |32 |20 |0.000 / 0.000 / 0.100 / 0.012 |0.824 / 0.883 / 19.872 / 0.492 |8602 / 8627 / 8644 / 6 | |
| |2000 |32 |24 |0.000 / 0.000 / 0.100 / 0.012 |0.826 / 0.879 / 21.842 / 0.509 |8610 / 8635 / 8654 / 6 | |
| |2000 |32 |28 |0.000 / 0.000 / 0.150 / 0.012 |0.836 / 0.894 / 18.071 / 0.448 |8707 / 8748 / 8791 / 12 | |
| |2000 |32 |32 |0.000 / 0.000 / 0.100 / 0.012 |0.823 / 0.870 / 13.917 / 0.398 |10491 / 10535 / 10575 / 11 | |
| |2500 |4 |4 |0.000 / 17.480 / 215.840 / 18.035 |0.844 / 0.896 / 17.723 / 0.444 |220 / 220 / 220 / 0 | |
| |2500 |8 |4 |0.000 / 17.960 / 369.960 / 18.503 |0.844 / 0.897 / 20.260 / 0.542 |221 / 221 / 221 / 0 | |
| |2500 |8 |8 |0.000 / 4.080 / 23.280 / 3.639 |0.874 / 0.928 / 21.505 / 0.472 |399 / 412 / 420 / 3 | |
| |2500 |12 |4 |0.000 / 17.520 / 204.760 / 17.973 |0.845 / 0.963 / 21.008 / 0.499 |221 / 221 / 221 / 0 | |
| |2500 |12 |8 |0.000 / 4.080 / 30.120 / 3.683 |0.869 / 0.946 / 18.277 / 0.488 |400 / 412 / 421 / 3 | |
| |2500 |12 |12 |0.000 / 0.880 / 4.600 / 0.747 |1.025 / 1.134 / 20.157 / 0.532 |6140 / 6934 / 7154 / 78 | |
| |2500 |16 |4 |0.000 / 17.240 / 173.400 / 17.729 |0.845 / 0.902 / 15.768 / 0.421 |222 / 222 / 222 / 0 | |
| |2500 |16 |8 |0.000 / 4.160 / 25.520 / 3.846 |0.868 / 0.929 / 11.497 / 0.362 |401 / 413 / 422 / 3 | |
| |2500 |16 |12 |0.000 / 0.200 / 1.080 / 0.164 |1.048 / 1.156 / 20.371 / 0.514 |8704 / 8924 / 9057 / 33 | |
| |2500 |16 |16 |0.000 / 0.200 / 1.000 / 0.166 |1.031 / 1.108 / 13.336 / 0.432 |11159 / 11486 / 11633 / 42 | |
| |2500 |20 |4 |0.000 / 17.600 / 154.600 / 17.373 |0.845 / 0.951 / 14.707 / 0.424 |223 / 223 / 223 / 0 | |
| |2500 |20 |8 |0.000 / 4.200 / 23.720 / 3.781 |0.868 / 0.932 / 20.062 / 0.477 |397 / 414 / 424 / 3 | |
| |2500 |20 |12 |0.000 / 0.040 / 0.320 / 0.044 |1.032 / 1.119 / 17.855 / 0.560 |10273 / 10533 / 10639 / 33 | |
| |2500 |20 |16 |0.000 / 0.040 / 0.320 / 0.043 |1.045 / 1.107 / 16.834 / 0.461 |10370 / 10621 / 10730 / 33 | |
| |2500 |20 |20 |0.000 / 0.040 / 0.400 / 0.044 |1.028 / 1.087 / 22.330 / 0.536 |11718 / 11995 / 12020 / 12 | |
| |2500 |24 |4 |0.000 / 17.480 / 156.160 / 17.171 |0.844 / 0.890 / 18.096 / 0.406 |223 / 223 / 223 / 0 | |
| |2500 |24 |8 |0.000 / 4.140 / 22.280 / 3.716 |0.868 / 0.915 / 19.031 / 0.458 |403 / 414 / 423 / 3 | |
| |2500 |24 |12 |0.000 / 0.000 / 0.160 / 0.017 |1.030 / 1.088 / 22.283 / 0.500 |11894 / 11934 / 11971 / 11 | |
| |2500 |24 |16 |0.000 / 0.000 / 0.120 / 0.017 |1.032 / 1.090 / 18.220 / 0.547 |11896 / 11947 / 11990 / 12 | |
| |2500 |24 |20 |0.000 / 0.000 / 0.120 / 0.018 |1.043 / 1.101 / 17.194 / 0.516 |12056 / 12115 / 12187 / 18 | |
| |2500 |24 |24 |0.000 / 0.000 / 0.120 / 0.018 |1.027 / 1.098 / 21.530 / 0.547 |14736 / 14798 / 14849 / 15 | |
| |2500 |28 |4 |0.000 / 17.720 / 199.480 / 17.641 |0.845 / 0.900 / 20.585 / 0.511 |224 / 224 / 224 / 0 | |
| |2500 |28 |8 |0.000 / 5.200 / 22.920 / 4.106 |0.869 / 0.914 / 12.212 / 0.359 |403 / 414 / 425 / 3 | |
| |2500 |28 |12 |0.000 / 0.000 / 0.080 / 0.012 |1.031 / 1.085 / 17.978 / 0.500 |12267 / 12329 / 12367 / 11 | |
| |2500 |28 |16 |0.000 / 0.000 / 0.120 / 0.013 |1.030 / 1.094 / 18.134 / 0.519 |11986 / 12331 / 12372 / 13 | |
| |2500 |28 |20 |0.000 / 0.000 / 0.120 / 0.011 |1.031 / 1.086 / 20.068 / 0.550 |12296 / 12341 / 12382 / 12 | |
| |2500 |28 |24 |0.000 / 0.000 / 0.080 / 0.012 |1.043 / 1.121 / 21.095 / 0.587 |12117 / 12486 / 12547 / 17 | |
| |2500 |28 |28 |0.000 / 0.000 / 0.080 / 0.012 |1.028 / 1.097 / 20.162 / 0.573 |14747 / 14802 / 14864 / 15 | |
| |2500 |32 |4 |0.000 / 18.200 / 187.680 / 18.797 |0.844 / 0.889 / 20.143 / 0.460 |224 / 224 / 224 / 0 | |
| |2500 |32 |8 |0.000 / 8.680 / 25.600 / 4.951 |0.868 / 0.923 / 20.998 / 0.525 |402 / 413 / 422 / 3 | |
| |2500 |32 |12 |0.000 / 0.000 / 0.080 / 0.012 |1.030 / 1.130 / 24.624 / 0.666 |12378 / 12414 / 12443 / 8 | |
| |2500 |32 |16 |0.000 / 0.000 / 0.080 / 0.012 |1.030 / 1.090 / 17.364 / 0.532 |12381 / 12416 / 12443 / 8 | |
| |2500 |32 |20 |0.000 / 0.000 / 0.120 / 0.012 |1.029 / 1.152 / 27.049 / 0.647 |12386 / 12418 / 12444 / 8 | |
| |2500 |32 |24 |0.000 / 0.000 / 0.080 / 0.013 |1.031 / 1.083 / 26.018 / 0.575 |12390 / 12427 / 12460 / 9 | |
| |2500 |32 |28 |0.000 / 0.000 / 0.120 / 0.013 |1.042 / 1.102 / 18.739 / 0.486 |12513 / 12568 / 12621 / 14 | |
| |2500 |32 |32 |0.000 / 0.000 / 0.120 / 0.012 |1.027 / 1.088 / 16.996 / 0.499 |14742 / 14802 / 14851 / 15 | |
| |3000 |4 |4 |0.000 / 17.833 / 186.867 / 17.498 |1.013 / 1.080 / 18.433 / 0.461 |220 / 220 / 220 / 0 | |
| |3000 |8 |4 |0.033 / 17.600 / 201.567 / 17.855 |1.013 / 1.074 / 19.563 / 0.528 |221 / 221 / 221 / 0 | |
| |3000 |8 |8 |0.000 / 4.300 / 26.467 / 3.786 |1.042 / 1.101 / 13.911 / 0.435 |403 / 414 / 422 / 3 | |
| |3000 |12 |4 |0.000 / 17.467 / 212.600 / 18.720 |1.014 / 1.070 / 21.257 / 0.509 |221 / 221 / 221 / 0 | |
| |3000 |12 |8 |0.000 / 4.167 / 30.600 / 3.794 |1.039 / 1.104 / 20.074 / 0.553 |405 / 414 / 422 / 3 | |
| |3000 |12 |12 |0.000 / 0.833 / 4.833 / 0.745 |1.233 / 1.322 / 20.558 / 0.638 |7453 / 7706 / 7923 / 67 | |
| |3000 |16 |4 |0.000 / 18.117 / 152.933 / 17.161 |1.013 / 1.101 / 21.535 / 0.564 |222 / 222 / 222 / 0 | |
| |3000 |16 |8 |0.000 / 4.200 / 25.633 / 3.817 |1.037 / 1.100 / 18.806 / 0.554 |405 / 415 / 423 / 3 | |
| |3000 |16 |12 |0.000 / 0.200 / 1.033 / 0.169 |1.258 / 1.331 / 22.352 / 0.682 |10292 / 10439 / 10570 / 37 | |
| |3000 |16 |16 |0.000 / 0.200 / 0.967 / 0.166 |1.238 / 1.317 / 18.279 / 0.589 |13313 / 13458 / 13612 / 43 | |
| |3000 |20 |4 |0.000 / 17.467 / 164.967 / 17.757 |1.013 / 1.113 / 19.652 / 0.508 |223 / 223 / 223 / 0 | |
| |3000 |20 |8 |0.000 / 4.167 / 24.267 / 3.743 |1.038 / 1.092 / 18.642 / 0.554 |405 / 416 / 425 / 3 | |
| |3000 |20 |12 |0.000 / 0.033 / 0.400 / 0.045 |1.240 / 1.303 / 14.404 / 0.517 |12293 / 12419 / 12571 / 35 | |
| |3000 |20 |16 |0.000 / 0.033 / 0.300 / 0.045 |1.256 / 1.327 / 17.597 / 0.537 |12406 / 12522 / 12655 / 35 | |
| |3000 |20 |20 |0.000 / 0.033 / 0.333 / 0.044 |1.237 / 1.308 / 20.509 / 0.558 |14100 / 14156 / 14250 / 11 | |
| |3000 |24 |4 |0.000 / 17.417 / 215.733 / 18.149 |1.013 / 1.091 / 21.116 / 0.517 |223 / 223 / 223 / 0 | |
| |3000 |24 |8 |0.000 / 4.233 / 21.767 / 3.773 |1.036 / 1.092 / 11.990 / 0.434 |406 / 416 / 426 / 3 | |
| |3000 |24 |12 |0.000 / 0.000 / 0.100 / 0.017 |1.239 / 1.343 / 19.689 / 0.541 |14036 / 14088 / 14126 / 12 | |
| |3000 |24 |16 |0.000 / 0.000 / 0.133 / 0.017 |1.239 / 1.311 / 14.029 / 0.532 |14050 / 14102 / 14146 / 13 | |
| |3000 |24 |20 |0.000 / 0.000 / 0.133 / 0.017 |1.255 / 1.327 / 25.879 / 0.611 |14218 / 14302 / 14381 / 20 | |
| |3000 |24 |24 |0.000 / 0.000 / 0.133 / 0.017 |1.233 / 1.299 / 20.804 / 0.615 |17434 / 17491 / 17548 / 17 | |
| |3000 |28 |4 |0.000 / 17.467 / 160.633 / 17.603 |1.014 / 1.069 / 15.224 / 0.466 |224 / 224 / 224 / 0 | |
| |3000 |28 |8 |0.000 / 4.733 / 22.600 / 3.937 |1.037 / 1.099 / 17.982 / 0.528 |407 / 417 / 426 / 3 | |
| |3000 |28 |12 |0.000 / 0.000 / 0.100 / 0.013 |1.239 / 1.301 / 20.579 / 0.688 |14494 / 14534 / 14579 / 12 | |
| |3000 |28 |16 |0.000 / 0.000 / 0.100 / 0.012 |1.237 / 1.299 / 16.532 / 0.499 |14490 / 14536 / 14575 / 12 | |
| |3000 |28 |20 |0.000 / 0.000 / 0.100 / 0.013 |1.239 / 1.337 / 19.234 / 0.591 |14503 / 14548 / 14594 / 13 | |
| |3000 |28 |24 |0.000 / 0.000 / 0.100 / 0.012 |1.254 / 1.328 / 17.902 / 0.580 |14666 / 14722 / 14789 / 18 | |
| |3000 |28 |28 |0.000 / 0.000 / 0.100 / 0.013 |1.234 / 1.300 / 20.500 / 0.552 |17439 / 17495 / 17558 / 17 | |
| |3000 |32 |4 |0.000 / 17.600 / 205.500 / 18.233 |1.012 / 1.059 / 11.735 / 0.414 |224 / 224 / 224 / 0 | |
| |3000 |32 |8 |0.000 / 7.450 / 23.967 / 4.786 |1.038 / 1.102 / 15.705 / 0.478 |405 / 416 / 424 / 3 | |
| |3000 |32 |12 |0.000 / 0.000 / 0.133 / 0.013 |1.238 / 1.335 / 18.066 / 0.600 |14599 / 14630 / 14661 / 9 | |
| |3000 |32 |16 |0.000 / 0.000 / 0.100 / 0.012 |1.238 / 1.324 / 20.202 / 0.642 |14596 / 14632 / 14662 / 9 | |
| |3000 |32 |20 |0.000 / 0.000 / 0.100 / 0.013 |1.238 / 1.313 / 17.383 / 0.513 |14601 / 14634 / 14663 / 9 | |
| |3000 |32 |24 |0.000 / 0.000 / 0.100 / 0.012 |1.239 / 1.300 / 19.292 / 0.560 |14607 / 14646 / 14677 / 10 | |
| |3000 |32 |28 |0.000 / 0.000 / 0.100 / 0.012 |1.254 / 1.332 / 20.109 / 0.578 |14761 / 14815 / 14872 / 16 | |
| |3000 |32 |32 |0.000 / 0.000 / 0.067 / 0.012 |1.234 / 1.304 / 18.132 / 0.537 |17436 / 17495 / 17546 / 17 | |
| |3500 |4 |4 |0.057 / 17.843 / 154.429 / 17.131 |1.183 / 1.241 / 18.127 / 0.543 |220 / 220 / 220 / 0 | |
| |3500 |8 |4 |0.000 / 18.057 / 187.514 / 17.697 |1.182 / 1.250 / 20.428 / 0.564 |221 / 221 / 221 / 0 | |
| |3500 |8 |8 |0.000 / 4.314 / 28.257 / 3.833 |1.213 / 1.299 / 20.333 / 0.672 |405 / 416 / 424 / 2 | |
| |3500 |12 |4 |0.057 / 17.571 / 161.857 / 17.664 |1.181 / 1.252 / 20.923 / 0.569 |221 / 221 / 221 / 0 | |
| |3500 |12 |8 |0.000 / 4.229 / 24.943 / 3.759 |1.208 / 1.275 / 22.113 / 0.543 |406 / 416 / 424 / 3 | |
| |3500 |12 |12 |0.000 / 0.871 / 4.657 / 0.786 |1.442 / 1.523 / 18.288 / 0.590 |8126 / 8383 / 8643 / 70 | |
| |3500 |16 |4 |0.000 / 18.114 / 163.343 / 17.771 |1.183 / 1.253 / 20.723 / 0.585 |222 / 222 / 222 / 0 | |
| |3500 |16 |8 |0.000 / 4.286 / 30.143 / 3.812 |1.207 / 1.298 / 20.047 / 0.593 |406 / 418 / 424 / 2 | |
| |3500 |16 |12 |0.000 / 1.057 / 5.457 / 0.698 |1.528 / 1.659 / 22.643 / 0.768 |2763 / 2824 / 2893 / 18 | |
| |3500 |16 |16 |0.000 / 0.200 / 1.029 / 0.166 |1.450 / 1.558 / 20.879 / 0.642 |15119 / 15300 / 15466 / 46 | |
| |3500 |20 |4 |0.057 / 18.029 / 156.029 / 17.445 |1.183 / 1.247 / 13.857 / 0.462 |223 / 223 / 223 / 0 | |
| |3500 |20 |8 |0.000 / 4.314 / 26.371 / 3.859 |1.206 / 1.311 / 22.816 / 0.630 |408 / 419 / 426 / 3 | |
| |3500 |20 |12 |0.000 / 1.057 / 4.771 / 0.694 |1.537 / 1.636 / 23.911 / 0.753 |2760 / 2824 / 2883 / 18 | |
| |3500 |20 |16 |0.000 / 0.057 / 0.343 / 0.042 |1.472 / 1.571 / 20.930 / 0.676 |14140 / 14282 / 14406 / 38 | |
| |3500 |20 |20 |0.000 / 0.057 / 0.257 / 0.043 |1.447 / 1.526 / 15.232 / 0.536 |16096 / 16146 / 16234 / 13 | |
| |3500 |24 |4 |0.057 / 17.029 / 212.571 / 17.691 |1.183 / 1.254 / 17.746 / 0.583 |223 / 223 / 223 / 0 | |
| |3500 |24 |8 |0.000 / 4.271 / 23.257 / 3.762 |1.207 / 1.268 / 15.615 / 0.492 |409 / 418 / 426 / 2 | |
| |3500 |24 |12 |0.000 / 1.057 / 6.229 / 0.706 |1.533 / 1.653 / 22.778 / 0.677 |2760 / 2824 / 2888 / 18 | |
| |3500 |24 |16 |0.000 / 0.000 / 0.114 / 0.017 |1.451 / 1.588 / 18.966 / 0.697 |16029 / 16088 / 16131 / 13 | |
| |3500 |24 |20 |0.000 / 0.000 / 0.114 / 0.017 |1.469 / 1.573 / 16.584 / 0.477 |16233 / 16316 / 16395 / 22 | |
| |3500 |24 |24 |0.000 / 0.000 / 0.114 / 0.017 |1.443 / 1.545 / 14.326 / 0.523 |19883 / 19959 / 20026 / 18 | |
| |3500 |28 |4 |0.029 / 17.371 / 256.857 / 18.779 |1.182 / 1.254 / 13.532 / 0.445 |224 / 224 / 224 / 0 | |
| |3500 |28 |8 |0.000 / 4.543 / 28.200 / 3.934 |1.208 / 1.317 / 22.788 / 0.594 |408 / 419 / 427 / 3 | |
| |3500 |28 |12 |84.371 / 84.371 / 84.514 / 0.021 |1.511 / 1.626 / 15.627 / 0.572 |1188 / 1204 / 1220 / 4 | |
| |3500 |28 |16 |0.000 / 0.000 / 0.086 / 0.012 |1.449 / 1.530 / 13.406 / 0.526 |16477 / 16528 / 16572 / 12 | |
| |3500 |28 |20 |0.000 / 0.000 / 0.114 / 0.012 |1.451 / 1.539 / 16.128 / 0.543 |16495 / 16542 / 16587 / 13 | |
| |3500 |28 |24 |0.000 / 0.000 / 0.086 / 0.012 |1.469 / 1.551 / 15.112 / 0.554 |16679 / 16744 / 16813 / 19 | |
| |3500 |28 |28 |0.000 / 0.000 / 0.086 / 0.012 |1.445 / 1.545 / 15.241 / 0.543 |19879 / 19962 / 20018 / 17 | |
| |3500 |32 |4 |0.057 / 17.371 / 188.286 / 17.775 |1.183 / 1.258 / 13.408 / 0.468 |224 / 224 / 224 / 0 | |
| |3500 |32 |8 |0.000 / 6.400 / 25.629 / 4.553 |1.207 / 1.271 / 12.497 / 0.443 |409 / 418 / 427 / 3 | |
| |3500 |32 |12 |99.086 / 99.086 / 99.086 / 0.000 |1.506 / 1.646 / 14.954 / 0.571 |920 / 920 / 920 / 0 | |
| |3500 |32 |16 |0.000 / 0.000 / 0.086 / 0.012 |1.449 / 1.537 / 15.306 / 0.511 |16582 / 16624 / 16657 / 10 | |
| |3500 |32 |20 |0.000 / 0.000 / 0.114 / 0.013 |1.450 / 1.558 / 15.277 / 0.547 |16587 / 16626 / 16657 / 10 | |
| |3500 |32 |24 |0.000 / 0.000 / 0.086 / 0.013 |1.451 / 1.555 / 20.496 / 0.661 |16604 / 16639 / 16678 / 10 | |
| |3500 |32 |28 |0.000 / 0.000 / 0.114 / 0.012 |1.468 / 1.554 / 15.893 / 0.489 |16768 / 16836 / 16895 / 17 | |
| |3500 |32 |32 |0.000 / 0.000 / 0.086 / 0.012 |1.445 / 1.548 / 20.310 / 0.537 |19889 / 19962 / 20025 / 17 | |
| |4000 |4 |4 |0.025 / 17.950 / 203.200 / 17.860 |1.351 / 1.431 / 14.530 / 0.524 |220 / 220 / 220 / 0 | |
| |4000 |8 |4 |0.050 / 17.175 / 216.800 / 17.734 |1.351 / 1.421 / 19.692 / 0.620 |221 / 221 / 221 / 0 | |
| |4000 |8 |8 |0.000 / 4.325 / 24.925 / 3.817 |1.382 / 1.475 / 19.092 / 0.560 |408 / 418 / 424 / 2 | |
| |4000 |12 |4 |0.025 / 17.538 / 175.650 / 18.066 |1.352 / 1.437 / 19.980 / 0.590 |221 / 221 / 221 / 0 | |
| |4000 |12 |8 |0.000 / 4.400 / 29.450 / 3.863 |1.378 / 1.477 / 22.190 / 0.643 |410 / 418 / 424 / 2 | |
| |4000 |12 |12 |0.025 / 0.900 / 5.775 / 0.799 |1.648 / 1.764 / 19.034 / 0.720 |8718 / 8981 / 9265 / 70 | |
| |4000 |16 |4 |0.050 / 17.575 / 176.200 / 17.005 |1.350 / 1.429 / 18.900 / 0.549 |222 / 222 / 222 / 0 | |
| |4000 |16 |8 |0.000 / 4.250 / 26.400 / 3.828 |1.376 / 1.451 / 18.224 / 0.601 |410 / 419 / 425 / 2 | |
| |4000 |16 |12 |0.275 / 0.950 / 4.275 / 0.615 |1.697 / 1.867 / 21.027 / 0.759 |2813 / 2882 / 2951 / 17 | |
| |4000 |16 |16 |0.000 / 0.200 / 1.075 / 0.168 |1.660 / 1.801 / 17.255 / 0.650 |16124 / 17130 / 17352 / 55 | |
| |4000 |20 |4 |0.050 / 18.325 / 148.475 / 17.187 |1.351 / 1.479 / 18.402 / 0.524 |223 / 223 / 223 / 0 | |
| |4000 |20 |8 |0.000 / 4.350 / 25.750 / 3.871 |1.375 / 1.446 / 24.560 / 0.671 |410 / 420 / 426 / 2 | |
| |4000 |20 |12 |0.275 / 0.975 / 4.400 / 0.635 |1.680 / 1.805 / 23.306 / 0.729 |2821 / 2883 / 2951 / 17 | |
| |4000 |20 |16 |0.000 / 0.050 / 0.325 / 0.042 |1.684 / 1.783 / 20.501 / 0.699 |15888 / 16038 / 16188 / 39 | |
| |4000 |20 |20 |0.000 / 0.050 / 0.325 / 0.042 |1.658 / 1.759 / 19.013 / 0.689 |18077 / 18134 / 18240 / 14 | |
| |4000 |24 |4 |0.000 / 17.625 / 164.300 / 17.292 |1.351 / 1.427 / 19.917 / 0.579 |223 / 223 / 223 / 0 | |
| |4000 |24 |8 |0.000 / 4.125 / 26.750 / 3.905 |1.375 / 1.442 / 18.641 / 0.557 |410 / 420 / 426 / 2 | |
| |4000 |24 |12 |0.275 / 0.975 / 4.275 / 0.610 |1.703 / 1.810 / 22.554 / 0.826 |2825 / 2883 / 2942 / 16 | |
| |4000 |24 |16 |0.000 / 0.000 / 0.100 / 0.017 |1.659 / 1.752 / 17.914 / 0.612 |18021 / 18073 / 18152 / 14 | |
| |4000 |24 |20 |0.000 / 0.000 / 0.125 / 0.017 |1.681 / 1.789 / 20.884 / 0.743 |18241 / 18331 / 18404 / 22 | |
| |4000 |24 |24 |0.000 / 0.000 / 0.150 / 0.017 |1.653 / 1.725 / 16.967 / 0.604 |22355 / 22426 / 22594 / 19 | |
| |4000 |28 |4 |0.050 / 18.125 / 167.050 / 17.329 |1.348 / 1.501 / 23.889 / 0.758 |224 / 224 / 224 / 0 | |
| |4000 |28 |8 |0.000 / 4.475 / 23.700 / 3.835 |1.374 / 1.588 / 21.053 / 0.732 |411 / 421 / 427 / 2 | |
| |4000 |28 |12 |67.225 / 75.800 / 76.550 / 0.261 |1.691 / 1.936 / 17.278 / 0.667 |1620 / 1704 / 2001 / 23 | |
| |4000 |28 |16 |0.000 / 0.000 / 0.100 / 0.012 |1.707 / 1.932 / 16.598 / 0.577 |18470 / 18520 / 18569 / 13 | |
| |4000 |28 |20 |0.000 / 0.000 / 0.075 / 0.012 |1.732 / 1.984 / 20.429 / 0.752 |18482 / 18536 / 18582 / 14 | |
| |4000 |28 |24 |0.000 / 0.000 / 0.075 / 0.012 |1.739 / 1.993 / 21.507 / 0.670 |18690 / 18766 / 18838 / 20 | |
| |4000 |28 |28 |0.000 / 0.000 / 0.100 / 0.012 |1.658 / 1.939 / 14.211 / 0.594 |22364 / 22431 / 22489 / 18 | |
| |4000 |32 |4 |0.000 / 17.950 / 206.150 / 18.020 |1.386 / 1.567 / 14.368 / 0.550 |224 / 224 / 224 / 0 | |
| |4000 |32 |8 |0.000 / 5.925 / 22.650 / 4.366 |1.395 / 1.594 / 16.171 / 0.515 |412 / 421 / 427 / 2 | |
| |4000 |32 |12 |88.250 / 88.675 / 89.125 / 0.127 |1.732 / 1.985 / 23.129 / 0.774 |1408 / 1486 / 1563 / 22 | |
| |4000 |32 |16 |0.000 / 0.000 / 0.100 / 0.012 |1.709 / 1.935 / 28.713 / 0.675 |18577 / 18616 / 18647 / 10 | |
| |4000 |32 |20 |0.000 / 0.000 / 0.100 / 0.012 |1.711 / 1.925 / 20.065 / 0.653 |18583 / 18618 / 18653 / 10 | |
| |4000 |32 |24 |0.000 / 0.000 / 0.100 / 0.013 |1.699 / 1.917 / 20.723 / 0.617 |18589 / 18633 / 18664 / 10 | |
| |4000 |32 |28 |0.000 / 0.000 / 0.075 / 0.012 |1.734 / 1.977 / 27.269 / 0.753 |18792 / 18858 / 18921 / 18 | |
| |4000 |32 |32 |0.000 / 0.000 / 0.100 / 0.012 |1.704 / 1.921 / 14.253 / 0.553 |22343 / 22431 / 22575 / 19 | |
| |4500 |4 |4 |0.044 / 17.911 / 173.911 / 18.150 |1.550 / 1.758 / 15.086 / 0.501 |220 / 220 / 220 / 0 | |
| |4500 |8 |4 |0.044 / 17.800 / 206.000 / 17.391 |1.551 / 1.768 / 13.653 / 0.545 |221 / 221 / 221 / 0 | |
| |4500 |8 |8 |0.000 / 4.222 / 26.378 / 3.844 |1.595 / 1.838 / 20.724 / 0.697 |411 / 419 / 424 / 2 | |
| |4500 |12 |4 |0.000 / 17.689 / 172.667 / 17.123 |1.561 / 1.751 / 15.085 / 0.578 |221 / 221 / 221 / 0 | |
| |4500 |12 |8 |0.000 / 4.300 / 24.644 / 3.826 |1.588 / 1.791 / 20.054 / 0.593 |410 / 419 / 424 / 2 | |
| |4500 |12 |12 |0.022 / 0.911 / 4.956 / 0.805 |1.915 / 2.172 / 19.644 / 0.615 |9234 / 9507 / 9768 / 70 | |
| |4500 |16 |4 |0.000 / 18.244 / 258.444 / 17.967 |1.526 / 1.758 / 15.643 / 0.552 |222 / 222 / 222 / 0 | |
| |4500 |16 |8 |0.000 / 4.200 / 26.978 / 3.931 |1.563 / 1.786 / 15.151 / 0.572 |413 / 420 / 425 / 2 | |
| |4500 |16 |12 |0.133 / 0.933 / 5.044 / 0.687 |1.972 / 2.210 / 22.173 / 0.741 |2864 / 2927 / 2980 / 15 | |
| |4500 |16 |16 |0.000 / 0.200 / 0.978 / 0.166 |1.947 / 2.221 / 21.386 / 0.766 |17912 / 18092 / 18533 / 55 | |
| |4500 |20 |4 |0.044 / 17.478 / 185.133 / 17.498 |1.561 / 1.765 / 13.399 / 0.524 |223 / 223 / 223 / 0 | |
| |4500 |20 |8 |0.000 / 4.333 / 29.822 / 3.854 |1.590 / 1.787 / 17.447 / 0.612 |412 / 421 / 426 / 2 | |
| |4500 |20 |12 |0.133 / 0.911 / 4.778 / 0.683 |1.953 / 2.210 / 24.534 / 0.768 |2875 / 2928 / 2982 / 15 | |
| |4500 |20 |16 |0.000 / 0.044 / 0.311 / 0.042 |1.987 / 2.216 / 14.466 / 0.611 |16982 / 17116 / 17239 / 36 | |
| |4500 |20 |20 |0.000 / 0.044 / 0.244 / 0.042 |1.944 / 2.176 / 20.109 / 0.655 |19151 / 19206 / 19254 / 15 | |
| |4500 |24 |4 |0.044 / 17.644 / 191.756 / 18.124 |1.562 / 1.747 / 15.134 / 0.531 |223 / 223 / 223 / 0 | |
| |4500 |24 |8 |0.000 / 4.178 / 28.111 / 3.838 |1.553 / 1.804 / 23.164 / 0.670 |413 / 421 / 426 / 2 | |
| |4500 |24 |12 |0.133 / 0.933 / 4.156 / 0.663 |1.951 / 2.182 / 18.935 / 0.663 |2869 / 2928 / 2982 / 16 | |
| |4500 |24 |16 |0.000 / 0.022 / 0.067 / 0.012 |1.943 / 2.170 / 17.937 / 0.713 |19107 / 19153 / 19196 / 13 | |
| |4500 |24 |20 |0.000 / 0.022 / 0.089 / 0.011 |1.977 / 2.262 / 24.969 / 0.761 |19338 / 19410 / 19500 / 20 | |
| |4500 |24 |24 |0.000 / 0.022 / 0.089 / 0.012 |1.929 / 2.189 / 18.456 / 0.712 |23435 / 23497 / 23550 / 17 | |
| |4500 |28 |4 |0.044 / 17.133 / 288.400 / 17.897 |1.563 / 1.759 / 19.175 / 0.641 |224 / 224 / 224 / 0 | |
| |4500 |28 |8 |0.000 / 4.378 / 24.867 / 3.803 |1.572 / 1.769 / 21.643 / 0.634 |415 / 422 / 427 / 2 | |
| |4500 |28 |12 |66.089 / 67.400 / 68.667 / 0.359 |1.930 / 2.183 / 28.176 / 0.759 |2026 / 2113 / 2214 / 27 | |
| |4500 |28 |16 |0.000 / 0.000 / 0.089 / 0.012 |1.954 / 2.170 / 20.834 / 0.751 |19176 / 19211 / 19240 / 9 | |
| |4500 |28 |20 |0.000 / 0.000 / 0.089 / 0.012 |1.942 / 2.170 / 21.282 / 0.708 |19191 / 19228 / 19261 / 10 | |
| |4500 |28 |24 |0.000 / 0.000 / 0.067 / 0.012 |1.982 / 2.234 / 29.563 / 0.905 |19410 / 19480 / 19557 / 18 | |
| |4500 |28 |28 |0.000 / 0.000 / 0.089 / 0.012 |1.944 / 2.180 / 21.368 / 0.749 |23437 / 23501 / 23562 / 17 | |
| |4500 |32 |4 |0.044 / 17.778 / 204.511 / 18.488 |1.564 / 1.801 / 23.871 / 0.680 |224 / 224 / 224 / 0 | |
| |4500 |32 |8 |0.000 / 5.489 / 22.289 / 4.225 |1.588 / 1.786 / 19.144 / 0.592 |413 / 422 / 427 / 2 | |
| |4500 |32 |12 |77.822 / 78.800 / 79.689 / 0.242 |1.926 / 2.172 / 21.370 / 0.726 |1838 / 1948 / 2039 / 26 | |
| |4500 |32 |16 |0.000 / 0.000 / 0.089 / 0.012 |1.955 / 2.213 / 24.360 / 0.725 |19171 / 19210 / 19238 / 9 | |
| |4500 |32 |20 |0.000 / 0.000 / 0.089 / 0.012 |1.949 / 2.186 / 17.274 / 0.660 |19179 / 19212 / 19241 / 9 | |
| |4500 |32 |24 |0.000 / 0.000 / 0.089 / 0.012 |1.889 / 2.182 / 28.660 / 0.825 |19192 / 19228 / 19262 / 10 | |
| |4500 |32 |28 |0.000 / 0.000 / 0.089 / 0.012 |1.903 / 2.039 / 21.219 / 0.700 |19423 / 19482 / 19549 / 18 | |
| |4500 |32 |32 |0.000 / 0.000 / 0.089 / 0.012 |1.870 / 2.004 / 20.273 / 0.742 |23437 / 23501 / 23566 / 17 | |
| |5000 |4 |4 |0.000 / 17.720 / 192.140 / 17.740 |1.684 / 1.829 / 14.606 / 0.612 |220 / 220 / 220 / 0 | |
| |5000 |8 |4 |0.060 / 17.700 / 186.580 / 17.279 |1.683 / 1.818 / 15.426 / 0.599 |221 / 221 / 221 / 0 | |
| |5000 |8 |8 |0.000 / 4.300 / 27.780 / 3.783 |1.715 / 1.913 / 20.698 / 0.741 |412 / 420 / 424 / 2 | |
| |5000 |12 |4 |0.000 / 17.040 / 206.400 / 18.593 |1.683 / 1.809 / 17.501 / 0.622 |221 / 221 / 221 / 0 | |
| |5000 |12 |8 |0.000 / 4.300 / 25.740 / 3.805 |1.710 / 1.811 / 19.765 / 0.670 |413 / 421 / 424 / 2 | |
| |5000 |12 |12 |0.020 / 0.920 / 5.200 / 0.842 |2.066 / 2.203 / 25.912 / 0.840 |9696 / 9977 / 10220 / 71 | |
| |5000 |16 |4 |0.000 / 17.700 / 190.640 / 16.754 |1.682 / 1.765 / 21.803 / 0.664 |222 / 222 / 222 / 0 | |
| |5000 |16 |8 |0.000 / 4.300 / 30.580 / 3.861 |1.709 / 1.813 / 20.311 / 0.618 |414 / 422 / 425 / 2 | |
| |5000 |16 |12 |0.000 / 0.880 / 5.140 / 0.747 |2.037 / 2.178 / 23.255 / 0.740 |2911 / 2962 / 3013 / 14 | |
| |5000 |16 |16 |0.000 / 0.200 / 1.120 / 0.171 |2.094 / 2.204 / 20.890 / 0.784 |20566 / 21208 / 21872 / 88 | |
| |5000 |20 |4 |0.000 / 17.620 / 127.520 / 17.257 |1.687 / 1.927 / 26.894 / 0.816 |223 / 223 / 223 / 0 | |
| |5000 |20 |8 |0.000 / 4.330 / 28.580 / 3.861 |1.710 / 1.909 / 18.186 / 0.643 |415 / 423 / 426 / 2 | |
| |5000 |20 |12 |0.000 / 0.900 / 5.340 / 0.743 |2.055 / 2.302 / 22.473 / 0.763 |2905 / 2963 / 3013 / 14 | |
| |5000 |20 |16 |0.000 / 0.040 / 0.320 / 0.043 |2.137 / 2.410 / 24.675 / 0.864 |19509 / 19910 / 20058 / 42 | |
| |5000 |20 |20 |0.000 / 0.040 / 0.280 / 0.042 |2.097 / 2.369 / 25.517 / 0.830 |21768 / 22416 / 22538 / 26 | |
| |5000 |24 |4 |0.040 / 17.450 / 201.900 / 17.504 |1.686 / 1.861 / 18.820 / 0.618 |223 / 223 / 223 / 0 | |
| |5000 |24 |8 |0.000 / 4.200 / 28.260 / 3.839 |1.713 / 1.923 / 19.929 / 0.679 |415 / 422 / 426 / 2 | |
| |5000 |24 |12 |0.000 / 0.900 / 4.780 / 0.766 |2.047 / 2.287 / 15.684 / 0.699 |2912 / 2963 / 3016 / 14 | |
| |5000 |24 |16 |0.000 / 0.020 / 0.100 / 0.012 |2.102 / 2.375 / 21.730 / 0.818 |22296 / 22351 / 22415 / 15 | |
| |5000 |24 |20 |0.000 / 0.020 / 0.080 / 0.012 |2.128 / 2.397 / 24.851 / 0.845 |22577 / 22658 / 22745 / 23 | |
| |5000 |24 |24 |0.000 / 0.020 / 0.080 / 0.012 |2.092 / 2.377 / 22.543 / 0.800 |27480 / 27560 / 27624 / 20 | |
| |5000 |28 |4 |0.000 / 17.860 / 158.380 / 17.534 |1.686 / 1.875 / 17.770 / 0.647 |224 / 224 / 224 / 0 | |
| |5000 |28 |8 |0.000 / 4.380 / 22.440 / 3.886 |1.712 / 1.969 / 22.211 / 0.772 |416 / 423 / 427 / 2 | |
| |5000 |28 |12 |59.020 / 60.660 / 62.740 / 0.463 |2.037 / 2.288 / 20.272 / 0.652 |2300 / 2388 / 2487 / 26 | |
| |5000 |28 |16 |0.000 / 0.000 / 0.100 / 0.012 |2.099 / 2.378 / 23.672 / 0.868 |22667 / 22723 / 22765 / 13 | |
| |5000 |28 |20 |0.000 / 0.000 / 0.080 / 0.012 |2.100 / 2.389 / 21.719 / 0.838 |22687 / 22742 / 22788 / 14 | |
| |5000 |28 |24 |0.000 / 0.000 / 0.080 / 0.012 |2.132 / 2.393 / 23.989 / 0.763 |22950 / 23027 / 23106 / 21 | |
| |5000 |28 |28 |0.000 / 0.000 / 0.080 / 0.012 |2.094 / 2.386 / 24.741 / 0.892 |27502 / 27565 / 27637 / 20 | |
| |5000 |32 |4 |0.000 / 18.060 / 189.520 / 18.378 |1.686 / 1.919 / 26.851 / 0.838 |224 / 224 / 224 / 0 | |
| |5000 |32 |8 |0.000 / 5.280 / 26.500 / 4.157 |1.714 / 1.930 / 21.178 / 0.750 |416 / 423 / 427 / 2 | |
| |5000 |32 |12 |69.800 / 70.920 / 72.440 / 0.341 |2.034 / 2.293 / 21.420 / 0.754 |2160 / 2261 / 2356 / 27 | |
| |5000 |32 |16 |0.000 / 0.000 / 0.080 / 0.012 |2.099 / 2.387 / 25.364 / 0.794 |21966 / 22798 / 22832 / 16 | |
| |5000 |32 |20 |0.000 / 0.000 / 0.080 / 0.012 |2.095 / 2.410 / 41.338 / 1.022 |22759 / 22800 / 22839 / 11 | |
| |5000 |32 |24 |0.000 / 0.000 / 0.100 / 0.012 |2.100 / 2.277 / 23.671 / 0.742 |22773 / 22818 / 22856 / 11 | |
| |5000 |32 |28 |0.000 / 0.000 / 0.100 / 0.012 |2.125 / 2.435 / 20.889 / 0.961 |22274 / 23099 / 23169 / 23 | |
| |5000 |32 |32 |0.000 / 0.000 / 0.060 / 0.012 |2.088 / 2.280 / 16.157 / 0.705 |27494 / 27565 / 27640 / 19 | |
| |
| Higher Cardinalities - sp=25, p=12,14,16 |
| ---------------------------------------- |
| |
| |card |sp |p |err l/m/h/std (% of actual) |time l/m/h/std (ms) |size l/m/h/std (b) | |
| |----------|----------|----------|----------------------------------------|----------------------------------------|------------------------------| |
| |55000 |25 |12 |0.000 / 1.031 / 5.431 / 0.928 |19.336 / 22.410 / 83.150 / 3.282 |3467 / 3501 / 3538 / 10 | |
| |55000 |25 |14 |0.000 / 0.470 / 2.525 / 0.428 |21.031 / 24.079 / 58.355 / 3.192 |11981 / 12073 / 12166 / 25 | |
| |55000 |25 |16 |0.000 / 0.140 / 0.751 / 0.124 |25.372 / 29.282 / 99.309 / 3.888 |41546 / 41834 / 42083 / 74 | |
| |60000 |25 |12 |0.002 / 1.080 / 5.500 / 0.936 |21.866 / 25.060 / 83.602 / 3.563 |3486 / 3519 / 3552 / 9 | |
| |60000 |25 |14 |0.002 / 0.483 / 2.762 / 0.432 |23.089 / 26.556 / 56.951 / 3.455 |12056 / 12141 / 12233 / 25 | |
| |60000 |25 |16 |0.000 / 0.145 / 1.552 / 0.135 |27.680 / 31.433 / 99.324 / 3.967 |42217 / 42469 / 42714 / 70 | |
| |65000 |25 |12 |0.000 / 1.053 / 5.968 / 0.927 |23.661 / 26.901 / 59.004 / 3.475 |3499 / 3534 / 3565 / 9 | |
| |65000 |25 |14 |0.000 / 0.483 / 2.517 / 0.431 |24.753 / 28.293 / 60.108 / 3.483 |12120 / 12206 / 12299 / 25 | |
| |65000 |25 |16 |0.000 / 0.158 / 1.455 / 0.158 |29.270 / 33.727 / 75.299 / 4.108 |42757 / 43002 / 43219 / 66 | |
| |100000 |25 |14 |0.001 / 0.520 / 2.814 / 0.451 |35.787 / 41.099 / 197.958 / 7.919 |12510 / 12598 / 12685 / 26 | |
| |
| **HLLPMeasurement tool usage** |
| |
| ``` |
| usage: HLLPMeasurement |
| -cd,--chart_delim <CHART_DELIM> Column delimiter for |
| the chart. Default is |
| pipe '|' |
| -cmn,--card_min <CARD_MIN> Lowest cardinality to |
| start running trials |
| from. Default 100 |
| -cmx,--card_max <CARD_MAX> Max cardinality to run |
| trials up to. Default |
| 1000 |
| -cp,--chart_padding <CHART_PADDING> Amount of padding to |
| use for each column. |
| Default 20 |
| -cs,--card_step <CARD_STEP> Quantity to increment |
| cardinality by for |
| each successive |
| measurement up until |
| the cardinality high |
| value. Default 100 |
| -efp,--error_format_percent <ERR_FORMAT_PERCENT> Format error in |
| percent instead of |
| absolute terms. |
| Default true. |
| -ep,--error_percentile <ERR_PERCENTILE> What percentile to |
| calculate between |
| min/max for error. |
| Default is the median |
| (50th percentile) |
| -h,--help This screen |
| -nt,--num_trials <NUM_TRIALS> Number of trials to |
| run. Default 1000 |
| -pmn,--p_min <P_MIN> Minimum sparse |
| precision to get |
| measurements for. |
| Default 4 |
| -pmx,--p_max <P_MAX> Maximum sparse |
| precision to get |
| measurements for. |
| Default 32 |
| -ps,--p_step <P_STEP> Increment precision |
| values by this step |
| amount when running |
| trials. Default 4 |
| -spmn,--sp_min <SP_MIN> Minimum sparse |
| precision to get |
| measurements for. |
| Default 4 |
| -spmx,--sp_max <SP_MAX> Maximum sparse |
| precision to get |
| measurements for. |
| Default 32 |
| -sps,--sp_step <SP_STEP> Increment precision |
| values by this step |
| amount when running |
| trials. Default 4 |
| -spt,--size_percentile <SIZE_PERCENTILE> What percentile to |
| calculate between |
| min/max for size. |
| Default is the median |
| (50th percentile) |
| -tp,--time_percentile <ERR_PERCENTILE> What percentile to |
| calculate between |
| min/max for time. |
| Default is the median |
| (50th percentile) |
| ``` |
