| commit | 3599dfd2690023db6a7af66512b3e44759eb34a1 | [log] [tgz] |
|---|---|---|
| author | Paul Rogers <par0328@yahoo.com> | Tue Jun 18 19:26:16 2019 -0700 |
| committer | Arina Ielchiieva <arina.yelchiyeva@gmail.com> | Mon Jul 15 13:27:28 2019 +0300 |
| tree | 165f3ac9bd1a64a7077159bb4219d5da06e4e3f9 | |
| parent | 2224ee1014cfdcc2b76f1a8290171ca63db76a53 [diff] |
DRILL-6951: Merge row set based mock data source The mock data source is used in several tests to generate a large volume of sample data, such as when testing spilling. The mock data source also lets us try new plugin featues in a very simple context. During the development of the row set framework, the mock data source was converted to use the new framework to verify functionality. This commit upgrades the mock data source with that work. The work changes non of the functionality. It does, however, improve memory usage. Batchs are limited, by default, to 10 MB in size. The row set framework minimizes internal fragmentation in the largest vector. (Previously, internal fragmentation averaged 25% but could be as high as 50%.) As it turns out, the hash aggregate tests depended on the internal fragmentation: without it, the hash agg no longer spilled for the same row count. Adjusted the generated row counts to recreate a data volume that caused spilling. One test in particular always failed due to assertions in the hash agg code. These seem true bugs and are described in DRILL-7301. After multiple failed attempts to get the test to work, it ws disabled until DRILL-7301 is fixed. Added a new unit test to sanity check the mock data source. (No test already existed for this functionality except as verified via other unit tests.)
Apache Drill is a distributed MPP query layer that supports SQL and alternative query languages against NoSQL and Hadoop data storage systems. It was inspired in part by Google's Dremel.
Please read Environment.md for setting up and running Apache Drill. For complete developer documentation see DevDocs.md
Please see the Apache Drill Website or the Apache Drill Documentation for more information including:
Apache Drill is an Apache Foundation project and is seeking all types of users and contributions. Please say hello on the Apache Drill mailing list.You can also join our Google Hangouts or join our Slack Channel if you need help with using or developing Apache Drill. (More information can be found on Apache Drill website).
This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.
The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code. The following provides more details on the included cryptographic software: Java SE Security packages are used to provide support for authentication, authorization and secure sockets communication. The Jetty Web Server is used to provide communication via HTTPS. The Cyrus SASL libraries, Kerberos Libraries and OpenSSL Libraries are used to provide SASL based authentication and SSL communication.