DRILL-7436: Fix record count, vector structure issues in several operators

Adds additional vector checks to the BatchValidator.

Enables checking for the following operators:

* FilterRecordBatch
* PartitionLimitRecordBatch
* UnnestRecordBatch
* HashAggBatch
* RemovingRecordBatch

Fixes vector count issues for each of these.

Fixes empty-batch (record count = 0) handling in several of the
above operators. Added a method to VectorContainer to correctly
create an empty batch. (An empty batch, counter-intuitively,
needs vectors allocated to hold the 0 value in the first
position of each offset vector.)

Disables verbose logging for MongoDB tests. Details are written to
the log rather than the console.

Disables two invalid Mongo tests. See DRILL-7428.

Adjusts the expression tree materializer to not add the LATE type
to Union vectors. (See DRILL-7435.)

Ensures that Union vectors contain valid vectors for each subtype.
The present fix is a work-around, see DRILL-7434 for a better
long-term fix.

Cleans up code formatting and other minor issues in each file touched
during the fixes in this PR.
37 files changed
tree: 3eed48a642d2ba5f9761c39ecf77e2822e8e0fc5
  1. .circleci/
  2. .mvn/
  3. common/
  4. contrib/
  5. distribution/
  6. docs/
  7. drill-shaded/
  8. drill-yarn/
  9. exec/
  10. logical/
  11. metastore/
  12. protocol/
  13. sample-data/
  14. src/
  15. tools/
  16. .gitignore
  17. .travis.yml
  18. header
  19. KEYS
  20. LICENSE
  21. NOTICE
  22. pom.xml
  23. README.md
README.md

Apache Drill

Build Status Artifact License

Apache Drill is a distributed MPP query layer that supports SQL and alternative query languages against NoSQL and Hadoop data storage systems. It was inspired in part by Google's Dremel.

Developers

Please read Environment.md for setting up and running Apache Drill. For complete developer documentation see DevDocs.md

More Information

Please see the Apache Drill Website or the Apache Drill Documentation for more information including:

  • Remote Execution Installation Instructions
  • Information about how to submit logical and distributed physical plans
  • More example queries and sample data
  • Find out ways to be involved or discuss Drill

Join the community!

Apache Drill is an Apache Foundation project and is seeking all types of users and contributions. Please say hello on the Apache Drill mailing list.You can also join our Google Hangouts or join our Slack Channel if you need help with using or developing Apache Drill. (More information can be found on Apache Drill website).

Export Control

This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.
The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code. The following provides more details on the included cryptographic software: Java SE Security packages are used to provide support for authentication, authorization and secure sockets communication. The Jetty Web Server is used to provide communication via HTTPS. The Cyrus SASL libraries, Kerberos Libraries and OpenSSL Libraries are used to provide SASL based authentication and SSL communication.