DRILL-8155: Introduce New Plugin Authentication Modes (#2516)

* Do not set the read-only hint on JDBC connections.

* Outline of different auth modes in storage-jdbc.

In this commit, a new `authMode` storage config supporting three new auth modes
is defined: shared user (default), user translation (user is translated to some
other user from the external storage) and impersonation (the external storage
and JDBC driver provide support for impersonating the Drill query user).

The JdbcStoragePlugin is enhanced to be able to work with a lookup table of
connection pools, where a pool is dedicated to each query user except when
in shared user auth mode.

Planning and execution time APIs are also enhanced to transmit a user
credentials object for the query user, instead of just its username.
This allows for the expansion of the UserCredentials protobuf type to
include some optional extra credentials, e.g. in an array of byte arrays.
These credentials may be relevant in the user tranlsation mode when a
credential provider must be accessed in order to obtain the creds to be used
for the external system.

* Rebased to current master and build fixes

* Build works, cred stuff added

* Credentials being saved and pushed down to storage plugin

* UI now closing properly

* User Translation working for HTTP plugin

* HTTP unit tests passing

* WIP

* Fixed import

* User Credentials now being stored in credential provider

* Working

* Fixed TPCH Unit Tests

* Fix CredProvider SerDe Test

* Added unit tests for JDBC

* Code cleanup

* Fix LGTM alerts

* Correct username now populating Group Scan

* Username to Subscan

* Remove PerUserUsernamePasswordCredentials class.

* Remove getUserCredentials from CredentialsProvider.

* Planning errors fixed

* Removed unused imports

* Fixed minor issues

* Unit test fixes

* WIP.

* Fix CodeQL Alert

* Ignore LGTM False Positive

* Fix tainted string LGTM alert

* Revert LGTM Comment

* Addressed review comments

* Use fixed size Guava caches in JDBC convetion and dialect factories.

These replace Maps with no size limit that might have grown without
bound. LRU eviction begins when the cache size limit is reached.

* Add a TTL to the JDBC dialect and convention caches.

Co-authored-by: James Turton <james@somecomputer.xyz>
87 files changed
tree: 8d06039289896c6ffaf9f5195a55fffacb73222d
  1. .github/
  2. .mvn/
  3. common/
  4. contrib/
  5. dev-support/
  6. distribution/
  7. docs/
  8. drill-shaded/
  9. drill-yarn/
  10. exec/
  11. hooks/
  12. logical/
  13. metastore/
  14. protocol/
  15. sample-data/
  16. src/
  17. tools/
  18. .asf.yaml
  19. .dockerignore
  20. .editorconfig
  21. .gitignore
  22. .lgtm.yml
  23. .travis.yml
  24. Dockerfile
  25. header
  26. KEYS
  27. LICENSE
  28. NOTICE
  29. pom.xml
  30. README.md
  31. start-build-env.sh
README.md

Apache Drill

Build Status Artifact License Stack Overflow Join Drill Slack

Apache Drill is a distributed MPP query layer that supports SQL and alternative query languages against NoSQL and Hadoop data storage systems. It was inspired in part by Google's Dremel.

Developers

Please read Environment.md for setting up and running Apache Drill. For complete developer documentation see DevDocs.md.

More Information

Please see the Apache Drill Website or the Apache Drill Documentation for more information including:

  • Remote Execution Installation Instructions
  • Running Drill on Docker instructions
  • Information about how to submit logical and distributed physical plans
  • More example queries and sample data
  • Find out ways to be involved or discuss Drill

Join the community!

Apache Drill is an Apache Foundation project and is seeking all types of users and contributions. Please say hello on the Apache Drill mailing list.You can also join our Google Hangouts or join our Slack Channel if you need help with using or developing Apache Drill (more information can be found on Apache Drill website).

Export Control

This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See http://www.wassenaar.org/ for more information.
The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code. The following provides more details on the included cryptographic software: Java SE Security packages are used to provide support for authentication, authorization and secure sockets communication. The Jetty Web Server is used to provide communication via HTTPS. The Cyrus SASL libraries, Kerberos Libraries and OpenSSL Libraries are used to provide SASL based authentication and SSL communication.