Release Notes - Sentry - Version 1.5.1

** Bug
    * [SENTRY-764] - Update the LICENSE file
    * [SENTRY-775] - Update the Changelog file in 1.5.0 release for some missing jiras

Release Notes - Sentry - Version 1.5.0

** Sub-task
    * [SENTRY-340] - Database implement for "with grant option"
    * [SENTRY-341] - Extend Thrift API for SentryStore to support "with grant option"
    * [SENTRY-342] - Grant check with grant option
    * [SENTRY-343] - Privileges query from database support for "With Grant Option"
    * [SENTRY-345] - Revoke check with grant option
    * [SENTRY-346] - Create new FileAppender used in log4j to keep all the logs
    * [SENTRY-347] - Generate the audit log in Json format
    * [SENTRY-349] - Extend Hive Hook with Grant Option
    * [SENTRY-366] - Update the versions on trunk after branching
    * [SENTRY-367] - Add end to end tests for audit log
    * [SENTRY-370] - Judgement of MSentryPrivilege implies child privileges
    * [SENTRY-377] - Add Hive e2e test for grantOption
    * [SENTRY-389] - Database implementation for column
    * [SENTRY-390] - Extend Thrift API to support column-level privilege
    * [SENTRY-391] - Extend sentrystore query for column level privilege
    * [SENTRY-392] - Authorization for column level security
    * [SENTRY-393] - Grant/Revoke and Show Grant info support for column level privilege
    * [SENTRY-394] - PolicyFile and ConfigImport support for column level privilege
    * [SENTRY-400] - Extending sentry metadata infrastructure to support the generic authorization model
    * [SENTRY-404] - Extending Sentry thrift interface and adding a processor for generic authorization model
    * [SENTRY-405] - Adding a general jdo access layer(sentrystore) to support the new authorization model
    * [SENTRY-406] - Support "WITH GRANT OPTION" for the audit log
    * [SENTRY-426] - Add upgrade scripts for column level privileges
    * [SENTRY-433] - Create common in-memory data structures for efficient storage of Path and Permission information
    * [SENTRY-434] - Create wrapper classes for the Path and Permission data structures to handle Full/Partial updates
    * [SENTRY-435] - Implement SentryAuthorizationProvider namenode plugin
    * [SENTRY-436] - Add Thrift endpoints to handle updates from HMS as well as request for updates from Sentry Namenode plugin
    * [SENTRY-437] - Create UpdateHandler and Forwarder that stores updates in a log to be sent to Sentry NN plugin
    * [SENTRY-438] - Modify SentryMetastorePostEventHandler to post updates to Sentry on HMS Path changes
    * [SENTRY-439] - Add e2e testcases for Sentry HDFS integration
    * [SENTRY-456] - Service discovery for SENTRY high availability
    * [SENTRY-459] - Security mode (Kerberos) support for SENTRY high availability
    * [SENTRY-463] - Refactor SentryServiceClientFactory: change "create SentryPolicyServiceClient" to static
    * [SENTRY-464] - Sentry service register and using InvocationHandler for SentryPolicyServiceClientFactory high availability
    * [SENTRY-479] - Create a Solr client to interactive with the sentry service
    * [SENTRY-481] - Add Solr e2e test for integrating with DB store
    * [SENTRY-503] - Add authentication support to SentryWebserver
    * [SENTRY-519] - Change Hive version to Apache Hive 0.15 for authorization V2
    * [SENTRY-548] - SentryStore support more actions for drop
    * [SENTRY-550] - SentryStore support more actions for rename
    * [SENTRY-595] - [UnitTest] In Kerberos mode, client should run under clientSubject
    * [SENTRY-608] - Add simple authorization support to SentryWebserver
    * [SENTRY-610] - [Unit Test]  TestDbPrivilegeAtTransform (and others) failing with NullPointerException @ HiveServer2.stop(HiveServer2.java:273)
    * [SENTRY-613] - Solr synchronize collection privileges with Sentry when metadata has been changed
    * [SENTRY-625] - Improve test cases in "TestPrivilegesAtColumnScope"
    * [SENTRY-628] - Add ZK based Sentry cache sync framework
    * [SENTRY-629] - Improve test cases in "TestPrivilegesAtTableScope"
    * [SENTRY-630] - Improve test cases in "TestViewPrivileges"
    * [SENTRY-632] - Support Sentry cache sync via ZK
    * [SENTRY-633] - Refactor SentryServiceIntegrationBase to reduce test time
    * [SENTRY-635] - Improve test cases in "TestPerDBConfiguration"
    * [SENTRY-638] - Improve test cases in "TestSentryStore"
    * [SENTRY-640] - Add core model for lily hbase indexer
    * [SENTRY-651] - Add policy engine for lily hbase indexer
    * [SENTRY-653] - Add simple Indexer model object test for sentry-core-model-indexer
    * [SENTRY-655] - Improve test cases in "SentryStoreIntegrationBase"
    * [SENTRY-659] - Periodic cleanup of ZK nodes
    * [SENTRY-680] - Create release branch for 1.5.0
    * [SENTRY-682] - Update changelog.txt, notice.txt, etc... for 1.5.0 release
    * [SENTRY-689] - Verify the patch on 1.4.0


** New Feature
    * [SENTRY-74] - Add column-level privileges for Hive/Impala
    * [SENTRY-331] - Add more granular privileges to the DBModel
    * [SENTRY-355] - Support metadata read privilege enforcement for Metastore pluging
    * [SENTRY-398] - Create the generic authorization model in Sentry
    * [SENTRY-432] - Synchronization of HDFS permissions with Sentry permissions
    * [SENTRY-477] - Sentry service should expose metrics
    * [SENTRY-478] - Solr Sentry plug-in integration with DB store
    * [SENTRY-501] - High availability for the SENTRY service(Zookeeper part)
    * [SENTRY-614] - Add authentication and simple authorization support to SentryWebserver


** Improvement
    * [SENTRY-31] - Enabling audit logs like HS2 impersonation
    * [SENTRY-179] - Generate audit trail for Sentry DBStore service actions
    * [SENTRY-326] - Add support for Hive 0.13
    * [SENTRY-327] - Support auth admin delegation via SQL construct 'with grant option'
    * [SENTRY-358] - Sentry service API to grant prvilege should return newly created privilege object
    * [SENTRY-359] - Support Sentry service API to retrieve applicable privileges for a given authorizable object
    * [SENTRY-417] - Allow all users "Show role GRANT" as long as they belong to that group
    * [SENTRY-420] - TestMovingtoProduction fails on real cluster
    * [SENTRY-422] - The URI object handling needs to be more robust
    * [SENTRY-471] - When running the command "mvn eclipse:eclipse", the sentry shouldn't default download the javadocs and source jars
    * [SENTRY-507] - Ban additional configs in getConfigVal()
    * [SENTRY-517] - MSCK REPAIR TABLE statements are not authorized
    * [SENTRY-572] - Upgrade solr version to 4.10.2
    * [SENTRY-574] - Add Sentry solr handler
    * [SENTRY-578] - Print detail error for TestHDFSIntegration when test failed
    * [SENTRY-598] - Hive binding should support enforcing URI privilege for transforms
    * [SENTRY-617] - Improve grant role to groups
    * [SENTRY-650] - Support drop privilege for truncate table


** Bug
    * [SENTRY-140] - Orphaned privileges should be garbage collected
    * [SENTRY-196] - Error in pom file prevents maven eclipse plugin from running
    * [SENTRY-208] - [flaky tests] Tests in TestSentryServiceIntegration and TestSentryStore often fail with "No current connection"
    * [SENTRY-225] - SimpleFileProviderBackend should not be required to pass sentry configuration object
    * [SENTRY-264] - SentryOnFailureHookContext missing database and table
    * [SENTRY-316] - Users should be allowed to see tables in a db on which the user has authorization without having to switch to the db.
    * [SENTRY-318] - Allow all users "Show GRANT" as long as they have the grant on that role.
    * [SENTRY-324] - Sentry need to be refactored to base on Hive 0.13 privilegeType class
    * [SENTRY-325] - Sentry needs to be refactored to based on HIVE 0.13 PreAddPartitionEvent API
    * [SENTRY-328] - Need to update DataNucleus version to support proper SQL generation for DB2
    * [SENTRY-334] - Handle errors more user firendly in db store when objects are not present.
    * [SENTRY-338] - Sentry policy import tool adds non-compatible comments to grant privilege statements
    * [SENTRY-339] - Remove PrivilegeName column and constructPrivilegeName() function
    * [SENTRY-344] - Fix pre commit build( TestSentryStore - Too many open files)
    * [SENTRY-350] - org.apache.sentry.tests.e2e.metastore.TestMetastoreEndToEnd failure caused by new table parameter (COLUMN_STATS_ACCURATE etc)
    * [SENTRY-357] - Not able to read policy files on HDFS for Solr
    * [SENTRY-362] - When sentry integrate into solr, the create instance of backend needs configure parameters from solrAuthzConf not hadoopConf
    * [SENTRY-368] - Remove unused field in SentryPolicyServiceClient.java
    * [SENTRY-373] - Trivial fix after Sentry-326
    * [SENTRY-375] - Sentry + Hive 0.13 integration test failure at org.apache.sentry.tests.e2e.hive.TestConfigTool
    * [SENTRY-376] - Sentry + Hive 0.13 integration test failure TestPrivilegesAtFunctionScope
    * [SENTRY-380] - Clean up some grantorPrincipal semantics
    * [SENTRY-381] - Define jackson.version
    * [SENTRY-388] - Solr Binding initKerberos should use supplied Configuration
    * [SENTRY-396] - The logic of Thrift multiplexedProcessor registers mutil processor isn't correct
    * [SENTRY-407] - Add schema upgrade script to handle schema changes in 1.5
    * [SENTRY-408] - The URI permission should support more filesystem prefixes
    * [SENTRY-409] - Do not print stack traces for SentryUserExceptions in Hive
    * [SENTRY-411] - Alter table set location does not strictly check for URI privileges
    * [SENTRY-412] - Sentry script should support an option to print product version
    * [SENTRY-413] - Fix alter table index rebuild
    * [SENTRY-414] - Alter table rename should require database level privileges
    * [SENTRY-416] - TestConfigTool.testQueryPermissions regressed
    * [SENTRY-421] - Metastore binding is not constructing in fully qualified URI sentry recognizable format
    * [SENTRY-423] - Hive command "SHOW TABLE EXTENDED LIKE... " failed with NPE
    * [SENTRY-424] - Rat check occasionally failing after derby upgrade
    * [SENTRY-425] - Reduce logging verbosity in SentryPolicyServiceClient when creating new connections
    * [SENTRY-428] - Sentry service should periodically renew the server kerberos ticket
    * [SENTRY-429] - When SENTRY Service using free port, the port should set to configuration.
    * [SENTRY-430] - Sentry Service does not use correct classpath when HIVE_HOME environment var is defined
    * [SENTRY-431] - Sentry db provider client should attempt to refresh kerberos ticket before connection
    * [SENTRY-441] - Improve the message for SemanticException
    * [SENTRY-442] - Sentry 331 follow on
    * [SENTRY-443] - "Show roles" regressed after Sentry-417
    * [SENTRY-444] - Update the schema upgrade scripts per the grantor principal changes
    * [SENTRY-445] - WITH GRANT OPTION does not allow delegated user to grant less permissive privileges
    * [SENTRY-446] - Missing comma in mysql 1.5 script
    * [SENTRY-447] - Fix thrift generated code related to grantor principal cleanup
    * [SENTRY-449] - Create testcases for Hive permanent UDF
    * [SENTRY-450] - Add new Hive UDFs to the whitelist
    * [SENTRY-451] - CAST is still broken after SENTRY-118 patch
    * [SENTRY-452] - Uri tests failing on real cluster
    * [SENTRY-454] - Hive metadata changes syncup with Sentry store should not run in error cases
    * [SENTRY-455] - Fixed Unit Tests: TestDbOperations#testIndexTable
    * [SENTRY-465] - Fix an upgrade issue and an Invalid sql script file name issue of derby
    * [SENTRY-466] - Return failure code when SentryClient was not successfully instantiated
    * [SENTRY-468] - Rename the oracle and postgre upgrade scripts to <jira-id>.<db-vendoer>.sql format
    * [SENTRY-469] - TListSentryPrivilegesByAuthRequest API should support impersonation
    * [SENTRY-470] - When the parameter of hive.sentry.server is uppercase string, the command "use default" will cause an error in Hive Server2 side
    * [SENTRY-472] - Hive binding should validate URI privileges on permenant function resource URI
    * [SENTRY-475] - SHOW GRANT ROLE from Hive always report with grant option as false
    * [SENTRY-482] - Fix typo in Sentry audit logs
    * [SENTRY-483] - The schema upgrade script for oracle missing terminating char for nested script
    * [SENTRY-484] - Sentry Service has does not audit ip address in secure environments
    * [SENTRY-487] - TestPrivilegesAtFunctionScope fails on the real cluster
    * [SENTRY-488] - Sentry list_sentry_privileges_by_authorizable API does not filter out roles/privileges for some cases.
    * [SENTRY-489] - Sentry DB upgrade fails on Oracle with "ORA-00905: missing keyword"
    * [SENTRY-494] - UNLOCK TABLE is not allowed
    * [SENTRY-496] - Sentry 1.5 postgres upgrade script has contains incorrect upgrade file name
    * [SENTRY-499] - The metastore client wrapper should load the authorization binding as HiveHook.HiveServer2
    * [SENTRY-500] - 1.4 to 1.5 upgrade needs to handle empty strings with __NULL__
    * [SENTRY-509] - upgrade HIVE version to 0.13.1-cdh5.3.0-SNAPSHOT in SENTRY
    * [SENTRY-511] - Always enable metric collection and do not fail when all metric reporters are disabled
    * [SENTRY-512] - Revert back to maven.compiler to java 6
    * [SENTRY-513] - Sentry web service may not be stopped completely
    * [SENTRY-523] - Add maven-thrift-plugin back into provider-db pom.xml
    * [SENTRY-525] - [Unit Test]org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationtestEnd2End test fails
    * [SENTRY-526] - Duplicate grant same but case sensitive privilges will throw exception
    * [SENTRY-528] - Dependent on multiple versions of servlet-api jars lead to throw an SecurityException when running solr e2e test in eclipse
    * [SENTRY-529] - [Unit Test]org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationtestEnd2End test fails after SENTRY-74
    * [SENTRY-533] - [Unit Test] TestHDFSIntegration.testEnd2End is failing due to NPE in Sentry Service
    * [SENTRY-534] - TestRuntimeMetadataRetrieval fails intermittently
    * [SENTRY-536] - Disable TestDPrivilegesAtFunctionScope from the cluster run profile
    * [SENTRY-540] - Fix Sentry test validating special chars in username due to HIVE-8916
    * [SENTRY-541] - Seperate udfuri privilege from anyPrivilege model
    * [SENTRY-543] - Sentry Store permission dump incorrect after recursive revoke
    * [SENTRY-544] - Do not add non HDFS path updates in Hive meta store Sentry plugin for HDFS sync
    * [SENTRY-545] - Disable Privilege Cleanup Thread
    * [SENTRY-547] - Drop table may dead lock, getChildPrivileges should in one transaction with revoke
    * [SENTRY-549] - SentryStore should support more actions
    * [SENTRY-552] - Sentry Store recursive revoke of privilege levels < ALL does not properly downgrade child privileges
    * [SENTRY-553] - Privilge implies failed if parent is server privilge and child privilge is URI privilge
    * [SENTRY-555] - Ensure groupName returned for dir objects within prefix but not associated with an authz object is not null
    * [SENTRY-556] - Remove NPE logging when Sentry Service is not reachable
    * [SENTRY-557] - Handle Situation when Metastore restarts Listeners thereby resetting the sequenceCounter
    * [SENTRY-558] - Make Metastore sync time period configurable
    * [SENTRY-559] - Allow prefix paths to be associated with authorizable objects
    * [SENTRY-560] - Sentry HDFS Syncup applies duplicate ACLs for the same scope, group and type
    * [SENTRY-563] - The interface listPrivilegesByRoleName may throw thrift exception if Authorizable is empty
    * [SENTRY-564] - Sentry metastore upgrade order is computed incorrectly
    * [SENTRY-566] - Sentry source assembly doesn't include sentry-hdfs module
    * [SENTRY-571] - Enable TestPrivilegesAtFunctionScope
    * [SENTRY-573] - Fix NPE caused when rename op is applied on authzObject with no explicit permissions
    * [SENTRY-575] - Table GRANTS should not Override Database GRANT in the Sentry HDFS plugin
    * [SENTRY-576] - Enable unit test TestSentryServiceIntegration.testListByAuthDB
    * [SENTRY-577] - Orphan cleaner should remove privilege is not ALL, SELECT or INSERT
    * [SENTRY-579] - Clean duplicate declaration of dependences
    * [SENTRY-586] - Remove remaining cdh hive dependence in pom.xml
    * [SENTRY-591] - create table should have output privilege in DB scope
    * [SENTRY-594] - Alter database should check output privilege instead of input
    * [SENTRY-599] - Sentry service may report incorrect status when service is restarting
    * [SENTRY-602] - Pre commit build script should update the snapshot dependencies
    * [SENTRY-604] - Clean duplicate dependences in poms
    * [SENTRY-609] - [Unit Test] Many tests failing with error "The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwxr-xr-x"
    * [SENTRY-615] - [Unit Test] Fix TestSentryWebServerWithKerberos#testPingWithUnauthorizedUser failed in Jenkins occasionally
    * [SENTRY-636] - Inaccurate log level in HiveServerFactory.java
    * [SENTRY-648] - Add e2e tests for Sentry HA
    * [SENTRY-652] - Sentry fails to parse spaces when HDFS ACL sync enabled
    * [SENTRY-654] - Calls to append_partition fail when Sentry is enabled
    * [SENTRY-658] - Connection leak in Hive biding with Sentry service
    * [SENTRY-660] - Support client principal and keytab configuration properties for Sentry HA to work with secure zookeeper
    * [SENTRY-664] - After Namenode is restarted, Path updates remain unsynched
    * [SENTRY-665] - PathsUpdate.parsePath needs to handle special characters
    * [SENTRY-669] - Drop database Hive statement removes the DB privileges even if the operation fails
    * [SENTRY-670] - Fix the Sentry build to remove snapshot and non apache dependencies
    * [SENTRY-673] - Keep consistent version of hadoop dependence
    * [SENTRY-674] - Update Apache Hive dependency to new release 1.1.0
    * [SENTRY-676] - Address Sentry HA issues in secure cluster
    * [SENTRY-690] - Remove SENTRY-645 patch from 1.5.0 release
    * [SENTRY-691] - upgrade schema tool: when upgrade oracle backed db from 1.4.0 - 1.5.0, gets syntax error
    * [SENTRY-693] - The generic model has not successfully revoke part of privileges from existed ALL privilege
    * [SENTRY-694] - Sentry leaving orphan rows in SENTRY_DB_PRIVILEGE


** Task
    * [SENTRY-354] - Test for update.distrib phase overriding
    * [SENTRY-382] - Build documentation
    * [SENTRY-415] - Add API to Sentry Service that allows clients to read the service's config values
    * [SENTRY-624] - Add upgrade scripts for general privilege model


** Test
    * [SENTRY-47] - Tests need to clean up the databases and tables it creates
    * [SENTRY-383] - Add TestPrivilegeWithGrantOption to cluster test profile
    * [SENTRY-688] - Verify the patch on 1.4.0