blob: 5d96434653aa292dfd025c7baa876013e95a5a15 [file] [log] [blame]
Release Notes - Sentry - Version 1.5.0
** Sub-task
* [SENTRY-340] - Database implement for "with grant option"
* [SENTRY-341] - Extend Thrift API for SentryStore to support "with grant option"
* [SENTRY-342] - Grant check with grant option
* [SENTRY-343] - Privileges query from database support for "With Grant Option"
* [SENTRY-345] - Revoke check with grant option
* [SENTRY-346] - Create new FileAppender used in log4j to keep all the logs
* [SENTRY-347] - Generate the audit log in Json format
* [SENTRY-349] - Extend Hive Hook with Grant Option
* [SENTRY-366] - Update the versions on trunk after branching
* [SENTRY-367] - Add end to end tests for audit log
* [SENTRY-370] - Judgement of MSentryPrivilege implies child privileges
* [SENTRY-377] - Add Hive e2e test for grantOption
* [SENTRY-389] - Database implementation for column
* [SENTRY-390] - Extend Thrift API to support column-level privilege
* [SENTRY-391] - Extend sentrystore query for column level privilege
* [SENTRY-392] - Authorization for column level security
* [SENTRY-393] - Grant/Revoke and Show Grant info support for column level privilege
* [SENTRY-394] - PolicyFile and ConfigImport support for column level privilege
* [SENTRY-400] - Extending sentry metadata infrastructure to support the generic authorization model
* [SENTRY-404] - Extending Sentry thrift interface and adding a processor for generic authorization model
* [SENTRY-405] - Adding a general jdo access layer(sentrystore) to support the new authorization model
* [SENTRY-406] - Support "WITH GRANT OPTION" for the audit log
* [SENTRY-426] - Add upgrade scripts for column level privileges
* [SENTRY-456] - Service discovery for SENTRY high availability
* [SENTRY-459] - Security mode (Kerberos) support for SENTRY high availability
* [SENTRY-463] - Refactor SentryServiceClientFactory: change "create SentryPolicyServiceClient" to static
* [SENTRY-464] - Sentry service register and using InvocationHandler for SentryPolicyServiceClientFactory high availability
* [SENTRY-479] - Create a Solr client to interactive with the sentry service
* [SENTRY-481] - Add Solr e2e test for integrating with DB store
* [SENTRY-503] - Add authentication support to SentryWebserver
* [SENTRY-519] - Change Hive version to Apache Hive 0.15 for authorization V2
* [SENTRY-548] - SentryStore support more actions for drop
* [SENTRY-550] - SentryStore support more actions for rename
* [SENTRY-595] - [UnitTest] In Kerberos mode, client should run under clientSubject
* [SENTRY-608] - Add simple authorization support to SentryWebserver
* [SENTRY-610] - [Unit Test] TestDbPrivilegeAtTransform (and others) failing with NullPointerException @ HiveServer2.stop(HiveServer2.java:273)
* [SENTRY-613] - Solr synchronize collection privileges with Sentry when metadata has been changed
* [SENTRY-625] - Improve test cases in "TestPrivilegesAtColumnScope"
* [SENTRY-628] - Add ZK based Sentry cache sync framework
* [SENTRY-629] - Improve test cases in "TestPrivilegesAtTableScope"
* [SENTRY-630] - Improve test cases in "TestViewPrivileges"
* [SENTRY-632] - Support Sentry cache sync via ZK
* [SENTRY-633] - Refactor SentryServiceIntegrationBase to reduce test time
* [SENTRY-635] - Improve test cases in "TestPerDBConfiguration"
* [SENTRY-638] - Improve test cases in "TestSentryStore"
* [SENTRY-640] - Add core model for lily hbase indexer
* [SENTRY-651] - Add policy engine for lily hbase indexer
* [SENTRY-653] - Add simple Indexer model object test for sentry-core-model-indexer
* [SENTRY-655] - Improve test cases in "SentryStoreIntegrationBase"
* [SENTRY-659] - Periodic cleanup of ZK nodes
* [SENTRY-680] - Create release branch for 1.5.0
* [SENTRY-682] - Update changelog.txt, notice.txt, etc... for 1.5.0 release
* [SENTRY-689] - Verify the patch on 1.4.0
** New Feature
* [SENTRY-74] - Add column-level privileges for Hive/Impala
* [SENTRY-331] - Add more granular privileges to the DBModel
* [SENTRY-355] - Support metadata read privilege enforcement for Metastore pluging
* [SENTRY-398] - Create the generic authorization model in Sentry
* [SENTRY-477] - Sentry service should expose metrics
* [SENTRY-478] - Solr Sentry plug-in integration with DB store
* [SENTRY-501] - High availability for the SENTRY service(Zookeeper part)
* [SENTRY-614] - Add authentication and simple authorization support to SentryWebserver
** Improvement
* [SENTRY-31] - Enabling audit logs like HS2 impersonation
* [SENTRY-179] - Generate audit trail for Sentry DBStore service actions
* [SENTRY-326] - Add support for Hive 0.13
* [SENTRY-327] - Support auth admin delegation via SQL construct 'with grant option'
* [SENTRY-358] - Sentry service API to grant prvilege should return newly created privilege object
* [SENTRY-359] - Support Sentry service API to retrieve applicable privileges for a given authorizable object
* [SENTRY-417] - Allow all users "Show role GRANT" as long as they belong to that group
* [SENTRY-420] - TestMovingtoProduction fails on real cluster
* [SENTRY-422] - The URI object handling needs to be more robust
* [SENTRY-471] - When running the command "mvn eclipse:eclipse", the sentry shouldn't default download the javadocs and source jars
* [SENTRY-507] - Ban additional configs in getConfigVal()
* [SENTRY-517] - MSCK REPAIR TABLE statements are not authorized
* [SENTRY-572] - Upgrade solr version to 4.10.2
* [SENTRY-574] - Add Sentry solr handler
* [SENTRY-578] - Print detail error for TestHDFSIntegration when test failed
* [SENTRY-598] - Hive binding should support enforcing URI privilege for transforms
* [SENTRY-617] - Improve grant role to groups
* [SENTRY-650] - Support drop privilege for truncate table
** Bug
* [SENTRY-140] - Orphaned privileges should be garbage collected
* [SENTRY-196] - Error in pom file prevents maven eclipse plugin from running
* [SENTRY-208] - [flaky tests] Tests in TestSentryServiceIntegration and TestSentryStore often fail with "No current connection"
* [SENTRY-225] - SimpleFileProviderBackend should not be required to pass sentry configuration object
* [SENTRY-264] - SentryOnFailureHookContext missing database and table
* [SENTRY-316] - Users should be allowed to see tables in a db on which the user has authorization without having to switch to the db.
* [SENTRY-318] - Allow all users "Show GRANT" as long as they have the grant on that role.
* [SENTRY-324] - Sentry need to be refactored to base on Hive 0.13 privilegeType class
* [SENTRY-325] - Sentry needs to be refactored to based on HIVE 0.13 PreAddPartitionEvent API
* [SENTRY-328] - Need to update DataNucleus version to support proper SQL generation for DB2
* [SENTRY-334] - Handle errors more user firendly in db store when objects are not present.
* [SENTRY-338] - Sentry policy import tool adds non-compatible comments to grant privilege statements
* [SENTRY-339] - Remove PrivilegeName column and constructPrivilegeName() function
* [SENTRY-344] - Fix pre commit build( TestSentryStore - Too many open files)
* [SENTRY-350] - org.apache.sentry.tests.e2e.metastore.TestMetastoreEndToEnd failure caused by new table parameter (COLUMN_STATS_ACCURATE etc)
* [SENTRY-357] - Not able to read policy files on HDFS for Solr
* [SENTRY-362] - When sentry integrate into solr, the create instance of backend needs configure parameters from solrAuthzConf not hadoopConf
* [SENTRY-368] - Remove unused field in SentryPolicyServiceClient.java
* [SENTRY-373] - Trivial fix after Sentry-326
* [SENTRY-375] - Sentry + Hive 0.13 integration test failure at org.apache.sentry.tests.e2e.hive.TestConfigTool
* [SENTRY-376] - Sentry + Hive 0.13 integration test failure TestPrivilegesAtFunctionScope
* [SENTRY-380] - Clean up some grantorPrincipal semantics
* [SENTRY-381] - Define jackson.version
* [SENTRY-388] - Solr Binding initKerberos should use supplied Configuration
* [SENTRY-396] - The logic of Thrift multiplexedProcessor registers mutil processor isn't correct
* [SENTRY-407] - Add schema upgrade script to handle schema changes in 1.5
* [SENTRY-408] - The URI permission should support more filesystem prefixes
* [SENTRY-409] - Do not print stack traces for SentryUserExceptions in Hive
* [SENTRY-411] - Alter table set location does not strictly check for URI privileges
* [SENTRY-412] - Sentry script should support an option to print product version
* [SENTRY-413] - Fix alter table index rebuild
* [SENTRY-414] - Alter table rename should require database level privileges
* [SENTRY-416] - TestConfigTool.testQueryPermissions regressed
* [SENTRY-421] - Metastore binding is not constructing in fully qualified URI sentry recognizable format
* [SENTRY-423] - Hive command "SHOW TABLE EXTENDED LIKE... " failed with NPE
* [SENTRY-424] - Rat check occasionally failing after derby upgrade
* [SENTRY-425] - Reduce logging verbosity in SentryPolicyServiceClient when creating new connections
* [SENTRY-428] - Sentry service should periodically renew the server kerberos ticket
* [SENTRY-429] - When SENTRY Service using free port, the port should set to configuration.
* [SENTRY-430] - Sentry Service does not use correct classpath when HIVE_HOME environment var is defined
* [SENTRY-431] - Sentry db provider client should attempt to refresh kerberos ticket before connection
* [SENTRY-441] - Improve the message for SemanticException
* [SENTRY-442] - Sentry 331 follow on
* [SENTRY-443] - "Show roles" regressed after Sentry-417
* [SENTRY-444] - Update the schema upgrade scripts per the grantor principal changes
* [SENTRY-445] - WITH GRANT OPTION does not allow delegated user to grant less permissive privileges
* [SENTRY-446] - Missing comma in mysql 1.5 script
* [SENTRY-447] - Fix thrift generated code related to grantor principal cleanup
* [SENTRY-449] - Create testcases for Hive permanent UDF
* [SENTRY-450] - Add new Hive UDFs to the whitelist
* [SENTRY-451] - CAST is still broken after SENTRY-118 patch
* [SENTRY-452] - Uri tests failing on real cluster
* [SENTRY-454] - Hive metadata changes syncup with Sentry store should not run in error cases
* [SENTRY-455] - Fixed Unit Tests: TestDbOperations#testIndexTable
* [SENTRY-465] - Fix an upgrade issue and an Invalid sql script file name issue of derby
* [SENTRY-466] - Return failure code when SentryClient was not successfully instantiated
* [SENTRY-468] - Rename the oracle and postgre upgrade scripts to <jira-id>.<db-vendoer>.sql format
* [SENTRY-469] - TListSentryPrivilegesByAuthRequest API should support impersonation
* [SENTRY-470] - When the parameter of hive.sentry.server is uppercase string, the command "use default" will cause an error in Hive Server2 side
* [SENTRY-472] - Hive binding should validate URI privileges on permenant function resource URI
* [SENTRY-475] - SHOW GRANT ROLE from Hive always report with grant option as false
* [SENTRY-482] - Fix typo in Sentry audit logs
* [SENTRY-483] - The schema upgrade script for oracle missing terminating char for nested script
* [SENTRY-484] - Sentry Service has does not audit ip address in secure environments
* [SENTRY-487] - TestPrivilegesAtFunctionScope fails on the real cluster
* [SENTRY-488] - Sentry list_sentry_privileges_by_authorizable API does not filter out roles/privileges for some cases.
* [SENTRY-489] - Sentry DB upgrade fails on Oracle with "ORA-00905: missing keyword"
* [SENTRY-494] - UNLOCK TABLE is not allowed
* [SENTRY-496] - Sentry 1.5 postgres upgrade script has contains incorrect upgrade file name
* [SENTRY-499] - The metastore client wrapper should load the authorization binding as HiveHook.HiveServer2
* [SENTRY-500] - 1.4 to 1.5 upgrade needs to handle empty strings with __NULL__
* [SENTRY-509] - upgrade HIVE version to 0.13.1-cdh5.3.0-SNAPSHOT in SENTRY
* [SENTRY-511] - Always enable metric collection and do not fail when all metric reporters are disabled
* [SENTRY-512] - Revert back to maven.compiler to java 6
* [SENTRY-513] - Sentry web service may not be stopped completely
* [SENTRY-523] - Add maven-thrift-plugin back into provider-db pom.xml
* [SENTRY-525] - [Unit Test]org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationtestEnd2End test fails
* [SENTRY-526] - Duplicate grant same but case sensitive privilges will throw exception
* [SENTRY-528] - Dependent on multiple versions of servlet-api jars lead to throw an SecurityException when running solr e2e test in eclipse
* [SENTRY-529] - [Unit Test]org.apache.sentry.tests.e2e.hdfs.TestHDFSIntegrationtestEnd2End test fails after SENTRY-74
* [SENTRY-533] - [Unit Test] TestHDFSIntegration.testEnd2End is failing due to NPE in Sentry Service
* [SENTRY-534] - TestRuntimeMetadataRetrieval fails intermittently
* [SENTRY-536] - Disable TestDPrivilegesAtFunctionScope from the cluster run profile
* [SENTRY-540] - Fix Sentry test validating special chars in username due to HIVE-8916
* [SENTRY-541] - Seperate udfuri privilege from anyPrivilege model
* [SENTRY-543] - Sentry Store permission dump incorrect after recursive revoke
* [SENTRY-544] - Do not add non HDFS path updates in Hive meta store Sentry plugin for HDFS sync
* [SENTRY-545] - Disable Privilege Cleanup Thread
* [SENTRY-547] - Drop table may dead lock, getChildPrivileges should in one transaction with revoke
* [SENTRY-549] - SentryStore should support more actions
* [SENTRY-552] - Sentry Store recursive revoke of privilege levels < ALL does not properly downgrade child privileges
* [SENTRY-553] - Privilge implies failed if parent is server privilge and child privilge is URI privilge
* [SENTRY-555] - Ensure groupName returned for dir objects within prefix but not associated with an authz object is not null
* [SENTRY-556] - Remove NPE logging when Sentry Service is not reachable
* [SENTRY-557] - Handle Situation when Metastore restarts Listeners thereby resetting the sequenceCounter
* [SENTRY-558] - Make Metastore sync time period configurable
* [SENTRY-559] - Allow prefix paths to be associated with authorizable objects
* [SENTRY-560] - Sentry HDFS Syncup applies duplicate ACLs for the same scope, group and type
* [SENTRY-563] - The interface listPrivilegesByRoleName may throw thrift exception if Authorizable is empty
* [SENTRY-564] - Sentry metastore upgrade order is computed incorrectly
* [SENTRY-566] - Sentry source assembly doesn't include sentry-hdfs module
* [SENTRY-571] - Enable TestPrivilegesAtFunctionScope
* [SENTRY-573] - Fix NPE caused when rename op is applied on authzObject with no explicit permissions
* [SENTRY-575] - Table GRANTS should not Override Database GRANT in the Sentry HDFS plugin
* [SENTRY-576] - Enable unit test TestSentryServiceIntegration.testListByAuthDB
* [SENTRY-577] - Orphan cleaner should remove privilege is not ALL, SELECT or INSERT
* [SENTRY-579] - Clean duplicate declaration of dependences
* [SENTRY-586] - Remove remaining cdh hive dependence in pom.xml
* [SENTRY-591] - create table should have output privilege in DB scope
* [SENTRY-594] - Alter database should check output privilege instead of input
* [SENTRY-599] - Sentry service may report incorrect status when service is restarting
* [SENTRY-602] - Pre commit build script should update the snapshot dependencies
* [SENTRY-604] - Clean duplicate dependences in poms
* [SENTRY-609] - [Unit Test] Many tests failing with error "The root scratch dir: /tmp/hive on HDFS should be writable. Current permissions are: rwxr-xr-x"
* [SENTRY-615] - [Unit Test] Fix TestSentryWebServerWithKerberos#testPingWithUnauthorizedUser failed in Jenkins occasionally
* [SENTRY-636] - Inaccurate log level in HiveServerFactory.java
* [SENTRY-648] - Add e2e tests for Sentry HA
* [SENTRY-652] - Sentry fails to parse spaces when HDFS ACL sync enabled
* [SENTRY-654] - Calls to append_partition fail when Sentry is enabled
* [SENTRY-658] - Connection leak in Hive biding with Sentry service
* [SENTRY-660] - Support client principal and keytab configuration properties for Sentry HA to work with secure zookeeper
* [SENTRY-664] - After Namenode is restarted, Path updates remain unsynched
* [SENTRY-665] - PathsUpdate.parsePath needs to handle special characters
* [SENTRY-669] - Drop database Hive statement removes the DB privileges even if the operation fails
* [SENTRY-670] - Fix the Sentry build to remove snapshot and non apache dependencies
* [SENTRY-673] - Keep consistent version of hadoop dependence
* [SENTRY-674] - Update Apache Hive dependency to new release 1.1.0
* [SENTRY-676] - Address Sentry HA issues in secure cluster
* [SENTRY-690] - Remove SENTRY-645 patch from 1.5.0 release
* [SENTRY-691] - upgrade schema tool: when upgrade oracle backed db from 1.4.0 - 1.5.0, gets syntax error
* [SENTRY-693] - The generic model has not successfully revoke part of privileges from existed ALL privilege
** Task
* [SENTRY-354] - Test for update.distrib phase overriding
* [SENTRY-382] - Build documentation
* [SENTRY-415] - Add API to Sentry Service that allows clients to read the service's config values
* [SENTRY-624] - Add upgrade scripts for general privilege model
** Test
* [SENTRY-47] - Tests need to clean up the databases and tables it creates
* [SENTRY-383] - Add TestPrivilegeWithGrantOption to cluster test profile
* [SENTRY-688] - Verify the patch on 1.4.0