[fix](ranger) Apply Ranger policy items written against groups, from Ranger's user store (#68220)
### What problem does this PR solve?
Issue Number: None
Related PR: #68203 (documents the behaviour; this PR changes it)
Problem Summary:
A Ranger policy item can name users, groups and roles. The requests the
`ranger-doris` and `ranger-hive`
sources build carry no groups - Doris has none of its own to hand over -
so every policy item written
against a group never matches: an allow on a group grants nothing to a
Doris user, a row filter or a mask
on a group does not apply, and a **deny on a group is silently ignored**
while a user-level allow on the same
table still admits the user. Operators who manage access by Ranger group
(so that nobody edits a policy
every time somebody joins a team) see the user denied on everything,
down to `SHOW DATABASES` and
`SWITCH`, and it starts working the moment the same user is put on the
item directly.
Ranger's answer for a plugin that cannot compute groups itself is the
**user store** - the users and groups
Ranger Admin holds, kept current by usersync - which a plugin downloads
next to its policies. Ranger 2.5+
reads the requesting user's groups out of it when
`ranger.plugin.<type>.use.rangerGroups=true`; the
plugin library on branch-3.0/2.1 is 2.4.0, which has no such setting, so
on those branches no
configuration can make group items work.
What this PR does:
- `RangerUserStoreGroups` (ranger-common): the two Ranger plugins Doris
embeds put the user store
enricher on the service definition they download (`setPolicies`),
exactly as `RangerBasePlugin` does for
`use.rangerGroups`, and the request builders read the user's groups out
of the downloaded store and set
them on every request (`RangerAccessRequestImpl.setUserGroups`). This
does not depend on the Ranger
version the plugin was built against.
- On by default. `ranger.plugin.doris.use.rangerGroups=false`
(`ranger.plugin.hive.` for the catalog-level
source) switches it off and restores the previous requests; the property
is the one Ranger itself reads
for the same thing, so one setting decides both. The refresh interval
and retriever class are read under
Ranger's own option names (`userStoreRefresherPollingInterval`, default
60000 ms).
- Each plugin logs one line at start-up saying whether groups are
attached and how to switch it off.
- Regression: `ranger_p2/test_ranger_group_policy` writes an access
policy, a row filter, a mask and a
deny against a Ranger group only, and checks them as a member of that
group, then after leaving it.
`httpTest` gains `op "put"` for the Ranger user update that bumps the
user store version.
Verified end to end against the `docker/thirdparties` Ranger (Admin
2.4.0): group allow / row filter /
mask apply, a group deny outranks a user allow, a Ranger role granted to
a group resolves, leaving the group
revokes within one user store refresh, and `use.rangerGroups=false`
gives the previous behaviour.
What this PR also changes: a Ranger plugin is built with its service's
policies, or not at all
`RangerBasePlugin.init()` survives a Ranger Admin it cannot reach: it
answers out of the local policy cache
when there is one, and out of no policies at all when there is not,
refusing every check until a later poll
succeeds. That was already so before this PR, and it is the wrong state
for Doris to start in: an FE with
`access_controller_type=ranger-doris` and no policies is one nobody can
use (no account bypasses the source),
and a catalog bound to `ranger-hive` in that state refuses every
statement, with nothing but a line in fe.log
to say why. So the plugins now refuse that state at the moment they are
built, and Ranger's own resilience
takes over afterwards. The load order at start-up, and where it fails:
```text
FE start with access_controller_type=ranger-doris (every FE: master, follower, observer)
Env()
└─ AccessControllerManager()
└─ RangerDorisAccessControllerFactory.create()
└─ new RangerDorisPlugin("doris") ... on the starting thread, synchronously
└─ LoadedRangerPlugin.init()
├─ RangerBasePlugin.init()
│ ├─ audit subsystem
│ ├─ PolicyRefresher.startRefresher()
│ │ ├─ loadRoles() ──REST──► Ranger Admin (fails: logged; roles from cache, or none)
│ │ ├─ loadPolicy() ──REST──► Ranger Admin
│ │ │ ├─ answered → setPolicies(policies) policies version ≥ 0
│ │ │ ├─ unreachable, cache found → setPolicies(cached policies) policies version ≥ 0
│ │ │ ├─ unreachable, no cache → setPolicies(null): no engine policies version -1
│ │ │ └─ service not found → cache cleared, no engine policies version -1
│ │ │ setPolicies() puts the user store enricher on the service definition:
│ │ │ user store ──REST──► Ranger Admin (fails: logged; no groups until it arrives)
│ │ └─ refresher thread + download timer start (from here on: Ranger's own resilience)
│ └─ chained plugins (none configured)
└─ policies version < 0 ?
├─ no → plugin built. INFO "loaded in N ms: policies version ...", WARN if no user store
└─ yes → cleanup(), throw IllegalStateException("... has no policies to authorize against ...")
└─ Env() fails → the FE does not start
CREATE CATALOG ... 'access_controller.class'='ranger-hive'
CatalogFactory.createCatalog()
└─ catalog.initAccessController(dryRun = true)
└─ RangerHiveAccessControllerFactory.create()
└─ RangerHiveAuditStack.startFor(serviceName)
└─ new RangerHivePlugin(serviceName) → the same LoadedRangerPlugin.init() as above
├─ built → controller released again (the stack stays up 300 s for the bind that follows) → CREATE succeeds
└─ threw → DdlException "Failed to init access controller: ..." → CREATE fails
Later binds of a ranger-hive catalog (the first statement against it, after ALTER CATALOG, after the 300 s idle stop):
the same init(); a failure fails that statement and the next statement tries again. Never a fallback to the
built-in source.
```
Configuration errors (no `policy.rest.url`, a REST timeout that is not a
number, an audit destination Doris
does not ship) fail the way they always did: inside
`RangerBasePlugin.init()`, before anything is started. A
missing user store is not a failure: an Admin that serves none cannot be
told from one that has not answered
yet, so policy items written against a group simply do not apply until
it arrives, and the plugin says so
with a WARN.diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/LoadedRangerPlugin.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/LoadedRangerPlugin.java
new file mode 100644
index 0000000..e5c7f89
--- /dev/null
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/LoadedRangerPlugin.java
@@ -0,0 +1,159 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog.authorizer.ranger;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.apache.ranger.plugin.service.RangerBasePlugin;
+import org.apache.ranger.plugin.util.ServicePolicies;
+
+import java.util.concurrent.TimeUnit;
+
+/**
+ * What the two Ranger plugins Doris embeds have in common: they are built with their service's policies or
+ * not at all, and the requests built over them carry the groups Ranger keeps for a user.
+ *
+ * <p><b>Built with the policies, or not at all.</b> {@code RangerBasePlugin.init()} downloads the service's
+ * roles and policies - and, with the enricher below on them, its user store - before it returns, and it
+ * survives a Ranger admin it cannot reach: it answers out of the local policy cache
+ * ({@code ranger.plugin.<type>.policy.cache.dir}) when there is one, and out of no policies at all when there
+ * is not, refusing every check until a later poll succeeds. That is the right thing for a plugin embedded in
+ * a service that has to keep running, and the wrong thing at the moment Doris binds one: an FE whose instance
+ * scope {@code access_controller_type=ranger-doris} governs, and which has no policies, is an FE nobody can
+ * use - no account bypasses the source - and a catalog bound to a {@code ranger-hive} source in that state
+ * refuses every statement against it, in both cases with nothing but a line in fe.log to say why. So
+ * {@link #init()} refuses that state instead: a load that ended with no policies, from the admin or the
+ * cache - or with policies Ranger could build no engine out of, which it also survives, by logging - stops
+ * what it started and throws, with the cause. That fails the FE start, the {@code CREATE CATALOG} (whose
+ * dry run builds the plugin), or the statement binding a catalog to its source again, which is where an
+ * operator sees it. A configuration the load could not use is refused before it starts, for the same
+ * reader; see {@link RangerUserStoreGroups#validate}. Once built, an outage of the admin is Ranger's to
+ * survive, as it always was: the refresher keeps the last policies it downloaded and keeps polling.
+ *
+ * <p>The user store is not part of that. A load that found the policies but no user store is logged and
+ * accepted - policy items written against a group do not apply until the store arrives, which the enricher
+ * keeps asking for - because "no store has arrived" cannot be told apart from "this admin serves none", and
+ * an admin from before the user store download existed, or one that fails that download while serving the
+ * policies, ran every deployment before groups were attached at all; see {@link RangerUserStoreGroups}.
+ *
+ * <p><b>Requests carry Ranger's own groups.</b> Doris has none to offer, and a policy item written against a
+ * group matches nothing without them; the store they are read from is asked for with the policies, see
+ * {@link #setPolicies} and {@link RangerUserStoreGroups}.
+ */
+public abstract class LoadedRangerPlugin extends RangerBasePlugin {
+ private static final Logger LOG = LogManager.getLogger(LoadedRangerPlugin.class);
+
+ /**
+ * The version of the policies the last {@link #setPolicies} was handed and could not build an engine
+ * out of; null when it built one, or was handed none. {@code RangerBasePlugin.setPolicies} catches
+ * whatever the engine's construction throws and leaves the plugin without an engine, which is the state
+ * "no policies" leaves it in as well, and not the same cause: {@link #init} reads this to tell the two
+ * apart when it refuses the plugin.
+ */
+ private volatile Long policiesWithoutEngine;
+
+ protected LoadedRangerPlugin(String serviceType, String serviceName, String appId) {
+ super(serviceType, serviceName, appId);
+ }
+
+ /**
+ * Loads the plugin - {@code RangerBasePlugin.init()}: roles, policies and user store, on this thread -
+ * and refuses to leave it without policies; see the class comment.
+ *
+ * @throws IllegalArgumentException for a configuration the load could not use, before it starts; see
+ * {@link RangerUserStoreGroups#validate}
+ * @throws IllegalStateException when the load ended with no policies from either the admin or the
+ * cache, or with policies no engine could be built out of
+ */
+ @Override
+ public void init() {
+ RangerUserStoreGroups.validate(getConfig());
+ LOG.info(RangerUserStoreGroups.describe(getConfig()));
+ long startedAtNanos = System.nanoTime();
+ try {
+ super.init();
+ if (getPoliciesVersion() < 0) {
+ Long refusedVersion = policiesWithoutEngine;
+ throw new IllegalStateException(refusedVersion == null
+ ? describeNoPolicies() : describeNoEngine(refusedVersion));
+ }
+ } catch (RuntimeException | Error e) {
+ // Stops what the load had started before it failed - the policy refresher and its download
+ // timer, and the engine if one was built - so that a plugin nobody will hold does not go on
+ // polling the admin. A configuration the load could not use fails inside RangerBasePlugin.init()
+ // before any of that is started, and stopping nothing is a no-op.
+ try {
+ cleanup();
+ } catch (RuntimeException | Error stopFailure) {
+ e.addSuppressed(stopFailure);
+ }
+ throw e;
+ }
+ LOG.info("Ranger service {} loaded in {} ms: policies version {}, roles version {}, user store"
+ + " version {}", getServiceName(),
+ TimeUnit.NANOSECONDS.toMillis(System.nanoTime() - startedAtNanos), getPoliciesVersion(),
+ getRolesVersion(), getUserStoreVersion());
+ if (RangerUserStoreGroups.enabledFor(getConfig()) && getUserStoreVersion() < 0) {
+ // Policies to answer out of, but no user store to read groups from: the admin could not be reached
+ // for it and none was cached. Requests carry no groups until one arrives - the enricher keeps
+ // asking - which is what this source sent before groups were attached at all, and which a policy
+ // item written against a group, a deny included, does not match.
+ LOG.warn("Ranger service {} loaded its policies but no user store: until one arrives, requests"
+ + " carry no groups and policy items written against a group do not apply", getServiceName());
+ }
+ }
+
+ /** Why the plugin was refused: what was asked for the policies, and where they could have been cached. */
+ private String describeNoPolicies() {
+ String prefix = getConfig().getPropertyPrefix();
+ String adminUrl = getConfig().get(prefix + ".policy.rest.url");
+ String cacheDir = getConfig().get(prefix + ".policy.cache.dir");
+ return "Ranger service " + getServiceName() + " (type " + getServiceType() + ") has no policies to"
+ + " authorize against: Ranger Admin at " + adminUrl + " could not be reached or does not know"
+ + " the service, and "
+ + (cacheDir == null
+ ? "no policy cache directory is configured (" + prefix + ".policy.cache.dir)"
+ : "no policy cache was found under " + cacheDir)
+ + ". Fix ranger-" + getServiceType() + "-security.xml in fe/conf, or bring Ranger Admin back,"
+ + " and try again";
+ }
+
+ /** Why the plugin was refused when its policies did arrive: nothing could be built out of them. */
+ private String describeNoEngine(long policiesVersion) {
+ return "Ranger service " + getServiceName() + " (type " + getServiceType() + ") has no policies to"
+ + " authorize against: its policies (version " + policiesVersion + ") were downloaded, but no"
+ + " policy engine could be built out of them; RangerBasePlugin.setPolicies logged the cause"
+ + " just before this. Fix what it names and try again";
+ }
+
+ /**
+ * Takes the policies Ranger downloaded, and asks for the user store with them, so that the requests the
+ * source over this plugin builds can carry the groups Ranger keeps for a user; see
+ * {@link RangerUserStoreGroups}. Done on the way in rather than left to Ranger, because Ranger only does
+ * it on its own from 2.5 on, and behind a property.
+ *
+ * <p>Notes, for {@link #init}, whether the policies handed over left the plugin without an engine.
+ */
+ @Override
+ public void setPolicies(ServicePolicies policies) {
+ RangerUserStoreGroups.addUserStoreEnricher(getConfig(), policies);
+ super.setPolicies(policies);
+ Long version = policies == null ? null : policies.getPolicyVersion();
+ policiesWithoutEngine = version != null && getPoliciesVersion() < 0 ? version : null;
+ }
+}
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java
index eb36f2c..81b2134 100644
--- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerAccessController.java
@@ -238,6 +238,15 @@
return context.getClientIp().filter(StringUtils::isNotBlank).orElseGet(subject::getHost);
}
+ /**
+ * The groups to put on a request about {@code subject}: the ones Ranger's own user store puts the user
+ * in, see {@link RangerUserStoreGroups}. Empty when the plugin has no user store, for a user the store does
+ * not know, and when the deployment has switched this off.
+ */
+ protected Set<String> groupsOf(AuthorizedSubject subject) {
+ return RangerUserStoreGroups.groupsOf(getPlugin(), subject.getUser());
+ }
+
private static boolean deferenceFrom(Map<String, String> properties) {
String configured = properties == null ? null : properties.get(DEFER_TO_GLOBAL_SCOPE_AUTHORITY);
if (configured == null) {
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerUserStoreGroups.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerUserStoreGroups.java
new file mode 100644
index 0000000..3053b0a2
--- /dev/null
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-common/src/main/java/org/apache/doris/catalog/authorizer/ranger/RangerUserStoreGroups.java
@@ -0,0 +1,212 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog.authorizer.ranger;
+
+import org.apache.logging.log4j.LogManager;
+import org.apache.logging.log4j.Logger;
+import org.apache.ranger.authorization.hadoop.config.RangerPluginConfig;
+import org.apache.ranger.plugin.contextenricher.RangerAdminUserStoreRetriever;
+import org.apache.ranger.plugin.contextenricher.RangerUserStoreEnricher;
+import org.apache.ranger.plugin.policyengine.RangerPluginContext;
+import org.apache.ranger.plugin.service.RangerAuthContext;
+import org.apache.ranger.plugin.service.RangerBasePlugin;
+import org.apache.ranger.plugin.util.RangerUserStoreUtil;
+import org.apache.ranger.plugin.util.ServiceDefUtil;
+import org.apache.ranger.plugin.util.ServicePolicies;
+
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.Set;
+
+/**
+ * Where a request built by a Doris source gets the groups a Ranger policy item may be written against.
+ *
+ * <p>A Ranger policy item names users, groups and roles, and Ranger's own plugins fill in the groups from
+ * the service they run in - the Hive plugin asks Hadoop's group mapping. Doris has nothing to ask: an
+ * account holds roles, and no part of the engine knows which groups a user is in. So a request that carries
+ * no groups matches no group item at all, and a service whose policies are kept by group - the normal way
+ * to run Ranger, so that nobody edits a policy each time somebody joins a team - grants nothing to a Doris
+ * user; and, worse, denies nothing either. An item that denies a group is exactly as silent about it.
+ *
+ * <p>What Ranger provides for a plugin in that position is its <b>user store</b>: the users and groups Ranger
+ * Admin itself holds, kept current by usersync from LDAP or the OS, which a plugin downloads next to its
+ * policies through a {@link RangerUserStoreEnricher}. Ranger 2.5 and later reads the groups of the request's
+ * user out of it when {@code ranger.plugin.<type>.use.rangerGroups} is set; the two sources Doris embeds do
+ * the same thing here, in the request builder, so that it neither depends on the Ranger version they were
+ * built against - branches still on 2.4 have no such setting - nor on an operator finding a Ranger property
+ * no Doris document names.
+ *
+ * <p>It is on by default, because a group item that is silently ignored is a bug in the deployment's eyes,
+ * not a behaviour anybody opted into; the property Ranger reads for this is the one that switches it off,
+ * so that a deployment which has already decided the question in Ranger's terms has decided it here too.
+ * Switched off, requests carry no groups and this source matches what it matched before.
+ */
+public final class RangerUserStoreGroups {
+ private static final Logger LOG = LogManager.getLogger(RangerUserStoreGroups.class);
+
+ /**
+ * Suffix of the property switching this off, {@code ranger.plugin.<type>.use.rangerGroups} in full: the
+ * name Ranger itself reads for the same thing, so that one setting decides both.
+ */
+ public static final String USE_RANGER_GROUPS = ".use.rangerGroups";
+
+ /** How often the plugin asks Ranger Admin for a newer user store, when nothing configures it. */
+ private static final long DEFAULT_REFRESH_INTERVAL_MS = 60 * 1000L;
+
+ private RangerUserStoreGroups() {
+ }
+
+ /**
+ * Whether the plugin configured by {@code config} attaches user store groups; on unless switched off.
+ *
+ * <p>Read strictly - {@code true}, {@code false}, or nothing - and not through Hadoop's {@code getBoolean},
+ * which takes any value it cannot read as the default: a mistyped opt-out ({@code flase}) would switch this
+ * on, in a deployment that has just decided the opposite, and Ranger reads the same property with the
+ * opposite default, so that the two would disagree about what a single setting says.
+ *
+ * @throws IllegalArgumentException for a value that is neither; {@link #validate} raises it before the
+ * load, so that a request never meets it
+ */
+ public static boolean enabledFor(RangerPluginConfig config) {
+ // No configuration at all - a plugin stubbed out in a test - has not switched anything off.
+ if (config == null) {
+ return true;
+ }
+ String property = config.getPropertyPrefix() + USE_RANGER_GROUPS;
+ String value = config.getTrimmed(property);
+ if (value == null || value.isEmpty() || value.equalsIgnoreCase("true")) {
+ return true;
+ }
+ if (value.equalsIgnoreCase("false")) {
+ return false;
+ }
+ throw new IllegalArgumentException("Ranger service " + config.getServiceName() + ": " + property + "="
+ + value + " is neither true nor false; leave it unset or set it to false to switch off the"
+ + " groups requests carry");
+ }
+
+ /**
+ * How often the plugin asks Ranger Admin for a newer user store, in milliseconds: what
+ * {@code userStoreRefresherPollingInterval} says, or a minute.
+ *
+ * <p>Read here rather than left to the enricher, which parses the option inside the policy engine's
+ * construction: a value that is not a number fails there, and one that is not positive fails a step
+ * later in {@code Timer.schedule}, after the enricher has downloaded the store and started its refresher
+ * thread. {@code RangerBasePlugin.setPolicies} catches both and leaves the plugin without an engine, which
+ * {@link LoadedRangerPlugin#init} would refuse for the wrong reason - and, in the second case, with that
+ * thread left behind. So both are refused before the load, by {@link #validate}.
+ *
+ * @throws IllegalArgumentException for a value that is not a positive number of milliseconds
+ */
+ static long refreshIntervalMsOf(RangerPluginConfig config) {
+ String property = RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION;
+ String value = config.getTrimmed(property);
+ if (value == null || value.isEmpty()) {
+ return DEFAULT_REFRESH_INTERVAL_MS;
+ }
+ long intervalMs;
+ try {
+ intervalMs = Long.parseLong(value);
+ } catch (NumberFormatException e) {
+ throw new IllegalArgumentException("Ranger service " + config.getServiceName() + ": " + property + "="
+ + value + " is not a number of milliseconds", e);
+ }
+ if (intervalMs <= 0) {
+ throw new IllegalArgumentException("Ranger service " + config.getServiceName() + ": " + property + "="
+ + value + " is not a positive number of milliseconds");
+ }
+ return intervalMs;
+ }
+
+ /**
+ * Refuses a configuration this cannot run with, before {@code RangerBasePlugin.init()} starts anything:
+ * an opt-out that is neither true nor false, and a refresh interval that is not a positive number of
+ * milliseconds. See {@link #enabledFor} and {@link #refreshIntervalMsOf} for where each would fail
+ * otherwise, and how much worse.
+ *
+ * @throws IllegalArgumentException naming the property and its value
+ */
+ public static void validate(RangerPluginConfig config) {
+ if (config == null || !enabledFor(config)) {
+ return;
+ }
+ refreshIntervalMsOf(config);
+ }
+
+ /**
+ * Puts a user store enricher on the service definition Ranger just downloaded, unless one is there or
+ * this is switched off, so that the plugin fetches and refreshes the user store {@link #groupsOf} reads.
+ *
+ * <p>Called by {@link LoadedRangerPlugin#setPolicies} before handing the policies on, on every
+ * call and not only the first: a download of policy deltas comes with its own copy of the service
+ * definition, which is why {@code RangerBasePlugin} re-adds the enricher on deltas too. The retriever
+ * class and the refresh interval are read under the option names Ranger itself reads them under, so that
+ * an operator who has tuned them for Ranger's own {@code use.rangerGroups} has tuned them here.
+ */
+ public static void addUserStoreEnricher(RangerPluginConfig config, ServicePolicies policies) {
+ if (policies == null || config == null || !enabledFor(config)) {
+ return;
+ }
+ String retriever = config.get(RangerUserStoreEnricher.USERSTORE_RETRIEVER_CLASSNAME_OPTION,
+ RangerAdminUserStoreRetriever.class.getCanonicalName());
+ String refreshIntervalMs = Long.toString(refreshIntervalMsOf(config));
+ // Ranger logs the addition itself, once per download that needed it; the operator-facing line about
+ // why the store is downloaded at all is the plugin's, written once when it starts (see describe).
+ if (ServiceDefUtil.addUserStoreEnricher(policies, retriever, refreshIntervalMs) && LOG.isDebugEnabled()) {
+ LOG.debug("Ranger service {} will download its user store every {} ms", policies.getServiceName(),
+ refreshIntervalMs);
+ }
+ }
+
+ /** One line for the plugin's start-up log, saying what this does for it and how to switch it off. */
+ public static String describe(RangerPluginConfig config) {
+ String property = config.getPropertyPrefix() + USE_RANGER_GROUPS;
+ return enabledFor(config)
+ ? "Ranger service " + config.getServiceName() + ": requests carry the groups Ranger's user store"
+ + " puts the user in, so that policy items written against a group apply; set "
+ + property + "=false to switch that off"
+ : "Ranger service " + config.getServiceName() + ": " + property + "=false, so requests carry no"
+ + " groups and policy items written against a group never apply";
+ }
+
+ /**
+ * The groups the user store {@code plugin} has downloaded puts {@code user} in.
+ *
+ * <p>Empty when this is switched off, when no store has arrived - Ranger Admin could not be reached for
+ * it and nothing was cached, which {@link LoadedRangerPlugin} says why it does not refuse - and when the
+ * store does not know the user, which is the case for every account that exists in Doris only. Empty
+ * and not null on purpose: a request with an empty group set matches items written against users and
+ * roles exactly as it did before.
+ */
+ public static Set<String> groupsOf(RangerBasePlugin plugin, String user) {
+ if (user == null || !enabledFor(plugin.getConfig())) {
+ return Collections.emptySet();
+ }
+ // The auth context the policy engine publishes is where Ranger's own request processing reads the
+ // store from; it is replaced together with the engine, and the store is carried over when it is.
+ RangerPluginContext pluginContext = plugin.getPluginContext();
+ RangerAuthContext authContext = pluginContext == null ? null : pluginContext.getAuthContext();
+ RangerUserStoreUtil userStore = authContext == null ? null : authContext.getUserStoreUtil();
+ Set<String> groups = userStore == null ? null : userStore.getUserGroups(user);
+ if (groups == null || groups.isEmpty()) {
+ return Collections.emptySet();
+ }
+ // A copy, because the set belongs to the store shared by every request until the next download.
+ return new HashSet<>(groups);
+ }
+}
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java
index 68c9c7c..88f93d6 100644
--- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessController.java
@@ -258,11 +258,16 @@
protected RangerAccessRequestImpl createRequest(AuthorizedSubject subject, AccessContext context) {
RangerAccessRequestImpl request = new RangerAccessRequestImpl();
request.setUser(subject.getUser());
+ // The groups are Ranger's own, out of the user store this source's plugin downloads: Doris has no
+ // groups to offer, and without them a policy item written against a group never matches, allow or
+ // deny. See RangerUserStoreGroups, including for how a deployment switches it off.
+ request.setUserGroups(groupsOf(subject));
// No user roles, unlike ranger-hive, which does send them. Not an oversight and not free to change:
// a request carrying roles matches policy items written against a role, so sending them would start
// granting - and denying - on policies this source has never matched, in every deployment that has
// any. That is a change to what an existing Ranger service decides and belongs with a release note
- // of its own, not here.
+ // of its own, not here. (Ranger's own roles are unaffected: with none sent, Ranger resolves the ones
+ // it holds for the user and now also for the user's groups.)
request.setClientIPAddress(clientAddressOf(subject, context));
request.setClusterType(CLIENT_TYPE_DORIS);
request.setClientType(CLIENT_TYPE_DORIS);
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactory.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactory.java
index ae8d380..69023fe 100644
--- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactory.java
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisAccessControllerFactory.java
@@ -59,9 +59,9 @@
* one Ranger service, which is bounded and not a leak that grows. Stopping it would cost more than that:
* a plain {@code ALTER CATALOG} detaches and re-attaches the catalog's access controller, and a plugin
* torn down and rebuilt between those two pays {@code cleanup()} on the DDL thread - it interrupts the
- * policy refresher and joins it without a timeout - and then two synchronous admin REST calls on the way
- * back up, since {@code RangerBasePlugin.init()} loads the service's roles and policies before it
- * returns.
+ * policy refresher and joins it without a timeout - and then three synchronous admin REST calls on the
+ * way back up, since {@code RangerBasePlugin.init()} loads the service's roles, policies and user store
+ * before it returns.
*/
private static RangerBasePlugin sharedPlugin;
private static final Map<Map<String, String>, Held> byConfiguration = new LinkedHashMap<>();
@@ -131,8 +131,8 @@
return held.controller;
}
}
- // Built with no lock held: RangerBasePlugin.init() loads the service's roles and its policies
- // over REST before it returns, so against a slow or unreachable Ranger admin doing it under
+ // Built with no lock held: RangerBasePlugin.init() loads the service's roles, policies and user
+ // store over REST before it returns, so against a slow or unreachable Ranger admin doing it under
// the lock queues every other binding's create - and close - behind the whole REST timeout.
// Losing the race that opens costs one plugin, stopped in the finally below.
built = new RangerDorisPlugin(SERVICE_NAME);
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisPlugin.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisPlugin.java
index 0da65aa..1a6c6b1 100644
--- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisPlugin.java
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/main/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerDorisPlugin.java
@@ -17,17 +17,24 @@
package org.apache.doris.catalog.authorizer.ranger.doris;
-import org.apache.ranger.plugin.service.RangerAuthContextListener;
-import org.apache.ranger.plugin.service.RangerBasePlugin;
+import org.apache.doris.catalog.authorizer.ranger.LoadedRangerPlugin;
-public class RangerDorisPlugin extends RangerBasePlugin {
+import org.apache.ranger.plugin.service.RangerAuthContextListener;
+
+/**
+ * The plugin over a Ranger service of type {@code doris}: built with the service's policies or not at all, so
+ * that an FE whose instance scope it governs starts with them or does not start; see
+ * {@link LoadedRangerPlugin}.
+ */
+public class RangerDorisPlugin extends LoadedRangerPlugin {
public RangerDorisPlugin(String serviceName) {
this(serviceName, null);
}
public RangerDorisPlugin(String serviceName, RangerAuthContextListener rangerAuthContextListener) {
super(serviceName, null, null);
- super.init();
- super.registerAuthContextEventListener(rangerAuthContextListener);
+ // Registered before the load, so that the listener hears of the engine the load installs.
+ registerAuthContextEventListener(rangerAuthContextListener);
+ init();
}
}
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/LoadedRangerPluginTest.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/LoadedRangerPluginTest.java
new file mode 100644
index 0000000..bc1038b
--- /dev/null
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/LoadedRangerPluginTest.java
@@ -0,0 +1,410 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog.authorizer.ranger;
+
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
+import org.apache.hadoop.conf.Configuration;
+import org.apache.ranger.admin.client.RangerAdminClient;
+import org.apache.ranger.authorization.utils.JsonUtils;
+import org.apache.ranger.plugin.contextenricher.RangerAbstractContextEnricher;
+import org.apache.ranger.plugin.model.RangerRole;
+import org.apache.ranger.plugin.model.RangerServiceDef;
+import org.apache.ranger.plugin.policyengine.RangerAccessRequest;
+import org.apache.ranger.plugin.util.GrantRevokeRequest;
+import org.apache.ranger.plugin.util.GrantRevokeRoleRequest;
+import org.apache.ranger.plugin.util.RangerRoles;
+import org.apache.ranger.plugin.util.RangerServiceNotFoundException;
+import org.apache.ranger.plugin.util.RangerUserStore;
+import org.apache.ranger.plugin.util.ServicePolicies;
+import org.apache.ranger.plugin.util.ServiceTags;
+import org.junit.jupiter.api.AfterEach;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.io.TempDir;
+
+import java.io.IOException;
+import java.nio.charset.StandardCharsets;
+import java.nio.file.Files;
+import java.nio.file.Path;
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+/**
+ * The contract of {@link LoadedRangerPlugin}, run through Ranger's real {@code RangerBasePlugin.init()} -
+ * policy refresher, policy cache and user store enricher included - against a Ranger admin this test stands
+ * in for: the plugin is built out of what the admin serves or what the cache holds, and not at all otherwise.
+ */
+public class LoadedRangerPluginTest {
+ private static final String PREFIX = "ranger.plugin.test";
+ /** Named in a refusal; never dialed, since {@link Admin} takes the REST client's place. */
+ private static final String ADMIN_URL = "http://ranger.invalid:6080";
+
+ /**
+ * Stands in for Ranger Admin. Ranger builds it by class name ({@code policy.source.impl}) and asks it for
+ * the roles, the policies and the user store exactly as it asks its REST client. Down, every call throws,
+ * which is all an unreachable admin is to Ranger; up, it serves what the test put here.
+ */
+ public static final class Admin implements RangerAdminClient {
+ private static volatile ServicePolicies policies;
+ private static volatile RangerUserStore userStore;
+ private static volatile boolean serviceUnknown;
+
+ private static void down() {
+ policies = null;
+ userStore = null;
+ serviceUnknown = false;
+ }
+
+ @Override
+ public void init(String serviceName, String appId, String configPropertyPrefix, Configuration config) {
+ }
+
+ @Override
+ public ServicePolicies getServicePoliciesIfUpdated(long lastKnownVersion, long lastActivationTimeInMillis)
+ throws Exception {
+ if (serviceUnknown) {
+ throw new RangerServiceNotFoundException("test");
+ }
+ ServicePolicies current = policies;
+ if (current == null) {
+ throw new Exception("connection refused");
+ }
+ return lastKnownVersion < current.getPolicyVersion() ? current : null;
+ }
+
+ @Override
+ public RangerRoles getRolesIfUpdated(long lastKnownRoleVersion, long lastActivationTimeInMills)
+ throws Exception {
+ if (policies == null) {
+ throw new Exception("connection refused");
+ }
+ return null;
+ }
+
+ @Override
+ public RangerUserStore getUserStoreIfUpdated(long lastKnownUserStoreVersion, long lastActivationTimeInMillis)
+ throws Exception {
+ RangerUserStore current = userStore;
+ if (current == null) {
+ throw new Exception("connection refused");
+ }
+ return lastKnownUserStoreVersion < current.getUserStoreVersion() ? current : null;
+ }
+
+ @Override
+ public ServiceTags getServiceTagsIfUpdated(long lastKnownVersion, long lastActivationTimeInMillis) {
+ return null;
+ }
+
+ @Override
+ public List<String> getTagTypes(String tagTypePattern) {
+ return Collections.emptyList();
+ }
+
+ // Administration this plugin never performs.
+
+ @Override
+ public RangerRole createRole(RangerRole request) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void dropRole(String execUser, String roleName) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public List<String> getAllRoles(String execUser) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public List<String> getUserRoles(String execUser) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public RangerRole getRole(String execUser, String roleName) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void grantRole(GrantRevokeRoleRequest request) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void revokeRole(GrantRevokeRoleRequest request) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void grantAccess(GrantRevokeRequest request) {
+ throw new UnsupportedOperationException();
+ }
+
+ @Override
+ public void revokeAccess(GrantRevokeRequest request) {
+ throw new UnsupportedOperationException();
+ }
+ }
+
+ /**
+ * A context enricher the policy engine cannot be built with: Ranger builds the enrichers a service
+ * definition names while it builds the engine, and does not catch what their init() throws there.
+ */
+ public static final class BrokenEnricher extends RangerAbstractContextEnricher {
+ @Override
+ public void init() {
+ throw new IllegalStateException("broken on purpose");
+ }
+
+ @Override
+ public void enrich(RangerAccessRequest request) {
+ }
+ }
+
+ private static final class TestPlugin extends LoadedRangerPlugin {
+ private TestPlugin(String cacheDir) {
+ // Service type "test" reads ranger-test-*.xml, none of which exist here: what the load needs is set
+ // below, and the admin it asks is the class above.
+ super("test", "test", null);
+ getConfig().set(PREFIX + ".policy.source.impl", Admin.class.getName());
+ getConfig().set(PREFIX + ".policy.rest.url", ADMIN_URL);
+ if (cacheDir != null) {
+ getConfig().set(PREFIX + ".policy.cache.dir", cacheDir);
+ }
+ }
+
+ /** Whether there is a policy engine to answer out of; what a refused plugin must not leave behind. */
+ private boolean hasEngine() {
+ return getPolicyEngine() != null;
+ }
+ }
+
+ @TempDir
+ Path cacheDir;
+
+ private TestPlugin plugin;
+
+ @BeforeEach
+ public void takeTheAdminDown() {
+ Admin.down();
+ }
+
+ @AfterEach
+ public void stopThePlugin() {
+ if (plugin != null) {
+ plugin.cleanup();
+ }
+ }
+
+ private TestPlugin build(String cacheDir) {
+ plugin = new TestPlugin(cacheDir);
+ plugin.init();
+ return plugin;
+ }
+
+ /** Policies of {@code version} with nothing in them, enough for Ranger to build an engine out of. */
+ private static ServicePolicies policies(long version) {
+ RangerServiceDef serviceDef = new RangerServiceDef();
+ serviceDef.setName("test");
+ ServicePolicies policies = new ServicePolicies();
+ policies.setServiceName("test");
+ policies.setServiceDef(serviceDef);
+ policies.setPolicyVersion(version);
+ policies.setPolicies(new ArrayList<>());
+ return policies;
+ }
+
+ /** What Ranger's refresher writes when the admin answers, where it reads it back when the admin does not. */
+ private void cache(ServicePolicies policies) throws IOException {
+ // <appId>_<serviceName>.json, the appId defaulting to the service type.
+ Files.write(cacheDir.resolve("test_test.json"),
+ JsonUtils.objectToJson(policies).getBytes(StandardCharsets.UTF_8));
+ }
+
+ private static boolean refresherRunning() {
+ return Thread.getAllStackTraces().keySet().stream()
+ .anyMatch(thread -> thread.getName().startsWith("PolicyRefresher(serviceName=test)"));
+ }
+
+ /** The thread the user store enricher starts to download the store, which cleanup() reaches through the engine. */
+ private static boolean userStoreRefresherRunning() {
+ return Thread.getAllStackTraces().keySet().stream()
+ .anyMatch(thread -> thread.getName().startsWith("RangerUserStoreRefresher(serviceName=test)"));
+ }
+
+ /** Built out of what the admin serves, groups included: the user store came with the policies. */
+ @Test
+ public void testBuiltWithWhatTheAdminServes() {
+ Admin.policies = policies(3L);
+ Admin.userStore = new RangerUserStore(1L, null, null,
+ ImmutableMap.of("user1", ImmutableSet.of("readers")));
+
+ build(cacheDir.toString());
+
+ Assertions.assertEquals(3L, plugin.getPoliciesVersion());
+ Assertions.assertEquals(1L, plugin.getUserStoreVersion());
+ // Through Ranger's own enricher, which the policies were handed on with: what a request carries.
+ Assertions.assertEquals(ImmutableSet.of("readers"), RangerUserStoreGroups.groupsOf(plugin, "user1"));
+ Assertions.assertTrue(RangerUserStoreGroups.groupsOf(plugin, "nobody").isEmpty(),
+ "a user the store does not know is in a group");
+ Assertions.assertTrue(refresherRunning(), "built, the plugin keeps polling");
+ }
+
+ /**
+ * The admin is down and what it served last time is in the cache: built out of that, the way Ranger's
+ * own plugins ride out an outage. No store came with it, which is logged and not refused.
+ */
+ @Test
+ public void testBuiltOutOfTheCacheWhenTheAdminIsDown() throws IOException {
+ cache(policies(7L));
+
+ build(cacheDir.toString());
+
+ Assertions.assertEquals(7L, plugin.getPoliciesVersion());
+ Assertions.assertTrue(plugin.getUserStoreVersion() < 0);
+ Assertions.assertTrue(RangerUserStoreGroups.groupsOf(plugin, "user1").isEmpty(),
+ "groups came from somewhere with no store");
+ }
+
+ /** Nothing from the admin, nothing in the cache: refused with the cause, and nothing left running. */
+ @Test
+ public void testRefusedWithNoPoliciesFromTheAdminOrTheCache() {
+ plugin = new TestPlugin(cacheDir.toString());
+
+ IllegalStateException refused = Assertions.assertThrows(IllegalStateException.class, plugin::init);
+
+ Assertions.assertTrue(refused.getMessage().contains("has no policies to authorize against"),
+ refused.getMessage());
+ Assertions.assertTrue(refused.getMessage().contains(ADMIN_URL), refused.getMessage());
+ Assertions.assertTrue(refused.getMessage().contains(cacheDir.toString()), refused.getMessage());
+ Assertions.assertFalse(plugin.hasEngine(), "an engine was left behind");
+ Assertions.assertFalse(refresherRunning(), "the policy refresher init() started is still running");
+ }
+
+ @Test
+ public void testRefusedWithNoCacheDirectoryConfigured() {
+ plugin = new TestPlugin(null);
+
+ IllegalStateException refused = Assertions.assertThrows(IllegalStateException.class, plugin::init);
+
+ Assertions.assertTrue(refused.getMessage().contains(PREFIX + ".policy.cache.dir"), refused.getMessage());
+ Assertions.assertFalse(refresherRunning());
+ }
+
+ /** A service the admin does not know - a mistyped service name - has no policies, cache or no cache. */
+ @Test
+ public void testRefusedForAServiceTheAdminDoesNotKnow() throws IOException {
+ cache(policies(7L));
+ Admin.serviceUnknown = true;
+ plugin = new TestPlugin(cacheDir.toString());
+
+ Assertions.assertThrows(IllegalStateException.class, plugin::init);
+
+ Assertions.assertFalse(plugin.hasEngine(), "an engine was left behind");
+ Assertions.assertFalse(refresherRunning());
+ }
+
+ /** A configuration the load cannot use fails as it always did: on the calling thread, with its cause. */
+ @Test
+ public void testAConfigurationTheLoadCannotUseIsRefusedWithItsCause() {
+ plugin = new TestPlugin(cacheDir.toString());
+ plugin.getConfig().set(PREFIX + ".policy.pollIntervalMs", "often");
+
+ Assertions.assertThrows(NumberFormatException.class, plugin::init);
+
+ Assertions.assertFalse(refresherRunning());
+ }
+
+ /**
+ * A refresh interval the enricher could not schedule is refused before the load starts. Left to the
+ * enricher, a non-positive interval fails in Timer.schedule after the store is downloaded and the
+ * refresher thread is up, inside the engine's construction: RangerBasePlugin.setPolicies catches it, the
+ * plugin is refused as if it had no policies, and cleanup() cannot reach the thread through an engine
+ * that was never built.
+ */
+ @Test
+ public void testRefusedBeforeTheLoadForAnIntervalTheEnricherCouldNotSchedule() {
+ Admin.policies = policies(3L);
+ Admin.userStore = new RangerUserStore(1L, null, null, ImmutableMap.of("user1", ImmutableSet.of("readers")));
+ plugin = new TestPlugin(cacheDir.toString());
+ plugin.getConfig().set("userStoreRefresherPollingInterval", "0");
+
+ IllegalArgumentException refused = Assertions.assertThrows(IllegalArgumentException.class, plugin::init);
+
+ Assertions.assertTrue(refused.getMessage().contains("userStoreRefresherPollingInterval=0"),
+ refused.getMessage());
+ Assertions.assertFalse(plugin.hasEngine(), "an engine was left behind");
+ Assertions.assertFalse(refresherRunning(), "the policy refresher was started");
+ Assertions.assertFalse(userStoreRefresherRunning(), "the user store refresher was started");
+ }
+
+ /** An opt-out Hadoop's getBoolean would have read as the default is refused, not switched on. */
+ @Test
+ public void testRefusedBeforeTheLoadForASwitchThatIsNeitherTrueNorFalse() {
+ Admin.policies = policies(3L);
+ plugin = new TestPlugin(cacheDir.toString());
+ plugin.getConfig().set(PREFIX + ".use.rangerGroups", "flase");
+
+ IllegalArgumentException refused = Assertions.assertThrows(IllegalArgumentException.class, plugin::init);
+
+ Assertions.assertTrue(refused.getMessage().contains(PREFIX + ".use.rangerGroups=flase"), refused.getMessage());
+ Assertions.assertFalse(refresherRunning());
+ }
+
+ /**
+ * The policies arrived and Ranger could build no engine out of them - here, a context enricher on the
+ * service definition whose init() throws; RangerBasePlugin.setPolicies logs that and keeps going with
+ * no engine, the same state as no policies at all. Refused for what happened, not for an admin that
+ * answered.
+ */
+ @Test
+ public void testRefusedWithPoliciesNoEngineCouldBeBuiltOutOf() {
+ ServicePolicies policies = policies(3L);
+ policies.getServiceDef().setContextEnrichers(Collections.singletonList(
+ new RangerServiceDef.RangerContextEnricherDef(1L, "broken", BrokenEnricher.class.getName(), null)));
+ Admin.policies = policies;
+ plugin = new TestPlugin(cacheDir.toString());
+
+ IllegalStateException refused = Assertions.assertThrows(IllegalStateException.class, plugin::init);
+
+ Assertions.assertTrue(refused.getMessage().contains("policies (version 3) were downloaded, but no policy"
+ + " engine could be built out of them"), refused.getMessage());
+ Assertions.assertFalse(refused.getMessage().contains("could not be reached"), refused.getMessage());
+ Assertions.assertFalse(plugin.hasEngine(), "an engine was left behind");
+ Assertions.assertFalse(refresherRunning());
+ }
+
+ /** The seam the groups hang on: policies are handed on with the user store enricher on their definition. */
+ @Test
+ public void testThePoliciesAreHandedOnWithTheUserStoreEnricher() {
+ plugin = new TestPlugin(null);
+ ServicePolicies policies = policies(1L);
+
+ plugin.setPolicies(policies);
+
+ Assertions.assertNotNull(plugin.getUserStoreEnricher(), "no user store enricher on the engine");
+ Assertions.assertTrue(policies.getServiceDef().getContextEnrichers().stream()
+ .anyMatch(enricher -> "userStoreEnricher".equals(enricher.getName())));
+ }
+}
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerUserStoreGroupsTest.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerUserStoreGroupsTest.java
new file mode 100644
index 0000000..1d72f17
--- /dev/null
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/RangerUserStoreGroupsTest.java
@@ -0,0 +1,206 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog.authorizer.ranger;
+
+import org.apache.ranger.authorization.hadoop.config.RangerPluginConfig;
+import org.apache.ranger.plugin.contextenricher.RangerAdminUserStoreRetriever;
+import org.apache.ranger.plugin.contextenricher.RangerUserStoreEnricher;
+import org.apache.ranger.plugin.model.RangerServiceDef;
+import org.apache.ranger.plugin.model.RangerServiceDef.RangerContextEnricherDef;
+import org.apache.ranger.plugin.util.ServiceDefUtil;
+import org.apache.ranger.plugin.util.ServicePolicies;
+import org.junit.jupiter.api.Assertions;
+import org.junit.jupiter.api.Test;
+
+import java.util.List;
+
+/**
+ * The half of {@link RangerUserStoreGroups} that runs when policies arrive: putting the user store enricher
+ * on the service definition, which is what makes the plugin download the store the other half reads.
+ */
+public class RangerUserStoreGroupsTest {
+
+ private static RangerPluginConfig config() {
+ // Service type "test" reads ranger-test-*.xml, none of which exist here: every property is at its
+ // default, which is the state a deployment that has never heard of this is in.
+ return new RangerPluginConfig("test", "test", null, null, null, null);
+ }
+
+ /** Policies as Ranger Admin hands them over: a service definition carrying no context enricher. */
+ private static ServicePolicies downloaded() {
+ RangerServiceDef serviceDef = new RangerServiceDef();
+ serviceDef.setName("test");
+ ServicePolicies policies = new ServicePolicies();
+ policies.setServiceName("test");
+ policies.setServiceDef(serviceDef);
+ return policies;
+ }
+
+ private static List<RangerContextEnricherDef> enrichersOf(ServicePolicies policies) {
+ return policies.getServiceDef().getContextEnrichers();
+ }
+
+ @Test
+ public void testOnByDefault() {
+ Assertions.assertTrue(RangerUserStoreGroups.enabledFor(config()));
+ }
+
+ /** The switch is read as Ranger reads its own copy of it, case and surrounding blanks aside. */
+ @Test
+ public void testReadsTheSwitchStrictly() {
+ RangerPluginConfig config = config();
+ config.set("ranger.plugin.test.use.rangerGroups", " False ");
+ Assertions.assertFalse(RangerUserStoreGroups.enabledFor(config));
+ config.set("ranger.plugin.test.use.rangerGroups", "TRUE");
+ Assertions.assertTrue(RangerUserStoreGroups.enabledFor(config));
+ config.set("ranger.plugin.test.use.rangerGroups", "");
+ Assertions.assertTrue(RangerUserStoreGroups.enabledFor(config));
+ }
+
+ /**
+ * Hadoop's getBoolean would take a mistyped opt-out as its default and switch this on; an operator who
+ * has just decided the opposite is told so, by property and value, rather than granted on groups.
+ */
+ @Test
+ public void testRefusesASwitchThatIsNeitherTrueNorFalse() {
+ RangerPluginConfig config = config();
+ config.set("ranger.plugin.test.use.rangerGroups", "flase");
+
+ IllegalArgumentException refused = Assertions.assertThrows(IllegalArgumentException.class,
+ () -> RangerUserStoreGroups.validate(config));
+
+ Assertions.assertTrue(refused.getMessage().contains("ranger.plugin.test.use.rangerGroups=flase"),
+ refused.getMessage());
+ Assertions.assertThrows(IllegalArgumentException.class, () -> RangerUserStoreGroups.enabledFor(config));
+ }
+
+ /**
+ * Ranger's enricher parses the interval inside the engine's construction, and a non-positive one
+ * fails in Timer.schedule after the refresher thread is up; both are refused before any of that.
+ */
+ @Test
+ public void testRefusesAnIntervalThatIsNotAPositiveNumberOfMilliseconds() {
+ for (String interval : new String[] {"0", "-1", "often", "5s"}) {
+ RangerPluginConfig config = config();
+ config.set(RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION, interval);
+
+ IllegalArgumentException refused = Assertions.assertThrows(IllegalArgumentException.class,
+ () -> RangerUserStoreGroups.validate(config), interval);
+
+ Assertions.assertTrue(refused.getMessage().contains(
+ RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION + "=" + interval),
+ refused.getMessage());
+ }
+ }
+
+ @Test
+ public void testAcceptsADefaultOrPositiveInterval() {
+ RangerPluginConfig config = config();
+ Assertions.assertDoesNotThrow(() -> RangerUserStoreGroups.validate(config));
+ Assertions.assertEquals(60000L, RangerUserStoreGroups.refreshIntervalMsOf(config));
+ config.set(RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION, " 5000 ");
+ Assertions.assertDoesNotThrow(() -> RangerUserStoreGroups.validate(config));
+ Assertions.assertEquals(5000L, RangerUserStoreGroups.refreshIntervalMsOf(config));
+ }
+
+ /** Switched off, no enricher reads the interval, so a bad one refuses nothing. */
+ @Test
+ public void testAnIntervalDoesNotMatterWhenSwitchedOff() {
+ RangerPluginConfig config = config();
+ config.set("ranger.plugin.test.use.rangerGroups", "false");
+ config.set(RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION, "0");
+
+ Assertions.assertDoesNotThrow(() -> RangerUserStoreGroups.validate(config));
+ }
+
+ /**
+ * The enricher is the one Ranger adds for its own {@code use.rangerGroups}, options included, so that a
+ * deployment that tuned the retriever or the interval for Ranger has tuned them here.
+ */
+ @Test
+ public void testAddsTheUserStoreEnricherRangerWouldHaveAdded() {
+ ServicePolicies policies = downloaded();
+
+ RangerUserStoreGroups.addUserStoreEnricher(config(), policies);
+
+ Assertions.assertTrue(ServiceDefUtil.isUserStoreEnricherPresent(policies));
+ RangerContextEnricherDef enricher = enrichersOf(policies).get(0);
+ Assertions.assertEquals(RangerUserStoreEnricher.class.getName(), enricher.getEnricher());
+ Assertions.assertEquals(RangerAdminUserStoreRetriever.class.getCanonicalName(),
+ enricher.getEnricherOptions().get(RangerUserStoreEnricher.USERSTORE_RETRIEVER_CLASSNAME_OPTION));
+ Assertions.assertEquals("60000",
+ enricher.getEnricherOptions().get(RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION));
+ }
+
+ @Test
+ public void testHonoursTheRetrieverAndIntervalRangerReads() {
+ RangerPluginConfig config = config();
+ config.set(RangerUserStoreEnricher.USERSTORE_RETRIEVER_CLASSNAME_OPTION, "com.example.Retriever");
+ config.set(RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION, "5000");
+ ServicePolicies policies = downloaded();
+
+ RangerUserStoreGroups.addUserStoreEnricher(config, policies);
+
+ RangerContextEnricherDef enricher = enrichersOf(policies).get(0);
+ Assertions.assertEquals("com.example.Retriever",
+ enricher.getEnricherOptions().get(RangerUserStoreEnricher.USERSTORE_RETRIEVER_CLASSNAME_OPTION));
+ Assertions.assertEquals("5000",
+ enricher.getEnricherOptions().get(RangerUserStoreEnricher.USERSTORE_REFRESHER_POLLINGINTERVAL_OPTION));
+ }
+
+ /** Called on every download, deltas included, so a second call must not stack a second enricher. */
+ @Test
+ public void testAddsItOnce() {
+ ServicePolicies policies = downloaded();
+
+ RangerUserStoreGroups.addUserStoreEnricher(config(), policies);
+ RangerUserStoreGroups.addUserStoreEnricher(config(), policies);
+
+ Assertions.assertEquals(1, enrichersOf(policies).size());
+ }
+
+ @Test
+ public void testLeavesTheDefinitionAloneWhenSwitchedOff() {
+ RangerPluginConfig config = config();
+ config.set("ranger.plugin.test.use.rangerGroups", "false");
+ ServicePolicies policies = downloaded();
+
+ RangerUserStoreGroups.addUserStoreEnricher(config, policies);
+
+ Assertions.assertFalse(RangerUserStoreGroups.enabledFor(config));
+ Assertions.assertFalse(ServiceDefUtil.isUserStoreEnricherPresent(policies));
+ }
+
+ /** What the refresher hands over when the service is gone: RangerBasePlugin copes with it, so must this. */
+ @Test
+ public void testToleratesNoPolicies() {
+ Assertions.assertDoesNotThrow(() -> RangerUserStoreGroups.addUserStoreEnricher(config(), null));
+ }
+
+ /** The start-up line names the property either way, so an operator reading the log knows the switch. */
+ @Test
+ public void testDescribesItselfAndTheSwitch() {
+ RangerPluginConfig config = config();
+ Assertions.assertTrue(RangerUserStoreGroups.describe(config).contains("policy items written against a"
+ + " group apply; set ranger.plugin.test.use.rangerGroups=false"));
+
+ config.set("ranger.plugin.test.use.rangerGroups", "false");
+ Assertions.assertTrue(RangerUserStoreGroups.describe(config).contains(
+ "ranger.plugin.test.use.rangerGroups=false, so requests carry no groups"));
+ }
+}
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerTest.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerTest.java
index b439734..6eae567 100644
--- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerTest.java
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-doris/src/test/java/org/apache/doris/catalog/authorizer/ranger/doris/RangerTest.java
@@ -32,10 +32,13 @@
import com.google.common.collect.Lists;
import com.google.common.collect.Sets;
import org.apache.ranger.plugin.policyengine.RangerAccessRequest;
+import org.apache.ranger.plugin.policyengine.RangerAccessRequestImpl;
import org.apache.ranger.plugin.policyengine.RangerAccessResource;
import org.apache.ranger.plugin.policyengine.RangerAccessResult;
import org.apache.ranger.plugin.policyengine.RangerAccessResultProcessor;
+import org.apache.ranger.plugin.service.RangerAuthContext;
import org.apache.ranger.plugin.service.RangerBasePlugin;
+import org.apache.ranger.plugin.util.RangerUserStore;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
@@ -56,7 +59,7 @@
public DorisTestPlugin(String serviceName) {
super(serviceName, null, null);
- // super.init();
+ // Never init(): it answers out of the overrides below, not out of a Ranger service.
}
@Override
@@ -131,7 +134,9 @@
} else if (!Strings.isNullOrEmpty(db)) {
result.setIsAllowed("ctl3".equals(ctl) && "db3".equals(db));
} else if (!Strings.isNullOrEmpty(ctl)) {
- result.setIsAllowed("ctl4".equals(ctl));
+ // ctl5 stands for a policy item written against a group rather than a user.
+ result.setIsAllowed("ctl4".equals(ctl)
+ || ("ctl5".equals(ctl) && request.getUserGroups().contains("readers")));
} else if (!Strings.isNullOrEmpty(cg)) {
result.setIsAllowed("cg1".equals(cg));
} else if (!Strings.isNullOrEmpty(sv)) {
@@ -353,4 +358,86 @@
Assertions.assertEquals(0, plugin.requests.get(),
"the default workload group was put to the policy engine");
}
+
+ /**
+ * Hands {@code plugin} the user store its policy engine would have published, mapping user1 onto
+ * {@code groups}: what the enricher does once Ranger Admin has answered the user store download.
+ */
+ private static void userStoreArrived(RangerBasePlugin plugin, String... groups) {
+ RangerUserStore userStore = new RangerUserStore(1L, null, null,
+ Collections.singletonMap("user1", Sets.newHashSet(groups)));
+ plugin.getPluginContext().setAuthContext(new RangerAuthContext(null, null, null, userStore));
+ }
+
+ /**
+ * A request carries the groups Ranger's user store puts the user in, and nothing else does: Doris has
+ * no groups of its own, so without this a policy item written against a group never matches.
+ */
+ @Test
+ public void testRequestCarriesTheGroupsOfRangersUserStore() {
+ DorisTestPlugin plugin = new DorisTestPlugin("test");
+ userStoreArrived(plugin, "readers", "analysts");
+
+ RangerAccessRequestImpl request = new RangerDorisAccessController(plugin, NOTHING_GRANTED_ELSEWHERE)
+ .createRequest(USER, AccessContext.NONE);
+
+ Assertions.assertEquals("user1", request.getUser());
+ Assertions.assertEquals(Sets.newHashSet("readers", "analysts"), request.getUserGroups());
+ Assertions.assertTrue(request.getUserRoles().isEmpty(),
+ "roles were attached, which this source has never done; Ranger resolves its own");
+ }
+
+ /** Until the plugin has a user store - not loaded yet, Ranger Admin unreachable - there are no groups. */
+ @Test
+ public void testNoGroupsBeforeTheUserStoreArrives() {
+ RangerAccessRequestImpl request = controller().createRequest(USER, AccessContext.NONE);
+
+ Assertions.assertTrue(request.getUserGroups().isEmpty());
+ }
+
+ /** A user the store does not know - one that exists in Doris only - is in no group. */
+ @Test
+ public void testAUserUnknownToTheUserStoreIsInNoGroup() {
+ DorisTestPlugin plugin = new DorisTestPlugin("test");
+ userStoreArrived(plugin, "readers");
+
+ RangerAccessRequestImpl request = new RangerDorisAccessController(plugin, NOTHING_GRANTED_ELSEWHERE)
+ .createRequest(AuthorizedSubject.of("somebody_else", "%"), AccessContext.NONE);
+
+ Assertions.assertTrue(request.getUserGroups().isEmpty());
+ }
+
+ /**
+ * The property Ranger itself reads for this switches it off, so that a deployment that has decided the
+ * question in Ranger's terms has decided it here too - and gets the requests this source built before.
+ */
+ @Test
+ public void testTheRangerPropertySwitchesGroupsOff() {
+ DorisTestPlugin plugin = new DorisTestPlugin("test");
+ userStoreArrived(plugin, "readers");
+ plugin.getConfig().set("ranger.plugin.test.use.rangerGroups", "false");
+
+ RangerAccessRequestImpl request = new RangerDorisAccessController(plugin, NOTHING_GRANTED_ELSEWHERE)
+ .createRequest(USER, AccessContext.NONE);
+
+ Assertions.assertTrue(request.getUserGroups().isEmpty());
+ }
+
+ /** The whole point: a policy item written against a group decides, once the user is in that group. */
+ @Test
+ public void testAPolicyItemWrittenAgainstAGroupDecides() throws AccessDeniedException {
+ DorisTestPlugin plugin = new DorisTestPlugin("test");
+ RangerDorisAccessController controller = new RangerDorisAccessController(plugin, NOTHING_GRANTED_ELSEWHERE);
+ AuthorizedResource table = AuthorizedResource.table("ctl5", "db", "tbl");
+
+ Assertions.assertThrows(AccessDeniedException.class,
+ () -> controller.checkPrivilege(USER, table, AccessRequirements.SELECT, AccessContext.NONE));
+
+ userStoreArrived(plugin, "readers");
+ controller.checkPrivilege(USER, table, AccessRequirements.SELECT, AccessContext.NONE);
+
+ userStoreArrived(plugin, "writers");
+ Assertions.assertThrows(AccessDeniedException.class,
+ () -> controller.checkPrivilege(USER, table, AccessRequirements.SELECT, AccessContext.NONE));
+ }
}
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java
index b4a91da..7e3fe60 100644
--- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessController.java
@@ -343,6 +343,9 @@
AccessContext context) {
RangerAccessRequestImpl request = new RangerAccessRequestImpl();
request.setUser(subject.getUser());
+ // Ranger's groups for the user, out of the user store this service's plugin downloads; a Hive
+ // service's policies are usually kept by group, and Doris has no groups of its own to send.
+ request.setUserGroups(groupsOf(subject));
request.setUserRoles(roles);
request.setClientIPAddress(clientAddressOf(subject, context));
request.setClusterType(CLIENT_TYPE_DORIS);
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java
index 5fc7528..82428c5 100644
--- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactory.java
@@ -70,7 +70,7 @@
* <p>A stack outlives the binding that started it, but not the process. The last binding letting go does
* not stop it, because a plain {@code ALTER CATALOG} detaches and re-attaches the catalog's access
* controller and a stack stopped between those two costs a {@code cleanup()} on the DDL thread - it
- * interrupts the policy refresher and joins it without a timeout - and two synchronous admin REST calls
+ * interrupts the policy refresher and joins it without a timeout - and three synchronous admin REST calls
* on the way back up. What happens instead is that a stack nothing reads any more is stopped after
* {@link #idleStackGraceSeconds}, and anything asking for that service again in the meantime cancels
* the stop. Without it the map would grow by one entry, one policy refresher thread and one download
@@ -168,9 +168,9 @@
return held.controller;
}
}
- // Nothing reads this service yet, and starting to read it talks to the Ranger admin twice
- // before it returns: RangerBasePlugin.init() loads the service's roles and its policies
- // synchronously, before the refresher thread starts. Built with no lock held, so that a slow
+ // Nothing reads this service yet, and starting to read it talks to the Ranger admin three
+ // times before it returns: RangerBasePlugin.init() loads the service's roles, policies and user
+ // store synchronously, before the refresher thread starts. Built with no lock held, so that a slow
// or unreachable admin cannot queue every other binding's create - and close - behind it.
// Losing the race that opens costs one plugin, stopped in the finally below.
built = RangerHiveAuditStack.startFor(serviceName);
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditStack.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditStack.java
index e223ff6..ab19977 100644
--- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditStack.java
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAuditStack.java
@@ -33,7 +33,7 @@
* <p>Bundled because the three are one lifetime - the handler is configured out of the plugin and the task
* exists to drain the handler - and because that lifetime is not a binding's. A catalog bound to this source
* is detached and re-attached by a plain {@code ALTER CATALOG}, and tearing a stack down between those two
- * costs a {@code cleanup()} on the DDL thread and two synchronous REST calls to the Ranger admin on the way
+ * costs a {@code cleanup()} on the DDL thread and three synchronous REST calls to the Ranger admin on the way
* back up. So {@link RangerHiveAccessControllerFactory} keeps one stack per Ranger service, hands it to
* every controller reading that service, and stops it only once nothing has read it for a while - while a
* controller built directly, a test or an embedding that owns its own, starts and stops one of its own.
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHivePlugin.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHivePlugin.java
index 18ddde8..170fe0c 100644
--- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHivePlugin.java
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/main/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHivePlugin.java
@@ -17,17 +17,25 @@
package org.apache.doris.catalog.authorizer.ranger.hive;
-import org.apache.ranger.plugin.service.RangerAuthContextListener;
-import org.apache.ranger.plugin.service.RangerBasePlugin;
+import org.apache.doris.catalog.authorizer.ranger.LoadedRangerPlugin;
-public class RangerHivePlugin extends RangerBasePlugin {
+import org.apache.ranger.plugin.service.RangerAuthContextListener;
+
+/**
+ * The plugin over a Ranger service of type {@code hive}: built with the service's policies or not at all, so
+ * that a catalog is bound to it with them or the binding fails; see {@link LoadedRangerPlugin}. The groups
+ * its requests carry are Ranger's own, where Hive's plugin would have asked Hadoop's group mapping, which
+ * Doris has no equivalent of.
+ */
+public class RangerHivePlugin extends LoadedRangerPlugin {
public RangerHivePlugin(String serviceName) {
this(serviceName, null);
}
public RangerHivePlugin(String serviceName, RangerAuthContextListener rangerAuthContextListener) {
super(serviceName, null, null);
- super.init();
- super.registerAuthContextEventListener(rangerAuthContextListener);
+ // Registered before the load, so that the listener hears of the engine the load installs.
+ registerAuthContextEventListener(rangerAuthContextListener);
+ init();
}
}
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactoryTest.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactoryTest.java
index cf8be16..f3f6da8 100644
--- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactoryTest.java
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerFactoryTest.java
@@ -136,7 +136,7 @@
* both detach a catalog's access controller and attach a new one, so the count of bindings on a service
* passes through zero as a matter of course. Stopped there and rebuilt, the plugin costs a
* {@code cleanup()} on the DDL thread - it interrupts the policy refresher and joins it with no timeout -
- * and two synchronous admin REST calls on the way back up.
+ * and three synchronous admin REST calls on the way back up.
*/
@Test
public void testAReAcquireWithinTheGraceKeepsThePluginUp() {
diff --git a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java
index 79a48e6..7485c5c6c 100644
--- a/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java
+++ b/fe/fe-authorization/fe-authorization-plugins/fe-authorization-plugin-ranger-hive/src/test/java/org/apache/doris/catalog/authorizer/ranger/hive/RangerHiveAccessControllerTest.java
@@ -28,11 +28,15 @@
import com.google.common.collect.ImmutableMap;
import com.google.common.collect.ImmutableSet;
+import org.apache.ranger.authorization.hadoop.config.RangerPluginConfig;
import org.apache.ranger.plugin.model.RangerPolicy;
import org.apache.ranger.plugin.policyengine.RangerAccessRequest;
import org.apache.ranger.plugin.policyengine.RangerAccessRequestImpl;
import org.apache.ranger.plugin.policyengine.RangerAccessResult;
+import org.apache.ranger.plugin.policyengine.RangerPluginContext;
import org.apache.ranger.plugin.policyengine.RangerPolicyEngine;
+import org.apache.ranger.plugin.service.RangerAuthContext;
+import org.apache.ranger.plugin.util.RangerUserStore;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.Test;
import org.mockito.ArgumentCaptor;
@@ -103,6 +107,38 @@
}
/**
+ * Groups, unlike roles, are not the engine's to know: Doris has none, so they are Ranger's own, read out
+ * of the user store the plugin downloads - which for a Hive service is where its policies' groups come
+ * from in the first place. Without them every policy item written against a group is dead to Doris.
+ */
+ @Test
+ public void testRequestCarriesTheGroupsOfRangersUserStore() {
+ AuthorizationContext context = Mockito.mock(AuthorizationContext.class);
+ Mockito.when(context.rolesOf(SUBJECT)).thenReturn(ImmutableSet.of());
+ // The plugin context as the policy engine leaves it once the user store enricher has run.
+ RangerPluginContext pluginContext = new RangerPluginContext(
+ new RangerPluginConfig("hive", "hive", null, null, null, null));
+ pluginContext.setAuthContext(new RangerAuthContext(null, null, null, new RangerUserStore(1L, null, null,
+ ImmutableMap.of("user1", ImmutableSet.of("analysts", "etl")))));
+
+ try (MockedConstruction<RangerHivePlugin> plugin = Mockito.mockConstruction(RangerHivePlugin.class,
+ (mock, settings) -> Mockito.when(mock.getPluginContext()).thenReturn(pluginContext));
+ MockedConstruction<RangerHiveAuditHandler> audit =
+ Mockito.mockConstruction(RangerHiveAuditHandler.class)) {
+ RangerHiveAccessController controller = new RangerHiveAccessController(
+ ImmutableMap.of("ranger.service.name", "hive"), context);
+ try {
+ Assertions.assertEquals(ImmutableSet.of("analysts", "etl"),
+ controller.createRequest(SUBJECT, AccessContext.NONE).getUserGroups());
+ Assertions.assertTrue(controller.createRequest(AuthorizedSubject.of("nobody", "%"),
+ AccessContext.NONE).getUserGroups().isEmpty(), "a user the store does not know");
+ } finally {
+ controller.close();
+ }
+ }
+ }
+
+ /**
* Only the checks the engine asks by name map onto a Hive access type. Anything else - a requirement put
* together for one statement, for instance - maps to one no Hive policy grants, rather than to a
* neighbouring access type that some policy might.
diff --git a/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogFactory.java b/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogFactory.java
index cf41028..13ffe40 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogFactory.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/datasource/CatalogFactory.java
@@ -192,7 +192,10 @@
catalog.setDefaultPropsIfMissing(false);
catalog.checkWhenCreating();
// This will check if the customized access controller can be created successfully.
- // If failed, it will throw exception and the catalog will not be created.
+ // If failed, it will throw exception and the catalog will not be created. A Ranger source refuses
+ // to be built without its service's policies, so a Ranger admin that cannot be reached and has
+ // left no policy cache fails the CREATE here, with the cause, rather than leaving a catalog
+ // behind that refuses every statement.
try {
catalog.initAccessController(true);
} catch (Throwable e) {
diff --git a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java
index fcb3c04..ce23f27 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/AccessControllerManager.java
@@ -788,8 +788,9 @@
if (isDryRun) {
// CREATE CATALOG validates its authorization properties by building the source and letting it go
// again, which is the only way to find out whether they are usable at all - the properties are the
- // source's to interpret. It is not free: a Ranger source builds a real plugin, policy refresher and
- // download timer included. Letting it go does not necessarily stop it on the spot - a Ranger
+ // source's to interpret - and, for a Ranger source, whether the service's policies can be had, since
+ // it refuses to be built without them. It is not free: a Ranger source builds a real plugin, policy
+ // refresher and download timer included. Letting it go does not necessarily stop it on the spot - a Ranger
// source shares one plugin per service and keeps an unread one for a grace period, so that the
// CREATE this validated for does not immediately rebuild what it just tore down - but a
// validation nothing follows does stop polling once that period is up.
diff --git a/regression-test/data/ranger_p2/test_ranger_group_policy.out b/regression-test/data/ranger_p2/test_ranger_group_policy.out
new file mode 100644
index 0000000..2bf4e79
--- /dev/null
+++ b/regression-test/data/ranger_p2/test_ranger_group_policy.out
@@ -0,0 +1,9 @@
+-- This file is automatically generated. You should know what you did if you want to edit this
+-- !group_select --
+10 XXXXXen10 SampleJ0
+5 XXXXXiv05 SampleE5
+6 XXXXXix06 SampleF6
+7 XXXXXev07 SampleG7
+8 XXXXXig08 SampleH8
+9 XXXXXin09 SampleI9
+
diff --git a/regression-test/framework/src/main/groovy/org/apache/doris/regression/action/HttpCliAction.groovy b/regression-test/framework/src/main/groovy/org/apache/doris/regression/action/HttpCliAction.groovy
index b5b8808..8dab41b 100644
--- a/regression-test/framework/src/main/groovy/org/apache/doris/regression/action/HttpCliAction.groovy
+++ b/regression-test/framework/src/main/groovy/org/apache/doris/regression/action/HttpCliAction.groovy
@@ -25,6 +25,7 @@
import org.apache.http.client.methods.HttpDelete
import org.apache.http.client.methods.HttpGet
import org.apache.http.client.methods.HttpPost
+import org.apache.http.client.methods.HttpPut
import org.apache.http.entity.StringEntity
import org.apache.http.entity.ContentType
import org.apache.http.impl.client.CloseableHttpClient
@@ -152,6 +153,11 @@
HttpDelete httpDelete = new HttpDelete(uri)
headers.each { k, v -> httpDelete.setHeader(k, v) }
result = executeRequest(httpDelete)
+ } else if (op == "put") {
+ HttpPut httpPut = new HttpPut(uri)
+ headers.each { k, v -> httpPut.setHeader(k, v) }
+ httpPut.setEntity(new StringEntity(body, ContentType.APPLICATION_JSON))
+ result = executeRequest(httpPut)
} else {
HttpPost httpPost = new HttpPost(uri)
headers.each { k, v -> httpPost.setHeader(k, v) }
diff --git a/regression-test/plugins/plugin_ranger.groovy b/regression-test/plugins/plugin_ranger.groovy
index 77c4286..d0907d5 100644
--- a/regression-test/plugins/plugin_ranger.groovy
+++ b/regression-test/plugins/plugin_ranger.groovy
@@ -16,6 +16,7 @@
// under the License.
import groovy.json.JsonOutput
+import groovy.json.JsonSlurper
import org.apache.doris.regression.suite.Suite
Suite.metaClass.createRangerUser = { String user, String password, String[] roles ->
@@ -80,7 +81,81 @@
}
}
+// A group in Ranger's own user store: what a policy item is written against so that nobody edits the policy
+// each time somebody joins a team. Doris has no groups, so the ranger-doris source reads a user's groups out of
+// this store; the suites put a user into a group here and write policies against the group only.
+// Tolerates a group that is already there, the way createRangerUser tolerates an existing user.
+Suite.metaClass.createRangerGroup = { String group ->
+ def rangerEndpoint = context.config.otherConfigs.get("rangerEndpoint")
+ def rangerUser = context.config.otherConfigs.get("rangerUser")
+ def rangerPassword = context.config.otherConfigs.get("rangerPassword")
+ def js = new JsonOutput().toJson([
+ "name": "${group}",
+ "description": "${group} desc",
+ "groupType": 1,
+ "groupSource": 0,
+ "isVisible": 1])
+ log.info("create group req: ${js} ".toString())
+ httpTest {
+ basicAuthorization "${rangerUser}","${rangerPassword}"
+ endpoint "${rangerEndpoint}"
+ uri "/service/xusers/secure/groups"
+ body js
+ op "post"
+ check { respCode, body ->
+ log.info("create group resp: ${body} ${respCode}".toString())
+ assertTrue(respCode == 200 || body.contains("Error creating duplicate object"))
+ }
+ }
+}
+
+// Reads one object out of Ranger Admin's user/group API as a map.
+Suite.metaClass.getRangerXObject = { String path ->
+ def rangerEndpoint = context.config.otherConfigs.get("rangerEndpoint")
+ def rangerUser = context.config.otherConfigs.get("rangerUser")
+ def rangerPassword = context.config.otherConfigs.get("rangerPassword")
+ def object = null
+ httpTest {
+ basicAuthorization "${rangerUser}","${rangerPassword}"
+ endpoint "${rangerEndpoint}"
+ header "Accept", "application/json"
+ uri path
+ op "get"
+ check { respCode, body ->
+ assertEquals(200, respCode, "GET ${path}: ${body}")
+ object = new JsonSlurper().parseText(body)
+ }
+ }
+ return object
+}
+
+// Rewrites the groups a Ranger user is in, to exactly `groups`. Goes through the user update that the
+// Ranger UI uses rather than the group-user mapping API, because in Ranger Admin 2.4 only the former bumps
+// the user store version - and the version is what tells the plugins there is a new store to download; a
+// membership changed without it is invisible to every plugin until something else bumps it.
+Suite.metaClass.setRangerUserGroups = { String user, List<String> groups ->
+ def rangerEndpoint = context.config.otherConfigs.get("rangerEndpoint")
+ def rangerUser = context.config.otherConfigs.get("rangerUser")
+ def rangerPassword = context.config.otherConfigs.get("rangerPassword")
+ def vxUser = getRangerXObject("/service/xusers/users/userName/${user}")
+ vxUser.groupIdList = groups.collect { getRangerXObject("/service/xusers/groups/groupName/${it}").id }
+ vxUser.groupNameList = groups
+ def js = new JsonOutput().toJson(vxUser)
+ log.info("set user groups req: ${js} ".toString())
+ httpTest {
+ basicAuthorization "${rangerUser}","${rangerPassword}"
+ endpoint "${rangerEndpoint}"
+ uri "/service/xusers/secure/users/${vxUser.id}"
+ body js
+ op "put"
+ check { respCode, body ->
+ log.info("set user groups resp: ${body} ${respCode}".toString())
+ assertEquals(200, respCode)
+ }
+ }
+}
+
Suite.metaClass.waitPolicyEffect {
sleep(6000)
// TODO: check if policy is effective by API
-}
\ No newline at end of file
+}
diff --git a/regression-test/suites/ranger_p2/test_ranger_group_policy.groovy b/regression-test/suites/ranger_p2/test_ranger_group_policy.groovy
new file mode 100644
index 0000000..83b7dd3
--- /dev/null
+++ b/regression-test/suites/ranger_p2/test_ranger_group_policy.groovy
@@ -0,0 +1,218 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+import org.apache.ranger.RangerClient
+import org.apache.ranger.plugin.model.RangerPolicy
+
+// Policies written against a Ranger GROUP and never against the user: how a deployment runs Ranger so that
+// nobody edits a policy each time somebody joins a team. Doris has no groups of its own, so these apply
+// only because the ranger-doris source reads the user's groups out of Ranger's user store - the store the
+// plugin downloads next to its policies. Every other suite here writes its policy items against a user.
+suite("test_ranger_group_policy", "p2,ranger,external") {
+ String enabled = context.config.otherConfigs.get("enableRangerTest")
+ String rangerEndpoint = context.config.otherConfigs.get("rangerEndpoint")
+ String rangerUser = context.config.otherConfigs.get("rangerUser")
+ String rangerPassword = context.config.otherConfigs.get("rangerPassword")
+ String rangerServiceName = context.config.otherConfigs.get("rangerServiceName")
+
+ if (enabled == null || !enabled.equalsIgnoreCase("true")) {
+ return
+ }
+
+ String db = 'ranger_group_db_1'
+ String table = 'ranger_group_tbl_1'
+ // The same database, no policy of any kind: what the group's SELECT must not open.
+ String otherTable = 'ranger_group_tbl_2'
+ String user = 'ranger_group_user_1'
+ String pwd = 'C123_567p'
+ String group = 'ranger_group_readers_1'
+ String accessPolicy = 'ranger_test_group_access_policy'
+ String rowFilterPolicy = 'ranger_test_group_row_filter_policy'
+ String maskPolicy = 'ranger_test_group_mask_policy'
+ String denyPolicy = 'ranger_test_group_deny_policy'
+ // A table the user is allowed on by name while the group is denied on it; the deny has to win.
+ String deniedTable = 'ranger_group_tbl_3'
+
+ sql """CREATE DATABASE IF NOT EXISTS ${db}"""
+ [table, otherTable, deniedTable].each {
+ sql """DROP TABLE IF EXISTS ${db}.${it}"""
+ sql """CREATE TABLE ${db}.${it} (
+ id BIGINT,
+ c1 VARCHAR(20),
+ c2 VARCHAR(20)
+ )
+ DISTRIBUTED BY HASH(id) BUCKETS 2
+ PROPERTIES (
+ "replication_num" = "1"
+ )"""
+ sql """INSERT INTO ${db}.${it} VALUES
+ (1, 'DataOne01', 'SampleA1'),
+ (2, 'DataTwo02', 'SampleB2'),
+ (3, 'DataThr03', 'SampleC3'),
+ (4, 'DataFou04', 'SampleD4'),
+ (5, 'DataFiv05', 'SampleE5'),
+ (6, 'DataSix06', 'SampleF6'),
+ (7, 'DataSev07', 'SampleG7'),
+ (8, 'DataEig08', 'SampleH8'),
+ (9, 'DataNin09', 'SampleI9'),
+ (10, 'DataTen10', 'SampleJ0')"""
+ }
+ sql """DROP USER IF EXISTS ${user}"""
+ sql """CREATE USER '${user}' IDENTIFIED BY '${pwd}'"""
+
+ // Ranger's side of the same names: the group, the user, and the membership. Nothing syncs users between
+ // Doris and Ranger, so the names are simply kept equal, as the other suites do for users.
+ createRangerGroup(group)
+ createRangerUser(user, pwd, ["ROLE_USER"] as String[])
+ setRangerUserGroups(user, [group])
+
+ RangerClient rangerClient = new RangerClient("http://${rangerEndpoint}", "simple", rangerUser, rangerPassword, null)
+ def dropPolicy = { String name ->
+ try {
+ rangerClient.deletePolicy(rangerServiceName, name)
+ } catch (Exception e) {
+ log.info("Policy not found: ${e.getMessage()}")
+ }
+ }
+ [accessPolicy, rowFilterPolicy, maskPolicy, denyPolicy].each { dropPolicy(it) }
+
+ def resourcesOf = { String tbl, String column ->
+ Map<String, RangerPolicy.RangerPolicyResource> resources = new HashMap<>()
+ resources.put("catalog", new RangerPolicy.RangerPolicyResource("internal"))
+ resources.put("database", new RangerPolicy.RangerPolicyResource(db))
+ resources.put("table", new RangerPolicy.RangerPolicyResource(tbl))
+ if (column != null) {
+ resources.put("column", new RangerPolicy.RangerPolicyResource(column))
+ }
+ return resources
+ }
+ def select = [new RangerPolicy.RangerPolicyItemAccess("SELECT")]
+
+ // Four policies, four Ranger Admin calls, and the FE polls for policies every few seconds: a poll
+ // between two of the calls brings a generation with only the first of them. So the access policy - the
+ // one the wait below watches for - is written last: neither a row filter nor a mask nor a deny grants
+ // anything, and once the FE answers on the access policy it holds the generation the other three are in.
+
+ // 1. A row filter for the group.
+ RangerPolicy policy = new RangerPolicy()
+ policy.setService(rangerServiceName)
+ policy.setName(rowFilterPolicy)
+ policy.setPolicyType(RangerPolicy.POLICY_TYPE_ROWFILTER)
+ policy.setResources(resourcesOf(table, null))
+ RangerPolicy.RangerRowFilterPolicyItem rowFilterItem = new RangerPolicy.RangerRowFilterPolicyItem()
+ rowFilterItem.setGroups([group])
+ rowFilterItem.setAccesses(select)
+ rowFilterItem.setRowFilterInfo(new RangerPolicy.RangerPolicyItemRowFilterInfo("id >= 5"))
+ policy.setRowFilterPolicyItems([rowFilterItem])
+ rangerClient.createPolicy(policy)
+
+ // 2. A column mask for the group.
+ policy = new RangerPolicy()
+ policy.setService(rangerServiceName)
+ policy.setName(maskPolicy)
+ policy.setPolicyType(RangerPolicy.POLICY_TYPE_DATAMASK)
+ policy.setResources(resourcesOf(table, "c1"))
+ RangerPolicy.RangerDataMaskPolicyItem maskItem = new RangerPolicy.RangerDataMaskPolicyItem()
+ maskItem.setGroups([group])
+ maskItem.setAccesses(select)
+ maskItem.setDataMaskInfo(new RangerPolicy.RangerPolicyItemDataMaskInfo("MASK_SHOW_LAST_4", null, null))
+ policy.setDataMaskPolicyItems([maskItem])
+ rangerClient.createPolicy(policy)
+
+ // 3. A deny written against the group, on a table the user is allowed on by name. Before groups were
+ // attached this deny was silently ignored and the user read the table - the dangerous half of the bug.
+ policy = new RangerPolicy()
+ policy.setService(rangerServiceName)
+ policy.setName(denyPolicy)
+ policy.setResources(resourcesOf(deniedTable, null))
+ RangerPolicy.RangerPolicyItem allowUserItem = new RangerPolicy.RangerPolicyItem()
+ allowUserItem.setUsers([user])
+ allowUserItem.setAccesses(select)
+ policy.setPolicyItems([allowUserItem])
+ RangerPolicy.RangerPolicyItem denyGroupItem = new RangerPolicy.RangerPolicyItem()
+ denyGroupItem.setGroups([group])
+ denyGroupItem.setAccesses(select)
+ policy.setDenyPolicyItems([denyGroupItem])
+ rangerClient.createPolicy(policy)
+
+ // 4. Access: SELECT on the table for the group. Last, see above.
+ policy = new RangerPolicy()
+ policy.setService(rangerServiceName)
+ policy.setName(accessPolicy)
+ policy.setResources(resourcesOf(table, null))
+ RangerPolicy.RangerPolicyItem accessItem = new RangerPolicy.RangerPolicyItem()
+ accessItem.setGroups([group])
+ accessItem.setAccesses(select)
+ policy.setPolicyItems([accessItem])
+ rangerClient.createPolicy(policy)
+
+ def tokens = context.config.jdbcUrl.split('/')
+ def defaultJdbcUrl = tokens[0] + "//" + tokens[2] + "/?"
+ def readable = { String tbl ->
+ return connect("${user}", "${pwd}", "${defaultJdbcUrl}") {
+ try {
+ sql """SELECT * FROM internal.${db}.${tbl}"""
+ return true
+ } catch (Exception e) {
+ log.info("not readable yet: ${e.getMessage()}")
+ return false
+ }
+ }
+ }
+
+ // The policies reach the FE within its policy poll interval; the membership reaches it with the next
+ // user store download, which the plugin makes every 60 seconds unless userStoreRefresherPollingInterval
+ // in ranger-doris-security.xml says otherwise. Hence waiting on the effect rather than a fixed sleep -
+ // and on the policy written last, which proves the whole set is there (see above).
+ logger.info("waiting for the group's SELECT to reach ${user}")
+ awaitUntil(180, 3) { readable(table) }
+
+ connect("${user}", "${pwd}", "${defaultJdbcUrl}") {
+ // Visible on the strength of the group's policy alone.
+ def databases = sql """SHOW DATABASES"""
+ assertTrue(databases.any { it[0] == db }, "the database the group may read is not listed")
+ sql """SWITCH internal"""
+ sql """USE ${db}"""
+
+ // Rows 5..10 only, c1 masked down to its last four characters: the row filter and the mask written
+ // against the group both apply, as do those written against a user.
+ order_qt_group_select """SELECT * FROM internal.${db}.${table}"""
+
+ // The group holds SELECT on one table, so the group grants nothing on the next one.
+ test {
+ sql """SELECT * FROM internal.${db}.${otherTable}"""
+ exception "denied"
+ }
+
+ // The deny written against the group outranks the allow written against the user.
+ test {
+ sql """SELECT * FROM internal.${db}.${deniedTable}"""
+ exception "denied"
+ }
+ }
+
+ // Taking the user out of the group takes the access with it, once the plugin has downloaded the change.
+ // Membership is Ranger's alone to keep, which is the whole reason a deployment grants by group.
+ setRangerUserGroups(user, [])
+ logger.info("waiting for ${user} to lose the group's SELECT")
+ awaitUntil(180, 3) { !readable(table) }
+
+ // And the deny written against the group no longer applies either: the user's own allow reads the table.
+ awaitUntil(180, 3) { readable(deniedTable) }
+
+ [accessPolicy, rowFilterPolicy, maskPolicy, denyPolicy].each { dropPolicy(it) }
+}
