add support for publishing percentile time series for the histogram m… (#1689)
* add support for publishing percentile time series for the histogram metrics cql-requests, cql-messages and throttling delay.
Motivation:
Histogram metrics is generating too many metrics overloading the promethous servers. if application has 500 Vms
and 1000 cassandra nodes, The histogram metrics generates 100*500*1000 = 50,000,000 time series every 30 seconds.
This is just too much metrics. Let us say we can generate percentile 95 timeseries for for every cassandra nodes,
then we only have 1*500 = 500 metrics and in applciation side, we can ignore the _bucket time series. This way there
will be very less metrics.
Modifications:
add configurable pre-defined percentiles to Micrometer Timer.Builder.publishPercentiles. This change is being added to
cql-requests, cql-messages and throttling delay.
Result:
Based on the configuration, we will see additonal quantile time series for cql-requests, cql-messages and throttling delay
histogram metrics.
* add support for publishing percentile time series for the histogram metrics cql-requests, cql-messages and throttling delay.
Motivation:
Histogram metrics is generating too many metrics overloading the promethous servers. if application has 500 Vms
and 1000 cassandra nodes, The histogram metrics generates 100*500*1000 = 50,000,000 time series every 30 seconds.
This is just too much metrics. Let us say we can generate percentile 95 timeseries for for every cassandra nodes,
then we only have 1*500 = 500 metrics and in applciation side, we can ignore the _bucket time series. This way there
will be very less metrics.
Modifications:
add configurable pre-defined percentiles to Micrometer Timer.Builder.publishPercentiles. This change is being added to
cql-requests, cql-messages and throttling delay.
Result:
Based on the configuration, we will see additonal quantile time series for cql-requests, cql-messages and throttling delay
histogram metrics.
* using helper method as suggested in review
* fixes as per review comments
* add configuration option which switches aggregable histogram generation on/off for all metric flavors [default=on]
* updating java doc
* rename method to publishPercentilesIfDefined
* renmae method
---------
Co-authored-by: Nagappa Paraddi <nparaddi@walmartlabs.com>
diff --git a/core/src/main/java/com/datastax/dse/driver/api/core/config/DseDriverOption.java b/core/src/main/java/com/datastax/dse/driver/api/core/config/DseDriverOption.java
index 74907c1..3ad6ed6 100644
--- a/core/src/main/java/com/datastax/dse/driver/api/core/config/DseDriverOption.java
+++ b/core/src/main/java/com/datastax/dse/driver/api/core/config/DseDriverOption.java
@@ -288,6 +288,34 @@
* <p>Value-type: {@link java.time.Duration Duration}
*/
METRICS_NODE_GRAPH_MESSAGES_SLO("advanced.metrics.node.graph-messages.slo"),
+ /**
+ * Optional list of percentiles to publish for graph-requests metric. Produces an additional time
+ * series for each requested percentile. This percentile is computed locally, and so can't be
+ * aggregated with percentiles computed across other dimensions (e.g. in a different instance).
+ *
+ * <p>Value type: {@link java.util.List List}<{@link Double}>
+ */
+ METRICS_SESSION_GRAPH_REQUESTS_PUBLISH_PERCENTILES(
+ "advanced.metrics.session.graph-requests.publish-percentiles"),
+ /**
+ * Optional list of percentiles to publish for node graph-messages metric. Produces an additional
+ * time series for each requested percentile. This percentile is computed locally, and so can't be
+ * aggregated with percentiles computed across other dimensions (e.g. in a different instance).
+ *
+ * <p>Value type: {@link java.util.List List}<{@link Double}>
+ */
+ METRICS_NODE_GRAPH_MESSAGES_PUBLISH_PERCENTILES(
+ "advanced.metrics.node.graph-messages.publish-percentiles"),
+ /**
+ * Optional list of percentiles to publish for continuous paging requests metric. Produces an
+ * additional time series for each requested percentile. This percentile is computed locally, and
+ * so can't be aggregated with percentiles computed across other dimensions (e.g. in a different
+ * instance).
+ *
+ * <p>Value type: {@link java.util.List List}<{@link Double}>
+ */
+ CONTINUOUS_PAGING_METRICS_SESSION_CQL_REQUESTS_PUBLISH_PERCENTILES(
+ "advanced.metrics.session.continuous-cql-requests.publish-percentiles"),
;
private final String path;
diff --git a/core/src/main/java/com/datastax/oss/driver/api/core/config/DefaultDriverOption.java b/core/src/main/java/com/datastax/oss/driver/api/core/config/DefaultDriverOption.java
index e7e75d9..71d0723 100644
--- a/core/src/main/java/com/datastax/oss/driver/api/core/config/DefaultDriverOption.java
+++ b/core/src/main/java/com/datastax/oss/driver/api/core/config/DefaultDriverOption.java
@@ -939,6 +939,41 @@
* <p>Value-type: List of {@link String}
*/
METADATA_SCHEMA_CHANGE_LISTENER_CLASSES("advanced.schema-change-listener.classes"),
+ /**
+ * Optional list of percentiles to publish for cql-requests metric. Produces an additional time
+ * series for each requested percentile. This percentile is computed locally, and so can't be
+ * aggregated with percentiles computed across other dimensions (e.g. in a different instance).
+ *
+ * <p>Value type: {@link java.util.List List}<{@link Double}>
+ */
+ METRICS_SESSION_CQL_REQUESTS_PUBLISH_PERCENTILES(
+ "advanced.metrics.session.cql-requests.publish-percentiles"),
+ /**
+ * Optional list of percentiles to publish for node cql-messages metric. Produces an additional
+ * time series for each requested percentile. This percentile is computed locally, and so can't be
+ * aggregated with percentiles computed across other dimensions (e.g. in a different instance).
+ *
+ * <p>Value type: {@link java.util.List List}<{@link Double}>
+ */
+ METRICS_NODE_CQL_MESSAGES_PUBLISH_PERCENTILES(
+ "advanced.metrics.node.cql-messages.publish-percentiles"),
+ /**
+ * Optional list of percentiles to publish for throttling delay metric.Produces an additional time
+ * series for each requested percentile. This percentile is computed locally, and so can't be
+ * aggregated with percentiles computed across other dimensions (e.g. in a different instance).
+ *
+ * <p>Value type: {@link java.util.List List}<{@link Double}>
+ */
+ METRICS_SESSION_THROTTLING_PUBLISH_PERCENTILES(
+ "advanced.metrics.session.throttling.delay.publish-percentiles"),
+ /**
+ * Adds histogram buckets used to generate aggregable percentile approximations in monitoring
+ * systems that have query facilities to do so (e.g. Prometheus histogram_quantile, Atlas
+ * percentiles).
+ *
+ * <p>Value-type: boolean
+ */
+ METRICS_GENERATE_AGGREGABLE_HISTOGRAMS("advanced.metrics.histograms.generate-aggregable"),
;
private final String path;
diff --git a/core/src/main/java/com/datastax/oss/driver/api/core/config/OptionsMap.java b/core/src/main/java/com/datastax/oss/driver/api/core/config/OptionsMap.java
index 8f5aa01..2c7a116 100644
--- a/core/src/main/java/com/datastax/oss/driver/api/core/config/OptionsMap.java
+++ b/core/src/main/java/com/datastax/oss/driver/api/core/config/OptionsMap.java
@@ -378,6 +378,7 @@
map.put(TypedDriverOption.COALESCER_INTERVAL, Duration.of(10, ChronoUnit.MICROS));
map.put(TypedDriverOption.LOAD_BALANCING_DC_FAILOVER_MAX_NODES_PER_REMOTE_DC, 0);
map.put(TypedDriverOption.LOAD_BALANCING_DC_FAILOVER_ALLOW_FOR_LOCAL_CONSISTENCY_LEVELS, false);
+ map.put(TypedDriverOption.METRICS_GENERATE_AGGREGABLE_HISTOGRAMS, true);
}
@Immutable
diff --git a/core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java b/core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java
index 2428be0..3f790e9 100644
--- a/core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java
+++ b/core/src/main/java/com/datastax/oss/driver/api/core/config/TypedDriverOption.java
@@ -388,6 +388,10 @@
/** The consistency level to use for trace queries. */
public static final TypedDriverOption<String> REQUEST_TRACE_CONSISTENCY =
new TypedDriverOption<>(DefaultDriverOption.REQUEST_TRACE_CONSISTENCY, GenericType.STRING);
+ /** Whether or not to publish aggregable histogram for metrics */
+ public static final TypedDriverOption<Boolean> METRICS_GENERATE_AGGREGABLE_HISTOGRAMS =
+ new TypedDriverOption<>(
+ DefaultDriverOption.METRICS_GENERATE_AGGREGABLE_HISTOGRAMS, GenericType.BOOLEAN);
/** List of enabled session-level metrics. */
public static final TypedDriverOption<List<String>> METRICS_SESSION_ENABLED =
new TypedDriverOption<>(
@@ -409,6 +413,12 @@
new TypedDriverOption<>(
DefaultDriverOption.METRICS_SESSION_CQL_REQUESTS_SLO,
GenericType.listOf(GenericType.DURATION));
+ /** Optional pre-defined percentile of cql requests to publish, as a list of percentiles . */
+ public static final TypedDriverOption<List<Double>>
+ METRICS_SESSION_CQL_REQUESTS_PUBLISH_PERCENTILES =
+ new TypedDriverOption<>(
+ DefaultDriverOption.METRICS_SESSION_CQL_REQUESTS_PUBLISH_PERCENTILES,
+ GenericType.listOf(GenericType.DOUBLE));
/**
* The number of significant decimal digits to which internal structures will maintain for
* requests.
@@ -433,6 +443,12 @@
new TypedDriverOption<>(
DefaultDriverOption.METRICS_SESSION_THROTTLING_SLO,
GenericType.listOf(GenericType.DURATION));
+ /** Optional pre-defined percentile of throttling delay to publish, as a list of percentiles . */
+ public static final TypedDriverOption<List<Double>>
+ METRICS_SESSION_THROTTLING_PUBLISH_PERCENTILES =
+ new TypedDriverOption<>(
+ DefaultDriverOption.METRICS_SESSION_THROTTLING_PUBLISH_PERCENTILES,
+ GenericType.listOf(GenericType.DOUBLE));
/**
* The number of significant decimal digits to which internal structures will maintain for
* throttling.
@@ -457,6 +473,12 @@
new TypedDriverOption<>(
DefaultDriverOption.METRICS_NODE_CQL_MESSAGES_SLO,
GenericType.listOf(GenericType.DURATION));
+ /** Optional pre-defined percentile of node cql messages to publish, as a list of percentiles . */
+ public static final TypedDriverOption<List<Double>>
+ METRICS_NODE_CQL_MESSAGES_PUBLISH_PERCENTILES =
+ new TypedDriverOption<>(
+ DefaultDriverOption.METRICS_NODE_CQL_MESSAGES_PUBLISH_PERCENTILES,
+ GenericType.listOf(GenericType.DOUBLE));
/**
* The number of significant decimal digits to which internal structures will maintain for
* requests.
@@ -701,6 +723,15 @@
DseDriverOption.CONTINUOUS_PAGING_METRICS_SESSION_CQL_REQUESTS_SLO,
GenericType.listOf(GenericType.DURATION));
/**
+ * Optional pre-defined percentile of continuous paging cql requests to publish, as a list of
+ * percentiles .
+ */
+ public static final TypedDriverOption<List<Double>>
+ CONTINUOUS_PAGING_METRICS_SESSION_CQL_REQUESTS_PUBLISH_PERCENTILES =
+ new TypedDriverOption<>(
+ DseDriverOption.CONTINUOUS_PAGING_METRICS_SESSION_CQL_REQUESTS_PUBLISH_PERCENTILES,
+ GenericType.listOf(GenericType.DOUBLE));
+ /**
* The number of significant decimal digits to which internal structures will maintain for
* continuous requests.
*/
@@ -774,6 +805,12 @@
new TypedDriverOption<>(
DseDriverOption.METRICS_SESSION_GRAPH_REQUESTS_SLO,
GenericType.listOf(GenericType.DURATION));
+ /** Optional pre-defined percentile of graph requests to publish, as a list of percentiles . */
+ public static final TypedDriverOption<List<Double>>
+ METRICS_SESSION_GRAPH_REQUESTS_PUBLISH_PERCENTILES =
+ new TypedDriverOption<>(
+ DseDriverOption.METRICS_SESSION_GRAPH_REQUESTS_PUBLISH_PERCENTILES,
+ GenericType.listOf(GenericType.DOUBLE));
/**
* The number of significant decimal digits to which internal structures will maintain for graph
* requests.
@@ -799,6 +836,14 @@
DseDriverOption.METRICS_NODE_GRAPH_MESSAGES_SLO,
GenericType.listOf(GenericType.DURATION));
/**
+ * Optional pre-defined percentile of node graph requests to publish, as a list of percentiles .
+ */
+ public static final TypedDriverOption<List<Double>>
+ METRICS_NODE_GRAPH_MESSAGES_PUBLISH_PERCENTILES =
+ new TypedDriverOption<>(
+ DseDriverOption.METRICS_NODE_GRAPH_MESSAGES_PUBLISH_PERCENTILES,
+ GenericType.listOf(GenericType.DOUBLE));
+ /**
* The number of significant decimal digits to which internal structures will maintain for graph
* requests.
*/
diff --git a/core/src/main/resources/reference.conf b/core/src/main/resources/reference.conf
index ee83280..4c58a16 100644
--- a/core/src/main/resources/reference.conf
+++ b/core/src/main/resources/reference.conf
@@ -1434,6 +1434,16 @@
// prefix = "cassandra"
}
+ histograms {
+ # Adds histogram buckets used to generate aggregable percentile approximations in monitoring
+ # systems that have query facilities to do so (e.g. Prometheus histogram_quantile, Atlas percentiles).
+ #
+ # Required: no
+ # Modifiable at runtime: no
+ # Overridable in a profile: no
+ generate-aggregable = true
+ }
+
# The session-level metrics (all disabled by default).
#
# Required: yes
@@ -1526,7 +1536,7 @@
# Modifiable at runtime: no
# Overridable in a profile: no
cql-requests {
-
+
# The largest latency that we expect to record.
#
# This should be slightly higher than request.timeout (in theory, readings can't be higher
@@ -1569,7 +1579,7 @@
# time).
# Valid for: Dropwizard.
refresh-interval = 5 minutes
-
+
# An optional list of latencies to track as part of the application's service-level
# objectives (SLOs).
#
@@ -1577,7 +1587,11 @@
# buckets used to generate aggregable percentile approximations.
# Valid for: Micrometer.
// slo = [ 100 milliseconds, 500 milliseconds, 1 second ]
-
+
+ # An optional list of percentiles to be published by Micrometer. Produces an additional time series for each requested percentile.
+ # This percentile is computed locally, and so can't be aggregated with percentiles computed across other dimensions (e.g. in a different instance)
+ # Valid for: Micrometer.
+ // publish-percentiles = [ 0.75, 0.95, 0.99 ]
}
# Required: if the 'throttling.delay' metric is enabled, and Dropwizard or Micrometer is used.
@@ -1589,6 +1603,7 @@
significant-digits = 3
refresh-interval = 5 minutes
// slo = [ 100 milliseconds, 500 milliseconds, 1 second ]
+ // publish-percentiles = [ 0.75, 0.95, 0.99 ]
}
# Required: if the 'continuous-cql-requests' metric is enabled, and Dropwizard or Micrometer
@@ -1601,6 +1616,7 @@
significant-digits = 3
refresh-interval = 5 minutes
// slo = [ 100 milliseconds, 500 milliseconds, 1 second ]
+ // publish-percentiles = [ 0.75, 0.95, 0.99 ]
}
# Required: if the 'graph-requests' metric is enabled, and Dropwizard or Micrometer is used.
@@ -1612,6 +1628,7 @@
significant-digits = 3
refresh-interval = 5 minutes
// slo = [ 100 milliseconds, 500 milliseconds, 1 second ]
+ // publish-percentiles = [ 0.75, 0.95, 0.99 ]
}
}
# The node-level metrics (all disabled by default).
@@ -1776,6 +1793,7 @@
significant-digits = 3
refresh-interval = 5 minutes
// slo = [ 100 milliseconds, 500 milliseconds, 1 second ]
+ // publish-percentiles = [ 0.75, 0.95, 0.99 ]
}
# See graph-requests in the `session` section
@@ -1789,6 +1807,7 @@
significant-digits = 3
refresh-interval = 5 minutes
// slo = [ 100 milliseconds, 500 milliseconds, 1 second ]
+ // publish-percentiles = [ 0.75, 0.95, 0.99 ]
}
# The time after which the node level metrics will be evicted.
diff --git a/metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerMetricUpdater.java b/metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerMetricUpdater.java
index c30dcc1..4785da1 100644
--- a/metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerMetricUpdater.java
+++ b/metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerMetricUpdater.java
@@ -15,7 +15,9 @@
*/
package com.datastax.oss.driver.internal.metrics.micrometer;
+import com.datastax.oss.driver.api.core.config.DefaultDriverOption;
import com.datastax.oss.driver.api.core.config.DriverExecutionProfile;
+import com.datastax.oss.driver.api.core.config.DriverOption;
import com.datastax.oss.driver.internal.core.context.InternalDriverContext;
import com.datastax.oss.driver.internal.core.metrics.AbstractMetricUpdater;
import com.datastax.oss.driver.internal.core.metrics.MetricId;
@@ -27,6 +29,7 @@
import io.micrometer.core.instrument.MeterRegistry;
import io.micrometer.core.instrument.Tag;
import io.micrometer.core.instrument.Timer;
+import java.util.List;
import java.util.Set;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.ConcurrentMap;
@@ -151,12 +154,31 @@
}
protected Timer.Builder configureTimer(Timer.Builder builder, MetricT metric, MetricId id) {
- return builder.publishPercentileHistogram();
+ DriverExecutionProfile profile = context.getConfig().getDefaultProfile();
+ if (profile.getBoolean(DefaultDriverOption.METRICS_GENERATE_AGGREGABLE_HISTOGRAMS)) {
+ builder.publishPercentileHistogram();
+ }
+ return builder;
}
@SuppressWarnings("unused")
protected DistributionSummary.Builder configureDistributionSummary(
DistributionSummary.Builder builder, MetricT metric, MetricId id) {
- return builder.publishPercentileHistogram();
+ DriverExecutionProfile profile = context.getConfig().getDefaultProfile();
+ if (profile.getBoolean(DefaultDriverOption.METRICS_GENERATE_AGGREGABLE_HISTOGRAMS)) {
+ builder.publishPercentileHistogram();
+ }
+ return builder;
+ }
+
+ static double[] toDoubleArray(List<Double> doubleList) {
+ return doubleList.stream().mapToDouble(Double::doubleValue).toArray();
+ }
+
+ static void configurePercentilesPublishIfDefined(
+ Timer.Builder builder, DriverExecutionProfile profile, DriverOption driverOption) {
+ if (profile.isDefined(driverOption)) {
+ builder.publishPercentiles(toDoubleArray(profile.getDoubleList(driverOption)));
+ }
}
}
diff --git a/metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerNodeMetricUpdater.java b/metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerNodeMetricUpdater.java
index 0f5dada..d6359bc 100644
--- a/metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerNodeMetricUpdater.java
+++ b/metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerNodeMetricUpdater.java
@@ -96,9 +96,9 @@
@Override
protected Timer.Builder configureTimer(Timer.Builder builder, NodeMetric metric, MetricId id) {
DriverExecutionProfile profile = context.getConfig().getDefaultProfile();
+ super.configureTimer(builder, metric, id);
if (metric == DefaultNodeMetric.CQL_MESSAGES) {
- return builder
- .publishPercentileHistogram()
+ builder
.minimumExpectedValue(
profile.getDuration(DefaultDriverOption.METRICS_NODE_CQL_MESSAGES_LOWEST))
.maximumExpectedValue(
@@ -111,9 +111,11 @@
: null)
.percentilePrecision(
profile.getInt(DefaultDriverOption.METRICS_NODE_CQL_MESSAGES_DIGITS));
+
+ configurePercentilesPublishIfDefined(
+ builder, profile, DefaultDriverOption.METRICS_NODE_CQL_MESSAGES_PUBLISH_PERCENTILES);
} else if (metric == DseNodeMetric.GRAPH_MESSAGES) {
- return builder
- .publishPercentileHistogram()
+ builder
.minimumExpectedValue(
profile.getDuration(DseDriverOption.METRICS_NODE_GRAPH_MESSAGES_LOWEST))
.maximumExpectedValue(
@@ -125,7 +127,10 @@
.toArray(new Duration[0])
: null)
.percentilePrecision(profile.getInt(DseDriverOption.METRICS_NODE_GRAPH_MESSAGES_DIGITS));
+
+ configurePercentilesPublishIfDefined(
+ builder, profile, DseDriverOption.METRICS_NODE_GRAPH_MESSAGES_PUBLISH_PERCENTILES);
}
- return super.configureTimer(builder, metric, id);
+ return builder;
}
}
diff --git a/metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerSessionMetricUpdater.java b/metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerSessionMetricUpdater.java
index bb361b8..f9387f1 100644
--- a/metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerSessionMetricUpdater.java
+++ b/metrics/micrometer/src/main/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerSessionMetricUpdater.java
@@ -63,9 +63,9 @@
@Override
protected Timer.Builder configureTimer(Timer.Builder builder, SessionMetric metric, MetricId id) {
DriverExecutionProfile profile = context.getConfig().getDefaultProfile();
+ super.configureTimer(builder, metric, id);
if (metric == DefaultSessionMetric.CQL_REQUESTS) {
- return builder
- .publishPercentileHistogram()
+ builder
.minimumExpectedValue(
profile.getDuration(DefaultDriverOption.METRICS_SESSION_CQL_REQUESTS_LOWEST))
.maximumExpectedValue(
@@ -80,9 +80,11 @@
profile.isDefined(DefaultDriverOption.METRICS_SESSION_CQL_REQUESTS_DIGITS)
? profile.getInt(DefaultDriverOption.METRICS_SESSION_CQL_REQUESTS_DIGITS)
: null);
+
+ configurePercentilesPublishIfDefined(
+ builder, profile, DefaultDriverOption.METRICS_SESSION_CQL_REQUESTS_PUBLISH_PERCENTILES);
} else if (metric == DefaultSessionMetric.THROTTLING_DELAY) {
- return builder
- .publishPercentileHistogram()
+ builder
.minimumExpectedValue(
profile.getDuration(DefaultDriverOption.METRICS_SESSION_THROTTLING_LOWEST))
.maximumExpectedValue(
@@ -97,9 +99,11 @@
profile.isDefined(DefaultDriverOption.METRICS_SESSION_THROTTLING_DIGITS)
? profile.getInt(DefaultDriverOption.METRICS_SESSION_THROTTLING_DIGITS)
: null);
+
+ configurePercentilesPublishIfDefined(
+ builder, profile, DefaultDriverOption.METRICS_SESSION_THROTTLING_PUBLISH_PERCENTILES);
} else if (metric == DseSessionMetric.CONTINUOUS_CQL_REQUESTS) {
- return builder
- .publishPercentileHistogram()
+ builder
.minimumExpectedValue(
profile.getDuration(
DseDriverOption.CONTINUOUS_PAGING_METRICS_SESSION_CQL_REQUESTS_LOWEST))
@@ -119,9 +123,13 @@
? profile.getInt(
DseDriverOption.CONTINUOUS_PAGING_METRICS_SESSION_CQL_REQUESTS_DIGITS)
: null);
+
+ configurePercentilesPublishIfDefined(
+ builder,
+ profile,
+ DseDriverOption.CONTINUOUS_PAGING_METRICS_SESSION_CQL_REQUESTS_PUBLISH_PERCENTILES);
} else if (metric == DseSessionMetric.GRAPH_REQUESTS) {
- return builder
- .publishPercentileHistogram()
+ builder
.minimumExpectedValue(
profile.getDuration(DseDriverOption.METRICS_SESSION_GRAPH_REQUESTS_LOWEST))
.maximumExpectedValue(
@@ -136,7 +144,10 @@
profile.isDefined(DseDriverOption.METRICS_SESSION_GRAPH_REQUESTS_DIGITS)
? profile.getInt(DseDriverOption.METRICS_SESSION_GRAPH_REQUESTS_DIGITS)
: null);
+
+ configurePercentilesPublishIfDefined(
+ builder, profile, DseDriverOption.METRICS_SESSION_GRAPH_REQUESTS_PUBLISH_PERCENTILES);
}
- return super.configureTimer(builder, metric, id);
+ return builder;
}
}
diff --git a/metrics/micrometer/src/test/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerNodeMetricUpdaterTest.java b/metrics/micrometer/src/test/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerNodeMetricUpdaterTest.java
index fbdfb7b..e5482aa 100644
--- a/metrics/micrometer/src/test/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerNodeMetricUpdaterTest.java
+++ b/metrics/micrometer/src/test/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerNodeMetricUpdaterTest.java
@@ -152,7 +152,8 @@
DriverOption lowest,
DriverOption highest,
DriverOption digits,
- DriverOption sla) {
+ DriverOption sla,
+ DriverOption percentiles) {
// given
Node node = mock(Node.class);
InternalDriverContext context = mock(InternalDriverContext.class);
@@ -174,6 +175,8 @@
when(profile.isDefined(sla)).thenReturn(true);
when(profile.getDurationList(sla))
.thenReturn(Arrays.asList(Duration.ofMillis(100), Duration.ofMillis(500)));
+ when(profile.isDefined(percentiles)).thenReturn(true);
+ when(profile.getDoubleList(percentiles)).thenReturn(Arrays.asList(0.75, 0.95, 0.99));
when(generator.nodeMetricId(node, metric)).thenReturn(METRIC_ID);
SimpleMeterRegistry registry = spy(new SimpleMeterRegistry());
@@ -190,6 +193,63 @@
assertThat(timer.count()).isEqualTo(10);
HistogramSnapshot snapshot = timer.takeSnapshot();
assertThat(snapshot.histogramCounts()).hasSize(2);
+ assertThat(snapshot.percentileValues()).hasSize(3);
+ assertThat(snapshot.percentileValues())
+ .satisfiesExactlyInAnyOrder(
+ valuePercentile -> assertThat(valuePercentile.percentile()).isEqualTo(0.75),
+ valuePercentile -> assertThat(valuePercentile.percentile()).isEqualTo(0.95),
+ valuePercentile -> assertThat(valuePercentile.percentile()).isEqualTo(0.99));
+ }
+
+ @Test
+ @UseDataProvider(value = "timerMetrics")
+ public void should_not_create_sla_percentiles(
+ NodeMetric metric,
+ DriverOption lowest,
+ DriverOption highest,
+ DriverOption digits,
+ DriverOption sla,
+ DriverOption percentiles) {
+ // given
+ Node node = mock(Node.class);
+ InternalDriverContext context = mock(InternalDriverContext.class);
+ DriverExecutionProfile profile = mock(DriverExecutionProfile.class);
+ DriverConfig config = mock(DriverConfig.class);
+ MetricIdGenerator generator = mock(MetricIdGenerator.class);
+ Set<NodeMetric> enabledMetrics = Collections.singleton(metric);
+
+ // when
+ when(context.getSessionName()).thenReturn("prefix");
+ when(context.getConfig()).thenReturn(config);
+ when(config.getDefaultProfile()).thenReturn(profile);
+ when(context.getMetricIdGenerator()).thenReturn(generator);
+ when(profile.getDuration(DefaultDriverOption.METRICS_NODE_EXPIRE_AFTER))
+ .thenReturn(Duration.ofHours(1));
+ when(profile.getDuration(lowest)).thenReturn(Duration.ofMillis(10));
+ when(profile.getDuration(highest)).thenReturn(Duration.ofSeconds(1));
+ when(profile.getInt(digits)).thenReturn(5);
+ when(profile.isDefined(sla)).thenReturn(false);
+ when(profile.getDurationList(sla))
+ .thenReturn(Arrays.asList(Duration.ofMillis(100), Duration.ofMillis(500)));
+ when(profile.isDefined(percentiles)).thenReturn(false);
+ when(profile.getDoubleList(percentiles)).thenReturn(Arrays.asList(0.75, 0.95, 0.99));
+ when(generator.nodeMetricId(node, metric)).thenReturn(METRIC_ID);
+
+ SimpleMeterRegistry registry = spy(new SimpleMeterRegistry());
+ MicrometerNodeMetricUpdater updater =
+ new MicrometerNodeMetricUpdater(node, context, enabledMetrics, registry);
+
+ for (int i = 0; i < 10; i++) {
+ updater.updateTimer(metric, null, 100, TimeUnit.MILLISECONDS);
+ }
+
+ // then
+ Timer timer = registry.find(METRIC_ID.getName()).timer();
+ assertThat(timer).isNotNull();
+ assertThat(timer.count()).isEqualTo(10);
+ HistogramSnapshot snapshot = timer.takeSnapshot();
+ assertThat(snapshot.histogramCounts()).hasSize(0);
+ assertThat(snapshot.percentileValues()).hasSize(0);
}
@DataProvider
@@ -201,6 +261,7 @@
DefaultDriverOption.METRICS_NODE_CQL_MESSAGES_HIGHEST,
DefaultDriverOption.METRICS_NODE_CQL_MESSAGES_DIGITS,
DefaultDriverOption.METRICS_NODE_CQL_MESSAGES_SLO,
+ DefaultDriverOption.METRICS_NODE_CQL_MESSAGES_PUBLISH_PERCENTILES,
},
{
DseNodeMetric.GRAPH_MESSAGES,
@@ -208,6 +269,7 @@
DseDriverOption.METRICS_NODE_GRAPH_MESSAGES_HIGHEST,
DseDriverOption.METRICS_NODE_GRAPH_MESSAGES_DIGITS,
DseDriverOption.METRICS_NODE_GRAPH_MESSAGES_SLO,
+ DseDriverOption.METRICS_NODE_GRAPH_MESSAGES_PUBLISH_PERCENTILES,
},
};
}
diff --git a/metrics/micrometer/src/test/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerSessionMetricUpdaterTest.java b/metrics/micrometer/src/test/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerSessionMetricUpdaterTest.java
index 09b3e44..1e2d210 100644
--- a/metrics/micrometer/src/test/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerSessionMetricUpdaterTest.java
+++ b/metrics/micrometer/src/test/java/com/datastax/oss/driver/internal/metrics/micrometer/MicrometerSessionMetricUpdaterTest.java
@@ -59,7 +59,8 @@
DriverOption lowest,
DriverOption highest,
DriverOption digits,
- DriverOption sla) {
+ DriverOption sla,
+ DriverOption percentiles) {
// given
InternalDriverContext context = mock(InternalDriverContext.class);
DriverExecutionProfile profile = mock(DriverExecutionProfile.class);
@@ -80,6 +81,8 @@
when(profile.isDefined(sla)).thenReturn(true);
when(profile.getDurationList(sla))
.thenReturn(Arrays.asList(Duration.ofMillis(100), Duration.ofMillis(500)));
+ when(profile.isDefined(percentiles)).thenReturn(true);
+ when(profile.getDoubleList(percentiles)).thenReturn(Arrays.asList(0.75, 0.95, 0.99));
when(generator.sessionMetricId(metric)).thenReturn(METRIC_ID);
SimpleMeterRegistry registry = spy(new SimpleMeterRegistry());
@@ -96,6 +99,61 @@
assertThat(timer.count()).isEqualTo(10);
HistogramSnapshot snapshot = timer.takeSnapshot();
assertThat(snapshot.histogramCounts()).hasSize(2);
+ assertThat(snapshot.percentileValues()).hasSize(3);
+ assertThat(snapshot.percentileValues())
+ .satisfiesExactlyInAnyOrder(
+ valuePercentile -> assertThat(valuePercentile.percentile()).isEqualTo(0.75),
+ valuePercentile -> assertThat(valuePercentile.percentile()).isEqualTo(0.95),
+ valuePercentile -> assertThat(valuePercentile.percentile()).isEqualTo(0.99));
+ }
+
+ @Test
+ @UseDataProvider(value = "timerMetrics")
+ public void should_not_create_sla_percentiles(
+ SessionMetric metric,
+ DriverOption lowest,
+ DriverOption highest,
+ DriverOption digits,
+ DriverOption sla,
+ DriverOption percentiles) {
+ // given
+ InternalDriverContext context = mock(InternalDriverContext.class);
+ DriverExecutionProfile profile = mock(DriverExecutionProfile.class);
+ DriverConfig config = mock(DriverConfig.class);
+ MetricIdGenerator generator = mock(MetricIdGenerator.class);
+ Set<SessionMetric> enabledMetrics = Collections.singleton(metric);
+
+ // when
+ when(context.getSessionName()).thenReturn("prefix");
+ when(context.getConfig()).thenReturn(config);
+ when(config.getDefaultProfile()).thenReturn(profile);
+ when(context.getMetricIdGenerator()).thenReturn(generator);
+ when(profile.getDuration(DefaultDriverOption.METRICS_NODE_EXPIRE_AFTER))
+ .thenReturn(Duration.ofHours(1));
+ when(profile.isDefined(sla)).thenReturn(false);
+ when(profile.getDurationList(sla))
+ .thenReturn(Arrays.asList(Duration.ofMillis(100), Duration.ofMillis(500)));
+ when(profile.getBoolean(DefaultDriverOption.METRICS_GENERATE_AGGREGABLE_HISTOGRAMS))
+ .thenReturn(true);
+ when(profile.isDefined(percentiles)).thenReturn(false);
+ when(profile.getDoubleList(percentiles)).thenReturn(Arrays.asList(0.75, 0.95, 0.99));
+ when(generator.sessionMetricId(metric)).thenReturn(METRIC_ID);
+
+ SimpleMeterRegistry registry = new SimpleMeterRegistry();
+ MicrometerSessionMetricUpdater updater =
+ new MicrometerSessionMetricUpdater(context, enabledMetrics, registry);
+
+ for (int i = 0; i < 10; i++) {
+ updater.updateTimer(metric, null, 100, TimeUnit.MILLISECONDS);
+ }
+
+ // then
+ Timer timer = registry.find(METRIC_ID.getName()).timer();
+ assertThat(timer).isNotNull();
+ assertThat(timer.count()).isEqualTo(10);
+ HistogramSnapshot snapshot = timer.takeSnapshot();
+ assertThat(snapshot.histogramCounts()).hasSize(0);
+ assertThat(snapshot.percentileValues()).hasSize(0);
}
@DataProvider
@@ -107,6 +165,7 @@
DefaultDriverOption.METRICS_SESSION_CQL_REQUESTS_HIGHEST,
DefaultDriverOption.METRICS_SESSION_CQL_REQUESTS_DIGITS,
DefaultDriverOption.METRICS_SESSION_CQL_REQUESTS_SLO,
+ DefaultDriverOption.METRICS_SESSION_CQL_REQUESTS_PUBLISH_PERCENTILES,
},
{
DseSessionMetric.GRAPH_REQUESTS,
@@ -114,6 +173,7 @@
DseDriverOption.METRICS_SESSION_GRAPH_REQUESTS_HIGHEST,
DseDriverOption.METRICS_SESSION_GRAPH_REQUESTS_DIGITS,
DseDriverOption.METRICS_SESSION_GRAPH_REQUESTS_SLO,
+ DseDriverOption.METRICS_SESSION_GRAPH_REQUESTS_PUBLISH_PERCENTILES,
},
{
DseSessionMetric.CONTINUOUS_CQL_REQUESTS,
@@ -121,6 +181,7 @@
DseDriverOption.CONTINUOUS_PAGING_METRICS_SESSION_CQL_REQUESTS_HIGHEST,
DseDriverOption.CONTINUOUS_PAGING_METRICS_SESSION_CQL_REQUESTS_DIGITS,
DseDriverOption.CONTINUOUS_PAGING_METRICS_SESSION_CQL_REQUESTS_SLO,
+ DseDriverOption.CONTINUOUS_PAGING_METRICS_SESSION_CQL_REQUESTS_PUBLISH_PERCENTILES
},
{
DefaultSessionMetric.THROTTLING_DELAY,
@@ -128,6 +189,7 @@
DefaultDriverOption.METRICS_SESSION_THROTTLING_HIGHEST,
DefaultDriverOption.METRICS_SESSION_THROTTLING_DIGITS,
DefaultDriverOption.METRICS_SESSION_THROTTLING_SLO,
+ DefaultDriverOption.METRICS_SESSION_THROTTLING_PUBLISH_PERCENTILES
},
};
}
