Otava requires Python 3.10 or later.
pip install apache-otava
This installs the Otava library and CLI with CSV, JSON, and Graphite support. Install the extra for any additional service you use, for example:
pip install 'apache-otava[postgres]' pip install 'apache-otava[bigquery,slack]'
See Installation for the complete list of extras.
or
docker pull apache/otava
The Docker image includes all optional integrations.
By default, Otava reads configuration from ~/.otava/otava.yaml. Create that file and add the data sources and tests you want to analyze.
To run the bundled CSV example from a source checkout, use its local configuration while running the commands in this guide:
export OTAVA_CONFIG=examples/csv/config/otava-local.yaml
[!TIP] See docs on specific data sources to learn more about their configuration - CSV, Graphite, PostgreSQL, or BigQuery.
Alternatively, it is possible to leave the config file as is, and provide credentials in the environment by setting appropriate environment variables. Environment variables are interpolated before interpreting the configuration file.
All test configurations are defined in the main configuration file. Otava supports reading data from and publishing results to a CSV file, Graphite, PostgreSQL, and BigQuery.
Tests are defined in the tests section. For example, the following definition will import results of the test from a CSV file:
tests: local.sample: type: csv file: tests/resources/sample.csv time_column: time metrics: [metric1, metric2] attributes: [commit] csv_options: delimiter: "," quote_char: "'"
The time_column property points to the name of the column storing the timestamp of each test-run. The data points will be ordered by that column.
The metrics property selects the columns that hold the values to be analyzed. These values must be numbers convertible to floats. The metrics property can be not only a simple list of column names, but it can also be a dictionary configuring other properties of each metric, the column name or direction:
metrics: resp_time_p99: direction: -1 column: p99
Direction can be 1 or -1. If direction is set to 1, this means that the higher the metric, the better the performance is. If it is set to -1, higher values mean worse performance.
The attributes property describes any other columns that should be attached to the final report. Special attribute version and commit can be used to query for a given time-range.
[!TIP] To learn how to avoid repeating the same configuration in multiple tests, see Avoiding test definition duplication.
otava list-groups otava list-tests [group name]
To list all available metrics defined for the test:
otava list-metrics <test>
[!TIP] For more details, see Finding Change Points and Validating Performance of a Feature Branch.
otava analyze <test>... otava analyze <group>...
This command prints interesting results of all runs of the test and a list of change-points.
A change-point is a moment when a metric value starts to differ significantly from the values of the earlier runs and when the difference is statistically significant.
Otava calculates the probability (P-value) that the change point was not caused by chance - the closer to zero, the more certain it is about the regression or performance improvement. The smaller the magnitude of the change, the more data points are needed to confirm the change, therefore Otava may not notice the regression immediately after the first run that regressed.
The analyze command accepts multiple tests or test groups. The results are simply concatenated.
$ otava analyze local.sample --since=2026-01-01T00:00:00Z INFO: Computing change points for test local.sample... time commit metric1 metric2 ------------------------- -------- --------- --------- 2026-01-01 02:00:00 +0000 aaa0 154023 10.43 2026-01-02 02:00:00 +0000 aaa1 138455 10.23 2026-01-03 02:00:00 +0000 aaa2 143112 10.29 2026-01-04 02:00:00 +0000 aaa3 149190 10.91 2026-01-05 02:00:00 +0000 aaa4 132098 10.34 2026-01-06 02:00:00 +0000 aaa5 151344 10.69 ········· -12.9% ········· 2026-01-07 02:00:00 +0000 aaa6 155145 9.23 2026-01-08 02:00:00 +0000 aaa7 148889 9.11 2026-01-09 02:00:00 +0000 aaa8 149466 9.13 2026-01-10 02:00:00 +0000 aaa9 148209 9.03