This document describes everything you need to know about DORA, and implementing this powerful and practical framework in DevLake.
Created six years ago by a team of researchers, DORA stands for “DevOps Research & Assessment” and is the answer to years of research, having examined thousands of teams, seeking a reliable and actionable approach to understanding the performance of software development teams.
DORA has since become a standardized framework focused on the stability and velocity of development processes, one that avoids the more controversial aspects of productivity and individual performance measures.
There are two key clusters of data inside DORA: Velocity and Stability. The DORA framework is focused on keeping them in context with each other, as a whole, rather than as independent variables, making the data more challenging to misinterpret or abuse.
Within velocity are two core metrics:
Stability is composed of two core metrics:
To make DORA even more actionable, there are well-established benchmarks to determine if you are performing at “Elite”, “High”, “Medium”, or “Low” levels. Inside DevLake, you will find the benchmarking table available to assess and compare your own projects.
DORA metrics help teams and projects measure and improve software development practices to consistently deliver reliable products, and thus happy users!
You can set up DORA metrics in DevLake in a few steps:
deployments
and incidents
DevLake now supports Jenkins, GitHub Action and GitLabCI as data sources for deployments
data; Jira, GitHub issues, and TAPD as the sources for incidents
data; Github PRs, GitLab MRs as the sources for changes
data.
If your CI/CD tools are not listed on the Supported Data Sources page, have no fear! DevLake provides incoming webhooks to push your deployments
data to DevLake. The webhook configuration doc can be found here.
Let's walk through the DORA implementation process for a team with the following toolchain
Calculating DORA metrics requires three key entities: changes, deployments, and incidents. Their exact definitions of course depend on a team‘s DevOps practice and varies team by team. For the team in this example, let’s assume the following definition:
Crash
or Incident
In the next section, we'll demonstrate how to configure DevLake to implement DORA metrics for the aforementioned example team.
blueprint
Visit the config-ui at http://localhost:4000
Create a project: ‘project1’. Go to ‘project1’ and create a blueprint. Let's name it “Blueprint for DORA”, add a Jira and a GitHub connection. Click Next Step
Select Jira boards and GitHub repos to collect, click Next Step
Click Add Transformation
to configure for DORA metrics
To make it simple, fields with a label are DORA-related configurations for every data source. Via these fields, you can define what are “incidents” and “deployments” for each data source. After all data connections have been configured, click
Next Step
Crash
and Incident
as “incident”, so choose the two types in field “incident”. Jira issues in these two types will be transformed to “incidents” in DevLake.deploy
and build-and-deploy
to deploy, so type in (?i)deploy
to match these jobs. These jobs will be transformed to “deployments” in DevLake. Note: The following example shows where to find GitHub action jobs. It's easy to mix them up with GitHub workflows.
Choose sync frequency, click ‘Save and Run Now’ to start data collection. The time to completion varies by data source and depends on the volume of data.
For more details, please refer to our blueprint manuals.
webhook
Using CircleCI as an example, we demonstrate how to actively push data to DevLake using the Webhook approach, in cases where DevLake doesn't have a plugin specific to that tool to pull data from your data source.
We recommend that you give your webhook connection a unique name so that you can identify and manage where you have used it later.
Create a Project first, choose Incoming Webhooks, then you can Add a Webhook
or Select Existing Webhooks
. And click “Generate POST URL”. DevLake will generate URLs that you can send JSON payloads to push deployments
and incidents
to Devlake. Copy the Deployment
curl command.
Now head to your CircleCI's pipelines page in a new tab. Find your deployment pipeline and click Configuration File
Paste the curl command copied in step 8 to the config.yml
, change the key-values in the payload. See full payload schema here.
version: 2.1 jobs: build: docker: - image: cimg/base:stable steps: - checkout - run: name: "build" command: | echo Hello, World! deploy: docker: - image: cimg/base:stable steps: - checkout - run: name: "deploy" command: | # The time a deploy started start_time=`date '+%Y-%m-%dT%H:%M:%S%z'` # Some deployment tasks here ... echo Hello, World! # Send the request to DevLake after deploy # The values start with a '$CIRCLE_' are CircleCI's built-in variables curl http://127.0.0.1:4000/api/plugins/webhook/1/deployments -X 'POST' -d "{ \"commit_sha\":\"$CIRCLE_SHA1\", \"repo_url\":\"$CIRCLE_REPOSITORY_URL\", \"start_time\":\"$start_time\" }" workflows: build_and_deploy_workflow: jobs: - build - deploy
If you have set a username/password for Config UI, you need to add them to the curl to register a deployment:
curl https://sample-url.com/api/plugins/webhook/1/deployments -X 'POST' -u 'username:password' -d '{ \"commit_sha\":\"$CIRCLE_SHA1\", \"repo_url\":\"$CIRCLE_REPOSITORY_URL\", \"start_time\":\"$start_time\" }'
Run the modified CircleCI pipeline. Check to verify that the request has been successfully sent.
You will find the corresponding deployments
in table.cicd_tasks in DevLake's database.
With all the data collected, DevLake's DORA dashboard is ready to deliver your DORA metrics and benchmarks. You can find the DORA dashboard within the Grafana instance shipped with DevLake, ready for you to put into action.
You can customize the DORA dashboard by editing the underlying SQL query of each panel.
For a breakdown of each metric's SQL query, please refer to the corresponding metric docs:
If you aren't familiar with Grafana, please refer to our Grafana doc, or jump into Slack for help.
:tada::tada::tada: Congratulations! You are now a DevOps Hero, with your own DORA dashboard!
To create the DORA dashboard with your own toolchain, please look at the configuration tutorial for more details.
If you run into any problem, please check the Troubleshooting or create an issue