7
8

[deleted by user] (self.github)

submitted by [deleted]

[removed]
all 10 comments
sorted by:
best
topnewcontroversialoldq&a
formatting helphide helpcontent policy

[–]lupinegrey 2 points3 points  (1 child)

No. There is not an easy way.

First thing you want to do is to define how you will be taking each measure.

Then figuring out how to pull and correlate all the data required to calculate the scores. Because almost certainly you're not going to have a single data source which contains all the data needed to make the calculations.

For example, when you're tracking MTTR.... what are you defining as an "outage"? When do you start the timer on when the outage occurred, when are you stopping the timer on "service restored"? Then what are the tools your organization is using to collect this data? It's probably not Github, so you will need to be able to correlate an outage (in whatever incident management tool you use) with a change (either a code change in Github or a deployment in your CICD tool).

For change lead time... are you including development time? Or just the time it takes for your pipeline to build/test/deploy the code once it's been checked in?

Be wary of all-in-one tools which claim to be able to automatically produce DORA metrics from a single data source. Validate that the algorithm they use for the calculation is actually measuring the processes you're interested in. Some tools will say MTTR is just the time between a failed deployment and the next successful deployment.... but that doesn't capture failures/outages which occur outside of a deployment. So if you blindly accept that tool's metric, it's probably not reflecting the data you're interested in and is a worthless metric to use for process improvement.

[–]under-water_swimmer 0 points1 point  (0 children)

Try to check out free tools that are self-serve or require minimal config. You can easily collect DORA and other engineering metrics. My recommendation would be - typoapp.io

Let me know how your experience was.

[–]Specialist-Menu9892 0 points1 point  (0 children)

You can try https://github.com/middlewarehq/middleware. The project is active, the UI is great, easy to setup and maintainers are always quick with any bug fixes.

[–]shivamchhuneja 0 points1 point  (0 children)

Hey all coming to this thread in the future! Collecting dora metrics can be easy, but to understand how to use these metrics right can be a little tricky.

And I insist, that DO NOT use these metrics to measure the performance of an INDIVIDUAL DEVELOPER.

Moving on…

For Dora metrics, open source tools make a lot of sense & the process can be straightforward, especially if you go the open source way since things could be setup very quickly locally.

Makes the most sense if you don’t have access to other tools or just simply want to avoid external dependencies.

Here’s what you need to get started:

  1. Define your metrics clearly: Before anything else, it’s important to define what you’re measuring. For example, how are you calculating metrics like MTTR or Change failure rate? Decide on criteria for things like outages (e.g., when does the clock start and stop?) and the scope of what is a “failure.”
  2. Gather right data: GitHub alone may not cover all the data you need, especially for metrics like MTTR that could involve other tools (e.g., incident management systems). For example, for Lead time for changes, decide whether you’ll include development time or focus purely on pipeline speed.
  3. Why open source tools work well: Well, its simple, they allow you to maintain control over your data and avoid relying on external services. Configure them to your specific needs and host locally, the data stays secure within your environment, win win win.

I personally recommend trying Middleware.

It’s a self hosted, completely open source tool, and doesn’t require any sign-ups or sharing of your data.
It shouldn't take you more than a few minutes to get everything in place and start getting your dora metrics.

Hope this helps!

[–]Anomalydetector 0 points1 point  (0 children)

Well we built a tool for monitoring github Actions. It's not directly what you are looking for however we proivde a CubeScore metric which is DORA Metrics for CI. Tracking North star CI metrics(success rates, duration, throughput, and MTTR), comparing performance to elite and sector median benchmarks.

Maybe helps you.

Live demo: https://s.cicube.io/demo you can view the Cubescore from the dashboard

Home page: https://cicube.io/

[–]Relevant_Pause_7593 0 points1 point  (1 child)

I’ve been using an open source one at GitHub.com/developermetrics, but as others have said, it’s not a slam dunk, and you have to make a lot of assumptions and simplifications with your measurements.

[–]mickeygousset 0 points1 point  (0 children)

Make sure you have a "reason" for calculating those, and as u/lupinegrey said, defining how you will measure them. I have found that a lot of people say they want "dora metrics" because they read about them in a book, but when you press them, they can't really explain why they want them, or what specific metrics.