The Challenges in Adopting DORA Metrics for DevOps success
The content here is under the Attribution 4.0 International (CC BY 4.0) license
The advent of DevOps changed how businessess face the integration between developers and operations leading to a shift in the cultural mindset of organizations focusing on the outcome and collaboration to provide customers an incresed outcome. As such, new challenges have arise in the approach of getting the metrics and adjusting the direction of the software development challenges.
There are two main points that we can refer to in the history of the devops history, the first was the launch of Accelerate, the book that became a reference in he field and he second one was the launch of The Phoenix Project another book that expose through examples how the day-to-day of bubinsess and its limitations with operations were, through a history the three ways of working were introduced and quicly beacme the another reference in the field.
The foundaton laied by those two examples and many others left the software industry with a sense that more could be achieved and better outcomes could be achieved through the change in the ways of working as well as adopting the 4 key metrics.
The metrics
The metrics Deployment Frequency, Change Failure Rate, Mean Time to Restore and Leat Time were the metrics used to differentiate what sucessful business were achieving in comparison with less successfull businessess. Those metrics accordinly to accelerate were devireved from the capabilities of softare development teams to deliver working software. Those turns out to be the four metrics that differentiates them.
Where the metrics comes from?
The starting point for the metrics is described in the book Accelerate, the methodology used and the data gathered to support those are described there.
Once those metrics were stabilished they bcome quicly the ideal metrics to me measured and as such, the desire to automate those also increased. In that advent, open source projects that offered those features became popular.
- Metrik - A cross-platform measurement tool that pulls data out of CD pipelines and analysis the four key metrics.
- FourKeys - Platform for monitoring the four key software delivery metrics of software delivery
Despite efforts in the adoption of automated tracking for the metrics, the challenge is a mix of infrastructure and business. It is possible to automate part of it using open source tools, however, it will be most likely that some adjustments will be needed as each business operates in a different fashion.
The ways of working
Not only the changing in the metrics disrrupted the software industry but the ways of working also suffered a mutation to adapt in an attempt to deliver value for customer faster. Adpting to a single framework (like Scrum) does not provide the benefits and the guidelines required to achieve such flow. There is a need to incorporate new practices that will allow such flow to be introduced in the value stream of the software delivery.
- Test-Driven Development
- Postmortem analysis
- Hands-on infrastructure
The practices in addition to the agile framework of choice will also provide the foundation that is needed to achieve the differentiator for business when comparing them to the competition.
The challenges
Despite of automation the challenges on the technical aspect of it are blurry for newcomers in this process, as you might have realised, the promise is big and the desired outcomes is even bigger promsing high stakes for business once they adopt it. However, in the details is where the challenges become interesting. Each of the metrics are focuses on different aspects of the technological stack leading to the need of integrated tools that can collect the metrics. Despite efforts on the open source projects, what is found is a lack of stadandrized fashion to collect the metrics due to the segmented solutions that software has.
The metrics rely on pipelines, cloud providers and code control version (at a minimum), for each of those, many providers exists and for each project in the industry the mix of teachnologies is unique.
Adopt a pipeline
The first challenge to be noted is that, if you are not using a CI/CD for software, this should be the first step, regardless of the technicalogical stack, for current standards, there are multiple vendors that can be used one of the notable ones are GitHub Actions, Jenkins and GitLab.
Which cloud provider
The traceabilility of the metrics and the adoption rate is coupled with the metrics, at the same time the metrics is coupled with the piepline and the infrastructure used. In that sense, for each software project adpting the atomtion might be required.
Commits standarization
Parts of the collectino of metrics comes from the git history. Git plays a central role in the process, it is the place which hooks get triggered to start pipelines, rollbacks or deployments. In that sense, standarization of commits are needed, diong so will enable the automation of pipelines based on them, helping the metrics to be collected.