DevOps: Bridging the Schism with Alerts

What the hell is DevOps anyways?

The term DevOps is thrown around a lot. As with any good buzzword, it started as a phrase coined by an industry trendsetter to describe a specific concept. Over time, DevOps has morphed into a nebulous construct and one now risks DevOps semantic satiation* by attending most technology conferences. Stepping back and examining the original concept described by “DevOps,” we get something along the lines of:

DevOps is the practice of operations and development engineers participating together in the entire service lifecycle, from design through the development process to production support…. DevOps is also characterized by operations staff making use of many of the same techniques as developers for their systems work.” [Source:]

Or, to simplify it further:

“DevOps = Development + Operations”

For those of you who got this far without rolling your eyes or saying, “No crap. Get on with it.” The reason you are hearing this buzzword every way to Sunday is that this simple concept has some very powerful benefits. It turns out that knowledge of physical infrastructure, and internal resource landscape (aka Ops knowledge) empowers developers to find simpler, faster, or more elegant (pick two?) solutions. Conversely, Operations teams are spending increasingly large portions of their time wearing the Developer hat as part of automating away infrastructure management.

## Wasn’t this article supposed to be about LogicMonitor?

Yes, as you may have expected, there was going to be some amount of LogicMonitor-specific context to this article. As a member of LogicMonitor’s TechOps team ( formerly known as OpsDev), I have always had a difficult time discussing DevOps in a serious context. Seriously, I was part of a team called “OpsDev” because my boss was so opposed to “DevOps.” Given our speed and the complexity of the projects we’re working on, growth of teams, etc., it’s not surprising that Ops and Dev teams haven’t always been in sync. We work together on projects, share many similar tools and practices for writing, testing, and deploying code, but, it has always felt a bit like playing volleyball (or badminton if we are working with the China-based Developers). No matter how many steps we took to improve build tools, or CI solutions we implemented, that feeling of separation always nagged me. “This isn’t how DevOps is supposed to feel.”

What finally bridged the schism was the last thing I would have expected. Alerts.

## The DevOps drum circle

A few months ago, we proposed a significant shift in the responsibilities of both Development and Operations. We wanted to increase the ownership of developers throughout the application lifecycle, as well as improve response time for both infrastructure and application issues.

Historically, once a release had gone through QA and was packaged appropriately, it became the sole property of Ops. Operations was in charge of releasing this new code, as well as handling any detected issues after release. There were some great things about this method, and some less great. As an enterprise monitoring company, one of the requirements for newly released components was a comprehensive DataSource suite so that Ops would be made aware of performance issues. However, after initial deployment, ongoing tuning of these DataSources was in the hands of those who knew least how configuration changes would affect performance.

It was these DataSources that triggered our release process transformation.

Developers know the applications they develop significantly better than Operations ever will. Developers know best how the application will respond to configuration changes. Once Developers start receiving pages, they will have added incentive to make sure that these DataSources are tuned to be as quiet as possible, while still alerting when issues occur. This means alerts for both infrastructure and application performance issues are routed to the most capable team, decreasing resolution time. Moving Operations into a more supportive role for releases forces Operations to massively increase the automation of maintenance actions before providing the keys to Development. Having both teams on the hook for the success of a release ends up bringing LogicMonitor even further into the software development lifecycle.

## Conclusion

Now THIS is starting to feel like DevOps. But, before we ride off into the sunset on our unicorns of satisfaction, this is not the end of the DevOps road. Most of our developers have never set foot in a data center, and I most certainly couldn’t write a Java switch statement without Googling. DevOps isn’t a destination, it’s a journey.

* Semantic satiation (also semantic saturation) is a psychological phenomenon in which repetition causes a word or phrase to temporarily lose meaning for the listener, who then perceives the speech as repeated meaningless sounds. [Source:]