This weekend I was catching up on some New Yorker issues, when an article by one of my favorite New Yorker authors, Atul Gawande, struck me as illuminating so much about tech companies and DevOps. (This is an example of ideas coming from diverse, unrelated sources – something part of the culture of LogicMonitor. Just yesterday, in fact, our Chief Network Architect had a great idea to improve security and accountability when our support engineers are asked to log in to a customer’s account – and this idea occurred to him while he and I were charging down the Jesusita trail on mountain bikes.)
The article, Atul Gawande: How Do Good Ideas Spread? : The New Yorker, is an exploration about why some good ideas (such as anesthesia) were readily adopted, while other just as worthy ideas (antisepsis – keeping germs away from medical procedures) did not. So how does this relate to DevOps and technology companies?
One of the challenges in technology companies – or indeed, any company – is to ensure best practices are actually employed. We all know we should have blameless, five-why investigations to determine root causes of failures and outages. We all know we should not regard issues as closed until we have monitoring in place to ensure it will be detected in the event it occurs again. The best practices of DevOps are not secret. They are mostly common sense.
So why aren’t they the norms at all businesses? Even at LogicMonitor, where we’ve been a DevOps shop for years (for the most part), we still find ourselves sometimes caught short because of, say, a lack of cross functional deployment planning.
An equally vexing issue is in the propagation of best practices amongst sales teams. If people want to jump in and set up LogicMonitor in their own environment, why should the sales person even attempt a demo? Yet the people that go through a demo become aware of what their monitoring could be, and generally have greater success than those that don’t. Sales people know this, but it’s easier to not attempt to guide the customer into a demo.
Atul argues that the difficulty in both issues is the same: the desired changes require modifying behavior at a point when differences in the outcomes are not visible (who knows if involving Ops (or a demo) at this point of the coding (or sales call) will change the success of this deployment?), and the changes are tedious, and go against the current norms.
Neither prescription (“Do this!”) nor incentives (“Do this and we’ll pay you!”) seem to help with changing norms.
Instead, the best approach seems to be using mentors, to call people’s attention to the way they are doing things, how they do not align with best practices, and discuss in a friendly way why they are not acting in a way that accords with what they know is optimum. This also aligns very nicely with the Agile concept of self-managing teams – the mentor and mentored discuss the issues, the best practices, and come up with ways to implement them that work in this situation. No one is directed to do anything. The key to this successful relationship, according to Atul, is that the mentor be “nice”. (No comment as to whether that criteria will be harder to find in Sales or DevOps.)
So, in your Sales teams and your Development and TechOps teams – ensure you have identified a mentor – someone who is willing to hold an objective view of the way the team is currently working, and how that relates to the best practices of the field, and gently guide discussions when there are differences. The best practices may not be the correct path in every situation – but divergence should be a conscious decision, not just a reversion to prior norms.