How Cedexis Deploys Puppet Enterprise and LogicMonitor Jointly to Support its Global Operations
Founded in 2009, Cedexis is building a faster Web. Cedexis offers visibility and control of Web performance through its community-based monitoring & analysis solution, Cedexis Radar, and its global traffic management platform, Cedexis Openmix.
Ops in the Cloud
Deploying their technology strictly in a cloud environment, Cedexis’ TechOps team follows a simple rule: “Never touch hardware.” Cedexis manages its dynamic host deployments globally across a range of managed hosting and cloud providers. To ensure uniformity across datacenters, Cedexis configures new machines identically via configuration automation tools in order to prepare each with a “blueprint” to take the Cedexis code.
Integrated Automation Technologies Play a Key Role in Cedexis Ops
“We rely on (LogicMonitor’s) SaaS-based monitoring service as much as we rely on Puppet Enterprise to manage our servers across 48 datacenters world-wide. I love the integration between the two products because it gives us full visibility into our infrastructure and allows us to manage resources at scale.”
— Marty Kagan, Cedexis Cofounder and CTO
Josh Cody, head of Cedexis’ TechOps explains, “We were early users of LogicMonitor’s performance monitoring platform and of Puppet Labs’ configuration management software. We recognized the need to use both products to facilitate the deployment and performance of the Cedexis app across the world. Puppet Enterprise helps us ensure machines and app versions are in synch. LogicMonitor then places those devices into monitoring, providing the operational assurance that each is deployed and functional.”
Cedexis’ Ops team urged LogicMonitor to create an integration between LogicMonitor and Puppet Enterprise to allow complete automation of the LogicMonitor Collector deployment. “The LogicMonitor Collector is deployed automatically via Puppet. Based on our DNS naming structure, a deployed machine is automatically discovered and placed in LogicMonitor. We know immediately that the new machine is up and running and how it’s performing,” says Cody. The integration allows Cedexis to take full advantage of LogicMonitor’s capabilities, automatically firing up custom app performance graphing based on logical group association. Later, Puppet initiates take down procedures to pull the device from the LogicMonitor platform, eliminating unwarranted alerts.
“LessOps” to DevOps at Cedexis: A Day of No Alerts!
Historically Cedexis has relied on its TechOps team for successful app deployment. Because developers tend to write error messages for people used to writing or running their own code, Cedexis Ops engineers didn’t have good definitions of app “hot spots” – code components that need monitors – and alert creep became problematic. When they identified this disconnect between team functions, a cultural shift was instituted. Cedexis’ blog memorialized the shift. “We wanted our LessOps approach to build more collaboration between operations and development, including cross-training. We hoped that integration between the two groups would lead to a more robust product. However, it would mean some changes that might not be that popular, such as adding developers to the on-call rotation.”
Cedexis’ new approach to DevOps has two key elements. First, developers are responsible for designing and responding to monitors for components they write. These messages, suddenly monitored by engineers, highlights places in the system where unneeded monitoring and reporting is happening. Minor, easy-to-fix issues that operations typically ignores are quickly discovered and addressed.
This led to another change that produced a more robust system. Working jointly with Ops, Cedexis developers now write the data sources for the components they code. Alerts based on these data sources facilitate more effective responses because they are directed, using LogicMonitor escalation chains, to the proper domain experts.
These changes ended the cold hand-off of code to Ops and not only improved the stability of the deployments but facilitated a cultural shift to an integrated DevOps environment between previously siloed Dev and Ops teams. After a few weeks of the new approach Cedexis actually experienced a day with zero alerts and zero error messages!
Cody summarizes the impact that technology has made in transition to DevOps. “It is huge to have these two key tools deployed automatically. Both LogicMonitor and Puppet are critical to our operations. Without them we could not do what we do. As an Ops guy I don’t want to work anywhere that doesn’t deploy these tools.”