John Debs watched the system administrator, perplexed.
As a development intern at a medium-sized company, he had noticed the technician moving slowly from computer to computer. Soon it dawned on him. The systems administrator was installing a security patch on each machine, individually. All 250 of them.
“Why don't you just script it?” John asked. The sysadmin mumbled something unconvincing and persisted with his painstaking task.
Years later, the memory has still left a deep impression on John, who went on to become a DevOps engineer for Lua Technologies and now works freelance: “He didn't care about improving the process for himself or other people.”
That's where DevOps comes in.
DevOps is a work philosophy that prioritizes collaboration between the people responsible for creating software (developers) and the people responsible for maintaining it (operations engineers). That's where the term comes from: Development + Operations = DevOps. At some companies, these two roles are even collapsed into one: Full cycle engineers, like the ones at Netflix, operate the same software that they build.
But collaboration is just one piece of the puzzle. At its core, DevOps is about streamlining engineering processes in order to improve business outcomes.
Teams using DevOps standardize and often automate repetitive procedures, such as pushing software changes or setting up new virtual servers. They use principles borrowed from lean manufacturing—pioneered by Toyota factories in the 1950s—and agile development to efficiently manage the flow of work from inception to operation.
DevOps also emphasizes experimentation and creativity, using short software development cycles to deploy many small, frequent updates instead of a few large ones. This allows teams to quickly create value for their end users. Using DevOps practices, releasing a great new feature or fixing a pesky bug takes hours or days, rather than weeks or months.
DevOps is a team responsibility. While some companies do have roles dedicated to leading the transition from a traditional project management framework to a DevOps one, or ones that advocate for DevOps on an ongoing basis, as John Debs puts it, “DevOps is about getting the whole engineering team involved in the process.”
While there are plenty of tools that you can use to support DevOps processes, DevOps isn't software—it's a mindset. Daphne Reed, Director of Information Security at Vidyard, puts it this way, “Tools are the building blocks which allow the DevOps mentality to thrive.”
“When higher-ups look at DevOps, they sometimes confuse it with systems administration,” says Gaurav Murghai, CEO at Softobiz. DevOps is so much more. It's a cultural change which not only integrates development and operations teams but also helps operations teams automate traditionally manual administrative tasks. Managing the development pipeline, creating scripts, and standardizing procedures are all outside the scope of traditional systems administration work.
It takes time to evaluate your existing workflows, devise ways to improve your processes, and shift peoples' mindsets. Developers and operations engineers can't learn new skills or take on new responsibilities overnight. They need time to learn and gain experience. Even though it might take some time, in the long run, the benefits of DevOps—better teamwork, more efficient development cycles, fewer outages, and shorter time to value—are worth it.
Historically, most companies divided development and operations into two teams: Development would design, write, and test code, then “throw it over the wall” to operations, who would then allocate server space, deploy the code, and take care of ongoing maintenance, such as patches and updates.
At large companies, this divide could be so profound that every stage in the process was the responsibility of a separate role. Before implementing DevOps, Netflix's development cycle looked like this:
Separating the people who create software from the people who operate and maintain it can cause huge problems for engineering teams. Miscommunications can easily occur, delaying or completely derailing scheduled deployments. For instance, a development team might not adequately scope the infrastructure requirements of a new release, causing operations to scramble to set up new servers at the last second. Or operations might search fruitlessly for the cause of an error, never thinking to talk to the developer who wrote the code in the first place.
The result is a finger-pointing blame game which fosters competition and even hostility between the two teams. Dialogue starts to sound like this:
—Your code is broken, not my machines!
—My code is fine, your servers are the problem!
Not very conducive to collaboration. But in the late 2000s and early 2010s, dialogues like this one were playing out across a wide range of companies. Any business who depended on software to sell its products struggled as a result of the disconnect. Soon enough, developers and operations engineers alike began to grow weary of the gap.
As frustrations simmered, engineers began to look for a better way. At the 2008 Agile Conference in Toronto, Patrick Debois and Andrew Schafer were already beginning to discuss how they could apply agile development principles to systems administration. But the real turning point happened the following year, at the 2009 O'Reilly Velocity Conference when two Flickr employees, John Allspaw and Paul Hanmmond gave a presentation: “10+ Deploys Per Day: Dev and Ops Cooperation at Flickr.” The DevOps movement was born.
In parallel to these new approaches, advances in technology created an environment in which collaboration, systematization, and automation could thrive. In particular, cloud computing allowed companies to efficiently manage their servers through virtual, rather than physical, networks.
The shift to the cloud enabled companies to stop treating their servers like “pets” and start treating them like “cattle” (a metaphor dreamt up by Bill Baker while he was an Engineer at Microsoft).
Here's how it goes: In a “pets” model, your servers all are configured manually. They have names, require individualized care and attention, and when they “get ill” (stop responding) you nurse them back to health. Caring for “pet” servers is a big part of the work done by traditional sysadmins. But in a “cattle” model, virtual servers are configured en masse by automated tools. They are numbered, maintained as a collective, and when they stop performing optimally, replaced with new ones—all with little to no human intervention. Any individual machine can fail without taking down the whole herd.
Because of this fundamental shift, the need for traditional sysadmins began to disappear, prompting many people from operations backgrounds to take on development responsibilities and more developers to play a more active role in operations.
Rather than divided roles and responsibilities, the DevOps software development life cycle emphasizes ongoing collaboration across all stages. The DevOps model removes the artificial barrier between people building software and people operating it, fostering inter-team cooperation. While members of integrated teams often still specialize, they all should understand how the system functions as a whole. “It's like majors and minors in school,” says John Debs. While your “major” might be software development, you would also have a “minor” in operations or vice versa.
Collaborating across all phases of the software development cycle empowers teams to work together to resolve issues, instead of waiting on each other to fix them. As Netflix notes in a blog post, “Teams that feel operational pain are empowered to remediate the pain by changing their system design or code; they are responsible and accountable for both functions.”
Exposure to the entire development cycle also helps teams anticipate problems before they happen, instead of reacting to them as they occur. Being proactive allows teams to be more innovative—as Daphne Reed explains, “When you're stressed, you're not being creative. When you're proactive and have time to think about what you're doing, processes get slicker.”
By achieving smooth collaboration and continuously improving engineering processes, teams can optimize time to value. Using DevOps principles, teams work on minimizing the amount of time it takes to get from a great idea to a finished product—meaning their users see benefits, faster. A better user experience means happier customers, who will continue to use your products and recommend you to their friends. That's a big win for your business.
And a bonus—DevOps creates a fantastic environment for professional growth. Ongoing learning and skill development are not only encouraged by managers but essential to team success. As a result, engineers are constantly expanding their knowledge and abilities.