In one of his first development jobs, Kent Beck was given an assignment that he estimated would take himself and another programmer six weeks to complete. But at the beginning of the project, his partner was reassigned to another task, leaving Kent to complete the project alone. Working on his own, he completed the project in twelve weeks—twice the allotted time, a reasonable adjustment considering he had half the team members he had been promised. Still, his managers were unhappy. They harassed Kent for the six-week “delay.” He felt like a failure even though his original estimate had been right on the nose.
This is just a sampling of the kinds of Dilbertesque expectations that burdened software developers in the 1990s. Companies were using sweeping, inflexible processes, like Waterfall—which had been designed in the 1970s to manage massive spacecraft projects, not meet the changing demands of software development in the internet age. As a result, it was difficult to make changes to a project that was already started, working software wasn't delivered until late in the process, and quality suffered.
Developers had enough. Independently of one another, programming leaders started to create their own “lightweight” processes as alternatives to the “heavyweight” processes in place at many companies. These new processes came in many forms—Extreme Programming, Scrum, and more—but their adherents sensed they shared similar philosophies. In February 2001, a group of seventeen programmers met in a ski lodge in Utah to try to find common ground.
Nestled in the snow-covered Wasatch mountains, Agile was born.
Agile, in the words of its founders, “is the ability to create and respond to change.” It's an approach to software development that prioritizes flexibility, collaboration, and frequent, small deployments. It's not a software development framework, rather, Agile is a mindset—a set of values and principles that helps companies make good decisions as they adapt to changing circumstances. The creators of Agile codified these values and principles in the Agile Manifesto and the Twelve Principles of Agile Software, which we'll discuss in more detail in the coming sections.
Agile arose as a reaction to the problems inherent to heavyweight software development processes, like Waterfall. The issue with heavyweight practices is that their success typically depends on following a strict plan, which can easily fall apart when teams face unanticipated obstacles. While some teams can fix this by getting better at planning, in most cases, change occurs that no one could have predicted—you have to reassign developers to fix an urgent issue, your customer changes their mind, or your competitor releases a feature that renders the one you were working on obsolete. In Agile, instead of trying to be predictive and anticipate every possible eventuality, you become adaptive and work on becoming flexible.
While these ideas weren't entirely new—iterative development practices had been around since the 1950s—the creation of Agile marked a new chapter. It's an adaptive development philosophy, not a defined process. Agile united developers with shared values, even if their individual approaches diverged. Crucially, it helped organizations rethink their software development processes—today 97 percent of organizations report using Agile methods. Let's explore this philosophy in more detail and discover what makes it so compelling.
The seventeen developers who met at that ski lodge in Utah—the “Agile Alliance,” as they fashioned themselves—created the Manifesto for Agile Software Development to summarize their new philosophy, along with the Twelve Principles of Agile Software to further support and explain their thinking.
The manifesto is a fascinating document for a number of reasons—“not the least of which was getting 17 people to agree to it,” wrote Agile alliance members Martin Fowler and Jim Highsmith. At just 68 words long, it's easily digestible, yet expresses some quite profound ideas about what developers should value and prioritize. Let's have a look.
The manifesto is notable, first of all, for its introduction. The signatories wanted to make it clear that—while they are considered experts and leaders in their field—software development is an ongoing practice and they're still learning. The choice of the word “uncovering” was deliberate, Martin and Jim say, “to assure (or frighten) the audience that the Alliance members don't have all the answers.” They simply wanted to share what they had learned and were continuing to learn.
The manifesto is also unusually “mushy” for a software development document (to borrow the phrasing of Agile Alliance member Bob Martin). It emphasizes creating human connections, both with teammates and customers. That's by design: “Software development is a people business,” said James Newkirk in a talk for the Agile Alliance. Building software, especially on a large scale, is a collaborative process—there are many stakeholders, both internal and external. The Agile Alliance wasn't arguing that having processes in place or negotiating contracts are bad practices, just that they're inadequate for successfully navigating software projects. Collaboration and teamwork take precedence.
Finally, the manifesto turns time-honored project management wisdom on its head. “Traditional project management practices assume that achieving a plan equals project success equals demonstrated customer value,” write Martin and Jim—but this simply isn't true in the modern software development landscape. Today's projects tend to be more volatile, with frequently-changing demands. A project's initial conception has little bearing on its ultimate success. It's more important for teams to be flexible than to execute a plan to the letter. Or, as Martin and Jim put it, “facilitating change is more effective than attempting to prevent it.”
Perhaps most significantly, the Agile Manifesto is not prescriptive. There are no rules about how teams should structure their workflows or how often they should meet. Rather, the manifesto offers a mindset that teams can adopt to help them prioritize the right things. When teams are faced with decisions, they can think back to the manifesto and use it to help them make choices in line with their values.
The Agile Alliance further supported their manifesto with twelve guiding principles. These help explain the creators' thinking and explore the manifesto's practical implications in more depth. In the words of the Agile Alliance, these are “the guiding practices that support teams in implementing and executing with agility.” Let's take a look:
While the Agile Manifesto promotes an idealized vision of software development, these Twelve Principles lay the foundational practices that help teams achieve it.
The simplicity of the Agile Manifesto and the Twelve Principles principles can be deceiving. While the ideals expressed in the manifesto are easy to understand, putting them into practice is harder than it looks—not to mention counterintuitive to the way many organizations function. Large, established companies typically have many processes in place. They may want to keep exhaustive software documentation and negotiate carefully-worded contracts to cover themselves for liability purposes. They might not be equipped to respond to adapt to change without extensive change management planning. So why bother with Agile at all? Let's take a look.
This isn't just anecdotal: Researchers at Baylor University have found a positive relationship between developers' use of Agile methods and their job satisfaction. The study found that members of Agile teams have more autonomy—Agile developers tend to be actively involved in project management and have a higher degree of control over the work they do—which promotes job satisfaction and employee well-being.
Since the early 2000s, case studies across a wide range of organizations have shown significant productivity gains from adopting Agile. According to the most recent State of Agile Report, 51 percent of teams cite productivity gains as a reason for adopting Agile. This may not be surprising. After all, Agile minimizes wasted time and resources by eliminating redundant meetings, unnecessary documentation, low-value features, and so on. But Agile teams may also be more effective because they're happier—as demonstrated by a University of Michigan study, positive teams are more productive.
Unlike plan-driven project management techniques, which attempt to scope the entirety of a project's requirements at the beginning, Agile methods focus on cultivating an ongoing conversation between developers and stakeholders. This fosters better communication between developers, executives, and customers—whether internal or external. With more visibility into the development process, stakeholders feel more involved and can offer valuable feedback on an ongoing basis.
As laid out in the Twelve Principles, teams using agile methods make smaller deployments, more frequently. These shorter iterations allow for faster feedback, meaning that teams can correct problems quickly and design quality into products from their earliest stages. One study found that teams using Agile used QA practices much more frequently than teams using Waterfall, and that these QA practices were built in from the earliest phases of software development.
Agile empowers teams to work more autonomously—releasing managers from the obligation of micromanaging their staff. Agile managers still conduct regular check-ins and offer their advice and support where needed, but they can trust that their teams will get the work done. Instead of constantly assessing everyone's progress, managers can focus on the big-picture problems they're meant to be solving, such as defining company direction and vision, prioritizing strategic goals, and assigning the right people to the right projects.
Agile is a philosophy that teams can use to make decisions about how best to develop software. While many Agile teams do use frameworks, like Scrum, these processes alone aren't Agile. As Agile expert James Newkirk explains, “Process itself is the means to an end. It is not the end.” The end goal, as stated in the Agile Manifesto, is “finding better ways of developing software.” Agile teams will have varying processes and approaches, but they are all working towards this same goal.
While some managers might flinch when they hear phrases like “self-organizing teams,” Agile does not mean chaos and anarchy. Agile teams don't reject process, they simply don't let process take priority over their teammates or the goals they're working toward. In fact, Agile teams do typically adopt some kind of software development lifecycle methodology, like Scrum, Lean Development, or Kanban—they just ensure the process is serving their needs, rather than sacrificing their needs to the process.
This is a contentious one, as the Agile Manifesto and Twelve Principles were written specifically with software teams in mind. But other teams can also benefit from adopting Agile principles—for instance, NPR uses Agile methods to create new radio programming and Lonely Planet uses Agile in their legal team. However, Agile works best when the conditions mirror the ones faced by software developers. The Harvard Business Review summarizes these conditions as follows: “The problem to be solved is complex; solutions are initially unknown, and product requirements will most likely change; the work can be modularized; close collaboration with end users (and rapid feedback from them) is feasible; and creative teams will typically outperform command-and-control groups.”