The art of points-based procurement for Agile projects has been saved
Limited functionality available
One of Agile’s key advantages is that it can accommodate change. But how can a contracting officer price something where the final output is expected to evolve? Points-based procurement may hold an answer.
As more government organizations turn to Agile development, contracting officers may find themselves at a crossroads. How can the procurement function effectively support the purchase of software in the absence of precise design specifications up front?
At the heart of the challenge is pricing.
In one direction lies the traditional approach to acquiring software solutions, with its emphasis on functional specifications that are typically tightly defined up front and the ability to write a “fixed price” contract. This pricing scheme can be comfortable, but it isn’t always well aligned with the Agile approach, in which a final design emerges over time through a collaborative, iterative process. It also may not be as safe as one might imagine, as change orders can drive the final project price above that initial “fixed” price.
In the other direction, there’s a new way of buying software development services, one based on the notion that work can be measured and sold in units known as points. This pricing approach, though typically well aligned with Agile, doesn’t have a long-standing track record or framework for execution in government procurement.
In Agile, points are a way to compare the effort between different tasks. The point scale is arbitrary, but it gives a sense of the relative amount of effort required to produce a specific piece of work.
Estimating the points associated with a chunk of work is best approached as a group activity, to reflect the wisdom and experience of the team. One of the best ways to estimate with a group is for each member to make an estimate in isolation, and then to compare estimates. If there is close consensus, great. But if not, a discussion can explore why each individual made the estimate he or she did.
In order to accurately reflect the lack of precision associated with point estimates, some teams use a point scale that is a modified Fibonacci sequence, such as 1, 2, 3, 5, 8, 13, 20, 40, 80, and so on. The idea behind this is that the larger a piece of work, the more difficult it can be to estimate the effort required with precision. Instead of wasting time arguing over whether a task is a 16 versus a 17, the team can have the much easier debate of whether to score it a 13 or a 20.
But why use points at all? There are several possible reasons. First, productivity levels can vary, making points a more reliable indicator of effort than estimates based strictly on time. For instance, depending on who is involved, a five-point story might take three days, or it might take six. Using points can also limit extraneous factors, such as pressure to complete a sprint by a certain date, that can disrupt an estimate based strictly on time. In addition, people just seem to be more effective at estimating relative values rather than absolute values.
A key reason to consider a points-based pricing approach, however, is that, because it is aligned with Agile development, it can facilitate the contracting relationship, ultimately leading to a better value for the contracting agency. We call this the “art” of points-based contracting because there is no cookie-cutter recipe for success. Contracting officers and agency heads need to use their wisdom, judgment, and experience to make sound choices.
Fortunately, the territory ahead isn’t wholly uncharted. Agile has enough history in the public sector to provide some insight into the pros and cons of points-based procurement, as well as some practical ideas for getting started.
Ceremony. A meeting to mark a milestone in the development effort, such as a kickoff meeting at the beginning of a sprint or a demonstration of the team’s work at the end of a sprint. (“Join us at the ceremony on Friday to take a look at the new credit card payment feature.”)
Epic. A series of user stories. (“The first story is to let constituents pay online with a credit card. The next story is that they can use other online payment platforms. Then they’ll be able to search for the records they want. Then they’ll be able to download each record in PDF format.”)
Points. A measure of the relative complexity of a sprint, as assessed by the development team based on experience. (“It generally takes a little over two weeks to build credit card functionality into a website like this. Given that history, this sprint is worth 22 points.”)
Product owner. A representative of the contracting organization who writes stories and works closely with the vendor to implement them. (“The product owner has been on-site with the development team, helping them understand the business objectives.”)
Velocity. How quickly a development team completes work, as measured in points. (“We produce an average of 10 points’ worth of work each week. So with that velocity, this 22-point story will take us just over two weeks to do.”)
Agile is a methodology for putting usable new software in people’s hands as quickly as possible. The basic approach is to break projects up into small, time-limited chunks called sprints (see the sidebar “Glossary of Agile terms.”). Each sprint addresses at least one user requirement, or story, quickly resulting in a working prototype. Users get frequent demos of the software as it is being developed, with a chance to offer their feedback to the development teams. The teams then generate a new version that incorporates this user feedback. This process is repeated until the functionality is right.
Agile’s iterative process can offer a number of advantages. For one thing, it can uncover problems or shortcomings sooner than a traditional “waterfall” approach. Sometimes, people don’t realize what they really need until they have something tangible to look at and/or work with. Other times, development teams run into unexpected roadblocks. Agile acknowledges these realities and provides a way to address them before they undermine a project.
Then, there’s customer satisfaction. Because users have input during Agile development, the resulting software is typically more in line with their expectations. The Agile method helps users gain value from a project earlier, rather than wait for everything to be delivered at the end—including nasty surprises. Further, an Agile approach can boost confidence in the development team and project sponsors. It can also help set appropriate customer expectations: Based on their experience with past iterations, users learn what the development team is realistically able to do.
The greatest benefit of Agile, however, may be that it makes room for and facilitates change. Few things typically remain constant during the course of a big software development project. If priorities shift, an Agile approach can accommodate the situation with minimal friction. In contrast, a fixed-fee contract generally operates on the implicit assumption that the original design will meet users’ needs and that nothing will change. Sometimes, in fact, this turns out to be the case. However, when priorities do shift, a fixed-fee project can sometimes require post-contract changes, including change orders, and may generate costly late surprises—some of the very issues that Agile is intended to mitigate.
While it may be helpful that Agile can accommodate change, how can a contracting officer price for that? How do you hit a moving target?
How can you price something where the final output is expected to evolve? Various pricing approaches deal with this in different ways, each with its own set of pros and cons.
Time and materials: One possibility is time and materials—which can be an effective approach for an Agile effort. But to the government, this approach may feel like writing a blank check.
Fixed fee: It is possible to use a fixed-fee contract for Agile development, but from the contractor’s perspective, this can be risky. After all, in Agile, the expectation is that the design will evolve, and if a government agency has free range to add features, the result could be a recipe for scope creep. Moreover, traditional fixed-price contracts lock in functionality—but in Agile development, locked-in functionality specs may fail to account for the type of design evolution that Agile is designed to produce. In fact, an attempt to rigidly specify functionality at the outset is fundamentally misaligned with how Agile builds software.
Small chunks of fixed fee: An agency could break an effort into smaller chunks for the purpose of issuing a series of small, fixed-price contracts, one for each chunk. This approach is somewhat consistent with Agile’s incremental development approach. As a pricing mechanism, however, this can be burdensome, as few parties to a transaction likely have much appetite for putting every two- or four-week sprint out to bid. It also might give incumbent vendors an unfair advantage over competitors who lack familiarity with the current project or who do not have an ongoing relationship with the product owner. Alternatively, jumping from vendor to vendor based on the bid on a particular chunk of work can be both administratively costly and risky, given that not only does each component have to work, but they all have to work together. Like a car, software can be built by multiple vendors, but significant work must be done to ensure that all the pieces mesh.
Points-based Agile procurement: This approach is similar to the “small chunks of fixed fee” approach, but it more fully embraces the language, methodology, and working pace of Agile development.
In points-based Agile procurement, development teams estimate project complexity through a system of points based on how much work is required to complete similar tasks. By adding up the points for each story in a project, experienced development teams can gauge the overall velocity of their efforts. For example, if over a series of four sprints an Agile team delivers 80, 100, 120, and 100 story points, its velocity is 100, and the team can reasonably estimate that in future sprints it will likely be able to produce roughly 100 points—all other things being equal (days in sprint, holidays, number of staff, and so on). These points form an arbitrary but internally consistent scale: A 20-point programming challenge should take twice as much effort as a 10-point one. Points put the focus on complexity, not on the time required. After all, some teams may be more productive than others, taking less time to complete the same task.
Points-based procurement, at its core, is all about empowering delivery teams—specifically, the product owners—and breathing flexibility into the Agile process. The agency ultimately governs how much it is willing to pay for a given number of development points, and then empowers product owners to deal with the day-to-day complexity of determining what trade-offs and decisions to make with respect to product features.
If contracting teams acquire software solutions based on points—under a blanket purchase agreement, for example—they can largely avoid change orders, budget overruns, and other costly distractions. The focus instead shifts to the programming challenge and the desired functionality. Long-term, points-based procurement can help alleviate other classic challenges of long-term development planning, as it places more authority for planning directly into the product owner’s hands. Long lead times (and their attendant risks) give way to sprints. Gantt charts and arbitrary deadlines are replaced by iterative processes, demos and other Agile ceremonies, and feedback loops. Change resistance and eleventh-hour training yield to ongoing, high-touch collaboration between the business’s product owner and the development team, whether internal, external, or a combination of the two.
Put another way, points-based procurement allows government contracting officers to acquire the capabilities of an Agile team, without caring about whether that capability is delivered by 8 people or 12 people, and without relying on a specific set of requirements or functionality. This way, executives could gain an extension of their own teams who can direct more attention to the business activities that align with the organization’s mission.
Ultimately, the use of points-based procurement can facilitate fixed-price purchasing without the need for creating functional specifications. In other words, a government agency and vendor with a good track record could agree on a fixed-fee contract that buys a certain number of story points without dictating what functionality those points will be put toward. Then, the client’s business leaders and Agile team can prioritize what to work on based on “bang for the buck”—that is, which stories will produce the most business value relative to story points consumed.
None of this means that traditional requirements-based software procurement is about to disappear. Specificity is a response to accountability, after all, and hours and days are typically easier to measure against external requirements. At some point, moreover, the subject of price does need to come up. How much a “point” costs relates back to the cost of the developer resources.
The biggest barrier to points-based pricing may be that a successful points-based approach requires a shared understanding between delivery and procurement—meaning that the procurement team should develop a good understanding of how points relate to effort. This means not only becoming comfortable with concepts such as “velocity,” “story,” and “epic,” but actually becoming familiar with how much technical effort is required to accomplish certain tasks. Is it reasonable that building a user interface should require 15 points? Why would adding a mobile capability to an app require 40 points? Only experience developed over time can allow someone to be confident that these point-based fees are reasonable.
In addition, there would need to be an education effort for political leaders, so that they can feel secure that the public interest is being served and that the agency is receiving appropriate value for the price it is paying.
In sum, there are many ways to price a contract for Agile development. You can use fixed fee. You can use time and materials or cost-plus contracts. You can break a big project into a series of small fixed-fee chunks. You can use a points-based approach, including one that includes a firm fixed price per user story point delivered. Or you can start with time and materials for the first few sprints, and then move to a “points-based” system for later sprints. All these approaches have benefits and limitations, and the best pricing structure could depend on many factors: the nature of the work, the organization’s appetite for experimentation, and the level of trust between an agency and its vendors, to name a few.
With all that said, if Agile is gaining steam in your organization and if your procurement organization seems open to the approach, you likely have a few openings for bringing points-based estimation into the public procurement realm. The key may be to take a stepwise approach, documenting effective practices as you go. Steps to consider include:
Get firsthand experience. Before attempting points-based procurement, focus on learning how to work with Agile contractors. Once your contract team develops a history of making deliverables in an Agile way, they—and you—could be more comfortable buying a design that’s not completely laid out.
Look to long-term projects. Even under a traditional contract, longer-term Agile projects can provide more opportunity for government employees and contracting personnel to work together and understand their own internal velocities. At the same time, conversations would tend to be less about price and more about the complexity of a piece of functionality. That depth of understanding can form the basis for making points-based procurement worthwhile.
Keep it simple—at first. Consider starting your points-based procurement journey with a time-and-materials contract for Agile development. From there, you might move to fixed-price contracts, and finally on to points-based contracting as the contract team’s experience grows. Another option: Consider using a points-based contract for a project that involves a single technology platform with a sophisticated software development kit. The lessons from this experience might then extend to projects with more diverse technologies or the need for more custom development.
Although Agile is no stranger to government organizations, a points-based measurement and pricing technique can be a significant departure from the traditional procurement mind-set. However, that needn’t stop contracting officers from exploring ways to apply what Agile has to offer. The results could well be greater flexibility, transparency, and value for the public.
There’s a growing body of literature on the subject of points-based software acquisition. To learn more about this rapidly-maturing area, consider starting with the following resources:
The American Council for Technology (ACT) and Industry Advisory Council (IAC) published a document for the procurement of Agile IT services.
The US General Services Administration maintains a website of information on modular contracting and Agile development. Here, among other things, you can find the Agile Delivery Services Blanket Purchase Agreement describing how the US federal government is trying to align acquisition practices with Agile delivery practices.
The contracting guidance that the White House published in 2012 still seems relevant for agencies who want to dive into Agile. Another White House publication, the TechFAR handbook, explains how government organizations can use Agile principles to acquire digital services in the context of the Federal Acquisition Regulation (FAR).