How Government digital teams can use Cost of Delay to prioritize work

Over the past year I’ve been learning a lot about cost of delay. In this blog post I explain what I’ve learned so far, and how it can be used by teams to prioritise work. My focus is on teams that are following the UK Government Digital Service Standard, but it can be of use to any team.

What is cost of delay?

There are always so many things that need doing, but it’s not possible to do them all. Teams need a way of deciding what to work on next, and managers want to be confident that teams are making the right decision.

Cost of delay is a simple measure that factors in value and urgency. When a cost of delay is given to all opportunities it enables more effective prioritisation decisions — and more effective communication of those decisions. An opportunity with a high cost of delay should be done sooner, because there is high benefit to be gained from prioritising it.

How to calculate cost of delay

Cost of delay = Value * Urgency

There are two key parts to cost of delay:

Value. What benefit could this opportunity deliver for your organisation or your users? In Government this means reducing costs, avoiding future costs, or saving time / money for citizens, businesses and other users.

Urgency. Something is urgent if doing it later reduces it’s total value (eg. if it’s needed for a specific date, or if a competitor might get there first otherwise)

In the example below, option 2 is higher priority because the cost of delay is higher.

CD3 (Cost of delay divided by duration)

CD3 (Cost of Delay Divided by Duration) = Value * Urgency / Duration

When prioritising work, it makes sense to factor in how much time it will take from the team to deliver the opportunity — the duration. This gives us cost of delay divided by duration — which can be abbreviated to CD3 (CDDD).

In the example below, option 1 is higher priority because the cost of delay divided by duration is higher.

My experience of using relative values

When I first started out with cost of delay I used relative values. Working in the public sector where profit isn’t a motive, I assumed that trying to put a £ value on opportunities would be too difficult to do. So I took what I knew of relative estimation for story points and got the team to give each opportunity a score for value, urgency and duration, then plugged those scores into a formula to give me my cost of delay. This in itself was a valuable activity — it got the team talking about value and quickly gave us an idea of what the best opportunities were. But it didn’t scale well to anyone outside the team, or team members who weren’t present for the discussion over value. It was also too easy for the scores to get influenced by the opinions of opinionated and influential individuals who others were only too happy to defer to.

I also used and created more complex models where opportunities were given scores for their alignment to various strategic objectives and the risk/complexity of delivering them. Answer 10–15 questions for each opportunity and out pops a score that tells you the cost of delay. I’ve created these models myself and used ones provided by consultants. None have worked for me — it’s much better to prioritise using a simple model that everyone gets then it is to create a score that people don’t understand or trust. Donald Reinertsen came to similar conclusions in his book ‘Managing the Design Factory’.

Why I prefer to calculate cost of delay in £

It isn’t actually that hard to put £ numbers on benefits. Start with an educated guess. For example, if you’re developing a new feature that will save each user 10 minutes per transaction, there are 100,000 transactions a month, and your user’s time is worth £15 per hour, then the potential benefit to users of that feature is £250,000 per year. Admittedly, some things such as enablers (foundational work that needs doing before user facing services can start delivering value) and information (discoveries and alphas are all about learning rather than delivering user value) are hard to put a value on, but it is still possible. The key thing is to write down the assumptions behind the numbers so that they can be challenged or validated by others.

Assumptions identified. Once you’ve made your educated guess, you can start validating some of the assumptions behind the guess. Will the new feature really affect all of those 250,000 transactions? Will it really save 10 minutes per transaction? Is your user’s time really worth £15 per hour? These are all things for the team to explore further.

Scales better. Relative estimates struggle when there’s lots of options to compare, or those who weren’t involved in assigning the estimates want to understand them. By visualising all of our assumptions it’s easy for those outside the team or new to the team to understand the reasoning behind the value of work. And by putting £ values on every benefit in a consistent way, it’s easy to compare opportunities within and across teams.

Increased accuracy. Sharing assumptions makes information more transparent, and enables more people to provide constructive feedback which can help improve the accuracy of estimates. The act of putting £ values on benefits also helps us overcome cognitive bias and better understand the true value of something. I’ve seen cases where relative estimates assigned two opportunities the same value score, but when the cost of delay was calculated in £ one was 10x the value of the other. It was an eye opener for me.

Gets benefits prediction started. Benefits prediction is an inescapable part of working on any Government project. In many teams, trying to quantify benefits is an afterthought. They struggle to predict what benefit their work will deliver, and they struggle even more to evidence that those benefits have been realised. Using cost of delay to prioritise brings conversations about benefits forward, and gives them extra significance. Early visualization and validation of assumptions puts teams in a better position to estimate and evidence benefits. These estimations can and should be refined as the team progress. This compliments the service manual recommendations on measuring the benefits of your service.

When cost of delay is useful

Pre-Discovery

Pre-Discovery, cost of delay can be used at a very high level to identify the problem space that the team should be focusing on.

Look for areas where:

• volume of users / transactions is high
• the organisation and users are experiencing a lot of pain
• the team have the capability to make an improvement
• value of benefit decays over time (high urgency)

During Discovery & Alpha

A key output of Discovery is to recommend which opportunities the team or organisation should pursue in the future.

For each opportunity considered, try to estimate the cost of delay & how long each would take your team to deliver. Make educated guesses about the values of each. Visualise your guesswork. Show your working and share with others to give them a chance to challenge assumptions. The tables below show how you might communicate benefits & assumptions for each opportunity, and compare the cost of delay across different opportunities. This is explained more at the end of this blog post.

The table below shows how to record benefits & assumptions for an opportunity

The table below shows how to compare the cost of delay of different opportunities. Option 3 should probably be done first.

The opportunities with the highest cost of delay & CD3 are the ones worth exploring further. For these opportunities, you want to start validating some of the assumptions you’ve made. Here are some techniques you might want to use to validate assumptions:

• Expert opinion. Sharing your assumptions with others in your organisation to get their opinion
• Talking to users. Ask them what they think
• Test with users. Test the current solution and your prototype with users. Record how long it takes for users to complete the task (and whether they complete it at all)
• Analysing the data. Do you have access to analytics tools or management information that can help? If not, can you start collecting data that will help give you more certainty?

User researchers and performance analysts will be able to help you with all of the above. If you have any of those in your organisation, ask for their assistance.

During Beta

Once real users are using the service, the team can start to use cost of delay and CD3 at a more granular level. Smaller features which take weeks or months to develop can be considered alongside larger features or new products. This enables the team to stay focussed on the opportunities that deliver the best value for money — whether that involves making small improvements to an existing product or moving into a new problem space.

There is a natural bias towards smaller features in CD3. This is a good thing — it encourages the team to break work done into smaller units which increases delivery flow and reduce risk.

At the programme level

Cost of delay informs decisions about where to focus investment through giving us information on the value of queues. In the table below, Team A has a backlog with a relatively high number of high cost of delay opportunities. The cost of delaying that team is higher. This indicates Team A should be prioritised when people assignment / resourcing decisions are being made. Increasing investment in the problem space that Team A are focussing on may also be a good option.

To effectively use cost of delay across multiple teams, there needs to be an effort to standardise how it is calculated. The assignment of monetary values to benefits should be done consistently. For example, time saving for users is a big source of benefits for most Government projects. They should all use a consistent value of citizen time.

How to calculate cost of delay

Calculating value

When calculating value, I like to consider what benefit the opportunity could deliver once it’s live.

For government projects, key benefits that we deliver are:

• Staff cost reduction (internal)
• Non staff cost reduction (internal)
• Cost avoidance (internal)
• Saving time for citizens (external)
• Saving citizens from paying a fee (external)
• Saving time for professional users (external)
• Saving professional users from paying a fee (external)
• Waiting time reductions (external)
• Opening data (external)
• Fraud reduction (external)

In the private sector and some parts of the public sector, you will also want to consider revenue.

To illustrate how to calculate value, let’s take the example of a feature to enable online signatures for an ID Card registration service with 1 million transactions per year. Without this feature, users complete the form online then have to print if off, sign and send it in. Let’s imagine a future state when this feature is live: with this feature they will be able to sign their application digitally in the web browser. We might assume:

• 90% of users use the digital service over other channels
• All digital service users save 10 minutes per transaction (from not having to print it off and scan it in)
• All digital users save 60 pence (from not having to post it in)
• Staff save 5 minutes per transaction (from not having to manually check the signatures)
• Digital service user time is valued at £7.83 per hour (minimum wage. Your organisation may have other standards for this value)
• Staff time is valued at £20 per hour

The potential value of doing this work is £3,224k per year. This is of course a simplification — but it helps us understand the ‘size of the prize’ from pursuing this opportunity relative to other opportunities we could pursue.

Value of enablers

Some pieces of work we do not because they deliver value themselves, but because they enable the delivery of future value. This is what I call an enabler. The best way I’ve found to assign benefit to an enabler is to attribute a fair portion of the future value that it unlocks. Where an enabling piece of work takes 3 months and it enables a valuable new feature that 9 months, it’s fair to attribute 25% of the end benefit of the new feature to the enabler, as 25% of the time it takes for the team to deliver that features comes from the work required to do the enabler. Back to the example of digital signatures, let’s say that one of the benefits of this feature is that it could enable future work to automate application processing.

Value of information

When developing Government services, much of the work we do is useful because it helps us generate more information. Information is useful because it reduces the likelihood of us making bad decisions. To factor in the value of information, consider how much you would be willing to pay for that information if it were available on the market, or how much it might cost you if you made an imperfect decision.

Benefit that is very hard to put a £ value on

If something is intangible and hard to put a figure on, it’s still worth trying. Just consider what the benefit is, and how much it’s worth to your organisation, and write down the assumption. That enables it to be compared against others opportunities. Recording the assumption also makes it clear to others what non monetary benefits are factoring in to decisions, and an idea of what weight they’ve been given. I record these values as non-monetary, so that they aren’t confused with more demonstrable benefits.

Calculating urgency

I don’t use £ values to estimate urgency. Instead, value is multiplied by urgency.

Urgency = 1 if value & cost won’t be affected by doing this opportunity later.

Urgency = 2 if delaying reduces potential value

For any business, urgency is usually driven by competition — the fear that if an opportunity isn’t seized quickly others will get there first. Government isn’t driven by this same sense of urgency. The key factors that will make Government opportunities more urgent is whether there’s a deadline attached to it, and the consequences of not hitting that deadline. The best example of urgency in Government in the UK at the moment is Brexit — anything that’s needed for when the UK comes crashing out of the Union is going to be high priority

Bringing it all together

Here is a google docs copy of the template I’ve been using to visualise assumptions from teams and calculate/compare the cost of delay and CD3. It consists of three tabs:

• global variables & assumptions
• benefits assumptions for each opportunity (use one per opportunity)
• summary table comparing all opportunities

Using this information wisely

Remember, this is a model. It is a simplification of reality that enables more effective decision making. Accuracy is reduced because factors such as take-up and operations cost are ignored. But it’s as accurate as it needs to be to enable good prioritisation decisions, and effective communication of those decisions.

More information

For anyone interested in learning more about cost of delay and CD3 I recommend Black Swan Farming. Much of what I discuss in this blog post is more comprehensively explained by Joshua Arnold — the site’s author. A big thank you to Joshua for answering many of my questions on this topic!

Do get in touch if you have any feedback on the techniques discussed in this post. I am still fairly new to cost of delay and keen to improve my practice.

Originally published at alaniswright.com on December 12, 2018.