Cost of Delay, Product Flow and the Role of AI

00:00:04: Johannes Schartau: Welcome to Episode 15 of transform together, the Holisticon podcast meetup. I'm Johannes Schartau, and today I'm talking to Joshua Arnold. I first came across Joshua's work in 2014, when I attended one of his sessions at Lean Kanban Central Europe. He spoke about Cost of Delay, and the ideas from that session have stuck with me ever since. In fact, just this week, I used a lot of those ideas in a workshop with a client. Joshua is the Chief Product Officer at Gallagher Security, and on his website, Black Swan Farming, he writes about topics like Cost of Delay, product development, and how organizations can make better decisions about value. Joshua, welcome to the podcast.

00:00:44: Joshua Arnold: Thanks for having me, Johannes. Good to be here.

00:00:48: Johannes Schartau: So since I was first introduced to you through this topic of Cost of Delay, it would be great to get started with that. My first question is: can you explain the topic of Cost of Delay briefly to the people who might not be familiar with it? What is it, and why does it matter?

00:01:05: Joshua Arnold: Sure. So there's a little bit of a head fake in it, in that it's not actually about cost at all. The short version of it would be that Cost of Delay is the impact of time on the outcomes that you're hoping to achieve, the benefits that you're hoping to achieve from the things that you're working on. There is a more formal definition, in that it's the partial derivative of the total expected value over the lifetime of whatever the thing is that you're building. And it's the partial derivative of that whole value with respect to time. So: how is that value leaking away? If you could have it today, what would that be worth? And if it's a week later, or a week after that, or so on and so forth, how much of that value is leaking away? What's interesting about it is that it combines two concepts that humans tend to be terrible at distinguishing between. We have a real tendency to conflate value and urgency, as if they're kind of the same thing. So we treat things that are urgent as if they're really valuable. And we sometimes treat things that are really valuable as if they're urgent, even though they might not be. The hard part is trying to tease those two concepts apart, treat them separately, and then recombine them in a way that helps us make better decisions. One of the kicker examples for me was when we did it at Maersk. I'd done it at a couple of organizations beforehand, but when we did it at Maersk, they allowed us to publish the results of what we found, which became the basis for an experience report that I wrote with a colleague. There, we surfaced some really interesting examples and plotted the distribution of Cost of Delay across the whole portfolio. And this power-law curve sort of popped out, with some outlying things. We discovered things that were around $200,000 a week of Cost of Delay, $200,000 that they were basically giving away, for what was actually a pretty small change. The change itself only took about a week to do, but for every week that they didn't do it, they were losing about $200,000 on it. What was super funny about that particular example was that they'd actually spent additional time trying to get the estimate really accurate. So they'd put a whole bunch more work into trying to get a really accurate estimate out of it, because of course the system was incentivized for accuracy of estimates. So they basically burned another couple of weeks trying to get a really accurate estimate, as opposed to just doing it, which would have gone faster than actually working on a more accurate estimate. So it's kind of a funny little example that we came across. I guess the reason why it's really important is that without some sense of the Cost of Delay, of the economic value of the things, how that value is leaking away over time, without that bit of information, without that bit of data, the whole product development system ends up optimizing for other things. Whether it's accuracy of estimates, or on time, on budget, or the usual suspects that you see out there. Or it'll optimize for keeping people happy, or those sorts of things. As opposed to trying to deliver the most value in the shortest amount of time, which is what you'd think they'd be incentivized to do. There's the famous line from Don Reinertsen that I love. He describes Cost of Delay as the golden key that unlocks many doors. He describes it as having this astonishing power to transform the mindset and how we think about development in an organization. In my experience, I've really found that to be true as well. I view it these days more as a way of thinking about the problem space and a way of framing it, rather than just a thing that you do in the process. It changes the way you think about it.

00:05:38: Johannes Schartau: Yeah, you just called it the golden key that unlocks things, and I'm personally sold. Why doesn't everyone do this?

00:05:48: Joshua Arnold: Yeah, I mean, it is curious, right? I think it starts with lots of different things that kind of drive it, but in general, a lot of organizations are pretty blind to queues. They don't realize that there's all of this waiting that's going on in the systems that they're working in. They feel the end result of that, like things take a long time, but they don't seem to realize that a part of the reason why things take so long is that there's actually a whole bunch of queues that are hidden in our organizations. So one, they're blind to those queues, and secondly, they're blind to what those queues are costing them. That kind of drives a whole bunch of behavior relating to it. I think there are a bunch of other optimizations that are happening. You look at the way we do approvals and funding in most organizations, and it tends to optimize for accuracy as opposed to speed. It's very rare that you come across an approval or funding process that is optimized for recognizing that every minute you spend, or every week that you spend trying to get through some approval or funding process, represents a loss of value. I'd say one of the other main drivers is that there's a certain level of bravery that's required to come up with those estimates in the first place. You need to be brave enough to be the first person to say, "Well, I think this is worth $5,000 a week to us." That sort of invites a whole bunch of people to attack that number, and that may feel personal to us, especially when a lot of the people in the product development system have been beaten over the head for bad estimates for many, many years. So as soon as you ask them to make an estimate of the value side of the equation, they're like, "No, that ain't happening. I know what's going to happen here. I'm going to get beaten up when I guess wrong." As a result, we end up falling back on a whole bunch of qualitative approaches and relative approaches that just don't give you the same level of understanding of the value that's leaking away. It massively weakens that. The thing that I kind of say to people is that those decisions don't disappear if you don't quantify the Cost of Delay. They just get made badly. It's actually Don Reinertsen who tells the classic joke of a couple of dudes out in the wilderness who come across a bear. One guy takes off, and the other guy stops and starts doing up his shoelaces. The guy running off is like, "What are you doing? You can't outrun a bear. Why are you tying up your shoelaces? It's not going to help. You can't outrun the bear." And he turns around and says, "I don't have to outrun the bear. I just have to outrun you." The analogy to product development is that we don't have to be perfect. We don't actually have to outrun the bear. We just have to outrun the thing that we're using as the alternative, which in most cases is just our gut intuition. Our guts aren't a very reliable guide for determining the difference between value and urgency and making better decisions in this world. I always encourage people to at least have a crack at it, even if they don't necessarily share the number with anyone else, if you're not feeling strong enough to do that. But there's still value in adopting a mindset around it so that you've got a bit of an understanding. The other thing you can do is go and find your friendly finance person, who's usually got their fingers on all the numbers. They know exactly what all these numbers are, and they love building a spreadsheet. Find me a finance person that doesn't love a little spreadsheet. Before you know it, they will build you a little Cost of Delay model according to whatever it is that you're looking at. You can generally find people in the organization who are willing to make those brave guesstimates and put some numbers together. And then once you put that number out there, people can say, "Oh, no, that's wrong." "Oh, really? Well, what do you have that tells us it's wrong? Do you have some better data? Happy days. Let's share that." Or, "Oh, we used the wrong number for what the employee cost is, the FTE cost is." Sweet. Tell me what the right number should be. Or the cost of customer acquisition: "Oh, no, that's not the right number for cost of customer acquisition." Okay, well, what is it then? Tell us what it is. What are your assumptions? So I think there are things that people can do, but more than anything: give it a crack.

00:11:07: Johannes Schartau: Yeah, that's also my experience. Once you make these assumptions visible, it really transforms the entire conversation, and it becomes much more of a joint discussion about what we believe, what we can see, and then, like you're saying, if someone is brave enough to take the first stab at it, then it triggers that whole conversation and invites other people. And it's really useful. I invite people on this podcast also for self-help for myself, I guess. Here's one of my experiences. Often, people really understand the general concept and say, "Yeah, that's great. We should start quantifying Cost of Delay. It absolutely makes sense." Then they start building the first list, and they start working with that concept. And then at some point later in the process, there's this dynamic that happens. Someone will come in and say, "Yeah, that's a great list. It totally makes sense. At the same time, the CEO asked for this, and he wants these five things done by next week. So we have to kind of push that in there." What's the response to this? Why do we see this happen? What is going on there?

00:12:29: Joshua Arnold: Yeah, I mean, I've definitely seen that myself. No surprise. In fact, I've talked and written about this for many years. So the classic highest-paid person's opinion, the HiPPO, comes in and basically squashes everyone and says, "Yeah, all good and well. This is what we're going to do." I think that's part of the nature of human organizations. Rightly or wrongly, a hierarchy tends to emerge. Even when it's not formal, there'll always be some sort of hierarchy. And when you don't have that language around value and urgency, then it tends to default to authority and politics, and "you scratch my back, I'll scratch yours," and quid pro quo setups. So I'm well familiar with those. I think you don't have to win all of the battles necessarily. It's okay to allow the system to be how the system is, but still to chip away at it and try to set in place a structure that will help to inform better decisions. One thing that I have actually done previously is, when the highest-paid person, the CEO or whoever, swoops in and insists that this is the one thing, then I literally take, "Okay, well, this was our previous highest Cost of Delay or CD3 thing, therefore this one must be higher than that. So let's just give it a nominal value of that, and then let's see how people react to that." So I'll almost turn the HiPPO insistence that this is the highest-value, highest-urgency thing for us to do into an assumed Cost of Delay. Now, if it turns out that that's not accurate, then at least you can always point back and say, "Well, that was the CEO who said it was that, so we just ran with it." So you don't have to win them all, I think. You can still try to get to that sorted list, use CD3 to get to a sorted list by duration. It still drives a lot of the right behaviors in terms of thinking about the cost of queues. It drives the right behaviors around understanding that if you take the 20% that you can find within a bigger chunk that is worth 80% of the value, then it's way higher in value. A lot of the gamification that happens, even in systems that tend to be highly political, can still get some of the right behaviors to emerge. The more you can make those Cost of Delay numbers visible, I'd say that's probably the bit that will help to drive the change. But like I said at the beginning, you don't have to win all of the battles. We can still allow humans to be human, unfortunately. None of us are perfect. No organization is perfect. Even at Maersk, where we ran this, there were still adjustments that we would make. That's okay. It's not like we're wedded to the number. You don't treat it like a GPS signal that, when it says drive off the cliff, you just blindly drive off the cliff. No, you are allowed to make human decisions around that. No framework is perfect. Sometimes the model itself is just lacking the nuance of some sense of, actually, that other thing over there is more valuable, and I just can't quite articulate it. That's okay.

00:16:29: Johannes Schartau: So other than this, when people start using Cost of Delay, what can they expect to go wrong in the beginning, or where do teams and organizations struggle when they just start out using this concept?

00:16:45: Joshua Arnold: I'd say the first mistake that people tend to make is that they think the Cost of Delay is really just the burn rate of the team. They think that that's the cost. And it's like, no, no, no. It's actually the value side of the equation. It's not the cost side of the equation. Just being clear about that: it's actually more about the benefits. Don't worry too much about the costs. I'd say the number one thing is that if people test it and they have an allergic reaction to it, it's probably because they've gotten stuck on precision as opposed to being directionally accurate. In all of the work that I've done in multiple organizations, we're talking orders-of-magnitude differences from the top end of it down to the long tail of stuff that's towards the bottom. So you only need to be within an order of magnitude to still be more accurate than your gut intuition. Don't worry too much about getting to a precise number. Certainly if you're using discounting and all of that kind of net present value stuff, that's a level of precision that I don't think any of us really have around the benefit side of the equation. If we think we're bad at doing estimates on how long things take, the lack of accuracy or precision on the benefit side of the equation is way, way higher. And then the other thing I'd say related to that is: our teams are expensive to run. They're not cheap. And we're not looking for a benefit-cost ratio that's just barely above one. We're looking for, for a month of this team's time, we really need to be generating something like ten times that value in terms of the outputs. So if it's ten or nine or eleven, it doesn't matter. Those are still big numbers compared to the cost of the team, so you're in the right ballpark. That's what I'd guide people to be a bit more focused on. The other little tweak I would tend to make about how people do this is that they tend to think in really big chunks. They think about the benefits in terms of a whole year. There's something magical that happens when you slice the time down to a week or a month, or maybe a quarter at the outside, because that drives a different behavior than if you're doing it at annual or five-year level. I always try to articulate the Cost of Delay as a per-week or a per-month number, because then I can do some mental maths with folks around, "Actually, is it worth us taking this different path here? Yes, it will cost another five grand to do it that way, but the Cost of Delay is 25 grand a month. So yeah, I'd do that every day of the week." There's lots you can then use it for. It's not just for prioritization. It's not just about understanding the benefits. It then becomes a way of seeing all of the things that are going on. You don't need those to be super accurate, and I'd say a smaller time slice means it becomes a more useful key to unlock lots of doors, as Don Reinertsen puts it.

00:20:35: Johannes Schartau: We're already kind of transitioning into the next topic. I started out in the agile space, and what I noticed at some point was that this whole idea of projects was kind of always causing issues for me in my workplace. And then I noticed that on your website, you were sharing content, and you were very clear about the topic that you believe using projects as the default model for delivering value is problematic. Can we get into that? Why do you say that?

00:21:09: Joshua Arnold: Yeah, I mean, how long have you got? The way it shows up is so funny. I think we've come a long way. Massive kudos to the team who famously didn't agree on much, but they agreed at least on the manifesto and really pushed the whole software world quite a long way towards where we need to go. Moving away from the old "software is like an infrastructure project, and therefore you plan the whole thing out in inordinate detail, as if we're pouring concrete and you've got to get all of the details right up front," that classic old waterfall process. Thankfully, a lot of that is gone now. I think most organizations understand that the way you do software is different. I'm not sure if they really understand why, but they know that that's the accepted norm now, to not do it that way. I still think they have struggled with the why. But the one bit that hasn't really changed is the funding model, which is still stuck in project land. Even the organization that I'm at at the moment still thinks in projects for some of the things that it does. So trying to tease them out of that model and start to think more of a continuous flow of changes, sliced and diced small enough so that we can tighten feedback loops, iterate on it when we need to, all that good stuff. The old PMI definition of this is kind of a temporary team with a fixed scope and a specific set of resources. A lot of that's already been dealt with. We've got modern teams that tend to be stable, long-lived teams. They don't really have an end date on them anymore. So I think the traditional definition of a project is already well broken. But there are still bits of it. The finance team still tends to think of things in projects. Mostly that's just because it's a funding model that they can wrap their head around. There's a whole game that goes on around CapEx and OpEx. If you're capitalizing any of the work that you're doing, there are certainly ways to do that that don't require a project bucket that you do it within. But I would say that the language matters. Project implies this sort of beginning, middle, end, as opposed to the product way of thinking about it, which is really just a continuous flow of learning and delivery. I feel like the world has come a long way, so I don't have to fight that battle so much anymore. But there are still aspects of the software and product development world that are still a bit stuck in that mindset. Thankfully, it's improved.

00:24:18: Johannes Schartau: What I find fascinating is when I talk to people about this, they have this really visceral reaction of, "But we cannot stop funding or budgeting." Could you briefly outline, in your ideal world, what is the alternative? What do you offer people? What does it look like if you let go of that annual budgeting cycle, for example?

00:24:39: Joshua Arnold: Yeah, I mean, this was something that we had a crack at at Maersk in order for us to get to where we wanted to go. It was probably one of the key unlocks for the work that we did, among other bits and pieces. We basically gave them options. We were like, "So, we really want to move away from projects. We would like to move to more of a continuous funding model." And we gave them some options. You could either fund for a fixed length of time. You could fund a buffer of changes, so we'll bring you 10 or 20 at a time, and you fund that group. We may change the order that we do them in, but at least we'll know we've got a continuous funding round. Or the alternative would be, as each one comes in, you fund them just in time. The funny thing is, when you sit down and run through those options with them, they're like, "Okay, well, actually, if you break down the projects to an epic or a user story or a feature level, all of a sudden the level of investment is so small, none of the overhead is justified." So they're kind of like, "Well, we'll just do a quarterly funding thing," which was how it basically ended up. So they would basically just look at it and go, "We'll just continuously fund you every quarter." And this was a big shift for Maersk. They came from a very heavy project governance model with a centralized PMO. It wasn't easy to unpick it. The key was to go and make friends with the finance team and get them on board with what we were trying to do. A big part of what changed their mind was us showing them the feast and famine that the project funding drove. They would have no money, so they basically were just trying to survive, doing bits and pieces that they could until the next project was funded. And then they had all this money, and they became this massive magnet for a whole bunch of changes to come through. So, of course, the project got bigger. Typical stuff, right? I've seen it so many times, it's hilarious. Showing them that feast and famine, and the impact it was having on the teams, versus, if you give us this continuous funding model, we won't have this feast and famine problem. We can just have this nice, smooth flow, and everything can work a bit better. Again, that was part of showing them the queues that were building up, and then obviously this feast and famine as those queues were flushed through. Lots I could talk about how that worked, but I'd say the first sensible step there is really to go and talk to your finance folks. Help them understand the problem that you're trying to solve, why it matters, and propose some alternatives and see what they go for.

00:27:45: Johannes Schartau: You recently started talking about a concept called ProductDevOps. What do you mean by that, and what is missing if an organization has maybe good DevOps, but not really established product thinking?

00:28:02: Joshua Arnold: Massive kudos to the folks that popularized DevOps, because I think they really helped to solve a problem that Scrum had come some way on. By forcing us to try to do a ship and an iteration of the product every two weeks, that drove a level of continuous integration, continuous delivery-ish, on a two-week, maybe originally three-week, down to two-week horizon. And then the whole DevOps thing was just squishing that until it was daily, hourly, or 10 times a day. That's been a massive unlock in terms of your ability to move, and the flow through the system is so much better. But of course, you can always flow shit through the pipe. There's no rule that says your continuous delivery pipeline has to be full of gold. It could equally be full of crap. What I tried to articulate with the shift to ProductDevOps was more like: yes, fast and safely is really important, but we need to make sure we're coupling that to a focus on finding the right things for us to deliver and to learn quickly in doing it. Without that learning, you're basically dead. You're just shipping rubbish, and you have no idea whether it's actually working for the end user or not. That feedback loop is missing. Your ability to choose and slice the opportunities in front of you is a key part of the whole product development puzzle. DevOps on its own is an important ingredient, but it's not the whole thing.

00:30:02: Johannes Schartau: How do you figure that AI plays into all of this? How can it help us? What I see at the moment is that people really look at the efficiency side, or just creating more output. Could it also, for example, help us quantify Cost of Delay better, or what else can it do for us?

00:30:23: Joshua Arnold: Early days, right? We're really still learning what's possible. I'm old enough now to have seen a few of these waves of new hotness come through. We've had the Agile wave, we've had the Lean wave, we've had the design thinking wave, we've had the cloud wave, we've had the DevOps wave, the product wave itself. Each of those promised to change everything. And in some ways, each of them did change everything. But with each wave, there's still an underlying question that needs to be answered about whether we're actually making meaningful progress, however you choose to define that. I don't think there's any particular magic there with the use of generative AI. What I have noticed is that generative AI and all of these wonderful new tools don't fix the dysfunction. They tend to amplify the dysfunction. It basically puts your blind spots, your habits, whatever habits you had before, on steroids. So you need to be even better at selecting what is the most valuable thing to go after, because all of a sudden the speed at which you can go after one of those things is so much higher. A lot of the false friends out there are just as dangerous as they ever were. What I'm really heartened by is that Kent Beck gifted us some amazing truths in XP, and a lot of what I would consider to be the goodness in the Agile wave is really rooted in a lot of what Kent Beck put forward all those years ago. I find it amusing that in 2026, the way to succeed with your AI genies beside you, helping you do the stuff, are the exact same things that we were talking about way back in 2001. Here we are, basically 25 years later, and it's a lot of the same skill sets. It's the fast feedback loops, being choosy about what you build, unit testing, continuous integration. There's a lot of similarity between what we were trying to implement then and what we need in order to keep these things on the rails now. I'm super excited to see where this goes. I'm using it myself. My teams have 200-odd people here in New Zealand, and every day or every second day, I have to go and approve another half a dozen: "I want to try out this tool." It's like, "Okay, cool." Click, click, click. So people are picking it up and testing and learning and playing with it. To your question, I do think about how this can help with quantifying Cost of Delay. I do think that this will help us ask braver questions. Rather than us having to come up with the Cost of Delay model and figuring out what A or B could be and being brave, we can use the AI to be brave for us. We can feed it some information about how Cost of Delay works and then ask it to do a rough sizing. I'm sure you have as well: you can do a ton of product research using these tools. It's just one step away to then say, "Okay, give me a Cost of Delay of that particular thing." And it'll spit out an answer. That's not to say it'll be right. But would it be any less accurate than yours or my guess? Probably not. Ultimately, we're still making an informed estimate. I haven't put a ton of time and energy into it. I did try to build a little Cost of Delay calculator tool probably a year ago now, and then I just got really busy. I haven't got to the point of just adding a new skill, which is calculating Cost of Delay, and just adding that to my Claude Code setup, but one day I'll probably have a crack at that. Let's see where it goes. I'm definitely in the mindset: I'm hiring right now. I'm out on market, probably hiring another 100 people. So I don't see this as something that should be considered a threat to software developers or engineers in general. I do think it's going to blur the lines between who does what and how it happens. We're definitely going to have smaller, faster-moving teams, I think, and there'll be a mix of people and agents doing all sorts of things. Internally, where I am at the moment, we're using it all over the place to speed up outcomes, and in particular, speed up outcomes that we thought were not possible. All of a sudden they become, "Oh, actually, maybe if we chuck that at an LLM and see what it can make of it..." And it's actually just discovered a way of doing the thing that we didn't think was possible. Or it will change the calculus around the cost and duration of doing something. A lot of those modernization projects that just never would have seen the light of day because they were going to be too big and too difficult now become doable. But I also suspect that the future is going to have a lot more software in it. You just look at how the number of repos that GitHub are juggling has gone through the roof. It feels a little bit like we've gone from an analog world, wet film, to digital cameras. The sum total of all of the photos that were ever taken in history was something like 20 million photos, or some crazy number like that. We now take something like 2.5 trillion photos per year. I feel like we're making that same transition with software. So there's going to be a Cambrian explosion of what we've got out there. A lot of it will be slop cannon dross, for sure. And that's okay if its main reason for existing is to prove out an idea. If you then have proven out that idea, and now you need to go and rewrite it in such a way that it is more scalable and secure and all those sorts of things, happy days. But the cost of testing an idea now has just dropped massively. It's dropped to almost zero. I do think the cost of tokens will probably go up at some point, but in the meantime we can chew through some cheap tokens and test a whole bunch of stuff. I'm excited about where it's going. I feel like the bottleneck has shifted from "how do we get this done?" to more "what should we be building?" And judgment, good judgment, has become the scarce resource, not the ability to write code. Let's see where that goes.

00:39:11: Johannes Schartau: Alright, final question. You already answered part of this, but I was wondering: what other transformational challenges do you see for organizations in the future, and what can we do today to prepare for those?

00:39:24: Joshua Arnold: A little bit like I was saying before, I feel like there's so much truth in some of the principles that have come out of the Agile movement. I still think that the core challenge stays the same. I'm hiring for curiosity, and I'm hiring for a sense of urgency, and I'm hiring for good judgment. Those things aren't really going to change. We're looking for people who have the ability to adapt. I feel like those capabilities aren't going anywhere. That's sort of an ultimate truth. Those fundamentals don't change: the value, the urgency, the flow, the fast feedback, the small batches, the ability to do distributed decision-making. All of that stuff just gets dialed up to 11. Underneath it all, we've got all these awesome new tools, but at the end of it, people are still the multiplier. People are the bits that make the magic happen. I think there's going to be a lot of change in organizational structures and how that works. If you're still stuck in project world and dealing with long budgets, get with the program on some of those as soon as possible. Those slower, more predictable-world model things are probably past their use-by date, very much so. Shifting to the new containers of DevOps and continuous delivery, all of those XP practices, and having funding models and organizational designs that can handle a good mix of people and agents doing interesting things, those are all going to be valid in the future. And for leaders, I think you're investing in the team as a capability, and the output is almost something you can throw away and rewrite, basically. We're really more interested in speeding up those feedback loops. And I guess making the Cost of Delay visible will hopefully get easier as well. Let's see how that plays out.

00:42:08: Johannes Schartau: Fantastic. Thank you for your time, Joshua.

00:42:11: Joshua Arnold: Not at all. It's good to chat.

00:42:15: Johannes Schartau: As always, this podcast is sponsored by Holisticon. We provide companies with holistic support, from strategy to technology to organizational change. And if this sounds interesting, you can find all the information you need at holisticon.de. And don't miss our next transform together session on June 9th at 2:00 PM Central European Summer Time. I will be talking to Lili David. She's the co-developer of Sociocracy 3.0, and she will let us know how to change an organization using that approach. See you then.