This is a transcript of episode 249 of the Troubleshooting Agile podcast with Jeffrey Fredrick and Douglas Squirrel.
Squirrel and Jeffrey have different takes on the value of estimating a tech team’s planned work at the beginning of a longer cycle, like a sprint or a quarter: Jeffrey says the discussion of an estimate often leads to insights and better outcomes, and Squirrel says you can get the same results without the estimates. Both agree that sharing and measuring against the estimates outside the team is counterproductive!
Show links:
Listen to the episode on SoundCloud or Apple Podcasts.
Introduction
Listen to this section at 00:11
Squirrel: Welcome back to Troubleshooting Agile. Hi there, Jeffrey.
Jeffrey: Hi, Squirrel. You and I were talking about doing a follow-up from our episode last week talking about estimates, and you-
Squirrel: You mean “lies,” don’t you? Estimates are lies!
Jeffrey: Laughter. That’s right, or predictions, or bets, depending on how you’re looking at them.
Squirrel: Bets aren’t lies, but estimates, man, not a fan.
Jeffrey: Okay, well, we’ll talk about that and we’re going to argue about estimates. But what really was unusual is not that you and I would disagree, but that you wrote a LinkedIn post, which is not like you. So what was this LinkedIn post that you wrote and why did you write it?
Squirrel: I’ve got these great friends at CTO Craft, Skipper and his crew are very, very clever folks. One of my clients came to me and said, “Hey Squirrel, that advice you gave me to not report on how well we’re delivering against our estimates, that’s exactly the opposite of what these other clever people wrote in their article.” And I said, “Oh, yeah, that’s because they’re wrong.” And he said, “Tell me more.” I said, “All right, I’ll tell everybody!” So they were like 4/5ths right. They got a whole bunch of different metrics that were really useful things like cycle time, and some really interesting ones like ramp-up time for your engineers. I thought it was great stuff, but man, they just managed to get one really kind of spectacularly wrong. They were arguing that you should report to management, to people in the executive team, the board of directors, whoever, on whether your technology team was meeting its estimates. And because estimates are lies, I just think that’s a crazy idea. So I wrote about why that was stupid.
Jeffrey: Now it’s great that you had that, and it’s interesting to compare and contrast because this past week my team closed that learning loop that I was talking about with our estimates, which I think of as predictions for Q3, we were looking back on Q3 OKRs what what we did versus what we predicted we would do. It was a really fun exercise, and one of the things that came out there were some things we didn’t get done that we thought we would, and that led to some really useful learning conversations.
Squirrel: But you weren’t having those conversations with the board of directors where you.
Estimating For Whom
Listen to this section at 02:28
Jeffrey: No, we were not. I would agree reporting engineering estimates and missing engineering estimates to management sounds like a terrible idea. It sounds like a way to take that tilted slider and just say “all we care about is predictability. The only way to get fired is to miss your estimate.” I think we probably talked about an example in the past of what it’s like to work for an insurance company, where they are so profitable that things can take as long as they want, at least for some insurance companies in the old days, and therefore everyone padded their estimates because the only way to stand out negatively was to miss your estimates because those are what was being reported on.
Squirrel: You were pointing this out to me before, Jeffrey, you put it really well, that it’s one of these great estimates that exemplifies gaming the metric. The moment you start to use this metric, you’re going to become slower.
Jeffrey: Exactly. It’s a general principle that once a metric becomes a target, it ceases to be a useful metric. Estimates are probably among the worst of those I can imagine. But I still find estimates tremendously useful and especially on longer-term time scales. I find them really useful, so it’s interesting that you and I are in having different views of estimates, where I’m not nearly as concerned about them as being lies because I think of them as being valuable to the team. It’s not management fodder, it’s not really something that you’re supposed to be reporting to other people, but I just find so much value both in creating the estimate—I see actually creating the estimates as a valuable activity, as a value-generating activity—and then reflecting on how things were different from our estimate as another value-generating activity. That doesn’t seem to be your experience though.
Squirrel: Yeah, it’s just not what I observe, because what I see is a significant risk that the estimates break quarantine. It’s really hard not to have them leak, and for somebody to say, “well gosh, you made some estimates, why don’t you tell the rest of us so that we can plan? Wouldn’t it be helpful?” and suddenly you’re back in the gaming the metric slowing yourself down world. So that risk seems high to me. And then the benefit seems not to be tied to the estimates. So let’s take the first one. You said that you found it helpful at the beginning of the quarter, because we’re talking here about a long learning loop, right Jeffrey? This is something you learned over not just one sprint or one week or one day, this is learning over a period. So at the beginning of the period, making an estimate helped the team…how? Again? What did the estimate itself do?
Jeffrey: Well, it’s not the estimate. It’s the estimating. It’s the act of generating the estimate among the team. I have often found it to be just a tremendously valuable activity to make sure we’re all thinking of the same thing.
Squirrel: So you could hold the estimation session and have this fantastic discussion and understand each other really well and make sure that you’re actually talking about the same thing and then burn all the estimates.
Jeffrey: Sure! Yeah, absolutely. We’d have a huge amount of value from that. That’s exactly right.
Squirrel: And I just have this great idea, Jeffrey, where you don’t bother coming up with the estimates so you don’t have to burn and add to the carbon in the atmosphere, you could just have the discussion about what you’re going to build.
Well, Yes, But Actually No
Listen to this section at 06:09
Jeffrey: In theory, yes. In practice, that doesn’t match my experience. Again, you could still burn them afterwards, we can talk about that separately. But there’s something about when people say, “look, I don’t understand why this is more than two days.” and someone else says, “What do you mean, this is at least two weeks!” Right? This is realistic. I’ve had people say this in a session. I just haven’t seen generating that kind of clarity of disagreement so succinctly, other than through asking people, “how long do you think this would take? What’s your kind of gut feel?” And when you find out those gut feels are far apart, that’s such an efficient way to check in with people on their sense of what’s happening. If you could say how you communicate that that doesn’t involve an estimate based on time, I’d love to hear it. I just don’t know. I would love to add more to my repertoire. I’m just saying this is the thing that I’ve seen work so effectively when we have those kind of discussions.
Squirrel: Maybe it comes down to the same thing, I’ll grant you that, but it sure seems to me that I see that kind of dialogue happen in teams using elephant carpaccio that are delivering very frequently. First of all, they’re not having it only at the beginning of the quarter. They’re having it continuously as they’re preparing the team.
Jeffrey: Yes, 100%.
Squirrel: We’re agreed about that. They might have a discussion about “how many slices is this,” or “can we get this done today,” or “what’s stopping us doing this this week?” “Why can’t we have this experiment live for real customers right away?” When they have that discussion they may say, “I’m going to need to add a whole new database for this,” and somebody else says, “But this person’s already done it.” Yes, there may be some discussion of time there, but what doesn’t happen is somebody says, “well, let’s figure out whether it’s seven and a half days, or four and a half sprints. Is this a 13 or a eight or a four or a 9 million?” I just think that kind of forced estimate discussion is not helpful. If it comes up organically, I’m happy with that.
Jeffrey: I don’t really see the difference, because you’re talking about people using whatever their local estimate currency units are. I don’t really care what a team particularly is talking about, but they’re probably are having some frame where they talk about level of effort.
Squirrel: Maybe here’s the way to say it. I wouldn’t say you start with the number of local units, days or sprints or whatever, but start with the effort and the steps that go into it. If that winds up being a discussion about time and somebody says, “yeah, but that would take at least three weeks and this couldn’t possibly take three weeks, nobody could take three weeks to add a password field,” then fine, let the discussion go that way. But I just think it’s not so helpful to start with “Now our job is to get out our planning poker cards and come up with whether this is an eight or a 13 or a 15 and a half and three quarters.”
Jeffrey: And why not? In my experience, “playing poker” is incredibly cheap. You just one, two, three, show your card. What’s wrong with that?
A Little Bird Told Me
Listen to this section at 09:33
Squirrel: “So, Jeffrey, could you just share with the board what those estimates are? That seems really interesting. How many of them were thirteens, and how well did you do on those? Were those really accurate?” See, that’s just such an easy conversation for someone outside to have.
Jeffrey: I guess I haven’t experienced that. I just haven’t experienced that problem. I will say on the longer term things, I’ve done plans going out a year saying, here’s what we’d expect to do. Here’s kind of what we’d expect to deliver over the course of a year, something roughly like this. This is one of things that comes with doing a startup and wanting to raise money.
Squirrel: By the way, I don’t object to this. This is so abstract and long term that I’m much less uncomfortable with it, because it’s not likely to lead to the problem that I’m concerned about where somebody says, “well, you said it would take exactly 47 weeks and 32 hours and it’s the 33rd hour. Why aren’t you done?” You’re not making a prediction of that variety. You’re really making a prediction in that case. “I predict that we can get something of value with lots of vagaries and difficulties and things that happen that we don’t expect within a year. Therefore, the payoff will be in time for our investors.” That I have no trouble with whatsoever, because it’s really not an estimate.
Jeffrey: Okay, interesting. I agree that there’s a scale between that kind of day-by-day, here’s the work we think we can get done today, or what we think we can get done tomorrow. There’s that very micro…planning, estimate, alignment, agreement, plan-making, whatever you want to call it. Then there’s the multi-quarter element. But there’s this middle one, where “overall this plan is going to take three quarters, what what part of it do we think we’re going to do within this quarter?” It is a completely artificial planning horizon, just as a way of saying “let’s break this thing up into some amount of chunks.” I still would say we found some value in that.
Squirrel: I don’t have massive objections to the way that you’re doing it. I just see it warped and twisted so often. The example I cited in the LinkedIn article is real from multiple clients where someone said, “Yeah, Squirrel, this project is right on track. I’m here in management. We know just what’s happening. The graphs show the velocity is really good, and the burn up chart shows the team’s nearly finished,” and they’ve never seen any working software.
Jeffrey: Yeah, okay.
Squirrel: That happens as a result of estimates so often. Someone feels comfortable with them, they think they’re truth! They could mistake the lie for a truth.
Jeffrey: Okay. Now I think we’re getting back into an area massive agreement again. Yeah. Estimates should not be considered a balm. Someone should not take comfort in the estimates. They should take comfort in the fact that they have seen working software.
Squirrel: That’s really the currency of value. But, Jeffrey, there’s something else that we want to talk about. I’m not sure there’s enough time this time. Could we come back next time maybe and talk about the value at the other end? Because we’ve talked a lot about how much value you find in estimates that people make at the beginning of some period like a quarter. I think actually the thing we didn’t get to talk about today was what the value is at the end, because you got this great retrospective value. What do you think about picking that up next week?
Jeffrey: I think that sounds fantastic.
Squirrel: Excellent. Thanks, Jeffrey.
Jeffrey: Thanks, Squirrel.