This is a transcript of episode 305 of the Troubleshooting Agile podcast with Jeffrey Fredrick, Douglas Squirrel.
Know what users think of your software as soon as it’s done - the secret is immediacy and frequency.
Show links:
Listen to the episode on SoundCloud or Apple Podcasts.
Listener Feedback
Listen to this section at 00:14
Squirrel: Welcome back to Troubleshooting Agile. Hi there Jeffrey.
Jeffrey: Hi, Squirrel. We had some feedback from listeners this week.
Squirrel: Yeah, but not quite the kind we we like as much. What happened was that we accidentally uploaded first, I think it was more or less blank. There wasn’t much on the podcast, the opening music and the closing music and nothing else. And then we uploaded a previous version of the podcast. So surprisingly, lots of people still listened. So we’re glad that you’re out there listening. Maybe you wanted to hear it again, but it wasn’t quite what we wanted to do. We really appreciated getting feedback from listeners who told us that that had happened, and I thought that was a very good illustration of the value of working in public and getting rapid responses.
Jeffrey: Yeah. So even if some of them had, you know, made a bit of fun of us and, you know, made comments about, you know, John Cage and, you know, 403 and The Sound of Silence, they thought maybe we were doing some avant garde art experiment. But the feedback was great. And it actually reminded me of a tweet of yours about the feedback and that the important part is immediacy and frequency. And you also said you had some skepticism there about annual objectives, quarter releases, monthly sprints. And I thought that would be great to talk about, especially given that we had that rapid feedback for ourselves.
Squirrel: And one thing I should say is that we didn’t have a process in place for getting that feedback, which we now have. So there’s something that we will be doing now that will let us know more quickly that the wonderful recording that we worked on, that we know exists, didn’t actually reach you. And that is precisely what I push all my clients to do much more quickly. There was one in the past week who’s getting feedback, say, every three months or four months or six months from clients.
Squirrel: And guess what? When they’re waiting for the software team to get something done that’s going to take three months, it often takes four or 5 or 7. And that means that the cycle time, the immediacy of their feedback, is very slow. What they’re getting feedback on is something they worked on months before, and so it’s not very immediate. And the frequency, of course, is very low. It’s not happening very often. And if you have low immediacy and low frequency, the value of your feedback is vastly diminished.
Squirrel: So what we’re going to do is make sure that we have much more rapid feedback on the podcast. But what we want listeners to do is to figure out how they can get much more rapid feedback on their objectives and their customer responses and the quality of their software. Because if you could, “what if you could do it every day?” is the question I always ask, because thatt’s one of yours, Jeffrey. Today is not too soon to get feedback on something you’ve done, and certainly six months from now is way too slow.
Jeffrey: Yeah, and I think what really stood out to me about this, and why I thought it was useful, is I am a big fan of, you know, say, monthly retrospectives or quarterly retrospectives and annual retrospectives. And I think there’s something about that to take time and look back over a longer period of time. I think where the mistake comes and I think this is what I’ve seen sometimes people say is, oh yeah, I’m going to hold this item until we have that, you know, retrospective in two weeks or a month or something like that.
Squirrel: Or until my performance evaluation, you know. “Yeah, this person isn’t really doing everything that she needs to. But, you know, we’ll bring that up in six months when we decide whether to give her a bonus.” No! Tell her today!
Jeffrey: That’s right, that’s right. And I’m a fan of the OKR process. And one of the things actually, this resonated with me, is some recent conversations about our internal OKRs where we had to say, look, the point about the OKR, yes, it’s a quarterly.
Squirrel: I always have to note OKRs is Objectives and Key Results. It’s a method of getting feedback. Go ahead. Get feedback to your staff. Yeah, go ahead.
Evaluate Regularly
Listen to this section at 04:12
Jeffrey: Yeah, we have this quarterly goal, your quarterly objective and key result, but we want you to be evaluating it at least weekly. You know where are you. And so it’s not this is not just a bookkeeping exercise. You know, you’re supposed to give in our terminology. You give it a RAG status Red, Amber, Green to say whether you’re are, you know, off track, whether you are behind or whether you’re on track and, and basically it’s not enough to go and say, “yep, we’re behind” or, “you know, we’re off track,” or, you know, “at risk”, no, no, no.
Jeffrey: If you say you’re at risk, we also want to know, what are you doing to get back on track. And we want you to be doing that every single week. Or even, you know, every single day. What we’re doing today to try to get us back on track. Did it work and be essentially looking and reflecting on that all the time? It’s not something you put off just to the end. So that’s what struck me about this, because I’ve been seeing people who are have these quarterly cycles or monthly cycles, but then they defer the conversations and defer the feedback and the thinking until those time periods.
Squirrel: And don’t get me wrong, I’m a big fan of periodic reflection, of doing objectives. You don’t want to evaluate someone’s bonus every day. That wouldn’t be very helpful, but you certainly want to give feedback that frequently. You can think of it as like circles within circles, right? So there might be some annual process where the board of directors gives feedback to the whole company on whether you’re achieving investment goals. They definitely don’t want to do that every day and shouldn’t, but you want to get your software in front of customers every single day if you possibly can, because that’s going to tell you where you are and what I like to call the glide path. Jeffrey, have you ever- we’ve talked about this before, but I can’t remember your answer to this -have you ever landed an airplane?
Jeffrey: I’ve actually used to fly gliders, so I have.
Squirrel: I thought so, yeah.
Jeffrey: Yes.
Squirrel: So, so many listeners will answer that, “well, no.” As most people that I ask, Jeffrey can answer yes, but so that means, Jeffrey that you can help them out by telling them what a glide path indicator is. Did you have one of those on your glider?
Jeffrey: Yes. What we had in the glider would be something that would tell us what our rate of descent was. So we could say, you know, are we on track to make it back? You know, the thing about a glider is you get pulled up up, in the air. We’d go to like 4000ft. And then you have to be thinking, you know, “do I have enough altitude to make it back to the airport where I’m going to land?”
Squirrel: Because you can’t just turn on the engine and go back.
Jeffrey: Haha that’s right.
Squirrel: You can’t go around and try again.
Jeffrey: Exactly.
Squirrel: You got one shot. Excellent. I may start using that one now, Jeffrey, because I didn’t realize it was gliders that you’d flown. But the point is that inside any airplane you’ll have some kind of indicator. And a frequent kind, at least in engine powered aircraft, is a little picture of an airplane. And it moves up and down, and it could be bang in the middle, which means you’re headed for the runway. But much more frequently it’s a little bit high, which means you’re going to overshoot the runway. You’re going to land in the trees beyond, or you’re too low. It’s below the line. And that’s showing you that you’re short of the runway. You’re going to crash before you get there.
Squirrel: And the point of the glide path indicator is not to have it bang in the center every single moment. The point of the glide path indicator is that you’re always moving it, and it gives you very frequent feedback, all the time. You’re a little high, you’re a little low, you’re a little high. You’re still high, you’re still high. Now you’re a little low. Now you’re on track. Now you’re a little high. Now you’re a little low.
Squirrel: And what you want is to be able to look back at that once you’re on the ground and to say, hang on a second. We were up. We were really high the whole time. Maybe we should pull back on the engines.
Squirrel: Or in your case, Jeffrey, you know, we should start the descent sooner. Something we might want. We might want to change something. So next time our feedback is not so consistently negative, that would be really useful. But what you don’t want to do is get to the ground and say, “you know, I think we were kind of high. I’m not really sure. But, you know, these trees, they they seem to be holding up the airplane. Something’s wrong here.”
Feedback from Customer
Listen to this section at 08:16
Squirrel: So the the point is, and this is something I emphasize so often to my clients, look for feedback from customers. Get your software in front of of people who will use it, even if you have to use proxies, as often as possible. I aim for for once a day, which seems outrageously fast to many people. I’m going to get production software live to real customers once a day, but if you can head toward that, even if you can’t hit it, that’s going to be so much faster than the cycle time that you probably have today.
Squirrel: The natural cycle time that comes about if you have, say, six month evaluations or annual performance reviews or something and you’re you’re holding your feedback, you’re holding the information, don’t hold it. Get the information, whether it’s about your staff or your software or about your quality or anything else. Can you get it today? That’s the question I’ll pose, and the challenge I’ll pose to our listeners.
Jeffrey: I really like that challenge because it really leads to very generative thinking. You know, you’re going to come up with new ideas, you’re going to do something different than you would have otherwise. We see this right now where we will release software, and then what I always want to look at in our Kanban board is, I want the last column to be ‘used in production’.
Jeffrey: So in a sense, we don’t really care that we released it. We want to know did people use it? How many people are using it? We actually have targets about how many people we think should be using it. And part of that is, you know, well, do they even know that it exists? How are we going to let people know that it exists? And once it’s out there, is it working? Are people getting the value from it we expected?
Jeffrey: And we’re not going to know those answer to those questions unless we’re calling people and talking to them. We’re looking at what they’re doing on the site and really investigating. It’s not enough to just deliver it and then kind of allow things to take their course, at least not to me, it’s not. We want to really know is this are we getting the value, are clients getting the value that we hope to deliver? And if not, what can we do about it? And and your thoughts of like, how can we get that feedback sooner is the kind of thing that leads to very different approaches that people might have.
Jeffrey: I remember we had a new version of our software at at TIM, which is a company that had a product called the Trade Idea Monitor, and we changed the input screen. And then we had a, you know, had people give people the option of using the new screen. And we were looking at, very regularly, how many people were using the new screen versus the old screen. And the product manager would call people, they would look and see the people, you know, “hey, I saw you were using the old screen. Why is that?”
Jeffrey: And that was extremely useful. Sometimes they simply didn’t know the new screen existed. Other times they said, “oh no, I prefer, you know, the old screen for this reason.” And then we could change things based on that feedback. But it only happened because that kind of proactive nature of actually seeking the feedback sooner, that really drove that behavior on his part and then had that learning benefit for the product.
Squirrel: And we had to think differently about it. I remember that some people were actually concerned that we were spying on them. They said, “how did you know?! You know, do you have a camera here over my desk?” They didn’t quite understand. This is a little earlier in the evolution of online software and software as a service. They didn’t quite understand how much we could observe about their behavior. We reassured them that we weren’t checking out their secret information they were typing in. We were just observing whether they used it or not.
Squirrel: But the the, the thinking to create that mechanism by which we could tell was what we were triggering by forcing ourselves to try to get much more frequent feedback. So that’s our challenge for listeners. And of course, the other way to keep in touch is come back next Wednesday when I promise, no four minutes, 33 seconds, no weird silences, no art experiments. We will actually have a podcast for you next Wednesday on Troubleshooting Agile. Thanks, Jeffrey.
Jeffrey: Thanks, Squirrel.