This is a transcript of episode 343 of the Troubleshooting Agile podcast with Jeffrey Fredrick and Douglas Squirrel.

Boost team results by measuring carefully and using consistent methods to move toward your goal–with “the Improvement Kata”.

Listen to the episode on SoundCloud or Apple Podcasts.

The Improvement Kata

Listen to this section at 00:14

Squirrel: Welcome back to Troubleshooting Agile. Hi there Jeffrey.

Jeffrey: Hi, Squirrel.

Squirrel: So I’ve got my lab coat on. I’m ready to be a scientist because what we talked about last week was that we’re going to be using scientific thinking. So, am I going to need a lab notebook? And I’m going to write down all my hypotheses, and that’s how we’re going to test everything in our software teams, or are we going to be doing something else?

Jeffrey: Actually, we are going to be doing that. But that’s not where we start. We’re going to talk about four steps in the improvement kata. And the last step is about keeping an experimental record, which is very much like your lab notebook.

Squirrel: Let’s remind listeners this is all coming from Mike Rother’s book, The Toyota Kata. All the links are in the show notes as usual. Go ahead, Jeffrey, where do we start in following this kata, this repeated set of steps, that we can use to scientifically think about improvement in our technology or any other organization.

Jeffrey: Well, you know, it’s funny because on the website Mike Rother actually created, he calls it the starter kata, where he lays out the steps in the improvement kata in a very simple way and the different artifacts you need. We’re kind of looking at this in four steps from the improvement kata side.

First: The Direction

Listen to this section at 01:25

Jeffrey: The first is you have a direction or challenge, right? So this might be like you have a vision. This is the long term vision of where you ultimately want to go. This might be strategy, major elements. So it could be something like: we’re trying to improve profitability of this product line, or we’re trying to reduce the cycle time, or we’re trying to improve capacity or, we’re trying to improve quality. But you have some dimension which you kind of know is the direction you want to go. And that’s really important, that’s going to kind of frame and limit the kind of things that you work on. So you need to know where directionally you want to go.

Squirrel: But you don’t necessarily expect to get there any time soon. This is quite a long term vision, right?

Jeffrey: That’s right. This is often like a three year or five year or even longer vision. It might be something you never achieve. At Toyota, the kind of ultimate vision is what they call ‘one piece flow’. That’s not an ideal that they expect to ever actually achieve, but it’s their North star. And that North Star is useful because one of the analogies that Mike Rother uses for this is that it’s like navigating with a compass rather than a map. We don’t know exactly the path we’re going to take, but we do know roughly the orientation we want to end up going. That’s the first thing you do is to get yourself oriented around what’s the process and direction you’re working on right now, what’s the scope of what you’re doing, in which direction you’re going. So that’s number one.

Squirrel: Great! So what do you do next?

Second: The Current Conditions

Listen to this section at 03:04

Jeffrey: Number two, the second step you do is to grasp the current condition. This is something that this is part of that discipline that a lot of times people often actually miss. What is the current state? How do you know the way things are currently?

Jeffrey: So you might say like, ‘well, we’ve decided we want to improve quality.’ Well how do you measure quality currently? Do you have a good handle on your defect rate? Do you know when you find defects, when your customers find defects?

Jeffrey: You want to reduce the time it takes to have a certain data recorded from the time the event happens in the world until it ends up in your database. And so you’re like, ‘well, what’s the process?’ You need to know all the steps. What are all the handoffs involved? How long does it take for things go through the different parts of the pipeline? Really understanding the current process and the current outcomes in a very detailed way is an important element.

Jeffrey: It could be something like, do you know what features your customers are using? Do you know how frequently they use them? Do you know the user journeys that are on? And have you a lot of times, and this is for me, very interesting, a lot of times there’s a lot of emphasis on companies on collecting data, but they very rarely take the time to go and analyze it. And this is the time in grasping the current condition where we’re going to go and analyze that data. We’re going to use all those loggings, use all those metrics we put into the product, and now gather insight to really understand what the current state is, because we need to understand it before we can improve it!

Squirrel: Fantastic. Then the Toyota folks would go and have the managers stand within a circle drawn in chalk on the ground that was right next to the assembly line. So this is quite a visceral understanding. It’s not just we have some data in a database somewhere, and there’s somebody who has some idea of what our customers are doing. We’re actually sitting next to some real users and watching them bang their phones in frustration, because our app is so hard to use.

Jeffrey: That’s right, yeah. That that process is called Gamba, which is like, go see. So it means like, yeah, you go talk to the actual operators, you go see what they’re actually doing. You observe, and the circle is to make sure that you stay there and keep your eyes on what’s happening.

Squirrel: They weren’t allowed to leave. You got in trouble if you, if you went outside the circle. So once you’ve been in your circle, you’ve got this understanding, then what do you do? What’s the next step of the scientific thinking?

Third: The Target Condition

Listen to this section at 05:32

Jeffrey: The next step is you’re going to set a target condition. And so the difference here is, before you have your long term vision. But now the target condition is something you believe is achievable in a relatively short period of time. So this could be a number of weeks or perhaps months. This is not a years long vision you’re talking about. There’s something we believe we could do even if we’re not sure how we’re going to get there.

Squirrel: Yeah. We know what it’ll be when we’re there, but we don’t know what the route is. Yeah, we want happy customers, or we want customers who at least don’t hate us and want to throw their phones in the river, but we don’t quite know how we’re going to make them happy.

Jeffrey: Yeah, but we probably want to measure it in a certain way. So we might see something like, ‘from what we’ve just done, the analysis on current condition. We now can say something very definitively like, we have this many of our customers using our site at least once a month, and we want to increase our engagement of once a month customers from X to Y percent.’ Right? So we have the metric, we’ve now identified the metric. And this could also be we want to improve production.

Jeffrey: We want to have the number of things we’re creating go up or the quality to improve. But we have the metric now we can say, ‘yep, this is how we’re going to measure success, how we know we’re done.’ I mean, this can be something I remember from one of the improvement katas at Toyota was ‘we want to reduce the price of this component by 70%,’ something like that. You know, it was a very high percentage. So that’s the point. You understand the direction, you understand your current state very well. And now you have your target that you believe you could achieve over a number of weeks that would be meaningful to achieve. And so now we’ve set the stage.

Squirrel: And listeners should notice, by the way, that that’s quite different from some of the ways that you set targets in other systems. It’s not incompatible with something like OKRs or 4DX or something. But you’re setting this quite short term activity and the moment you achieve it, you’re going to be going on to the next one. So this is a highly iterative process appropriate for conditions of uncertainty. Go ahead.

Fourth: The Experiments

Listen to this section at 07:44

Jeffrey: Yep. And now we’re ready for you to get your lab coat out. Because the fourth process of the improvement kata is you’re going to experiment to overcome the obstacles that stand between your current state and the target state. What do we mean by that? You could think about this as there’s a set of things that you’d say like, ‘well, what prevents us from being there today?’ This actually can be very fun in a group, people come up with objections about why what you’re saying is not possible. And they’d say like, ‘that’s crazy! How could we double the number of users per month? We don’t even know why they come to the site.’.

Jeffrey: Great. Okay. One of the obstacles we have: we don’t know why they come. Or we don’t know this, or we don’t have that. We’re lacking something they need. Cool. Write that down. All those objections become obstacles. And so you’re going to create this collection of obstacles that you’re going to have in your obstacle parking lot. And you’re going to say which one— because this is, very like you said, this is very iterative serial process —which one of these obstacles are we going to try to overcome? And we’re going to overcome it by experiments. So we’re going to have our lab notebook, and we’re going to say, ‘what’s the experiment we’re going to run?’ We’re going to write our hypothesis of what we think is going to happen. We’re going to record what actually happened, and we’re going to say, well, what did we learn from that step? And we’re going to keep doing that cycle over and over again until the obstacle is overcome. Then we’ll have a new state, and maybe we’ll discover new obstacles along the way, but then we’re going to say, ‘great, now we’ve overcome this obstacle. What’s the next obstacle?’

Squirrel: And I just want to underline, we’re expecting some of those experiments to have negative results. That’s success. That’s what it looks like to be doing science. You try a couple of mixtures of chemicals, and they don’t turn into gold, and you say, ‘that didn’t turn into gold.’ A lot of us do an awful lot of alchemy. I’m guilty of this sometimes where we say, ‘well, this should have worked. And it just there was a special case. And so it didn’t happen, but it’s still valid.’

Jeffrey: Yes. That’s right.

Squirrel: The experiment tells you: that didn’t work! So you better try something else. So we better be used to the idea that there are successful experiments that have negative results.

Jeffrey: Yes. And there’s some art to this about choosing the right experiment, keeping the experiments short. And the idea here is, again, rapid iteration. This is something that I’ve seen used very successfully on data science projects. And you can imagine some of the obstacles are things like, ‘well, we don’t actually know where all the data lives within the company. We know we have it, but we don’t know where it is.’

Jeffrey: ‘Okay, we now we know where the data lives, but it’s not in our data science environment. It’s not in Databricks or something like that. Great.’

Jeffrey: ‘Now we’ve loaded it all in okay. Now what problems? Well, it’s not all mapped together. The tables don’t map. Great. We need to do that.’ And so you have a whole series of things and that you can identify as obstacles, and you overcome them one at a time.

Jeffrey: But the key thing is that in this experimentation phase, at any given moment, there is one obstacle you’re working on, and there’s one experiment you’re running, and that’s it. And if we’ve done that, those are the four steps of the improvement kata.

Squirrel: Fantastic okay. And by the way, the hint is in the name. It is called data science. I’m glad we do that. But we don’t talk enough about computer science, those of us who are programmers and developers and so on, we think of ourselves as being sort of disciplined, but we’re not as disciplined as scientists. And I think this kind of scientific thinking could be really helpful to those of our listeners who are finding themselves a bit adrift and saying, ‘wait a minute, I’m not sure where we are. We’re trying to work on seven things at once.’.

Squirrel: If you had this kind of discipline, if you were able to insert this scientific thinking into your culture, I think that would lead to a lot of clarity. And you might not go any faster, but you might know where you were going. And that certainly seems to be an advantage to me.

Squirrel: So, Jeffrey, I want to cover this in our final episode next week, how do you create that culture? How do you get to it? Because we’ve laid out some steps, and now we’ve said, ‘oh, this sounds great to do,’ but one of our favorite things to do is to go beyond that and say, ‘what do you do about it?’ Mike Rother is unusual in giving us a coaching kata that tells us more about how to implement this, how to make it happen in your company.

Squirrel: So, listeners, we’d love for you to come back and listen to us talk about that next week. Uh, and of course, you can also get in touch with us and ask us questions.

Squirrel: Something I do want to mention as well is that in just a couple of weeks, I’m going to be launching a new podcast with guests! So on that one, I’m going to be focusing on new material, new ideas from different folks. Jeffrey, you and I sometimes have guests. My podcast is going to be even more focused on getting those new ideas. It’s called the Insanely Profitable Tech Podcast. And if you’re interested, go on over to agileconversations.com, drop me a note, and I’ll get you all signed up for that. If you’re not getting enough podcasts from us.

Squirrel: But don’t worry, we’re not going anywhere, and we’ll be back again next week to cover the coaching kata. So please join us again next Wednesday for another episode of Troubleshooting Agile. Thanks, Jeffrey.

Jeffrey: Thanks, Squirrel.