This is a transcript of episode 279 of the Troubleshooting Agile podcast with Jeffrey Fredrick and Douglas Squirrel.
Are you building a house of cards? On this week’s episode, Squirrel talks about how to test the value of what you’re doing using mark to market - and how not to be Silicon Valley Bank! Listen to find out why you should aim to bring new software to the market every day and what methods of testing you can use to make sure that you’re always adding value to your organisation.
Show links:
Listen to the episode on SoundCloud or Apple Podcasts.
Read the previous episode in this series here.
Introduction
Listen to this section at 00:11
Squirrel: Welcome back to Troubleshooting Agile. Hi there, Jeffrey.
Jeffrey: Hi, Squirrel. I’m so happy to be back because last week you promised us a follow up to explain this concept you have of “mark to market.” What is that? How do you go about doing that? You said it has broad effects and is good for lots of things, including the try-hard problem from last week, but it sounds like that’s almost a side effect of something much more central.
Squirrel: The try-hard problem from last week of course comes from a miscalibration, and marking to market is a term I stole from the finance world where people have exactly this problem. Listeners might have noticed that there were some banks recently that had some difficulty in actually giving their customers their money back. Part of the reason they got into that trouble is that they had assets on their books that had a certain value, and maybe those assets would have had that value in the future, but if somebody actually wanted to sell those assets right now today, in other words, to take those assets to the market and sell them, then they would not have the value that their books said they did, and therefore they didn’t have the value in assets they belived themselves to. Therefore, they couldn’t meet all the depositor’s demands. So the process that cures that which finance folks have come up with is not a perfect process, but it works a lot better, is to mark the assets to market. And there exist assets in finance that are hard to do that with as there are in software as well. But the notion is that if you possibly can, you want to say “What would happen if we actually liquidated this entire portfolio of stuff that we have here, what could we get for it?” If you do that frequently, then you get into a lot less trouble. An example of a company that got into a lot of trouble kind of on purpose with this sort of thing is Enron, who listeners might remember had a whole bunch of crazy derivatives and wacky assets and so on. They claimed they were marking to market, but if they ever took the those bits of rubbish to an actual market, no one would have paid a penny for them.
Jeffrey: FTX did something similar.
Squirrel: Oh yes, of course. Sorry I forgot. Cryptocurrency is prime case in intentionally not marking to market. It’s a sort of house of cards. Nothing is actually measured.
Jeffrey: Of course the accidental version is with Silicon Valley Bank that you mentioned, where they had some very safe assets, unlike FTX. Things that really do have value, but they weren’t something they could they could draw upon immediately: their market current market value was much lower and that caused all kinds of problems for them. But how do you apply this to software? I’m very familiar with it in finance, but what does software mark to market mean?
You’ve Built Something. Who Cares?
Listen to this section at 03:12
Squirrel: It means that ideally every single day—which is also the standard in finance, you want to mark to market daily if you can, that would give you the most accurate picture of your portfolio in its value—you want to take your software to the market and find out whether people actually want to buy it. So the extreme version of this—and few of my clients get this far, but you can get a long way toward it—is to release new software every day. That’s itself a challenge, but a pretty well understood process these days. Engineers know how to do it, even if they’re not saying how or they don’t believe you that they they know how. And then not only to produce the software, but to release it, give it to customers and get actual feedback. So if you have high enough volumes, if you’re in some sort of retail environment, then you can make the changes, get that software to the customers, and observe their behavior. People often talk about this as being data-driven, but they often pay much more lip service to that than actual action because often the data is not available. If you’re releasing your software every two weeks or every month or, heaven forbid, every quarter or every year, you don’t have an opportunity to change very much. So you don’t get an opportunity to see whether what you’ve just done has had any effect. You can wind up being just like one of these unfortunate banks where you have what you think is a bunch of assets, you think you’ve completed a bunch of features and changes to your software that are valuable, but you don’t know. You haven’t actually tried it in the market, and so you can’t tell whether they’re actually useful. So if you can get to the stage where you’re releasing very frequently, in retail that might be every day, in my biotech clients that might be every two weeks. That’s still astonishingly fast for them. Can you then layer on top of that some form of customer feedback. In retail, Mark Zuckerberg can make a change to the Facebook login page and see whether more people log in or fewer. And when he knows that and he can try it on a percentage of his people, then he’s got a mark to market. He knows whether that feature that somebody in Facebook built gave him more value or less. You may not be able to do that. You might be selling to large businesses who are not taking your new software that frequently or who don’t apply the changes or who aren’t as available. But with whatever frequency you can, you can mark to market by getting their feedback, by talking to internal proxies for them, maybe customer service or sales and saying, “Is this something you can sell? Is this something that will address the problem that your customers have?” All of those are ways to mark whether your software actually had the effect that you wanted it to. Astonishingly often when I introduced this idea and have people start doing it, a lot of the things they thought were really valuable turned out not to be. But that’s great news! It’s great to find that out, just like it would have been very helpful for Silicon Valley Bank to know that its assets did not match the value that they thought that they did, because then you can take action, you can change your assets, you can change your approach, you can change your marketing. There are lots of things you can do. But while you believe your portfolio has a certain value and it doesn’t and you don’t know that, you have a silent killer roaming around ready to knock you out and you don’t know it.
If That Sounded Familiar…
Listen to this section at 06:49
Jeffrey: Right. We’ve talked in this podcast now for many years and this sounds to me like what you’ve done now is kind of an evolution of something you used to talk about a lot, which was elephant carpaccio.
Squirrel: Exactly. That’s the first stage: you get that daily delivery.
Jeffrey: Right. So it sounds like the elephant carpaccio is the technical side. And now you’re going further to the business side and saying, “Great, now you have this regular cadence of delivery, be testing it in the market all the time. Don’t just assume it’s valuable because you build it. You want validation that there’s actual value here.”
Squirrel: Indeed. And lots of listeners may be saying, “Well, my company could never be that rational about it. There’s too much politics, there’s too many demands. The highest paid person in the room gets to decide,” and those sorts of things. The interesting thing is that you can mark to market without anybody’s permission. So if you produce some software, you could go ask some customers about it. There’s usually no dire provision that says you can’t talk to anyone or release something as a hidden feature that’s only available to certain folks or demonstrated or talk to sales. There’s usually something that you can do individually to get some impression of what the market value is and that can be very powerful information and can often change behavior if you can show it.
Jeffrey: I’ve got all kinds of ideas coming to mind. Yeah, this is a great idea. I think the powerful thing is once you have this concept just sort of asking, “What are the options we have for getting feedback on value?” You’ve mentioned several, like you could try contacting users, you could use proxies internally. Might be something you could release and see if people discover it. I’m thinking you could do an email to some segment of users. If they read the email, do they click the link?
Squirrel: Yeah, exactly.
Jeffrey: All kinds of things you could try to determine what would be proof of value to people. Now the ultimate proof would be like, not just do they buy it, but do they use it? Do you see that behavior change? To me, that would be the strongest signal. But the concept here of “How quickly can we test the value of what we’ve built?” It seems like a fantastic idea. I love the analogy of mark to market here because you’re right, otherwise we have this thing that we say, “Well, this is an asset.” I’ve often seen people do this, saying “Of course we have a moat in the market. We spent four years and $40 million making this product.”
Squirrel: That is no moat at all. It’s like having a ditch around your castle, but there’s no water in it, because anybody could come marching across and build something for $4 million and four months that might be better than what you have. Until you mark it to market, you have no idea.
Jeffrey: Yeah. That’s a fantastic insight. I love that. Can we take a moment to tie this back to our try-hard problem from last episode?
An Unproblematic Problem
Listen to this section at 10:00
Squirrel: Of course, I forgot we started there. So that was last week’s episode about folks who are trying hard but actually aren’t that competent.
Jeffrey: That’s right. The problem was there can be people who are very visible, very visibly trying, very visibly committed, but they’re not actually delivering. How does the mark to market approach solve that?
Squirrel: Well, see, I don’t even think about it. So what I’m thinking about is how do I measure the whole software team’s market value? What is this software team producing? And if the answer is zero, then that’s not going to end well for anybody. If the answer is it’s got significant value, but only in these areas, then I’ve got some pointer to where we should concentrate more resources, where we should boost skills, where we should improve. And if it’s, “Gee, it’s pretty good across the board,” then I know the team’s performing pretty well, that it’s focused on the right things and we’re producing value. I don’t find usually that the hard part is figuring out which people in the team might match that strategy once I know what the actual value is. That is difficult, it’s not like it’s a walk in the park to determine, “Oh yeah, you know, it’s actually Squirrel who really just doesn’t understand good design and anything he builds, people have trouble using it. And Jeffrey really understands good design. He builds beautiful front ends and people find it easy to use what he builds.” If you ask people in the team who are working with you and me, they could probably figure that out pretty quickly. So that’s not the hard part. The hard part is figuring out whether people want to use what it is in the first place, whether you’re talking about the ones that you build or that I build that drive more value, that create more opportunities in the market, that get us more revenue. Once you determine where the value is, it’s relatively easy to backfill that, to do the reverse engineering to work out, “Where did that come from?”
Jeffrey: I see. So you’re really just looking at “Is the system producing value or not,” before you worry about the people-components within it?
Squirrel: Yes, exactly. And that means that if you’ve got a try-hard, somebody who doesn’t know that they are not competent, we’re rapidly going to discover that because we’re going to look at what they’re producing. We’re looking at their contribution. We say this person is doing a lot in this area, but it’s not actually producing value.
Jeffrey: Right. That makes it makes perfect sense. I can understand now why you would see that this is your starting point you’re going to worry about, because you’re getting at the key thing. “Are we producing value?” That’s the number one concern that people should have that should supersede anything else.
Squirrel: If you have the right tools—and engineers know how to produce these tools, and know how to run an elephant carpaccio type process—then you’re able to mark to market frequently. It’s the frequency that makes the big difference.
Jeffrey: Got it. Well, fantastic. I appreciate you explaining that. I’ve got a new analogy in my vocabulary now, and having worked in finance now for several years, that means much more to me today than it would have ten years ago.
Squirrel: Fantastic. But even for our listeners who aren’t in finance, I hope this has been a helpful idea for you. Thanks, Jeffrey.
Jeffrey: Thanks, Squirrel.