I’ve been doing a bit of reading on lean analytics this week. It has reminded me of the many MI packs I’ve seen, updated, and developed in risk management. (Note that I’ve not read the Lean Analytics book by Ben Yoskovitz and Alistair Croll though, which is where most of this stuff comes from.)
There is lots that makes sense (and isn’t anything new) in lean analytics. Using proportions and rates rather than absolute values, making comparisons across time, and so on, are all still good advice. The notion of a “Vanity Metric” is also a nice one to keep in mind: something that makes you feel good and looks impressive, but isn’t actually either very informative or actionable, is of little value.
The one thing that I do have a problem with, is the lean emphasis on the “One Metric That Matters”. This is meant to be a single piece of information that will drive everything in your venture. I can see how concentrating on the key MI would be advisable, especially for an early-stage business, but I think they are taking this a bit too far.
You can absolutely go too far in the other direction as well. I was once tasked to completely rebuild a portfolio credit risk MI pack. The old one had grown over the years, and was now so big that it took two weeks to produce (each month!), and it was suspected that nobody even looked at half the pages. So I started scoping out the replacement: what were the best parts of the old pack to keep (everyone had their favourites), and what new information should be added (everyone had lots of ideas). And of course; very quickly the planned new pack became even bigger than the old one.
So by all means have your one main metric, but do keep an eye on other relevant data as well. You won’t get swamped by a well thought-out set of relevant metrics. If for example you are looking at active users at two months after registration as your main figure, then make sure that you also know your early indicators: page views, registrations, actives at one month and so on.