🧪 Why A/B Testing Features Can Be a Double Failure

I was discussing with a friend the other day (he’s also a product manager) why running A/B tests on features can be a double screw-up.

First, it exposes that, as a product manager, you don’t actually know how to turn a feature idea into a clear question like “what exactly do I want to achieve?” — and then into some kind of dendro-fecal MVP.

Second, it exposes that, as an analyst, you don’t know how to assess the impact of a feature without depriving the product of 50% of its potential growth for a while.

Obviously, this is a very simplified recap of that conversation. But my advice to any product manager has always been the same: learn how to ask questions that can be answered using the data you already have — or using data from a slightly modified version of the product.