Over recent times, Product Management has transformed from Art to Science. The maturity in product thinking has been accelerated by tools, evolving thought processes and problem solving skill sets. So much so, that the next age of product managers shall be called ‘Product Scientist’, whose core strength or job description shall be experiment design.
However, there are a few mistakes that prevent us from becoming good scientists. Let me take you through my memory lane over different organizations where I introspect some defining moments or mistakes in my product journey that helped me learn and unlearn.
Lessons That Shaped My product Journey:
1) Talking to user with a solution bias: Confirmation Bias wrecks Genuine Hypothesis
We have an innate desire to be ‘right’ all the time, every time. Hence, even as we talk to our users, we already start thinking in terms of how best to solve the problem basis what we already know. Unfortunately, to be an effective product scientist, we all need to take a step back and learn, unlearn what we know.
If I look back, we were trying to improve the retention of our music platform. The first thing that comes to anyone’s mind to improve retention in a music platform is “relevance”. The more relevant content the better the retention. I remember, when I asked a few users as well, they confirmed, yes it will make them listen more but our platform had the best curated content of our time. Surely it was relevant to the main segment we were catering to, then why were people dropping off? We had to look beyond the obvious and live the life of a customer to figure out what was amiss. It turns out time between two songs and inability to play sideloaded music were the biggest pain points, which we figured through our feet-on-street agents!
What if we had gone with an open mind earlier, we could have learnt this even faster!
2) Actioning without a why?
Bias for action is an asset for a product person, but actioning without a why is suicidal. This shows up more when we are into the execution mode and desperately want to check a few boxes that we got things done. We often think ‘If a certain thing is working for our competition, chances are it will work for us’. Unfortunately, this may or may not be always correct without going deeper and understanding ‘the why?‘. Another downside of this is, we are impaired by our experiences and are held captive by our own lenses.
I remember, our sister concern was doing really well in another geography when it came to hyper local, hyper simple buy-sell experiences. We followed suit, only to find out such constructs don’t work really well in some categories & geographies where our core strength lies and we had to modify the experience to accommodate our customers but only after the negative experiences that our users had to go through.
Although the trick was to learn and respond quickly but the blood bath could have been avoided, if we had gone deeper and understood ‘the why?’ better.
3) Looking at only micro level metrics without the big picture
The tunnel view of experiments optimizes for local maxima but disturbs the bigger picture by either shifting the problem elsewhere or deteriorating the overall experience which kills the product in the longer run.
In yesteryear, we were trying to improve the funnel conversion of booking bus tickets, we went deeper into the funnel and optimised the steps in the online flow and created tools to manage the offline steps. No matter what we did, the conversions didn’t seem to improve beyond a certain point.
After going deeper, talking to potential customers, we figured that the issue was not about the funnel that we were managing but the problem was elsewhere. Competitors offered similar solutions, but clubbed loyalty products with ticket booking that ensured there was some money left for the next trip. Another factor that had a detrimental effect was around managing communication for commuters to make decisions.
No matter how efficient we could have made our funnel, the skin in the game was loyalty that reduced entry barriers and better communication to manage the trip, which can’t be done effectively by just integrating 3rd party booking sites.
What if we looked at correlation between money in the wallet and conversion to look beyond our funnel?
4) Going for the overkill: Not keeping it simple
We product managers are dreamers and we want to conquer the world with our products but this enthusiasm leads us into a “silver bullet” thinking, where we give into the gamble of going all in!
Several years earlier, we wanted to compete with real estate verticals and while talking to customers the problem identified was related to lead quality. We tried connecting the dots and came up with a solution that we will build a landing page with all kinds of filters & capturing of additional attributes that helps buyers converge on the most relevant real estate properties. Given the complexity of the platform, it took us about 3-4 sprints to launch this.
The results were really bizarre, the overall conversion was poor. On further investigation, we found the problem was that the user looked at us as a ‘discovery’ platform where the intent is lower than what is observed in a vertical platform. Since the intent was low, any obstruction in discovery led to lower conversion.
What if we could have scaled down the experiment and gathered more insights early?
5) Not being proactive in managing stakes and expectations
The currency of a product scientist is credibility through results. Sometimes, more than even results, it is the empathy towards the users and also towards the stakeholders, who are trying to make sense of the product vision you defined on the impact in technology, business, marketing and sales. The runway to experiments depends on your proactiveness to co-create with the stakeholders. This not only mitigates the risk but also brings in different viewpoints which greatly enhances the chances of success. “We are in this together” is no longer just a war cry, it has become the way of working in order to consistently deliver results.
Some years back, I was frustrated at how businesses ‘dictated’ what needs to be done. I even abruptly left my job because it felt very suffocating. Now if I reflect back, I was so wrong! This meant I wasn’t able to take them with me, I wasn’t able to communicate the higher level purpose/ long term vision that I wanted to work on and I wasn’t able to give them space to co-own this vision and take inputs on validating ideas.
These are just some of the life lessons that shaped up my product journey in this pursuit of becoming a better product scientist. Looking forward to learning-unlearning and sharing more such stories.
Written by:
Jasjit Singh,
Associate Director, Dealer Experience & Platform Monetisation, OLX Group
