Machine learning questions every product manager must ask

These days, every product manager (PM) is anxious on how to ride the machine learning bandwagon. And rightly so, machine learning (ML) has caught up with the imagination of everyone and unequivocally, seems like a good bet to ride the wave of next generation product capabilities. I’ve had many a conversations with PMs who ask if using ML for this would be good? Given that most PMs are not machine learning experts, their inquisitiveness is understandable. I also know PMs who went ahead and enrolled for an introductory course in machine learning, only to later realize the course talked more about ML mathematical basics rather than practical decision-making on ML related use cases in products. 

To help them, I’ve built a framework for PMs to guide them on the decision making on whether to integrate ML element into their product and if yes, how to decide on its implementation across the product life cycle. And truly, none of this requires any in-depth knowledge of machine learning mathematical fundamentals. Read on.

To start with, machine learning shouldn’t go into your product for the sake of it. No matter what the advocates tell you, merely having ML elements in your product will not necessarily make it better. ML must suffice necessary use cases. What could those be? I’m mentioning five scenarios, meeting any of which, PMs must consider using ML elements.

Use Cases for practical machine learning integration in the product

  1. Helping users find what they are searching for in the product (improving search)
  2. Giving users, by recommending or clustering, what they might be interested in (recommendations)
  3. Classifying products and user inputs for better segmentation and ease of user experience (segmentation)
  4. Giving user predictive output, if that is a feature of the product (user expectation)
  5. Anomaly detection (payments, bots, etc.)

Once a use case has been identified and zeroed upon, then the work on actual ML implementation should begun. The market these days is flooded with open-source ML libraries, service providers with out-of-the-box ML products and abundance of available talent who can work on customized ML solution for your product. Whatever route you choose, there are important questions I’ve defined that should be answered before, during and after ML element integration into the product. Here they go!

Infographic – Key questions to ask while integration Machine Leaning elements in the product

Infographic-Key Questions to Ask for Machine Learning Integration into Product

Infographic-Key Questions to Ask for Machine Learning Integration into Product

Pre-ML Integration

1. Is the ML algorithm solving the problem to be solved?

Appropriate – This is the most basic question. It is important to filter, amidst the excitement of enabling the product with built-in ML features, if the ML solution addressing the exact defined problem in the product? For e.g., if your product design is a hindrance to discoverability, will a recommendation engine solve the issue completely? Maybe not.

2. How critical is the ML integration with the core user workflow of the product? 

Vital – Is the ML element really necessary for the core user workflow? Not every product might require a ML element in its core functionality. Adding ML might not necessarily add to product usability.

3. Is the product being held up for ML integration? 

Critical – Is holding up the product for ML integration really necessary? For e.g., ML element might be needed to make the product experience really personalized for the user. But one must also ask if the product has enough armor, as of today, to make it really personalized for different users? Maybe that would be a good feature to have once the product has a lot to choose from – in the future.

4. Is data needed being accurately captured for the ML to work? 

Fueled – All ML algorithms are powered by data that trains the model. It is imperative that analytics is already capturing enough data that can be fed into the ML model for training. If not, pre-existing ML models will never improve.

5. Do you have baseline and well-defined metrics to compare pre- and post- ML integration? 

Measurable – Absolutely essential to define what was the product baseline metrics for the intended use case pre-ML and what are the expected metric levels from ML integration.

During ML Integration

1. ML workflow must frictionlessly integrate with the product workflow 

Unobtrusive – Adding ML element should not alter the product workflow or delay its action in a way that it becomes difficult for the user to operate with. If that happens, while ML metric might still look good, the overall product engagement will suffer.

2. Product decisioning, bound by common sense and design logic, must over-ride ML decisioning if needed

Flexible – It is extremely important that the decision to over-ride ML logic must reside with the product manager and implemented, if necessary. ML models are after all, models only and can not always be relied upon over common sense.

Post ML Integration

1. Practical ML use case, as revealed by market exposure, might be different from that envisioned by the product designing team 

Acknowledge – Often, the users of the product might use the ML element for a different use case than what was intended by the designers of the product. A recommendation engine meant to increase the purchase quantum (units) for the user might result in her choosing just one, though from a wider variety of choices, courtesy the reco engine. It is still a use case – helpful for the user, but not necessarily translating into more buying. PMs must be able to acknowledge such feedback as the process helps in further improving the product.

2. ML products will make error in judgement. The trade-off vis-a-vis business objectives must be notified and observed

Evaluation – Sometimes, as mentioned in the last question, ML element does not necessarily augment business objectives. In such scenarios, it is important to evaluate the existing product and ML element functionalities, aiming for a better outcome for the product.

3. Learn to admit failure gracefully in the early product phase itself

Admission – And if it is revealed that the ML element isn’t really serving the intended purpose, rather rendering the product a little difficult to use, removing it should not be considered a regressive move simply because the element is called ML (something that is toast of the town right now)!

Anubhav, a data scientist, writes about new developments and future trends in the machine learning and data analytics domain.
He can be reached at anubhav@thinkbigdata.in
Follow him on Twitter at: https://twitter.com/think_bigdata

Leave a Reply