4 challenges of assessing Product Manager's skills and capabilities
Why is it very hard and what to do about it
The first step to growing (or helping others grow) in your career is to understand where you are at. This usually implies assessing your skills, capabilities, knowledge, and performance.
Assessing professionals in any field can be a challenging task, but this is especially true for Product Managers. While some disciplines have clearer metrics and objective criteria to evaluate performance, assessing Product Managers' skills and impact is a more complex endeavor. Unlike engineers or marketers, the scope of a Product Manager's responsibilities and the impact of their work can be difficult to quantify. Their success often depends on intangible factors like collaboration, strategic thinking, and stakeholder management.
In this article, we will explore some of these unique challenges and (following our practical approach) provide some options and strategies for assessing Product Managers effectively, enabling a better approach to career growth or a better path to form high-performing product teams.
1. The diversity of required skills
Unlike other functions, product managers require proficiency in many different areas, as varied as the more technical aspects of agile and collaborating with engineer teams, the empathy required to do effective user research, and the fundamentals of business management and P&Ls.
There are several challenges associated with this diversity. First of all, assessing all of it takes time. As a manager, you will not know the expertise of all the Product Managers in your team in all these areas. As an individual contributor, it would be difficult to discuss your strengths and weaknesses in all these areas simultaneously.
What can we do about it?
Focus! Either as a manager or individual contributor, you need to start the discussion of what aspects are more critical to assess and develop for the current role, product type and maturity, and company challenges.
Decide what areas are more critical to improve
Deep dive into the gap analysis of those ones
2. The different requirements per product type or scope
The diversity of skills also arises from the different needs required by the different types of product roles. The abilities required to do product management in a mass consumer product will be very different than an API product targeted to developers, which in turn will be different from an enterprise accounting product (not to mention other specializations like B2C/B2B, AI, growth, etcetera).
While we can describe a somewhat similar “product function” in all cases, how we do that function may be quite different (while other functions, such as software development, could be more stable among these different companies).
What can we do about it?
Clarify expectations. After mapping your skills, you will realize that some would be more important for your current role (or your collaborator’s current role if you are a product lead).
For example, if you work with an internal product with a few users, learning A/B testing might not give you any advantage (in terms of improving your current performance or getting a promotion).
Make a clear agreement between the PM and Lead about what those critical areas are, and set clear expectations of how “good looks” for those.
3. The collaborative nature of the role
It is typically said that the level of success of a PM is the level of success of their product. However, the impact of the product is affected (positively or negatively) by other roles like development, UX, marketing, sales, operations. And even by the situation, like high-tech debt or multiple dependencies.
That doesn’t change the fact that the success of the PM is the success of the product. But, when thinking about skills and development, this reality may conceal the value that the PM adds to the product's success.
The natural consequence is that either “bad PMs” are promoted due to favorable circumstances or “good PMs” are punished and not promoted due to unfavorable ones. And in both instances, there is no real plan for them to get out of those scenarios.
What can we do about it?
Isolate factors and assess the ability to collaborate and resolve conflicts.
Start by acknowledging the potential problems, and assess the PM's contribution in that situation. As an example: if the impact is slow due to dependencies or lack of tech skills, we can still assess discovery results to check that what was selected for delivery was the best possible solution, even when it will take time to see the impact.
But this is not the end of it. As a PM, you should also tackle whatever problem prevents faster impact for the user and the company. Sometimes these problems would be outside your direct control, but you can still surface the problem, gather evidence, and facilitate decisions among participants to make things better. Many times PMs are uniquely positioned to do it, given their interaction with multiple areas on behalf of delivering better results. This is a critical area for the PM to develop, and going back to the 2 previous points, there are scenarios in which this should be selected as the top skill to develop.
4. Subjectivity and The Dunning-Kruger effect
The Dunning-Kruger effect is a cognitive bias in which people with a low ability or knowledge in a particular area tend to overestimate their competence and expertise, while those with a high ability or knowledge tend to underestimate their competence relative to others. This effect occurs because individuals who lack knowledge or skills may not have the necessary knowledge or skills to recognize their own deficiencies, which can lead them to overestimate their abilities and make poor decisions. The Dunning-Kruger effect was described in 1999 and observed in various fields and can result in overconfidence and inaccurate judgments.
What’s the risk here? Junior PMs (or non-experienced leads) will overestimate attributes. My personal favorite is Strategy, where so many people rank themselves or their PMs high when they don’t even have a strategy in place.
What can we do about it?
Making expectations explicit and coaching to gain shared understanding. Given the subjective nature of most product skills, this is not an easy task. For leaders, it requires detailing very clearly what a good result is.
A clear example can be Roadmaps. If you put as an expectation for some strategic skill, “Have a clear 6 to 12 months roadmap”, it would be very easy to create a crappy artifact to “tick the box.” But creating some explanation of what a good roadmap is can be demanding. Also, it may create a heavy template-driven process that can undermine the needed flexibility of innovative product work.
So fixing this requires a combination of an expectation definition plus the coaching work needed to gain a shared understanding of what can be a good implementation of that expectation in the context where the PM is at the moment.
Note: this is mostly the work we do at Continuous upskilling, starting with a detailed assessment, to later clarify in coaching sessions given each individual’s context, and using case studies to refine the understanding and abilities of each participant.
Expectations and Action Plans
In this scenario, full of ambiguities and lack of standards, having an initial assessment of performance is already hard. But that is not all! After understanding the status and skill development priorities, you must create growth expectations and action plans to get there!
If you are a leader, this can be the most fruitful thing you can do for your people. Marty Cagan says coaching “is the single most important responsibility of every people manager to develop the skills of their people.” So if you think this requires more time than what you have available, you may need to reconsider your priorities and time allocation.
What if you are an individual contributor? What can you do if your people manager is not creating those standards and action plans? You have to propose them proactively. No manager will refuse to discuss a proposal to improve a certain skill. They may have a different view of the priorities or what good looks like, but this is invaluable feedback you can get if you present a plan to debate!
In any case, I’m heavily working on these problems. If it resonates and you need further help, connect with me!