My Product Management Interview

My most recent post on LinkedIn cross-posted here.

————————————

A few months ago, I was asked to speak to graduates aspiring to be PMs at a startup school about my product management interview process. Now, to be clear, I haven’t worked for any of the valley unicorns and I’ve surprisingly never had formal interview training. Most of my process is “startup organic” by making mistakes and absorbing feedback from others, as well as books like Cracking the PM Interview.

So, with that in mind, here’s my interview process and some questions.

Background Notes

  • I like to ask the same questions. I find asking from the same pool of questions, enables me (or a team) to better pattern match the candidate’s response
  • I lean towards open-ended case style questions where the question is really a story and we build along the way with follow-up questions
  • I prefer questions that don’t have a right or wrong answer. This is much more about understanding how someone thinks and less so about remembering exact frameworks
  • I generally avoid looking at resumes and/or LinkedIn profiles. I sometimes can’t avoid this if I have to solicit and/or screen the candidate but when I can, I find this helps me eliminate any potential bias
  • Each of my questions have a specific goal, whether it’s strategy, design, analytics or other. Ultimately, I believe product management is a mindset

Questions

Q1: Tell me about a (software or hardware) product experience that you disliked? (something you haven’t worked on)

  • How do you describe the product and background? From a UX perspective or more high-level? From a customer perspective?
  • Are you specific?
  • Let’s talk through some edge cases
  • Are you inductive and/or deductive?
  • How would you fix it? Let’s design, whiteboard, and iterate?
  • What if the fix didn’t work?

If not obvious, my goal with this question is to see if you have product empathy, if you can focus on what you think the problem is, and how you would design to fix and test it. I generally steer this question towards design (personal bias) but that’s not required.

The “not so great” answers are often very high-level discussions about why iTunes sucks — but without any product detail. Use the whiteboard and start at the top-left so you have enough room to answer.

Q2: You are the PM of Google Music and you need to present to the CEO a strategy to compete with Apple Music. Discuss

  • Do you ask clarifying questions? How opinionated are you? Do you create a hypothesis? Are you open to alternative ideas?
  • How do you organize your thoughts?
  • And I like to interject with alternate potential angles-of-attack (e.g. geo, biz model, go-to-market etc.)
  • Where does your thinking bias towards? (e.g. market vs product vs model, or other)

Honestly, this question could literally be about any major product between two companies. I intentionally choose something crazy big to see how the candidate narrows, organizes, and focuses.

Using the whiteboard is a must here. The worst answers are very disorganized in their approach and often only consider one insertion point and/or may not have a strong hypothesis. Sometimes, I may even ask the candidate to try to outline some slides to see if they can organize their ideas.

Q3: You are the CEO of an airline and you have unlimited budget (full reign to change anything you want with the plane or the airport). Design a faster way to board a plane.

  • How out-of-box and/or creative are you with your thinking?
  • Are you analytical? Did you think about the edge cases? I usually hint along with scenarios you may have missed (e.g. handicap, families, seating preferences)

Failing to ask any clarifying questions is not good (and this is thematically true amongst most of my questions). As mentioned before, there are generally no right or wrong answers. Some answer with very creative ideas: “park the plane into the airport and let people board directly as they arrive to the gate.” Others have very in-the-box answers but maybe very customer-centric: “mandatory use of a mobile boarding pass and potentially giving people a real-time seat assignment based on preferences and order of arrival.”

As I’ve written, the goal from these questions is not necessarily to rule you out as a candidate but rather to identify your strengths. Are you creative, analytical, or organized? Then I determine if that best matches what we need at the company at that time.

Q4: Flow diagram the product development process (week to week) (eg Agile, Sprint or other)

  • Can you organize your thoughts?
  • Have you really managed and iterated on a product with an engineering team?
  • How do you deal with different scenarios (e.g. CEO asks for a feature last minute; or your team keeps missing dates)?
  • What ratios and graphs would you like to have or review in this process? Sometimes, I’ll draw some odd plots of odd ratios or parameters and ask the candidate to interpret (for example, a burn-down graph that doesn’t burn down until the day before the sprint ends).

Usually, I can pretty quickly ascertain whether the candidate has actually iterated a product with an engineering team from v0 to v1 and to v1.XX. This is not to say that all product managers need to be hands-on project or program managers, but this question provides more data points to match against the role we need in the company.

Q5: Psuedo-code a load balancer

  • Can you write an algorithm?
  • Can you draw a simple architecture diagram?
  • Do you know basic data structures?
  • Can you optimize and determine edge cases?

With this question, I’d always start with an architecture diagram while asking clarifying questions. The best outcomes usually start simple and then iterate on edge cases and performance. I don’t really care about programming language or syntax but mostly about whether the candidate has some analytical chops to earn the respect of the engineering team and whether he/she can think technically particularly when writing requirements for something like backend APIs. I like this example of a Load Balancer because there are so many opportunities to increase complexity while continuing to optimize, so you can easily determine the technical ability of the candidate.

Q6: You are the PM of the Gmail app for the iPhone (generally indifferent about which mail app and which platform, but it should be something the candidate uses). You’ve been asked to present an analytics report to the management team. Outline your slides

  • Can you define assumptions and goals to measure?
  • Can you identify the key metrics? What ratios are relevant and how would you represent this data (e.g. over time, per user, per session?)
  • How would you get and normalize this data?

I love this question, and the product can really be anything, but I like to choose products that the candidate has used. The “not so great” answers tend to be very disorganized and/or are lacking very obvious actionable metrics. I can usually tell pretty quickly how much time this potential PM has spent in growth or engagement — the vocabulary they use to describe an action or metric can be revealing. That said, just because you may not know the right terms, doesn’t mean you can’t deduce it and I’ve definitely had experiences interviewing candidates that had obviously never performed analytics but were smart enough to deduce what they would want to do.

The second part of this question requires organizing these metrics into presentable slides (at an outline level). I come back to slides often because I feel a large part of product management success is championing a strategy and plan and winning over your various constituents (employees, peers, management) and so becoming a pseudo-salesperson is a must.

Q7: What are you most proud of from your career?

This question is very telling. I find those that have managed teams often discuss moments where they had to carry a team through a tough period. Those that have been more IC (individual contributor) will often talk about how they designed a specific feature which was received well. Those that lean more project management will often highlight how they shipped their product on time and so forth. The answers are varied but they can signal strengths and humility.

The “not so great” answers are always too broad and do not highlight a specific accomplishment. This is certainly in the bucket of questions I’d suggest prospective interviewees to think about before starting an interview process.

Summary

In short, these are a subset of ~15 types of question templates I use. I’ll decide on the pool of questions to ask with my interview team based on what we need for the role (e.g. hiring a VP of Product would put greater emphasis on being able to mentor product managers, resolve resources, recruit, break strategy down into tactical roadmaps).

Lastly, I immediately (almost always the same day) summarize my feedback on the candidate via email to the interview team (after everyone has had a chance to complete their interview). I will often also score the candidate’s performance on the questions I was responsible for asking. In the end, we, the interview team, have a quick meeting, discuss the candidate, and make a decision.

I would love to hear reader responses on how I can improve my process, what I might be missing in terms of questions, and what tweaks I could make.

It should also be noted that I’m not sure I could reasonably pass my own interview process! But, hey, it’s all about surrounding yourself with smarter people.

Reflecting on the Productivity Category

My most recent post on LinkedIn cross-posted here.

————————————

Having spent the last 4 years working in the productivity category, I wanted to share some learnings from the space. Note, everything below is slanted towards an early-stage startup ultimately selling prosumer to SMB.

1/ Individually vs Group Useful
Sometimes called single-player vs multi-player mode but the premise is the same. Is your app useful to the individual or useful to a group (eg workplace).

For example, Asana is useful in a group but not so much as an individual. Evernote is useful to the individual but not so much in a group. Rare apps like Dropbox are both but this is not a requirement. Given that productivity apps typically touch personal data, they are not generally viral. Brainstorm ways to be both individually and group useful.

2/ Direct-to-Prosumer vs Top-Down
Is your app B2C or B2B – the trend is to drive prosumer adoption and then to sell to the SMB. Yammer pioneered this model; Xobni extended this model and in some instances it can even backfire such as when Microsoft asked their employees to uninstall the Xobni plugin.

If your app is direct-to-prosumer, you need to think about whether you can really get to 100s of M of users or 10s of M at a really high price-point. Evernote is probably the best example and is still struggling with accelerating freemium growth.

With SMB deployment, you can charge a higher SAAS price-point plus drive more seats. Ideally, you can achieve this without a sales team like Slack and Github have demonstrated but realistically, you will need an inside sales team and ignoring this reality is why I’ve seen many productivity app developers fail.

3/ Replacing In a Category vs Creating a Category
Are you an app the SMB already pays for or something they don’t yet pay for? Slack, as an example is a new category in that SMBs did not previously pay for chat. However, if are a “Todo” app, you are probably competing with an already existing tool such a Jira that the SMB pays for. Getting a company to switch tools is hard and thus why I recommend targeting super-SMBs when replacing in a category.

4/ Create Lock-In
Most productivity apps aggregate some cloud data (files, CRM etc). In the PIM (email, calendar, address book) category, we aggregated email accounts (Google, Exchange, iCloud). Being an aggregator means we are the presentation layer. But without any content exclusively stored in our system, the user can switch presentation layers without penalty.

To create lock-in, some options:
(a) Store some content exclusively
(b) Require upfront customization such as Salesforce
(c) Introduce paid; this will be the best thing you can ever do and will improve all of your metrics
(d) Achieve network effect. Hard to pull of but if you get it, hold on to it!

5/ Be Pervasive
Productivity is a lifestyle and is integrated across personal and work. Although mobile-first is my recommendation, don’t discount laptop usage at the workplace. Apps must be avail on all screens otherwise you are destined to fail.

6/ Have a Macro Thesis
Product management in productivity is hard – there is no 80/20. Workflows are unique to each individual and attempting to change and behave like a coach is exceptionally hard except when managed down (eg everyone has to use Expensify). It’s best to embrace the existing workflow and improve while also maintaining a macro product thesis. Without a thesis, your resultant app will look like the Settings dialogue in Office.

7/ Don’t Be Too Smart
As I’ve written previously, I believe predictive intelligence will be a new layer on all apps. But in the productivity, trust is always the #1 feature. Failing to sync an email will be an immediate deal-breaker. Optimize on precision (accuracy) vs recall (# of results) and ask the user when unsure versus being too smart. Although most contact mergers are pretty good, the 1 out of 10 times they fail is why many don’t embrace.