Conducting Impactful User Research

Posted: 1 Mar 2023

Whichever company understands their customers best will most likely win the market. Deep customer empathy helps you create a better product and positions it as irresistible to your target market.

The fastest and most effective way to get to know your customers is through user research.

This guide will help you learn from your customers through every stage of the startup journey. From pre-product to ongoing improvements. This should be an ongoing process rather than a box to be ticked. We’ll talk through how to approach user research and the different methods you can employ.

At Connect, we believe that user research is an underutilised secret weapon in the journey towards product-market fit.

Don’t find customers for your products, find products for your customers.

Seth Godin

If you think user research is expensive, you should look at the cost of building the wrong thing.

Mario Maruffi, Product Design Lead at Photobox

To find ideas, find problems. To find problems, talk to people.

Julie Zhou, former VP Product Design at Facebook

Having someone tell you that your baby is ugly is super important. That’s what our researcher does for us.

Jonathan Moshinsky, CEO of Stitch

What is user research? 🤔

Simply put, user research is the process by which you talk to your target customers. This can range from a simple conversation, to surveys, to asking them to complete tasks while you watch. The objective is to learn more about your target customers.

When you know more about them, you can:

  • Build the right product for them, with less time wasted building things they don’t care about.
  • Understand what they need and make sure what you’re building solves their problems.
  • Refine your positioning to ensure your brand and offering resonates.
  • Understand who you should be targeting and selling to.

🚨 Different names for “user research”

 

There are many different ways to refer to the act of talking to customers. Terms like “customer development” and “UX research” have slight differences, but at the very early stages I put them all together as “user research”.

I don’t love “customer development” as it implies a task that can be completed, nor “UX research” as it implies it’s just about understanding the user journeys. “User research” should be an ongoing, repeated job that covers all parts of your offering.

Afreen Saulat goes further: “I would actually push folks to think about person-based research. I have been moving away from the term user because it denotes that the person is simply using your product/service but actually, its a synergetic relationship. So, even seeing as “ethnographic research” may be beneficial”.

📚 Further reading

The Mom Test is a seminal book on customer interviews. It focuses on early stage validation of business ideas, but it is equally valuable for understanding ongoing research. It’s full of absolute gems and will make anyone a better researcher. Andrew Muir Wood adds: “the reason the Mom Test is so good is that it’s written by a non-researcher! He’s very good at explaining things plainly that are instinctive to researchers.”

This blog post summarises a story from Shreyas Doshi (ex-product at Stripe, Twitter and Google), and is a fantastic example of how to avoid some traps of customer research.

The best early stage research methods

There are loads of ways of conducting research. The most useful for early stage startups are those that are insightful, low-effort and repeatable. In other words, whatever gives you the highest leverage for your time.

Here are the research methods that are typically most helpful at early stages with some tips on how to run them:

1. User interviews 💬

AKA: Find people, and speak with them.

While simplistic, user interviews can give you a huge amount of information. It’s useful to conduct these types of interviews as much as you can. Even if you’re not trying to learn anything new at that moment, there’s always the chance of an off-hand “aha” moment.

User interviews are flexible, but make sure to plan how you’ll analyse them, and turn them into actionable insights.

Andrew writes: “I once worked with a startup that had done 150 interviews and had no idea what to do with all the data and recordings. Having a bunch of unstructured, rambling conversations with a random mix of people, without clear goals and structured analysis is likely to result in all the biases creeping in.”

💡 Author’s opinion:

I don’t think I’ve ever had a user interview that hasn’t revealed something interesting. Whether it’s a major insight, hearing a specific turn of phrase, or reading people’s body language when a topic comes up, there’s so much to take away any time you speak with a target customer.

How to run them:
Recruitment, or finding people to speak to, can take some effort. You can pull people from your existing users, from your sales pipeline, and from social media. You should advertise the fact that you’re trying to find users to talk to, although be aware you’ll get participants biased to those that know about you. Generally speaking it’s useful to think about the bias in any recruitment method that you use.

There are agencies and tools like UserTesting that can help with recruitment, but they can also be quite expensive.

Top tips:

  • Automate recruitment wherever you can. Set up a Calendly and share the link automatically with new users. Add the link to content that you put into the world. Any passive stream of people is a big timesaver.
  • Charlotte adds “think about where your target users are most likely to be found”. For example if you’re developing something for photographers, find clubs nearby or online, or if you’re targeting parents, try Mumsnet.
  • Let people speak – in an interview, just listen and guide with prompts. You’re not there to sell. Ask open-ended questions to get them talking (some examples below). Every time you ask a question, they reset to the ‘easy’ answer. The most unique insights are always buried under a few minutes of chat, so let them keep talking.
    • Tell me about how you currently do X?
    • What does “good” look like in your role?
    • What happens if you do X well? What about badly? What about not at all?
    • Anything else that doesn’t have an answer that is “yes”, “no”, or a number.
  • At the beginning of interviews, it’s useful to ask people to stack rank their biggest pain points. Once you start talking about the problem you solve, you’ll both feel like it’s the most important thing in the world. It’s useful to get unbiased context up front, before you dig into what you most care about.
  • Charlotte’s top tip is to think of the interview like a funnel – you know what you need to find out but if you start with that it will bias what you find so start broader and then slowly narrow to that – if it doesn’t come up naturally early on you know it’s not a pain point.
  • If they’re veering off course, nudge them rather than asking a whole new question. Say things like “you mentioned X earlier, can you tell me more about that?”. Or “you said Y, what does that mean to you?”. This helps them to focus, without the reset of a new question.
  • Prepare a script of where you’d like the conversation to go ahead of time. You don’t have to stick to it, but there’s nothing worse than realising you don’t have any questions to ask. Also the more consistent your interviews are, the easier they are to analyse.
  • Ideally do interviews with two people – one to ask questions and one to take notes. If there’s only one person, record the session using Clearword, Dovetail, or even Quicktime (with permission!). If possible, I’m always a huge fan of bringing engineers, designers, and others into interviews. Seeing customers in the flesh can be inspirational and give them extra context in their work.
  • If you do have more than one person involved, Charlotte suggests asking everyone who observed what their takeaways were – useful to get the perspectives of people who are looking through a different lens to you.
  • Finish by asking people “is there anything else I should have asked about?”. People will often have something on their mind, and be politely waiting for it to come up in conversation. Also try “who else should I talk to?” for interview recruitment.
  • Write down the words people use. This can seem like a small detail, but for brand and positioning it is absolute gold. People love seeing their own language repeated back to them.

📚 Tools:

There are loads of great tools that can help with your interviews for free. Zoom has free transcription, and Otter.ai also gives you 600 mins a month.

💎 Example: Intuit

Intuit is a huge and mature company, but the product team still undertook customer interviews that they called “follow me homes”. These were interviews with existing customers that took place in their homes or offices. Intuit said “Being fully immersed in the customer context helped us learn that they were trying to use our products with patchy wifi, or in noisy environments.” This was super helpful to building a market-winning product.

💎 Example: TransferWise

Once per month, the leadership team would personally handle customer contacts and complaints. Whilst obviously being a good motivator for the whole team, it also helped them build strong understanding of customers. While helping customers, they’d ask who they’re sending money to, what this transfer was for, etc.

This regular touchpoint helped build an empathy muscle that the product team adopted too.

2. User testing 🧑‍🔬

AKA: Asking users to try out your product.

The earlier you test designs with real users, the quicker you can learn. This not only builds a better product but saves valuable engineering time in doing so.

You can test designs at any point of the design/development journey: wireframes, low-fidelity designs, high-fidelity designs, clickable prototypes, fully built product. This should be fully baked into your product process.

How to run them:

  • Get your prototype ready. These can be just screens, but the more interactive the better. Zeppelin and Figma a great tools for making quick prototypes.
  • Work out what you want to learn. You’ll always get general insights, but the more prepared you are the better. Have a rough script that you want them to follow.
  • Ask people to try to use the product (e.g. “try to place an order”) and watch what they do.
  • In-person or remote can both work. If remote, make sure to get permission to record the session. Sharing this with the whole team is much more valuable than going back to them with your own insights.

Top tips:

  • Resist temptation to help participants or answer questions during the test. If they get stuck, that’s a very useful signal in itself. The objective is to see how they act when you’re not there so do not guide them.
  • It’s useful to emphasise at the start that there’s no ‘wrong’ behaviour from your users – if something doesn’t work as expected, then that means you built it wrong. Emphasising this can help users to relax a bit.
  • Ask them to think out loud as they go. If this doesn’t come naturally, ask questions like “what do you think will happen when you click that button?” or “what is this page telling you?”. If your prototype is a static image you can ask what they would expect to happen when they took an action.
  • User testing will never tell you for sure that people will use your product as expected. You’ll still need to analyse live results and/or run A/B tests. But it will tell you where people struggle to use the product. Use it to find issues for users, not confirm benefits.

3. Customer panel 📱

AKA: Having a group of relevant people that you can ask questions to.

It’s sometimes useful to have a group of people that you can bounce ideas off and ask questions of quickly. Recruitment takes a long time, so having people waiting to ask questions of can make things go much quicker.

 

How to run them:

  • Pick a communication forum, like WhatsApp or Slack. WhatsApp is less formal, but some people don’t like to use Slack.
  • Decide whether to group people together or to address them individually. Each has their merits.
  • Whenever you have a question that needs answering, ask your people!

Top tips:

  • Try not to overwhelm people by asking them questions too frequently. Save it for when you really want to learn something.
  • For the most important questions, ask people individually. With groups sometimes you’ll either get a few voices that dominate, or no replies at all. If there’s something really critical that you want to understand, drop people a note privately to maximise your response rate.
  • Make sure to touch base semi-frequently at least. It makes the panel feel useful and wanted, and stops things going stale.
  • Reward them if you can! As you’re not asking that much from them, this doesn’t need to be a financial reward. But it is nice to give them early access, discounts, or a shout out whenever you can. Make them feel a bit special.

4. Surveys 🧑‍💻

AKA: Asking lots of people their opinion.

Surveys are a useful way to understand the feedback of many people in a single go. It’s a great tool when you know what you want to learn, and when you have access to many people.

Surveys are useful when you’ve already done some research so you have an idea of likely answers to the question.

How to run them:

  • Pick your survey tool. Typeform is great for this, but there are loads out there.
  • Make a plan for getting respondents. If you’re getting them yourself you can use incentives like £/€5 per survey, or they enter for a chance to win a larger prize. If you need to, you can use tools like Attest and SurveyMonkey which give you a way to reach their audience (for a fee).
  • Work out your qualifying questions – you’re not going to want to hit everyone, so make sure you’re clear upfront about who you do want, and what you want from them. Try not to have the “right” answer too obvious if you can help it, so that you don’t have people lying just so they can complete the survey. For example, ask users to pick their age from one of multiple groups rather than just “are you 18-20? Yes/No”.

Top tips:

  • Ask people for their contact details and permission to follow up if you need to. You might see one of their answers as really interesting and want to know more.
  • If you want to get proof points for a sales pitch or investment, use multiple-choice questions. This will keep your data very tidy. If you want to learn more, give people space to write their own answers. However, Charlotte warns that this can generate a lot of data to analyse very quickly, and too many open text fields will likely reduce response rate.
  • Use a form-builder like Typeform to build conditional surveys i.e. a survey where you can ask different people different questions, depending on their early answers.
  • Don’t give too large a reward. If you do, you’ll attract people that aren’t relevant and might lie to get the reward.

Common arguments against user research

There are a few reasons founders give for not conducting user research. These are valid to some extent, but they shouldn’t get in the way of talking to your customers frequently.

  1. I’m not experienced at it and might get it wrong: Research is a skill that can always be improved, but that doesn’t mean that it’s not worth doing. Research is like communication – some will always be better than none.
  2. I can’t find people: Yes you can. If you’re building a business, you know how to acquire users. Use the same methods for your research recruitment. Some tips below.
  3. Customers will ask for faster horses: The Henry Ford quote goes: “if I’d asked customers what they wanted, they’d have asked for faster horses!”. This is sometimes used as an excuse not to speak to customers, because they don’t know what they want. Frankly, it’s a poor excuse. You don’t have to build what customers ask for, but you should still talk to them. Drill into what they really want. Make sure to ask people about what they do, and how they feel – not what you should build for them.

How to recruit participants 🎣

Finding participants can be tough – it’s a skillset in and of itself. Generally speaking, I recommend approaching it as you would finding customers.

Some useful sources:

  • LinkedIn – especially useful for B2B. Find people that look like your target customers and ask them if they’ll speak to you. People are surprisingly happy to help founders learn.
  • Shout outs on socials (personal and company) – this will obviously bias towards fans, but it will get the word out that you want to speak to certain people. You can pay to promote these to certain segments too, which can be impactful.
  • Agencies like Userinterviews and Respondent get participants for you. The downside is that they sometimes struggle to reach particularly niche demographics and the cost can add up quite quickly. They also bias towards tech-savvy users. However, the plus side is clear – they can take this whole job off your hands!
  • Don’t forget that you have access to existing customers too. You have access to them in-product, via email, and can even just pick up the phone (GDPR-permitting). It’s amazing how easy it is to just chat to people that use your product frequently. Charlotte advises to anonymise in notes wherever possible and to have a plan for deleting personally identifiable information (such as video recordings).
  • Afreen shared her consent form for participants here, so that you can see how to capture it. It’s important for participants to know how you’ll use their data, when it will be deleted, and generally what they’re signing up for.

📚 Tools for recruitment:

  • Phantom Buster is a really useful way of automating LinkedIn and Twitter outreach. It’s not super fast, because they don’t want it to appear automated. But it will keep your outreach up while you sleep.
  • Calendly is a game-changer, if you don’t already use it. You can define the times you want to set aside for research, and people can book in on their own.
  • Dovetail seems popular for recording conversations and analysing them later. Otter has a good free tier too, but it’s not specifically built for user research. One of our portfolio companies, Clearword, also records meeting notes automatically.
  • You can use Thank You Codes for payment. If you want to hook up integrations with your systems, you can use Tremendous. If you want to pay with pre-paid Visa cards, Tango Card is an option too.

Paying participants 💸

Paying participants for their time is a good idea for two reasons. Firstly, it makes it easier to recruit. Secondly, it makes you less likely to get bias in your sample – people willing to talk to free may just be fans or friends of the business, which doesn’t give you a well rounded picture.

Common amounts to spend are around $5-10 (or £/€ equivalent) for a survey, or you could have one larger ‘prize’ that participants get a shot at. For interviews, a rule of thumb from Andrew is $1 per minute for B2C, and $1.50 per minute B2B.

As a general rule, the greater the salary of the person you want to talk to, the more compensation they’ll need. There’s a point at which money stops mattering though – when talking to very senior business leaders it’s often not worth offering compensation.

Afreen adds that if you’re talking to people about emotionally sensitive topics or they’re an expert in an area that you’re trying to understand, you should probably look to compensate them more than the standard requirement for that time.

How to offer compensation:

  • Most common is Amazon vouchers. This is basically a “cash equivalent” for most people. Be aware though – this can bias who you attract. It also can be a faff to manage across countries – Amazon.fr codes can’t be used on Amazon.de, for example.
  • You can offer straight up cash too, although this can be a pain to manage. If you use an agency for participant recruitment, they can handle this for you and can give guidance on recommended amounts.
  • Charitable donations often go down well with people, especially higher earners.

💸 Thank You Codes

I built a service to manage participant payments: Thank You Codes. It lets you pay an amount per user, and sends them a link with a choice of voucher (including Amazon, and charity). It’s also completely free.

Some commonly asked questions about user research:

Who should do user research?

Others might disagree, but I believe anyone can do user research. The alternative is often just not doing it all and guessing about your users, which is far worse.

As a startup should be optimised for learning, it doesn’t make sense to gate-keep user research. Every interaction with a potential or real user is an opportunity for research.

When is the right time to hire a researcher?

There’s no right answer here – it really depends on the company. Two factors to take into account are:

  1. Your team’s comfort and experience conducting research.
  2. The level of uncertainty that you operate in.

To explain the second point a bit more – a company building a completely new category may need research earlier than one simply improving existing and well-understood tools.

Afreen shares some signs that you need a researcher on the team:

  • When you feel like you’re hitting a wall with the insights you are getting and need a broader view.
  • When you’re starting to grow rapidly, there are many product opportunities and you need to figure out which way to go.
  • If you are trying to go beyond conducting short interviews and gain deep analysis that can give you a roadmap for at least a year.

There is also the option of bringing a consultant in early to set you on the right path/help upskill the team. This can be a low-commitment way to explore a research function in your company.

Glossary 📖

There are several different types of research you can undertake. Each has pros and cons dependent on your speed of learning. You don’t need to learn these phrases off by heart, but it’s useful to understand the terminology and how the research can be applied.

Quantitative:
Using larger sample sizes to understand customers better (e.g. surveys, usage metrics). Great for answering a clear question. Charlotte adds: It’s likely to be more useful later down the line once you already have a good understanding of the problem area (otherwise you could be asking the wrong ‘clear question’ and not know).

Qualitative: Leveraging more subjective insights (e.g. customer interviews). Great for when you don’t know what questions you should be asking.

Attitudinal: Taking into account what people say to understand how they think. Great for when you want to learn more about how to position your product.

Behavioural: Looking at the actions people take. Great for when you want more certainty about what someone will actually do.

Generative: Understanding new areas to generate ideas and opportunities. (e.g. open-ended conversations). Great for when you have a lot of uncertainty or are pre-product.

Formative: Analysing how users react to an existing idea or product direction whilst it’s still in the design phase. (e.g. testing out specific designs). Great for when you want to refine an existing journey or behaviour.

Summative: Seeing how a feature is being used once it has been built (e.g. A/B test). Great for figuring out how well improvements work.