You probably already know that a person’s preferences are subjective. What person A loves or thinks looks amazing, person B might view as just ho hum.
However, if you take a handful of people and ask their preferences on something, those responses can help you obtain a more objective result. That’s preference testing in a nutshell. Preference testing is also sometimes referred to as desirability testing.
Why Bother with Preference Testing?
There’s nothing worse than putting a lot of blood, sweat, and tears into designing something, only to have it fall flat once launched. If your audience doesn’t give a lick, that means you just flushed a lot of time, money, and energy down the drain.
With preference testing, you can save yourself some of that pain. Preference testing is a research tool that allows you to obtain feedback from users on multiple versions of the thing you want to test, before it’s launched into the wild.
Preference Testing vs A/B Testing
Though similar, preference testing is not the same as A/B testing. Think of preference tests as the thing you do before doing A/B tests. A/B testing typically involves evaluating 2 versions of something that already exists and focuses more on user behavior.
Preference testing is done earlier, while you’re still in the development phase, and you might test up to 6 variations, not just 2. You’re also focused on user perception instead of behavior.
Users are asked to choose which option they prefer most, and more importantly, they’re asked to specify why. Asking the question why helps you obtain qualitative data to drive your development and decision-making.
Preference tests are used primarily to determine aesthetic value and emotional appeal. It evaluates how well the concept or design communicates the desired message, and how it influences things like trustworthiness and credibility.
Ways to Use Preference Testing
The earlier in the process you can implement preference testing, the better. For instance, when designing an ecommerce website, you already know there will be several design elements involved. You can use preference testing to evaluate things like:
- Color palette
- Logo variations
- Overall site layout
- Content and copy
- Typography
- Headlines and CTAs
- Navigation and menu design
- Product photos vs lifestyle photos
- Promotional videos
- Package options
- Different advertisements for the same product
- Just about anything, really!
The biggest benefit of preference tests is that they allow you to obtain useful feedback early in the design process, so you have plenty of time and resources to make necessary changes before launching to the public.
Moreover, the responses received from preference tests are simple to analyze and understand. Often, there’s a clear winner, and the qualitative responses received can be useful for designing other elements for your brand.
Basic Anatomy of a Preference Test
If you’ve decided to try preference testing, there are several valuable considerations to keep in mind.
Define Your Main Goal or Objective
Preference testing isn’t a tool merely to discover what design a user likes better. You really want to structure your questions to dig deeper and address specific challenges you want to overcome.
For instance, instead of simply asking what version a user prefers of the iterations you are testing, ask something specific related to your goal.
So, if your goal is to create a simplified checkout experience, ask users which webpage iteration leads them to the checkout page the easiest, and ask why, along with any other follow-up questions that are relevant.
If your goal is to make your product photos stand out from competitors selling similar items, ask users which iterations capture their attention immediately, and again, ask why.
Always ask why a user has chosen their preference, because it’s the “why” that can help you better understand how to give your audience what they want most.
Define the Type of Feedback You Want––Qualitative or Quantitative
You can collect both with preference testing, but by asking that valuable why question, and creating additional follow-up questions that go beyond which iteration they prefer most, you can unearth helpful data to drive development.
Define What Elements You Want to Test
You can use preference testing for all kinds of things. Maybe it’s a wireframe, maybe it’s your brand color palette, perhaps it’s videos about your products or offerings. Sometimes you might want to do a format comparison, such as product photos vs product videos.
That’s fine too, whatever you need to do to get the best answers to help narrow down your decisions. Just be sure that what you’re asking them to evaluate aren’t barely perceptible changes––there should be noticeable differences so your users understand clearly what you’re asking them to judge.
Try to limit your test to no more than 3 iterations at a time, although you can go up to 6 if need be. Too many options can lead to decision fatigue, and make it more difficult for your users to pick their preference.
Define How You Plan to Measure Your Results
Remember that “why” question! Always ask users why they chose their choice. You can use a simple survey to gather your preference test data and leave space to expound on their responses.
You can ask multiple choice questions, or ask them to rank their choices based on a scale. Specific yet open-ended questions are also ideal to gain deep qualitative insights.
You could even offer word choice options, like asking them to share 3 or 4 words that best describe the iteration they are evaluating, or providing 3 or 4 words they can choose from that best describe it. Sometimes numerical ratings can be helpful, or even a mixed bag of all of the above!
Additional Tips for Effective Preference Testing
Specificity Matters
Remember that your main question and your follow-up questions should be specific to your goals. Specific questions get specific answers, which makes them easier to analyze. They can still be open-ended to leave room for expounding upon, but if they’re too general, you won’t get the best data you could have if you’d been more specific.
Avoid Confusion
If you’re preference testing smaller design elements that are part of a larger picture––for example, a navigation menu on an ecommerce site––narrow down the element you want feedback for. You can still show the full website so they can get a good idea of how it looks within its whole context, but zoom in on what you’re testing or highlight it in some way to help minimize confusion.
Ready to Get Started with Preference Testing?
Getting stuck in the development stage is common. We’ve designed UserInput to help you with your research experience. We’ll help you formulate your own preference test, assign reviewers to collect data from, and facilitate quick and easy results you can use to help drive development forward. Ready to get unstuck?
