The Importance of Discovery Testing

A fresh idea to improve your website has hit you like a lightning bolt. It’s exciting and new (or it’s something your CEO wants because your competitor is already doing it) and you need it now! You already have an agency in mind, and the authority to kick off the project.

Let’s gooooo!

Hold on a second. We all get enchanted by shiny new things sometimes. But what do you actually know? You know what you want, and you have the means to get it, but there are several other questions you need to answer before you commit to any new project.

Do your audiences find this valuable, and will they be able to use it?

If you are charging, how much will you need to charge to get a return on your investment? Can your audiences afford it?

Does this support your organization’s mission, and is it in-line with your values?

What are the actual project costs? Not just budget; but also in-house production workload, training/on-boarding staff, and re-allocating other company resources to get the project launched.

Can you make it better than your competitors?

First, let’s define “Discovery”

The Discovery phase of a website project is the first, and most important, phase of the web development process. During discovery, the project team will use research and testing to gather and analyze all the critical information they need to ensure successful outcomes for your organization and audiences.

Research 

• Client and customer interviews, surveys, and questionnaires.
  
• Defining key performance indicators (KPI).
  
• Generating reports from historical data.
  
• Technical Research, and proof-of-concept prototyping.
  
• Wireframing and user flows.
  

Critical Information 

• Profiles for all target audiences.
• The client's business goals.
• The market, competitors, and contemporaries.
• The website's scope, features, and functionality.
• A project plan for the content, design, and technical implementation.
• Risks, limitations, and guardrails.
  
  

Successful Outcomes

• Your audiences want or need this solution.
• Your audiences will be able to adopt the new solution.
• It is possible to complete the project within the technical, budget, schedule, and manpower constraints.
• This is in-line with the organization’s mission, vision, and values.
• The project’s solutions are better than any of your competitors, or will be a new innovation in the market.
  
  

The Types of Discovery Testing

These five types of testing should not be treated like an ordered checklist. They should be utilized in whichever order or frequency best fits your project Discovery process. Then after Discovery is complete, regularly repeat these tests to ensure that you have not strayed from the original project plan.

Testing Usability

Developing small-scale prototypes for individual features or upgrades, and asking your users to review before the entire product is built. We encourage our clients to test every sketch, mockup, and prototype throughout the process to gather incremental feedback about how their audiences interact with the design and features.

These prototypes will vary, depending on what you are testing, and what you intend to learn from the test.

Qualitative Value

Simply the user's response. Asking your audiences directly what they love about your current product offerings, what they dislike, and what is on their wish list. Learn if they would pay for a premium experience, or are they content with basic functionality.

Quantitative Value

How effective is your project idea? Ask what are the key indicators your audiences are monitoring to determine the success of adopting your project. And ask them how much they would expect these factors to improve for them by switching to you.

Testing Demand

Testing the demand for a new feature is a simple process, and can help avoid wasting time on a feature that only a very vocal minority of your users actually want.

To run this test, just add a link to the new feature on your current live website. The feature doesn’t exist yet, but you can drive these clicks to a landing page where they can sign up to be notified when the feature is launched. They also become great test subjects once the project is underway.

By collecting analytics on how many clicks, and how many subscribers you receive in comparison to your average website traffic, you will be able to accurately understand if there is a demand for the feature.

Testing Feasibility

If you are trying out a new technology, or creating a brand new feature from the ground up, start with a small-scale proof of concept. Utilize as little code, content, and user interface as possible to know for sure how the remainder of the development process would need to go. The proof of concept is meant to be throw-away code. Do not invest time producing a production-level feature, when you’re not yet sure the end result will work.

Not only will we learn if the feature is literally possible (not just theoretically); we will also have a better understanding of the limitations of the technology, the risks involved, and the time it will take to complete. While you have the prototype up and running, share it with some of your target audience to get a head-start on usability testing.

Testing Viability

The most difficult test in the group, but even if every other test comes back with great results, ultimately it’s the final deciding factor when it comes to moving forward with a new project or not. Your business is a complex system with internal and external stakeholders that you have a responsibility to. You have to collect and analyze every perspective from these stakeholders to determine if your new project will hurt the organization in any way. Not just changes in revenue, but also reputation, customer satisfaction and retention, potential for legal action, and more.

• Can the sales guys sell it?
• Can the security guys secure it?
• Will the funders be willing to fund it?

In a large enough organization, there may even be a separate department with different marching orders producing a project in direct conflict with the one you have in mind. Or the resources you are counting on may have already been allocated elsewhere. Without thoroughly interviewing your internal stakeholders, you run the risk of digging the entire company into a hole that it will not be able to dig itself out of.

The Types of Discovery Testing

Test early and test often. Allowing your target audiences every opportunity to provide feedback “direct from the source” will ensure that your new project is delivering exactly what they need in exactly the right way. “What should we test?” and “How often?”, are easy enough to answer: everything and always. The more important question is “How do we get enough test subjects?”

1. A dedicated customer discovery program. Recruit a task-force of testers from your current audiences that are excited to see what is coming up next.
   
2. Advertise for test users in a public forum, or related industry group.
   
3. Volunteers from within the company or members of the association.
   
4. Conferences and trade shows. Give out SWAG in exchange for feedback.
   

Applying Discovery Testing to Your Project

Most of the time, when a client comes to NeigerDesign to implement project objectives, they have already been thoroughly vetted internally, and are a clear need for their target audiences. Our discovery phase begins in the middle of theirs—where we only need to focus on testing the execution of those project objectives. But regardless of who initiated the research, by the end of the Discovery phase all of these testing techniques should be leveraged at least once to collect and analyze critical information, and ensure successful outcomes.

Concepts for this blog post have been inspired by and adapted from:

Cagan, Marty. Inspired: How to Create Tech Products Customers Love. Audible Studios, 2018. Chapters 47-56