The folktale “The Emperor’s New Clothes” is a cautionary example of receiving and acting on bad information. In this tale, the emperor’s servants and citizens tell him what he wants to hear rather than what he needs to hear. Acting on bad information, the emperor naively parades through his empire undressed and ill-prepared for his public appearance.
EdTech companies often engage in user testing to collect feedback from potential users to inform their product development. In these research settings, users are introduced to a product concept or prototype and asked to provide honest feedback. Like the folktale, users in these research settings will sometimes offer false approvals of product prototypes, may incorrectly remember previous experiences, or be inadvertently led to offer responses that do not reflect their true intentions and behaviors.
This post will consider three ways to get accurate information from your target users during user testing in order to effectively “dress” your EdTech product.
1. Talk to Just Enough of the Right People
Good information begins with talking to the people who will provide the feedback you need to support your product development. In conducting user testing, focus your recruitment on your product’s ultimate target users. In our equity-first approach at WGU Labs, we intentionally over-sample populations often overshadowed by majorities in order to give a meaningful voice to this segment of users.
When determining user testing sample size, the goal is to reach — but not go beyond — data saturation, or the point where the research no longer yields novel, impactful results relevant to your product’s current phase of development. An unnecessarily large sample size will require a higher investment and can lead to a large, inaccessible data set and subsequent analysis paralysis. You can begin to recruit enough of the right people by:
- Recruiting users among specific fields, departments, or job roles, and according to identified user criteria (e.g., demographics, geography, organization type)
- Recruiting users who would be involved in the product’s adoption, implementation, and management (e.g., administrators, department chairs, technology support)
- Avoiding users with personal connections (e.g., family, friends, colleagues) who may offer potentially biased feedback
- Beginning with a small sample size (e.g., 30-40 for surveys, 5-7 for interviews or user testing sessions per user testing round) and increasing sample size as needed
- Viewing each user testing engagement as one part of a long-term user research commitment
2. Demonstrate Openness to Constructive Feedback
While testing concepts with potential users, demonstrate a genuine interest in open and honest feedback — especially feedback that may contradict the ideas and assumptions behind your product. This can be accomplished by:
- Providing examples of constructive feedback in your instructions (e.g., “You may see something that looks off, is unclear, or that doesn’t resonate…”)
- Asking for more details and thoughts through follow-up questions (e.g., “Can you tell me more about…”)
- Prompting users during prolonged moments of silence or obvious confusion (e.g., “It looks like this page is difficult to understand…”)
- Avoiding confirming questions that lead users to predetermined conclusions (e.g., “Would it be accurate to say...")
3. Ground User Testing in Real-World Application
Your products need to be designed to live within the user’s existing physical, technological, and cultural environment. During user testing, users will continuously assess how your product could fit into their existing lives. Gathering this information will assist you in understanding the feasibility of your product in the authentic environments of your ultimate user. You can accomplish this by:
- Testing the product in a physical environment that mirrors the ultimate use setting (e.g., classroom, college dorm, home office)
- Creating user tasks that mirror real-world applications
- Listening for “if” statements that reveal user’s perceived constraints (e.g., “if faculty weren’t so busy…”)
- Asking open-ended questions to understand user assumptions, motivations, and intentions to use (e.g., “When would you use this product?” “What problem is this product trying to solve?”)
The importance of aligning your product with user needs
Given that one of the reasons EdTech companies fail is because they build products that don’t align with user needs, user research is an essential part of product development. Simply administering surveys or conducting interviews and user testing sessions is not enough. For your product to be properly “dressed” for the market, you need to ensure that you’re getting the right information from your user research.
The Accelerator at WGU Labs leverages the talent of 30-plus education and business professionals to boost your company’s impact and business scale. Interested in learning more? Let’s chat.
