{
"@context": "https://schema.org",
"@type": "FAQPage",
"mainEntity": [
{
"@type": "Question",
"name": "What are UX product metrics in EdTech?",
"acceptedAnswer": {
"@type": "Answer",
"text": "UX product metrics in EdTech, such as the number of user downloads or clicks on a platform, provide data on how users are engaging with an EdTech product. However, these metrics alone may not accurately reflect how the product is being used in real academic environments."
}
},
{
"@type": "Question",
"name": "Why might UX product metrics be misleading?",
"acceptedAnswer": {
"@type": "Answer",
"text": "UX product metrics can be misleading if they are not understood within the context of the technology adoption culture of the site where the EdTech product is implemented. For example, differences in school policies regarding technology use or the presence of other similar tools can skew the interpretation of these metrics."
}
},
{
"@type": "Question",
"name": "How does technology adoption culture affect EdTech product use?",
"acceptedAnswer": {
"@type": "Answer",
"text": "The technology adoption culture of a school affects how students and instructors engage with an EdTech product. Factors such as access to devices during the school day, the presence of similar technologies, and dependencies on other tools or instructor input can all influence the effectiveness of the product and the interpretation of UX metrics."
}
},
{
"@type": "Question",
"name": "What should EdTech companies consider about technology adoption culture?",
"acceptedAnswer": {
"@type": "Answer",
"text": "EdTech companies should consider whether students can access the product during the school day, if the product duplicates other existing technologies, and whether effective use of the product relies on other technologies or instructor input. Understanding these factors can help provide a more accurate picture of how the product is used and how to support its implementation."
}
},
{
"@type": "Question",
"name": "Why is mixed-methods research important in EdTech product evaluation?",
"acceptedAnswer": {
"@type": "Answer",
"text": "Mixed-methods research, which combines quantitative data from UX metrics with qualitative insights from users and administrators, is important because it provides a more comprehensive understanding of how an EdTech product is used in real academic environments. This approach helps identify potential barriers and opportunities for improving user engagement."
}
}
]
}
User experience (UX) product metrics – such as number of user downloads or clicks on your platform – are a compelling source of data for EdTech companies: they are easy to obtain and give an indication of how folks are engaging with your product. But what if these product metrics don’t paint an accurate picture of how your product is being used? What if relying on only UX product metrics led to erroneous conclusions about your product?
A great example is a recent EdTech research evaluation of an app product that was newly implemented in a middle and high school. The UX product metrics indicated that more students from the high school had downloaded the app compared to students from the middle school. If we took these metrics at face value, we might have concluded that there was something about the middle school that impeded adoption and use of the product.
When we dove into conversations with administrators and students, however, we learned that the UX product metrics were misleading. The culture of technology adoption at the middle school was actually more positive than the high school. We learned even more from talking with the students from the middle school who were surprisingly more familiar with the product and its features, and seemed to use the app much more frequently than students from the high school.
Our research team at Labs learned the importance of understanding the context in which an EdTech product is being used and the value of qualitative data from users. Even the best designed product is constrained by the site it is implemented in – or what we refer to as the “technology adoption culture” of the school an EdTech product is implemented in.
Given the impact of the tech adoption culture on understanding how users engage with a product, here we share three things to consider about a school’s technology adoption culture that may affect how your EdTech product is used:
1 – Can Students Access Your Product During School Day?
If your EdTech product supplements the academic experience outside actual learning instruction, consider whether students can access your product when you need them to. Does effective usage depend on students accessing the product at specific times throughout the day? Is your product reliant on phones or other tech devices?
For example, if your product requires students to check their phones to receive updates or complete tasks, consider whether students are allowed to access their phones or devices during the school day. If the school does not allow access to devices during or between classes, you must consider how this constraint of the research site may impact the interpretation of your UX product metrics compared to another school that permits phone access. UX product metrics must be understood in context.
2 – Is Your Product Duplicating Other Technology at the School?
Schools are implementing more EdTech tools now than ever before, especially in the wake of the virtual learning wave. It’s crucial to consider whether the utility of your product might be dampened because it duplicates aspects of other products being used at the school.
If students have access to multiple EdTech tools or apps that play very similar roles in their learning experience, what about your product stands out from the rest? Will students become fatigued during their decision making process of which product to use for similar purposes? Your UX product metrics alone can’t answer these questions. To increase the likelihood of your EdTech product being used and users gaining unique value from it, thoroughly investigate how your product will coexist with other EdTech already in place.
3 – Does Effective Use of Your Product Rely on Other Tech?
Many EdTech products rely, in part, on other technologies or input from instructors. If effective use of your product depends on, for example, linking to a learning management system or an instructor inputting assignments that link with your product.
It’s necessary to consider how impactful these dependencies are, and how they may skew the interpretation of your use metrics. How reliable are these dependencies over the academic term? If instructor behavior impacts your product use, how are you supporting them in the use of your product in addition to the student users? Without this context, your UX metrics may not show a full or accurate picture of product engagement.
As much as we would like EdTech to always be flawlessly implemented and used, the reality is that these products are not used in a vacuum. Hurdles will always appear that prevent the most effective user engagement. By understanding the realities of the context your product operates in through mixed-methods research – which requires going beyond your UX product metrics – you will be able to more accurately understand how your EdTech product is used in real academic environments.
The Accelerator at WGU Labs leverages the talent of 30-plus education and business professionals to boost your company’s impact and business scale. Interested in learning more? Let’s chat.
