End-user testing is a phase of software development in which the intended audience tests the application to provide feedback to the development team. Typically, companies will employ strategies for end-user testing such as observation, artificial environments, surveys, and focus groups. These methods may be invasive and may provide a less accurate representation of actual users when applied in settings where the software is merely supplemental to a larger experience. The rise of software in spaces like amusement parks, museums, and fairs begs for a different approach to end user testing. Hence, the driving question: How can user testing on a software application be effectively conducted in a public environment? This study designs and evaluates a minimally invasive end-user testing model that allows users to interact with the app naturally as they enjoy the recreational space without being followed and observed. The proposed model combines in-app analytics and brief post-visit interviews with end users. This model was used for testing at a children’s museum using an app developed by local university undergraduates. These data sources were then evaluated to determine their capabilities for finding bugs in the app. Findings suggest that this model allows for authentic interactions from real users, leading to the identification of several issues that may otherwise go unnoticed with traditional testing methods. Interviews alone resulted in vague information; analytics alone did not identify all bugs in the software. The combination resulted in more bugs being identified and with enough information for developers to address them. This study offers an alternative to conventional end-user testing in contexts where user engagement with the software is secondary to a larger, in-person experience.
Natural Interaction, Real Feedback: A Qualitative Assessment of Minimally Invasive Model for End-User Testing in Recreational Environments
Category
Student Abstract Submission