User experience evaluation methods: current state and development needs

Authors : Arnold P. O. S. Vermeeren , Effie Lai-Chong Law , Virpi Roto , Marianna Obrist , Jettie Hoonhout , Kaisa Väänänen-Vainio-Mattila Authors Info & Claims

Pages 521 - 530 Published : 16 October 2010 Publication History 398 citation 12,346 Downloads Total Citations 398 Total Downloads 12,346 Last 12 Months 1,316 Last 6 weeks 82 Get Citation Alerts

New Citation Alert added!

This alert has been successfully added and will be sent to: You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below. Manage my Alerts

New Citation Alert!

Abstract

The recent shift of emphasis to user experience (UX) has rendered it a central focus of product design and evaluation. A multitude of methods for UX design and evaluation exist, but a clear overview of the current state of the available UX evaluation methods is missing. This is partly due to a lack of agreement on the essential characteristics of UX. In this paper, we present the results of our multi-year effort of collecting UX evaluation methods from academia and industry with different approaches such as literature review, workshops, Special Interest Groups sessions and an online survey. We have collected 96 methods and analyzed them, among other criteria, based on the product development phase and the studied period of experience. Our analysis reveals development needs for UX evaluation methods, such as early-stage methods, methods for social and collaborative UX evaluation, establishing practicability and scientific quality, and a deeper understanding of UX.

References

ENGAGE, Report on the evaluation of generative tools and methods for 'emotional design'. Deliverable D15.3. EU project Engage 520998 (2006).

Ganglbauer, E., Schrammel, J., Deutsch, S., and Tscheligi, M. Applying Psychophysiological Methods for Measuring User Experience: Possibilities, Challenges and Feasibility. Workshop on User Experience Evaluation Methods in Product Development. August 25, 2009. Uppsala, Sweden.

Gaver, B., Dunne, T., and Pacenti, E. Design: Cultural probes. Interactions 6, 1 (Jan. 1999), 21--29.

Green, W., Dunn, G., and Hoonhout, J. Developing the scale adoption framework for evaluation (SAFE). In: Proc. of the 5th COST294-MAUSE Open Workshop "Meaningful Measures: Valid Useful User Experience Measurement (VUUM)", Iceland June 2008. Also published in the ACM Library.

Hassenzahl, M., and Tractinsky, N., User Experience - a research agenda. In: Behavior & Information Technology, 25(2), (2006) pp. 91--97.

Hassenzahl, M. 2008. User experience (UX): towards an experiential perspective on product quality. In Proc. of the 20th international Conference of the Association Francophone D'interaction Homme-Machine. IHM '08, vol. 339. (2008) ACM, New York, NY, 11--15.

Hoonhout, H. C. M. Let the game tester do the talking: think aloud and interviewing to learn about the game experience, In: Isbister, K., Schaffer, N. (eds.), Game Usability: Advice from the Experts for Advancing the Player Experience, San Fransisco, CA: Morgan Kaufmann Publishers, (2008) 65--77.

HUMAINE D9j: Final report on WP9. 30 th January, 2008.

ISO DIS 9241-210:2010. Ergonomics of human system interaction - Part 210: Human-centred design for interactive systems (formerly known as 13407). International Standardization Organization (ISO). Switzerland.

Isomursu M, Tähti, M., Väinämö, S., and Kuutti, K. Experimental evaluation of five methods for collecting emotions in field settings with mobile applications. International Journal of Human-Computer Studies, Volume 65(4), April 2007, pp. 404--418.

Isomursu, M. (2008). User experience evaluation with experimental pilots. In: Väänänen-Vainio-Mattila, K.; Roto, V.; Hassenzahl, M. Now Let's Do It in Practice: User Experience Evaluation Methods in Product Development, Workshop at CHI2008 (2008).