“Current quantitative methods ask designers to rely on 'average' opinions”, says Karapanos “Designers have every right to distrust statistics. What is the value of questionnaires where users are asked to rate products on qualities users don't recognize or think important themselves?” 

Therefore observing users over a longer period of time and asking them to tell about their experiences in their own words, can be an inspiring source of information for designers. But it's very labor intensive and time consuming to do so. And how to handle the vast load of information you get in this way? One of the things Karapanos addressed in his thesis was a method to provoke detailed user experiences in retrospective. 

He used a survey tool iScale in which people did sketch how positive or negative they think of their mobile phone over time. At the same time they could link experiences in using the product to this time line. He found that people using iScale reported more experiences compared to free recall of experiences. They could also give more contextual information, and they were more accurate in recalling concrete details of the events. “In this way we can collect more information of people using a product in different contexts”, says Karapanos. 

Karapanos also looked at ways to analyze these large amounts of user experiences in a meaningful way. For example by visualizing the similarity between narratives. “I used this idea from information retrieval”, Karapanos explained. If you have a number of documents you can compute the similarity by counting the occurrence of the same words in those different documents. He combined this with encoding the narratives, for example by labeling different words which have a similar function in the narrative. For example, 'friend', 'brother' and 'colleague' are all encoded as 'relevant other'. 

“In this way you create an easy way to navigate through the extended data. Besides this similarity analysis can help in interpreting the data in different ways”, explains Karapanos. In one study he let people describe three experiences with an iPhone every day for four weeks. The narratives were encoded by terms like 'learnability', 'usefulness', 'stimulation' and 'long-term usability' and by features of the product, like 'audio' or 'camera'. “The data show that the experiences and their impact change over time. At first Learnability and ease of use are important factors. But what makes the product cool after a few weeks? And what will stimulate prolonged use of a product like iPhone? “This depends on how the product becomes embedded in one's personal and social life”, Karapanos explains. One example is how products become part of daily rituals, or part of our self-identity. Karapanos argues that “traditionally in our field we have been treating interactive products such as mobile phones as tools that serve predefined purposes. But it’s not really like that! People live with products, products are part of their lives. We need methods for capturing such rich information in a meaningful way for designers.” 
 
The research was carried out under the auspices of the J.F. Schouten Graduate School of User-System Interaction Research, and sponsored by the Dutch Ministry of Economic Affairs within the IOP-IPCR program,through the ’Soft Reliability’ project, under the auspices of Philips and Océ. Ir. Evangelos Karapanos did defend his dissertation `Quantifying Diversity in User Experience' on Tuesday 23 March, at 4 P.M.