Metrics

[Tullis2008] Tom Tullis and Bill Albert, Measuring the user experience: collecting, analyzing, and presenting usability metrics. Amsterdam ; Boston: Elsevier/Morgan Kaufmann, 2008, ISBN: 978-0-12-373558-4.

Transcript

So, if you think of these different types of data, and we see them over here in the first column. We can think about metrics to use for them. So, for example, for nominal data that we can think of metrics like success and failure.  And the common statistics we will apply are things like frequency analysis and chi-squared tests.  Or for ordinal data, we can think about ranking.  Not surprisingly, we can use again frequencies and chi-square, but we can also think in terms of Wilcoxian rank-sum tests or Spearman rank correlations.  For interval data, for example, it's quite often the case that we have a survey, we might have five different levels: I completely disagree with it, I disagree, I don't really care - either way, I really agree, I agree somewhat, and I really agree with it. These five levels might be a five-level Likert scale.  We might use system usability scales, MOS scores, etc.  And for these types of data, we can use descriptive statistics, things like averages, medians, [and] standard deviations. We can apply tests to them like Student's t-test or ANOVA or correlation or regression type analysis. And for ratio type of data, things like pass completion time or packet interarrival time, we can apply all of the above, but now we can often apply to it the geometric mean.  And, if we look at Tullis [and Albert] book - you can find lots more examples about this.