

7), with their common scientific use deeply rooted in positivist tradition. “Reliability and validity are tools of an essentially positivist epistemology” (Watling, as cited in Winter, 2000, p. This could be something that many researchers, analysts or product manager who usually live in the quantitative world feel difficult to fathom. For example, in that study paper the authors mentioned reliability and validity of their qualitative analysis in statements like “the inter-coder reliability was 0.85” and “to ensure the validity”, but I found this extremely confusing, at least from its current text without further clarifications. The second thing is about reliability and validity in qualitative analysis.

That is something that “forcing codes” could hardly bring, I am afraid.


Braun and Clarke (2006) argued that when researchers go through the data analysis process, it is critical to “involve a constant moving back and forward between the entire data set, the coded extracts of data that you are analysing, and the analysis of the data that you are producing”. While I agree that Braun and Clarke (2006) are sort of critical to the idea of “emergence” because of its implied passiveness that they believed, researchers always should play active roles in the creation of themes by staying sensitive, reflective, and open. Next, the coders used the agreed-upon code book to code the interview data.” Though this might seem to be a working approach to use thematic analysis (a typical qualitative analysis method) in their specific study, the authors may have ignored a severe problem that could emerge in their coding process, i.e., how could they possibly ensure all codes that they are going to develop on a new upcoming interview document will definitely fall to these 135 previously-defined code set? What’s more, this seems to be a typical “forcing” approach in qualitative data analysis. The paper mentioned that “… two coders convened, discussed, and converged their codes into a code book of 135 unique codes …. There are mainly two things that I would like to reflect upon. Nevertheless, I noticed several methodological issues unexplained in their paper, yet I think this may be an opportunity to be a little bit verbose and loquacious to talk about my reflections of qualitative analysis, which is increasingly important in UX research and product analysis across various technology industries. Recently I read an interesting CHI paper which was a qualitative study and was related to our research project. Coding in A Qualitative Analysis In a Real Research Project (blurred the picture on purpose)
