Qualitative research has a bad reputation -and, I’m sad to say, for a reason. Here is a collection of some problems and pitfalls I’ve noticed in qualitative research theses and papers and suggestions on how to avoid them.
Why Qualitative Research?
What is qualitative research? It is not a lacking or poorly done quantitative research. Nor is it a preliminary, “let’s-give-it-a-go-and-get-it-done” approach that let’s you get off the hook of writing a thesis in the easiest possible way. Qualitative research is research done on verbal data, such as interviews, diary entries, conversations, or newspaper articles.
Let’s start from asking why do you want to do qualitative research. If you choose it to get an easy way out with your thesis, you are making the choice for the wrong reasons. Analyzing verbal data is not necessarily any easier than analyzing numerical data, at least if properly done. Hence, don’t choose to do interviews or analyze other kind of verbal data just because you don’t want to do statistics.
Firstly, do some reading. Read up on how to collect and analyze qualitative data. Read some good qualitative papers published in reputable journals of your field. Make notes on how they present doing the data collection, analysis, and conclusions. Simply put: Get rid of the mindset that you can just chit chat with a bunch of people and write up what you think was most interesting in what they said, and call that “qualitative research interviews”.
Once you are doing your research, be ready to explain to yourself and the readers of your thesis/paper why you choose to do qualitative research. What drives you to choose this approach to data collection and analysis? Why is this the best way to answer your research questions?
Check out this video by Dr. Leslie Curry from Yale University:
Data Collection and Preparation
Next, define your data collection method properly. If you are conducting an interview, put time and attention into developing your set of questions. Explain in your thesis the development of your interview protocol and share it as an appendix. You save time and your nerves (and the nerves of your supervisor) by planning ahead how you collect your data and by creating a good instrument for doing it, whether it is an open-ended questionnaire, a reflective diary format, or something else.
Surprisingly often, theses and papers include no note about the degree of detail in transcribing the data. Make sure to at least state if the transcription was done verbatim or not. Transcribing your data (making e.g. interview tapes or videorecorded conversations into written form) is often a big part of your work. It is also the first chance for you to really dig deeper into your data and get to know it well. I recommend you put time and attention into conducting the transcribing as well as possible.
For tips on transcribing, check out QualPage
Digging into the Data
Often, people don’t explain at all how they conducted the data analysis itself. They might explicate carefully the development of their interview protocol, but then jump into presenting the categorization of the responses without showing how and why those categorizations were created.
Qualitative research is not about setting up pre-defined categories and then smashing your data into them. Qualitative data analysis is about reading and re-reading. It is an evolving process: as you re-read the data, your way of categorizing and systematizing it develops. You have to stay self-aware of why and how you place attention onto your material. Also, make sure you truly investigate the textual data and that you do it with an open mind -you are not searching for proof that your previous ideas are correct. Keep your attention on the text and not on what you think it says. Be explicit in how you read your data, where you put attention, and how your reading develops during the process. To make sure you can be transparent about these things in your thesis or paper, make notes of your progress to keep track on the various phases of analysis. Keep your mind open. In poor theses and papers, it is often evident that the researcher/student is using the data to prove what they have pre-decided is the truth.
Make it clear to yourself what the process of data collection, analysis, and presenting the results is. Like I said the previous paragraph, in qualitative research, you often surf back and forth between these different stages. You go back to the data after you already have your preliminary results, such as a categorization system, and re-read the data to modify your system. However, when you write up your results, be sure you don’t mix up things and write something like you “coded the results”. Results emerge from coding, not the other way round.
Moreover, saying something like you “coded” the data is not the answer to a good analysis. Coding is not an analysis method per se. How, what, and why did you code? How did you define the way in which you coded the data? It is incredibly common to see people use quantitative words and phrases in their qualitative work to create the illusion of rigorous analysis. Just because you insert the word “factor” in a qualitative paper doesn’t magically make it good, often just the opposite. If you end up using quantitative terminology, make sure you know what you are doing.
People often think that describing your data is good enough for results in qualitative research. It’s not. Try and dig deeper. Think abstract. You have to show how you systematized and classified the data in one way or another, how you picked up more abstract patterns from it. Just telling your reader what your interviewees said is not a results section. Copy-pasting your field notes is also not a results section. For what a good results section looks like, find a few much-cited papers in good journals of your discipline and see how they put it up.
A common phenomenon is that the researcher, in wanting to prove they are data-driven, says that something “emerged” from the data. Data is data. The results don’t just “emerge” from there like a Loch Ness monster from the lake. You as the researcher make them emerge by placing attention onto certain features of the text and disregarding others.
Furthermore, if you conducted e.g. interviews, show quotes to make it transparent why and how you ended up to certain conclusions. Don’t overdo this, though. You cite your interviewees to show why you classified the data in a certain way, not because the quotations themselves are the “results”.
As I pointed out previously, sometimes people use concepts from quantitative research and transport them into a qualitative context, where they end up making very little sense. If you decide to use concepts such as “factor” or “variable”, make sure the way you define and use them is actually understandable in the context of qualitative thesis or paper. However, chances are that “factor” and “variable” make zero sense in a qualitative work. Your data is not made of numbers; it is made up of words, sentences, utterances, metaphors, arguments, and the like. Conceptualize it accordingly.
The Big Monsters of Reliability and Validity
The questions of reliability and validity can be a challenge to tackle in a qualitative thesis. The two all too common approaches are: 1. stating that in a qualitative study, they are a challenge, but not explaining why this is so and what you have done to make sure your reliability and validity (or, rather, trustworthiness/rigor and consistency) are as good as possible and 2. taking a quantitative research framework and imposing it onto a qualitative setting. Reliability and validity in a qualitative research are simply not the same thing as they are when you are operating with quantitative data. Also, if you use concepts such as “external validity” or “internal validity”, again, make sure you are not just transporting meanings from a quantitative framework and forcing them into a qualitative one, because chances are you are doing exactly that.
I have read in a thesis a statement that the reliability of the interviews means that you get the same responses every time you conduct the interview protocol with new interviewees. This is so clearly wrong that if you don’t yet see what is wrong with it, you are most likely in need of a heavier crash course in qualitative study than this post.
For reliability and validity, check out for example these resources:
Golafshani, N. (2003). Understanding Reliability and Validity in Qualitative Research. The Qualitative Report, 8(4), 597-607.
Cypress, B. S. (2017). Rigor or Reliability and Validity in Qualitative Research: Perspectives, Strategies, Reconceptualization, and Recommendations. Dimensions of Critical Care Nursing, 36(4), 253-263. doi: 10.1097/DCC.0000000000000253
Simon, M. &Goes, J. Reliability and Validity in Qualitative Studies.
The Researcher’s Influence
In qualitative research, it is very important that the researcher acknowledges their personal influence in the data collection and analysis. This is the cornerstone of reliability and validity in qualitative research. Do not treat your influence as merely a disturbance or noise. In qualitative research, the researcher is the instrument, in good and bad. Make it as good as possible while acknowledging the bad.
It is not enough to say that in e.g. conducting interviews, you had an influence in the setting. What kind of an influence? What kind of glasses were you wearing when analyzing the data? Your age, sex, nationality, and educational as well as socio-economical background are just some of the many of variables that influence how you approach and analyze your topic. Be aware and transparent.
Keep studying more. Read up on scientific thinking process in general and educate yourself on the differences between quantitative and qualitative research. Keep your concepts clear.
While reading, make sure that the resources you use in your work and put in your reference list are of good quality. If I look at your reference section and find out that a conference presentation from the 1980´s is your only guide in analyzing qualitative data, I’m not going to be very happy. Not that you should care of my happiness. Find better sources for your own good.
Once more: Make sure you do qualitative research because you want to answer research questions that cannot be answered with e.g. experiments, numerical data, and statistical analyses. Have fun, enjoy the process, and make a thesis/paper you can truly be proud of!
Some other resources about interviews and qualitative research in general:
Kent Löfgren about analyzing interview data on YouTube: https://www.youtube.com/watch?v=DRL4PF2u9XA
Sarah J. Tracy and Margaret M. Hinrichs: Big Tent Criteria for Qualitative Quality