Dropping the ball

Research is a juggling act. I think to be a successful researcher you must juggle at any given time at least four of the following: data collection, reading, writing, analysis, reflexivity -- always reflexivity and reflection.

Am I leaving anything out? Probably.

This whole semester I thought I was doing well, when, in fact, at most I was tossing around maybe two things at a time. Big deal.

After reading the coding and analysis literature this week, I realize I have dropped a major ball, the analysis one.

I also realize I have some new questions. (See end of post for questions).

Coding, analysis, and interpretation
Coding is a commonly accepted approach to analysis, and, as such, it has its share of critics, who claim it is an over-glorified form of reductionist frequency counting.

In the opening chapters of the new edition of his coding manual, Saldaña (2013) addresses the critics. He says coding is not the end-all, be-all of analysis. Nor is it a discrete stage of analysis done in isolation from other aspects of research.

Saldaña argues that coding does not distance the researcher from his or her data:

If you are doing your job right as a qualitative researcher, nothing could be further from the truth. Coding well requires that you reflect deeply on the meanings of each and every datum. Coding well requires that you read and reread and reread yet again as you code, recode, and recode yet again. Coding well leads to total immersion In your data corpus with the outcome being exponential and intimate familiarity with its detail, subtleties, and nuances. (p. 39)

I so need to do this. I haven't been handling the data well at all. I have been amassing it, stockpiling it, peering at it occasionally with heavy heart as I do with hampers of dirty laundry. In fact, I now have piles of neatly folded, laundered clothes, and no grasp of my study. I will choose laundry any day over the challenge of data analysis.

The readings on coding and analysis cause me to feel a pit in my stomach as I realize how far removed I have become from the actual texts of my study. I haven't transcribed in weeks. I haven't touched, much less reflected on, field notes, emails, and other artifacts from last semester. I acquired a thick folder of documentation from one of the key informants for my study, and the documents still sit where I left them two weeks ago.

Worse, I have been continually conducting interviews with participants for the last three months, without the benefit of immersion in the data to guide or shape my interactions with those participants.

In their discussion on the process of line-by-line microanalysis, Strauss and Corbin (1998) write, "We are moved through microanalysis by asking questions, lots of them, some general but others more specific. Some of these questions may be descriptive, helping us ask better interview questions during the subsequent interviews" (p. 66).

I should be forming theoretical questions that probe relationships between concepts and then asking these question during follow-up interviews.

I just concluded the first round of interviews. Time is running out for follow-ups, as my participants are classroom teachers who will not relish being interviewed over summer vacation. This is a potential problem.

Analytic memos
And because I have not been immersed in the data, I also have not generated a single analytic memo about the data, not since last semester. This is another problem.

Analytic memos are the raw materials for what will ultimately become the “theoretic text,” in which the researcher “finally sees the theoretic interpretation – core thesis – he or she wants to put forward” (Piantanida & Garman, 2009, Ch. 13, para 1). They are initiated by “aha moments” and “conceptual leaps” that put “the myriad individual, idiosyncratic, and situational details into a meaningful, coherent, theoretic perspective” (para 2).

According to Saldaña, "Virtually every qualitative research methodologist agrees: whenever anything related to and significant about the coding or analysis of the data comes to mind, stop whatever you are doing and write a memo about it immediately" (p. 42).

Richardson (1994) refers to analytic memos as "theoretical notes." These are the researcher's hunches, hypotheses, connections, and/or critiques about what is being seen and heard in the field. The researcher opens up his or her texts to interpretation and a "critical epistemological stance" (p. 526).

Memo writing goes hand-in-hand with analysis. All sorts of memos may be generated during research, but the analytic ones mark the researcher's first attempts at creating findings.

In writing workshop, instructors guide their students: "Don't get it right, get it writ." This advice applies to the research memo as well. Saldaña says just write the memo; worry about the title and category later. He explains,

I simply write what is going through my mind, then determine what type of memo I have written to title it and thus later determine its place in the data corpus. Yes, memos are data; and as such they, too, can be coded, categorized, and searched with CAQDAS programs. Dating each memo helps keep track of the evolution of your study. Giving each memo a descriptive title and evocative subtitle enables you to classify it and later retrieve it through a CAQDAS search. (p. 42)

Grounded Theory and ATLAS.ti
One thing that has become more transparent to me in the coding and analysis readings is the connection between grounded theory and ATLAS.ti.

Some researchers distrust CAQDAS tools because “enduring foundationalist epistemologies are clearly being drawn on in their design and programming” (Brown, 2002). After reading portions of Strauss and Corbin’s (1998) handbook on grounded theory, I finally see what the critics are talking about. Key features of ATLAS seem to be borrowed directly from the grounded theory genre: open codes, in vivo codes, network views (Strauss and Corbin call them “diagrams”), and the integration of memos.

Strauss and Corbin did not claim to know much about computers in their 1998 volume, but they specifically mention ATLAS.ti as “more geared toward theory building” (p. 276) and reproduce a memo from ATLAS developer Heiner Legeiwe that says as much.

Questions:

• Both the grounded theory guidelines as well as Saldaña’s book refer to the use of categories. How does one denote categories in ATLAS.ti? I have been using prefixes to organize codes upfront, so my prefixes are definitely not categorical. They are simply organizational/topical in nature. Is this what code families are for? Does it matter?
• Saldaña refers to “subcodes” and “subcategories.” How can these be represented in ATLAS.ti?

References
Brown, D. (2002). Going digital and staying qualitative: Some alternative strategies for digitizing the qualitative research process. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 3(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/viewArticle/851

Piantanida, M., & Garman, N. B. (2009). The qualitative dissertation: A guide for students and faculty (2nd ed., Kindle version.). Thousand Oaks, CA: Corwin.

Richardson, L. (1994). Writing: A method of inquiry. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 516–529). Thousand Oaks, CA: SAGE Publications, Inc.

Strauss, A., & Corbin, J. M. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. Thousand Oaks, CA: SAGE Publications.

Saldaña, J. (2013). The coding manual for qualitative researchers (2nd ed.). Los Angeles: SAGE Publications Ltd.

Dropping the ball

Reflecting on Watt (2007)

Watt's 2007 piece, "On Becoming a Qualitative Researcher," is a first-person narrative of a novice researcher who "put reflexivity to the test by keeping a research journal" (pp. 82-83).

Unlike positivist studies, which are assessed by familiar standards of validity and reliability, the rigor of qualitative inquiry is determined more subjectively, beginning with the researcher herself through a process of reflexivity. In the absence of precise designs and formulas to guide their practice, qualitative researchers, Watt explains, must be willing to self-scrutinize. 

Reflexivity, as defined by Glesne (2006), "involves critical reflection on how researcher, research participants, setting, and phenomenon of interest interact and influence each other" (p. 6). Reflexivity statements "provide a kind of map to the decisions you make so that the reader can better understand (and question) the interpretations you make" (p. 127).

As a student, Watt understood reflexivity on a theoretical level but wasn't sure what it looked like in practice.

This is the third time in as many years that I have read Watt's article, and with each new reading, I gain new insights. Not surprisingly, this time around, my insights were colored by the fact that I am now immersed in my own dissertation project.

What did surprise me was just how intimately connected the practice of reflexivity is to improving trustworthiness in research, a topic I will try to address at the end of this post.

A refined sense of the reflexive process
First, putting aside textbook definitions, my understanding of reflexivity has grown more refined, especially as it relates to the act of writing and developing identity.  I was really struck by the influence of a (somewhat) regular writing regime on Watt's own personal development as much as her research process. This is the kind of exploratory and personal journal writing that Laurel Richardson advocates in "Writing: A Method of Inquiry," which Watt refers to several times.

In her research journal, Watt summarized her readings from the literature, reflected on events in the field, recorded participant data, and made notes about her methodology. "Through using writing as a method of inquiry I was able to make links between how I carried out my study, reflective journal entries, and the literature on qualitative methodology" (p. 98).  In other words, she developed the deliberative stance as advocated by Piantanida and Garman (2009) and came to recognize the "centrality of writing as a way of coming to know" (Ch. 9, “Experiential Text as a Content for Theorizing,” para 12).

More importantly, through writing, the deliberative stance becomes internalized: "Reflective writing allowed me to meaningfully construct my own sense of what it means to become a qualitative researcher" (Watt, p. 83).

On becoming an expert
Writing is one way to negotiate moments of conflict and disequilibrium. Like discussion and other forms of peer mediation, reflective writing creates a feedback loop -- a conversation with oneself -- for improving practice and developing expertise.

Yet, as Watt points out, it is during moments of crisis and tension that novice researchers typically slow down or shut down their writing entirely.

For instance, when faced with the task of data analysis, Watt was at a "complete loss" (p. 95), and so she made lots of charts and diagrams. She explains, "I was so focused on the need to do something with the data that I did not consider journaling as a means to think things through, on both a personal and a research level. That was a mistake. In retrospect, this was perhaps the time I needed it most" (p. 96).

The mistake is common in my field, education. Like novice qualitative researchers, new teachers experience a profound sense of disequilibrium. Overwhelmed and isolated, they put reflection on the back burner.

But reflexivity is the antidote to repeating mistakes, and building upon a foundation of subjective experience is a hallmark of expertise. Watt cites Eisner's (1991) concept of educational "connoisseurship," in which seasoned practitioners leverage their subjectivities in productive ways. Piantanida and Garman also cite Eisner's work in their Chapter 5 discussion of deliberation as a means of attaining phronesis, "a valuing of wisdom that can guide action within the complexities of unfolding experience."

Whether it's "critical subjectivity," "connoiseurship," or "phronesis," these ideas are very useful to me because they are nuanced versions of what I would simply call "developing expertise," and teachers developing expertise about technology is the topic of my dissertation. For purposes of my study, I have found the CHAT concept known as "expansive learning" (Engestrom) to be helpful, but I like how Eisner's term goes directly at the teacher-learner, teacher-researcher experience. I may need to borrow or develop a more precise term other than "expertise" (something else to reflect upon later).

At any rate, I have noted before in this blog the parallel developmental trajectories of novice qualitative researchers and teachers-as-learners insofar as stance, disposition, and expertise are concerned. And after re-reading Watt, I have a renewed sense of how reflexivity mediates this process.

The Multiple Realities perspective
I have also referred previously in this blog to the Multiple Realities perspective, a central notion in my theoretical frameworks. I was excited to make a connection between Watt's article and Labbo and Reinking's (1999) seminal theoretical piece, "Negotiating the Multiple Realities of Technology in Literacy Research and Instruction."

Although Labbo and Reinking do not specifically use the term "reflexivity," they put forth the Multiple Realities perspective as a way for New Literacies researchers to monitor and leverage subjectivities in a way that strengthens research-to-practice connections:

For example, when a question related to instructional  practice begins with the phrase "What  does the research say about...?" we  believe it should be followed by an explicit consideration of which reality or set of realities is being considered. Doing so means that the answer will inherently be more complex than citing a string of studies and drawing conclusions from them. It also suggests that it may be important to identify the realities to which a potential answer does not apply or why a question is not a relevant or particularly good one within certain realities. (p. 488)

The Multiple Realities perspective, then, is a framework for guiding reflexive thinking within New Literacies research projects. I had never thought about it in this way until now! Maybe I need to be more intentional about referencing it in the methods section of my Chapter 3.

Implications for use of digital tools
Finally, a note about digital technologies and the role they serve in supporting reflexive practice: Watt's descriptions of her struggles to corral and make sense of the data, the codes, and the categories highlighted the value (for me) of digital tools generally, and CAQDAS in particular. This last insight will only become more compelling with the passage of time (Watt's article is now six years old) and my own increasing awareness of how to use digital tools for qualitative inquiry.

Trustworthiness
The common thread that connects each of the above ideas is how they promote trustworthiness in qualitative research. The discipline of regular, reflexive writing -- "chronicling one's thinking" -- helps the researcher mediate the meaning of her experiences and continually develop her expertise.

As Watt explains, "Revisiting my study has strengthened my confidence in my ability to negotiate the  complex process of qualitative inquiry, and I now see myself as a researcher. The multiple layers of reflection drawn upon in writing and revising this paper have made me more cognizant of how far I have come, and have taken me further along the path to becoming a qualitative researcher" (p. 98).

Reflexivity develops an internal authority of self-knowledge (identity) and self-efficacy, alternate "measures," if you will, of excellence and trustworthiness in research.  Reflexivity develops an authentically authoritative voice that delivers the research findings and interpretations in a way that resonates with readers.

References

Glesne, C. (2005). Becoming qualitative researchers: An introduction (3rd ed.). Boston: Allyn & Bacon.

 Labbo, L. D., & Reinking, D. (1999). Negotiating the multiple realities of technology in literacy research and instruction. Reading Research Quarterly, 34(4), 478–492. doi:10.1598/RRQ.34.4.5

Piantanida, M., & Garman, N. B. (2009). The qualitative dissertation: A guide for students and faculty (2nd ed., Kindle version.). Thousand Oaks, CA: Corwin.

Watt, D. (2007). On becoming a qualitative researcher: The value of reflexivity. The Qualitative Report, 12(1), 82–101.

Reflecting on Watt (2007)

A question about reflexive statement and assumptions

Last week I presented my Chapter 3 to my workshop group. I wanted feedback on growing the logic-of-justification for case study method within a Cultural-Historical Activity Theory (CHAT) approach. To do this  part well, I know I must first clear the CHAT hurdle: does the reader understand what it is and how my use of case study relates to it?

As an evolving theoretical frameworks, CHAT is not widely used in education studies, even less so in the field of literacy studies. Roth and Lee (2007) call it a "best kept secret."

So, the biggest item of feedback I received during workshop last week had nothing to do with the incorporation of case study theory into my research design. It had to do with my explication of the underlying assumptions of my study, namely CHAT. The group wanted to know more about the assumptions of CHAT, which I only allude to in Chapter 3. It is within Chapter 1 that I discuss -- by way of a reflexivity statement -- the underlying assumptions of CHAT, as well as other elements of my frameworks (New Literacies theory and the multiple realities perspective).

My statement of reflexivity, as it currently appears in Chapter 1, outlines who I am as a researcher (background, interests, professional experience, and worldview) and how these aspects of my identity influence my choice of topic, question, frameworks, and methodological approach. Then, I get into the nitty-gritty of the underlying assumptions of my approach. But since these are shared assumptions, assumptions belonging to an entire research tradition/genre, should they not be in the methods section of the study?

My question: does it make sense to split my reflexive section? In other words, should I keep the personal statement of reflexivity where it is, but save the built-in assumptions of my research genre for Chapter 3? 

I would like to submit my Chapter 3 revisions as party of my mid-term project report for EP 659, so feedback on this point will be greatly appreciated.

Additional project update: I read most of Susanne Friese's (2012) book about ATLAS.ti last week, and posted my reading notes.

References:

Roth, W-M., and Lee, J-L. (2007). “‘Vygotsky’s Neglected Legacy’: Cultural-Historical Activity Theory.” Review of Educational Research 77(2), 186–232. doi:10.3102/0034654306298273.

A question about reflexive statement and assumptions

Building your head

Section Two of Piantanida and Garman’s The Qualitative Dissertation (2009, Kindle edition) describes the conceptual work one must perform prior to conducting a qualitative research project.  A considerable amount of mental heavy-lifting precedes the more technical aspects of “doing” research.

If I actually knew anything about weight training and conditioning (or physical exercise in general, for that matter), I could get really carried away right now with metaphor. Suffice it to say, I felt pretty fit while reading some portions of Section Two and woefully out-of-shape while reading others.

I was reminded of the phrase “building your head” from H. L. Goodall’s (2000) Writing the New Ethnography (p. 51). Scholarship is a process of locating yourself in the “storyline” of your discipline.  It means reading widely and writing daily.  Piantanida and Garman would call it developing oneself as the “instrument of inquiry.” Whatever you call it, it’s a heavy workout.

Qualitative researchers must do this conceptual work if they want their research to be taken seriously and to resonate with their desired audience. As Piantanida and Garman explain, qualitative researchers approach topics and issues of inquiry with different assumptions and knowledge claims that don’t conform to prevailing positivist and postpositivist research paradigms. Our work is cut out for us.

Image by MR. LIGHTMAN, courtesy of freedigitalphotos.net 

The fundamental challenge of qualitative inquiry is answering the question: What is legitimate knowledge? According to Piantanida and Garman, “This question pushed qualitative researchers to articulate more clearly the nature of knowledge claims they were making” (Ch. 5, “Considerations of worldview,” para 5).

I wish I had known about this book last semester. The Reflective Interludes, particularly 5.1 and 7.1, provide terrific prompts for drafting the “Assumptions” section of a dissertation proposal. At any rate, I have identified some goals to work on – dare I say, “workout goals”? – for the coming semester:

Resist the “residue” of positivism and postpositivism

Piantanida and Garman refer to the “default language” of positivism and the fact that for so long quantitative research served as “a shorthand descriptor for a world-view embedded in the Western European ‘Enlightenment’” (Ch. 5, “Considerations of worldview,” para 4). Amen. I was 37 years old and at the tail-end of my master’s coursework before I even knew what qualitative research was! All in a tizzy, I did not think I could complete a master’s thesis without at least collecting some survey data and learning SAS. At that point, my adviser sat me down and explained that there were alternative methods that did not require an experimental design and a large n.

Over the last several semesters I have made concerted effort to curtail my use of words and phrases that suggest postpositivist assumptions. Two that come to mind are “achievement” (erroneously used as a synonym for “learning”) and “best practices” (falsely generalizing about instructional practice without respect for context and learner).

Yet, even as we come around to our own enlightenment, the language of the dominant paradigm is hard to resist. It just creeps up on you. Now, at the tail-end of my doctoral studies, I have had committee members scrutinize my word choice, calling into question seemingly benign words such as "factors" and "confirmed."

And, now, per Piantanida and Garman, I may need to more carefully consider use of the word “data.” That will be difficult!

I really do appreciate the distinction drawn in Chapter 8 between “data” and “text.” It jibes with my professional roots as an English/Language Arts teacher, trained in reader-response theory. Piantanida and Garman anticipate readers’ objections that the distinction borders on playing with semantics. They argue that representations of social phenomena, as opposed to representations of the physical world, “are always and inevitably interpretations offered by a particular person with a particular perspective and a particular position” (“From Data to Text,” para 4).

This is the essence of Louise Rosenblatt’s transactional theory of reading and writing, which views reader and text as inextricably bound in dynamic, mutually conditioning relationship. I cut my teeth on this stuff as a young preservice teacher! What’s not to love? I just wonder how forcing the distinction between “data” and “text” (much less forms of text – raw, experiential, discursive, and so on) might stall collaborative dialogue within one’s research community. It would certainly raise eyebrows among some dissertation committee members. 

Nonetheless, I am intrigued and would like to explore implications for more conscientious use of these terms in my own research journey.

Avoid simplistic dualisms

I understand that for the immediate task at hand – the dissertation – it is important for me to build a “conceptual frame” (Ch. 5, “Considerations of worldview,” para  17). As Piantanida and Garman recommend, “…[F]or purposes of conceptualizing a study, it is useful to sort out what form of knowledge claims one hopes to make by the end of the research process” (Ch. 5, “Considerations of worldview,” para 16).

This poses an interesting balancing act between being true to one’s core values, on the one hand, and acting pragmatically within the immediate research context, on the other.

What I struggle with is the possibility of being boxed in or branded by my conceptual frame, which, at present, is defined by a sociocultural perspective and a desire to seek insight and meaning rather than definitive proofs. I wonder if I limit myself by not seeking a more solid grounding in quantitative methods. Some of my colleagues are pursuing both research certificates, and, whether they are doing it for future job security in academe or because they see both paradigms as potentially valuable to their research agendas, I admire their hard work and ambition.

Piantanida and Garman seem somewhat dismissive of this pragmatic stance, but I like it, especially in the way it opens up potential for collaboration between camps. In my own readings on professional development and digital technologies, I see ways in which both kinds of research studies and knowledge claims can serve each other.

Maintain stance of "self as instrument"

I must see myself as the “instrument of inquiry,” all else is ancillary. This is the most crucial "workout routine" of all, and it is an especially difficult exercise for the teacher-researcher, coming from the culture of disempowerment that largely defines K-12 education. In the field of literacy especially, where decentralized decision-making, canned curricula, and scripted instruction prevail, thinking, planning, and acting instrumentally can be unfamiliar territory.

In the Chapter 6 section on voice and stance, I was struck by how the different researcher stances paralleled certain teacher dispositions. The researcher’s worldview affects her subject-object orientation with regard to participants. Similarly, a teacher’s worldview dictates how she regards her students. Are they subjects in or objects of her instructional design? The third stance described in this section, in which the researcher risks challenging her own certitude and self-understanding, is, in fact, what teacher development expert Gerald Duffy (1998) has called the “thoughtful adaptation” of effective teachers (p. 778).

This process of “thinking through one’s relationship with that which is under study” (Ch. 6, “Stance and voice,” para 1) reinforces Piantanida and Garman’s earlier assertion that the dissertation is an orientation to a life of scholarly inquiry. It is as much an opportunity for personal development as it is an obstacle to surmount in the quest for professional advancement. It is the “thinking through” that stimulates “a reforging of one’s ontological stance,” leading to new understanding (“phronesis”). This is the ultimate goal of qualitative research: to share these new understandings in hopes that others may recognize "new possibilities for their way of being in practice.” 

Does that not fulfill the “legitimate knowledge” requirement?

References

Duffy, G. G. (1998). Teaching and the balancing of round stones. Phi Delta Kappan, 79(10), 777–80.

Piantanida, M., & Garman, N. B. (2009). The qualitative dissertation: A guide for students and faculty (Second Edition, Kindle edition.). Thousand Oaks, CA: Corwin.

Building your head