Reflecting on Watt (2007)

Watt's 2007 piece, "On Becoming a Qualitative Researcher," is a first-person narrative of a novice researcher who "put reflexivity to the test by keeping a research journal" (pp. 82-83).

Unlike positivist studies, which are assessed by familiar standards of validity and reliability, the rigor of qualitative inquiry is determined more subjectively, beginning with the researcher herself through a process of reflexivity. In the absence of precise designs and formulas to guide their practice, qualitative researchers, Watt explains, must be willing to self-scrutinize. 

Reflexivity, as defined by Glesne (2006), "involves critical reflection on how researcher, research participants, setting, and phenomenon of interest interact and influence each other" (p. 6). Reflexivity statements "provide a kind of map to the decisions you make so that the reader can better understand (and question) the interpretations you make" (p. 127).

As a student, Watt understood reflexivity on a theoretical level but wasn't sure what it looked like in practice.

This is the third time in as many years that I have read Watt's article, and with each new reading, I gain new insights. Not surprisingly, this time around, my insights were colored by the fact that I am now immersed in my own dissertation project.

What did surprise me was just how intimately connected the practice of reflexivity is to improving trustworthiness in research, a topic I will try to address at the end of this post.

A refined sense of the reflexive process
First, putting aside textbook definitions, my understanding of reflexivity has grown more refined, especially as it relates to the act of writing and developing identity.  I was really struck by the influence of a (somewhat) regular writing regime on Watt's own personal development as much as her research process. This is the kind of exploratory and personal journal writing that Laurel Richardson advocates in "Writing: A Method of Inquiry," which Watt refers to several times.

In her research journal, Watt summarized her readings from the literature, reflected on events in the field, recorded participant data, and made notes about her methodology. "Through using writing as a method of inquiry I was able to make links between how I carried out my study, reflective journal entries, and the literature on qualitative methodology" (p. 98).  In other words, she developed the deliberative stance as advocated by Piantanida and Garman (2009) and came to recognize the "centrality of writing as a way of coming to know" (Ch. 9, “Experiential Text as a Content for Theorizing,” para 12).

More importantly, through writing, the deliberative stance becomes internalized: "Reflective writing allowed me to meaningfully construct my own sense of what it means to become a qualitative researcher" (Watt, p. 83).

On becoming an expert
Writing is one way to negotiate moments of conflict and disequilibrium. Like discussion and other forms of peer mediation, reflective writing creates a feedback loop -- a conversation with oneself -- for improving practice and developing expertise.

Yet, as Watt points out, it is during moments of crisis and tension that novice researchers typically slow down or shut down their writing entirely.

For instance, when faced with the task of data analysis, Watt was at a "complete loss" (p. 95), and so she made lots of charts and diagrams. She explains, "I was so focused on the need to do something with the data that I did not consider journaling as a means to think things through, on both a personal and a research level. That was a mistake. In retrospect, this was perhaps the time I needed it most" (p. 96).

The mistake is common in my field, education. Like novice qualitative researchers, new teachers experience a profound sense of disequilibrium. Overwhelmed and isolated, they put reflection on the back burner.

But reflexivity is the antidote to repeating mistakes, and building upon a foundation of subjective experience is a hallmark of expertise. Watt cites Eisner's (1991) concept of educational "connoisseurship," in which seasoned practitioners leverage their subjectivities in productive ways. Piantanida and Garman also cite Eisner's work in their Chapter 5 discussion of deliberation as a means of attaining phronesis, "a valuing of wisdom that can guide action within the complexities of unfolding experience."

Whether it's "critical subjectivity," "connoiseurship," or "phronesis," these ideas are very useful to me because they are nuanced versions of what I would simply call "developing expertise," and teachers developing expertise about technology is the topic of my dissertation. For purposes of my study, I have found the CHAT concept known as "expansive learning" (Engestrom) to be helpful, but I like how Eisner's term goes directly at the teacher-learner, teacher-researcher experience. I may need to borrow or develop a more precise term other than "expertise" (something else to reflect upon later).

At any rate, I have noted before in this blog the parallel developmental trajectories of novice qualitative researchers and teachers-as-learners insofar as stance, disposition, and expertise are concerned. And after re-reading Watt, I have a renewed sense of how reflexivity mediates this process.

The Multiple Realities perspective
I have also referred previously in this blog to the Multiple Realities perspective, a central notion in my theoretical frameworks. I was excited to make a connection between Watt's article and Labbo and Reinking's (1999) seminal theoretical piece, "Negotiating the Multiple Realities of Technology in Literacy Research and Instruction."

Although Labbo and Reinking do not specifically use the term "reflexivity," they put forth the Multiple Realities perspective as a way for New Literacies researchers to monitor and leverage subjectivities in a way that strengthens research-to-practice connections:

For example, when a question related to instructional  practice begins with the phrase "What  does the research say about...?" we  believe it should be followed by an explicit consideration of which reality or set of realities is being considered. Doing so means that the answer will inherently be more complex than citing a string of studies and drawing conclusions from them. It also suggests that it may be important to identify the realities to which a potential answer does not apply or why a question is not a relevant or particularly good one within certain realities. (p. 488)

The Multiple Realities perspective, then, is a framework for guiding reflexive thinking within New Literacies research projects. I had never thought about it in this way until now! Maybe I need to be more intentional about referencing it in the methods section of my Chapter 3.

Implications for use of digital tools
Finally, a note about digital technologies and the role they serve in supporting reflexive practice: Watt's descriptions of her struggles to corral and make sense of the data, the codes, and the categories highlighted the value (for me) of digital tools generally, and CAQDAS in particular. This last insight will only become more compelling with the passage of time (Watt's article is now six years old) and my own increasing awareness of how to use digital tools for qualitative inquiry.

Trustworthiness
The common thread that connects each of the above ideas is how they promote trustworthiness in qualitative research. The discipline of regular, reflexive writing -- "chronicling one's thinking" -- helps the researcher mediate the meaning of her experiences and continually develop her expertise.

As Watt explains, "Revisiting my study has strengthened my confidence in my ability to negotiate the  complex process of qualitative inquiry, and I now see myself as a researcher. The multiple layers of reflection drawn upon in writing and revising this paper have made me more cognizant of how far I have come, and have taken me further along the path to becoming a qualitative researcher" (p. 98).

Reflexivity develops an internal authority of self-knowledge (identity) and self-efficacy, alternate "measures," if you will, of excellence and trustworthiness in research.  Reflexivity develops an authentically authoritative voice that delivers the research findings and interpretations in a way that resonates with readers.

References

Glesne, C. (2005). Becoming qualitative researchers: An introduction (3rd ed.). Boston: Allyn & Bacon.

 Labbo, L. D., & Reinking, D. (1999). Negotiating the multiple realities of technology in literacy research and instruction. Reading Research Quarterly, 34(4), 478–492. doi:10.1598/RRQ.34.4.5

Piantanida, M., & Garman, N. B. (2009). The qualitative dissertation: A guide for students and faculty (2nd ed., Kindle version.). Thousand Oaks, CA: Corwin.

Watt, D. (2007). On becoming a qualitative researcher: The value of reflexivity. The Qualitative Report, 12(1), 82–101.

Reflecting on Watt (2007)

Notes on ATLAS.ti from Friese (2012)

For my dissertation, I have created a hermeneutic unit (HU) in ATLAS.ti. It resides in a folder on my desktop, and I am gradually pulling in documents related to my project, starting with annotated literature and reading notes on Cultural-Historical Activity Theory (CHAT), identity theory, case study, and interviewing methods. I am grouping my literature using the document family function.

For now, my focus is on the document families for case study and interviewing, which I am reading, analyzing, and coding for purposes of writing a more substantial logic-of-justification (Piantanida & Garman, 2009). As part of my project contract for EP659, I will incorporate this synthesis of my readings into my Chapter 3 overhaul.

This is only the second time I have used ATLAS.ti to conduct a literature review. The first time, I found that most of my analysis occurred outside of the software, and I used ATLAS to simply organize and categorize my annotations. The software was helpful, but I did not allot enough time to achieve a true synthesis with it.

Now, as before, I feel my attempts with ATLAS are constrained by a lack of time, and I am attempted to revert back to my "old ways." Why bother taking time to learn a digital tool for a process that people have conducted successfully for years with paper and pencil?

But I want my use of CAQDAS to count for something this time around. So, with that in mind, I began reading the eBook version of Susanne Friese's (2013) QualitativeData Analysis with ATLAS.ti.

Friese's book appeals to those novices and skeptics who may ask, "...[I]f the computer doesn’t do the coding, then what is it good for?" (p. 1). She argues that CAQDAS opens up data coding to a host of new possibilities, and new users must understand the potential value-added of CAQDAS software or risk a cursory application of it. Friese describes an all-too-familiar scenario: "...[W]hen I started to use software to analyze qualitative data in 1992, I did what most novices probably do: I looked at the features, played around a bit, muddled my way through the software and the data, gained some insights and wrote a report. It worked – somehow. But it wasn’t very orderly or systematic" (p. 2).

Friese argues that nowhere in the extant literature on CAQDAS is there a systematic guide for its use. In her book on ATLAS.ti, she refrains from being overly prescriptive -- how can she be with a system that, at last count, offered more than 400 sub-menus? Friese simply outlines the approach she has honed over the last 20 years. Passages laden with technical how-to are couched in methodological terms. The “how” is linked to the “why,” so readers can appreciate the intended analytical rigor behind each skill-building exercise.

In consideration of my next big foray into ATLAS.ti, I read with an eye toward learning new methodological and technical practices and techniques. Here are some key take-aways:

Methodological advice and ah-ha's:

• Learn to interpret the numerals inside brackets following each code. The first number refers to the number of times a code has been used (its "groundedness"); the second number relates to the code's "density" and has to do with how the code functions within the network of other codes. I never understood what the second number signified, until now. Interestingly, ATLAS offers network views, but the networks are created manually through the user's interpretive process. This is discussed more in-depth in Chapter 7.  
• As an early analytic move, comment on each PDoc and group PDocs into families as you add them to the HU. Document families are a prerequisite for running certain kinds of data queries later on. 
• Refer to Chapter 4 for an interesting discussion of how to use the coding tools in ATLAS. The tools are modeled after principles of Corbin and Strauss' grounded theory, but that does not limit how they are applied. Chapter 4 does an excellent job of weaving methodological advice with technical how-to, such as the use and pitfalls of in vivo coding. 
• Be diligent about defining codes with the comment tool. Code definitions evolve over time, and sometimes over the course of a project, it is possible to forget what a code originally stood for. This is another reason to use the drag-and-drop method afforded by the Code Manager (Friese's preference) instead of the list option (what I am accustomed to using). By keeping the Code Manager open in the workspace, definitions are constantly viewable as you select and apply each code. 
• Manage memo settings (p. 138). This step is crucial, as writing memos in ATLAS goes hand-in-hand with use of the query tool. I have been taking field notes and writing memos in Evernote, and I have been using Evernote's tagging utility to label my notes thus: methodological note, theoretical note, personal note, and observational note (based on Richardson's scheme, which I wrote about in a previous post). I like using the Evernote app on my iPad in the field; it's less intrusive. And I generally avoid using my PC laptop (which runs ATLAS) for any form of data entry because I dislike the keyboard. Following Friese's suggestion, I adjusted the memo types in ATLAS to align with the memo types/tags I already use. Now, as I begin coding and analyzing my texts with ATLAS, I may continue generating and labeling memos using my personal scheme. Next, I need to consider how to get my Evernote data into ATLAS. Copy and paste into internal text documents within ATLAS? (See pp. 55-56.) 
• Create analytic memos based on the research questions (pp. 143-145, p. 148). It is likely that most of my subsequent memos, while working in ATLAS, will be theoretical or methodological in nature. Friese suggests a special class of analytic memos called "research question" memos for when the user enters into a second level of analysis that involves querying the data and finding relations. Research question memos may be generated at the start of the project and added to and revised over the duration of the project. She explains,"In your first research question memos, the answers will probably be descriptive. But in time they will become more abstract as you get ideas for further questions, add new research question memos and basically take it one step further at a time, gaining more and more understanding, exploring more and more details of your data landscape and starting to see relations between them" (p. 145). 
• After coding the texts/data, review Chapter 6 for ideas about how to query the data. The last half of Chapter 6 is very procedural and skills-based and will make more sense to a reader who already has a list of codes and has started to conceptualize those codes. Friese reviews the query tool, the co-occurrence explorer, and the Codes-Primary-Documents-Table, which can be used to find relationships and patterns. At this stage of analysis, the research question memos can be used to keep record of queries and results (as pictured on p. 144). Further, the memos, if set up correctly and linked to quotations, can be used to generate output that serves as "building blocks" for the findings chapter. 
• Consider incorporating ATLAS.ti into the dissertation defense. ATLAS can support the presentation of findings in a number of ways listed on pp. 219-221.

Methodologically, this passage from page 1 of Chapter 6 says it all:

A lot of the analysis happens as you write, not by clicking on some buttons and outputting some results. You need to look at what the software retrieves, read through it and write it up in your own words in order to gain an understanding of what is happening in the data. Most of the time insights need to be worked at and are not revealed to you immediately by looking at the results. Simply seeing that there are, say, 10 quotations is not enough; numbers are sometimes useful, they hint that there might be something interesting there, but the important step is to take a closer look and to see what’s behind them. (p. 133)  

Technological advice and ah-ha's:  

• When the Code Manager is open, navigate through a long list of codes by pointing the cursor on the Code Manager, and typing the first few letters of the desired code. If I ever amass the average number of 120-200 codes, this practice will be useful.  
• When preparing transcripts in a word processor, save them as rich text (.rtf files) so that ATLAS does not need to convert them. Rich text is the standard file format in ATLAS, not .doc or docx. Typically, after transcribing in InqScribe, I format transcripts in MS Word. It would not be difficult for me to Save As .rtf. This is something to consider; although, in the past I have used .doc files in ATLAS and did not incur any problems. 
• Many suggestions in Chapter 2 for formatting transcripts! I have not yet loaded any transcripts into my dissertation HU, and Friese's formatting guidelines should be easy to implement. These include marking speakers with unique identifiers that do not appear in the text (i.e. "INT" for "interviewer," and so on), double spacing between turns, and breaking up long turns with empty lines. Some of these preparations are to facilitate use of the ATLAS automatic coding tool, which I don't know if I even want or need to use, but Friese suggests getting into the habit anyway.  
• Learn the ATLAS.ti file protocol. Friese calls it "the biggest hurdle" in ATLAS project management, and on pp. 36-37 she thoroughly demystifies this aspect of the program. It's a matter of understanding "external references," a system of document storage and retrieval that prevents a project file from becoming too big and unwieldy. I now have a firmer understanding of how the one folder for all data rule works and why I should follow this rule. More importantly, the name of each data file should be analytically helpful, not generic like "transcript1," "transcript2," and so on. Choose a file-naming system early. It will add transparency and efficiency to the project. The data files (primary documents) can be sorted in the document manager, providing a quick glimpse of your sampling. 

One immensely telling detail: the total number of menus and submenus in ATLAS.ti equaled 443 at the time Friese published her handbook. The point is that working with ATLAS can be highly individualized. The interface is designed to provide users with multiple options for performing the same function. Thus, through practice, the user develops his or her own preferred workflow and routines.  (I like this as it basically sums up my whole approach to teaching and learning with technology.)

References 
Friese, S. (2012). Qualitative data analysis with ATLAS.ti. (eBook.). Los Angeles: SAGE Publications Ltd.

Piantanida, M., & Garman, N. B. (2009). The qualitative dissertation: A guide for students and faculty. (2nd ed., Kindle version.). Thousand Oaks, CA: Corwin.

Notes on ATLAS.ti from Friese (2012)

Reflection on Atlas.ti

Last night in class, we practiced performing data analysis with Atlas.ti, and the exercise confirmed a nagging suspicion: I have grossly underutilized the functions of this CAQDAS tool. There are so many features and options inside the Atlas.ti environment, and, to quote our instructor, "at least 12 different ways" to perform each function. As we gathered up our things to go home, one of my classmates remarked that she felt like she had "run a marathon."

I've run a few laps already with Atlas.ti, using it to code data for two small projects within the past year.  Yet, even I was mentally and physically wasted after last night's workout.

Maybe "triathlon" is a better way to describe what it's like to work in the flexible and multifaceted environment of Atlas.ti. Konopasek (2008) referred to the "sophisticated interface" of CAQDAS tools in general and then specifically described Atlas.ti's "visualisation" capabilities, in which the researcher's "thoughts or mental operations can easily be stored, recollected, classified, linked, filtered out in great numbers...and made meaningful in sum."

Some rights reserved by hmcotterill

Is this why the Atlas.ti developers named their product after the mighty hero of Greek myth, the one who bore the weight of the world on his shoulders?

Something else strikes me as powerful about Atlas.ti and the other digital tools we are exploring. Again and again in the EP604 course readings, I've noticed the suggestion that technology is blurring the lines between the strict, paradigmatic camps -- quantitative vs. qualitative, positivist vs. constructivist.

Seale (2010) tells us that the "counting" capabilities of CAQDAS software "is a reminder that the days of a great divide between qualitative and quantitative research work have now largely passed" (p. 255). And I am intrigued by Konopasek's comparison of Atlas.ti to a "textual laboratory." The metaphor literally co-opts the venue most commonly associated with scientific and positivist inquiry.

I am no statistician, but I like the way Atlas.ti enables the qualitative researcher to perform quantitative functions such as frequency counts and the "Word Cruncher," not as an end-all-be-all of analysis, but as a jumping off point for deeper exploration of connections, patterns, and new meanings.

For example, the first time I used the software, I ran a frequency count for a particular phrase in my transcripts just to confirm a hunch before I started coding.  I realize I could have used the "find" and "comment" tools in MSWord to perform these simple operations, but it was what I was able to do after coding that sets Atlas.ti apart from a word processor. I began looking for the co-occurrence of three specific a priori codes based on the TPACK framework (technology, pedagogy, and content).  Turns out, that didn't happen much in my transcripts, but Atlas.ti did help me to see more than twenty intersections between the "technology" and "pedagogy" codes.  This sent me down an altogether different and fruitful path of inquiry.

These previous experiences with Atlas.ti are the equivalent to running sprints. Now, after having received some guided, hands-on instruction, I have a clearer vision of how Atlas.ti. can function as an all-inclusive research notebook, containing fieldnotes, comments, memos, codes, and a seeming infinite variety of visual, textual, and statistical outputs generated by the researcher. I am ready to go the distance.

Reflection on Atlas.ti