CHAT and education research (Part II of a series)

I recently stumbled upon a 2011 paper by Lee, who argues for increased use of Cultural-Historical Activity Theory (CHAT) in research on educational change.

I could not have found this at a more opportune time. Lee presents a good overview of CHAT and its place in the sociocultural tradition. I needed this refresher as well as new ways to think about applying CHAT as an interpretive lens to the data set for my project.

Sociocultural theory
First, Lee introduced the family of theories that compose the sociocultural tradition:

"A heterogeneous grouping of frameworks (e.g. distributed/situated cognition, practice theory, communities of practice, cognitive apprenticeships), they are united in accounting for behavior and learning beyond purely mental conceptions housed within human minds. Giving primacy instead to social interactions with other human beings and designed artifacts as the catalysts for cognition, the process of human development becomes inextricably linked with participation in culture and history rather than being dictated by biology" (p. 403).

Sociocultural theorists seek to dissolve dichotomies based on traditional Western philosophy, such as cognition and identity, person and group. Sociocultural theory provides for a  "richer, non-reductionist, and more humane approach towards educational practice and research" (p. 404).

OK, so what about CHAT?

CHAT is the newest member of the family, and it is growing in popularity. It is comprised of both a theoretical perspective and a concrete analytical method (activity theory), and the purpose of Lee's article is to argue that CHAT "can address some of the major shortcomings or gaps in educational change research" (p. 404).

The diagram (again!)
Lee provides an excellent explanation of the expansion of Vygotsky’s classic triangle diagram into the activity systems triangular diagram (both pictured below):

"While Vygotsky first proposed mediating tools (e.g. semiotic sign systems such as mathematics and language or physical tools such as writing equipment), contemporary versions of CHAT include other mediators such as rules, community, and the division of labor that are all dialectically linked. Together, these permit the subject (i.e. the agents) to transform the object (that which is to be changed) to produce an outcome" (p. 407).

Over the last century, theorists expanded Vygotsky's simple triangle diagram to show that his original conception of "tools" is but one kind of mediator in an activity system. Lee explains, "Although any one or more of the mediators in an activity system can be foregrounded, the rest are not absent and are in fact indispensible to describe the 'hows' and 'whys' of subjects' transformations of objects," (p. 407). This last part about "transformations of objects" is another way to view learning: learning equates to change in both subject(s) and object(s) as a result of activity.

In the remainder of his article, Lee:

• outlined five current challenges and shortcomings in educational research,
• described CHAT and how curriculum researchers have used it, and
• discussed future implications for use of CHAT in education research.

Five challenges in educational research
As I summarize Lee's five "predicaments" in educational research, I will note specific connections to and implications for my current project in italics:

1. Failing to acknowledge role of context. I cite "context neutrality" as a major impetus in my dissertation project on teachers and technology.
2. Seeking simplification instead of embracing complexity. Educational research often objectifies students rather than viewing them as subjects with their own objectives (objects of activity). Consequently, students are often an "under represented stakeholder voice in educational change research" (p. 414). "Unpacking the various configurations of the object," as Lee put it, is a major advantage of CHAT. In my project the participants/students are literacy teacher-learners. My goal is to understand their unique perspectives on instructional technology (IT) practices and processes. This would include their various goals and objectives with regard to IT. By glimpsing this complexity, I hope to understand variation in the overall participant experience, which might inform my own future enactments as a facilitator of online and blended learning in literacy teacher education. In my project, I rely on a companion perspective, known as "multiple realities" (Labbo & Reinking, 1999), which confronts that “common and unfortunate tendency to treat technology in relation to literacy as a monolithic, unidimensional topic and a corresponding tendency to oversimplify its use or potential use in literacy instruction” (p. 479). When IT is critiqued as misguided and ineffective, it is quite possibly due to a failure on the part of reformers, researc hers, and policymakers to acknowledge the multiple realities surrounding its implementation.
3. Failing to critique the role of power and politics: "While maintaining research ethics and researcher reflexivity have risen to the fore, one still has to be constantly mindful of ethical considerations as there is a strong tendency towards objectifying participants due to inherent power differentials. Rather than dismissing conflicts and tensions in and around educational systems as something negative or to be avoided altogether, their inevitability can be a vital impetus for improvement and renewal" (Lee, p. 406). This idea is also highly complementary to the multiple realities perspective.
4. Ignoring identity and emotions. Emotions are important to stimulating educational change but are often treated as nothing more than steppingstones to leverage change. (Consider the way educators discuss social engagement as a way to hook kids into learning, i.e. treating engagement as a step toward learning rather than integral to learning.) "Identity, which is the sense of who we are and what we stand for" (Lee, p. 406) is overlooked precisely because it is so integrated and bound up with learning, but identity and emotion are "highly pertinent, as analytic concepts to understand why people do the things that they do" (p. 406). As theoretical concepts, identity and emotion enable researchers to "better recognize and predict the extent to which people see themselves as agents with power to transform an activity system and to improve their own lives" (p. 417). Lee introduces the concept of "4th generation CHAT," which applies innovative surveying and interview techniques to reveal "these notoriously elusive social-psychological dimensions" (p. 417). In my project I deployed a data generation technique called “interactive interviewing,” in which researcher and participant each assume overlapping roles as expert and guide. The researcher and participant each bring a story to the interaction, and as they converse, they stimulate each other’s story, reflexively co-reconstructing experience through conversation (Ellis, Kiesinger, & Tillman-Healy, 1997).
5. Introducing new reforms and innovations too quickly. (I am reminded of the education truism "reform fatigue.") This last challenge introduces an apparent contradiction between CHAT, which demands a "historical," long view on reform, and the reality of rapid change in school-based reforms and interventions. Lee argues that CHAT allows for "nested levels of analysis," including that of local, situated events enacted in real-time. "This plane still remains within a larger historical frame, thereby giving both long- and short-term views of meaning-making by participants that are necessary for building an inclusive picture of change" (p. 417). So, CHAT can provide insight into new practices with respect to human agency, such as when careful "adherence to new modes of instruction turn out to short-circuit the original objectives of planners. These subtle mismatches in the object of activity have usually been only uncovered through the careful recording of classroom practices at the enacted or interactional level."  I see a major implication for my own study, which engaged participants in several new technology practices, particularly online digital video analysis.

Implications for my research
CHAT, as both theory and method, directly speaks to these challenges, as outlined above. For the second half of his article, Lee demonstrates the merits of CHAT using data from a study of inquiry-based science curriculum in a Singaporean secondary school.

CHAT also excels in interpretive, small-scale, teacher-oriented studies of educational change.  I hope to demonstrate this in my own study.

My first order of business is to unpack those many “configurations of the object.” To do this, I have engaged over the last several months in "boundary-crossing" (p. 408) with participants to identify barriers, obstacles, and contradictions within the online course. The interactive interviews served this purpose, as will upcoming “member reflections” (Tracy, 2010), which I am conducting in July.

Next, I must tell the story of the “’hows’ and ‘whys’ of subjects’ transformations of objects.” This first level of analysis will result in a case study narrative, which I am currently drafting. My case study narrative must take pains to situate the story of the course pilot within specific cultural and historical contexts.

Last, I will conduct the second stage of analysis – drafting triangle models of activity systems and jotting down ideas about different "mediators." I must analyze the tensions arising within/without the system and deliberate on ways to leverage these tensions for change. One mediating tension I am particularly curious about is how a teacher’s self-understanding (identity) as a technology user influences (or is influenced by) the online learning experience.

Questions about identity are best explored through 4th-generation activity theory. In a final post to this series, I will look at the work of two CHAT theorists, Roth and Stetsenko, who, in their own ways, have developed a unified theory for understanding identity development within complex learning systems. Their work to resolve tensions and gaps within the CHAT tradition partly inspired the design of my study.

References

Ellis, C., Kiesinger, C., & Tillmann-Healy, L. (1997). Interactive interviewing: Talking about emotional experience. In R. Hertz (Ed.), Reflexivity and voice (pp. 119–149). Thousand Oaks, CA: SAGE.

 Labbo, L. D., & Reinking, D. (1999). Negotiating the multiple realities of technology in literacy research and instruction. Reading Research Quarterly, 34(4), 478–492. doi:10.1598/RRQ.34.4.5

 Lee, Y. J. (2011). More than just story-telling: Cultural–Historical Activity Theory as an under-utilized methodology for educational change research. Journal of Curriculum Studies, 43(3), 403–424.

Tracy, S. J. (2010). Qualitative quality: Eight “big-tent” criteria for excellent qualitative research. Qualitative Inquiry, 16(10), 837–851.

CHAT and education research (Part II of a series)

What is CHAT? (Part 1 of a series)

I've spent the first half of this year immersed in methods literature (qualitative research, case study, transcription, reflexivity, analysis, for example).

I've fallen out of touch with the substantive theory of my project, which is a study of teacher-learners and their dispositions and mindsets toward digital technologies used in an online, graduate Reading Education course.

New articles and PDFs have piled up, so I am slowly digging into these, beginning with some readings in Cultural-Historical Activity Theory (CHAT), which I will elaborate on in a series of posts, beginning with this one.

The tricky thing about CHAT is it is both a substantive theory applied to human activity and learning as well as a nuts-and-bolts analytical method, should the researcher feel occasion to deploy it. (I, in fact, am going to try my hand at activity systems analysis this summer, in the final stages of my dissertation work.)

CHAT theorists whose writings have proven significant to my thinking about teacher experiences with technology include Yew-Jin Lee, Wolff-Michael Roth, and Anna Stetsenko. I've set up Google Scholar alerts to make sure I do not miss any new publications by these scholars; in the meantime, I am playing catch-up with past publications. In a follow-up to this post, I will summarize a 2011 article by Lee titled More than Just Storytelling: Cultural-Historical Activity Theory as an Under-Utilized Methodology for Educational Change Research.

But, first, a quick explanation of how I plan to use activity theory (the “AT” in “CHAT”).

Activity theory and activity systems analysis provide a way to understand the multiple realities that impinge on teacher education programs, in general, and tech-infused programs, in particular. I will apply activity systems analysis to understand various participants' developmental paths along the novice-to-expert spectrum and how their self-understandings influenced (and were influenced by) the online course that I helped facilitate in Fall 2012.

Scandinavian theorist Yrjo Engeström innovated the mediational triangle diagram, which represents an activity system as a basic unit of analysis and is commonly pictured in activity theoretical studies (Kaptelinin & Nardi, 2006, p. 99).



According to Roth (2004), an under-explored topic in studies of activity systems is "changing identities in changing communities as a result of praxis" (p. 6). By examining one or more "salient contradictions," activity systems analysis can shine light on questions about identity because "engagement with the contradictions leads to change, in the conditions concretely experienced by the participants and in their identities" (p. 7).

And a marker of teacher expertise is a willingness to engage with contradictions, an idea expressed throughout the literature (e.g. Koehler & Mishra, 2008).

Roth (2004), Lee (2011), and Stetsenko (2010), who, in their defense of activity theory against claims that it is too static and structured, make a case for its use in the study of identity development.

Despite the seeming calcification of Engeström's famous activity systems diagram, Roth argues that the model is "inherently dynamic" (p. 2) because activity embodies change.

Roth cites two features to argue his point. First, subject and object are a "dialectic unit." The object is bound to the subject, and it is tied to subject identity. People rely on object outcomes to build their identities.

Second, the idea "of practical activity and learning as coinciding with changing life conditions" can be traced all the way back to Marx and Engels and has been taken up time and again in the sociocultural tradition, by theorists such as Engeström and Lave. "That is, although the Engeström triangle depicts the structure of activity, it is inherently a dynamic structure continuously undergoing change in its parts, in its relations, and as a whole" (Roth, 2004, p. 4).

What are the implications of this? Roth asks, “If participation in activity changes the identity of the subject, what are the effects of the alienating structures of schooling?” Readers might think of standardized testing, one-size-fits-all curriculum requirements, teacher-centered instruction, and tightly regimented and routinized technology access (or zero technology access, in some cases). The list goes on.

As for teacher professional development, the activity that I am studying, it is no less influenced by “alienating structures.” Identifying these structures and how they affect teacher-learners is the purpose behind my study.

References

Kaptelinin, V., & Nardi, B. A. (2006). Acting with technology: Activity theory and interaction design. Cambridge, MA: MIT Press. Retrieved from http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=11004

Koehler, M. J., & Mishra, P. (2008). Introducing TPCK. In AACTE Committee on Innovation and Technology (Ed.), Handbook of Technological Pedagogical Content Knowledge (TPCK) for educators (pp. 3–29). New York: Routledge.

Lee, Y. J. (2011). More than just story-telling: Cultural–Historical Activity Theory as an under-utilized methodology for educational change research. Journal of Curriculum Studies, 43(3), 403–424.

Roth, W.-M. (2004). Activity theory and education: An introduction. Mind, Culture, and Activity, 11(1), 1–8. doi:10.1207/s15327884mca1101_1

Stetsenko, A. (2009). Teaching–learning and development as activist projects of historical Becoming: Expanding Vygotsky’s approach to pedagogy. Pedagogies: An International Journal, 5(1), 6–16. doi:10.1080/15544800903406266

What is CHAT? (Part 1 of a series)

A new "pressure point"

I remember well Piantanida and Garman’s (2009) advice about constructing a logic-of-justification for a research genre. It is not simply a matter of locating a recipe for the study. Instead, one must recognize and sort through the “epistemological and methodological pressure points” in the literature and choose which ideas will best guide the study at hand (Ch. 7, Conventions of a Genre and Logic-of-Justification, para 2).

I had forgotten that Piantanida and Garman had specifically referenced grounded theory as one such genre framed by “contentious literature”!

So, as I began my background reading of constant comparative analysis (with its roots in grounded theory), I was not expecting to encounter a new pressure point.

The tension for me, however, is not within the grounded theory tradition itself but between grounded theory’s constant comparative method and Stake’s (1995) case study method.

I thought I had this part pretty well sewn up: case study research methods, followed by constant comparative analysis, and concluding with activity systems analysis. The path, while not necessarily easy, was at least clear.

Now, a bump in the path. 

Points of departure
On the subjects of generalization and sampling, I see a need to reconcile the different analytic stances of Strauss and Corbin versus Stake. With constant comparative method originating in the grounded theory tradition, and with the purpose of grounded theory being a systematic progression from descriptive to theoretical, is constant comparative analysis compatible with Stake's case study approach?

Time and again, Stake emphasizes that “case studies are undertaken to make the case understandable” (p. 85). He writes, “The function of research is not necessarily to map and conquer the world but to sophisticate the beholding of it” (p. 43). A skilled case researcher should organize his or her report in such a way as to stimulate a resonance with readers, who draw on their own past experiences with past cases to form “naturalistic generalizations” (p. 85).


Case research, then, is never about sampling: “Our first obligation is to understand this one case” (p. 4, emphasis added).

On the other hand, Strauss and Corbin (1998) present a step-by-step process of microanalysis that moves beyond description to conceptualizing and classifying and (eventually) theory building. The basic operations of theory building are 1) asking questions and 2) making theoretical comparisons -- analytic tools that guide and direct theoretical sampling.

In grounded theory, the researcher makes theoretical comparisons when in doubt or confused by the data. "The object, then, is to become sensitive to the number and types of properties that might pertain to phenomena that otherwise might not be noticed or noticed only much later" (Strauss &Corbin, p. 82). Properties and dimensions of one thing are used as tools for examining another.

Strauss and Corbin say the point of theoretical comparison is to move beyond describing and pinning down "facts." The issue, they say, is to move from the particular to the general.

The sticking point
How does the process of theoretical comparison fit with case study analysis?

Stake is clear: case studies don't produce generalizations, they refine them. Through counter example, a case study may help modify an existing generalization, but "the real business of case study is particularization, not generalization. We take a particular case and come to know it well, not primarily as to how it is different from others but what it is, what it does" (p. 8).

Particularization is the avenue to understanding, not generalization.

Or, has constant comparative method (and coding and memo writing, for that matter) become so commonplace within the broad spectrum of qualitative inquiry that it can be logically applied in descriptive case study? Has the constant comparative method become independent of grounded theory as a thematic analysis tool? At what point did the tools of grounded theory work cross over into the mainstream?

For instance, Saldaña (2013) presents six coding methods from the grounded theory “coding canon” in the newest edition of his book: in vivo, process, initial (formerly “open”), focused, axial, and theoretical (formerly “selective”). He says they all can be used in other non-grounded theory studies (p. 51).

At my breaking point

Part of my confusion, I realize now, stems from the fact that I have never clearly resolved in my head if my case is, according to Stake's terms, an "intrinsic" one or an "instrumental" one. In an intrinsic study the unique case itself is the center of attention; in instrumental study the case(s) are selected purposefully based on the researcher’s need to more fully understand an issue. Whatever the case, it’s not a process of sampling.

I think my study is somewhere in the middle: the case was practically handed to me on a silver platter, but my interest is instrumental and pre-existing. I embraced the opportunity to study the case in question because of my own a priori interests.

Stake warns that it is often not easy to categorize one’s work based on these distinctions. The researcher’s interest in the case (intrinsic versus instrumental) dictates the methods of analysis.

Where once I appreciated Stake’s broadminded stance (“..I encourage you readers to be alert for tactics that do not fit your style of operation or circumstances. Before you is a palette of methods” [Introduction, p. xii].), I now barely comprehend his musings on the “mystical side of analysis” (p. 72).

Stake writes,

Where thoughts come from, whence meaning, remains a mystery. The page does not write itself, but by finding, for analysis, the right ambience, the right moment, by reading and rereading the accounts, by deep thinking, then understanding creeps forward and your page is printed. (p. 73)

Whaaat?

Points on a continuum
Must I choose between Stake’s riddles and Strauss and Corbin’s rigidity?

In short, no. But I definitely need to write a clear articulation of the purpose, nature, and selection of my case as part of the logic-of-justification for my analytic methods.

For Stake, the case researcher must be equally inclined toward inductive analysis, which he calls "categorical aggregation," and interpretive analysis, or "direct interpretation." An intrinsic case study requires more direct interpretation, as there is little time or need to aggregate categorical data. Intrinsic case studies are more descriptive, with emphasis on particularization. In contrast, instrumental case studies are more theoretical, with emphasis on induction and generalization.

These analytical methods reside along a paradigmatic continuum with no hard-and-fast boundaries. Stake writes, "The quantitative side of me looked for the emergence of meaning from the repetition of phenomena. The qualitative side of me looked for the emergence of meaning in the single instance" (p. 76).

As with every other stage of the process, reflexivity is key:

Each researcher needs, through experience and reflection, to find the forms of analysis that work for him or her....The nature of the study, the focus of the research questions, the curiosities of the researcher pretty well determine what analytic strategies should be followed: categorical aggregation or direct interpretation. (p. 77)

The point, I think, is it is up to me to argue compatibility between the constant comparative method and Stake. The type and purpose of the case, the conceptual structure of the study, and reflexive management of evolving research questions will determine where I land along the analysis continuum.

References
Piantanida, M., & Garman, N. B. (2009). The qualitative dissertation: A guide for students and faculty (2nd ed., Kindle version.). Thousand Oaks, CA: Corwin.

Saldaña, J. (2013). The coding manual for qualitative researchers (2nd ed.). Los Angeles: SAGE Publications Ltd.

Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: SAGE Publications.

Strauss, A., & Corbin, J. M. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. Thousand Oaks, CA: SAGE Publications.

A new "pressure point"

Dropping the ball

Research is a juggling act. I think to be a successful researcher you must juggle at any given time at least four of the following: data collection, reading, writing, analysis, reflexivity -- always reflexivity and reflection.

Am I leaving anything out? Probably.

This whole semester I thought I was doing well, when, in fact, at most I was tossing around maybe two things at a time. Big deal.

After reading the coding and analysis literature this week, I realize I have dropped a major ball, the analysis one.

I also realize I have some new questions. (See end of post for questions).

Coding, analysis, and interpretation
Coding is a commonly accepted approach to analysis, and, as such, it has its share of critics, who claim it is an over-glorified form of reductionist frequency counting.

In the opening chapters of the new edition of his coding manual, Saldaña (2013) addresses the critics. He says coding is not the end-all, be-all of analysis. Nor is it a discrete stage of analysis done in isolation from other aspects of research.

Saldaña argues that coding does not distance the researcher from his or her data:

If you are doing your job right as a qualitative researcher, nothing could be further from the truth. Coding well requires that you reflect deeply on the meanings of each and every datum. Coding well requires that you read and reread and reread yet again as you code, recode, and recode yet again. Coding well leads to total immersion In your data corpus with the outcome being exponential and intimate familiarity with its detail, subtleties, and nuances. (p. 39)

I so need to do this. I haven't been handling the data well at all. I have been amassing it, stockpiling it, peering at it occasionally with heavy heart as I do with hampers of dirty laundry. In fact, I now have piles of neatly folded, laundered clothes, and no grasp of my study. I will choose laundry any day over the challenge of data analysis.

The readings on coding and analysis cause me to feel a pit in my stomach as I realize how far removed I have become from the actual texts of my study. I haven't transcribed in weeks. I haven't touched, much less reflected on, field notes, emails, and other artifacts from last semester. I acquired a thick folder of documentation from one of the key informants for my study, and the documents still sit where I left them two weeks ago.

Worse, I have been continually conducting interviews with participants for the last three months, without the benefit of immersion in the data to guide or shape my interactions with those participants.

In their discussion on the process of line-by-line microanalysis, Strauss and Corbin (1998) write, "We are moved through microanalysis by asking questions, lots of them, some general but others more specific. Some of these questions may be descriptive, helping us ask better interview questions during the subsequent interviews" (p. 66).

I should be forming theoretical questions that probe relationships between concepts and then asking these question during follow-up interviews.

I just concluded the first round of interviews. Time is running out for follow-ups, as my participants are classroom teachers who will not relish being interviewed over summer vacation. This is a potential problem.

Analytic memos
And because I have not been immersed in the data, I also have not generated a single analytic memo about the data, not since last semester. This is another problem.

Analytic memos are the raw materials for what will ultimately become the “theoretic text,” in which the researcher “finally sees the theoretic interpretation – core thesis – he or she wants to put forward” (Piantanida & Garman, 2009, Ch. 13, para 1). They are initiated by “aha moments” and “conceptual leaps” that put “the myriad individual, idiosyncratic, and situational details into a meaningful, coherent, theoretic perspective” (para 2).

According to Saldaña, "Virtually every qualitative research methodologist agrees: whenever anything related to and significant about the coding or analysis of the data comes to mind, stop whatever you are doing and write a memo about it immediately" (p. 42).

Richardson (1994) refers to analytic memos as "theoretical notes." These are the researcher's hunches, hypotheses, connections, and/or critiques about what is being seen and heard in the field. The researcher opens up his or her texts to interpretation and a "critical epistemological stance" (p. 526).

Memo writing goes hand-in-hand with analysis. All sorts of memos may be generated during research, but the analytic ones mark the researcher's first attempts at creating findings.

In writing workshop, instructors guide their students: "Don't get it right, get it writ." This advice applies to the research memo as well. Saldaña says just write the memo; worry about the title and category later. He explains,

I simply write what is going through my mind, then determine what type of memo I have written to title it and thus later determine its place in the data corpus. Yes, memos are data; and as such they, too, can be coded, categorized, and searched with CAQDAS programs. Dating each memo helps keep track of the evolution of your study. Giving each memo a descriptive title and evocative subtitle enables you to classify it and later retrieve it through a CAQDAS search. (p. 42)

Grounded Theory and ATLAS.ti
One thing that has become more transparent to me in the coding and analysis readings is the connection between grounded theory and ATLAS.ti.

Some researchers distrust CAQDAS tools because “enduring foundationalist epistemologies are clearly being drawn on in their design and programming” (Brown, 2002). After reading portions of Strauss and Corbin’s (1998) handbook on grounded theory, I finally see what the critics are talking about. Key features of ATLAS seem to be borrowed directly from the grounded theory genre: open codes, in vivo codes, network views (Strauss and Corbin call them “diagrams”), and the integration of memos.

Strauss and Corbin did not claim to know much about computers in their 1998 volume, but they specifically mention ATLAS.ti as “more geared toward theory building” (p. 276) and reproduce a memo from ATLAS developer Heiner Legeiwe that says as much.

Questions:

• Both the grounded theory guidelines as well as Saldaña’s book refer to the use of categories. How does one denote categories in ATLAS.ti? I have been using prefixes to organize codes upfront, so my prefixes are definitely not categorical. They are simply organizational/topical in nature. Is this what code families are for? Does it matter?
• Saldaña refers to “subcodes” and “subcategories.” How can these be represented in ATLAS.ti?

References
Brown, D. (2002). Going digital and staying qualitative: Some alternative strategies for digitizing the qualitative research process. Forum Qualitative Sozialforschung/Forum: Qualitative Social Research, 3(2). Retrieved from http://www.qualitative-research.net/index.php/fqs/article/viewArticle/851

Piantanida, M., & Garman, N. B. (2009). The qualitative dissertation: A guide for students and faculty (2nd ed., Kindle version.). Thousand Oaks, CA: Corwin.

Richardson, L. (1994). Writing: A method of inquiry. In N. K. Denzin & Y. S. Lincoln (Eds.), Handbook of qualitative research (pp. 516–529). Thousand Oaks, CA: SAGE Publications, Inc.

Strauss, A., & Corbin, J. M. (1998). Basics of qualitative research: Techniques and procedures for developing grounded theory. Thousand Oaks, CA: SAGE Publications.

Saldaña, J. (2013). The coding manual for qualitative researchers (2nd ed.). Los Angeles: SAGE Publications Ltd.

Dropping the ball

ATLAS.ti and network views: No magic wands

After reading Susanne Friese's book, I knew what functions of the ATLAS.ti program I most wanted to put to work: memos, queries, and network views. I have tried unsuccessfully in the past to use these features, but they require a certain amount of finesse and a certain amount of insight into their underlying logic that I just did not possess.

My usual strategy for learning a new technology (pushing a lot of buttons, clicking a lot of links, and generally just fooling around with it until it works) just wasn't cutting it.

So, I turned to Friese, whose book outlines the strategic use of memos for running data queries and integrating findings. Memos are instrumental to the entire analytical process but play an especially important role during writing and production of the final report (read: DISSERTATION).

Friese encourages: "In order to see the benefits of it, you have to try it." She says some people may call it "magic," but it is really just "the added value of approaching analysis in a systematic way utilizing the options available" (p. 142).

Stop expecting "magic." Start applying a "systematic" approach. That is what I needed to do.

Memos
So, I started small, with one primary document (Watt's 2007 article on researcher reflexivity), one memo, and one writing task. I wanted to use the memo tool to develop some discursive text -- a blog post -- based on Watt's article.  Before I began reading and coding the article in ATLAS, I opened the Memo Manager, created a memo, gave it a "proper" title, and categorized it by type.

Meanwhile, I also created, titled, and categorized a free memo to record my process of using memos in ATLAS. Yes. A memo about memo'ing -- a "meta memo," which, in turn, became this blog post.

According to Friese, memos can be linked to quotations, to codes, and to other memos. I tried this and linked the Watt memo to the PDoc itself, to specific quotations within the PDoc, and to significant codes. Now what?

Queries & Network Views
Friese says memos serve an analytic function (a "container of ideas") and a technical function. After coding the data (in my case, the Watt literature), the user may probe more deeply by running queries. Queries serve a data retrieval function by probing connections between PDocs, quotes, codes, and memos in depth. The memo serves as a place to record descriptions, interpretations, and ideas based on these probes. Friese calls it "second-level conceptual analysis" (p. 7).

However, for the task at hand, I simply had one PDoc, one memo, a few codes, and a handful of quotes. Upon further consideration, I decided to save the query function for another day. For purposes of developing my blog post, I turned my attention to the network view and memo output functions. I was especially intrigued by the potential of network views for displaying data, creating concept maps, and developing writing heuristics.

Projects created within ATLAS.ti are referred to as hermeneutic units (HUs), based on the science of text interpretation (I know because I finally took the time to look it up.). The HU consists of links that the user creates between all sorts of object nodes:  PDocs, quotes, codes, and memos.  Thus, the HU is really a network, and network views are detailed perspectives on different aspects of the network. 

The thing is, there truly is nothing "magical" about network views, which is why I never had any success with them in the past! I possessed a fundamental misconception of this tool: I thought when I clicked the display network option, the network would magically populate the screen like a flow chart, with all the nodes logically mapped out and connected for me. 

On the contrary, when the user displays the network for an item and all its nodes, the user must manipulate the objects, linking and labeling them in a manner that makes sense to the user. This is why Friese has an entire chapter (Chapter 7) devoted to the creation and manipulation of network views.

My Process
I took some time to explore the quotes and codes associated with the Watt PDoc. How could I use the network view to glean new insights from Watt (2007) in relation to my current project, the dissertation? This exploration was messy and recursive. I spent a lot of time clicking on the quote nodes, which allowed me to read them in full. I found that it was more fruitful to right-click on the quotations and Display in Context. This resulted in more reading and, in some cases, more coding. Finally, under the Display menu, I opted for Quotation Verbosity, Full Text, so I could work with the quotations in full as I arranged them in the network view.

Arranging the codes and experimenting with the Code-to-Code Relations Editor helped build my understanding of exactly what I wanted to say in my Watt reflection. Naming the semantic relationships between the code nodes led me to see how most of the codes had one thing in common: they each connected somehow to the "trustworthiness" code.

I discovered I could double-click on the memo object to display its text in full, and I could double-click on the memo text to edit it. So, as I arranged the network view, I revised and wrote whole new portions of the memo. To illustrate points in the memo, I copied and pasted selected quotes from the context of the article. Toggling in this manner, from network view to memo and back again, became tedious, so I broke my paperless rule and exported the (mostly) finalized network view as a graphic file to my desktop and printed it. It sat on my physical desktop as I finalized the blog post. I would not have done this if I had a larger computer screen or dual monitors.

As can be seen in the "before" and "after" screenshots, I only scratched the surface of what is possible with network views. Using the network view did not make my writing process more efficient -- it took as much, or possibly more time to compose the blog post -- but it did help me conceptualize something new about an article I had already read multiple times over the last three years.

Network View BEFORE

Network View AFTER

I drafted the Watt blog post entirely in ATLAS. I am sure there are multiple ways to go about this; I saved the memo as an .rtf file to my desktop, then copied, pasted, and did final edits in my blog editor.

A final word about Memo Output
So, in the end, I did not utilize the memo output function. With the memo output function, the user selects the memo, right-clicks, selects Output and Selected Memo with Quotations, and chooses Editor as the destination. An editable document is generated that "includes everything you need to write the results chapter of your research report or paper" (p. 146).  However, I can see that if I was working with multiple memos and needed to string them together into one coherent piece of text, such as a chapter of findings, I might want to use this option to pull everything into a more robust word processing program.

ATLAS.ti and network views: No magic wands

Notes on ATLAS.ti from Friese (2012)

For my dissertation, I have created a hermeneutic unit (HU) in ATLAS.ti. It resides in a folder on my desktop, and I am gradually pulling in documents related to my project, starting with annotated literature and reading notes on Cultural-Historical Activity Theory (CHAT), identity theory, case study, and interviewing methods. I am grouping my literature using the document family function.

For now, my focus is on the document families for case study and interviewing, which I am reading, analyzing, and coding for purposes of writing a more substantial logic-of-justification (Piantanida & Garman, 2009). As part of my project contract for EP659, I will incorporate this synthesis of my readings into my Chapter 3 overhaul.

This is only the second time I have used ATLAS.ti to conduct a literature review. The first time, I found that most of my analysis occurred outside of the software, and I used ATLAS to simply organize and categorize my annotations. The software was helpful, but I did not allot enough time to achieve a true synthesis with it.

Now, as before, I feel my attempts with ATLAS are constrained by a lack of time, and I am attempted to revert back to my "old ways." Why bother taking time to learn a digital tool for a process that people have conducted successfully for years with paper and pencil?

But I want my use of CAQDAS to count for something this time around. So, with that in mind, I began reading the eBook version of Susanne Friese's (2013) QualitativeData Analysis with ATLAS.ti.

Friese's book appeals to those novices and skeptics who may ask, "...[I]f the computer doesn’t do the coding, then what is it good for?" (p. 1). She argues that CAQDAS opens up data coding to a host of new possibilities, and new users must understand the potential value-added of CAQDAS software or risk a cursory application of it. Friese describes an all-too-familiar scenario: "...[W]hen I started to use software to analyze qualitative data in 1992, I did what most novices probably do: I looked at the features, played around a bit, muddled my way through the software and the data, gained some insights and wrote a report. It worked – somehow. But it wasn’t very orderly or systematic" (p. 2).

Friese argues that nowhere in the extant literature on CAQDAS is there a systematic guide for its use. In her book on ATLAS.ti, she refrains from being overly prescriptive -- how can she be with a system that, at last count, offered more than 400 sub-menus? Friese simply outlines the approach she has honed over the last 20 years. Passages laden with technical how-to are couched in methodological terms. The “how” is linked to the “why,” so readers can appreciate the intended analytical rigor behind each skill-building exercise.

In consideration of my next big foray into ATLAS.ti, I read with an eye toward learning new methodological and technical practices and techniques. Here are some key take-aways:

Methodological advice and ah-ha's:

• Learn to interpret the numerals inside brackets following each code. The first number refers to the number of times a code has been used (its "groundedness"); the second number relates to the code's "density" and has to do with how the code functions within the network of other codes. I never understood what the second number signified, until now. Interestingly, ATLAS offers network views, but the networks are created manually through the user's interpretive process. This is discussed more in-depth in Chapter 7.  
• As an early analytic move, comment on each PDoc and group PDocs into families as you add them to the HU. Document families are a prerequisite for running certain kinds of data queries later on. 
• Refer to Chapter 4 for an interesting discussion of how to use the coding tools in ATLAS. The tools are modeled after principles of Corbin and Strauss' grounded theory, but that does not limit how they are applied. Chapter 4 does an excellent job of weaving methodological advice with technical how-to, such as the use and pitfalls of in vivo coding. 
• Be diligent about defining codes with the comment tool. Code definitions evolve over time, and sometimes over the course of a project, it is possible to forget what a code originally stood for. This is another reason to use the drag-and-drop method afforded by the Code Manager (Friese's preference) instead of the list option (what I am accustomed to using). By keeping the Code Manager open in the workspace, definitions are constantly viewable as you select and apply each code. 
• Manage memo settings (p. 138). This step is crucial, as writing memos in ATLAS goes hand-in-hand with use of the query tool. I have been taking field notes and writing memos in Evernote, and I have been using Evernote's tagging utility to label my notes thus: methodological note, theoretical note, personal note, and observational note (based on Richardson's scheme, which I wrote about in a previous post). I like using the Evernote app on my iPad in the field; it's less intrusive. And I generally avoid using my PC laptop (which runs ATLAS) for any form of data entry because I dislike the keyboard. Following Friese's suggestion, I adjusted the memo types in ATLAS to align with the memo types/tags I already use. Now, as I begin coding and analyzing my texts with ATLAS, I may continue generating and labeling memos using my personal scheme. Next, I need to consider how to get my Evernote data into ATLAS. Copy and paste into internal text documents within ATLAS? (See pp. 55-56.) 
• Create analytic memos based on the research questions (pp. 143-145, p. 148). It is likely that most of my subsequent memos, while working in ATLAS, will be theoretical or methodological in nature. Friese suggests a special class of analytic memos called "research question" memos for when the user enters into a second level of analysis that involves querying the data and finding relations. Research question memos may be generated at the start of the project and added to and revised over the duration of the project. She explains,"In your first research question memos, the answers will probably be descriptive. But in time they will become more abstract as you get ideas for further questions, add new research question memos and basically take it one step further at a time, gaining more and more understanding, exploring more and more details of your data landscape and starting to see relations between them" (p. 145). 
• After coding the texts/data, review Chapter 6 for ideas about how to query the data. The last half of Chapter 6 is very procedural and skills-based and will make more sense to a reader who already has a list of codes and has started to conceptualize those codes. Friese reviews the query tool, the co-occurrence explorer, and the Codes-Primary-Documents-Table, which can be used to find relationships and patterns. At this stage of analysis, the research question memos can be used to keep record of queries and results (as pictured on p. 144). Further, the memos, if set up correctly and linked to quotations, can be used to generate output that serves as "building blocks" for the findings chapter. 
• Consider incorporating ATLAS.ti into the dissertation defense. ATLAS can support the presentation of findings in a number of ways listed on pp. 219-221.

Methodologically, this passage from page 1 of Chapter 6 says it all:

A lot of the analysis happens as you write, not by clicking on some buttons and outputting some results. You need to look at what the software retrieves, read through it and write it up in your own words in order to gain an understanding of what is happening in the data. Most of the time insights need to be worked at and are not revealed to you immediately by looking at the results. Simply seeing that there are, say, 10 quotations is not enough; numbers are sometimes useful, they hint that there might be something interesting there, but the important step is to take a closer look and to see what’s behind them. (p. 133)  

Technological advice and ah-ha's:  

• When the Code Manager is open, navigate through a long list of codes by pointing the cursor on the Code Manager, and typing the first few letters of the desired code. If I ever amass the average number of 120-200 codes, this practice will be useful.  
• When preparing transcripts in a word processor, save them as rich text (.rtf files) so that ATLAS does not need to convert them. Rich text is the standard file format in ATLAS, not .doc or docx. Typically, after transcribing in InqScribe, I format transcripts in MS Word. It would not be difficult for me to Save As .rtf. This is something to consider; although, in the past I have used .doc files in ATLAS and did not incur any problems. 
• Many suggestions in Chapter 2 for formatting transcripts! I have not yet loaded any transcripts into my dissertation HU, and Friese's formatting guidelines should be easy to implement. These include marking speakers with unique identifiers that do not appear in the text (i.e. "INT" for "interviewer," and so on), double spacing between turns, and breaking up long turns with empty lines. Some of these preparations are to facilitate use of the ATLAS automatic coding tool, which I don't know if I even want or need to use, but Friese suggests getting into the habit anyway.  
• Learn the ATLAS.ti file protocol. Friese calls it "the biggest hurdle" in ATLAS project management, and on pp. 36-37 she thoroughly demystifies this aspect of the program. It's a matter of understanding "external references," a system of document storage and retrieval that prevents a project file from becoming too big and unwieldy. I now have a firmer understanding of how the one folder for all data rule works and why I should follow this rule. More importantly, the name of each data file should be analytically helpful, not generic like "transcript1," "transcript2," and so on. Choose a file-naming system early. It will add transparency and efficiency to the project. The data files (primary documents) can be sorted in the document manager, providing a quick glimpse of your sampling. 

One immensely telling detail: the total number of menus and submenus in ATLAS.ti equaled 443 at the time Friese published her handbook. The point is that working with ATLAS can be highly individualized. The interface is designed to provide users with multiple options for performing the same function. Thus, through practice, the user develops his or her own preferred workflow and routines.  (I like this as it basically sums up my whole approach to teaching and learning with technology.)

References 
Friese, S. (2012). Qualitative data analysis with ATLAS.ti. (eBook.). Los Angeles: SAGE Publications Ltd.

Piantanida, M., & Garman, N. B. (2009). The qualitative dissertation: A guide for students and faculty. (2nd ed., Kindle version.). Thousand Oaks, CA: Corwin.

Notes on ATLAS.ti from Friese (2012)

New transcription routines

I have been preparing transcripts from virtual class recordings and interactive/co-generative interviews for my dissertation project.  As I have done in the past with other projects, I am creating verbatim transcripts in a standardized format to “aid the handling, comparison, and sharing of language data” (Lapadat & Lindsay, 1999, p. 70). 

My primary tool for transcription, is InqScribe software, which has many features that I have come to rely on, especially the control for playback speed and the embedded timecodes that enable me to jump to specific locations in my audio files. However, one aspect I do not like about InqScribe is its output. I must copy and paste the raw transcript into a Word document, where I bold, italicize, and perform other formatting functions as needed.

My journey down the transcription road (more like grueling death march) happened to coincide with my recent venture into the case study literature.  For this reason, I closely attended to what the case study methodologists had to say about interviewing and transcribing, and I wondered if my readings would influence my process of data collection and handling.  In fact, they have, and I have added a few "new routines" to my process. 

Gisting

Somewhere along the way, I heard about this practice of capturing the "gist" of researcher-participant exchanges  -- maybe in ethnography? I can't remember. 

Nevertheless, to gist an interview or conversation, the researcher writes down everything that he or she can recall as soon as possible following the encounter. 

Gisting is essentially a good habit of all fieldwork. We were taught in intro to qualitative research to immediately review notes and reflect in writing after the fact. Sadly, it is not a habit I ever developed. Typically, when I am in the field, I am running hither and yon, usually because I have overscheduled the day and am late to the next engagement. As qualitative researchers, this may be our most debilitating mistake in the interview process. I realize this now.

In the short passage on interviewing techniques in The Art of Case Study Research, Stake says that the single most important thing the case researcher can do is "insist on ample time and space immediately following the interview to prepare the facsimile and interpretive commentary" (p. 66). In fact, Stake argues persuasively against any transcription, saying the "facsimile" is all anyone really wants to see. Participants are likely to be put off by the length of a typical transcript and "the inelegance of their own sentences" (p. 66). 

Briefly, Stake's interview tips are:

1. rather than tape record or write furious notes, listen and make a few notes
2. ask for clarification
3. after the interview (within a few hours) reconstruct the account in your own words
4. submit the facsimile to the participant for accuracy and stylistic improvement

I think I have the first two steps down pat, but I am not prepared to give up recording. I am a fairly decent listener and always take a few notes, and my notes work like markers for later navigation through the recording. On the downside of that, I have never mastered the art of discreetly noting the passage of time or approximate minutes transpired during an interview. (It seems rude to glance at my cell phone or clock during the interview. I don't want to wrongly cue the speaker that we are out of time.)

Insofar as reconstructing the account immediately after the interview (Step 3), this is something I would like to do better. What I have been doing based on Stake's advice is to deliberately stop after every 10 or so minutes of transcription and write and reflect on key ideas, impressions, and insights that accumulated in my brain. This has become my version of gisting the interview, and I'm very pleased with the results! When I finally finish transcribing -- and, unfortunately, this does drag out the transcription process even more so than ever -- I have a nice narrative summary of the interview with my interpretive commentary woven throughout. I insert timestamps as critical reference markers, should I or a participant want to go into the original transcript to read the exact words. 

Maintaining trustworthiness

I am starting to include the gist of the interview with the verbatim transcript when I submit it to the interviewee as part of my new member checking routine. So far, I have received one reaction from an interviewee, and it was wholly positive.  Stake says to expect this. Member checking is a necessary step, even when participants don't respond. If the researcher is anxious his or her facts or interpretations are off-base, it is up to the researcher to probe more deeply.

Member checks are one of several strategies identified by Yamagata-Lynch (2010) for maintaining trustworthiness in activity theoretical studies (the method of my dissertation). Other strategies include prolonged engagement, persistent observation, and triangulation. I feel relatively confident about upholding these standards during my study, but I have never purposefully applied member checks before. This is new territory for me.

While Yin does not use the term "member check," he says a major procedure in doing a case study is inviting participants and informants to view drafts of the report. Yin fixates on "corroboration" and "validity" as the main reason for member checks. He says participants may disagree with interpretation but should validate facts of the case. The case study is not complete until the researcher resolves these disagreements. 

I would argue the purpose is to check interpretations for "coherence," per Piantanida and Garman (2009). Yin allows for this possibility, too. In cases where "validity" or "objective truth" may not be the point, the process of member checking is helpful for identifying different participant perspectives, which will be portrayed in the final case study report. And, Yin says, the researcher ultimately has the "discretionary option" and does not have to accommodate participants' reinterpretations (Yin, Ch. 6, "Reviewing the Draft Case Study: A Validating Procedure," para 3).

As with gisting, member checking adds time. Also, Yin advises, participant-reviewers may use the opportunity to open a fresh line of dialogue about the study. The researcher must plan ahead and budget time accordingly.

I have only started to do member checks, but to expedite the process, I am finding the Evernote notetaking app to be especially useful.

It was not my plan initially to use Evernote so deliberately in my data collection process. In January, during my very first interview with a teacher-learner, my digital audio recorder was making me nervous, indicating it was low on both memory and battery life. I didn't have a back-up recording device -- or so I thought! I remembered I could use the built--in recording feature of Evernote. I opened my iPad, opened Evernote, created a note using the participant's pseudonym as a title, and clicked on the Record Audio button in the top menu. With tablet devices becoming almost ever-present in daily life, it was easy to place the iPad unobtrusively on the table between me and the interviewee while we talked. It may even have been less distracting than the microrecorder.

Later, when I prepared to send the transcript with gisting to the participant, it occurred to me that I could use the Share tool in Evernote and the original audio file would embed in an email. So, I copied and pasted the gist and the transcript from Word into the Evernote note with audio and sent it. 

I am hopeful that the member checking routine will enrich my study. According to Yin, "Often, the opportunity to review the draft also produces further evidence, as the informants and participants may remember new materials that they had forgotten during the initial data collection period" (Yin, Ch. 6, "Reviewing the Draft Case Study: A Validating Procedure," para 2).

In the event this does not happen (and it may not), I still wonder if this could not serve as the basic premise of eventual follow-up interviews, asking participants to react to the short gist of their first interview (as opposed to, "What did you think of the transcript?")? 

Group update

Last week our group workshopped Brian's Chapter 3. I learned a lot from reading Brian's work and got some ideas about interviewing as well as a lead on another possible resource, ‪Learning From Strangers: The Art and Method of Qualitative Interview Studies by Weiss. The group also selected a reading for the week of  Feb. 24, Edwards (1998). We plan to discuss the reading and workshop some of Nalani's early coding efforts from a pilot study she conducted.

References

Lapadat, J. C., & Lindsay, A. C. (1999). Transcription in research and practice: From standardization of technique to interpretive positionings. Qualitative Inquiry, 5(1), 64 –86. doi:10.1177/107780049900500104

Piantanida, M., & Garman, N. B. (2009). The qualitative dissertation: A guide for students and faculty (2nd ed., Kindle version.). Thousand Oaks, CA: Corwin.

Stake, R. E. (1995). The art of case study research. Thousand Oaks, CA: SAGE Publications.

Yamagata-Lynch, L. C. (2010). Activity systems analysis methods: Understanding complex learning environments. New York: Springer.

Yin, R. K. (2008). Case study research: Design and methods (4th ed., Kindle version.). Los Angeles: SAGE.

New transcription routines