Frequently advanced quarrels (FAQ) with re-using qualitative data

This section attempts to address some of the most typical objections mounted against the re-use of qualitative data. In July 2001, Qualidata hosted a conference, "Celebrating classic sociology: pioneers of contemporary British qualitative research." Several of the participants reflected on the opportunities - and challenges - of secondary analysis of qualitative data. The talks (and other papers cited below) have been drawn on to address the most common challenges to re-using data. The objections and responses presented here provide overviews only of arguments that raise fundamental questions of epistemology and philosophy of social research. The cited papers provide deeper treatments.

I don't have the context to use someone else's data

Re-use is incompatible with reflexivity

Isn't being close to data always better?

I have to collect primary data

New data are always better

The data have been used up

One can't re-use data for 'audit-like' replication

References

I don't have the context to use someone else's data

Probably the most frequently raised objection has to do with secondary researchers lacking the context of the original investigators. They were not there to get to know respondents as individuals, see interviews settings, observe body language, hear intonation, and absorb huge quantities of background material prior to entering the field.

One somewhat facile response is that for several research purposes, deep contextual knowledge is not essential. For example, archived data can be used to enhance descriptions of primary data, they can be used for methodological insights (to model an interview schedule), and they can be used for teaching. See: Re-using qualitative data

Somewhat more controversially, it has to be said that the unique, deep, profound, reflexive relationship between scholar and subject matter is, at times, mythologised. Especially in large projects, it is rare for one researcher to be the sole custodian of all context. In light of this, the question about re-use has to be reframed: if shared context had to be built amongst a team of researchers in the original project, can an additional researcher share 'enough' of that context at a later date?

Yes and no. It would never be argued that 'the same' context can ever be reconstructed. The practical question is: can sufficient context be constructed to satisfy the purposes of the secondary researcher? That depends, of course, with the most salient factors being how well documented the original project was and the specifics of the secondary enquiry. Good documentation (e.g. research diaries, meeting notes, records of major project decisions) and a good fit between archival data and secondary research questions do much to enhance the usefulness of secondary data. The opportunities are further enhanced if the secondary researcher has access to at least one member of the original team.

Re-use is incompatible with reflexivity

"Any attempt to go further than [historical and methodological exploration] is incompatible with an interpretive and reflexive methodology" (Mauthner, et al., 1998). Even where there is agreement about the interpretive and reflexive nature of qualitative work, opinions differ on the implications for archiving. One position, taken by Mauthner, et al., argues that original context cannot be recovered by a secondary researcher, and it is essential for the evaluation of qualitative research. Thus, secondary research for any generative purpose is eliminated. A less restrictive approach is taken by Fielding (2001) who sees that context is a challenge faced by both primary and secondary researchers. The particulars may differ, but both have the responsibility to "monitor the effects of reflexivity" (Fielding, 2001).

A truly reflexive approach would honour the uniqueness of every project, so that secondary analysis should be evaluated on its own merits, which would include, no doubt, the original project context, but also expand the understanding of context to include the contemporary situation as well.

In summary, it is incontestable that the original researcher will always have the most context, especially in the pure case of a single researcher having personally collected all the data. But eventually, the objective of research and analysis is to convince colleagues, readers, and others of one's arguments, even though they do not share (and indeed can never share) the original context.

Isn't being close to data always better?

Isn't being as close as possible to my data all the time the best way to do research? Actually, no. Nearly all official methods handbooks and certainly most lived experience confirms that the best research combines detailed and intimate knowledge of one's data with distance and perspective. Usually, these are linked to phases of the project, closer at the start, more distant in the period of final analysis. The trick, of course, is finding the right balance. There is no more one 'correct' distance from data than there is from a painting. It depends on what one is looking for: brushstrokes or composition? If secondary researchers have sufficient context, then the fresh perspective they bring to the data can even be an advantage. In the situation when secondary researchers are asking new questions of the data, they are exempt from the criticism that they are sampling in such a way as to support their existing beliefs or hypotheses.

I have to collect primary data

I have to collect primary data because I'm a: student, world famous researcher, or anything in-between. This is a real and legitimate issue. Many, if not most, MA and PhD programmes require students to collect primary data. It should not be any other way; there is no substitute for collecting and analysing your own primary data. But archived data can still be invaluable to students (and even famous researchers) by providing: background cases of similar projects, insider details of methodology (how were coding schemes arrived at?), the possibility of enhancing generalisability by expanding sample size or by constructing illuminating historical or comparative datasets (Hammersley, 1997).

New data are always better

We would have to disagree. There are a number of situations where archived data might actually be preferred:

• inaccessible populations (i.e. the very rich, the very remote) - for any group that is difficult to access, archived data may be the only source available

• vulnerable populations (most typically, these might be the old or infirm) - the stress of undergoing repeated interviews argues for extracting as many research insights as possible without imposing further burdens

• over-researched populations - as with the previous case, archived data can spare informants the burden of repeated investigations

The data have been used up

It is a common misconception that archived data are not worth re-analysing. The best people to answer here are those who have either re-used data or let their own material be re-used.

"Another thing is that now we spend more time looking at the apparently unproductive interviews, looking at what they did not say. Some of the ones which at first look to having nothing in them, if you sit down and brainstorm for a while, you get a lot more out of them" (Marsden, 2001).

"Even though you have put them in the cabinet for 20 years, you can come back to this store of material and re-think issues...And, of course, coming now to the final bit, there is no reason why other people cannot do this" (Brown, 2001).

"At least five times as many major publications came out of it [collection of Edwardian interviews] as the original research team could have produced. That has been an enormous source of satisfaction to me" (Thompson, 2001).

One can't re-use data for 'audit-like' replication

We couldn't agree more.

References

Brown, G. (2001) 'Qualidata and the Preservation of Meaning', transcript of conference talk, Celebrating classic sociology: pioneers of contemporary British qualitative research, University of Essex, 5-6 July.


Corti, L. (2000) 'Progress and Problems of Preserving and Providing Access to Qualitative Data for Social Research - The International Picture of an Emerging Culture', Forum Qualitative Sozialforschung / Forum: Qualitative Social Research [Online Journal], 1(3). Available at: www.qualitative-research.net/fqs-texte/3-00/3-00corti-e.htm.


Corti, L. and Thompson, P. (2003) 'The secondary analysis of archived qualitative data' in C. Seale et al. (eds) Qualitative Research Practice, Sage: London.


Fielding, N. (2001) 'Automating the Ineffable: getting the maximum from archived data', transcript of conference talk, Celebrating classic sociology: pioneers of contemporary British qualitative research, University of Essex, 5-6 July.


Hammersley, M. (1997) 'Qualitative data archiving: some reflections on its prospects and problems', Sociology 31(1): 131-142.


Heaton, J. (2000) 'Secondary analysis of qualitative data: a review of the literature', Social Policy Research Unit Report, Ref: R000222918, University of York, August.


Marsden, D. (2001) 'The changing experience of researching family and intimate relationships', transcript of conference talk, Celebrating classic sociology: pioneers of contemporary British qualitative research, University of Essex, 5-6 July.


Mauthner, N.S., Parry, O. and Backett-Milburn, K. (1998) 'The data are out there, or are they? Implications for archiving and revisiting qualitative data', Sociology 32(4): 733-745.


Thompson, P. (2001) 'Pioneering the life story method', transcript of conference talk, Celebrating classic sociology: pioneers of contemporary British qualitative research, University of Essex, 5-6 July.