Bibliometrics and the Culture of Open Access [v0.1]

This is version 0.1 (revised 18 May 2001) of an abstract to be presented at altmetrics11.

Clifford Tatum
Paul Wouters
Centre for Science and Technology Studies
Leiden University, the Netherlands

The possibility of an Open Access Citation Advantage (OACA) continues to inspire researchers and advocates. The question of a citation advantage has important implications not only for Open Access as an alternative scholarly publishing model, but also for the careers of individual researchers. In both cases, increased citation translates into increased impact. However, in the efforts to improve upon studies of OACA, increased methodological rigor seems at the same time to remove much of the cultural context of openness. As factors of Open Access such as self-selection, collaboration proximity, and publication of draft texts, are found to influence citation, they are often eliminated from subsequent studies. With a conception of Open Access defined so narrowly, many important dimensions are rendered invisible. In an increasingly polarized debate[1], improvements in the validity of data and methods may have come at the expense of gains in understanding. What do the results from these analyses tell us about the complexities of publishing models and about research trajectories of individual researchers?

Whereas early research exhorted the virtues of Open Access publishing based on results that revealed a citation advantage from publishing in open access journals, more recent studies claim much of this early research was flawed. Methodological refinements since then have included parsing different forms of open access, such as the crucial distinction between Open Access journals (Gold OA) and author self-archiving (Green OA) (e.g. Harnad and Brody, 2004) and the development of new techniques that strive to isolate open access from influencing factors (e.g. Davis, et al. 2008, Davis 2010). In the new techniques category several factors have been shown to either influence citation behavior directly or obscure the ability to measure an open access citation advantage. These factors include author self-selected deposit of early draft articles in a repository (Kurtz M., et al., 2007), early exposure of draft versions (Moed 2007), the availability of an article at multiple open access points (Xia, et. al 2010), and the impact of physical proximity of collaborators (Lee, et al. 2010). Authors of these studies typically frame the condition or factor as a bias that artificially inflates the number of citations. However, this systematic elimination of biases for the purpose of isolating OACA assumes that the practices can be as unchanged once removed from cultural context.

Missing from techniques such as the the randomized control approach are not only the factors specifically eliminated to control for bias, but also a host of cultural and technological scaffolding that has emerged over time and through multiple evolutionary iterations that together create particular configurations of openness. By classifying open access practices, such as self selection or early publication of drafts, as “confounding effects” the authors of a recent study cite previous research that shows increased citation because of articles that are selectively chosen to be published openly or when pre-print manuscripts are published early (Davis, et al. 2008:1). But these open access practices, in addition to being a performance of particular cultural values, are embedded within a larger social and cultural milieu where publication and citation are significant criteria in the allocation of academic reward. This sort of classification problem is explained by Dehue as “the choice of categories [that] depends on human decisions and cultural conventions” (Dehue 2002:86).

Also not present in the debate of OACA is the view that open access contributes to intertextual scholarly discourse. When in compliance with content interoperability standards, open access articles contribute to the possibility of making explicit intertextual references through hyperlinking (Mitra 2006) from one text to another across different communication platforms. Through hyperlinking, documents, collections of documents, and related audio and visual resources are structured across the web (Halavais 2008, 43). This “textured connectivity” of scholarly discourse is created both by human and machine (such as databases) actors (Beaulieu and Simakova 2006). Unlike traditional citations in printed text, the immediacy of hyperlinks facilitates the construction of intertextual discourses, which are dynamic in both production and consumption. From this vantage point, citations by themselves would appear to be insufficient as an indicator of impact. A more comprehensive definition of a ‘citation’ would need to include its role in discursive practices on the web.

Moreover, when taking this more inclusive view of scholarly discourse and citation practices, citation advantage becomes only one dimension in a dynamic system. In a related area of interest, ethnographic approaches used to study Open Source software development revealed interesting insights about the implications of openness in collaborative practice. Kelty (2008) identifies a crucial characteristic of Open Source communities, whereby software, which in this case is the object and medium of openness, “is a kind of experimental system: its practices can be adopted, adapted, and modulated in new contexts and new places, but it is one whose rules are collectively determined and frequently modified” (Kelty, 2008:98). Open Source software development is, of course, a different community of practice, where reward systems vary in pragmatics, if not their ideals.

As the Internet and the Web are essential prerequisites for the possibility of Open Access, we propose a framework for Open Access that considers the interface and infrastructure of both social and technological configurations of openness. In this framework, the interface of openness is conceived as the place where users act upon digital media to communicate with others. Correspondingly, the possibility for openness is conceived as a technological infrastructure, which is the result of social agents selecting, configuring, and implementing associated communication resources. As such, analytical focus is aimed at individual acts of openness framed as the result of interaction between human agency, social structure in the form of situated practices, and material structure in the form of digital media. Following Orlikowski, we view technology as “both an enabler of, and a constraint on, human action” (Orlikowski 1992, 25). The conceptual framing of openness as contingent upon interface and infrastructure recognizes the dual role of technology in facilitating both agency and structure (Wouters et al. 2008).


1. See for example, The Open Citation Project, where ongoing discourse is manifest trough a well-maintained bibliography and active discussion through enabled comments. (Accessed 05 February 2011)


Beaulieu, Anne, and Elena Simakova. 2006. “Textured connectivity: An ethnographic approach to understanding the timescape of hyperlinks.” Cybermetrics 10 (1).

Davis, Philip M. 2010. “Does Open Access Lead to Increased Readership and Citations? A Randomized Controlled Trial of Articles Published in APS Journals.” The Physiologist, 53 (6), December 2010

Davis, Philip M, Bruce V Lewenstein, Daniel H Simon, James G Booth, and Mathew J L Connolly. 2008. “Open access publishing, article downloads, and citations: randomised controlled trial.” BMJ 337(jul31 1): a568-a568.

Dehue, Trudy. 2002. “A Dutch treat: randomized controlled experimentation and the case of heroin-maintenance in the Netherlands.” History of the Human Sciences 15(2): 75 -98.

Halavais, Alexander. 2008. “The hyperlink as organizing principle.” In The Hyperlinked Society: Questioning Connections in the Digital Age, eds. Joseph Turow and Lokman Tsui, 39-55. Ann Arbor: University of Michigan Press.

Harnad, Stevan and Tim Brody. 2004. “Comparing the Impact of Open Access (OA) vs. Non- OA Articles in the Same Journals.” D-Lib Magazine, Vol. 10 No. 6, June 2004

Kelty, Christopher M. 2008. Two Bits: The Cultural Significance of Free Software. Duke University Press Books.

Kurtz, Michael J et al. 2005. “The effect of use and access on citations.” Information Processing and Management: an International Journal 41: 1395–1402.

Lee, Kyungjoon, John S. Brownstein, Richard G. Mills, and Isaac S. Kohane. 2010. “Does Collocation Inform the Impact of Collaboration?” PLoS ONE 5(12): e14279.

Mitra, Ananda. 2006. “Characteristics of the WWW Text: Tracing Discursive Strategies.” Journal of Computer-Mediated Communication 5(1): 0-0.

Moed, Henk F. 2007. “The effect of ‘open access’ on citation impact: An analysis of ArXiv’s condensed matter section.” Journal of the American Society for Information Science and Technology 58: 2047–2054.

Orlikowski, Wanda. 1992. “The duality of technology: rethinking the concept of technology in organizations.” Organization Science 3 (3): 398-427.

Wouters, Paul, Katie Vann, Andrea Scharnhorst, Matt Ratto, Iina Hellsten, Jenny Fry, and Anne Beaulieu. 2008. “Messy shapes of knowledge – STS explores informatization, new media, and academic work. The Virtual Knowledge Studio.” In Handbook of science and technology studies, ed. Edward Hackett, Olga Amsterdamska, Michael Lynch, and Judy
Wajcman, 319-351. Cambridge, MA: MIT Press.

Xia, Jingfeng, Rebekah Lynette Myers, and Sara Kay Wilhoite. 2011. “Multiple open access availability and citation impact.” Journal of Information Science 37(1): 19 -28



  1. Posted May 2, 2011 at 10:28 am | Permalink

    […] human agency, social structure in the form of situated practices […]. I could not agree more. What do you mean by “situated practices”? I think I understand but would like to make sure. Many thanks to you to both. Aalam

  2. Clifford Tatum
    Posted May 5, 2011 at 5:46 pm | Permalink

    Aalam, thanks for the comment and question.

    In the sentence related to your question, situated practices are identified as a form of social structure. This refers to collective practices (human actions) that are shaped by local context and at the same time expressions of norms in that context. This analytical construction is influenced by Orlikowski’s work (see references). In the present case then, if Open Access practices emerge in an academic context, they would be influenced by their situatedness among existing academic practices, in a field or discipline, and within a particular institutional setting.

  3. Posted May 6, 2011 at 11:56 am | Permalink

    Clifford, thank you. Looking forward to further discussing this very point with you in Coblenz. All best. Aalam

  4. Posted May 17, 2011 at 12:21 am | Permalink


    Tatum & Wouters make the most common mistake regarding Open Access (OA):

    OA does not mean solely — or even primarily — publishing in OA journals (“Gold OA”). More articles are made OA by publishing them in a subscription journal and making the articles OA by self-archiving them in the author’s institutional repository (or a central repository) (“Green OA”) immediately upon acceptance for publication.

    Only 20% of journals are Gold OA (and mostly not the top journals) but over 60% of journals have endorsed Green OA self-archiving by their authors (and that includes most of the top journals).

    All that’s needed is for authors’ institutions and funders to mandate Green OA self-archiving (as Harvard, MIT, NIH and nearly 200 hundred other institutions and funders have done).

    That’s why the OA citation advantage (OACA) is important: it provides an incentive to mandate Green OA self-archiving.

    Most of the findings on the OACA (including the Harnad & Brody 2004 study misdescribed as being as being based on Gold OA) are based on Green OA, not Gold OA (though the findings of course also apply to Gold OA), because only Green OA allows within-journal comparisons of OA and non-OA article citation impact. Between-journal comparisons are comparing apples vs oranges.

    Tatum & Wouters do not seem to have understood the Gargouri et al (2010) findings comparing mandatory vs self-selected (Green) OA self-archiving. The fact that the OACA was significant and of the same size for both means that the OACA is not an artifact of author self-selection bias, and this finding was based on a far larger and broader sample size than the studies that failed to find an OACA.

    All the good things Tatum & Wouters envision will come to pass. But first we need OA. And for that, more careful attention will have to be paid to the facts and findings. I hope the misunderstandings will be cleared up in the revised version for altmetrics11.

  5. Clifford Tatum
    Posted May 18, 2011 at 9:16 am | Permalink


    Hi Stevan, Thank you for the constructive feedback! The errors noted have been corrected (all in the second paragraph) and the other points will be elaborated in the full text.

    In your last paragraph you state, “All the good things Tatum & Wouters envision will come to pass.” I take this to be tacit support of our proposed framework (Is this a correct interpretation?).

    To elaborate a bit, some factors identified as biasing OACA, are in my view indicators of possible diffusion of openness values into other aspects scholarly practice. For example, posting pre-publication drafts on or would seem to actually extend the formal definition of OA to include ‘informal’ scholarly communication in addition to articles that have already been peer reviewed and are perhaps on their way to formal publication.

    If publication of early drafts occurs in significant numbers, it would seem to be an indication that the practice of OA does not stop at the boundaries of ‘formal’ scholarly communication. In the context of our argument, choosing to exclude some practices (such as depositing early drafts in a repository) while focusing on others (such as depositing peer reviewed, finished articles in a repository) and still calling it OA, seems at best to be an imprecise use of the term open access.


Post a Comment

Your email is never shared. Required fields are marked *