ACM Web Science Conference 2012 Workshop
Evanston, IL, 21 June 2012

Judit Bar-Ilan
Department of Information Science, Bar-Ilan University, Ramat Gan, Israel


The first volume of the Journal of the American Society for Information Science and Technology (JASIST) was published in 2001. JASIST continues JASIS (the Journal of the American Society of Information Science (1970-2000) and the American Documentation (1950-1969). JASIST is one of the leading information and library science journals. In this work we considered all articles published by JASIST between 2001 and 2011. We collected citation data from three sources: Web of Science, Scopus and Google Scholar and compared them with readership data from Mendeley.

Data were collected in April 2012, thus the partial data retrieved for 2012 publications were excluded from the analysis. Data were collected from the Web of Science (WOS), where articles, reviews and proceedings papers were selected (editorials, book reviews, letters, biographical items and bibliographies were excluded), and a total of 1637 items were retrieved. A similar process at Scopus resulted in the retrieval of 1628 items. An extensive search and data cleansing in Google Scholar, using Publish or Perish, resulted in five additional items. Data were also collected from Microsoft Academic Search, but are not included in this report. This process resulted in the identifications of 1651 items categorized as articles and/or reviews published by JASIST between 2001 and 2011. Items labeled as “proceedings papers or conference papers” are articles published in JASIST that are based on earlier published papers in proceedings, and thus can be considered as articles as well (31 such items were identified in WOS and 23 in Scopus).

All 1651 articles were manually searched on Mendeley. There were several reasons why we chose not utilize the Mendeley API (see also Bar-Ilan, Haustein, Peters, Priem, Shema & Terliesner, 2012). In case of multiple records for the given article, only one of them is retrieved by the API, and this record is not necessarily the record with the largest number of readers (in any case we were interested in comprehensive readership counts). Title searches are “tricky”, because the Mendeley search engine does not handle well special characters like hyphens, question marks, colons, and non-standard ASCII characters like ø or é. Title searches could not be replaced by DOI searches because not all the records in Mendeley have DOIs and even if they do, sometimes the DOI is not correct. And finally there were mistakes introduced by the readers, e.g. partial or slightly incorrect titles or incorrect identification of the source (JASIST). Such “suspected” items were carefully checked and were only included when we were quite confident, that the record refers to the specific item published by JASIST (e.g. correct DOI, exact abstract or the correct preview page). In order to be as comprehensive as possible we used title searches, as phrases. In most cases the subtitles were excluded from the search phrase and special characters were avoided. When necessary the name of one of the authors (usually the first author) was added to the search in order to reduce the number of retrieved records. The search results were scanned, duplicates (called “matching titles”) were expanded and the number of readers in all matching results were summed and recorded. Although probably some misspelled or misidentified items were missed in this process, we took reasonable steps to collect comprehensive readership results from Mendeley.

All, but 46 items were located in Mendeley, i.e. Mendeley covers 97.2% of the JASIST articles published between 2001 and 2011. Bar-Ilan et al. (2012) report 82% coverage of Mendeley out of more than 1,100 articles published by scientometric researchers, and Li, Thelwall and Giustini (2012) found that in 2010, Mendeley’s coverage for articles published in 2007 in Nature and Nature was 94% and 93% respectively. It should be noted that all articles published in Nature and Science were cited at least twice according to WOS, and the most cited items were cited more than 1,000 times by then. In comparison, the most highly cited JASIST article in our dataset was cited 289 times in WOS and 398 times in Scopus, and 17.2% , 12.4% and 7.6% of the items in the dataset were reported as uncited by WOS, Scopus and GS respectively. The coverage of Mendeley is extremely impressive, especially since the records are not created through systematic indexing as in the other databases, but by the users.

Table 1 displays the ten most-read articles. The Spearman correlations between Mendeley readership counts and WOS, Scopus and Google Scholar (.458, .502 and .519 respectively) are significant and of medium strength. The significance strength is similar to what was reported in previous studies, and seems to indicate that readership counts measure something different from citations. Priem, Piwowar and Hemminger (2012) found that there is higher correlation between Mendeley readership counts and downloads of PLoS articles than between these readership counts and citation counts on WOS and Scopus. Although this is expected it may also indicate that readership counts have more to do with reading than with citation, which is an active process. This is similar to visiting a Web site vs. linking to it, as reflected in the differences between PageRank and visiting popularity counts.

Author(s) Title PY Reader count
Liben-Nowell, D;
Kleinberg, J
The link-prediction
problem for social networks
2007 322
Jansen, BJ; Zhang,
MM; Sobel, K; Chowdury, A
Twitter Power:
Tweets as Electronic Word of Mouth
2009 280
Bates, MJ Fundamental forms of
2006 81
Stvilia, B; Twidale, MB; Smith, LC; Gasser, L Information quality
work organization in Wikipedia
2008 73
Chen, CM CiteSpace II:
Detecting and visualizing emerging trends and transient patterns in
scientific literature
2006 71
Zins, C Conceptual
approaches for defining data, information, and knowledge
2007 70
Rieh, SY Judgment of
information quality and cognitive authority in the Web
2002 69
Borlund, P The concept of
relevance in IR
2003 64
Saracevic, T Relevance: A review
of the literature and a framework for thinking on the notion in information
science. Part II: Nature and manifestations of relevance
2007 61
Metzger, MJ Making sense of credibility
on the web: Models for evaluating online information and recommendations for
future research
2007 60

We conclude that Mendeley is a major altmetric data source, however further studies are needed to understand the meaning of Mendeley readership counts.


Bar-Ilan, J., Haustein, S., Peters, I., Priem, J., Shema, H., & Terliesner, J. (2012). Beyond citations: Scholars’ visibility on the social Web. To be presented at STI2012.

Li, X., Thelwall, M., & Giustini, D. (2012). Validating online reference managers for scholarly impact measurement. Scientometrics, 91(2), 461-471.

Priem, J., Piwowar, H., & Hemminger, B. H., (2012). Altmetrics in the wild: Using social media to explore scholarly impact. arXiv:1203.4745v1

Creative Commons License

This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 Unported License.


  1. Judit Bar-Ilan
    Posted June 16, 2012 at 7:14 am | Permalink

    Erratum: The readership count of 322 for the Liben-Nowell & Kleinberg paper resulted from combining the readership counts for the JASIST article and the CIKM proceedings paper with the same title. The actual readership count should be 186, and then the top-read JASIST paper is “Twitter power” by Jansen et al.

  2. loay
    Posted March 19, 2014 at 1:56 pm | Permalink

    What is the impact factor of JASIST?

  3. Judit Bar-Ilan
    Posted March 30, 2014 at 11:44 am | Permalink

    The 2012 impact factor of JASIST is 2.005

7 Trackbacks

  1. […] with three papers on the list relating to ENCODE. Company co-founder and CEO Victor Henning says several studies have examined and confirmed a strong correlation between Mendeley readership statistics and […]

  2. By What were the top papers of 2012 on social media? on December 26, 2012 at 7:12 am

    […] on the list relating to ENCODE. Company co-founder and chief executive Victor Henning says that several studies have examined and confirmed a strong correlation between Mendeley readership statistics and […]

  3. […] papers on a list relating to ENCODE. Company co-founder and arch executive Victor Henning says that several studies have examined and reliable a clever association between Mendeley readership statistics and […]

  4. […] Bar-Ilan also presented a study of Mendeley’s coverage of JASIST, a premier information science journal. Not only did Mendeley’s coverage of that journal […]

  5. By Entrevista com William Gunn | SciELO em Perspectiva on November 7, 2013 at 12:38 pm

    […] JASIST@mendeley […]

  6. By Interview with William Gunn | SciELO in Perspective on November 7, 2013 at 12:41 pm

    […] JASIST@mendeley […]

  7. By Entrevista con William Gunn | SciELO en Perspectiva on November 7, 2013 at 12:46 pm

    […] JASIST@mendeley […]

Post a Comment

Your email is never shared. Required fields are marked *