Do Chinese and American contributions
in top journals have an equal citation
Jielan DING1,2 & Ronald ROUSSEAU3,4
1National Science Library, Chinese Academy of Sciences, Beijing 100190, China
2University of Chinese Academy of Sciences, Beijing 100049, China
3Information and Library Science (IBW), University of Antwerp (UA), Antwerp B-2000, Belgium
4Department of Mathematics, KU Leuven, Leuven B-3000, Belgium
):1-10, Accepted: Jun. 16, 2015
This work is supported by the National Natural Science Foundation of China (Grant No.: 71173185).
R. Rousseau (email@example.com, corresponding author) proposed the research idea,
planned and designed the outline, wrote the first draft and revised the paper. J.L. Ding (dingjielan@
mail.las.ac.cn) performed data analysis, joined discussion of the findings and contributed to
writing the paper.
Purpose: We want to contribute to the evaluation of Chinese research, focusing on contributions in top journals.
Design/methodology/approach: Using a Mann-Whitney test we investigate if contributions in Nature, Science or the Proceedings of the National Academy of Sciences of the United States of America (PNAS) by Chinese or American authors only, i.e. articles for which all authors have a Chinese or an American address, have a different citation potential.
Findings: There is no reason to state that Chinese and American contributions in these top journals have a different citation potential.
Research limitations: Because of the small numbers involved we were not able to make a distinction between publications in Nature, Science or the Proceedings of the National Academy of Sciences of the United States of America.
Practical implications: These results suggest that the better Chinese research results are of a similar level as those by American colleagues.
Originality/value: It is well-known that the number of citations per publication by Chinese authors is still lagging with respect to leading scientific nations and in particular compared with the USA. We have shown that this difference does not necessarily hold in contributions in Nature, Science or the Proceedings of the National Academy of Sciences of the United States of America.
Although it is common knowledge that China’s increase in publications over the
latest decade is astonishing
, it is also well-known that the number of citations per
publication is still lagging with respect to leading scientific nations and in particular
compared with the USA[2,3]
. Yet, numbers of citations are averages over all
publications, at least as included in an international database. One may expect that
citations per paper differ according to the level or the type of the journal. For
instance, the intellectual requirements for acceptance of submissions in Nature,
or the Proceedings of the National Academy of Sciences of the United States
in short) can hardly be compared with those for most national
journals. These three journals are the ones we will focus on. Just as a short-hand,
they will be referred to as top journals.
On the one hand, it is known that press releases are common in most American
Universities making sure that scientific results obtained by American Universities
are well-publicized. On the other hand, few Chinese universities dissipate English
language press releases. Of course, also leading journals issue press releases
in that respect American and Chinese publications are treated on an equal footing.
These observations lead to the following hypotheses:
1) Chinese and American publications in Nature, Science and PNAS are of equal
citation potential as shown by received citations in the long run;
2) But American publications receive more citations than Chinese ones over a
short period of time.
2 Data collection
Publication and citation data were retrieved from Thomson Reuter’s Web of Science
on May 30, 2015. We only considered the so-called top journals and restricted
publications to those of article type. Citations received by articles published in
the years 2004 to 2008 were collected year by year. Complete data are shown in
Appendix (I–V). The terms “American” and “Chinese” publications are
operationalized as articles for which all authors’ addresses are in the USA or in
China. By way of interest we also collected information for articles with at least
one American or one Chinese address. Table 1 shows the total number of articles
published in each of the three journals: maybe surprisingly, Nature
a slightly decreasing trend in number of articles over the period 2004–2008, while
has a slightly increasing trend.
Number of publications (only article type) in the three journals
The percentage of articles with at least one American address stays more or less at
about 72%, while articles with only American addresses decrease somewhat (from
49% to 45 %). The percentage of articles with a least one Chinese address increases
from about 2% to 3%, while the percentage of articles with only Chinese addresses
stays small at about 0.3% (Table 2).
Numbers of American and Chinese articles
We use the Mann-Whitney test with the following null-hypothesis.
H0: The distribution of received citations for American articles in top journals is, in the long
run (i.e. till the year 2014) the same as that for Chinese ones.
H1: The alternative hypothesis is that they are different.
The same hypotheses are studied using a three year citation window, including
the year of publication.
It has been stated by Huber and Wagner-Döbler
that the Mann-Whitney test can
be used on informetric data, regardless of the fact that such data often have many
ties. These ties reduce the discriminatory power of the test but do not preclude its
use. For each publication year we performed two tests: one for citations received
from the year of publication till the year 2014, and the other for a 3-year citation
window consisting of the period: Publication year till publication year plus 2. Actual
calculations were performed using SPSS 16.0, which includes a correction for ties.
Table 3 shows the results for the ten tests. Considering data referring to citations
received till the end of the year 2014 there is never a reason to reject the nullhypothesis
that American and Chinese publications have the same distribution of
received citations. For the short term, the citation window of the null-hypothesis can be rejected in two cases, namely, for the year 2006 (p=0.038) and for the year
2007 (p= 0.072).
Results of Mann-Whitney test for 10 tests
Note: *c: number of Chinese only articles; **a: number of American only articles.
Figures 1 to 5 illustrate the average number of cumulative citations for American
and Chinese authors only, as well as for publications with at least one American
or one Chinese address from the year 2004 to the year 2008.
Average number of cumulative citations of papers published in 2004.
Average number of cumulative citations of papers published in 2005.
Average number of cumulative citations of papers published in 2006.
Average number of cumulative citations of papers published in 2007.
Average number of cumulative citations of papers published in 2008.
Depending on the publication year citations to American publications dominate
or Chinese ones do. These figures do not suggest that the average citation performance
of one country is systematically higher than that for the other. For each publication
year the average number of citations for collaborated American publications is
higher than that for American only publications. This suggests that American
scientists, i.e. scientists with an American address, benefit from international
collaboration. We investigated this phenomenon in a separate publication
found that it is not as straightforward as could be expected. Concretely, we found
that, statistically, American scientists publishing in Nature and Science do not benefit from international collaboration. This statement also holds for communicated
submissions to PNAS, but not for direct and for contributed submissions.
5 Discussion and conclusion
It is clear that, statistically, there is no reason to assume that citation distributions
of American only and Chinese only contributions in Nature, Science and PNAS
differ. Of course, this does not exclude the fact that there may be differences. For
instance, it may be that fewer Chinese contributions belong to the top 5% most-cited
articles in these journals (but note that numbers are too small to make such a
conclusion). Yet, none of the Chinese contributions was uncited, not even for the
3-year period, while several American ones remained uncited for the complete
period under investigation. We stress the caveat that this investigation is, by
necessity, based on a small numbers of Chinese contributions. For the same reason
it was impossible to take the relative numbers of articles published in each of the
three journals into account.
We conclude by stating that there is no indication that Chinese and American
contributions in top journals have a different citation potential. Yet, there might be
a small tendency, depending on the publication year, for American only articles to
be cited earlier.
Yearly citation data of papers published in 2004
Yearly citation data of papers published in 2005
Yearly citation data of papers published in 2006
Yearly citation data of papers published in 2007
Publ. Citations received
Yearly citation data of papers published in 2008
Cheetham, A. China still rising (interview by Nic Fleming). Nature, 2015, 520(7549): S34- S35. Retrieved on August 3, 2015, from http://www.nature.com/nature/journal/v520/n7549_supp/full/520S34a.html
Zhou, P., & Leydesdorff, L. The emergence of China as a leading nation in science. Research Policy, 2006, 35(1): 83-104. Retrieved on August 3, 2015, from http://www.sciencedirect.com/science/article/pii/S0048733305001794
Leydesdorff, L., Wagner, C. S., & Bornmann, L. The European Union, China, and the United States in the top-1% and top-10% layers of most-frequently cited publications: Competition and collaborations. Journal of Informetrics, 2014, 8(3): 606-617. Retrieved on August 3,2015, from http://www.sciencedirect.com/science/article/pii/S1751157714000509
Kiernan, V. Public relations practices at medical journals. Learned Publishing, 2014, 27(1):5-13. Retrieved on August 3, 2015, from http://chinesesites.library.ingentaconnect.com/content/alpsp/lp/2014/00000027/00000001/art00002?token=004a1036275c277b42573a6728483f256245517975592f653b672c57582a72752d704f3876
Huber, J.C., & Wagner-Döbler, R. Using the Mann-Whitney test on informetric data. Journal of the American Society for Information Science and Technology, 2003, 54(8):798-801. Retrieved on August 3, 2015, from http://onlinelibrary.wiley.com/doi/10.1002/asi.10252/abstract;jsessionid=8EFAB79AD211D51FF3AEDA891B1004EF.f01t03
Rousseau, R., & Ding, J. L. Does international collaboration yield a higher citation potential for US scientists publishing in highly visible interdisciplinary journals? Journal of the Association for Information Science and Technology. 2016 (To appear). DOI:http://dx.doi.org/10.1002/asi.235