Outline
Do Chinese and American contributions in top journals have an equal citation potential?
Jielan DING1,2E-mail to the corresponding author & Ronald ROUSSEAU3,4Corresponding authorE-mail to the corresponding author
1National Science Library, Chinese Academy of Sciences, Beijing 100190, China
2University of Chinese Academy of Sciences, Beijing 100049, China
3Information and Library Science (IBW), University of Antwerp (UA), Antwerp B-2000, Belgium
4Department of Mathematics, KU Leuven, Leuven B-3000, Belgium
2015, 8(2):1-10, Accepted: Jun. 16, 2015
This work is supported by the National Natural Science Foundation of China (Grant No.: 71173185).
R. Rousseau (ronald.rousseau@kuleuven.be, corresponding author) proposed the research idea, planned and designed the outline, wrote the first draft and revised the paper. J.L. Ding (dingjielan@ mail.las.ac.cn) performed data analysis, joined discussion of the findings and contributed to writing the paper.

Abstract
Purpose: We want to contribute to the evaluation of Chinese research, focusing on contributions in top journals.

Design/methodology/approach: Using a Mann-Whitney test we investigate if contributions in Nature, Science or the Proceedings of the National Academy of Sciences of the United States of America (PNAS) by Chinese or American authors only, i.e. articles for which all authors have a Chinese or an American address, have a different citation potential.

Findings: There is no reason to state that Chinese and American contributions in these top journals have a different citation potential.

Research limitations: Because of the small numbers involved we were not able to make a distinction between publications in Nature, Science or the Proceedings of the National Academy of Sciences of the United States of America.

Practical implications: These results suggest that the better Chinese research results are of a similar level as those by American colleagues.

Originality/value: It is well-known that the number of citations per publication by Chinese authors is still lagging with respect to leading scientific nations and in particular compared with the USA. We have shown that this difference does not necessarily hold in contributions in Nature, Science or the Proceedings of the National Academy of Sciences of the United States of America.
Keywords

1 Introduction
Although it is common knowledge that China’s increase in publications over the latest decade is astonishing[1], it is also well-known that the number of citations per publication is still lagging with respect to leading scientific nations and in particular compared with the USA[2,3]. Yet, numbers of citations are averages over all publications, at least as included in an international database. One may expect that citations per paper differ according to the level or the type of the journal. For instance, the intellectual requirements for acceptance of submissions in Nature, Science or the Proceedings of the National Academy of Sciences of the United States of America (PNAS in short) can hardly be compared with those for most national journals. These three journals are the ones we will focus on. Just as a short-hand, they will be referred to as top journals.
On the one hand, it is known that press releases are common in most American Universities making sure that scientific results obtained by American Universities are well-publicized. On the other hand, few Chinese universities dissipate English language press releases. Of course, also leading journals issue press releases[4] and in that respect American and Chinese publications are treated on an equal footing.
These observations lead to the following hypotheses:
1) Chinese and American publications in Nature, Science and PNAS are of equal citation potential as shown by received citations in the long run;
2) But American publications receive more citations than Chinese ones over a short period of time.
2 Data collection
Publication and citation data were retrieved from Thomson Reuter’s Web of Science on May 30, 2015. We only considered the so-called top journals and restricted publications to those of article type. Citations received by articles published in the years 2004 to 2008 were collected year by year. Complete data are shown in Appendix (I–V). The terms “American” and “Chinese” publications are operationalized as articles for which all authors’ addresses are in the USA or in China. By way of interest we also collected information for articles with at least one American or one Chinese address. Table 1 shows the total number of articles published in each of the three journals: maybe surprisingly, Nature and Science have a slightly decreasing trend in number of articles over the period 2004–2008, while PNAS has a slightly increasing trend.
Table 1    Number of publications (only article type) in the three journals

The percentage of articles with at least one American address stays more or less at about 72%, while articles with only American addresses decrease somewhat (from 49% to 45 %). The percentage of articles with a least one Chinese address increases from about 2% to 3%, while the percentage of articles with only Chinese addresses stays small at about 0.3% (Table 2).
Table 2    Numbers of American and Chinese articles

3 Methods
We use the Mann-Whitney test with the following null-hypothesis.
H0: The distribution of received citations for American articles in top journals is, in the long run (i.e. till the year 2014) the same as that for Chinese ones.
H1: The alternative hypothesis is that they are different. The same hypotheses are studied using a three year citation window, including the year of publication.
It has been stated by Huber and Wagner-Döbler[5] that the Mann-Whitney test can be used on informetric data, regardless of the fact that such data often have many ties. These ties reduce the discriminatory power of the test but do not preclude its use. For each publication year we performed two tests: one for citations received from the year of publication till the year 2014, and the other for a 3-year citation window consisting of the period: Publication year till publication year plus 2. Actual calculations were performed using SPSS 16.0, which includes a correction for ties.
4 Results
Table 3 shows the results for the ten tests. Considering data referring to citations received till the end of the year 2014 there is never a reason to reject the nullhypothesis that American and Chinese publications have the same distribution of received citations. For the short term, the citation window of the null-hypothesis can be rejected in two cases, namely, for the year 2006 (p=0.038) and for the year 2007 (p= 0.072).
Table 3    Results of Mann-Whitney test for 10 tests

Note: *c: number of Chinese only articles; **a: number of American only articles.
Figures 1 to 5 illustrate the average number of cumulative citations for American and Chinese authors only, as well as for publications with at least one American or one Chinese address from the year 2004 to the year 2008.
Fig. 1    Average number of cumulative citations of papers published in 2004.
Average number of cumulative citations of papers published in 2004.

Fig. 2    Average number of cumulative citations of papers published in 2005.
Average number of cumulative citations of papers published in 2005.

Fig. 3    Average number of cumulative citations of papers published in 2006.
Average number of cumulative citations of papers published in 2006.

Fig. 4    Average number of cumulative citations of papers published in 2007.
Average number of cumulative citations of papers published in 2007.

Fig. 5    Average number of cumulative citations of papers published in 2008.
Average number of cumulative citations of papers published in 2008.

Depending on the publication year citations to American publications dominate or Chinese ones do. These figures do not suggest that the average citation performance of one country is systematically higher than that for the other. For each publication year the average number of citations for collaborated American publications is higher than that for American only publications. This suggests that American scientists, i.e. scientists with an American address, benefit from international collaboration. We investigated this phenomenon in a separate publication[6] and found that it is not as straightforward as could be expected. Concretely, we found that, statistically, American scientists publishing in Nature and Science do not benefit from international collaboration. This statement also holds for communicated submissions to PNAS, but not for direct and for contributed submissions.
5 Discussion and conclusion
It is clear that, statistically, there is no reason to assume that citation distributions of American only and Chinese only contributions in Nature, Science and PNAS differ. Of course, this does not exclude the fact that there may be differences. For instance, it may be that fewer Chinese contributions belong to the top 5% most-cited articles in these journals (but note that numbers are too small to make such a conclusion). Yet, none of the Chinese contributions was uncited, not even for the 3-year period, while several American ones remained uncited for the complete period under investigation. We stress the caveat that this investigation is, by necessity, based on a small numbers of Chinese contributions. For the same reason it was impossible to take the relative numbers of articles published in each of the three journals into account.
We conclude by stating that there is no indication that Chinese and American contributions in top journals have a different citation potential. Yet, there might be a small tendency, depending on the publication year, for American only articles to be cited earlier.
Appendix I:    Yearly citation data of papers published in 2004

Appendix II:    Yearly citation data of papers published in 2005

Appendix III:    Yearly citation data of papers published in 2006

Appendix IV:    Yearly citation data of papers published in 2007 Year Publ. Citations received

Appendix V:    Yearly citation data of papers published in 2008


References
1 Cheetham, A. China still rising (interview by Nic Fleming). Nature, 2015, 520(7549): S34- S35. Retrieved on August 3, 2015, from http://www.nature.com/nature/journal/v520/n7549_supp/full/520S34a.html. DOI:10.1038/520S34a.
2 Zhou, P., & Leydesdorff, L. The emergence of China as a leading nation in science. Research Policy, 2006, 35(1): 83-104. Retrieved on August 3, 2015, from http://www.sciencedirect.com/science/article/pii/S0048733305001794. DOI:10.1016/j.respol.2005.08.006.
3 Leydesdorff, L., Wagner, C. S., & Bornmann, L. The European Union, China, and the United States in the top-1% and top-10% layers of most-frequently cited publications: Competition and collaborations. Journal of Informetrics, 2014, 8(3): 606-617. Retrieved on August 3,2015, from http://www.sciencedirect.com/science/article/pii/S1751157714000509. DOI:10.1016/j.joi.2014.05.002.
4 Kiernan, V. Public relations practices at medical journals. Learned Publishing, 2014, 27(1):5-13. Retrieved on August 3, 2015, from http://chinesesites.library.ingentaconnect.com/content/alpsp/lp/2014/00000027/00000001/art00002?token=004a1036275c277b42573a6728483f256245517975592f653b672c57582a72752d704f3876. DOI:10.1087/20140102.
5 Huber, J.C., & Wagner-Döbler, R. Using the Mann-Whitney test on informetric data. Journal of the American Society for Information Science and Technology, 2003, 54(8):798-801. Retrieved on August 3, 2015, from http://onlinelibrary.wiley.com/doi/10.1002/asi.10252/abstract;jsessionid=8EFAB79AD211D51FF3AEDA891B1004EF.f01t03. DOI:10.1002/asi.10252.
6 Rousseau, R., & Ding, J. L. Does international collaboration yield a higher citation potential for US scientists publishing in highly visible interdisciplinary journals? Journal of the Association for Information Science and Technology. 2016 (To appear). DOI:http://dx.doi.org/10.1002/asi.235.