Country Comparisons of Human Stroke Research Since 2001
A Bibliometric Study
Background and Purpose—This is the first bibliometric comparison between countries of the development of stroke research over time.
Methods—Clinical and epidemiological articles on stroke published 2001 to mid-2011 were identified in Science Citation Index Expanded. Article fractions, citation fractions, h-index, and international collaboration were calculated using the BibExcel software and adjusted for population size and gross domestic product.
Results—The United States dominated with 28.7% of the sum of article fractions and 36.2% of the sum of citation fractions. The United States, Japan, the United Kingdom, and Germany together accounted for 52.1% of articles and 61.0% of citations. When adjusted for population size or gross domestic product, several small European countries, together with Israel and Taiwan, ranked the highest. Per population, there was a negative association (r=0.60) between burden of stroke (disability-adjusted life-years lost) and number of articles per population. In China, South Korea, and Singapore, the annual growth of stroke articles was more than twice the worldwide average. Whereas multinational collaboration was common within Europe and North America, it was relatively uncommon between Asian countries.
Conclusions—The Big 4 in scientific literature on stroke, as to both number of articles and citations, are the United States, Japan, the United Kingdom, and Germany. Many small European countries have, in relation to their size, a high scientific production. Several countries with rapidly expanding economies have very fast growth of scientific production on stroke. Our results emphasize the need for stroke research in countries with a high population burden of stroke and they highlight the role of multinational collaboration.
Bibliometrics is increasingly used to evaluate research by quantity as well as quality. Examples of its use, at least as a supplementary instrument, are ranking of journals, ranking of candidates that apply for academic positions and awards, allocation of resources to research groups, and external reviews of the scientific excellence of institutions or universities.1 It has been argued that the strength of bibliometrics is its intrinsic validity in contrast to surrogates such as external grant funding.2 At the macro level, it may also be used to evaluate scientific performance at a regional, national, or multinational level.
Despite its wide application in other research domains, there is very limited literature3,4 in which bibliometrics has been used for benchmarking of performance in stroke research or neurological research at large. A recent editorial in Stroke used bibliometrics to compare the impact of selected neuroscience journals.5 An extensive validation of bibliometric data on Swedish neurological research showed that a presumed decline in international competitiveness was not attributed to methodological bias.6
In this benchmarking study, we provide information on countries' proportions of all articles on human stroke research, their impact, and time trends during the years 2001 to 2011. The methodological approach for the present study is similar to that applied in an overview of public health research.7 Crude data as well as data adjusted by population size and gross domestic product (GDP) are reported. The relation between number of stroke publications and population burden of stroke is analyzed. In view of the close linkage between clinical and epidemiological research, we have included both. Preclinical experimental research is not covered.
Materials and Methods
The Science Citation Index Expanded database was used to identify clinical and epidemiological articles on stroke. It includes >8300 major journals across 150 disciplines and was accessed through Web of Science. The study covered a period from January 2001 to August 2011. For time trend analyses, we included only full years (2001–2010). Citation analyses were based on articles during the years 2001 to 2010 and cited up to August 2011.
Inclusion criteria were (1) clinical and epidemiological articles on stroke, including all subtypes listed in MESH; (2) original articles and reviews; (3) articles published in English; and (4) time period from January 2001 to August 2011. In time trend analyses, we included only countries with ≥100 articles published during 2001 to 2010. Exclusion criteria were (1) nonhuman studies; and (2) articles other than original studies and reviews; thus, conference proceedings, editorials, letters, and book chapters were excluded. Although the main analysis concerned only English language article, articles in other languages were also recorded in a supplementary analysis.
The search strategy is presented in Supplemental Table I (http://stroke.ahajournals.org). In addition to “stroke,” we used synonyms to stroke (“cerebrovascular disease[s]” and “apoplexy”) and all stroke subtypes listed in MESH as search terms. Because the term “stroke” is also included in the cardiology term “stroke volume,” we excluded articles with the latter term. A random sample of 455 articles (1% of all articles) was reviewed to validate the search.
The software BibExcel8 was used to analyze the downloaded records from the search: editing country addresses, calculation of article fractions, citation fractions, and country coauthorships. The analyses are based on sums of article fraction and sums of citation fractions. When an article has authorship from >1 country, each country is assigned a fraction of this article and its citations depending on the number of addresses, for example, for an article with 2 addresses from 1 country and 1 address from another country, the first country is assigned two thirds of an article when quantity is analyzed and two thirds of its citations.
As a supplementary analysis, the impact of publications was also measured by the h-index.9 This index is constructed so that it discounts the disproportionate weight of highly cited articles or articles that have not yet been cited. Using BibExcel, we adopted it to estimate the impact of stroke publications at the country level.
The number of articles and citations are reported as percentages of all articles and citations, both as crude proportions and proportions adjusted by number of inhabitants and gross national product. Data on population sizes were derived from UN statistics10 and data on gross national product from The World Bank11 supplemented by data on Taiwan from the International Monetary Fund database.12 European Union countries were combined both as member states in 2001 (EU-15) and in 2011 (EU-27). For analyses of number of articles in relation to population burden of stroke, we used World Health Organization country data on disability-adjusted life-years lost by cerebrovascular disease.13
Poisson regression was used to model the sum of article fractions. Year was included as a continuous covariate assuming a log-linear relationship. Separate models were fitted for each country and the regression coefficient×100 was used to estimate the annual relative change (%). To compare time trends, an additional model included all countries and the independent variables country, year, and country-by-year interaction. Estimates are presented with 95% CIs and the level of significance was 0.05. Approximate 95% CIs for ranks were calculated by using a Monte Carlo simulation procedure, similar to that described by Marshall et al,14 based on 10 000 iterations. Here, the sums of article and citation fractions for each country were assumed to follow a Poisson distribution. R Version 2.14 was used for the simulation. For other statistical analyses, SAS 9.2 was used.
The literature search yielded a total of 44 877 clinical and epidemiological articles on stroke published in the English language from January 2001 to August 2011. In a validation of 445 randomly selected articles (1% of all articles), we identified 41 articles that did not fulfill the inclusion criteria (13 on neurological conditions other than stroke, 23 on other diseases, and 5 nonhuman studies; overall positive predictive value 90.8%). The distribution of countries among articles that did not fulfill the inclusion criteria (United States 32%; United Kingdom, Japan, and Germany together 24%; other countries 44%) was similar to that of the entire set of articles (see subsequently). Terms that were added late in the search process, for example, more subclassifications of stroke, resulted in very few additional articles, not identified at previous steps of the search, indicating high sensitivity.
Articles identified by our search strategy did not always have stroke as a major focus but as 1 of many disorders (including, for instance, stroke as 1 of several manifestations of cardiovascular disease). If the condition that the term “stroke” should be present in the title, abstract, or authors' key words was added to possibly retrieve more stroke-specific articles, it had very little impact on the ranking of countries (data not shown).
When our SCI search was extended to articles published in any language, the great majority of articles retrieved were in English (93.2%). Other relatively frequent languages were German (1.9%), Spanish (1.7%), French (1.1%), and Russian (0.9%). No other language accounted for >0.4%.
Excluding review articles had little impact on the ranking of countries, whether article fractions or citations fractions were compared, even if citation fractions were, in general, somewhat lower (data not shown).
Country Proportions of All Articles and Citations
Figure 1 shows the top 30 countries by quantity of articles, expressed as sums of article fractions. The United States had a very marked dominance (28.7% of the sum of article fractions). Next in ranking by quantity were the United Kingdom, Japan, and Germany (7%–8%).
The sums of citation fractions for articles published during the years 2001 to 2010 are shown in Figure 2 (left panel). Here, the US dominance is even more evident (36.2%). Belgium, Denmark, France, Sweden, and the United Kingdom also had a bigger proportion of citations than proportion of articles. For all other countries, the proportion of citations was equal to or lower than the proportion of articles.
When the h-index was used to further assess the impact of stroke publications (Figure 2, right panel), most countries ranked about the same as when citation fractions were used. A general pattern was, however, that the Scandinavian countries (Denmark, Finland, Norway, and Sweden) ranked somewhat higher when h-index was used, whereas East Asian countries (China, Japan, South Korea, and Taiwan) ranked lower.
Country Proportions Adjusted for Population Size and GDP
When sums of article fractions were adjusted for population size, several small European countries, together with Israel, ranked the highest (Table 1). Among the 4 countries with the largest production in absolute numbers, the United Kingdom ranked considerably higher per million inhabitants than the United States, Germany, and Japan. When the European Union countries were analyzed together, the article fraction adjusted for population size was somewhat lower than the US fraction but higher than the Japanese fraction (Table 1).
There was a very close relationship between GDP and number of publications on stroke (r=0.98 for the top 30 countries as to sum of article fractions). When the countries were ranked by sum of article fractions adjusted for GDP, Israel had the largest production of scientific articles during 2001 to 2011 followed by Taiwan and the same small European counties that ranked high in the population-adjusted comparisons. Again, the United Kingdom ranked highest among the 4 countries with most published articles. Countries with big economies such as the United States, China, and Japan, although having a high proportion of article fractions (Figure 1), tended to rank relatively low after adjustment for GDP. The European Union countries together had a higher GDP-adjusted article fraction than the United States and Japan (Table 1).
In citation analyses, Sweden ranked the highest, whether adjusted by population or economy size (Table 1). Several other small European countries, plus the United Kingdom, also ranked high. It may be noted that Asian countries with a high production of articles (China, Japan, and South Korea) ranked relatively low in citations after adjustment for population size or GDP. The citation fraction for articles from the European Union countries combined was similar to that in the United States whether adjusted for population size or GDP (Table 1).
Relation to Country Burden of Stroke
Among the 29 countries with the largest number of stroke publications during the years 2001 to 2010 (comparative World Health Organization data on disability-adjusted life-years are not available for Taiwan), there was a strong negative correlation between the population burden of stroke (measured as age-standardized disability-adjusted life-years attributed to cerebrovascular disease per population) and the number of publications per population (r=0.60; P=0.001). Several countries with a high burden of stroke had a relatively low number of stroke articles per population (eg, China, India, Turkey), whereas some countries with a relatively low burden of stroke had a high number of publications per population (eg, Canada, Israel, and Switzerland).
In all countries with a quantity of articles that permitted reliable analyses (≥100 articles during study period), there was an increase in the sum of article fractions over the 2001 to 2010 period. The overall annual increase was 10.4% (95% CI, 10.1%–10.8%). As emerges from Table 2, left columns, the increasing pattern was similar in most countries, being in the range of 9% to 14% annual increase. However, in 3 countries, China, South Korea, and Singapore, the annual rate of increase was more than twice the international average. In Israel, Japan, and the United Kingdom, the rate of increase was significantly slower than the worldwide average (Table 2).
When the article quantity ranking of countries in the beginning of the study period (2001–2002) was compared with that at the end of the period (2010–2011), the top 5 countries were identical at the 2 time points. Eight countries had advanced ≥5 positions, 5 of them in Asia (China, India, Iran, South Korea, and Taiwan), 2 in Europe (Greece and Poland), and 1 in South America (Brazil). Three countries lost ≥5 positions (Finland, Israel, and Sweden). Due to the large number of article fractions, 95% CIs of the rankings were narrow.
Because recently published articles have fewer citations than articles published earlier (articles published in 2002 had the highest total number of citations), annual trends in citations by country are not reported here. To illustrate change over time, we instead provide citation ranking data at the beginning and the end of the study period (Table 2, right columns). Three countries improved their citation fraction ranking from 2001 to 2002 to 2009 to 2010 by 5 positions or more: China, India, and South Korea. Among the top 10 countries in 2001 to 2022, Belgium, Japan, and Sweden each lost ≥3 ranking positions. Among countries further down the list, Austria, Finland, Hungary, Norway, Singapore. and Turkey lost ≥5 positions. It should be noted, however, that Belgium and Singapore each had 1 very frequently cited article in 2001, making their positions at the beginning of the study period spuriously high. As for article fractions, the 95% CIs of the rankings were narrow.
In Figure 3, multinational collaboration, as reflected by articles with authorship from >1 country, has been mapped., showing links with at least 50 coauthored articles. The pattern was essentially the same as for sums of article and citation fractions with the United States as the leading country. However, there were a few countries with few strong multinational authorships in view of their overall production of stroke articles: China, Japan, and South Korea. Whereas there was frequent coauthorship between west European countries, there were no strong coauthorships between Asian countries.
Worldwide, there has been a >10% annual increase in clinical and epidemiological stroke publications during the last century. We have documented the US dominance in clinical and epidemiological stroke research and the rapidly increasing contribution of Chinese and South Korean research to the scientific literature on stroke. When adjusted for population size and GDP, many small European countries rank remarkably high in stroke research production, but, over the years, many of them seem to have lost some of their competitiveness.
The addition of relevant new articles was minimal when, by the end of the search procedure, new search terms were added. This indicates a high sensitivity of our search strategy. Validation of randomly selected articles indicated a reasonably good specificity. Articles resulting from the search strategy that were found not to be relevant were similarly distributed between countries as relevant articles. It is highly unlikely that any bias caused by the limited proportion of irrelevant articles was systematic enough to significantly affect the comparisons between countries.
The fact that our search was not restricted to stroke or neurological journals may be seen both as a strength and a weakness. The search strategy resulted in a broad spectrum of articles, that not all had stroke as a major focus. For instance, reports on stroke as 1 of several outcomes or stroke as an adverse effect of therapy were included. Thus, we studied countries' contribution to the collected worldwide knowledge on stroke rather than success or failure of a country's policy to promote specific stroke research.
SCI Expanded covers all important medical journals published in English, but its coverage of non-English literature is limited.15 This may explain why a big country like Russia (with stroke research predominantly published in Russian language journals) does not appear, in any aspect, among the top 30 producers of stroke research. Correspondingly, research published in Spanish, Chinese, and Japanese may be underrepresented in our analyses of research production. Because non-English articles are infrequently cited in the international literature, comparisons of citations are less prone to be affected by selective language coverage in SCI.
The use of impact factors to evaluate quality of healthcare research has often been criticized, perhaps most eloquently by Seglen as early as 1997.16 Concerns have also been expressed by organizations such as the European Association of Science Editors.17 The sum of citation fractions and the h-index, as we have used, are a more direct measures of the impact stroke articles have had than journal impact factors. The need for adjustment for speciality has also been emphasized.17,18 Because our study was restricted to a relatively uniform research area, there was no obvious need for adjustment by speciality.
The dominance of the United States in both quantitative and qualitative analyses was impressive, but much less so when population size and GDP were taken into account. It should be noted that, due to the size of the US economy (nearly 3 times bigger than that of any other country), the US ranking came out as quite low when adjusted for GDP. The share of US articles of total number of stroke articles in the world (sum of article fractions 29%) is approximately the same or somewhat smaller than reported from other clinical disciplines, for example, infectious diseases, cardiopulmonary diseases, and public health epidemiology.7,19,20 Together, the big 4 (United States, Japan, United Kingdom, and Germany) produce the majority of the world's stroke literature: 52% of the sum of article fractions and 61% of the sum of citation fractions.
Many small countries in western Europe (most notably the Scandinavian countries), together with Israel, rank high in the population and GDP-adjusted comparisons. However, several of them, for instance all Nordic countries such as Denmark, Finland, Norway, and Sweden, together with Israel, have lost rank positions during the last decade. This may represent a regression-to-the-mean phenomenon in a very competitive international setting, but it may also be an expression of emerging problems in research organization, innovation culture, and funding in these countries. Exceptions to the trend in small, previously successful countries to lose positions are The Netherlands and Switzerland, both keeping their high rankings.
The much cited Clinical Research Roundtable at the Institute of Medicine has characterized US clinical research as “increasingly encumbered by high costs, slow results, lack of funding, regulatory burdens, fragmented infrastructure, incompatible databases, and a shortage of qualified investigators and willing participants.”21 It is beyond the scope of this article to analyze to what extent some countries may have been more successful than others to overcome these obstacles. However, we note that, first, the United States has, despite these challenges, kept its dominating position in stroke research and, second, that common to many of the successful small countries are strong multinational research networks. It is notable that multinational collaboration is much less common outside North America and Europe.
There was a very strong negative correlation between the population burden of stroke and stroke research production. The links are complex with socioeconomic factors seemingly playing a major role. Our results highlight the obvious need for high-quality stroke research in countries with a high burden of the disease.
It seems that countries with rapidly expanding economies strengthen their positions. China and South Korea have emerged as leading countries in stroke research, more so in quantitative terms than in impact. Common to them is very strong economic growth. Brazil, India, and Taiwan are also examples of countries with an expanding economy that rank considerably higher in stroke research now than a decade ago. The rate of increase in number of articles has been less than the international average in Japan despite a well-recognized stroke research history, perhaps related to economic growth below the international average during the last decade.
To conclude, we have provided benchmarking data on the quantity of scientific publications and on impact in terms of citation rates and h-index in different countries during the last decade. We have also mapped multinational collaboration. Irrespective of bibliometric measure, there is a striking association between economic strength (and growth) and volume and impact of stroke research. Considering their population size and GDP, several small European countries rank high both as to publication volume and citation rates. During the decade, there has been a very conspicuous increase in the Asian contribution to the scientific literature on stroke.
Source of Funding
This study was supported by Vinnvård, a joint initiative by Vinnova, the Vårdal Foundation, and the Swedish Ministry of Health.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.111.637249/-/DC1.
- Received August 25, 2011.
- Revision received November 23, 2011.
- Accepted December 1, 2011.
- © 2012 American Heart Association, Inc.
- Thomas B,
- A. M.,
- Sandercock P
- Lo EH,
- Fisher M
- Clarke A,
- Gatineau M,
- Grimaud O,
- Royer-Devaux S,
- Wyn-Roberts N,
- Le Bis I,
- et al
- Persson O
- Hirsch JE
Department of Economic and Social Affairs Population Division, United Nations. World Population Prospects. 2010 revision. Available at: http://esa.un.org/unpd/wpp/index.htm. Accessed August 11, 2011.
The World Bank: World Development Indicators database. GDP ranking. Available at: http://data.worldbank.org/data-catalog/GDP-ranking-table. Accessed August 12, 2011.
International Monetary Fund, World Economic Outlook Database: Nominal GDP list of countries. Data for the year 2010. Available at: www.imf.org/external/pubs/ft/weo/2011/01/weodata/index.aspx. Accessed August 11, 2011.
World Health Organization. Disease and injury country estimates. Burden of disease. Available at: www.who.int/healthinfo/global_burden_disease/estimates_country/en/index.html. Accessed August 13. 2011.
- Marshall EC,
- Spiegelhalter DJ
- Seglen PO
European Association of Science Editors. EASE statement on Impact factors. Available at: www.ease.org.uk/artman2/uploads/1/EASE_statement_IFs_final.pdf. Accessed August 5, 2011.
- Falagas ME,
- Michalopoulos AS,
- Bliziotis IA,
- Soteriades ES