Effect of English Proficiency and Research Funding on Acceptance of Submitted Articles to Stroke Journal
Science is a shared and global undertaking that requires cumulative contributions from scientists from all around the world. Peer-reviewed publications, which are an essential part of the dissemination of scientific research, provide information on innovative interventions and discoveries,1 guide funding agency grant-making decisions, constitute a major criterion in academic faculty promotion, affect the receipt of career milestone awards, and can be pivotal in the allocation of public health resources.2,3
As the channels through which peer-reviewed publications become accessible to the scientific world, journals are science’s gatekeepers, wielding tremendous influence on whether various scientific works receive broad readership and recognition. Given that the English language has become the international lingua franca of scientific communication, with most high effect journals being published in English,4,5 authors’ native language may be a barrier to the publication of otherwise scientifically worthy articles, thereby limiting international knowledge exchange among scientists. Indeed, it is conceivable that novel ideas or projects not properly articulated in English may be inadvertently assigned low-priority scores, whereas less innovative, but well-written articles (ie, from countries with high English proficiency) may receive high-priority scores and eventually be published. Another nonscientific issue that may affect an article acceptance could be economic: articles from countries with relatively lower overall research funding may be held to a higher threshold of acceptance, based on a perception of potentially less developed scientific culture and rigor.
Worldwide, the journal Stroke is generally regarded as the premier scientific journal covering cerebrovascular disorders. For Stroke, priority scores depend on the experimental design, effect, innovation, and quality of the article, but other factors also influence the acceptance rate as highlighted in a previous report.6 A better understanding of language barriers and economic factors influencing the acceptance rate would be informative for Stroke readers, scientists, and authors when submitting a new article and for editors when making decisions. Our goal is to determine whether the native language in the country of the corresponding author and research funding influence the likelihood of acceptance.
We analyzed all 14 949 original contributions submitted to Stroke journal from January 2004 to December 2011. All noninvited original article submissions were considered eligible for this study. The status of each article was followed up until September 2012. Data elements for the present study included: article ID, submission date, request revision date, number of requested and accepted reviewers, reviewer’s recommendation, most recent editorial decision, subject codes, key words, article type, decision date, publication online date, publication print date, country, city, and affiliation of the corresponding author, and number of authors per article. Details of the general editorial process have been published elsewhere.6 We also compared trends before and after July 1, 2010, when a new triaging system (led by a senior editor and one reviewer) and assignation of reviewers were introduced.
We used language as a proxy measure of nonacademic factors that could potentially influence reviewers’ recommendations and editorial decisions. As research funding is a potential confounder and a proxy measure of research investment and training, we analyzed absolute research funding as the gross expenditures in research and development, as percentage of the gross domestic product (GDP) and GDP by country during the study period. GDP is usually adjusted for purchasing power parity, an economic technique to determine the relative value of different currencies, thus making economic indicators internationally comparable. This information was obtained from the World Bank (http://data.worldbank.org) and a summary report (Table I in the online-only Data Supplement).7 Research funding from Nigeria, Vanuatu, Cameroon, Jamaica, Bahrain, Georgia, and Oman were not available. Together with missing country data (n=1089), the analysis, including economic indicators, was available for 13 860 (92.7%) of 14 949 submitted original contributions. Priority scores based on reviewers’ recommendations were used as proxy of scientific quality ranging from 1 (low) to 4 (high).
The first language was classified as English or non-English, depending on whether English was the native or mother language in the country of the corresponding author. We also analyzed English proficiency. The native language of each country was determined according to accepted definitions,8 as well as British Council criteria (learning and research in English Next from http://www.britishcouncil.org/learning-research-english-next.pdf) and exceptions made for the Test of English as a Foreign Language and International English Language Testing System tests when applying to medical schools (Table II in the online-only Data Supplement).
English proficiency was determined according to the Education First English Proficiency Index (EPI) that ranks countries by the average level of English skills among adults.9 It ranges from 38 (low) to 100 (high).
The main outcome of interest was the final editorial decision classified as accepted or rejection.
χ2 test was used to compare categorical variables; ANOVA or Kruskal–Wallis tests were used to compare mean and median differences for continuous variables.
We used generalized estimating equations to fit the models (link function: logit), to account for clustering of patients within countries. Compound symmetry (independent) was selected as the working correlation structure. This is a standard statistical technique used when other variables and characteristics of treatment that affect outcome are thought to exist and expected to cluster within predetermined groups.10 Logistic regression analysis was used to compare the acceptance rate by country from a previous report6 and after controlling for English proficiency and research funding (Table 1). Native English-speaking countries (United States, Canada, United Kingdom, Australia, and New Zealand) were the reference category when comparing English proficiency with the outcome of interest. In analyzing the effects of nonscientific factors on the rates of acceptance, we controlled for several scientific factors, including year of submission, country, study type (clinical or basic science), % of GDP as research, number of authors per article, priority scores, and English proficiency. Linear regression analysis was completed to determine the association between the variation in research funding and the probability of acceptance by country. R2 was used to evaluate the variance explained by exposure and additional variables. Also, the assumptions of linearity and homoscedasticity were tested using graphical methods (plots observed versus predicted and residuals versus predicted values) and multicollinearity effects using the variable inflation factor (variable inflation factor values >10 or 1/variable inflation factor <0.1 suggests of collinearity).
A trend analysis was completed to determine whether the language effect was present after Editorial changes. A sensitivity analysis was completed by also adjusting for scientific quality and number of authors in the articles, and by including countries where English is considered a second language (eg, India and Pakistan) or when several dialects/multiple languages (eg, South Africa, Cameroon, Ghana, Gambia, Kenya, Nigeria, Zambia, and Zimbabwe) were available as per the African Academy of Languages (http://www.acalan.org, accessed November 12, 2013). We also explored the consistency of our findings by evaluating language disparities for articles mainly focused on thrombolysis, acute stroke care, rehabilitation, stems cell or angiogenesis, endovascular treatment, and randomized trials or submitted to the basic science section.
Model discrimination was assessed by the area under curve (equivalent to the c statistic). Statistical analysis was performed using STATA version 9 (StataCorp LP, College Station, TX). All tests were 2 tailed, and P values <0.05 were considered significant. The protocol was approved by the Stroke Editorial Office and the American Heart Association.
Of 14 949 original contributions submitted to Stroke during the study period, there were 9259 articles from non-English speakers and 5690 submissions from English speakers.
The mean number of authors per article was 6.9±3.3. The mean priority score was 2.16±0.81 and was significantly higher for accepted articles (3.15 versus 1.9; P<0.0001). Articles submitted from English-speaking countries had a higher priority scores than those from non–English-speaking countries (2.36 versus 2.03; P<0.0001).
Overall, the acceptance rate was significantly higher for English-speaking countries (29.9% versus 15.8%; P<0.0001). Similar findings were observed for articles submitted to the basic science section (acceptance rate for English-speaking countries, 33.1% versus 17.6 for non–English speaking countries; P<0.0001) and among those with the main focus on thrombolysis (33.2% versus 22.6%; P<0.0001). Accepted articles were from countries with significantly higher English proficiency (mean EPI, 80.4 versus 71.6; P<0.0001). Moreover, articles accepted for publication originated from countries with substantially higher investment in research and development (mean expenditures in billions of US$ adjusted by purchase per parity: US$192.12 versus US$145.96). The top 10 countries submitting original contributions for Stroke are ranked among the richest countries in the world as per the purchasing power parity-GDP during the study period (2004–2012; Table 1).
Interestingly, the higher likelihood of acceptance for submissions from the United States, Canada, and United Kingdom became not significant after adjusting for native language, research funding, section (basic science versus others), and year of submission. Conversely, articles from Japan, South Korea, and China were less likely to be accepted after controlling for confounders (Table 1). These results remained unaltered after adjusting for priori scores (Table III in the online-only Data Supplement).
For every million US$ invested in research (β=0.270; P<0.0001), the acceptance rate increased by 27%. Similar findings were observed for expenditures in research as a % of GDP. For every 1% increase in research funding (as % of GDP; β=0.0200; P<0.0001), the likelihood of acceptance increased by 2%. Figure 1 showed the higher odds of acceptance by incremental research funding (as % of GDP) by country (P<0.0001). The results remained consistent after adjusting for the scientific quality of the articles (for US$ millions invested in research, β=0.255; P<0.0001 and for research investment as % of GDP, β=0.0157; P<0.0001).
Univariate analysis showed that articles submitted from non–English-speaking countries were less likely to be accepted (odds ratio [OR], 0.44; 95% confidence interval [CI], 0.40–0.47; Table 2). Multivariable regression analysis accounting for clustering revealed that submissions from non–English-speaking countries had a 38% lower odds of being accepted (OR, 0.62; 95% CI, 0.53–0.74) after adjusting for volume, continent, year of submission, and research expenditures. Similar findings were observed after adjusting for priority scores and number of authors per manuscript (OR, 0.70; 95% CI, 0.60–0.82; area under curve, 0.931). These results remained consistent when analyzing the odds of acceptance for English-speaking authors for specific topics (eg, thrombolysis, rehabilitation, and randomized clinical trials; Table 2).
Figure 2 represents the lower odds of acceptance in countries with lower English proficiency after adjusting for confounders, including priority scores. The results remain unaltered in the sensitivity analysis (by including countries where English is considered a second language).
More importantly, the analysis remained consistent when using the EPI or categories of English Proficiency. Countries with higher EPI (OR, 1.02; 95% CI, 1.013–1.027) were more likely to have articles accepted.
Multivariable linear regression analysis showed that English proficiency (β=0.0033; P<0.0001) was associated with acceptance independently of level of funding (β=0.0146; P<0.0001) by country and the scientific quality of the articles (β=0.006; P<0.0001).
There was no violation of the principles of linearity or evidence of collinearity among the studied variables.
For example, English proficiency explained 71.9% of the variability (R2=0.719) of article acceptance. This variability remained constant after adjusting for editorial factors (eg, section and year of submission; R2=0.721), research funding (R2=0.738), and the scientific quality of articles (R2=0.739). Together, these results suggest that English proficiency was a strong and independent predictor of article acceptance irrespective of other measured confounders (note the modest increment of R2 by adding other relevant variables to the adjusted models), including priority scores given by reviewers.
Trends in the Influence of Native Language on Article Acceptance
There was no difference in the acceptance rate before and after editorial changes (21.2% versus 21.0%; P=0.83). Before editorial changes, articles submitted from non–English-speaking countries (n=11 872) were 52% less likely to be accepted for publication (OR, 0.48; 95% CI, 0.36–0.63) after accounting for clustering and adjusting for year of submission, section, and research funding. Similar findings were observed after July 2010 (OR, 0.44; 95% CI, 0.29–0.67). Similar results were observed by adjusting for priority scores. There was no interaction (OR, 0.84; 95% CI, 0.68–1.04) between the period of editorial changes with native language.
The progress of biomedical sciences is based on an effective peer-review process to disseminate relevant interventions and strategies, novel discoveries, and practical clinical approaches.11 The acceptance of articles is mostly driven by scientific factors, including the design, effect, innovation, and quality.6,12,13 Nonscientific factors, such as clarity of presentation and potential biases, have not been extensively assessed. Most medical journals have a single-blinded review process, meaning that authors do not know the reviewers, whereas the name of the authors and their affiliations are available to reviewers.14–16 In this large observational study, including ≈15 000 original contribution submissions to Stroke during the past 8 years, we analyzed the acceptance rate by the native language and the influence of research funding in the country of the corresponding author. We found that submissions from non–English-speaking countries received lower priorities scores and had significantly lower odds of acceptance independently of research funding. The EPI explained ≥70% of the variability of article acceptance. Interestingly, the higher chance of acceptance for articles originated from the United States and Canada disappeared after adjusting for native language, research funding, and scientific quality. Submissions from Japan, South Korea, and China had 32% to 48% lower odds of being accepted after controlling for these factors. The results remained consistent irrespective of how English proficiency was accounted (as a continuous variable [EPI] or a categorical [eg, English-speaking countries, EPI groups]) and in the sensitivity analysis. Furthermore, original contributions from countries with low research funding (either in absolute international US$ or as % of adjusted GDP) were less likely to be accepted. The results remained consistent after adjusting for relevant confounders, including the scientific quality of articles.
A few studies have explored the influence of native language on the outcome of article submissions. Okike et al12 analyzed 1173 articles submitted to The Journal of Bone and Joint Surgery between 2004 and 2005. The authors found articles from countries other than the United States or Canada were significantly less likely to be accepted (OR, 0.51; 95% CI, 0.28–0.92; P=0.026). Corresponding authors with ≥10 previous publications were 2-fold (OR, 2.01; 95% CI, 1.33–3.05) and more likely to be successful. They found no significant association between the primary language and the acceptance.12
Other studies found an unconscious preference for American and Canadian research.17 For example, a study analyzing US (n=2355) and non-US (n=1297) reviewers’ recommendations for articles submitted to JAMA found that US reviewers were more likely to recommend acceptance of articles submitted by authors from the US authors more often than did non-US reviewers (P=0.001). Moreover, US reviewers ranked articles submitted from US more favorably (P=0.001) than non-US submissions. There was also a similar trend for non-US reviewers, who ranked articles submitted by US authors slightly more favorably than articles submitted by non-US authors (P=0.09).18 Interestingly, a study that evaluated the effect of blinded review on abstract acceptance (n=13 455) to the American Heart Association scientific sessions from 2000 to 2004 found that an open review of abstracts favored authors from the United States, English-speaking countries outside the United States, and prestigious academic institutions. A blinded review process reduced the reviewer bias by ≈34%.19 None of these studies adjusted for research expenditures or GDP.
What our study shows is that research conducted in the richest countries (based on GDP or research funding) and those with high English proficiency are more likely to be published independently of the scientific quality, thus driving the research agenda. These findings can be explained by nonscientific factors influencing the acceptance rate at 2 different levels (eg, editorial and research funding). For example, it is possible that some reviewers are more critical of the quality of research or give lower priority scores to borderline articles written by non-English speakers. Contrarily, regional constraints (eg, low investment in research, suboptimal research training, limited education in how to write scientific articles, etc.) may also explain the parallel lower submissions and acceptance rates for non–English-speaking countries and for those with low expenditures in research.20 Other financial incentives (eg, scientists receiving bonuses per performance and submitting higher number of lower quality publications) may also play a role.21
The relatively low improvement in academic productivity (ie, 2% increase in the number of accepted publications) for each point increase in the GDP is not surprising. Previous studies revealed that National Institutes of Health research grant funding (≈$1.7 million) leads to only 1 additional publication during the following 5 years, representing a modest 7% increase.22 Moreover, a considerable number of high-quality studies in cardiovascular research are conducted without direct financial support, especially by young clinical investigators.23
The present study has some limitations and strengths. First, we analyzed the corresponding author country of origin as a proxy of native language. It is possible that trainees (eg, visitors and fellows) from non–English-speaking countries submitted articles as the designated corresponding author. This phenomenon could have led to a misclassification causing an underestimation of the true effect of language effect on article decisions. Second, we do not have access to each individual article to assess its grammar. Third, we do not have information about research training in each country. However, we adjusted the analysis by priority scores given by reviewers and for macroeconomic indicators that reflect research investment to ameliorate this limitation.
Despite these limitations, this comprehensive analysis of all original contributions submitted to Stroke ≥8 years with a complete outcome assessment provides evidence about the effect of native language and research funding on the acceptance of article submissions.
Potential solutions to attenuate this phenomenon include the implementation of editorial services or a coaching process for non-English speakers, standardized assessment by English- and non–English-speaking reviewers for each article, and a double-blinded review system. Alternatively, medical journals may consider asking reviewers to ignore the writing quality or downplay their comments in editorial decisions. This report in conjunction with the support of different scientific organizations may help increase the awareness about the potential effect of economic disparities, research investment, and training.
In summary, original contributions related to cerebrovascular disease from countries with low English proficiency scores, low GDP, or research funding are less likely to be accepted for publication irrespective of their scientific quality. Together, our findings have practical implications especially when considering the low English proficiency and research funding among countries with the highest burden of stroke (eg, low- and middle-income countries). A better understanding of nonacademic factors influencing reviewers’ recommendations and editorial decisions is relevant to assure equal access to publication of high-quality research articles irrespective of the native language of the authors or research investment on the country of origin. This is a complex problem with no simple solutions. It would require a multicountry and multidisciplinary effort involving patients, healthcare providers, policymakers, world leaders, and scientific organizations to ameliorate the effects of English proficiency and make research funding more cost-effective toward local and regional goals.
We thank Rebecca Seastrong, Managing Editor at the Stroke Editorial Office, for her invaluable support and assistance. We appreciate the support from the Applied Health Research Centre at the Li Ka Shin Institute and the University of Toronto Stroke Program. We are also grateful to the Stroke Journal Editorial office and the American Heart Association (AHA) for providing access to the data. The opinions, results, and conclusions reported in this article are those of the authors and are independent from the Stroke Editorial Office, the AHA, and Heart and Stroke Foundation of Canada.
Sources of Funding
This study was supported, in part, by the American Heart Association and Heart and Stroke Foundation of Canada (HSFC). Dr Saposnik is supported by Distinguished Clinician-Scientist Award from the HSFC.
Dr Fisher receives compensation as the editor of Stroke. Dr Saposnik, Dr Ovbiagele, S. Raptis, Dr Fisher, and Dr Johnston all participated in the conception, design, analysis, interpretation of the results, drafting the article, and made a critical revision of the article.
Guest Editor for this article was Bo Norrving, MD, PhD, FESO, FAHA.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.114.005413/-/DC1.
- Received March 10, 2014.
- Revision received March 30, 2014.
- Accepted April 11, 2014.
- © 2014 American Heart Association, Inc.
- Jennings C
- Saposnik G,
- Johnston SC,
- Raptis S,
- Ovbiagele B,
- Fisher M
- 7.↵2012 global. R&D funding forecast. R&D Magazine. 2011;1–35.
- 9.↵Education First. English Proficiency Index. Toronto: Education First; 2013. http://www.ef.com/ca/epi/downloads. Accessed March 28, 2014.
- Hanley JA,
- Negassa A,
- Edwardes MD,
- Forrester JE
- Halligan M,
- Zecevic A
- Okike K,
- Kocher MS,
- Mehlman CT,
- Heckman JD,
- Bhandari M
- Rothwell PM,
- Martyn CN
- Wong VS
- Lynch JR,
- Cunningham MR,
- Warme WJ,
- Schaad DC,
- Wolf FM,
- Leopold SS