Published online before print August 24, 2006, doi: 10.1161/01.STR.0000240509.05587.a2
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
(Stroke. 2006;37:2588.)
© 2006 American Heart Association, Inc.
Original Contributions |
Frequency and Determinants of Nonpublication of Research in the Stroke Literature
From the University of California at Los Angeles (UCLA) Stroke Center (N.S., J.L.S., L.I.K., B.O.), Calif; and the Department of Neurology (N.S., A.G.O., J.L.S., B.O.), UCLA Medical Center, Calif.
Correspondence to Nerses Sanossian, MD, Department of Neurology, University of Southern California, LAC+USC Medical Center, 1200 N State St, #5640, Los Angeles, CA 90033. E-mail sanossia{at}yahoo.com
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionReferences |
|---|
Background and Purpose— Selective nonpublication will yield publication bias and a published literature imperfectly representative of the full range of scientific findings. We evaluated the proportion of research abstracts presented at the leading United States research meeting in stroke, the International Stroke Conference (ISC), which were subsequently published as full-length articles and investigated the factors associated with full manuscript publication.
Methods— Features of all abstracts presented at the annual ISC meeting in February 2000 were analyzed. Search of the National Library of Medicine PubMed database and written communication with abstract authors was performed to determine conversion of abstracts to fully published manuscripts over the subsequent 5 years.
Results— Among the 353 abstracts presented at the 2000 International Stroke Conference, 108 were oral presentations and 245 posters. Overall, 202/353 (62.3%) resulted in full-length publications, with a median time to publication of 15 months. In multivariate analysis, factors increasing likelihood of full-length publication were: platform rather than poster presentations (odds ratio [OR] 3.0, 95% CI, 1.6 to 5.5), authors with a university affiliation (OR 2.2, 95% CI, 1.2 to 4.1), and European region of origin (OR 2.2, 95% CI, 1.1 to 4.4), whereas topic concerning community/risk factors decreased the likelihood of publication (OR 0.3, 95% CI, 0.16 to 0.74). Positive results, multicenter collaboration and industry sponsorship did not affect publication rate.
Conclusions— Approximately 1 of every 3 abstracts presented at an international stroke meeting was not published as a full manuscript within 5 years. Poster abstracts were less likely to be published in full manuscript form than oral presentations.
Key Words: nonpublication • publication rates • stroke
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Several abstracts presented at annual scientific meetings never come to full manuscript publication in peer-reviewed journals.1 Publication bias toward studies that favor new therapies and positive outcomes has been known to occur for the past 40 years, perhaps leading to a published literature imperfectly representative of the full range of scientific findings.2 Indeed, the increasing use of meta-analyses underscores the need to ensure that published literature represents the totality of evidence when addressing treatment questions. A prior study demonstrated evidence of modest publication bias in the stroke clinical trial literature, with negative trials published less often than statistically expected.3
In this study, we undertake a detailed investigation of the key transition step from published abstract to published full-length manuscript among a broad corpus of investigations, not just controlled clinical trials, in order to obtain further insight into determinants of nonpublication in the cerebrovascular literature. We aimed to determine the frequency and causes of nonpublication of research presented in abstract form at the annual International Stroke Conference (ISC) of the American Stroke Association (ASA) in the year 2000.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
Data were collected on all abstracts accepted for presentation at the year 2000 ISC meeting, using a standardized form. This information included type of presentation (platform or poster), regions of origin for authors, pharmaceutical or industry sponsorship, sample size, ischemic versus hemorrhagic stroke, whether a treatment was studied, whether a treatment was standard of care (ie, intravenous tissue plasminogen activator or aspirin) or investigational, type of study (observational, clinical trial, experimental, etc), mode of data collection (prospective or retrospective), imaging modalities used, clinical outcome measures, and laboratory outcomes.
There is no standardized definition of a positive result across the broad range of study types in this analysis. We modified a commonly used definition4: results were positive if they met the following criteria: (1) the studied variable produced beneficial results, or supported the abstract hypothesis or objective, and either (2a) the probability value for the variable effect was reported as <0.05, or (2b) no test of statistical significance was reported regarding the variable effect.
The National Library of Medicine PubMed database was searched in August 2005 for evidence of subsequent publication of abstracts in full-manuscript form. Searches used pertinent abstract keywords and first, second, or last author names. Data on publication date, journal, and reference were collected. Time to manuscript publication was calculated relative to the February 2000 date of the ISC.
For all abstracts which were either not located in PubMed, or for which it was not clear whether the publication found related directly to the abstract, a standardized questionnaire was emailed to the abstract authors. Author contact information was obtained first by searching scholarly databases for other listed publications by abstract authors (names cross-verified with affiliations), by searching the American Academy of Neurology 2005 membership address book, and by using the Google search engine. If an email address was not available for the primary author, the email address of the second or last author on the abstract was used. Questionnaires were designed to confirm/disconfirm nonpublication and to determine whether full-length publication was planned or no longer planned.
The individual impact of each independent predictor variable was measured by Fisher Exact or 
2 to compute probability values. Fifteen variables were evaluated in a backward logistic regression to determine which factors influence publication rates. In the multivariate analysis, candidate variables entered into the model using a liberal P<0.15 criterion.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
A total of 353 abstracts were presented at the 2000 ISC and published in the January 2000 issue of Stroke: Journal of the American Heart Association. Of these, 108 were oral presentations and 245 were posters. On initial search of the PubMed database, 189 abstracts were identified as published as full articles. An additional 31 full articles were identified as published via questionnaires returned by the authors for a total of 202 published articles (62.3% of total).
The frequency of publication of abstracts for each factor studied for association with publication is listed in Table 1. Publication rates were greater for oral abstracts than posters (75.7% versus 56.3%, P=0.02). Publication rates varied substantially among different topic categories (P=0.03). The category "results of recent clinical trials" had the highest rate of publication, 100%, and "community/risk factors" the lowest, 48%.
|
Positive results were reported in 220/353 (62.3%) by the predefined criteria. Presentation of positive results did not affect publication rate (64.0% versus 59.4%, P=0.39). Positive results were not a factor in publication rate when controlling for type of presentation (oral 79.7% versus 69.8%, P=0.24; poster 57.5% versus 54.1%, P=0.61).
The majority of abstracts (71%) had a listed author affiliated with a United States institution. In addition 17% of abstracts had an author with European affiliation, 10% with Asian/Australian affiliation, 8% with Canadian affiliation and 0.3% with a South/Central American affiliation. Although author region was not associated with publication rate on univariate analysis, abstracts with European authors were more likely to be published relative to those with a US author in the multivariate model (odds ratio 2.2, 95% CI, 1.1 to 4.4).
The majority of abstracts had university affiliation (269/353, 76.2%). Abstracts with university affiliation tended to be published more frequently than those without (64.7% versus 54.8%, P=0.10). The 29 abstracts pertaining to novel therapies were published at a higher rate than those on other topics (82.8% versus 60.5%, P=0.018). Factors with no univariate association with publication rates were: sample size, prospective design, retrospective design, primarily ischemic stroke, primarily hemorrhagic stroke, multicenter collaboration, and industry sponsorship.
Abstracts were published as articles in 57 different journals. The most common journals of publication are listed in Table 2. The median time to publication from presentation was 15 months (interquartile range 7 to 27, range: 18 to 69 months; Figure). Among fully published studies, 73% were published within 2 years of the abstract presentation. Time to full length publication did not differ between posters and oral presentations (14 months versus 16 months, P=0.451).
|
|
Multivariate logistic regression analysis (Table 3) revealed that factors independently associated with increased likelihood of full length publication were oral presentation, university affiliation, and European region of origin. Community risk factors as a presentation category was independently associated with reduced likelihood of full-length publication. A trend to association with increased likelihood of full-length publication was seen for Canadian region of origin, presentation category of vascular pathophysiology and new therapy as study topic. Trends to association with reduced odds of full-length publication were observed for presentation categories of diagnosis and recovery.
|
Among the 161 abstracts initially identified as unpublished or without clear evidence of publication after initial PubMed database search, author email address was identified in 157 of 161. Repeated attempts were made to contact study authors with a standardized questionnaire to determine the fate of these abstracts. At least 1 author of 105/157 (66.8%) responded to the survey (Table 4). Based on author responses, 31 additional abstracts were determined to have been published and 74 were confirmed as not published. Of the 74 confirmed unpublished abstracts, 73% were never submitted for publication, 16% were submitted and rejected, and 11% were in various stages of submission, review, and revision. The commonly reported reasons for nonpublication are shown in Table 4 and included lack of time, lack of coauthor participation, on-going study, methodological limitations, other publications in the field, and low priority given to publication.
|
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
We found that 1 of every 3 abstracts presented at the 2000 ISC was not published as a full manuscript within 5 years. This nonpublication rate is relatively low compared with rates found in several other subspecialties (supplemental Table I, available online at http://stroke.ahajournals.org). A recent meta-analysis has shown an overall nonpublication rate of 56% for abstracts presented at academic meetings.1 The relatively lower rate of nonpublication for ISC stroke abstracts may reflect the high scientific quality of the conference, but may also in part reflect a high survey respondent rate bolstered by our repeated attempts to contact study authors to confirm full article publication.
|
Poster abstract presentations were less likely to be published than oral presentations, which is consistent with findings from prior studies.1 This difference likely reflects the fact that oral presentation abstracts were deemed by meeting peer reviewers to have the greatest relevance and scientific merit, as well as the overall greater quantity of poster abstracts compared with oral abstracts at scientific meetings.
Publication rates of abstracts varied by presentation category with vascular pathophysiology, experimental ischemia, and recent clinical trials abstracts published at high rates and community risk factors, recovery, and diagnosis abstracts at lower rates. A variety of factors likely underlie these differences, including greater competing duties for clinical than basic scientist researchers and a privileging of therapeutic over population and diagnostic research in the clinical research community.
The majority of abstracts were published as full articles within 2 years of presentation. Of note, however, publication of abstracts from the 2000 ISC continued through the 2005 year. At the time of the writing of this article, there were an additional 8 abstracts in submission, revision, or prepublication designation according to author survey results, so the final conversion to full-length publication rate among this corpus of studies may increase slightly by the end of the decade.
This study did not identify positive-outcome bias across the broad class of stroke publications studied. Positive-outcome bias refers to the preferential publication of research with positive outcomes.4 The lack of a positive-outcome bias in this study likely in part reflects study of only accepted abstracts and not submitted abstracts. It could well be that a positive-outcome bias may exist in the selection of abstracts for presentation to the ISC. Positive-outcome bias has been documented in acceptance rates of abstracts to national meetings in prior publications.4–6 In addition, lack of a positive-outcome bias may in part reflect the broad topic range of studies analyzed. Positive-outcome bias likely is more pronounced in therapeutic than observational, population, and basic science studies.
This study identified several barriers to full publication of abstracts. The most commonly cited reason for nonpublication of an abstract was lack of time, which probably reflects the pressures of clinical practice and obtaining ongoing competitive funding for research. Other important barriers to publication were the low priority given to publication by authors and a lack of coauthor participation in drafting manuscripts. Only a minority of authors attributed the nonpublication of their abstracts to methodological weaknesses.
This study has limitations. Data were collected only for accepted abstracts and do not reflect those abstracts submitted for presentation and rejected. Although the PUBMED database was thoroughly searched for evidence of publication, publications in nonindexed journals may have been mistakenly coded as unpublished. There were a small number of abstracts for which confirmation of nonpublication could not be elicited from authors and the actual publication rates may be slightly higher than reported. Because searches were performed based on first, second, and senior author names, it is conceivable that data presented in ISC abstracts was published by other coauthors, although we suspect that this happened infrequently. The study did not address the number of submissions needed to achieve final publication for individual abstracts.
In conclusion, although the overall rate of nonpublication of stroke abstracts presented at the 2000 ISC was comparatively low, there is certainly room for improvement because selective publication of abstracts has the potential to lead to bias in the stroke literature.
References 7 through 23 are cited in the supplemental Table I.
|
Acknowledgments |
|---|
Disclosures
None.
Received April 17, 2006; revision received May 30, 2006; accepted June 6, 2006.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionReferences |
|---|
1. von Elm E, Costanza M, Walder B, Tramer M. More insight into the fate of biomedical meeting abstracts: A systematic review. BMC Medical Research Methodology. 2003; 3: 12. Available at: http://www.biomedcentral.com/1471–2288/3/12. Accessed August 17, 2006.
2. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet. 1991; 337: 867–872.[CrossRef][Medline] [Order article via Infotrieve]
3. Liebeskind DS, Kidwell CS, Saver JL. Empiric evidence of publication bias affecting acute stroke clinical trials. Stroke. 1999; 30: 268A.
4. Callaham ML, Wears RL, Weber EJ, Barton C, Young G. Positive-outcome bias and other limitations in the outcome of research abstracts submitted to a scientific meeting. JAMA. 1998; 280: 254–257.
5. Miguel-Dasit A, Marti-Bonmati L, Sanfeliu P, Aleixandre R. Scientific papers presented at the European Congress of Radiology 2000: Publication rates and characteristics during the period 2000–2004. Eur Radiol. 2006; 16: 445–450.[CrossRef][Medline] [Order article via Infotrieve]
6. Preston CF, Bhandari M, Fulkerson E, Ginat D, Koval KJ, Egol KA. Podium versus poster publication rates at the Orthopaedic Trauma Association. Clin Orthop Relat Res. 2005; 437: 260–264.
7. Wang JC, Yoo S, Delamarter RB. The publication rates of presentations at major spine specialty society meetings (NASS, SRS, ISSLS). Spine. 1999; 24: 425–427.[CrossRef][Medline] [Order article via Infotrieve]
8. Boldt J, Maleck W. ‘Schicksal’ wissenschaftlicher beitrage auf grossen anasthesiekongressen. Anaesthesist. 1999; 48: 802–806.[CrossRef][Medline] [Order article via Infotrieve]
9. Bowrey DJ, Morris-Stiff GJ, Clark GW, Carey PD, Mansel RE. Peer-reviewed publication following presentation at a regional surgical meeting. Med Educ. 1999; 33: 212–214.[CrossRef][Medline] [Order article via Infotrieve]
10. Daluiski A, Kuhns CA, Jackson KR, Lieberman JR. Publication rate of abstracts presented at the annual meeting of the orthopaedic research society. J Orthop Res. 1998; 16: 645–649.[CrossRef][Medline] [Order article via Infotrieve]
11. Evers JL. Publication bias in reproductive research. Hum Reprod. 2000; 15: 2063–2066.
12. Gavazza JB, Foulkes GD, Meals RA. Publication pattern of papers presented at the american society for surgery of the hand annual meeting. J Hand Surg [Am]. 1996; 21: 742–745.[Medline] [Order article via Infotrieve]
13. Hamlet WP, Fletcher A, Meals RA. Publication patterns of papers presented at the annual meeting of the american academy of orthopaedic surgeons. J Bone Joint Surg Am. 1997; 79: 1138–1143.
14. Kiroff GK. Publication bias in presentations to the annual scientific congress. ANZ J Surg. 2001; 71: 167–171.[CrossRef][Medline] [Order article via Infotrieve]
15. Landry VL. The publication outcome for the papers presented at the 1990 ABA conference. J Burn Care Rehabil. 1996; 17: 23A–26A.[Medline] [Order article via Infotrieve]
16. Marx WF, Cloft HJ, Do HM, Kallmes DF. The fate of neuroradiologic abstracts presented at national meetings in 1993: Rate of subsequent publication in peer-reviewed, indexed journals. AJNR Am J Neuroradiol. 1999; 20: 1173–1177.
17. Murrey DB, Wright RW, Seiler JG, Day TE, Schwartz HS. Publication rates of abstracts presented at the 1993 annual academy meeting. Clin Orthop. 1999; 359: 247–253.
18. Nguyen V, Tornetta P, Bkaric M. Publication rates for the scientific sessions of the OTA. Orthopaedic Trauma Association. J Orthop Trauma. 1998; 12: 457–459.[CrossRef][Medline] [Order article via Infotrieve]
19. Riordan FA. Do presenters to paediatric meetings get their work published? Arch Dis Child. 2000; 83: 524–526.
20. Sanders DS, Carter MJ, Hurlstone P, Hoggard N, Lobo AJ. Full publication of abstracts presented at the british society of gastroenterology [Abstract]. Gastroenterology. 2000; 118: A217.
21. Eloubeidi MA, Wade SB, Provenzale D. Factors associated with acceptance and full publication of gi endoscopic research originally published in abstract form. Gastrointest Endosc. 2001; 53: 275–282.[Medline] [Order article via Infotrieve]
22. Jackson KR, Daluiski A, Kay RM. Publication of abstracts submitted to the annual meeting of the Pediatric Orthopaedic Society of North America. J Pediatr Orthop. 2000; 20: 2–6.[CrossRef][Medline] [Order article via Infotrieve]
23. Timmer A, Hildsen RJ, Cole J, Hailey D, Sutherland LR. Publication bias in gastroenterological research - a retrospective cohort study based on abstracts submitted to a scientific meeting. BMC Med Res Methodol. 2002; 2: 7.[CrossRef][Medline] [Order article via Infotrieve]
This article has been cited by other articles:
 |
|
 |
|
  K. Prasad Delays in publishing the results of clinical trials harm patients, and public health Practical Neurology, January 1, 2008; 8(1): 6 - 7. [Full Text] [PDF]
|
|
|
|
 |
|
 Minerva BMJ, October 28, 2006; 333(7574): 926 - 926. [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||