MSS15: CITATION VERSUS DISRUPTION IN THE MILITARY: ANALYSIS OF THE TOP DISRUPTIVE MILITARY RESEARCH PUBLICATIONS
Joshua Dilday, DO1; Shea Gallagher, MD1; Ryan Bram, MD2; Elliot Williams, MD1; Areg Grigorian1; Brent Emigh1; Kazuhide Matsushima, MD1; Morgan Schellenberg, MD1; Kenji Inaba, MD1; Matthew Martin, MD1; 1LAC+USC Medical Center; 2Tripler Army Medical Center
Objectives: The analysis of surgical research using bibliometric measures has recently become increasingly prevalent. Citation counts (CC) or indices are commonly utilized markers of research quality, but may not adequately capture the most impactful military research. In an effort to identity innovative research, a novel disruption score (DS) was developed to capture literature that changes paradigms or future research directions. We sought to analyze the most disruptive and developmental versus most cited research in the military trauma literature.
Methods: The top scientific trauma papers by DS and by CC were identified via a professional literature search. All publications in military journals were included. Military trauma-related keywords were used to query additional top surgical journals for military-focused publications. The identified publications were then linked to the iCite NIH tool to quantify total CC and related metrics. The top 100 disruptive (DR) and developmental (DV) publications by DS were analyzed based upon the area of focus, citation, and perceived clinical impact, and then compared to the top 100 papers by CC.
Results: A total of 32,040 articles published between 1954-2014 were identified. The average DS and CC were 0.01 and 22, respectively. Most papers were published in Mil Med (68%). The top 100 disruptive papers were frequently published in Mil Med (48%) and Lancet (18%), with mean DS of 0.31. The most disruptive paper was the 4th most cited paper; the most cited article was the 7thmost disruptive. The most cited papers averaged a DS of 0.02 and were commonly found in JAMA (28%) and J Trauma (27%). CC papers had significantly more citations than both the DR and DV papers (514 vs. 103 vs. 96, p<0.05) Only 5 publications were on both the top 100 DR and top 100 CC lists, and 6 were on both the top DV and CC lists. CC was not correlated with DV (r=.13; p=.18) and only weakly correlated with DR (r=.26; p<0.05).
Conclusion: Disruption score is a novel metric to identify publications that changed military paradigms and future research directions that are not captured when using standard citation counts. The most disruptive and developmental papers are distinct and have little overlap with the top military papers identified by standard citation count metrics, with only a weak correlation between the citation counts and disruption scores. Multiple bibliometric measures should be employed in order to avoid overlooking impactful military trauma research.