(Stroke. 1998;29:871-872.)
© 1998 American Heart Association, Inc.
Interrater Agreement on a Simple Neurological Score in Rats
Leonardo Pantoni, MD
Department of Neurological and Psychiatric Sciences
Luciano Bartolini, PhD
Department of Preclinical and Clinical Pharmacology
Giovanni Pracucci, MD;
Domenico Inzitari, MD
Department of Neurological and Psychiatric Sciences,
University of Florence,
Florence, Italy
To the Editor:
Strong and reliable outcome measures are required in laboratory studies
that aim to appraise the extent of the damage in animals subjected to
various forms of cerebral ischemia. In this regard, the
assessment of histological changes, such as the volume
of infarcted tissue or the number of necrotic cells, is considered the
gold standard. However, the assessment of functional outcome can also
be useful in animal studies that evaluate the effect of new therapeutic
agents, since the clinical examination is effortless and not time
demanding; moreover, physical testing of the animals can be repeated
over time and thus provide data on the evolution of the neurological
deficit. A simple neurological score to evaluate sensorimotor
performance in rats has recently been developed by Garcia et
al.1 It explores six different functions and
attributes to each a 3- or 4-point score. The total score, which
correlates closely with the severity of the
histological injury (in particular with the number of
necrotic neurons) in a model of middle cerebral artery (MCA) occlusion
in the Wistar rat,1 has been used as one of the
outcome measures in studies that estimate the effect of new
drugs.2 3 Because of its simplicity and strong
correlation with histological damage, this score might
be used in other laboratories employing experimental procedures to
cause cerebral ischemia. In this regard, evaluation of its
reproducibility could be extremely useful, because it is known that the
validity of scales evaluating neurological deficit can be affected by
interobserver variability.4 5
We have conducted a study to evaluate the impact of personal
judgment in the use of this scale and to explore which of the six
different functions might be particularly affected by interobserver
variability. Thirty-one male Charles River Wistar rats weighing 270 to
310 g were studied. Twenty rats had permanent occlusion of the
right MCA through use of an intraluminal
filament,6 2 had a sham operation for MCA
occlusion (as in the preceding group but with the filament withdrawn
within 30 seconds), 3 underwent ligation of the right common carotid
artery, 3 underwent a bilateral ligation of the common carotid artery,
2 had a sham operation for carotid artery ligation (ie, exposure of the
arteries in the neck), and 1 was a normal control. All the surgical
procedures were carried out under general anesthesia with
halothane.
Two observers consecutively and independently carried out the
neurological examination of each rat 1 to 3 days after surgery,
according to published guidelines.1 The
investigators, both experienced in laboratory procedures involving
animals, were blinded to the surgical procedure that the animals had
undergone, with the exception of the normal control animal (who was
easily identifiable by absence of skin incision). To measure the level
of interobserver agreement, we used the weighted 
coefficient, based
on a formula proposed by Cohen.7 Conventionally,
values are considered as follows: 0.01 to 0.20, slight; 0.21 to
0.40, fair; 0.41 to 0.60, moderate; 0.61 to 0.80, substantial; and 0.81
to 0.99, almost perfect.8 (A value of +1
indicates perfect agreement.) The statistical significance was
evaluated by means of 95% confidence limits.
Considering the entire group of animals, the interobserver agreement
was substantial for each of the items and almost perfect for total
score. When the analysis was restricted to the rats with
permanent MCA occlusion (ie, those in which sensorimotor deficits were
present), the agreement worsened, in particular on the items
"movements symmetry" and "body proprioception"; the agreement
on the total score remained substantial (Table
)
.
View this table:
[in this window]
[in a new window]
|
Table 1. Interobserver Agreement on Each of the Six Sensorimotor
Functions and Total Score of the Scale of Garcia et
al1
|
|
Whereas previous papers correlated the extent of neurological deficit
following disparate cerebral ischemic procedures with the area
or volume of infarct,9 10 the sensorimotor score
developed by Garcia et al1 shows a strong
correlation with the number of necrotic neurons in rats with permanent
or transient MCA occlusion. Therefore, it might also represent
a useful tool for the assessment of functional outcome in animals with
limited ischemic brain damage. Testing by this method does not
require training of the rats or purchase of expensive equipment. In
addition, results can be expressed in a numerical score compatible with
analysis by statistical means. In the original
article1 the extent of rat neurological deficit
was assessed by two raters, but data on interobserver variability were
not provided. Interrater reliability is an essential requirement for a
scale that measures functional outcome in neurological assessment.
Differences reported can be accounted for by the interrater variability
instead of true variations in the observed phenomenon. This has been
emphasized in studies evaluating interobserver variability in the
neurological examination of stroke patients.5 11
Our study shows that the reliability of this brief rat neurological
scale is fair, although not completely free from interrater
variability, particularly in such items as movements symmetry and body
proprioception. We believe that this variability may partly depend on
dissimilar behavior of the animals in different moments rather than on
raters' variability. This flaw is in part lessened by the substantial
agreement on the total score that is the only score finally used.
Future studies based on the premise that a certain agent is
effective in reducing the ischemic insult to the brain
parenchyma should be based on (1 ) the knowledge of the chronology and
topography of the lethal neuronal injury as it exists in the absence of
therapeutic intervention; (2 ) a measure of the degree of the
neurological deficit induced by the injury; and (3 ) a verification of
the close relationship between (1 ) and (2 ). Based on the above-reported
characteristics of the scale and its rather fair reliability, we
suggest that the score of Garcia et al may be used by researchers to
evaluate the functional outcome of rats undergoing a variety of
experimental procedures aimed at provoking an insult to the brain
parenchyma that results in a sensorimotor deficit.
References
1.
Garcia JH, Wagner S, Liu K-F, Hu X-j. Neurological
deficit and extent of neuronal necrosis attributable to middle cerebral
artery occlusion in rats: statistical validation.
Stroke. 1995;26:627–635.[Abstract/Free Full Text]
2.
Garcia JH, Liu K-F, Relton JK. Interleukin-1 receptor
antagonist decreases the number of necrotic neurons in rats
with middle cerebral artery occlusion. Am J Pathol. 1995;147:1477–1486.[Abstract]
3.
Garcia JH, Liu K-F, Bree MP. Effects of CD11b/18
monoclonal antibody on rats with permanent middle cerebral artery
occlusion. Am J Pathol. 1996;148:241–248.[Abstract]
4.
Sisk C, Ziegler DK, Zileli T. Discrepancies in
recorded results from duplicate neurological history and
examination in patients studied for prognosis in cerebral vascular
disease. Stroke. 1970;1:14–18.[Abstract/Free Full Text]
5.
Tomasello F, Mariani F, Fieschi C, Argentino C, Bono
G, De Zanche L, Inzitari D, Martini A, Perrone P, Sangiovanni G.
Assessment of inter-observer differences in the Italian Multicenter
Study on Reversible Cerebral Ischemia. Stroke. 1982;13:32–35.[Abstract/Free Full Text]
6.
Zea Longa E, Weinstein PR, Carlson S, Cummins R.
Reversible middle cerebral artery occlusion without craniectomy in
rats. Stroke. 1989;20:84–91.[Abstract/Free Full Text]
7.
Cohen J. Weighted kappa: nominal scale agreement with
provision for scaled disagreement or partial credit. Psychol
Bull. 1968;70:213–220.[Medline]
[Order article via Infotrieve]
8.
Landis JR, Koch GG. The measurement of observer
agreement for categorical data. Biometrics. 1977;33:159–173.[Medline]
[Order article via Infotrieve]
9.
Bederson JB, Pitts LH, Tsuji M, Nishimura MC, Davis
RL, Bartkowski H. Rat middle cerebral artery occlusion: evaluation of
the model and development of a neurological examination.
Stroke. 1986;17:472–476.[Abstract/Free Full Text]
10.
Persson L, Hårdemark H-G, Bolander HG, Hillered L,
Olsson Y. Neurologic and neuropathologic outcome after middle cerebral
artery occlusion in rats. Stroke. 1989;20:641–645.[Abstract/Free Full Text]
11.
Shinar D, Gross CR, Mohr JP, Caplan LR, Price TR, Wolf
PA, Hier DB, Kase CS, Fishman IG, Wolf CL, Kunitz SC. Interobserver
variability in the assessment of neurologic history and examination in
the Stroke Data Bank. Arch Neurol. 1985;42:557–565.[Abstract/Free Full Text]
Response
Julio H. Garcia, MD
Departments of Pathology,
Case Western Reserve University,
and Henry Ford Hospital,
Detroit, Michigan
The study by Pantoni and associates described above adds an
important evaluation of the interrater agreement on the method to
assign a neurological score originally designed to determine whether
there exists a reliable correlation between the extent of sensorimotor
deficit (induced in rats by a middle cerebral artery occlusion) and the
numbers of necrotic neurons identified by histological
methods 7 days after the original injury.1 From these and
additional studies we emphasize the following important issues.
(1 ) In this species (male Wistar rats from Charles River; identified
as Cr1 (WI)Br), permanent occlusion of one middle cerebral
artery induces pannecrosis (infarction) of the entire
arterial territory only 3 to 4 days after the
injury.2
(2 ) Necrotic neurons (defined by histological
criteria) appear quickly (within 12 hours) in the striatum, while in
the cortex large numbers of necrotic neurons become identifiable only
several (2 to 3) days later.3
(3 ) Reopening the artery after 60 minutes induces a brain lesion whose
features are remarkably different from those of the infarction
(pannecrosis) that develops after 4 days in all rats in which the
artery was not reopened.4
(4 ) The "area of pallor" that becomes visible in sections stained
with hematoxylin-eosin 24 to 48 hours after a permanent
arterial occlusion never appears in brains subjected to
transient arterial occlusion.2 4
Thus, under these conditions it is essential to evaluate numbers of
necrotic neurons as a logical end point.
(5 ) In experiments based on short-term (<30 minutes)
arterial occlusions followed by long-term (up to 28 days)
reperfusion, there is a lapse of 3 to 4 days between the time of the
injury and the appearance of necrotic neurons in the
cortex.4 This emphasizes once more the wisdom or the
necessity of relying on counts of necrotic neurons as an end point that
accurately reflects the degree of brain injury caused by
arterial occlusions of variable duration.
We are pleased to learn that an independent laboratory has verified the
reliability of this simple method to evaluate sensorimotor responses in
the rat.
References
1.
Garcia JH, Wagner S, Liu K-F, Hu X-j. Neurological
deficit and extent of neuronal necrosis attributable to middle cerebral
artery occlusion in rats: statistical validation. Stroke. 1995;26:627–635.
2.
Garcia JH, Yoshida Y, Chen H, Li Y, Zhang ZG, Lian L,
Chen S, Chopp M. Progression from ischemic injury to infarct
following middle cerebral artery occlusion in the rat. Am J
Pathol. 1993;142:623–635.[Abstract]
3.
Garcia JH, Liu K-F, Ho K-L. Neuronal necrosis after
middle cerebral artery in Wistar rats progresses at different time
intervals in the caudoputamen and the cortex.
Stroke. 1995;26:636–643.[Abstract/Free Full Text]
4.
Garcia JH, Liu K-F, Ye Z-R, Gutierrez JA. Incomplete
infarct and delayed neuronal death after transient middle cerebral
artery occlusion in rats. Stroke. 1997;28:2303–2310.[Abstract/Free Full Text]
