The Pediatric Stroke Outcome Measure
A Validation and Reliability Study
Background and Purpose—The Pediatric Stroke Outcome Measure (PSOM) is an objective, disease-specific outcome measure containing 115 test items suitable for newborn to adult ages. The PSOM measures neurological deficit and function across 5 subscales: right sensorimotor, left sensorimotor, language production, language comprehension, and cognitive/behavior yielding a final 10-point deficit score. The goal of this study was to examine PSOM construct validity in measuring neurological outcome in pediatric stroke survivors and interrater reliability (IRR) for both prospective and retrospective scoring.
Methods—For construct validity, PSOM subscale scores were correlated with scores on standardized neuropsychological measures matched by functional domain. We assessed IRR by comparing same-day “live” PSOM scores from 2 independent raters in 10 children (prospective IRR) and by comparing PSOM scores estimated from medical dictations across 5 raters in another 10 children (retrospective IRR).
Results—We analyzed PSOM scores from 203 children with ischemic stroke. PSOM subscales show good construct validity (ρ=0.2–0.4; P<0.05). PSOM subscale scores of normal/abnormal demonstrate strong agreement for domain-matched neuropsychology scores (alternative chance-corrected statistic=0.4–0.8). IRR was excellent with the 2 prospective raters' scores in almost perfect agreement (intraclass correlation coefficient, 0.93; 95% CI, 0.76–0.98). Retrospective IRR demonstrated strong agreement with an intraclass correlation coefficient of 0.77 (95% CI, 0.56–0.92).
Conclusions—The PSOM is a valid and reliable outcome measure for pediatric stroke. It is useful for retrospective scoring from health records and prospective serial longitudinal outcome assessments and is ideally suited for prospective clinical trials in pediatric stroke.
Stroke during childhood is an increasingly recognized cause of significant long-term morbidity that creates a substantial burden of illness per affected individual.1 Childhood stroke incidence is 5 to 8 per 100 000 children annually with approximately 50% ischemic including arterial ischemic stroke (AIS) or cerebral sinovenous thrombosis (CSVT).2,3 Neonatal incidence is higher at 1 in 3000 to 5000 live births.4,5 Resultant neurological deficits are reported in 50% to 90% of children and include motor, language, and cognitive deficits.1,6–10 Outcomes research and clinical trials require a feasible, valid, and reliable outcome measure in pediatric stroke.
Outcome measures developed for diffuse cerebral pediatric disorders including cerebral palsy, HIV, adrenoleukodystrophy, and head trauma1 may be insensitive to the focal and sometimes mild deficits that result from pediatric stroke. The Rankin Scale, Barthel Index, and other adult stroke scales11 are not applicable to young children due to their reliance on self-reporting and independence in activities of daily living. Finally, across the adult years, expected performance is similar in contrast to children in whom abilities change considerably with maturation from infancy to teenage years.
The Pediatric Stroke Outcome Measure (PSOM) is an objective disease-specific measure of neurological recovery after childhood stroke. The PSOM was developed and implemented from 1994 in a prospective outcome study in our institutional Children's Stroke Clinic in Toronto, Canada.1,12 Currently, the PSOM is also in use in the International Pediatric Stroke Study13 and multiple other pediatric stroke studies.8,14,15
In the current study we sought to analyze the validity and reliability of the PSOM.
Beginning in 1994, children diagnosed with acute AIS or CSVT at the Hospital for Sick Children, Toronto, Canada, were prospectively enrolled in a longitudinal pediatric stroke outcome study. Children with outcome assessments completed before May 31, 2010, were included in the current PSOM validation study. Subjects included children aged newborn to 18 years. Parental consent and child assent (when appropriate) were obtained for all study subjects.
The diagnosis of AIS and CSVT met previously published clinical and radiographic criteria.1 AIS required a sudden-onset focal neurological deficit1 (for infants <6 months age, seizures or lethargy was sufficient) and corroborating corresponding neuroimaging evidence of focal infarction conforming to an established cerebral artery territory. Children with delayed diagnosis of perinatal AIS presenting with pathological hand dominance/hemiparesis <1 year age were also included. For CSVT, the diagnosis required seizures, lethargy or focal neurological deficits, and radiographic demonstration of cerebral vein or sinus occlusion either with or without parenchymal bleeding or infarction.
Pediatric Stroke Outcome Measure
The PSOM was developed, piloted, and refined by 3 pediatric neurologists in the Children's Stroke Clinic (G.d.V., D.M., R.C.). It was adapted, with permission, from a neurological outcome scale developed by Dr Isabel Rapin and a pediatric stroke adjudication questionnaire.16–18 The PSOM represents a structured, classical, pediatric neurological examination containing 115 test items encompassing cognition, language, cranial nerve, motor, sensory, cerebellar, and gait functions. Test items are organized sequentially across development from early infancy to teenage years with a scoring option of “not age-appropriate” for each item. For infants <2 years, primitive reflexes and developmental examination items are included. Completion time for examination and scoring averages 20 minutes.
On completion of the PSOM examination, the neurologist scores a Summary of Impressions containing 5 subscales: right sensorimotor, left sensorimotor (each with subcategories), language production, language comprehension, and cognitive/behavioral. Subscale scoring is 0 (no deficit), 0.5 (mild deficit, normal function), 1 (moderate deficit, decreased function), or 2 (severe deficit, missing function). The PSOM total score is the sum of the 5 subscale scores and ranges from 0 (no deficit) to 10 (maximum deficit).
Children were serially examined with the PSOM in the Children's Stroke Clinic concurrent with clinically indicated visits at 3 to 6 months and 12 months poststroke and thereafter at 2- to 5-year intervals to 18 years of age. Serial and parallel standardized neuropsychological and quality-of-life outcomes were also obtained within the main outcome study.19–23
Children with a neuropsychological assessment and a PSOM completed within a specified intertest time interval were included in the current study. The required maximum intertest interval varied over time from the acute stroke to account for the shifting rate of neurological changes over time. Maximum intertest time intervals were: for <6 months poststroke, 1-month intertest interval; 6 to 12 months poststroke, 6-month interval; 1 to 4 years poststroke, 1-year interval; and ≥4 years poststroke, 2-year interval. The PSOM/neuropsychological test pair most remote from the acute stroke was selected.
To assess construct validity, PSOM subscale scores were compared with standardized neuropsychological measures matched to each subscale based on functional domain/content relevance. These domain-matched combinations were defined a priori (Table 1). Descriptions of the neuropsychological measures are in the online-only Data Supplement Appendix I. Correlation of PSOM subscale scores with scores from the corresponding neuropsychological measures was analyzed using Spearman correlation. The predictive value of PSOM was assessed using linear regression (neuropsychological standardized test scores as the dependent variable and PSOM subscale scores as the independent variable). Using the linear regression model, we defined misclassification rate as percentage of observed values that are not within the predicted values 95% CI. We used PSOM subscale raw scores (4 levels) and standardized used measures scores for correlation and prediction analyses. For agreement analysis, we used an alternative chance-corrected statistical test with PSOM subscale scores dichotomized to normal (PSOM=0) and abnormal (PSOM >0) and neuropsychological standardized test scores dichotomized as normal and abnormal (defined as >1 SD below the mean). Correlation or agreement was defined as poor for coefficient <0.25, fair to moderate for coefficient >0.25 but <0.75, and excellent for coefficient >0.75.
For validation of retrospective PSOM scoring from medical records, we compared PSOM subscale scores obtained by “live” in-clinic PSOM examination on 10 children with PSOM subscale scores generated by 5 raters from medical dictations on the same children (Spearman correlation). The abstracters included 3 staff pediatric neurologists, a senior pediatric neurology resident, and an allied health professional member of the stroke team (PhD in Occupational Therapy). Raters had varying levels of experience with the PSOM ranging from 15 years (Stroke Clinic Medical Director) to no prior experience (neurology resident visiting Stroke Clinic). For each patient, the dictation notes from stroke clinic visits were photocopied, edited to remove identifying patient information, and then photocopied again. These copies were then distributed to the 5 raters who estimated scores for the 5 PSOM subscales.
We assessed interrater reliability (IRR) in 2 ways. First, as previously published,1 10 children underwent same-day in-clinic PSOM examinations by 2 neurologists (G.d.V. and R.C.) blinded to each other's ratings (prospective IRR). Second, we assessed IRR of PSOM scores abstracted from medical dictations on another 10 children by 5 independent raters (retrospective IRR; same 10 children used for validation of retrospective scoring).
For both methods of reliability testing, PSOM scores were analyzed as raw scores (4 levels for subscales and 0–10 for PSOM total scores). Intraclass correlation coefficient (ICC) using 2-way random effects model and absolute agreement definition was used to estimate scoring agreement for PSOM ordinal data (subscale scoring from 0 to 2). Interrater agreement definitions were for poor ICC=0.0 to 0.2, fair ICC=0.3 to 0.4, moderate ICC=0.5 to 0.6, strong ICC=0.7 to 0.8, and almost perfect ICC >0.8.
Statistical packages used were SPSS Version 20 (SPSS Science, Chicago, IL) and SAS Version 9.3 (SAS Institute, Cary, NC).
The study sample consisted of 203 children including 156 (76.8%) with AIS and 47 (23.2%) with CSVT (Table 2). Age at stroke ranged from newborn to teenage years. There were 49 (24%) neonates (including presumed perinatal ischemic stroke) and 124 (61.1%) males. Mean interval between stroke and PSOM was 4.5 years (range, 0–13.8 years). Intervals from PSOM testing to neuropsychological tests were within 6 months in 40% of subjects and within 12 months in 85% (details in online-only Data Supplement Table I). PSOM total scores over time for the study sample are presented in Figure 1. PSOM total scores tended to cluster at the lower end of the 0 to 10 range with approximately one third scoring 0 of 10 and another approximately 20% scoring 0.5. The remainder had moderate to severe deficits with scores ranging from of 1 to 7. Fewer than 2% scored >5.
As predicted, fair to moderate correlations were found for the majority of the PSOM–neuropsychological test pairings (Table 1). Scores on the sensorimotor right and sensorimotor left subscales were significantly correlated with performance on the Grooved Pegboard Test for the right hand and left hand, respectively (P<0.001). Scores on the PSOM language production subscale correlated significantly with performance on standardized neuropsychological tasks of verbal fluency (P=0.032 and P=0.008) and expressive vocabulary (P=0.001) as well as standardized questionnaire reports from parents regarding functional communication (P=0.014). PSOM scores on the language comprehension subscale correlated significantly with performance on standardized neuropsychological tasks of receptive vocabulary (P<0.001) and standardized questionnaire reports from parents regarding functional communication (P=0.01). Of note, the PSOM language production and comprehension scores (online-only Data Supplement Figure I) were not significantly correlated with verbal IQ, suggesting that these subscales specifically evaluate language expressive and comprehensive functioning and not general cognitive functioning. Finally, the PSOM cognitive/behavioral subscale was significantly associated with standardized neuropsychological measures of overall intellectual ability (P<0.001), verbal reasoning (P=0.011), perceptual reasoning (P=0.004), mental manipulation of information (P<0.001), and visuomotor processing speed (P<0.001). In addition, the cognitive–behavioral subscale was significantly associated with standardized parent questionnaire reports regarding overall executive function (P<0.001), behavior regulation (P<0.001), social skills (P=0.001), and functional independence at home (P=0.031). The relationship between PSOM language comprehension and the Peabody Picture Vocabulary Test is presented in online-only Data Supplement Figure I. Because of the large number of correlations calculated, we have included only this 1 as an example. For the majority of the correlations (21 of 25), the linear regression model was statistically significant (Table 1). Misclassification rates were also low (range, 1%–7%), further supporting the predictive validity of the PSOM subscales.
There was moderate agreement between normal/abnormal PSOM subscale scores with scores on corresponding domain-matched neuropsychological measures (Table 1). For each corresponding pair, the percent of overall agreement ranged from 49% to 86% with the 2 language and 2 sensorimotor subscales showing stronger agreement with neuropsychological measures (alternative chance-corrected=0.45–0.82) than the cognitive–behavioral subscale (alternative chance-corrected=0.42–0.63 for overall executive function and social skills in comparison to alternative chance-corrected=−0.02 to 0.36 for behavior regulation and functional independence at home). This finding is consistent with the broad range of difficulties and symptoms that could result in a designation of abnormal on the cognitive–behavioral subscale.
Validation of Retrospective PSOM Scoring
PSOM scoring abstracted from medical dictations was valid compared with same-day live PSOM scoring for right sensorimotor (ρ=0.86, P=0.002), left sensorimotor (ρ=0.84, P=0.005), language production (ρ=0.76, P=0.011), and cognitive and behavioral (ρ=0.86, P=0.002) subscales. Correlations for language comprehension failed to reach significance.
The IRR analyses included 20 children (10 for prospective and another 10 for retrospective IRR) with age at stroke ranging from newborn to age 13 years, deficit severities ranging from normal (PSOM total score 0) to “severe deficit” (total PSOM >0), and including AIS and CSVT stroke types (details in online-only Data Supplement Table II).
Prospective IRR was excellent with the 2 raters' prospective total scores in almost perfect agreement (ICC=0.93; 95% CI, 0.76–0.98;Table 3).
Retrospective IRR of PSOMs abstracted from medical dictations was also strong with an ICC of 0.77 (95% CI, 0.56–0.92) for total PSOM score. Figure 2 shows the PSOM score distribution by rater. There is strong agreement among the 5 raters in the following functional domains: right sensorimotor (ICC=0.78; 95% CI, 0.57–0.93), left sensorimotor (ICC=0.79; 95% CI, 0.55–0.94), language production (ICC=0.73; 95% CI, 0.50–0.91), and cognitive and behavioral (ICC=0.96; 95% CI, 0.90–0.99) subscales. Poor agreement is observed for language comprehension with ICC=0.19 (95% CI, −0.06 to 0.67; Table 4).
The current study demonstrates, in a large cohort of children with pediatric stroke, that the PSOM is both valid and reliable. These findings are important because the PSOM is the only disease-specific measure of neurological outcome for pediatric stroke and is currently in use in multiple research studies of this population.1,8,13,15
Our construct validity analyses confirm that the PSOM subscale scores demonstrate relevant functional impairments in cognition/behavior, sensorimotor, and language abilities. The PSOM cognitive–behavioral subscale significantly correlated with standardized neuropsychological measures of overall intellectual ability, verbal and perceptual reasoning, and parental behavior/cognitive questionnaires.
Fair to moderate correlations between PSOM and neuropsychological scores were expected, because these 2 measures evaluate outcome in different ways. Specifically, our results indicate that PSOM is an effective screening tool for significant sensorimotor, language, and cognitive–behavioral deficits resulting from pediatric stroke. In contrast, neuropsychological assessment taps into the same domains as those evaluated on the PSOM but in a much more detailed and objective manner such that more subtle weaknesses can be detected. Full standardized neuropsychological assessments require significant financial and time resources and the use of the 20-minute PSOM examination in screening children for cognitive–behavioral deficits is therefore of value. However, a child with a normal PSOM cognitive–behavioral subscale score could still have subtle weaknesses in cognition, behavior, or complex aspects of information processing that are only detectable on formal neuropsychological assessment. In addition, the narrow rating scale of the PSOM (ranging from 0 to 2) also contributes to findings of moderate correlations in this study.
Finally, our article demonstrates that PSOM scores estimated from health records closely approximated in-clinic PSOM scores, validating this method of PSOM scoring. Both in-clinic and chart-abstracted PSOM scoring demonstrated excellent reliability. These findings support the use of the PSOM for quantifying neurological function from past health records.
Accurate outcome data are essential for informing prognosis, treatment prioritization, and selecting clinical trial end points in pediatric stroke.1 The PSOM evaluates the direct impact of stroke on the child's neurological function. In focal brain injury, including stroke, objective neurological outcome measures using observation by skilled health professionals provide a direct measure of the severity of the brain injury. Treatments that are intended to directly reduce infarct severity, including neuroprotective or acute reperfusion treatments, are most likely to demonstrate their effects with the use of direct outcome measures. Indirect measures, including self-report for older children, or parental proxy report for younger children, are more likely to measure a combination of direct brain injury and indirect experiential aspects of the illness. We have previously demonstrated that parental impressions of a child's poststroke recovery frequently underestimate and overestimate that child's neurological deficits compared with a trained pediatric neurologist.1 This may be caused by the parents' incorporation of indirect effects into their assessment of a child's status. Indirect effects of the illness that are experienced by the child who experiences a stroke include pain from phlebotomy and other medical procedures, child and parental anxiety regarding the acute loss of function and anticipated future consequences, fear of a recurrent stroke, and, later, the emotional and social consequences of challenges in reintegration of the impaired child. Studies of these indirect effects of stroke including quality-of-life evaluations require indirect measures, including child-report or parent-report outcome measures. In quality of life and other studies of the full impact of stroke, the use of the PSOM can elucidate the contribution of the actual infarct severity to the child's quality of life.
There are a number of limitations to the current study. First, not all consecutive children in the stroke clinic consented to neuropsychological testing. However, because this is a validation study, the sample need not be completely representative of the children seen over the study interval. In fact, we did obtain data on a wide range of ages, stroke types, and deficit severities in the sample used for this study. Moreover, the results from the neuropsychological tests were normally distributed, which downplays the concerns regarding referral bias. Another limitation is that the PSOM is likely to be biased toward motor and sensory impairments as opposed to cognitive, language and behavioral deficits. This could result from the fact that the PSOM is administered by trained neurologists, and individual test items are more frequently focused on motor and sensory impairments. One area for future development might be to expand the cognitive–behavioral component of the PSOM to better capture these deficits. Difficulties in behavior, emotions, and peer relationships interfere with everyday life occur in approximately half of all children with hemiplegia after stroke.24,25
In conclusion, the current study demonstrates that the PSOM demonstrates strong measurement domains and is a valid and reliable measure of deficit types and neurological impairment after pediatric stroke. The PSOM is ideal for use in both outcomes studies and clinical trials geared to improving outcomes in pediatric stroke.
Sources of Funding
Supported by the Auxilium Foundation and The Heart and Stroke Foundation of Ontario.
We are extremely grateful to the many Pediatric Stroke Fellows from the Hospital for Sick Children who have contributed so much to the PSOM database, to our patients, and to our program since 1998. Thank you to our patients and families for their participation in research. Thank you to Ann-Marie Pontigon and Jeffrey Templeton for expert statistical assistance. The PSOM form and instructions are available by e-mailing the senior author at.
The online-only Data Supplement is available with this article at http://stroke.ahajournals.org/lookup/suppl/doi:10.1161/STROKEAHA.111.639583/-/DC1.
- Received October 3, 2011.
- Accepted February 10, 2012.
- © 2012 American Heart Association, Inc.
- deVeber G,
- MacGregor D,
- Curtis R,
- Mayank S
- Lynch JK,
- Hirtz DG,
- deVeber G,
- Nelson KB
- Agrawal N,
- Johnston SC,
- Wu YW,
- Sidney S,
- Fullerton HJ
- Govaert P,
- Matthys E,
- Zecic A,
- Roelens F,
- Oostra A,
- Vanzieleghem B
- Gordon AL,
- Ganesan V,
- Towell A,
- Kirkham FJ
- Kwon S,
- Hartzema AG,
- Duncan PW,
- Min-Lai S
- Golomb MR,
- Fullerton HJ,
- Nowak-Gottl U,
- deVeber G
- Beslow LA,
- Licht DJ,
- Smith SE,
- Storm PB,
- Heuer GG,
- Zimmerman RA,
- et al
- Lo W,
- Zamel K,
- Ponnappa K,
- Allen A,
- Chisolm D,
- Tang M,
- et al
- Rapin I
- Rapin I
- Rapin I
- Rapin I
- McLinden A,
- Baird AD,
- Westmacott R,
- Anderson PE,
- deVeber G
- Hetherington R,
- Tuff L,
- Anderson P,
- Miles B,
- deVeber G
- Friefeld SJ,
- Westmacott R,
- MacGregor D,
- Deveber GA
- Westmacott R,
- MacGregor D,
- Askalan R,
- deVeber G
- Goodman R