Letter by Iosa et al Regarding Article, “Reliability and Validity of Bilateral Ankle Accelerometer Algorithms for Activity Recognition and Walking Speed After Stroke”
To the Editor:
We read with great interest the article of Dobkin and colleagues1 recently published in Stroke. They proposed a method that could be very helpful to assess walking speed and mobility-related activity of people with stroke in the home and in the community. This method has the advantages2 of using 2 wearable, wireless, light, and low-cost triaxial accelerometers located above each ankle, and of using Bayesian Recognition Algorithms to analyze their signals. However, in our opinion, statistical analyses performed by the authors may be inappropriate, risking belittling their work.
They computed the Pearson correlation coefficient (r) to assess the reliability and validity of their method. In many studies, r has been computed to assess the validity of a new measure.3 But the correlation coefficient between 2 variables (x,y) measures the strength of their linear relation (y=a×x+b), neither their agreement (y=x) nor a causal relationship (x implies y) are assessable by the r-value.3
To prove the validity of their method, the authors reported a significant correlation (r=0.98; P=0.001) between walking speed values estimated by Bayesian Alghorithms and those computed by the stopwatch-timed method (considered as true values). However, they did not report the underlying linear regression equation. Using all 29 trials (Table 2 of that article), we found an equation very close to the equality line: y=0.991×x−0.013. This result, more than r-value or the observed small mean difference between estimated and true values, strongly support the validity of the proposed method. However, we have also observed an underestimation of the true values in 23 of the 29 trials. What might have caused this systematic underestimation in 79% of the cases? Conversely, the overestimation observed in 5 trials was related only to 2 subjects. Could that possibly have been caused by some kind of error during their calibration indoor trials?
Then, to test the validity of the method over the whole range of speeds, we applied the method suggested by Bland and Altman.3 We did not find any correlation (r=−0.04) between observed differences and true values. Moreover, most of these differences (26 of 29) lay between their mean ±2SD. The other 3 trials were in the middle of the speed range with differences between −14% and 13% of the relevant true values. This approach seems more appropriate to demonstrate good agreement between estimated and true values. It should have also been used to assess the validity of step counting or for data of home and community walking.
In this study, the reliability was also assessed by computing r. In a previous similar study, it was evaluated by an analysis of variance.4 Both of these approaches seem less appropriate than is computing the intraclass correlation coefficient.2 For the 8 test–retested subjects, it was 0.980, confirming a reliable consistency of estimated values with true ones.
More specific and appropriate analyses provided stronger evidences of the reliability and validity of the method proposed in this study. Nevertheless, the method could be further improved by reducing underestimation and deeply investigating its reliability, taking into account other aspects related to calibration and accelerometer misplacements.2
Generally, the confusion among correlation, agreement, and/or causation may lead to an overstatement of the results, as observed in many studies.3 A meaningful example was offered by Matthews, who reported a significant correlation between number of storks and number of births in 17 European countries (r=0.62; P=0.008).5 To emphasize the risks of overstating the meaning of a high r-value, he humorously titled his article, “Storks Deliver Babies (P=0.008).”
Marco Iosa, PhD
Giovanni Morone, MD
Stefano Paolucci, MD
Movement and Brain Laboratory
Fondazione Santa Lucia I.R.C.C.S.
Stroke welcomes Letters to the Editor and will publish them, if suitable, as space permits. Letters must reference a Stroke published-ahead-of-print article or an article printed within the past 3 weeks. The maximum length is 750 words including no more than 5 references and 3 authors. Please submit letters typed double-spaced. Letters may be shortened or edited. Include a completed copyright transfer agreement form (available online at http://stroke.ahajournals.org and http://submit-stroke.ahajournals.org).
- © 2011 American Heart Association, Inc.
- Dobkin BH,
- Xu X,
- Batalin M,
- Thomas S,
- Kaiser W
- Saremi K,
- Marehbian J,
- Yan X,
- Regnaux JP,
- Elashoff R,
- Bussel B,
- et al