Response to Letter by Middel and van Sonderen
We thank Drs Middel and van Sonderen for their comments on our prior response to a letter from Dr Sivan.1 Middel and van Sonderen argued for the inappropriateness of directly using Cohen’s original criteria2 to estimate the standardized response mean (SRM), which was used in our study.3 The criteria for effect size d were originally used in 2 independent samples2 (pp 20 to 27). However, when the criteria are applied in the 1-sample case, the effect size index needs to be adjusted as follows.
Effect size d=dz′×√2, where dz′=SRM, and the √2 is adjusted for the doubling of sampling error, not for twice the number of degrees of freedom. Although Cohen’s table is for 2(n−1) degrees of freedom, the effect of underestimation (only n−1 are actually available in 1-sample cases) of degrees of freedom is negligible except in small samples. However, the doubling of sampling error would have a substantial effect and thus Cohen proposes to multiply dz′by √2 to compensate2 (pp 46 to 48).
When an estimate of r (the correlation between the values for paired observations) is available, a preferable adjusted procedure is to use the following index2 (p 49): Effect size d=d4′/√1−r, where d4′=effect sizepooled (ESp was proposed for use in the letter from Middel and van Sonderen).
As shown in Cohen’s illustrative examples 2.6 and 2.72 (pp 50 to 52), the effect size d can be calculated from these 2 formulas and the estimated values are nearly the same: (1) effect size d=dz′√2, when no estimate of r is available; or (2) effect size d=d4′/√1−r, when the r can be estimated. Our study design is similar to Cohen’s illustrative example 2.7; therefore, in our response1 to the letter of Sivan, we used the second formula (d=d4′/√1−r=ESp/√1−r) to calculate the adjusted effect size d.
In their letter, Middel and van Sonderen proposed to transform the SRM into the ESp and then to apply the Cohen’s criteria to interpret the ESp values. That is, based on these 2 formulas: effect size d=SRM×√2=ESp/√1−r; therefore, ESp=SRM ×√2×√1−r. However, to our knowledge, the ESp and the SRM (2 indices are for 1-sample cases) both need to be adjusted into the effect size d to more accurately apply Cohen’s criteria to interpret the changes. As in Cohen’s illustrative examples 2.72 (pp 51 to 52):
Effect size d=SRM×√2=0.74 or effect size d=ESp/√(1– 0.8)=0.74
Then the effect size d (0.74) instead of the ESp (0.33) can be used to find the power value in Cohen’s tables of statistical power.
The effect size statistics for assessing responsiveness are values expressed in the units of SD, which may lead to some discrepancies when different SDs are used (eg, SDpretest, SDpooled, or SDchange scores). We propose that in addition to the d index, researchers may estimate and report the proportions of minimal detectable change and minimal clinically important change, which are expressed in the same units as the scores of outcome measures, to evaluate the responsiveness.4
Hsieh YW, Wu CY, Lin KC. Response to Letter by Sivan. Stroke. 2009; 40: e710–e711.
Cohen J. Statistical Power Analysis for the Behavioral Sciences, II ed. Hillsdale, NJ: Lawrence Erlbaum Associates; 1988.
Hsieh YW, Wu CY, Lin KC, Chang YF, Chen CL, Liu JS. Responsiveness and validity of three outcome measures of motor function after stroke rehabilitation. Stroke. 2009; 40: 1386–1391.
Portney LG, Watkins MP. Foundations of Clinical Research: Applications to Practice, III ed. Upper Saddle River, NJ: Pearson/Prentice Hall; 2009.