A
Power Struggle: Between- vs. Within-Subjects -Designs in Deductive
Reasoning Research
V. A. Thompson & J. I. D. Campbell |
This
experiment examined the relative merits of using within-and between-subjects
designs to investigate deductive reasoning. Two issues were investigated:
1) the potential for expectancy and fatigue effects when using within-subjects
designs, and 2) the relative power of within- vs between-subjects
designs. Participants were presented with problems in a standard
belief--bias paradigm in which the believability of putative conclusions
varied orthogonally to their validity. The belief bias effect, as
well as the effect of validity, and the interaction between beliefs
and validity, were not affected by reasoners' expectations regarding
the number of problems they had to solve. The effect of beliefs and
the belief by validity interaction were only marginally affected
by the number of problems solved, despite adequate power to observe
an effect. Thus, neither expectancy nor fatigue appear to have affected
performance, suggesting that there are few drawbacks to using a within-subjects
design. In contrast, however, a power analysis clearly established
the desirability of using within- relative to between-subjects designs.
Within- subjects designs require far fewer participants to detect
effects of comparable size; this was especially true for higher-order
(interaction) effects. Finally, we provide a power analysis of within-
and between-subjects designs that should be of general utility to
researchers planning studies using proportions as a dependent measure. Key words: belief bias, deductive reasoning, statistical power, expectancy effects, carryover effects |