The Solomon four-group design is a type of experiment where participants get randomly assigned to either 1 of 4 groups that differ in whether the participants receive the treatment or not, and whether the outcome of interest is measured once or twice in each group. The four groups in this design are (see figure below): The objective is to assess the efficacy of the treatment (or intervention). Important notes regarding the Solomon four-group design:
Advantages of the Solomon four-group designWhy would we need 4 groups? More specifically, what additional information can we get with a Solomon four-group design compared to a standard experimental design with 2 groups (1 treatment group and 1 control group)? First, let’s start with: How the Solomon four-group design resembles a two-group experimental design?Notice that the first 2 groups of this design resemble the pretest-posttest control group design. And therefore, the first 2 groups are enough to control threats to internal validity such as the effects of:
(For a detailed explanation, see my other article: Pretest-Posttest Control Group Design) What makes the Solomon four-group design better than a standard two-group design?The Solomon four-group design was developed to:
What is pretest sensitization?Pretest sensitization (a.k.a. interaction between pretest and treatment) occurs when the use of a pretest increases or decreases the responsiveness of the participants to the study intervention. For example: Consider a pretest that contains questions that non-deliberately make participants more aware and concerned regarding the consequences of smoking. Such pretest may sensitize participants to make them more responsive to a smoking cessation intervention. In fact, it could be that this intervention is only effective for participants who took such pretest. Why pretest sensitization is an external validity problem?The general population from which participants were selected for the study will not be pretested, therefore, will not be sensitized to the intervention. Consequently, the effect of the intervention on the general population will be different than the one detected in the study. Therefore, the study will suffer from a generalizability (a.k.a. an external validity) problem. How the Solomon four-group design helps detecting pretest sensitization?An interaction between pretest and treatment (or pretest sensitization) would exist if the pretest influences the outcome of the posttest, only in the presence of the intervention. This can be assessed in 3 steps: Step #1: Calculate the effect of the pretest on the outcome of the posttest in the presence of the intervention: This is the difference between 1B and 3B. Step #2: Calculate the effect of the pretest on the outcome of the posttest in the absence of the intervention: This the difference between 2B and 4B. Step #3: Compare the result of step 1 with that of step 2 (that is “the difference between 1B and 3B” with “the difference between 2B and 4B”): A difference between these 2 quantities means that the effect of the pretest is different in the presence of the intervention, in other words, there is an interaction between pretest and treatment. Limitations of the Solomon four-group designThe Solomon four-group design is not used a lot in practice because of its high cost, since compared to a standard design with 2 groups it requires twice the sample size, time, materials, resources, and personnel/work. Example of a study that used the Solomon four-group designDehghan et al. investigated VR (Virtual Reality) technology as a method to reduce anxiety in children undergoing surgery. Design detailsA Solomon four-group design was used in order to control any pretest sensitization. The pretest and posttest measured the anxiety score by using a standardized questionnaire. The intervention used was a VR technique that simulates step-by-step going into an operation room. ResultsThe study concluded that VR technology reduced pre-operative anxiety in children by acting as a distraction method. LimitationsWhile the treatment allocation was done at random, participants included in the study were sampled in a non-random way from the population of all children undergoing surgery. This means that the study participants may not represent well the population of interest, therefore causing a problem of generalizability of the study results. Note however, that this external validity issue is not specific to the Solomon four-group design, instead it is a consequence of the sampling method used. References
Further reading
Which of the following statements is true of Solomon for group evaluation design?Which of the following statements is true of Solomon four-group evaluation design? It combines both pre-test/post-test comparison group and post-test-only/control group designs.
Which of the following statement best differentiates formative evaluation from summative evaluation?Which of the following statements best differentiates formative evaluation from summative evaluation? Formative evaluation focuses on how to make a training program better, whereas summative evaluation helps to determine the extent to which trainees have changed after training.
Which of the following training evaluation outcomes is Level 1 criteria in Kirkpatrick's five level framework?Level 1: Reaction
The first level of criteria is “reaction,” which measures whether learners find the training engaging, favorable, and relevant to their jobs.
Which of the following statements is true of lesson plans quizlet?Which of the following statements is true of lesson plans? They ensure that both the trainee and the trainer are aware of the course and program objectives.
|