Rank the steps involved in applied behavior analysis according to the order in which they occur.

Abstract

Social validity refers to the social significance of intervention goals, social acceptability of intervention procedures, and social importance of effects (Kazdin, 1977. Behavior Modification, 1, 427; Wolf, 1978. Journal of Applied Behavior Analysis, 11, 203–214). Assessing social validity is an important component of applied behavioral analytic inquiry. By assessing social validity, practicing professionals are empowered to revise and refine interventions with positive outcomes. In this chapter, we defined the term social validity and offered behavioral practitioners pragmatic direction and recommendations regarding social validity assessment. We provided a concise discussion of the history of social validity, explaining how the assessment of social validity became—and continues to be—a priority in the field of applied behavior analysis. Then, we focused on practical applications of social validity: (1) the importance of assessing social validity at each level of prevention, (2) the importance of assessing social validity from all stakeholders involved in any intervention effort, and (3) the range of methods for measuring social validity. We concluded with a summary of practical guidelines, noting salient essential learning from this chapter.

Welfare of others

Social validity is concerned with measuring the impact of treatment goals, procedures, and effects on not only the direct recipients of treatment but also on others that may indirectly influenced by the treatment. Social validity assessment frequently incorporates treatment mediators, family members, friends, peer groups, etc. This inclusion in social validity assessment offers a method for measuring and ensuring that the welfare of others is considered in treatment programs. Developing a more standardized process for conducting social validity assessments could increase the focus on others welfare, by expecting that a wide array of individuals are included in the process. Conducting social validity assessments in a manner that includes numerous others may also reveal variables that have previously gone unrecognized.

Is Social Validity Important?

Social validity has been viewed as essential to the survival of the field of applied behavior analysis and as a diversionary trap that would lead to the demise of the field (Baer & Schwartz, 1991). They state, “We know little about the accurate and valid assessment of what gets called social validity—and we know a great deal less about the survival of fields” (p. 231). While either of these views may potentially be correct, what appears most important is to gain an understanding of the construct and what role it might play in benefiting society. If social validity has some value to society, it should be embraced and further utilized in the field of behavior analysis rather than avoided and misunderstood. Baer and Schwartz state that the primary problems facing the field of applied behavior analysis with regard to social validity were how to assess it and how to apply it to improve behavior interventions.

In some cases it may be argued that social validity is not important. An expert may develop a program that is considered highly effective at achieving a desired outcome, which may on the surface appear to be a very useful program regardless of whether nonexperts agree. An extreme hypothetical example of this might be as follows:

An expert is assigned a case in which a client is displaying aggressive acts toward others within a residential facility. The client frequently strikes, pushes, and bites others throughout the day during typical daily activities. The expert designs a program intended to reduce the frequency of aggressive acts toward others. The expert trains the staff at the facility to place the client in a time-out room following each aggressive act. The program is determined to show an immediate reduction in the frequency of aggressive acts when compared to data obtained prior to the treatment.

On the surface, this treatment may appear to have produced the desired outcome (reduction of aggression), but it actually neglects to mention a number of critical elements that would need to be considered before the treatment could truly be considered a success. First, the treatment does not clarify the amount of time the client would be required to remain in the time-out room. This problem has been highly relevant in intermediate care facilities, due to the potential for abusing this type of procedure. The staff at the facility may choose to leave the client in the time-out room for extended periods of time because the aggressive behavior that the patient displays is difficult to manage and can be avoided while he or she remains in the time-out room. In addition, the procedure may not be truly effective because the time spent in isolation reduces the opportunities for the client to engage in aggressive behavior; this lack of opportunity would certainly reduce the frequency of aggressive acts. There are numerous other concerns regarding this program, such as the potential lack of socialization that may occur, lack of training in appropriate skills, and so on.

The point of this example is not to identify all the potentially inappropriate elements, but rather to demonstrate that just because a program is considered effective does not mean that it will be considered appropriate by those closely involved in implementation of the program, or that it should even be implemented in the first place. For instance, other treatments may be available that can potentially produce similar outcomes and avoid some of the deficiencies apparent in the program described. If these other treatments are available, they might certainly be more appropriate than the treatment described, although they may not produce more effective results.

Kazdin (1980) points to three reasons why determining the acceptability of treatments was important to overall social validity. The first reason was that several techniques for treating problem behaviors may be available that have comparable efficacy, but may not be equally acceptable to the person receiving the treatment. The second reason involves ethical and legal issues, which require that procedures avoid infringing on the rights of the individuals receiving the treatments. The final reason addressed by Kazdin (1980) involves the value of identifying specific variables that could influence the acceptability of treatments. If influential variables could be identified, they could be used to encourage individuals to seek out treatment, initiate treatment, and adhere to treatment regimens.

With the increased dissemination of treatments involving less intrusive procedures and changes in educational law, a reexamination of the concept of social validity appears warranted. Michaels, Brown, and Mirabella (2005) report that experts were making fewer recommendations for the use of consequence-based treatments than they had in the past. Educational legislation such as the No Child Left Behind Act (No Child Left Behind Act of 2001, 20 U.S.C. § 6301 et seq.) and the Individuals with Disabilities Education Act Amendments (IDEA, 1997, 20 U.S.C. § 1401) has mandated the incorporation of parents, teachers, principals, and others in the development and implementation of treatments for children receiving special education services. More recent changes to IDEA have also promoted the use of empirically based treatments in the least restrictive environment. In light of these changes, the concept of social validity may benefit from further evaluation in terms of relevance to decisions made regarding treatments for behavior change programs.

What is social validity?

The construct of social validity has been conceptualized in many different ways. Schwartz and Baer (1991) pointed out that the numerous meanings have been ascribed to the construct of social validity, which have thus catapulted the construct into a mass of confusion. Even the terminology associated with social validity has been used in various ways that make determining exactly what social validity consists of extremely confusing at times. While the literature within the field of behavior analysis coined the term “social validity,” the term has not been used consistently even within the field where it was first used. Other fields of social sciences as well as business and marketing have used various terminology to describe similar constructs such as social importance, social relevance, social significance, consumer satisfaction, clinical importance clinical significance, educational relevancy, applied relevance, applied importance, ecological validity, cultural validity, and cultural significance. The numerous terms which have in some instances been used interchangeably with social validity may have resulted in some confusion about exactly what is involved in defining, measuring, and evaluating social validity. While social validity can be associated with several different disciplines and defined in many different ways, the type of social validity referred to within this book will be the social validity which was developed from the field of applied behavior analysis. There have been several proposed variations on what social validity is, how it should be assessed, and how it should be used within the field of applied behavior analysis and its potential applications within other areas such as medicine, nursing, and the health sciences. Several of these variations will be described within this book, but each of these will be discussed in relationship to the original definition of social validity proposed by Wolf (1978).

For the purposes of this book the term social validity will refer to the evaluation of the degree of acceptance for the immediate variables associated with a procedure or program designed to change behavior. This definition is derived from the definition provided by Wolf (1978; refer to Table 1.1). He defined social validity as (1) the social significance of the goals of treatment; (2) the social appropriateness of the treatment procedures; and (3) the social importance of the effects of treatments. The most frequent method for determining the degree of acceptance for a procedure or program has been to ask those receiving, implementing, or consenting to a treatment about their opinions of the treatment. These opinions are then used to make decisions about current or future uses of the treatment.

Table 1.1. Wolf’s three component definition of social validity.

Social Validity

Broadly defined, social validity concerns the appropriateness and acceptability of ABA interventions as both process and outcome measures (Kazdin, 1977; Wolf, 1978). One type of social validity, expert validation, has to do with the selection of intervention objectives, skills, and procedures. Thus, as an initial step in their program to reduce swimming stroke errors, Koop and Martin (1983) developed lists of possible errors from swimming instruction books and distributed the lists to team coaches “who independently rank-ordered the errors according to their relative importance in detracting from swimming speed,” refining the lists further “by deleting errors on which coaches strongly disagreed, as well as errors that were unanimously considered unimportant” (p. 449).

In the same vein, Stokes, Luiselli, and Reed (2010), Stokes, Luiselli, Reed, and Fleming (2010) and Harrison and Pyles (2013) referred to recommendations from the American Football Coaches Association (1995) and sought verification by active high school and collegiate coaches when developing task analyses for improving blocking and tackling skills. Boyer et al. (2009) provided another example of expert validation by constructing their video modeling intervention with child gymnasts from videotaped performances of USA National Team members in competition, advice from a high-ranking gymnastics judge, and consultation with coaches of elite athletes.

A second type of social validity assessment compares preintervention, intervention, and postintervention measures against a high-performing normative sample. Such assessment enables practitioners to select reasonable intervention-targeted criteria. Furthermore, the resulting data can confirm that intervention results matched or exceeded an acceptable standard. In the previously cited football study by Stokes, Luiselli, Reed, and Fleming (2010), the participants were five offensive linemen that coaches judged as having the poorest pass blocking skills on a high school varsity team. After preparing a 10-step task analysis and recording protocol for the study, the researchers documented the pass blocking performance of the three highest rated starting offensive linemen from videotaped games against opposing teams during the preceding season. The average correct pass blocking accuracy for these linemen was 80%, and the range of 70-90% was adopted as the practice and game performance criteria in the study.

A further goal of social validity assessment in ABA-sports research asks athletes and coaches to rate the acceptability of intervention procedures they received and how satisfied they were with implementation and outcomes. Smith and Ward (2006) noted that beyond acquiring measures of acceptability and satisfaction, this type of social validity assessment may uncover unanticipated intervention effects and enable researchers to adjust procedures in future studies. On the most practical level, social validity assessment allows sports practitioners to select intervention procedures that have the greatest appeal and by extension will be implemented with good integrity.

A few research examples show the usual process of socially validating procedural acceptability and satisfaction. Following the gymnastics intervention, Boyer et al. (2009) gave a 5-point Likert-style questionnaire to the gymnasts, coach, and assistant coaches to assess “how much they liked the procedure, whether they would recommend it to others, how easy it was, how helpful it was, and how effective it was in skill development” (p. 857). Koop and Martin (1983) had a swimming coach determine the degree to which he considered intervention to be effective, useful, and easy to implement. The swimmers also documented how much they liked the procedures or found them useful. In Stokes, Luiselli, Reed, and Fleming (2010), the football players rated their satisfaction with baseline and intervention procedures on a number-coded questionnaire as poor (1), fair (2), good (3), and excellent (4). The social validity assessment with a novice golfer by Fogel, Weil, et al. (2010) required her to complete 6-point ratings (strongly disagree, disagree, slightly disagree, slightly agree, agree, strongly agree) for questions such as “My swing is better after this training,” and “Learning the pivot via TAG is contributing to my long term golf goal.”

The benefits of social validity assessment notwithstanding, it is possible that some respondents may not view the most effective procedures favorably. For example, young and beginning athletes might prefer procedures that are not highly strenuous or difficult independent of how quickly they learn target skills. More established performers are likely to discount procedures that do not produce rapid results and competitive success. Coaches, too, will likely judge procedures differently based on the level of play they are instructing, implementation complexity, familiarity, and so on. We suggest that these and similar variables be considered when designing and interpreting the data from social validity assessment questionnaires.

Strategies to boost the social validity of treatment goals

There are several strategies which could potentially improve the selection of socially significant treatment goals and enhance overall social validity.

Steps to boost social validity—Developing Socially Valid Goals for Treatment

Incorporate semistructured interviews;

Gather information from multiple sources;

Consider different aspects of the individual’s life;

Rank treatment goals hierarchically;

Operationalize expected outcomes in both clinical and social (lay) terminology.

Ease of Implementation

Instruments or methods for collecting social validity data need to offer a quick and efficient means for gathering information. In many cases social validity data is never collected, and if the available procedures are not easy to implement, then they will most likely never be used. Considering the time and effort that is required to conduct a functional assessment of behavior, develop a treatment, train treatment mediators, collect data on behavior changes, conduct integrity checks, and so on, it seems highly important that social validity measures not be overly time consuming. Schwartz and Baer (1991) recommended that social validity questionnaires be highly comprehensive and allow consumers numerous options for responding. When a highly comprehensive questionnaire is used, time allowances should be recognized, especially when these questionnaires are used in conjunction with other similar instruments such as functional assessment interviews. Each of these types of instruments takes time to complete, and in many cases there may be a considerable amount of redundancy across different forms, scales, interviews, and other materials.

It might be beneficial for consultants to work toward synthesizing some of the different measures into a comprehensive tool that addresses numerous variables, such as functional assessment background data and social validity data, and also reduces redundancy. At present, such an instrument does not appear to be available. Another option would be to rely upon other related factors that can offer insight into consumer preferences and be indicative of social validity, such as quality-of-life measures, measures of habilitation, vocational aptitudes measures, and so forth,, from which information may be available. Information from other successful programs may be indicative of treatment goals that are considered socially valid by specific consumers. The easier it is to obtain information about potentially socially significant treatment goals, the more readily the measures will potentially be used by consultants.

Assessment model of social validity

Gresham and Lopez (1996) conceptualized social validity as a product of the social validity assessments. They described how social validity could be enhanced by improving upon the procedures used to assess social validity. Social validity could be influenced if social validity assessment procedures were used more frequently during the treatment process and provided more useful data which could serve toward making decisions. They proposed the need for recognition and incorporation of several practices for conducting assessments of the social validity of school-based practices. Their recommendations included the incorporation of multiple sources of information which went beyond the opinions of consumers’ treatments and relied more heavily on data obtained from normative comparisons from sources such as developmental norms, structured interviews, and archival data. Each of their recommendations described methods for measuring social validity in relevant manner which could be used to make modifications during the treatment process in order to increase the overall social validity of a treatment. While they stated that consumers’ opinions were important toward marketing a treatment or a program, they espoused that the opinions of consumers’ were not always in the best interest of the individual targeted for treatment. By focusing on assessment methods other than consumer opinion, they felt that more socially valid treatments could be provided to their clients.