Which of the following characteristics does the new intelligence test possess?

(iii) Overview

Professionals in the field of intelligence testing have described the third edition of the Wechsler Intelligence Scale for Children in a number of different ways. Some critics feel that the WISC-III reports continuity, the status quo, but makes little progress in the evolution of the assessment of intelligence. Such critics note that despite more than 50 years of advancement in theories of intelligence, the Wechsler philosophy of intelligence (not actually a formal theory), written in 1939, remains the guiding principle of the WISC-III (Schaw, Swerdilik, & Laurent, 1993). One of the principal goals for developing the WISC-III stated in the manual was merely to update the norms, which is “hardly a revision at all” (Sternberg, 1993). Sternberg (1993) suggests that the WISC-III is being used to look for a test of new constructs in intelligence, or merely a new test, the examiner should look elsewhere.

In contrast to these fairly negative evaluations, Kaufman (1993) reports that the WISC-III is a substantial revision of the WISC-R and that the changes that have been made are considerable and well done. “The normative sample is exemplary, and the entire psychometric approach to test development, validation, and interpretation reflects sophisticated, state-of-the-art knowledge and competence” (Kaufman, 1993). For Kaufman, the WISC-III is not without its flaws but his overall review of the test is quite positive. One of Kaufman's (1993) main criticisms is that the Verbal tasks are highly culturally-saturated and school-related, which tend to penalize bilingual, minority, and learning-disabled children. He suggests that perhaps a special scale could have been developed to provide a fairer evaluation of the intelligence of children who are from the non-dominant culture or who have academic difficulties. Another criticism raised by Kaufman is that too much emphasis is (placed on a child's speed of responding on the WISC-III. It is difficult to do well on the WISC-III if you do not solve problems very quickly. This need for speed penalizes children who are more reflective in their cognitive style or who have coordination difficulties. The speed factor may prevent a gifted child from earning a high enough score to enter into an enrichment class or may lower a learning disabled child's overall IQ score to a below average level, just because they do not work quickly enough. Although the WISC-III clearly has had mixed reviews, it is one of the most frequently used tests in the field of children's intelligence testing.

The Nature of Intelligence

Concomitant with the rise of intelligence testing came a series of debates over the characteristics of the object being measured. In 1904 British psychologist Charles Spearman (1863–1945) used early test data to argue for the unitary nature of intelligence, explaining performance on mental tests in terms of general intelligence (g) and specific abilities (s). While numerous researchers – including Karl Pearson (1857–1936), Goddard, and Terman – accepted his analysis, others were skeptical, insisting instead that intelligence was composed of a number of primary independent abilities. Edward L. Thorndike (1874–1949) in the United States was among the first to articulate this position. During the 1920s, he was soon joined by two statistically sophisticated psychologists, L.L. Thurstone (1887–1955) in the U.S. and Godfrey Thomson (1881–1955) in the U.K. Both used mathematical methods similar to Spearman's (factor analysis) to show, not that there was one central factor, as had Spearman, but that there were a small number of significant and independent abilities. In the period after World War II, psychologists continued to put forward a range of interpretations of the composition of intelligence: while Hans Eysenk (1916–97) remained convinced of the reality of g (general intelligence), for example, Philip E. Vernon (1905–87) proposed a hierarchical model of intelligence that inter-linked specific skills, general abilities, and overall intelligence, and Joy P. Guilford (1897–1987) contended that intelligence was composed of as many as 150 independent factors. Later influential additions to these conceptualizations included Howard Gardner's (1943–present) seven discrete types of intelligence and Robert J. Sternberg's (1949–present) triarchic model of intelligence.

Overshadowing all of the arguments over intelligence, however, has been the nature–nurture question, particularly with regard to racial, ethnic, and socioeconomic groups. Figures such as Galton, Spearman, Pearson, and Terman argued early in the twentieth century strenuously for the nature position, with Galton producing studies on identical twins that have served as a model to this day for investigations into the relative weights of heredity and environment. During the 1900s to 1930s, when eugenics was at the apex of its popularity, arguments in favor of intelligence as an inheritable biological entity ran strong, and were used to justify calls for immigration restriction and for sterilization of the mentally ‘unfit,’ as well as for the creation of multitracked secondary schools. Claims about the ‘inferiority’ of non-European racial groups had been long-standing in the West, buttressed from the middle of the nineteenth century on with arguments about differences in overall levels of intelligence. During the early twentieth century these arguments continued to be promulgated using data on intelligence produced by the new intelligence tests, and indeed to be extended to less favored segments of the so-called ‘white race,’ including lower socioeconomic groups and peoples deriving from parts of southern and eastern Europe. There was also a particular worry about those individuals feared to be of subnormal intelligence, with professional scholars and public figures often decrying what they termed ‘the menace of the feebleminded.’

At the same time, however, anthropologists were beginning to put renewed emphasis on culture as the primary determinant of human behavior, claims strengthened during the middle of the century by studies carried out especially at the Iowa Child Welfare Research Station, where IQ was found to change depending on nutrition and educational environment. Moreover, during the 1920s and 1930s a number of psychologists and sociologists in the United States, including many prominent African-American scholars, challenged much of the data on which claims about group-level differences in mental ability were based, demonstrating that the World War I testing data, for example, could be more convincingly interpreted as showing that intelligence scores were influenced markedly by the quality of education received.

During the post-World War II period the debate continued to be pressed from both sides, with increasingly sophisticated twin studies showing high IQ correlations between identical twins separated at birth, while at the same time other researchers were teasing out ever more complicated connections between intelligence development and such factors as nutrition, family child-rearing practices, socioeconomic status, and quality of education received. In the 1980s, James R. Flynn among others pointed out the significant increases in IQ test scores around the globe (now known as the ‘Flynn effect’), and used those dramatic changes to argue that intelligence tests were capturing only limited aspects of the underlying entity called intelligence and that performance on these tests was strongly affected by a range of environmental factors. There was also a wholesale retreat on claims about intelligence at the level of groups, with most mainstream researchers arguing that group-level differences, even if they existed, were more than swamped by individual variation and environmental factors.

While few experts would deny the influence of both genes and the environment, the vociferous debate in the mid-1990s over The Bell Curve (Herrnstein and Murray, 1994), with its claims that IQ is hereditary and a prime determinant of an individual's future, indicates that broad disagreements persist about intelligence and show little likelihood of quick resolution. What they reveal as well is that the language of native intelligence has remained a powerful vehicle for discussions of a range of social issues, from the organization of an educational system to the value of affirmative action programs to the just allocation of social resources. Because of its associations with individual merit and its history of use for racial and class discrimination, intelligence continues to attract both strong proponents and vigorous critics, and remains a concept with wide cultural and political resonance.

SCOPE

The literature about brief cognitive assessment is very large, and it is not possible in a single chapter to comprehensively consider all aspects of this area. This chapter deals mainly with issues about and specific tests for the group probably seen most often by psychologists for brief intelligence testing: school-age children about 5–16 years of age. Considered first are various contexts for brief cognitive testing in school or clinic settings. Desirable characteristics and limitations of brief cognitive tests are discussed next, followed by reviews of some individually administered tests for brief intelligence testing. The occasionally overlooked, but often valuable, role of parent-informant data in screening for child cognitive dysfunction is also addressed. After review of the aforementioned issues and measures, general recommendations for the best practice of brief cognitive assessment are offered. This chapter concludes with note of issues pertinent to this topic but that cannot here be covered in great detail. These issues include the evaluation of children who are members of minority groups or who do not speak English as a first language.

Standardized Instruments

Standardized instruments, such as intelligence tests or personality tests, compare the client’s performance to a normative group. These instruments may be useful in determining whether or not the child’s functioning or behavior is discrepant from most children of a similar age and background, and, if so, in what direction. These instruments may also provide diagnostic or clinical categories based on the pattern of scores. Standardized instruments are particularly vulnerable to cultural bias and distortion issues related to the nature of the normative population and to psychometric concerns such as whether or not the construct being examined is comparable across cultures (Dana, 2005).

5.3 Operationalizing the Concepts

Intelligence tests are routinely used in a variety of settings including business, education, civil service, and the military.134 Psychometricians attempt to use test scores to measure a factor (a component of T in the notation of Section 4). The working hypothesis in the intelligence testing business is that specific tests measure only a single component of T and that tests with different “content domains” measure different components. We first discuss the origins of the measurement systems for intelligence, and we then discuss their validity.135

Abstract

Alfred Binet's impact on the field of psychology is unequaled, with the possible exception of Sigmund Freud. In retrospect, it appears that quite ironically Alfred Binet's most outstanding contribution is the Stanford–Binet Intelligence Scale, which owes its popularity (though not its conceptualization) largely to the work of Lewis Terman rather than to Binet himself. The intelligence scale became the tree that hid the forest: Much of Binet's exceptional, and often revolutionary, research was largely ignored on account of the overwhelming visibility that his work on intelligence testing achieved. A thorough review of his bibliography shows that his research interests spread a wide range of topics, including but not limited to anatomy, animal magnetism, audition, chess players, court witnesses, double consciousness, fetishism in love, graphology, intellectual exhaustion, intelligence, intelligence assessment, hallucinations, hysteria, inhibition, language, literary creation, memory, mental alienation, mental images, moral responsibility, movement, pedagogy, perception, phrenology, physiology, reasoning, retardation and academic underachievement, the soul, and suggestibility. Though at first glance these topics may seem unrelated, there really are two consistent threads throughout Binet's life and work. The first is a commitment to experimental psychology. The second is his pursuit to understand the complexities of the human mind in order to establish ‘an individual psychology.’

Historical Background

The study of intellectual development and its measurement historically has involved debate about several theoretical issues, all of which have implications for test design. One issue concerned the structure of intelligence, which was described as either a general global intellectual capacity, complex but unified, or a combination of many separate abilities. The determinants of individual differences in intellectual capacity and rate of development also were the subject of much debate when Bayley first started the construction of her scales. For example, proponents of the view that intelligence is predetermined by either the child’s genetic heritage or by a maturational timetable argued that intellectual capacity is fixed and unchanging across the lifespan, while others maintained that environmental influences contributed to changes in performance on intelligence tests over time. Finally, an important theoretical issue concerned whether cognitive development can be characterized as qualitative change, involving reorganization and restructuring at each new developmental level, or as a quantitative change involving the steady and incremental accumulation of knowledge with no qualitative change in the nature of intelligence. During the first half of the twentieth century, these issues influenced both the intelligence testing movement and the movement to catalog the course of human growth in large longitudinal studies, as exemplified by the work of Alfred Binet and Arnold Gesell.

In 1905, Binet developed an intelligence test for young children to determine placement in special remedial classes in the French school system. Binet’s test contained items selected to assess complex mental processes such as memory, attention, and comprehension. Its structure was based on the concept of general intelligence, which was widely accepted in the early years of the twentieth century. According to this view, intelligence tests could be constructed of a variety of items measuring performance in different domains combined together into a single scale, rather than divided into separate subscales for different abilities, since each individual item presumably tapped into the same underlying level of general intelligence. Binet test scores for individual children were fairly stable over time, leading some to conclude that this general intellectual capacity was constant throughout the lifespan, although Binet himself was opposed to this idea.

Working with a longitudinal sample in the early 1920s, Gesell developed test items designed to measure age-related changes in infant abilities in five areas: postural, presensory, perceptual, adaptive, and language–social behavior, and created a test divided into five corresponding scales. Gesell believed that responses to test items presented in early infancy were precursors to more mature forms of related behaviors observed at later ages. His five separate scales with no overall composite score implied that Gesell believed abilities in these areas developed independently. Gesell’s position was that predetermined biological maturation accounted for the development of these separate abilities, which unfolded in a fixed sequence, and that heredity determined the upper limits of an individual’s capacity and rate of development.

Even though Bayley’s views differed from many of the theoretical positions of Gesell and Binet, both the content and the structure of the Bayley Scales were strongly influenced by their work. For example, much of the content of her early scales consisted of developmentally ordered items taken directly from Gesell’s work, and she employed Gesell’s method of administering the same test items at different ages in order to elicit a range of different responses. Bayley shared Gesell’s interest in motor development, although she emphasized the inseparability of motor and mental skills in early development. She disagreed with Gesell that items measuring infant abilities could be organized into different subscales, believing that mental abilities were not divided into separate factors, particularly in early infancy. Accordingly, the Bayley Scales were structured like the Binet scales, with items testing performance in different domains grouped together by age levels rather than divided into separate subscales for different kinds of mental abilities. Despite the influence of Binet and Gesell, Bayley did not subscribe to most of the underlying assumptions of the intelligence-testing movement, particularly the idea of a fixed intelligence.

The notion of intellectual capacity as fixed and unchanging across the lifespan, based on the idea that intelligence was genetically predetermined, had two important implications for infant assessment: it should be possible to predict later intelligence from performance on tests given in infancy, and performance on intelligence tests should not be influenced by environmental factors such as level of parental education or the amount of stimulation in the home. Bayley’s early research directly addressed these issues and helped to formulate her own theoretical perspective on the structure of intelligence and the nature of intellectual growth. Evidence from her longitudinal studies demonstrating that performance on infant tests did not predict later outcome led her to conclude that intellectual capacity is not immutable. If intelligence is not fixed, but rather changes over time, consideration of the determinants of intellectual growth necessarily included environmental factors. Thus, Bayley’s developmental perspective led her to take an interactionist position on this issue, and her work supported the idea that maturation and environment had different effects on the course of cognitive development at different points in time. For example, she found that in early infancy, individual differences in performance on the mental scale appeared to be a function of different rates of sensorimotor maturation, while environmental influences on mental scores became more evident in the second year. Her earlier finding that correlations between mental and motor scores decreased with age in typically developing infants supported this conclusion. Bayley took a similarly developmental perspective on the question of whether intelligence was best described as a general mental ability or as separate factors, writing that early in development mental abilities consist of the most simple and basic functions, out of which more complex functions gradually emerge and are eventually differentiated into separate abilities in different domains.

CHANGES IN BEHAVIORAL ASSESSMENT

Contrasted to the field of psychological assessment in general, behavioral assessment as a specialty has had a history of about four decades. However, in these three decades we have witnessed some remarkable changes in the thinking of behavioral assessors. Probably as a strong overt reaction to the problems perceived by behavioral assessors in traditional psychological evaluation, many of the sound psychometric features of that tradition were initially abandoned. Indeed, in some instances it appears that “the baby was thrown out with the bath water.” As we already have noted, consistent with the idiographic approach to evaluation and treatment, little concern was accorded to traditional issues of reliability and validity. (The exception, of course, was the obsessive concern with high interrater reliability of observations of motoric behavior.) This was particularly the case for the numerous self-report inventories developed early on to be consistent with the motoric targets of treatment (e.g., some of the fear survey schedules).

There were many other aspects of traditional evaluation that also were given short shrift. Intelligence testing was eschewed, norms and developmental considerations were virtually ignored, and traditional psychiatric diagnosis was viewed as anathema to behavior therapy. However, since the late 1970s this “hard line” has been mollified. With publication of the second, third, and fourth editions of Behavioral Assessment: A Practical Handbook and emergence of two assessment journals (Behavioral Assessment and Journal of Psychopathology and Behavioral Assessment), greater attention to cherished psychometric principles has returned. For example, the external validity of role playing as an assessment strategy in the social skill areas has been evaluated by Bellack and his colleagues (cf. Bellack, Hersen, & Lamparski, 1979; Bellack, Hersen, & Turner, 1979; Bellack, Turner, Hersen, & Luber, 1980) instead of being taken on faith. Also in numerous overviews the relevance of the psychometric tradition to behavioral assessment has been articulated with considerable vigor (e.g., Adams & Turner, 1979; Cone, 1977, 1988; Haynes, 1978; Nelson & Hayes, 1979; Rosen, Sussman, Mueser, Lyons, & Davis, 1981). Looking at behavioral assessment today from a historical perspective, it certainly appears as though the “baby” is being returned from the discarded bath water.

Also, in recent years there have been several calls for a broadened conceptualization of behavioral assessment (e.g., Bellack & Hersen, 1998; Hersen, 1988; Hersen & Bellack, 1988; Hersen & Last, 1989); Hersen & Van Hassett, 1998). Such broadening has been most noticeable with respect to the use of intelligence tests in behavioral assessment (Nelson, 1980), the relevance of neuropsychological evaluation for behavioral assessment (Goldstein, 1979; Horton, 1988), the importance of developmental factors especially in child and adolescent behavioral assessment (Edelbrock, 1984; Harris & Ferrari, 1983; Hersen & Last, 1989), and the contribution that behavioral assessment can make to pinpointing of psychiatric diagnosis (Hersen, 1988;Tryon, 1986, 1998).

Primary Factors

In the late 1930s, Thurstone administered 56 mental tests to 200 students from the University of Chicago and 40 YMCA college students, who volunteered to participate in the test sessions. Mean age of participants was 19.7 years. The test battery covered a variety of mental tasks that were verbal, spatial, or numerical by nature. Tests measured fluency of production, abstraction, reasoning, or rote learning of paired associates. Some tests were adapted from existing materials, while others were especially designed and developed by Thurstone and his wife in order to cover a broad range of tasks. Many of these tests or equivalent tests are still being used in contemporary intelligence testing, e.g., figure rotation, surface development, arithmetic speed, number series, verbal analogies, syllogisms, and vocabulary. After collecting the data, a matrix of intercorrelations was first calculated from scores on the 56 tests. Almost all intercorrelations appeared to be positive, and the modal correlation coefficient was about 0.35. Next, 12 centroid factors were extracted. Orthogonal rotation to simple structure rendered nine primary factors with positive and near-zero loadings, but without substantial negative loadings. These nine factors were psychologically interpretable through the identification of common elements in tests that loaded 0.40 or more on the rotated factors. In the following paragraphs, a keyword and content description will be given for each primary factor, along with examples of pure tests that typically loaded on that factor only.

Spatial ability (S): The 13 tests that substantially loaded on this factor had a visual or spatial nature in common. This first primary factor should not be confused with the next factor, perceptual ability, as spatial abilities relate to mental imagery, rather than the perception of stimuli. Two representative tests were “flags,” which required the rotation of nation flags, and “pursuit,” which required participants to follow lines from start to end in a complex line pattern.

Perceptual ability (P): The nine tests that loaded on this factor to a large extent represented the facility in finding or recognizing items in a perceptual field. For instance, it involved the perception of an object that was embedded in irrelevant material. Two representative tests were “identical forms,” which simply required participants to pick an object identical to the stimulus object out of an array of highly similar objects, and “word grouping,” which required the categorization of easy words through the speeded perception of apparent relations among them and the identification of words that did not belong in these categories.

Numerical ability (N): The eight tests that highly loaded on this factor had in common a numerical nature. These tasks demanded a considerable proficiency in numerical calculation and reasoning. Pure numerical tests included those of arithmetic speed, such as “addition,” “multiplication,” and “division.” The tests for numerical reasoning, however, also loaded on various other factors.

Verbal relations ability (V): The 13 tests that clearly loaded on this factor concerned the logical relations between ideas and the meaning of words. This fourth primary factor should be separated from the next verbal factor, word ability, as verbal relations pertained to the classification and association of ideas and semantics, rather than to the production of isolated words. Two characteristic tests were “inventive opposites,” in which participants had to find two words with a meaning opposite to the stimulus word, and “verbal analogies,” which required participants to find a relationship between two given words and to choose a target word by applying that relationship to another stimulus word.

Word ability (W): The six tests that loaded on this fifth primary factor were characterized by a fluency in dealing with isolated words. Both “disarranged words” and “anagrams” were relatively pure W-tests, which required participants to rearrange a jumbled sequence of characters in order to obtain a meaningful word.

Memory ability (M): The five tests that substantially loaded on this sixth factor concerned the recall or recognition of paired associates that had been presented shortly before. The two most unambiguous tests were “word-number,” which involved the memorization and recall of paired associates of stimulus words and response numbers, and “initials,” which required the memorization and recall of a list of names with initials.

Inductive ability (I): The five tests that loaded on this seventh factor asked participants to find a rule or principle underlying a set of stimulus items in a test. Two representative tests were “areas,” which required participants to determine the total white surface area from increasingly complex figures, and “tabular completion,” which required participants to fill in missing numerical entries in a table by examining column headings. Even though “number series” loaded highest on induction, apparently this test also loaded on other factors.

Restriction in solution ability (R): This far less distinct factor concerned tasks that involved some sort of restriction in obtaining a solution. J. Guilford might have referred to the description of this factor as “convergent production.” Seven tests loaded on this factor, but it is not entirely obvious that the two most representative tests, “sentence completion” and “mechanical movements,” had task elements in common. Sentence completion required participants to add one appropriate word to an incomplete sentence, whereas mechanical movements asked participants to determine the direction of movements for interacting gear wheels.

Deductive ability (D): Finally, the ninth factor also appeared to be less well defined. The four tests that loaded on this last factor required participants to apply a rule to target stimuli. Relatively pure tests for D concerned the verbal syllogisms tasks of “reasoning” and “false premises.” Participants had to judge whether an inference (e.g., “Mr. White is wealthy”) logically followed from the given premises (“All wealthy men pay taxes. Mr. White pays taxes.”).

As Thurstone rejected the existence of g, he also strongly opposed the use of a single IQ index as a general indicator of mental ability. He preferred a description of mental abilities in terms of an individual profile of factor scores on the primary abilities. In fact, he even tried to relate such individual mental profiles to the vocational interests of his participants. He selected certain atypical participants with extreme profiles as case studies and argued that their profile of mental abilities matched their vocational preferences. For instance, one student with a profile high on verbal relations V and perception P, but low on the problem-solving factor R, appeared to pursue a career as an actor. These highly selected case studies, however, are not entirely convincing. Indeed, Thurstone acknowledged that more than 90% of his participants were not extremely profiled. Thurstone, however, maintained that more pure, that is, factorially less complex, tests would further substantiate the simple structure of primary abilities.