AnalysisHow do youth assess students’ 21st Century life and career skills and theirlearning environments?
Get answer to your question and much more
How may the results of this study be utilized in enhancing the 21st Century
Ask a REL Response
Assessing Students' 21st Century Skills — December 2018
Question
Could you provide research on best practices and techniques for teachers to provide student feedback on 21st century skills?
Response
Following an established REL West research protocol, we conducted a search for research reports and resources on best practices and techniques for teachers to provide student feedback on 21st century skills (i.e., soft skills, such as critical thinking, collaboration, communication, and creativity). The sources included ERIC, Google Scholar, and PsychInfo. (For details, please see the methods section at the end of this memo.)
We have not evaluated the quality of references and the resources provided in this response. We offer them only for your reference. Also, we searched for references through the most commonly used sources of research, but the list is not comprehensive and other relevant references and resources may exist. References are listed in alphabetical order, not necessarily in order of relevance.
Research References
Care, E., Scoular, C., & Griffin, P. (2016). Assessment of collaborative problem solving in education environments. Applied Measurement in Education, 29(4), 250–264. Abstract retrieved from //eric.ed.gov/?id=EJ1111391
From the abstract: “The Assessment and Teaching of 21st Century Skills (ATC21S) project undertook a research and development plan that included conceptualization of 21st century skills and assessment task development. Conceptualization focused on the definition of 21st century skills. This article outlines the particular case of one of these skills, collaborative problem solving, from its definition and identification of subskills, to development of a method for its assessment. The theories contributing to construct definition are described. These have implications for the particular clusters of subskills that are of interest for educational assessment. An approach to assessment task creation is illustrated through the deconstruction of a well-known reasoning task, and its re-development to sample both the cognitive and social aspects of collaborative problem solving. The assessment tasks are designed to generate formative feedback for teachers in order to identify levels of ability within and between their students and support tailoring of instruction differentially for improvement.”
Devedzic, V., Tomic, B., Jovanovic, J., Kelly, M., Milikic, N., Dimitrijevic, S., & Sevarac, Z. (2018). Metrics for students’ soft skills. Applied Measurement in Education, 31(4), 283–296. Abstract retrieved from //www.tandfonline.com/doi/abs/10.1080/08957347.2018.1495212?journalCode=hame20
From the abstract: “This article presents a systematic approach to defining, applying, evaluating, refining, and revising metrics for students’ soft skills—their abilities like critical thinking, problem solving, leadership and responsibility, communication, and collaboration. The importance of these skills in educational and work settings is growing rapidly. While such skills are easy to notice, they are hard to measure. Metrics do exist, but vary from one case to another, and are often rather implicit and vague. Contrary to that, this article proposes the use of precisely specified, measurable, low-inference indicators (metrics) to assess soft skills. The article also introduces an open set of principles that can be used to guide the specification of concrete, evidence-based metrics for different soft skills. Two case studies are used to illustrate the approach. These case studies are part of a larger research effort that has developed an open set of metrics for different soft skills; some of them are discussed in the article extensively. Generalizing the metrics used in specific educational contexts is also discussed.”
Gunn, T. M., & Hollingsworth, M. (2013). The implementation and assessment of a shared 21st century learning vision: A district-based approach. Journal of Research on Technology in Education, 45(3), 201–228. Abstract retrieved from //eric.ed.gov/?id=EJ1090570
From the abstract: “Traditional educational patterns are giving way to more innovative modes and methods of learning, primarily due to radical technological changes that have increased the availability of information and improved communication. In an attempt to address this shift toward 21st century skills and learning, a single school district began a large-scale, district-wide 3-year professional development study involving the implementation of technology, differentiated instruction, and assessment for learning upgrades and teacher in servicing. After 3 years, results suggest that significant changes did occur with respect to teachers’ knowledge and use of information and communication software, the purposes for communication, differentiated instruction and assessment practices, and levels of information and communication usage. In particular, the number of hours of professional development a teacher reported having undertaken was the most notable factor affecting levels of significance.”
Lucas, B. (2016). A five-dimensional model of creativity and its assessment in schools. Applied Measurement in Education, 29(4), 278–290. Abstract retrieved from //www.tandfonline.com/doi/abs/10.1080/08957347.2016.1209206?journalCode=hame20
From the abstract: “Creativity is increasingly valued as an important outcome of schooling, frequently as part of so-called ‘21st century skills.’ This article offers a model of creativity based on five Creative Habits of Mind (CHoM) and trialed with teachers in England by the Centre for Real-World Learning (CRL) at the University of Winchester. It explores the defining and tracking of creativity’s development in school students from a perspective of formative assessment. Two benefits are identified: (a) When teachers understand creativity they are, consequently, more effective in cultivating it in learners; (b) When students have a better understanding of what creativity is, they are better able to develop and to track the development of their own CHoM.”
REL West note: This is an international study. Given its subject is relevant to your request, we included it here for your information.
Sabatini, J. P., O’Reilly, T., Halderman, L. K., & Bruce, K. (2014). Integrating scenario-based and component reading skill measures to understand the reading behavior of struggling readers. Learning Disabilities Research & Practice, 29(1), 36–43. Abstract retrieved from //onlinelibrary.wiley.com/doi/abs/10.1111/ldrp.12028
From the abstract: “In recent years, researchers, educators, and policy makers have called for a new generation of reading comprehension assessments (e.g., Partnership for 21st Century Skills, 2008). Advocates of this movement argue for a deeper type of reading assessment, one that captures students’ ability to not only understand single texts in isolation but also to engage in purposeful, multisource integration of sources. While this shift in how we define and measure reading comprehension is laudable, assessments must also measure the fundamental reading skills that may impede higher-level comprehension processes. This article presents data from two assessments that were designed to work in tandem to provide a more complete picture of reading comprehension. Middle school students were given a component skills battery which measured core reading skills such as word recognition, decoding, vocabulary, and morphology, as well as a second assessment designed to measure reading comprehension. Reading comprehension was measured using a scenario-based assessment approach, which required students to read a range of sources to fulfill a particular reading goal. The results indicate that students, including struggling readers, were able to read, understand, and problem solve in complex learning environments, but students’ ability to do so was often tempered by their basic reading skills. We argue that including a measure of component skills alongside a measure of higher-level comprehension is beneficial in interpreting student performance. Accordingly, we present the results on the scenario-based measure as a function of reading component skills and argue for the value of using this approach for struggling readers.”
Soland, J., Hamilton, L. S., & Stecher, B. M. (2013). Measuring 21st century competencies. Guidance for educators. Santa Monica, CA: RAND Corporation. Retrieved from //asiasociety.org/files/gcen-measuring21cskills.pdf
From the introduction: “Public school systems are expected to promote a wide variety of skills and accomplishments in their students, including both academic achievement and the development of broader competencies, such as creativity, adaptability, and global awareness. The latter outcomes, which are often referred to as ‘21st century skills’ or ‘21st century competencies,’ have recently taken a more central role in policy discussions, because they are seen as critical components of college and career readiness. For example, in the United States, more than forty states have adopted the Common Core State Standards (CCSS), which are designed to embody a broader view of the knowledge and skills needed for success in college and careers. This growing emphasis on outcomes beyond simple academic content knowledge is the result of a confluence of factors, including perceptions among some business and government leaders that globalization, technology, migration, international competition, and changing markets require a greater emphasis on these outcomes than was required in the past. As a result, school systems are facing increasing pressure to produce graduates with this range of competencies (i.e., knowledge, skills, attitudes, and dispositions), a demand that generates challenges in terms of pedagogy and assessment.”
Webb, M. E., Prasse, D., Phillips, M., Kadijevich, D. M., Angeli, C., & Laugesen, H. (2018). Challenges for IT-enabled formative assessment of complex 21st century skills. Technology, Knowledge and Learning, 23(3), 441–456. Abstract retrieved from //link.springer.com/article/10.1007/s10758-018-9379-7
From the abstract: “In this article, we identify and examine opportunities for formative assessment provided by information technologies (IT) and the challenges which these opportunities present. We address some of these challenges by examining key aspects of assessment processes that can be facilitated by IT: datafication of learning; feedback and scaffolding; peer assessment and peer feedback. We then consider how these processes may be applied in relation to the assessment of horizontal, general complex 21st century skills (21st CS), which are still proving challenging to incorporate into curricula as well as to assess. 21st CS such as creativity, complex problem solving, communication, collaboration and self-regulated learning contain complex constructs incorporating motivational and affective components. Our analysis has enabled us to make recommendations for policy, practice and further research. While there is currently much interest in and some progress towards the development of learning/assessment analytics for assessing 21st CS, the complexity of assessing such skills, together with the need to include affective aspects means that using IT-enabled techniques will need to be combined with more traditional methods of teacher assessment as well as peer assessment for some time to come. Therefore learners, teachers and school leaders must learn how to manage the greater variety of sorts and sources of feedback including resolving tensions of inconsistent feedback from different sources.”
REL West note: This is an international study. Given its subject is relevant to your request, we included it here for your information.
Additional Organizations to Consult
ASCD – //www.ascd.org/about-ascd.aspx
From the website: “Founded in 1943, ASCD (formerly the Association for Supervision and Curriculum Development) is a nonprofit, nonpartisan organization that represents more than 114,000 educators from more than 127 countries, including 57 affiliate organizations. ASCD is dedicated to excellence in learning, teaching, and leading so that every child is healthy, safe, engaged, supported, and challenged. Our diverse, nonpartisan membership is our greatest strength, projecting a powerful, unified voice to decision makers around the world.”
REL West note: One chapter in a book, published by ASCD, is relevant to this request, as follows:
Dean, C. B., Hubbell, E. R., Pitler, H., & Stone, B. (2012). Setting objectives and providing feedback. In Dean et al. (Eds.), Classroom instruction that works: Research-based strategies for increasing student achievement (pp. 1–21). Alexandria, VA: ASCD. Retrieved from //www.ascd.org/publications/books/111001/chapters/Setting-Objectives-and-Providing-Feedback.aspx
Excerpt: “Providing students with opportunities for self- and peer assessment speaks to these 21st century skills. By allocating time for students to reflect upon their own learning and to give and receive feedback from peers, we help them develop skills they will need throughout their K–12 years, in college, and in the workplace. The tools available now (e.g., survey tools, blogs, wikis) strongly amplify students’ and teachers’ abilities to access, collaborate, and get feedback on their work. Encouraging students to use the many resources available online and through social media and engage in the feedback process helps them become self-directed learners. Students could post their writing to a blog and solicit feedback, or they could create a rubric and ask peers to give feedback on their writing with an online form. Teachers could provide young students with a simple graphic organizer in which they assess their own work, ask for peer feedback, and receive feedback from the teacher.”
Colorado State Board of Education – //www.cde.state.co.us/cdeboard
From the website: “Members of the Colorado State Board of Education are charged by the Colorado Constitution with the general supervision of the public schools. They have numerous powers and duties specified in state law. Individuals are elected on a partisan basis to serve six-year terms without pay.”
REL West note: One report published by the Colorado State Board of Education is relevant to this request, as follows:
Colorado State Board of Education. (2015). Work group reports and recommendations. Executive summary – 21st century skills. Denver, CO: Author. Retrieved from //www.cde.state.co.us/postsecondary/21st-century-skills-draft-summary-recommendation
Excerpt: “21st century assessments are still new and emerging, and they are very useful for teachers and students. These assessments help teachers to be more intentional about already-existing practice. This data is not about content measurement of understanding; rather they help measure progression of skill development. This involves modeling teaching, assessing, and most importantly, giving students feedback on 21st century skills. Available measurements are useful for formative purposes, but can be more difficult to use for accountability purposes. Please see sample rubrics in the Best Practices section.”
The Forum for Youth Investment – //forumfyi.org
From the website: “The Forum for Youth Investment is a nonprofit, nonpartisan action tank dedicated to helping communities and the nation make sure all young people are ready by 21—ready for college, work, and life. Informed by rigorous research and practical experience, the Forum forges innovative ideas, strategies and partnerships to strengthen solutions for young people and those who care about them. Founded in 1998 by Karen Pittman and Merita Irby, two of the country’s top leaders on youth issues and youth policy, the Forum is a trusted resource for policymakers, advocates, researchers, and practitioners. The Forum provides youth and adult leaders with the information, connections, and tools they need to create greater opportunities and outcomes for young people.”
REL West note: One report published by the Forum for Youth Investment is relevant to this request, as follows:
Wilson-Ahlstrom, A., Yohalem, N., DuBois, D., & Adler, P. J. (2011). From soft skills to hard data: Measuring youth program outcomes. Washington, DC: Forum for Youth Investment. Retrieved from //www.search-institute.org/wp-content/uploads/2017/11/DAP-Ready-by-21-Review.pdf
From the overview: “Youth programs operating during the non-school hours are important partners that work alongside families and schools to support learning and development. Some programs prioritize academics; others prioritize enrichment, recreation or leadership development; others weave together a combination of these. Whether focused on sports, art or community service, most of these programs aim to develop cross-cutting skills that will help young people be successful now and help ensure they are ready for college, work and life. Helping to build what are often referred to as “social-emotional” or “21st century skills” is an important contribution that many youth programs make and more could be making. Yet these efforts remain underrepresented in the program evaluation literature, in part because they cannot be measured using administrative records or other databases to which schools and programs might have easy access. Practitioners and funders regularly ask us for advice about how to measure these skills. In response we developed this guide, which summarizes information about tools that programs can use to measure youth progress in these areas. The guide builds on and complements several related resources available in the field (for a listing, see Other Collections of Youth Outcome Measures). Our goal is to help practitioners choose conceptually grounded and psychometrically strong measures of important skills and dispositions that cut across academic achievement and other distal youth outcomes like risk behavior, mental health and employment. We also hope to encourage the development of additional measures in areas where our review reveals gaps. In a time of increasing pressure on programs to improve policy-relevant outcomes, we want to facilitate access to good measurement tools. This can help advance the out-of-school time (OST) field and facilitate collaboration among practitioners working toward common goals, both in school and out.”
The National Academies of Sciences, Engineering, and Medicine – //www.nationalacademies.org
From the website: “Society is facing an array of complex policy questions. The National Academies of Sciences, Engineering, and Medicine are distinctively qualified to provide nonpartisan, objective guidance for decision makers on pressing issues. As we have done since our founding in 1863, we marshal the energy and intellect of the nation’s critical thinkers to respond to policy challenges with science, engineering, and medicine at their core. Through a meticulous process of information collection, evidence analysis, and deliberation, our studies provide blueprints for progress. By shining a spotlight on subjects and facilitating dialogue across disciplines, our work advances understanding of critical issues. The needs of the nation—and therefore the topics we study—change over time, but our commitment to putting sound advice to work for the public good does not.”
REL West note: Two workshop summaries published by National Academies of Sciences, Engineering, and Medicine are relevant to this request, as follows:
Koenig, J. A. (2011). Assessing 21st century skills: Summary of a workshop. Washington, DC: National Academies of Sciences, Engineering, and Medicine. Retrieved from //www.ncbi.nlm.nih.gov/books/NBK84218/pdf/Bookshelf_NBK84218.pdf
From the description: “The routine jobs of yesterday are being replaced by technology and/or shipped off-shore. In their place, job categories that require knowledge management, abstract reasoning, and personal services seem to be growing. The modern workplace requires workers to have broad cognitive and affective skills. Often referred to as ‘21st century skills,’ these skills include being able to solve complex problems, to think critically about tasks, to effectively communicate with people from a variety of different cultures and using a variety of different techniques, to work in collaboration with others, to adapt to rapidly changing environments and conditions for performing tasks, to effectively manage one’s work, and to acquire new skills and information on one’s own. The National Research Council (NRC) has convened two prior workshops on the topic of 21st century skills. The first, held in 2007, was designed to examine research on the skills required for the 21st century workplace and the extent to which they are meaningfully different from earlier eras and require corresponding changes in educational experiences. The second workshop, held in 2009, was designed to explore demand for these types of skills, consider intersections between science education reform goals and 21st century skills, examine models of high-quality science instruction that may develop the skills, and consider science teacher readiness for 21st century skills. The third workshop was intended to delve more deeply into the topic of assessment. The goal for this workshop was to capitalize on the prior efforts and explore strategies for assessing the five skills identified earlier. The Committee on the Assessment of 21st Century Skills was asked to organize a workshop that reviewed the assessments and related research for each of the five skills identified at the previous workshops, with special attention to recent developments in technology-enabled assessment of critical thinking and problem-solving skills. In designing the workshop, the committee collapsed the five skills into three broad clusters as shown below: (1) Cognitive skills: nonroutine problem solving, critical thinking, systems thinking; (2) Interpersonal skills: complex communication, social skills, team-work, cultural sensitivity, dealing with diversity; and (3) Intrapersonal skills: self-management, time management, self-development, self-regulation, adaptability, executive functioning. ‘Assessing 21st Century Skills’ provides an integrated summary of the presentations and discussions from both parts of the third workshop.”
Hilton, M. (2010). Exploring the intersection of science education and 21st century skills: A workshop summary. Washington, DC: National Academies of Sciences, Engineering, and Medicine. Retrieved from //www.nap.edu/catalog/12771/exploring-the-intersection-of-science-education-and-21st-century-skills
From the description: “An emerging body of research suggests that a set of broad ‘21st century skills’—such as adaptability, complex communication skills, and the ability to solve non-routine problems—are valuable across a wide range of jobs in the national economy. However, the role of K–12 education in helping students learn these skills is a subject of current debate. Some business and education groups have advocated infusing 21st century skills into the school curriculum, and several states have launched such efforts. Other observers argue that focusing on skills detracts attention from learning of important content knowledge. To explore these issues, the National Research Council conducted a workshop, summarized in this volume, on science education as a context for development of 21st century skills. Science is seen as a promising context because it is not only a body of accepted knowledge, but also involves processes that lead to this knowledge. Engaging students in scientific processes—including talk and argument, modeling and representation, and learning from investigations—builds science proficiency. At the same time, this engagement may develop 21st century skills. Exploring the Intersection of Science Education and 21st Century Skills addresses key questions about the overlap between 21st century skills and scientific content and knowledge; explores promising models or approaches for teaching these abilities; and reviews the evidence about the transferability of these skills to real workplace applications.”
NWEA – //www.nwea.org
From the website: “NWEA is a research-based, not-for-profit organization that supports students and educators worldwide by creating assessment solutions that precisely measure growth and proficiency—and provide insights to help tailor instruction. For 40 years, NWEA has developed Pre-K–12 assessments and professional learning offerings to help advance all students along their optimal learning paths. Our tools are trusted by educators in more than 9,500 schools, districts, and education agencies in 145 countries.”
REL West note: One report published by NWEA is relevant to this request, as follows:
NWEA. (2018). Assessing soft skills: Are we preparing students for successful futures. Portland, OR: Author. Retrieved from //www.nwea.org/content/uploads/2018/08/NWEA_Gallup-Report_August-2018.pdf
From the foreword: “The NWEA Assessment Perceptions Study, our ongoing investigation into public perceptions of K–12 assessment, shows a pressing need for more discussion, attention, and focus regarding how we measure student growth and success. Previous studies commissioned by NWEA in 2012, 2014, and 2016 revealed the need for better assessment literacy among administrators, teachers, and parents—a better understanding of the purpose, value, and instructional use of different assessment types. While I believe there has been progress on this front under the Every Student Succeeds Act (ESSA), there is still a long road ahead before we make truly significant progress to support learning for all students. This year, we broadened the survey to address fundamentally important questions: How do we know if we are preparing students for their futures? What skills will students need, and how do we gather evidence of this learning? The results presented in this report quickly reveal an opportunity and call to improve assessment. They show that we are not aligned in the value of assessments given today—especially when it comes to the time they take, their purpose, and how results are communicated. Interestingly, there is agreement on the importance of measuring what we call ‘soft skills’ such as critical thinking, problem-solving, and teamwork. This year, for the first time, our study also included personal interviews with educators and parents. These reinforced our findings and also added a new perspective. Educators agree that soft skills are critical to student success and need to be directly taught and reinforced. But it was also acknowledged that these skills start at home.”
P21, Partnership for 21st Century Learning – //www.p21.org
From the website: “P21’s mission is to serve as a catalyst for 21st century learning by building collaborative partnerships among education, business, community, and government leaders so that all learners acquire the knowledge and skills they need to thrive in a world where change is constant and learning never stops.”
REL West note: One report published by P21 is relevant to this request, as follows:
Partnership for 21st Century Learning. (2007). 21st century skills assessment. Washington, DC: Author. Retrieved from //www.p21.org/storage/documents/21st_Century_Skills_Assessment_e-paper.pdf
Excerpt: “Meeting the demands of today’s world requires a shift in assessment strategies to measure the skills now prized in a complex global environment. The Partnership for 21st Century Skills believes that such a shift is vital to the widespread adoption of 21st century skills in our schools. We must move from primarily measuring discrete knowledge to measuring students’ ability to think critically, examine problems, gather information, and make informed, reasoned decisions while using technology. In addition to posing real world challenges, such assessments should accept a range of solutions to a task. For example, one possible assessment of 21st century skills would focus more on a student’s operational skills, such as her expertise in using multiple sources appropriately and efficiently, rather than on whether or not a correct response was submitted.”
Method
Keywords and Search Strings
The following keywords and search strings were used to search the reference databases and other sources:
(“Teacher feedback” OR “student feedback” OR “teacher assessment”) AND (“21st century skills” OR “soft skills”)
(Measuring OR assessing) AND (student) AND (“21st century skills” Or “soft skills”)
Databases and Resources
We searched ERIC for relevant resources. ERIC is a free online library of over 1.6 million citations of education research sponsored by the Institute of Education Sciences. Additionally, we searched Google Scholar and PsychInfo.
Reference Search and Selection Criteria
When searching and selecting resources to include, we consider the criteria listed below.
- Date of the Publication: References and resources published within the last 15 years, from 2003 to present, were included in the search and review.
- Search Priorities of Reference Sources: Search priority is given to study reports, briefs, and other documents that are published and/or reviewed by IES and other federal or federally funded organizations and academic databases. Priority is also given to sources that provide free access to the full article.
- Methodology: Priority is given to the most rigorous study designs, such as randomized controlled trials and quasi-experimental designs, and we may also include descriptive data analyses, survey results, mixed-methods studies, literature reviews, or meta-analyses. Other considerations include the target population and sample, including their relevance to the question, generalizability, and general quality. Priority is given to publications that are peer-reviewed journal articles or reports reviewed by IES and other federal or federally-funded organizations. If there are many research reports available, we select those with the strongest methodology, or the most recent of similar reports. When there are fewer resources available, we may include a broader range of information.
This memorandum is one in a series of quick-turnaround responses to specific questions posed by educational stakeholders in the West Region (Arizona, California, Nevada, Utah), which is served by the Regional Educational Laboratory West at WestEd. This memorandum was prepared by REL West under a contract with the U.S. Department of Education’s Institute of Education Sciences (IES), Contract ED-IES-17-C-0012, administered by WestEd. Its content does not necessarily reflect the views or policies of IES or the U.S. Department of Education nor does mention of trade names, commercial products, or organizations imply endorsement by the U.S. Government.