Received: May 02, 2023, Manuscript No. isr-23-100616; Editor assigned: May 05, 2023, Pre QC No. isr-23-100616 (PQ); Reviewed: May 19, 2023, QC No. isr-23-100616; Revised: May 24, 2023, Manuscript No. isr-23-100616 (R); Published: May 31, 2023, DOI: 10.17719/jisr.2023.100616
Socio-emotional and motivational skills are routinely measured using self-reports in large-scale educational assessments. Measures exploiting test-takers behaviour during the completion of questionnaires or cognitive tests are increasingly used as alternatives to self-reports in the economics of education literature. We compute behavioural measures of socio-emotional and motivational skills using data from the Programme for International Student Assessment (PISA). However, these measures are only limitedly correlated among themselves and have low correlations with self-report measures of the same constructs. This is likely a reflection of the fact that behavioural measures are representations of the test taker current ‘state’, rather than descriptions of the participant view of their own ‘trait’ like the self-report measures. Moreover, the low correlation across measures suggests that they capture different behavioural responses to the test-taking situation. These differences are still limitedly understood because the measures are constructed ex-post using collateral information collected during the administration of assessments rather than developed ex ante in line with theoretical models of human cognition and affect.
Traditionally, large-scale assessments in education have focused primarily on measuring cognitive skills and academic knowledge. However, there is a growing recognition that socio-emotional and motivational skills play a crucial role in a student's overall development and success. As a result, educators and policymakers are increasingly interested in incorporating behavioral measures of socio-emotional and motivational skills into large-scale assessments. This article examines the importance of assessing these skills and explores the potential benefits and challenges associated with incorporating behavioral measures in large-scale assessments.
The emphasis on socio-emotional and motivational skills - the term we adopt in this work - and the interest in how they can be promoted has prompted policy makers as well as researchers to consider how their measurement can best be integrated in benchmarking and accountability systems at international, regional and national levels. Standardized low-stakes large-scale tests such as the Programme for International Student Assessment, the Programme for International Assessment of Adult Competences, the IEA's Trends in International Mathematics and Science Study, the Progress in International Reading Literacy Study (PIRLS) as well as regional and national standardized tests have, in recent years, introduced the measurement of selected socio-emotional and motivational skills within questionnaires administered to participants, alongside the measurement of cognitive skills.
The increased attention devoted to understanding the role played by socio-emotional and motivational skills has also spurred a parallel and lively debate in the economics of education literature over what exactly low-stakes large-scale tests measure and the importance of factors such as test taking motivation as a determinant of differences in measures of cognitive skills across countries and population subgroups
Understanding Socio-Emotional and Motivational Skills
Socio-emotional and motivational skills encompass a wide range of abilities that are essential for personal, social, and academic well-being. These skills include self-regulation, resilience, empathy, teamwork, self-efficacy, and motivation. They influence a student's capacity to manage emotions, build positive relationships, set and achieve goals, and navigate challenges effectively. Assessing these skills provides a more comprehensive understanding of a student's capabilities beyond traditional academic measures.
The Value of Behavioral Measures
Behavioral measures offer several advantages in assessing socio-emotional and motivational skills within large-scale assessments:
Holistic Assessment: Incorporating behavioral measures allows for a more comprehensive evaluation of a student's abilities, taking into account their non-cognitive skills alongside academic performance. This provides a more accurate picture of their overall development and readiness for future challenges.
Real-World Relevance: Socio-emotional and motivational skills are critical for success in various aspects of life, including education, careers, and personal relationships. Assessing these skills through behavioral measures helps ensure that students are equipped with the skills necessary for navigating the complexities of the real world.
Personalized Support: Behavioral measures can identify specific areas where students may require additional support and interventions. This information can guide educators in tailoring instructional strategies and interventions to address individual student needs effectively.
Enhanced Accountability: Including socio-emotional and motivational skills in large-scale assessments sends a strong message about the value placed on these skills in educational settings. It encourages schools and policymakers to prioritize the development of these skills alongside academic achievement.
Challenges and Considerations
Incorporating behavioral measures of socio-emotional and motivational skills in large-scale assessments comes with certain challenges that need to be addressed:
Measurement Validity: Developing reliable and valid measures for assessing socio-emotional and motivational skills can be complex. It requires careful consideration of factors such as cultural context, age appropriateness, and alignment with desired outcomes.
Standardization and Comparability: Ensuring consistency and comparability across different schools and contexts is a significant challenge in large-scale assessments. Standardizing behavioral measures while considering diverse cultural backgrounds and educational settings is crucial to maintain fairness and equity.
Teacher Training and Buy-In: Successfully implementing behavioral assessments requires adequate teacher training and support. Educators need to understand the purpose of these assessments, be equipped with the necessary skills to administer them, and embrace the value of assessing socio-emotional and motivational skills.
Ethical Considerations: Collecting behavioral data raises ethical concerns regarding student privacy, informed consent, and data security. It is essential to establish robust protocols to protect student information and ensure transparency in data usage.
This study provides a comprehensive evaluation of the indicators of PSEC that can be derived by examining students’ behaviour when completing the PISA cognitive tests and questionnaires. A first contribution of the study consists in demonstrating that these measures are only limitedly affected by the use of different administration protocols or instruments. Our findings show that country differences in the behavioural measures tend to be relatively stable over successive cycles of the survey.
The results of this study confirm previous findings on the existence of considerable differences in measures of socio-emotional and motivational measures both across and within countries. Behavioural measures indicate that boys have lower levels of PSEC than girls, while boys tend to report that they are more perseverant. Socio-economically disadvantaged students tend to report lower levels of self-reported perseverance, but the gaps between the two groups of students are less marked according to behavioural indicators. These differences across the two sets of measures suggest that self-reports partly reflect subjective judgments that are influenced by gender and socio-economic status.
The behavioural measures of PSEC are also found to be strongly correlated with achievement across all PISA domains, and this correlation is robust to the inclusion of self-reported measures. A novel finding based on longitudinal data from four countries is that some measures of students’ behaviour on the test, such as performance decline, are also strongly associated with life outcomes ten years after the test.
Assessing socio-emotional and motivational skills through behavioral measures in large-scale assessments holds tremendous potential for enhancing educational practices and promoting holistic student development. By incorporating these measures, educators and policymakers can gain valuable insights into students' non-cognitive abilities, tailor instruction to individual needs, and foster the growth of crucial skills necessary for success in the 21st-century world. However, careful attention must be paid to measurement validity, standardization, teacher training, and ethical considerations to ensure the effectiveness and fairness of these assessments. Ultimately, a comprehensive assessment approach that encompasses both cognitive and non-cognitive skills can pave the way for more inclusive and student-centered education systems.