April 29, 2018
Looking Below the Surface to Close Achievement Gaps and
Improve Career Readiness Skills
Marilee Bresciani Ludvik, Ph.D.
While institutions have been investing in efforts to close achievement gaps, they still remain. With a growing emphasis on data analytics, some institutions may discover additional interventions that will benefit students while others may simply reinforce past behavior and potentially increase achievement gaps. This manuscript utilizes emerging neuroscience to introduce malleable learning dispositions that align with desired career readiness skills. In addition, this manuscript shares inquiry methodology that can help institutions ensure they are not creating more harm with the use of their data analytic strategies and potentially improving career readiness skills for all students.
There has been a great deal of emphasis on using data analytics to close achievement gaps among varying identity groups and intersecting of identity groups as defined by persistence, graduation rates, and time-to-degree. For some institutions, applying just in time academic and student support initiatives predicted as necessary by data analytics has been fruitful. For other institutions, this approach may be less welcomed as it may not account for institutional leaders’ desire to understand individual students’ needs or to critically examine how well the institution is transforming its alleged historical deficit mindset. Furthermore, using historically predictive analytics without an understanding of how those analytics intersect with students’ attainment of desired career readiness skills could potentially increase achievement gaps as opposed to decreasing them. With increasing emphasis on preparing career readiness competencies such as social emotional intelligence, self-awareness, global citizenship, compassion, pro-social behavior, and lifelong learning skills and abilities, this manuscript seeks to offer a different lens through which to collect data in order to close achievement gaps while also ensuring optimal career readiness preparation.
In 2016 and 2017, a synthesis of learning and development research was published by the Institute of Educational Sciences and the National Academies of Sciences respectively. In 2018, the National Academies of Sciences released another synthesis of research in a book entitled How People Learn II: The Science and Practice of Learning. Within these manuscripts and this book, decades of research reported how malleable desired career readiness skills are and subsequently provided some ways in which they could be cultivated and assessed within in- and out-of-class educational settings. What was also made clear in this book is that culture and context play an important role in understanding how people learn. “Learning does not happen in the same way for all people because cultural influences pervade development from the beginning of life” (p.22). And while many scholars have been exploring the influence of internal and external influences on learning, the research is still in a nascent stage.
While there is no question that socio-cultural groupings of students and the intersection of socio-cultural groupings of students is illuminating achievement gaps across the country, there are many complications to identifying ways to improve learning and development based on socio-cultural groupings and the predictive metrics that accompany those conversations. To quote from How People Learn II (NAS, 2018) “Research on genetic differences among population groups has established that there are not scientifically meaningful genetic differences among groups commonly identified as belonging to different races (Smedley and Smedley, 2005). It has long been recognized by social scientists that race is a social construction and that criteria for inclusion in a racial category or definition of particular groups as racial ones have varied over time (see, e.g., Figueroa, 1991; Kemmelmeier and Chavez, 2014; Lopez, 2006)” (p.24). Furthermore, perspectives on what constitutes culture and how it relates to learning and development have changed over time thus further complicating data analytics.
Having said this, there are a number of studies that illustrate how culture plays a role in basic cognitive processes that help learners understand and organize the world, such as attention, memory and perception of self and others, as well as the cognitive processes that shape learning (Chua et al., 2005; Cole, 1995; Rogoff and Chavajay, 1995; Markus and Kitayama, 1991; Nisbett et al., 2001; Gelfand et al., 2011; Kitayama and Cohen, 2007; Kronenfeld et al., 2011; Medin and Bang, 2014; Segall et al, 1966). Furthermore, students’ environmental experiences and personal choices change certain portions of their brains necessary for learning and development. And since there is clear evidence that human beings have a wide variety of diverse environmental experiences and personal choices and that not all human beings have the same opportunities to learn and develop, neurodiversity is a fact that educators must contend with simply based on the variety of lived experiences each student has already had prior to matriculating to each institution.
Given that neurodiversity exists and its presence may not be easily identifiable by social groupings, how might we consider allowing for malleable learning dispositions (e.g., desired career readiness skills) that could be culturally constructed in order to ensure the closing of achievement gaps while also optimizing career readiness learning and development? To respond to this, let us consider the iceberg analogy of learning dispositions presented in Kuh et al (2018) and adapted here. In Figure 1 below, you see a list of several learning dispositions listed, underneath the surface of the water, where researchers have affirmed that educational environments can contribute to improving. The understanding from cognitive, social, and emotional neuroscientists is that these dispositions are indeed malleable and the assumption is that it is our responsibility to cultivate these toward positive goal-oriented behavior such as degree attainment. Furthermore, many of these learning dispositions map directly onto desired career-readiness skills expressed by employers.
Figure 1. Iceberg Analogy of Learning Dispositions
Figure 1 illustrates that many of our efforts to identify achievement gaps within our institutions rests in large part on the measurements of indicators listed above the water line. Measurement tools such as tests, standardized exams, time-to-degree, and persistence are easy to gather measures. And many current data analytic practices are seeking to understand students’ behavior as it correlates with or predicts these indicators. That kind of data may be useful to many institutions, however, it neglects to account for a great deal of underlying conditions or learning dispositions that are known to contribute to easy to identify learning. These learning dispositions also tightly align with employer desired career readiness skills. How do we get at better understanding those, particularly given neurodiversity?
Figure 1 is presented in an iceberg analogy to showcase a portion of Otto Scharmer’s (2009) organizational behavior change Theory U. In Scharmer’s organizational behavior change theory, leaders must conduct their own deep dive into understanding why their performance metrics are the way they are. The deep dive process, illustrated in Figure 2, requires an understanding of patterns of past behavior, which data analytics can shed light upon. However, understanding past patterns of behavior is not simply gathering data to identify a pattern, rather, according to Scharmer, the intention is to unearth the identification of deep-seated beliefs, values, mental models, and systemic structures to explain what informs the creation of those identified patterns of behavior. Analyzing the systemic structures that contribute to the patterns of behavior involves awareness of the values, assumptions, and mental models that have shaped these behavior patterns. As you can see by the model, this requires refraining from acting upon the easy to identify performance indicator data and instead, leveraging leaders’ increasing awareness brought about by intentional reflection to examine ways of being and doing that have caused their organizations’ past failures. It is an exploration of the systems of belief, values, and attitudes that have informed policies, practices, and behavioral expectations which reside underneath the obvious question as to why the performance indicators might look the way they do.
Figure 2. Otto Scharmer’s Iceberg Model
Returning to Figure 1, if organizational leaders begin collecting data on how known malleable learning dispositions is cultivated in every individual and then compare those strategies within and across groupings and sub-groupings of individuals using pre- and post- assessment measures along with first-person direct self-report experience, then perhaps we can begin to better understand how organizational and individual context and culture influence easy to identity above the surface data. Engaging in this kind of inquiry requires an investment of time to collaborate, design, and pilot evidence-based strategies known to cultivate learning dispositions. It also requires evidence that can be meaningfully compared across groups. Thoughtfully administered pre- and post- assessment measures across varying learning and development opportunities analyzed by various groupings and sub-groupings of students along with gathering individual students’ voices of their learning experience can signal to leaders what is working for whom, under what conditions, and why. This kind of inquiry is likely not possible for many institutions under the current systemic structural assumptions that one size fits all and the assertion that assessing students’ learning and development is a waste of time. It also may not be possible unless organizational leaders are really willing to think critically at how educational opportunities are designed and delivered and how those who contribute to expected learning and development are hired, on-boarded, provided with professional development to adopt and adapt learning science design and evaluation, and required to do this kind of work as a part of their employment contract.
There are a number of free, valid and reliable pre-and post-learning disposition/career readiness questionnaires and measures available to assess desired skills. We refer to these as equity indicators because neuroscientists have reported that these are malleable skills but we don’t yet know how individual students’ lived experiences may have already cultivated these or whether they need to be cultivated within their higher education experience. What we do know is that they are related to desired career-readiness skills and students’ ability to demonstrate what they do know and have learned.
As such, the time-consuming portion of this diving underneath the surface inquiry process is gathering meaningful student voice. Embedded reflective journal prompts, digital narratives, 360-degree evaluations, and thoughtfully constructed reflective student portfolios provide a wealth of data about students’ internal processes of meaning making. While this type of inquiry is time-consuming and that there is no such thing as more time, without organizational leaders gaining a deeper understanding of what is working well for whom via the use of pre-and post- learning disposition measures (e.g., equity measures), we can’t know where to allocate the precious resource of time and to whom and when. Gathering pre-and post-learning disposition data along with first-person self-direct report of experience could ground dialogue for priority decisions around who needs something different than what we have been providing in order to succeed. This is equity.
Figure 3 summarizes the context of this invitation to higher education administrators. Neurodiversity exists; while there are some very real genetic and epigenetic differences in some students that influence their ability to learn and develop in expected ways, it remains a fact that not every human being is experiencing the same thing externally or internally in any given moment. All of that is shaping each person’s ability to learn and develop even when the same opportunity for learning and development is provided. Organizational and individual culture and context do matter when it comes to ensuring every student has an opportunity to achieve. While we have historically offered one size fits allor one size fits this social group models to meet the increasing demand of access to education, we have plenty of evidence of not meeting expectations of employers for all students’ high achievement of career readiness skills. In order to fix this problem, we can look at historical data to see who is predicted to succeed in the one size fits all models in the hopes we can change students’ behavior or provide interventions that reinforce that historical behavior. Or we can seek to better understand who our students are, how they are experiencing what we provide them, and do so in tandem with the measurement of the malleable learning and development skills employers want us to positively influence. We can also understand how specific students acquisition of these skills is influencing their persistence, time-to-degree, cumulative GPA, graduation rates and employability. Without this type of evidence, we risk never searching below the easy to identify indicators in order to avoid reinforcing continued achievement gaps.
Figure 3. Educational Context for Equity
In closing, if institutional leaders seek to ensure they are not perpetuating achievement gaps or inadvertently increasing them, and they also seek to assure the cultivation of malleable learning dispositions that ensure career readiness, then they may find it useful to respond to the following questions.
- What malleable learning dispositions does our institution value?
- How well do our valued learning dispositions map to our employers’ desired career readiness skills?
- Where are we providing opportunities for these skills to be cultivated, how, and to whom specifically?
- How are we gathering first-person direct self-report of these learning experiences from the students?
- How are we collecting evidence that the desired learning disposition/career readiness skills were acquired?
- How are we comparing this evidence gathered across social grouping and sub-groupings to identify how well the cultivation of these skills is allowing certain groupings and sub-groupings of students opportunities to achieve?
- How is what we are learning from this evidence providing us with opportunities to re-think our mental models, beliefs, values, and behaviors around previously conceived notions for how all students succeed?
- And how well are we using this data and dialogue to refine specific experiences so that all students have an opportunity to achieve at high levels?
Chua, H.F., Boland, J.E., and Nisbett, R.E. (2005). Cultural variation in eye movements during
scene perception. Proceedings of the National Academy of Sciences of the United States
of America, 102(35), 12629–12633.
Cole, M. (1995). Culture and cognitive development: From cross-cultural research to creating
systems of cultural mediation. Culture & Psychology, 1, 25–54.
Figueroa, P. (1991). Education and the Social Construction of “Race.”New York: Routledge.
Gelfand, M.J., Raver, J.L., Nishii, L., Leslie, L.M., Lun, J., Lim, B.C., Duan, L., Almaliach, A., Ang, S., Arnadottir, J., Aycan, Z., Boehnke, K., Boski, P., Cabecinhas, R., Chan, D., Chhokar, J., D’Amato, A., Ferrer, M., Fischlmayr, I.C., Fischer, R., Fül.p, M., Georgas, J., Kashima, E.S., Kashima, Y., Kim, K., Lempereur, A., Marquez, P., Othman, R., Overlaet, B., Panagiotopoulou, P., Peltzer, K., Perez-Florizno, L.R., Ponomarenko, L., Realo, A., Schei, V., Schmitt, M., Smith, P.B., Soomro, N., Szabo, E., Taveesin, N., Toyama, M., Van de Vliert, E., Vohra, N., Ward, C., and Yamaguchi, S. (2011). Differences between tight and loose cultures: A 33-nation study. Science, 332(6033),1100-1104. doi: 10.1126/science.1197754.
Kemmelmeier, M., and Chavez, H.L. (2014). Biases in the perception of Barack Obama’s skin
tone. Analyses of Social Issues and Public Policy, 14, 137–161. doi 10.1111/asap.12061.
Kitayama, S., and Cohen, D. (2007). Handbook of Cultural Psychology. New York: Guilford.
Kronenfeld, D.B., Bennardo, G., de Munck, V.C. and Fischer, M.D. (eds). (2011). A Companion
to Cognitive Anthropology.Hoboken, NJ: Blackwell.
Kuh, G. D., Gambino, L. M., Bresciani Ludvik, M., & O’Donnell, K. (2018, February).Using ePortfolio to document and deepen the impact of HIPs on learning dispositions (Occasional Paper No. 32). Urbana, IL: University of Illinois and Indiana University, National Institute for Learning Outcomes Assessment. Retrieved from http://learningoutcomesassessment.org/occasionalpaperthirtytwo.html
L.pez, I.H. (2006). White by Law.New York: New York University Press.
Markus, H.R., and Kitayama, S. (1991). Culture and the self: Implications for cognition, emotion, and motivation. Psychological Review, 98(2),224–253. doi.org/10.1037/0033-295X.98.2.224.
Medin, D.L., and Bang, M. (2014). Who’s Asking? Native Science, Western Science, and Science Education. Cambridge, MA: MIT Press.
Nisbett, R.E., Peng, K., Choi, I., and Norenzayan, A. (2001). Culture and systems of thought:
Holistic versus analytic cognition. Psychological Review, 108(2),291–310.
National Academies of Sciences, Engineering, and Medicine. (2018). How People Learn II: Learners, Contexts, and Cultures. Washington, DC: The National Academies Press. doi: https://doi.org/10.17226/24783.
National Academies of Sciences, Engineering, and Medicine. (2017). Supporting
Students’ College Success: The Role of Assessment of Intrapersonal and Interpersonal Competencies. Washington, DC: The National Academies Press. https://doi.org/10.17226/24697.
Rogoff, B., and Chavajay, P. (1995). What’s become of research on the cultural basis of cognitive development.American Psychologist, 50(10), 859–877.
Scharmer, O. (2009). Theory U: Leading from the future as it is emerging.Oakland, CA: Berrett-Koehler Publishers.
Segall, M.H., Campbell, D.T., and Herskovits, M.J. (1966). The Influence of Culture on Visual
Perception. Indianapolis, IN: Bobbs-Merrill Company.
Smedley, A., and Smedley, B.D. (2005). Race as biology is fiction, racism as a social problem is real: Anthropological and historical perspectives on the social construction of race. American
Zelazo, P.D., Blair, C.B., and Willoughby, M.T. (2016). Executive Function: Implications for Education (NCER 2017-2000)Washington, DC: National Center for Education Research, Institute of Education Sciences, U.S. Department of Education.