The Evolution of Assessment and Its Forces of Change
Posted September 02, 2015 by Dan LeClair
- Executive Vice President and Chief Operating Officer - AACSB International
Historically, assessment programs have been designed either to demonstrate accountability or to foster curricular improvements. Each approach leads us to a different place. Accountability favors more quantitative data and standardized instruments, enabling comparisons across institutions. Improvement is open to qualitative evidence and favors systems that enable comparisons over time. And observed gaps between expectations and performance have different meanings, especially to faculty, depending on whether data are intended for improvement or accountability.
Continuous improvement has always been the main driver for AACSB accreditation standards related to assessment; in fact, it was the sole driver until 2001, when expectations for accountability were added to the standards under the umbrella of assurance of learning (AoL). The addition was motivated by the desire to shift the weight of accreditation from inputs to outcomes. It was no longer enough to show improvement based on evidence; schools would be held accountable for achieving the learning goals they established for programs.
By most accounts, AACSB succeeded in tilting assessment programs toward accountability, stimulating deeper campus conversations about learning and about how to demonstrate that it has been achieved. But the underlying tension between improvement and accountability did not disappear. AACSB heard concerns about unanticipated consequences, including claims that the standards discouraged schools from using indirect measures, such as surveys of graduates and information provided by practitioner stakeholders; proliferation of cookie-cutter approaches (and learning goals) across schools; and assessment programs that seemed oddly disconnected from the curriculum. AACSB responded with revisions that gave AoL more context as a part of curriculum management, but did not withdraw the emphasis on accountability. Indirect measures would be encouraged, but direct measures of learning would still be required.
While it is too early to tell what impact these changes will have, it is clear that assessment for accountability as well as improvement is here to stay. And its role in business schools is being shaped as much by the changing environment as it is by accreditation.
Markets—companies and consumers—are emerging as key drivers of more transparency about learning. They are demanding more publicly available evidence about outcomes to help them make choices. But that’s not all. Companies, and students seeking jobs they offer, are beginning to expect credentials that have been earned by learners to link more directly to the competencies they have mastered. University degrees have not been good at that. As Keven Carey, author of The End of College: Creating the Future of Learning and the University of Everywhere, says, “Traditional college degrees are deeply inadequate tools for communicating information.”
These expectations will lead schools to capture more information as learning unfolds. While schools have been getting better at assessing outcomes at the end and, to a lesser extent, at the start of programs, there are many more opportunities to capture useful information in the middle, throughout the program, demonstrating progression and not just achievement. Focusing more on the middle, for example by storing the creative work of students, can provide a richer information set from which schools and their students can draw, to demonstrate competencies, determine future learning pathways, and make their case to employers.
Technology is shaping the future of assessment in at least three ways. First, new digital tools are helping schools to collect better data, and do so more easily, as well as empower educators to use it effectively. Products and platforms such as TracDat, eLumen, and LiveText provide digital solutions to schools and are connecting to myriad data sets to create more useful information.
Second, digital advances are changing the higher education market structure, with new companies such as Degreed emerging as intermediaries, enabling students and companies to compare credentials—degrees, specializations, certificates, and more—based presumably on learning outcomes.
Third, technology is having a profound impact on the types of outcomes being expected from business education. Of course, widespread instantaneous access to information has increased the importance of “doing” relative to “knowing.” Now there is evidence that technology has increased the importance of “social skills” relative to “quantitative skills.” Measuring our ability to apply knowledge in the workplace and our social skills development will take on a more elevated focus on most assessment programs.
The 2013 AACSB Accreditation Standards were designed to encourage more innovation and engagement with practice and to take the outcomes orientation a step further—toward measuring impact. Individually and together these objectives for accreditation will challenge schools to think differently about assessment, in terms of both accountability and improvement.
Curricular innovation is absolutely necessary in today’s environment. The question is, how will assessment programs that were built mostly for continuous improvement support more discontinuous and radical change? Fostering innovation will require assessment systems that are more flexible and agile and facilitate more comprehensive views of the curriculum—its content, pedagogy, structure, and purposes. Assessment for innovation will require more organizational agility—embedded assessments, more frequent reflection, and a culture of change. “Closing the loop” (demonstrating that assessment information has been used to initiate change), and closing it quickly, has never been more important. To foster innovation, evidence about learning also must be combined with information about changing markets.
AACSB accreditation standards are encouraging a stronger intersection between academe and practice. One way is by strengthening the learning that comes from experiences built into the curriculum. More experiential learning will create new opportunities to assess learning progress and outcomes, especially from stakeholders beyond faculty, and especially about the behavioral skills that matter most in workplace.
Finally, a movement toward evaluating schools based on impact will lead to new questions about our approaches to assessment. How will our experiences with assessment help us to think beyond learning goals and examine the longer-term impact and success of our graduates? What impact does our research have on learning outcomes? How do outreach activities and extracurricular activities impact learning and career success? These questions will continue to stretch the boundaries of assessment in higher education.
To summarize, the future of assessment is not about either improvement or accountability, it is about both. And it is increasingly about the capacity of schools to change—to innovate, to adapt to changing market expectations, and to demonstrate value and impact.