* Prof. Tjosvold would like to thank Robin Stanley Snell, professor of management and director of business programs at Lingnan University, for his contributions to the development of this article.

Student learning is central to all institutions of higher learning, and affirmation that graduates can achieve established learning expectations should be a key component of any high-quality academic program. As the AACSB Assurance of Learning (AoL) Standards identify, learning goals can be established in a wide variety of ways, and due to the differences in mission, faculty expectations, student body composition, and other miscellaneous factors, schools can vary greatly in how they demonstrate their goals.

Such variability is important, particularly for the Asia region, which serves as home for numerous business schools whose mission or core assumptions differ at the institutional level through their emphasis on research, liberal arts, or even teaching.

One Assurance of Learning Journey
For some business schools considering accreditation, establishing a successful AoL program might seem to be a daunting task. In seeking initial AACSB Accreditation, which we eventually gained early in 2011, the Faculty of Business at Lingnan University found the standards for Assurance of Learning (AoL) particularly challenging. As with other business schools, it took us several years to build and refine our AoL programs, generate sufficient faculty involvement, and demonstrate that we were able, on a continuous basis, to ‘close the loop.’

The mission and core assumptions of our Faculty of Business derives, in large part, from our parent university—resulting in an emphasis of a liberal arts education and ‘value addedness’ for students. This in turn, presented the following challenges and practical lessons in how we were to go about successfully shaping our AoL programs.

Applying Our Mission and Core Assumptions to AoL
Shortly after entering the pre-accreditation process, we held faculty-wide meetings to review the learning goals for our programs. In the case of our BBA, we sought to match the program level learning goals with the graduate attributes that already had been established for the university as a whole.

Reflecting the liberal arts mission, these graduate attributes emphasized broad and generic human qualities rather than discipline-specific knowledge and skills. Accordingly, the learning goals that we confirmed for the BBA program were almost exclusively generic in nature, and many of the core business and management disciplines and functions lacked corresponding goals at the program level.

The AACSB Assurance of Learning Standards white paper points out that direct assessment can take many different forms, including stand-alone testing and pre-program screening. However for us, most of these alternatives were discouraged because of how the university level mission was being interpreted internally. It was considered a matter of university policy to emphasize narrative-based assessment tasks.

Faculty members were strongly encouraged to provide nuanced feedback to students about their academic progress, and although permissible, multiple-choice formats and stand-alone testing were not recommended. Furthermore, the university’s emphasis on ‘value added’ education appeared to rule out the use of screening at entry as a means to demonstrate that we were closing the loop. These interpretations of the university’s mission steered us toward assuming that the basis of data collection for our AoL program should be course-embedded assessment, using consensus rubrics.

The Need for Balanced Learning Goals
In practice, these decisions and assumptions about doing AoL at Lingnan University made it very difficult to follow AACSB exhortations and spread faculty involvement in the AoL program beyond the capstone course. Many faculty members felt ill-prepared to participate in course-embedded assessment and we suffered a long impasse. Their diffidence was encapsulated in a private remark by one faculty member, widely admired for expertise in discipline-based teaching and research, who said, ‘I teach my subject, I don’t teach the skills of analysis, problem solving and decision making!’

In a concerted attempt to instill wider faculty member involvement in AOL, we enlisted several overseas consultants to work closely with business departments and faculty members. One consultant taught us our first main lesson: balance general learning goals with discipline-specific learning goals. He urged us to unfreeze our program level learning goals and to arrive at more goals for knowledge and skills within the business and management disciplines. This intervention served to unlock the door, and empowered faculty members to express their passion for discipline-based teaching. A number of articulate proposals ensued and the program level learning goals were duly revised. While this was happening, our program director liaised closely with the coordinators of the various core courses, securing their commitment to serve as venues for the course-embedded assessment of particular program level learning goals.

Learning the Art of Rubric-Writing
Learning to develop analytical consensus rubrics was another major challenge. We had begun with a rubric for the capstone course that comprised a list of criteria, along with space for qualitative comments as student feedback. This was inadequate for aggregating data about the overall performance of the student cohort on each performance dimension, and needed to be changed.

We drew on outside help to acquire the necessary guidance and insights about rubric development, and our second lesson was to make the rubrics no more complex than is absolutely necessary. Developing consensus rubrics that were fit to use for our own AoL programs required us to combine language-based, subject-based, and educational assessment-based expertise. This took a long time, and necessitated much discussion across the business school, which required the hands-on involvement of the program director.

Collecting Data
We developed a ‘commitment table’ to summarize what we thought had been agreed about the key aspects of our AoL programs, such as the venues for the various course-embedded assessments, and the associated consensus rubrics. However, we were subsequently surprised to discover that consensus among a course teaching team about a particular rubric did not mean that the rubric would actually be used. Some academics assumed that our AoL programs were mere ‘paper systems,’ and some academics ‘forgot’ to use the rubrics, or lost their data. These were serious omissions, as it distorted the sampling process and rendered our overall data set unreliable.

Our third lesson, therefore, was not to rely on oral promises about collecting data for AoL but to insist upon formal commitment and accountability in writing. Academic quality assurance work is routine-driven. A new AoL system takes time to get stamped in as an organizational routine, and this requires constant and specific reminders at various meetings and via emails. The program office reviews the commitment table at a meeting with the core course coordinators at the beginning of each academic year. In the middle of the academic term, core course coordinators receive emails informing them of the submission deadline for the assessment data. Reminders are issued a few weeks before the deadline. These constant reminders have proved effective, so far!

Aggregating Data
Following several ‘drought’ years, during which AoL data was not forthcoming, the program office eventually became flooded with data. Data aggregation into an AoL data summary report was an arduous and time-consuming task because instructors reported their data idiosyncratically. Our fourth main lesson from experience was that it is necessary to develop standardized assessment report forms, based on the rubrics, for course coordinators to complete. They are now requested to provide simple arithmetical counts corresponding to the cells in their rubrics. This might seem obvious, but it took us a couple of years to realize the need to do this.

Closing the Loop
After the first couple of iterations, we found that the quality targets for most, but not all, of our BBA program learning goals were being met. However, it was the exception that caught our attention. Our fifth main lesson was that a properly executed AoL program can flag up surprising shortfalls in student attainment of program level learning outcomes. We were actually pleased to find an exception, not only because it alerted us to an area for improvement but also because this was a clear sign to AACSB that we were not ‘cooking the books,’ and that we were treating the data objectively and looking hard at how to make adjustments in program delivery. Suffice to say that when we made the necessary adjustments, there were sufficient improvements.

Lingnan University and its business school are in many ways unique. We hope, nonetheless, that these lessons will provide useful guidance to the increasing numbers of business schools in Asia that are embarking on their own AoL journeys.