Beyond Rankings: Building Better Research Intelligence

• Knowing where a school ranks is only one part of understanding its research performance.
• CHART combines transparent research benchmarks with flexible tools that allow schools to examine performance through different priorities and assumptions.
• By revealing strengths, gaps, and potential pathways to improvement, research intelligence can help leaders make more informed, mission-aligned decisions.

 
Rankings have become an unavoidable part of business school life. They shape reputation, influence faculty recruitment and partnerships, and sometimes affect decisions about resources and strategy.

Yet, for all the attention they receive, most rankings answer only one question: Where does a school stand? They are much less useful in answering the questions academic leaders want to know: Why did we receive this result? Which parts of our research performance are improving? Where are we falling behind? What kinds of investments might make a difference going forward?

The Limitations of Existing Rankings

A ranking that does not enable a school to explore its position is mostly a scoreboard. Business schools need information that helps them understand their performance and make informed decisions about how to improve.

This is not a uniquely Canadian problem. Business schools around the world operate in very different national systems, yet many are assessed through rankings that rely on reputation, emphasize a narrow set of outputs, or offer little visibility into their methodologies.

Large and small schools are typically placed on the same scale without accounting for faculty size or budget. Total research output can favor larger schools with greater faculty capacity, while per-capita results—research performance relative to faculty size—can favor smaller schools with a few highly productive researchers. Neither measure tells the whole story on its own.

Reputation measures also present a significant challenge because, although reputation is not meaningless, it can be very slow to change and difficult to verify. A school may continue to rank highly because it is widely believed to be strong, even when current research data do not support that view. Another school may have improved substantially in research performance, but its reputation may take years to catch up to its reality. The result can be a system that reinforces established views rather than one that tests them against current evidence.

Designing a More Useful Framework

These limitations became clear at the University of Calgary’s Haskayne School of Business as we worked toward an ambitious goal of becoming one of Canada’s top five business schools for research. Many well-known rankings focus primarily on programs, student experience, employability, or reputation, with research accounting for only a small (if any) portion of the result. The small number of rankings that do focus on research often rely on publications in a narrow list of journals.

Our response was to develop the Calgary-Haskayne Adaptive Research Tracker, or CHART, a transparent research-benchmarking framework initially built for Canadian business schools. The authors of this article contributed to CHART’s build—Sandy Hershcovis as project lead and Aja Rieger as manager of the project development team that also included a data scientist, the university’s Office of Institutional Analysis, and its Office of the Vice-President (Research).

CHART’s design was informed by consultations with senior leaders from six business schools in Canada’s U15 leading research-intensive public universities, as well as broader discussions with research deans and associate deans at a national business school leadership conference.

These consultations focused on the questions leaders wanted to answer through benchmark data, the indicators they considered meaningful, the need for consistent faculty definitions, and the trade-offs between comparability and mission. The team selected indicators and weights before collecting data, using three criteria: strategic relevance, consistent measurement across institutions, and public or reproducible data.

The consultation process did not produce one universally preferred definition of research excellence. Leaders differed on their opinions about appropriate journal lists, other indicators, and weights. That disagreement became an important design insight: CHART needed a consistent benchmark for comparison, but it also needed a way for schools to question the defaults and examine performance through a lens that better reflected their own missions.

The inaugural benchmark includes 28 Canadian business schools that chose to participate in the project. They supplied faculty lists using a common definition of full-time, research-stream faculty. The participating group spans schools of different sizes, missions, and institutional contexts.

From Research Rankings to Research Intelligence

Canada served as the defined setting for us to build and test the framework, but the issues CHART addresses extend well beyond one country. The framework includes two connected tools:

• CHART-Index provides a consistent benchmark across 28 Canadian business schools. The same indicators and methodology are applied to each school, creating a common point of comparison. Results are available in both absolute and per-capita terms, allowing users to distinguish between research strength arising from institutional scale and from productivity relative to faculty size.
• CHART-Index Flexible (CHART-IF) allows users to change assumptions, adjust priorities, and explore “what if” scenarios. For example, a school can examine what might happen if it increased research funding, hired additional research faculty, improved publication performance, or placed greater emphasis on research impact, helping its leaders decide how to improve its position.

The methodology is fully documented, and the indicators use publicly available or independently verifiable sources so users can see how results were produced. The goal is not simply to produce another ranking; it is to give academic leaders greater visibility into what drives their results and how different strategic choices could affect future performance.

What CHART Measures

The fixed benchmark brings together four broad dimensions of research performance: 

• Research Inputs: The total value of competitive grants awarded through Canada’s three federal research agencies and held by a school’s faculty as principal investigators.
• Scholarly Outputs: Publications by a school’s faculty in FT50 journals, identified through OpenAlex using author and institutional-affiliation data.
• Research Impact: Academic and policy impact of publications attributed to a school, rather than faculty members’ broader publication records. Academic impact reflects publications in the top 10 percent most cited among comparable works. Policy impact captures citations in policy documents, identified through DOI matches in Overton.
• Researcher Recognition: Senior and emerging research honors held by a school’s faculty, including Canada Research Chairs and recognition through the Royal Society of Canada.

Together, these measures show the resources that support research, the work scholars produce, the reach of that work, and the recognition researchers receive. They are not the only forms of excellence, but these indicators can be measured consistently through a single, mostly open platform where users can compare and analyze data.

From Data to Decisions

The practical value of CHART lies in moving from the fixed benchmark ranking to a more detailed diagnosis of research performance using CHART-IF.

Consider a school that is near the middle of the overall benchmark but performs very differently across the underlying measures. It may rank highly in research funding and policy citations while lagging in publication output. A conventional ranking would show only the middle position. CHART would reveal a more specific pattern: The school successfully attracts resources and produces work that reaches policy audiences, but it may struggle to convert funded projects into publications within the defined journal set.

In response, leaders might examine whether the pattern is concentrated in particular disciplines, whether publishing in such outlets is a school objective, whether publication performance is improving over time, and whether faculty have the time, research supports, doctoral capacity, or mentoring needed to move work through the publication pipeline.

In CHART-IF, users could model whether a realistic increase in publication performance would meaningfully change the school’s position. If the change requires modest effort, it may represent an actionable opportunity. If it would require a large shift, the school may decide that another goal is more consistent with its mission and resources.

A second school might perform strongly per capita but weaker in the overall comparison because it has a small faculty. Its leaders might conclude that it is already performing exceptionally well for its size and resist pressure to chase the scale of larger institutions. Or they could test whether a few targeted research-stream hires would strengthen the school’s total output and overall position without reducing per-capita productivity.

Conversely, a large school with strong absolute results but weaker per-capita performance might focus more resources on supporting existing faculty rather than further expansion.

In this sense, there is no single prescription for improving a school’s CHART position. The data may show several pathways; the practical value lies in making those pathways and their scales visible so leaders can weigh them against cost, feasibility, and school mission.

One Part of a Wider Solution

We are conscious that CHART does not escape all the problems associated with ranking systems. Although it considers a broader range of research indicators than many existing rankings, it still does not capture the full extent of researchers’ contributions. For example, efforts that cannot be easily measured include the originality of an idea, the long-term influence of a body of work, the value of research to a particular community, or the ways scholarship shapes teaching, professional practice, and public debate.

Many business schools are responding to calls—including from AACSB—to emphasize and measure impact, but impact remains difficult to quantify and measure. CHART therefore serves as one quantitative input into a fuller, mission-aligned portfolio of impact evidence rather than a definitive account of research excellence. Its methodology makes the choices behind the framework visible. With CHART-IF, users can question those choices, change the weighting of indicators, and consider what the tool may be leaving out.

This openness is important because measurement can influence institutional priorities and behavior. CHART is intended to support judgment and decision-making—not replace them. Its results are most useful when they initiate, rather than settle, a discussion about research strategy.

Expanding Beyond Canada

Our experience in Canada is a first step toward a broader question: What would research benchmarking look like if it were designed to help business schools learn and grow in mission- and context-aligned ways, rather than compete? Answering that question internationally will require collaboration. AACSB’s framework makes a related case for collective action, calling on accrediting bodies, scholarly societies, journals, funders, and ranking organizations to coordinate rather than work in isolation. CHART is one contributor to that shared effort.

Haskayne’s longer-term goal is to gradually expand CHART beyond Canada. The framework will need to evolve as it enters new regions, while still offering a common basis for comparison across regions. Business schools in different countries will need to be key partners in determining which indicators are meaningful, which comparisons are fair, and which forms of research contribution should be included.

Rankings will continue to attract attention from schools and their stakeholders. By making the drivers, assumptions, and possible pathways visible, CHART intends to turn research rankings into research intelligence, helping business schools understand not only where they are but how they got there and what choices will help them move forward.