Data-Driven Diversity

When companies realize they are falling short in improving their operations, expanding their offerings, or connecting with customers, they typically define what they want to achieve, identify relevant metrics, and then try out various strategies until the metrics reveal progress toward their goal. It’s a practice that works, and businesses use it to address any problem they truly care about. Hence the aphorism “We measure what we treasure.”

But something odd is going on when it comes to diversity, equity, and inclusion (DEI). Although tracking data is key to doing better in this arena, most companies have yet to adopt evidence-based, metrics-driven practices—even though they’ve acknowledged DEI as a moral imperative and recognize how it can help their bottom line.

That makes no sense. The fact is, without metrics to measure their current status and monitor progress, DEI efforts will always amount to shooting in the dark. And that can be very costly, as CFOs are starting to realize. According to Harvard Kennedy School’s Iris Bohnet, U.S. companies spend roughly $8 billion a year on DEI training—but accomplish remarkably little. This isn’t a new phenomenon: An influential study conducted back in 2006 by Alexandra Kalev, Frank Dobbin, and Erin Kelly found that many diversity-education programs led to little or no increase in the representation of women and minorities in management.

If you want meaningful change, it’s not enough to simply tout the importance of diversity. Think about it this way: Suppose a firm with weak sales decided to address the situation by conducting an earnest, companywide conversation about how much everyone values sales and then organizing a national Celebrate Sales Month. Would you expect a big jump in sales as a result?

Probably not. A company that’s committed to solving its problems uses metrics to identify trouble spots, establish baselines, and measure progress. So why aren’t companies doing that in the DEI arena?

Much of the answer has to do with risk. All too often, when an HR chief or a DEI head proposes a metrics-based plan for achieving DEI goals, it gets rejected because others in the company worry about the legal exposure it creates.

To be fair, some types of diversity metrics can indeed be useful to plaintiffs’ employment lawyers, because they can provide concrete evidence of discrimination against a protected group. It’s understandable, then, that in-house attorneys and midlevel managers may be uncomfortable collecting certain information for fear of helping adversarial lawyers write class-action briefs.

But although it can be risky to gather diversity data, there are also risks in not doing so. Companies today acquire data about virtually everything else, so their failure to track diversity statistics sends a message of indifference—or, worse, may be taken as evidence that the company has allowed bias to flourish.

In fact, there is no cause for alarm about embracing DEI metrics. Companies regularly handle sensitive information and shoulder legal risk to achieve their business goals. How many firms, for example, would refuse to analyze and fix their cybersecurity vulnerabilities because they were worried that doing so might expose them to a data-breach lawsuit? The crucial question corporate leaders have to ask is this: Do we really want to wait until after we’ve been sued to learn that our DEI record is problematic?

One of us (Joan) is an academic who has studied inequality in the workplace for more than 30 years, advised companies on implementing metrics-driven DEI practices, and taught employment law. The other (Jamie) is an employment lawyer who assists companies with DEI initiatives. This article, which reflects recent work we’ve done with colleagues, will help you take stock of your DEI goals, assess your tolerance for risk, and adopt practices that will make it easier to reach your goals while also mitigating the risks.

Outcome metrics indicate only whether you have a problem. Process metrics will tell you exactly where to focus your attention to bring about meaningful change.

There’s really no mystery about how to implement a metrics-based approach to diversity that gets results. Your organization probably already has protocols for handling sensitive information, whether it’s product-recall secrets or customer data, and you’ve probably developed thoughtful procedures for conducting sensitive internal investigations. You need to use those protocols and procedures to handle DEI data as well.

Choose the Right Metrics

Many companies assume that diversity metrics are all about the “body count”—how many women, people of color, and perhaps members of other underrepresented groups they employ and in what positions. Those are outcome metrics, and they’re important. They’re a good indicator of bias; they’re vital for establishing a baseline against which progress can be measured; and they’re necessary for assessing the effectiveness of various interventions.

Any company committed to DEI goals needs to attentively track outcome metrics. Doing that, and making the numbers public, is even required by law in some contexts. But outcome metrics indicate only whether you have a problem, not where it’s arising or how to fix it. And you may find yourself with a PR challenge: If all you’re doing in your DEI work is tracking demographics and then haphazardly trying to address the issues that surface, you’re likely to end up reporting the same outcomes year after year. That can be a public-relations disaster, and of course it can also have a corrosive effect on employee morale.

To do better, you need process metrics, which can pinpoint problems in employee-management processes such as hiring, evaluation, promotion, and executive sponsorship. If your outcome metrics tell you, say, that you don’t have enough women or people of color on your staff, process metrics will tell you where exactly to focus your attention to bring about meaningful change. Examples of these metrics include the speed at which people of color move up the corporate ladder and the salary differential between men and women in comparable jobs.

So what kind of problems can process metrics help you identify? Work we’ve done with colleagues has shown that process metrics can reveal the “prove-it-again bias” that obliges women and nonwhite men to provide more evidence of competence than white men do. Process metrics can also locate “tightrope bias,” which rewards white men for being authoritative and ambitious but often penalizes members of other groups for behaving in the same way. On that front, for example, a 2020 study by Shelley Correll and coauthors found that tech company workers who “took charge” tended to receive the highest ratings on performance evaluations—but only if they were men.

The value of process metrics can be easily seen in the context of hiring. Problems generally arise in one or more of four areas: recruitment, résumé review, interviewing, and the making and negotiating of offers. In each case, if you identify a problem, there’s a different fix. For example, to diversify your candidate pools, the solution may be additional outreach—perhaps to historically Black colleges and universities. To correct for bias in the review of résumés, you need to train reviewers to identify and interrupt bias. (The simple two-page training worksheet available at biasinterrupters.org can help; our research shows that worksheets of this sort, when read aloud to employees, can correct for bias against both women and people of color.) To reduce interview bias, at a minimum you’ll need a set of structured interview questions so that everyone is asked the same thing, including some skills-based or job-specific questions to ensure that what candidates are asked is relevant. You’ll also need specialized training for your interviewers, to teach them not to write off women and people of color as either “too meek” or “too much.”

You should incorporate both outcome and process metrics in your plan for achieving your diversity goals. But you also need to consider your tolerance for the risks that come with a metrics-based approach.

Determine Acceptable Risk

Employment lawyers control for risk all the time, so we recently assembled seven prominent ones to discuss how companies can best protect themselves when using diversity metrics. The group included the former general counsel of a large multinational corporation, other in-house lawyers, and law-firm attorneys who represent employers. Together we created a clear and simple road map for organizations to follow.

The first step is to assess risk tolerance. Some companies are highly risk-tolerant when it comes to DEI metrics. They believe that diversity, equity, and inclusion are core values, and they’re absolutely committed to fostering them. They see inclusion as an important business goal, so they’re willing to shoulder some risk to achieve it.

Other companies are much warier of getting into hot water with DEI data, sometimes because they’re just generally risk-averse. The good news for highly risk-averse companies is that even if they’re unwilling to use metrics, they can still use bias interrupters. Ample research shows, for example, that bias in performance reviews can be interrupted by giving managers a structured form to use, with objective criteria for evaluation, and asking them for concrete evidence to justify all their ratings. When we did this with one company, in conjunction with an hourlong workshop, levels of bias fell sharply, and every group, including white men, got more action-oriented feedback.

Of course, using bias interrupters without tracking metrics is less than ideal, because it means you can’t determine a baseline. Lacking that, you can’t measure progress or objectively assess whether your interventions have worked. But using bias interrupters without metrics is certainly better than doing nothing.

In the end, the CEO and other leaders in the C-suite—not midlevel in-house lawyers or HR managers—are the ones who must decide how much risk to shoulder in the DEI context. Once they do, they need to make their position clear to others in the company. Here’s a solid rule of thumb: You should be willing to risk as much in the DEI arena as you would in any other arena in which you have important business goals.

Create a Plan for Action

Many companies are committed to fostering DEI and would like to adopt a metrics-based approach, but some are highly focused on doing so in a way that will minimize their legal exposure. The following guidelines will work for companies with differing appetites for risk.

Be ready to act on what you find. This is crucial for every organization. Before you begin collecting and analyzing data, make sure you have buy-in at the top and the budget to take persistent, reasonable measures to remedy any problems you find. Remember that you don’t need to solve every problem immediately, and your response doesn’t have to be perfect. But it does have to be prompt—don’t wait around for six months or a year.

Start small. Launching a pilot is a good idea, because it will help you fine-tune your intervention in an iterative fashion before rolling it out companywide—a sensible course of action when it comes to both organizational change and risk management. Starting small also helps ensure that you can identify an effective way to make progress toward your goal without undue delay. Pilots should be overseen by a cross-disciplinary change-management team that has a clear mandate, specific goals, and a limited time frame. That team will need an engaged executive sponsor and a manager who is a diversity champion, or at least someone who is open-minded and willing to be guided by HR or the project team. Don’t try to solve every problem in one fell swoop, and keep in mind that making progress on a single issue may require a multifaceted intervention.

Build the business case. Building the business case means persuading key stakeholders at your company that attention to diversity, equity, and inclusion can help them succeed in their jobs. Make clear to your CEO and the board how it can enhance your firm’s products and services, its public image, and its profits. If you are a consumer-products company, show managers how effective DEI programs can make it easier to connect with diverse customers. If you are a tech company, emphasize how inclusion can help you avoid bias in your artificial intelligence. (Facial recognition systems, for example, have been notorious for misidentifying people of color.) If you are in finance, point to the finding by Sheen S. Levine and colleagues that ethnically diverse teams are 58% more likely than homogeneous ones to price stocks correctly. Managers also need to know that team performance stands to benefit from racial and gender diversity. Scholars have found, for instance, that a team’s collective intelligence is more than twice as important as individual members’ intelligence in determining how well the team performs, and that gender diversity predicts higher collective intelligence. Other research indicates that when groups are racially mixed, they engage in less groupthink and work harder.

Control expectations through careful messaging. Don’t create expectations you can’t fulfill. The most effective messaging is that the company is wholeheartedly committed to unlocking the potential of its most important asset—its human capital. But DEI challenges reflect the fact that bias commonly colors many talent-management activities (from recruiting employees to developing and retaining them), and it’s often a factor in informal workplace interactions too. The best path to achieving DEI goals is to aim for a sustained series of small, incremental improvements. Success will take time, so as John Kotter recommends in his organizational-change model, use your metrics to measure and celebrate wins.

Consider limiting access to your metrics. A key difference between risk-averse and risk-tolerant companies is openness about diversity metrics. Some companies widely share enough data to paint a clear picture of the company’s DEI profile, and this approach has many advantages. In our work, especially with companies with a high level of antidiversity backlash, we have found that letting managers see key statistics can supercharge buy-in.

But that is not the only option. Risk-averse companies can restrict the dissemination of DEI data in the same way that they restrict the dissemination of any other sensitive information, giving access only to those who are already accustomed to handling such material. For such companies, the real question is this: To create an effective DEI program, who needs to know your metrics and corrective-action plans? You’ll want somebody to analyze the data and take the lead on developing the plans. Typically, that’s someone in human resources. You’re also likely to want your head of HR, your DEI officer, the members of your project team, and its executive sponsor to be informed.

All companies should think carefully about which data to share widely and which to hold closer to the vest. Sometimes keeping the circle tight can be important for morale, because any problems you find might take some time to solve. Not everybody in the company has to know every metric you are tracking.

Create a DEI data protocol. Remember that when it comes to DEI risks, problems often arise not from the metrics themselves but from what people do as a clearer picture of the company comes to light: the notes they take, the emails they send, the discussions they engage in. Anyone with access to diversity metrics needs to be trained to know what is and isn’t permitted on this front and how to recognize information that should be shared only orally, not in writing. And there should certainly be no joking on paper: Often that’s where the trouble starts. People also need to be trained to avoid drawing legal conclusions about the way any employee is treated. That’s a job for lawyers.

Risk-averse companies should consider creating a data protocol with several key characteristics. It should name and describe your project. It should identify the members of your project team, including a well-trained “data protocol officer” who is in charge of properly gathering and using sensitive information. It should clarify who is authorized to analyze the data. It should establish a procedure for adding new members to the team. It should delineate the scope of the team’s work. And it should make two more things clear: that nobody may share sensitive information outside the team without the data protocol officer’s approval, and that any violation of the protocol may lead to disciplinary action.

Risk-averse companies may also want to consider limiting written communications until the company’s leaders have determined their DEI priorities. The safest approach is for members of the project team to initially present the data orally to senior leaders, with no note-taking allowed—a perfectly routine and sensible practice. The leaders can then decide which problems to address, in which order, and communicate their plans in writing.

Create a sound data-retention policy. Most companies already have a policy in place regarding how long to keep other kinds of data. If possible, adapt that for use with DEI data. But whether you choose that course of action or start from scratch, make sure that your policy complies with local and other laws. And follow the general principle that you should retain your data only as long as necessary to identify problems and measure the effectiveness of specific DEI interventions.

Consider framing key documents so that they qualify as “privileged.” Some highly risk-averse companies try to structure certain critical documents, particularly corrective-action plans, in ways that maximize their chances of being protected by attorney-client privilege. This can be a good idea in certain contexts, notably if your company has already been sued for discrimination. That said, be careful if you choose this approach. It may shield you from risk, but it may also undermine your efforts to make your workplace more diverse, equitable, and inclusive. Research has shown that when companies devise and frame DEI initiatives as responses to legal risk, employees often go into a defensive crouch.

And that, of course, is not a good stance from which to make progress.

About the Research

The framework presented in this article grew out of the meetings of a working group on the topic of tracking diversity metrics while controlling for legal risk. The group consisted of Cynthia Thomas Calvert (principal at Workforce 21C), Laura Maechtlen (partner at Seyfarth Shaw), Gilda Malek (deputy general counsel for employment and litigation at Confluent), Joyce M. Norcini (former general counsel at Nokia Siemens Networks), Wayne Stacy (executive director of the Berkeley Center for Law & Technology), Emily Gould Sullivan (vice president of the legal group at Ross Stores), Janine Yancey (CEO of Emtrain), and the authors. Heather Lanyi provided outstanding research assistance.

Copyright 2022 Harvard Business School Publishing Corp. Distributed by The New York Times Syndicate.