Do Hospitals Led by Physician CEOS Have Better Leapfrog Scores?

More than 30 years ago, patient safety emerged as a goal worthy of prioritization.(1-4) In 1999, the Institute of Medicine’s (IOM) report “To Err Is Human” provided clarity in terms of the definitions, scope, and magnitude of medical errors in healthcare.(5, 6) In follow-up, the IOM provided a broader framework that positioned patient safety among the five other quality domains and led to an emphasis on patient safety and quality that persists today.(6)

The Leapfrog Hospital Grade is a well-accepted benchmark that allows consumers to compare hospitals over time and to each other based on several patient safety metrics, and use these scores to guide their healthcare decisions.(8) The ratings are derived from the Leapfrog Hospital Survey and responses are run through publicly available scoring algorithms.(9)

While Leapfrog publishes Hospital Safety Grades for more than 2,700 hospitals in the United States, certain hospitals are exempt from participating in the survey, including critical access hospitals, long-term care and rehabilitation facilities, and several federal and specialty hospitals.(9)

The chief executive officer (CEO) has oversight for the hospital’s strategy, culture, and day-to-day operations, all of which influence patient safety. As hospitals increasingly seek physician CEOs, the independent contribution of a physician CEO to the organization’s outcomes, including quality and patient safety, become relevant.(10-11)

Only a few empirical studies and meta-analyses have investigated the impact of physician leaders on patient outcomes; these studies are dated and equivocal in terms of their results.(12-17) Consequently, we investigated whether hospital physician-CEOs had any effect on patient outcomes, represented by Leapfrog Hospital Safety Grades.

Method and Study Design

Data were derived from the 2016 American Hospital Association (AHA) Annual Survey Database and the Leapfrog Hospital Survey. The AHA Annual Survey Database is a comprehensive hospital database produced primarily from the AHA Annual Survey of Hospitals and reports information on hospital organizational structure, service lines, utilization, finances, and staffing.(18) The final sample consisted of acute care hospitals that voluntarily participated in the 2016 AHA Annual Survey of Hospitals and the Leapfrog Hospital Survey.

Using publicly available data, we identified each hospital CEO as holding either a Doctor of Medicine or Doctor of Osteopathy degree. Then, using the Leapfrog website (www.LeapfrogGroup.org ) each hospital’s Leapfrog Grade was recorded for the spring and fall of 2017. Hospitals that did not have a Leapfrog Hospital Safety Grade in both spring and fall of 2017 were excluded from analysis.

Variables

Independent Variables

The AHA Annual Survey of Hospitals included categorical and continuous variables and the first and last name of each hospital’s CEO for the year 2016. Categorical variables included state, bed size, and a control variable to indicate hospital ownership and governance. Continuous variables were total facility admissions, adjusted admissions, total facility inpatient days, adjusted patient days, total facility Medicare discharges, total facility Medicaid discharges, total births (excluding fetal deaths), total surgical operations, emergency department visits, total facility personnel full-time equivalent (FTE), and adjusted average daily census.

Coded Independent Categorical Variables

Bed Size: The AHA database included a predefined eight-level variable for bed size. Similar to Leapfrog, we excluded hospitals with a bed size of 24 or less and compressed the variable into three categories: 25–199, 200–399, and 400 or more. A bed size of 25–199 was chosen as the reference group in the logistic regression models to intuitively compare results against larger hospitals.

Region: Hospitals were coded into four statistical regions defined by the U.S. Census Bureau.(19) Each hospital was assigned to either West, Midwest, South, or Northeast based on the state that was provided in the AHA database. The South was chosen as the reference in the regression model because it was the largest category of the four regions.

Control: A predefined 18-category variable representing the controlling body over the hospital was provided in the AHA database. Additionally, AHA provided groupings into four categories: (1) Government, nonfederal; (2) Nongovernment, not-for-profit; (3) Investor-owned, for-profit; and (4) Government, federal.

Gender: The CEO’s gender was identified using the CEO’s name and assigning a gender, dichotomously ascribed as male or female. For those names that were ambiguous with respect to gender, for example Randy or Chris, a specific search using LinkedIn and Google was used to identify the hospital CEO’s gender.

Physician Status: The CEO’s medical education was identified using a specific search in LinkedIn and/or Google. Physician status data were collected only for hospitals that had a published Leapfrog Hospital Safety Grade. The CEO’s name was searched and dichotomously ascribed as physician or non-physician based on the person being identified as holding a medical or osteopathic doctoral degree. Non-physician CEOs were chosen as the reference group in the logistic regression models because it was the largest and most normative group to study key differences against physician CEOs.

Dependent Variables from Leapfrog

There were three dependent variables, one for each logistic regression model. All were derived from the Leapfrog Hospital Safety Grade with five ordinal categories ranging from the highest grade of an “A” to the lowest grade of an “F” in both spring and fall.

The first model included an outcome generated by combining the spring and fall Leapfrog grades. Each letter grade was converted into a numerical score with A = 5, B = 4, C = 3, D = 2, and F = 1. The spring and fall scores were combined to generate an overall score ranging from 2 to 10 for the year 2017, with the most desirable score being 10.

The overall score was categorized into three ordinal groups: low grade (score of 2–4), medium grade (score of 5–7), and high grade (score of 8–10), representing a combined score of spring and fall. In addition, the spring and fall grades were analyzed separately in regression models with the assigned letter grade (A, B, C, D, and F) as the dependent variable.

Leapfrog scores embed many granular patient safety metrics. The overall grade, measured from A to F, is well validated as an outcome; includes hospital level input, particularly around improvement efforts; is measured at six-month intervals; is peer reviewed; and is consistently used to benchmark hospitals to one another.(19) In addition, given that there were two Leapfrog grades for the year of data, we compared the difference in scores from the fall to the spring to determine if there was improvement, deterioration, or stabilization based on the exposure of having a physician CEO.

Statistical Analysis

Using Statistical Analysis Software (SAS) version 9.4, we examined the frequency distribution of physician status, gender, bed size, and region across Leapfrog grade categories and examined the chi-square test p-values for a significance level of alpha<0.05 (see Table 1). We also tested for trends in Leapfrog grades across bed size categories using Cochran-Mantel-Haenszel test. Finally, we included those categorical independent variables in a bivariate logistic regression; those with a significance level of alpha<0.05 were included in the multivariate logistic regression models.

Included in this analysis are three different multivariate logistic regressions models and their respective odds ratios (OR). First, we conducted a regression model with the outcome being the combined score of either low, medium, or high, modeled toward the value of high (see Table 2). Next, we ran regression models for spring and fall separately to verify our results of the combined score. Both spring and fall models were conducted with a five-level outcome variable consisting of the Leapfrog Safety Grade (A, B, C, D, or F) modeled toward a value of “A” (see Table 3).

Continuous variables representative of patient volumes were tested for normality and their descriptive statistics and p-values from the Kruskal-Wallis test were compared for differences across the three categories of Leapfrog grades at a significance level of <0.05. Finally, the change in Leapfrog scores from spring to fall were examined by subtracting the generated spring score from the generated fall score. The frequency and proportions of those who improved, deteriorated, or remained stable were compared across physician CEO groups.

Results

As illustrated in Table 1, the 2016 AHA database contained 6,239 hospitals; of those, 1,926 (31%) had a Leapfrog Hospital Safety Grade published in both the spring and fall of 2017. The 4,313 hospitals that did not have a published Leapfrog Safety Grade were excluded from regression analysis. Hospitals that participated with Leapfrog were significantly larger, with 18% having a bed size of 400+ compared to 4% of hospitals that did not participate with Leapfrog (p<.0001). Those that participated with Leapfrog had a greater proportion of not-for-profits (67% vs 42%, p<.0001), whereas hospitals that did not participate with Leapfrog had a greater proportion of for-profits (30% vs 21%, p<.0001).

Among hospitals with a published Leapfrog Safety Grade, 139 (7%) were led by physician CEOs and 1,787 (93%) were led by non-physician CEOs. Physician CEOs were more often leading larger hospitals, with 37% of physician CEOs at hospitals with a bed size of 400+ compared to 17% of non-physician CEOs (p<.0001). A majority (81%) of hospitals with physician CEOs were not-for-profit compared to 66% of non-physician CEOs (p<.0001).

As illustrated in Table 2, findings from the bivariate analysis found that CEO gender was not significantly associated with Leapfrog grade; thus, gender was not included in the multivariate regression models. Physician status also had no association with Leapfrog grade, but was included in the multivariate models because it is the basis of the primary research question.

The combined Leapfrog grade was found to trend lower as bed size categories increased (p = 0.0165), indicating an inverse relationship with 58% percent of small hospitals (bed size of 25–199) receiving high Leapfrog grades compared to 49% of hospitals with a bed size of 400 or more. Bed sizes of 200–399 and 400+ were associated with a lower Leapfrog grade (OR: 0.76, CI: 0.62–0.93, p = 0.0085 & OR: 0.72, CI:0.56–0.92, p = 0.0085 respectively), when compared to hospitals with a bed size of 25–199. In contrast, hospitals located in the Midwest were found to be associated with a higher Leapfrog grade (OR: 1.29, CI:1.01–1.63, p = 0.0352) when compared to hospitals located in the South.

We verified that our results using the combined Leapfrog grade were consistent when examining the spring and fall Leapfrog grades separately. As Table 3 illustrates, the same relationships persisted, showing no association between physician status and Leapfrog grade for either spring or fall. The inverse relationship between bed size and Leapfrog grade persisted in both spring and fall, with the strongest association found in the spring. Hospitals with a bed size of 400+ were less likely to receive a spring grade of “A” when compared to hospitals with a bed size of 25–199 (OR: 0.65, CI: 0.52–0.81, p = 0.0002). Additionally, 38% of hospitals with a bed size of 25–199 received an A in the spring versus 22% of hospitals with a bed size of 400+ (p = 0.0008).

The patient volume indicators detailed in Table 4 were not normally distributed across combined Leapfrog grade categories; thus, nonparametric testing methods were used and medians, instead of means, were compared. The Kruskal-Wallis tests demonstrated significant differences in patient volumes across the three categories of combined Leapfrog Safety Grade, specifically for total facility admissions, inpatient days, adjusted patient days, Medicaid discharges, and adjusted average daily census. The descriptive statistics in Table 4 show that hospitals with the largest median patient volumes were in the low Leapfrog grade category.

Table 5 demonstrates the change in Leapfrog grades from spring to fall by physician status. The proportion of hospitals that improved, diminished, and remained stable in their Leapfrog grades were nearly the same for both physician and non-physician CEOs. The overall change in Leapfrog grades ranged from a decrease of two letter grades to an increase in up to four letter grades from spring to fall. In both CEO groups, roughly 65% of hospitals had the same letter grades for both spring and fall, with non-physicians having slightly more hospitals receiving two “A’s” (38% versus 34%) than physician CEOs.

Discussion

Using AHA and Leapfrog data, we sought to understand the relationship between the CEO’s physician status and Leapfrog Safety Grade for the hospitals they lead. We found that hospitals that participated in the 2017 Leapfrog Survey represented only 31% of hospitals in the AHA database and that among those that participated in the survey, a relatively low percentage of those CEOs were medical or osteopathic doctors. When present, physician CEOs tended to be male and work at hospitals with larger bed sizes and larger patient volumes.

With increased attention paid to value in care delivery and clinical outcomes,(1-7) one might assume that hospitals led by physician CEOs would be more likely to have higher Leapfrog grades than those led by non-physician CEOs; however, we found that both physician and non-physician CEOs led hospitals in nearly equal proportions of Leapfrog grade categories, even when controlling for bed size and region.

Our data suggest that bed size may be the most prominent factor known to influence Leapfrog grades. We found that as bed size increases, the odds of receiving a high grade inversely trended downward, with this relationship persisting across categorical and continuous variables representing hospital size and patient volumes. We verified that the same relationship was present when examining the spring and fall grades individually.

Our findings demonstrated that median patient volume indicators were highest in the low Leapfrog grade category. While few studies have examined the relationship between hospital size and patient safety, mixed results have been reported. Schlenker, et al., were unable to detect an association between patient volumes and outcomes; however, the data are older and less powerful.(20)

Romano, et al., found that rural hospitals had lower rates of adverse events than urban teaching hospitals(21); however, the existing literature concurs that adverse safety events are highly correlated to the clinical specialty they take place in, such as trauma, surgical, and emergency departments.(2, 22)

It is possible that services correlated with safety events are not offered at smaller hospitals, thus causing them to receive higher patient safety ratings. In addition, geographic region may play a role in Leapfrog Safety Grade, as we found that hospitals in the Midwest scored higher than hospitals in the South.

Finally, we found no material differences in Leapfrog Safety Grade between male and female CEOs.

Few studies have investigated the association between medical and non-medical hospital leadership and their respective performance outcomes.(23) Slonim, et al., sought to understand the relationship between physician status and Hospital Consumer Assessment of Healthcare Providers and Systems (HCAHPS) scores in 12 rating categories.(24) They found that, when controlling for bed size, a physician CEO was significantly associated with higher scores in care transition, discharge information, overall hospital rating, pain management and recommended hospital rating.(24)

Slonim, et al., found no association between physician status and cleanliness rating, medication communication, doctor and nurse communication, quietness rating, and staff responsiveness rating.(24) Similarly, Slonim, et al., found that as bed size increased, the odds of scoring higher decreased across all 12 HCAHPS rating categories and that gender had no association with HCAHPS rating.(24)

This research also has important limitations. First, it is a cross-sectional cohort study of a single year; therefore, it may not be representative of trends in performance. Second, the designation of a CEO’s gender and status as a physician were performed using publicly available information and is subject to miss-classification bias. We believe that these biases, while possible, are limited by the use of multiple data sources to interrogate the variables of gender and physician status. Third, there may be unknown variables that were not captured in the AHA dataset.

Another important limitation is that this sample of 1,926 hospitals is not a representative sample of all U.S. hospitals and that data would have been more comprehensive had Leapfrog scores been available for all hospitals. There may be biases that lead some CEOs to be more or less likely to participate in the Leapfrog Hospital Survey.(25-26) Our sample represents only hospitals that voluntarily participated with Leapfrog and we were unable to assess outcomes for hospitals that did not participate in the survey.

In addition to this participation bias, there could be a systematic bias in generating the Leapfrog grades, given the inverse relationship we identified between bed size and grade. It is recommended that future research collect data on all CEOs in the AHA database and use alternative outcomes beyond Leapfrog scores so that the relationship between physician status and outcome might be better understood.

Despite these limitations, this study has significant strengths. First, this study provides one of the first national perspectives of physician CEOs and their Leapfrog performance compared to their non-physician colleagues in hospitals greater than 25 beds. Second, this is both an empiric and more contemporary study than what currently exists in the literature. Finally, the focus on specific and well-validated outcomes, namely Leapfrog scores, that are important for a hospital’s value based performance are essential to understand the role of personal traits, like physician status, on outcome.

Physician CEOs bring unique experiences and expertise to their positions that may reveal themselves in terms of objective performance measures. Conducting further empiric studies using contemporary datasets at scale and investigating other important hospital outcomes are warranted.

References

1. Hiatt HH, Barnes BA, Brennan TA, et al. A Study of Medical Injury and Medical Malpractice. The New England Journal of Medicine. 1989;321: 480–84. doi: 10.1056/NEJM198908173210725.

2. Brennan TA, Leape LL, Laird NM, et al. Incidence of Adverse Events and Negligence in Hospitalized Patients. Results of the Harvard Medical Practice Study I. The New England Journal of Medicine. 1991;324:370–76. doi: 10.1136/qshc.2002.003822.

3. Leape LL, Brennan TA, Laird N, et al. The Nature of Adverse Events in Hospitalized Patients. Results of the Harvard Medical Practice Study II. The New England Journal of Medicine. 1991;324:377–84. doi: 10.1056/NEJM 199102073240605.

4. Localio AR, Lawthers AG, Brennan TA, et al. Relation Between Malpractice Claims and Adverse Events Due to Negligence. Results of the Harvard Medical Practice Study III. The New England Journal of Medicine. 1991;325:245–51. doi: 10.1056/NEJM199107253250405.

5. Institute of Medicine (US) Committee on Quality of Health Care in America. To Err is Human. Washington, DC: National Academies Press (US);2000.

6. Institute of Medicine (US) Committee on Quality of Health Care in America. Crossing the Quality Chasm. Washington, DC: National Academies Press (US);2001.

7. President’s Advisory Commission on Consumer Protection and Quality in the Health Care Industry. Quality First: Better Health Care for All Americans. Washington, DC: Author;1998. https://govinfo.library.unt.edu/hcquality/final/index.htm .

8. The Leapfrog Group. History. The Leapfrog Group. www.leapfroggroup.org/about/history .

9. The Leapfrog Group. About the Grade. How Safe is Your Hospital? The Leapfrog Group. www.hospitalsafetygrade.org/about-the-grade .

10. Staff. The Case for Physician CEOs. Leadership & Management. Becker’s Hospital Review. March 2, 2015. www.beckershospitalreview.com/hospital-management-administration/the-case-for-physician-ceos.html .

11. Figueroa J, Krubert C. Why a Physician May Be the Best Choice to Lead an Ailing Organization. Medical Economics. September 16, 2019. www.medicaleconomics.com/view/why-physician-may-be-best-choice-lead-ailing-organization .

12. Clay-Williams R, Ludlow K, Testa L, Li Z, Braithwaite J. Medical Leadership, a Systematic Narrative Review: Do Hospitals and Healthcare Organizations Perform Better When Led By Doctors? BMJ Open. 2017;7(9):e014474. doi:10.1136/bmjopen-2016-014474.

13. Schneller ES, Greenwald HP, Richardson ML, et al. The Physician Executive: Role in the Adaptation of American Medicine. Health Care Manage Rev. 1997;22(2):90–96. doi:10.1097/00004010-199704000-00010.

14. Schultz FC, Pal S, Swan DA. Who Should Lead a Healthcare Organization: MDs or MBAs? J Healthc Manag. 2004;49(2):103–16. PMID: 15074119.

15. Clay-Williams R, Braithwaite J. Doctors in Executive Management: A Systematic Review of the Peer-reviewed Literature. Sydney, Australia: Centre for Clinical Governance Research, Australian Institute of Health Innovation, University of New South Wales; 2012. https://www.mq.edu.au/__data/assets/pdf_file/0010/81478/DoctorsinExecutiveManagement_Mar13.pdf .

16. Xirasagar S, Samuels ME, Curtin TF. Management Training of Physician Executives, Their Leadership Style, and Care Management Performance: An Empirical Study. Am J Manag Care. 2006;12(2):101–8. PMID: 16464139.

17. Xirasagar S, Samuels ME, Stoskopf CH. Physician Leadership Styles and Effectiveness: An Empirical Study. Med Care Res Rev. 2005;62(6):720–40. doi:10.1177/1077558705281063.

18. American Hospital Association. AHA Annual Survey Database. 2016. www.ahadata.com/aha-annual-survey-database .

19. US Census Bureau. Census Regions and Divisions of the United States. Census.gov . www2.census.gov/geo/pdfs/maps-data/maps/reference/us_regdiv.pdf .

20. Schlenker RE, Hittle DF, Hrincevich CA, Kaehny MM. Volume/outcome Relationships in Small Rural Hospitals. J Rural Health. 1996;12(5):395–409. doi: 10.1111/j.1748-0361.1996.tb00807.x. PMID: 10166136.

21. Romano PS, Geppert JJ, Davies S, Miller MR, Elixhauser A, McDonald KM. A National Profile of Patient Safety in U.S. Hospitals. Health Aff (Millwood). Mar-Apr 2003;22(2):154–66. doi: 10.1377/hlthaff.22.2.154. PMID: 12674418.

22. Loux SL, Payne SMC, Knott A. Comparing Patient Safety in Rural Hospitals by Bed Count. Advances in Patient Safety: From Research to Implementation (Volume 1: Research Findings). Rockville, MD: Agency for Healthcare Research and Quality (US); 2005. www.ncbi.nlm.nih.gov/books/NBK20441 .

23. Clay-Williams R, Ludlow K, Testa L, Li Z, Braithwaite J. Medical Leadership, A Systematic Narrative Review: Do Hospitals and Healthcare Organisations Perform Better When Led by Doctors? BMJ Open. 2017;7(9):e014474. https://doi.org/10.1136/bmjopen-2016-014474 .

24. Slonim AD, See H, Shreve L, Slonim S. Do Hospitals Led by Physician CEOs Have Better HCAHPS Scores? Physician Leadership Journal. July-August 2021;8(4):38–46.

25. Smith SN, Reichert HA, Ameling JM, Meddings J. Dissecting Leapfrog: How Well Do Leapfrog Safe Practices Scores Correlate With Hospital Compare Ratings and Penalties, and How Much Do They Matter? Medical Care. 2017;55(6):606–14. https://doi.org/10.1097/MLR.0000000000000716 .

26. Staff. Leapfrog Safety Grades Penalize Hospitals for Transparency, Study Suggests. Becker’s Hospital Review. March 29, 2017. www.beckershospitalreview.com/quality/leapfrog-safety-grades-penalize-hospitals-for-transparency-study-suggests.html?oly_enc_id=9841H3880512J2D .