Payer Class Bias in Patient Satisfaction Surveys

Patient satisfaction is playing an increasingly prominent role in the provision and evaluation of healthcare in the United States. In recent years, the CMS has been transitioning to value-based payment systems. Since the Medicare Access and CHIP Reauthorization Act (MACRA) was passed in 2015, many providers who care for Medicare patients have been required to participate in quality reporting or face automatic reimbursement penalties.(1,2) These quality measures include patient satisfaction, which is assessed through questionnaires such as the Consumer Assessment of Healthcare Providers and Systems or the Press Ganey Medical Practice Survey. Results from patient satisfaction surveys are then used to evaluate provider performance, determine physician reimbursement, and compare institutions.

Inclusion of these survey scores in the value-based payment model assumes that patient satisfaction reflects value and quality of care. However, questions remain regarding the validity of this assumption. An increasing number of studies have implicated inherent biases that may influence patient satisfaction. Several studies have suggested that nonmodifiable patient factors, such as gender,(3) travel distance,(4) ethnicity,(5) education level,(5) and insurance status,(3,6,7) can influence patient satisfaction scores. If this is true, providers’ scores may be more reflective of their patient population than actual quality of care.

Given the impact of patient satisfaction surveys on provider evaluation and reimbursement, it is important to understand the factors that affect scoring. To date, no review analyses have been published on whether patient healthcare coverage status biases scoring of physicians. Understanding the relationship, if any, between payer mix and satisfaction scores is especially salient due to the widespread implementation of value-based payment systems from MACRA 2015. This systematic literature review and meta-analysis is the first to investigate whether there is evidence of bias in patient satisfaction scoring of physicians in the outpatient clinic setting based on insurance status.

Materials and Methods

A focused literature search for the years 2015 through 2020 was conducted on PubMed for the terms: “patient satisfaction” and “insurance status”. We limited our search to peer-reviewed publications in English, in the outpatient clinic setting, and with patient satisfaction scoring of physicians. P-values of less than .05 were considered statistically significant. Random-effects methods were used for statistical analysis of combined patient satisfaction data across studies.(8) A forest plot of odds ratios was generated using R software (dmetar package).(9) In the absence of odds ratios provided directly by the primary source, odds ratios were calculated using provided mean, median, standard deviation, and effect size values.(10) Odds ratio meta-analysis was conducted only for the Medicare patient group because data pertaining to other insurance groups was lacking or heterogeneously collected across studies. Each study included in the meta-analysis was weighted by sample size.

Results

The results of the literature search are listed in Tables 1, 2, and 3. Table 1 includes all 16 publications meeting the inclusion criteria along with the corresponding method of surveying patient satisfaction. The Press Ganey Survey was the tool most commonly used to measure patient satisfaction (10 out of 16 studies). (Press Ganey Associates is a third-party healthcare consulting group that administers patient satisfaction surveys for hospitals reporting to the CMS.) Other patient satisfaction surveys used included variations of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) and Patient Satisfaction Questionnaire Short Form (PSQ-18).

The 16 publications included a total of 39,015 presumably unique patients who completed satisfaction surveys. The results of each study pertaining to the relationship between insurance status and patient satisfaction are presented in Table 2. Ten out of 16 studies found a statistically significant (p <.05) difference in patient satisfaction scores across third-party payer (insurance) groups. Five out of 16 studies found differences in scoring across insurance groups that were not statistically significant. Only 1 out of 16 studies found no difference in how physicians were rated across insurance groups.

Although most studies found differences in satisfaction scores across insurance groups, there is less agreement on which groups specifically implicate a positive or negative bias. Table 3 presents the findings from all selected studies sorted by the following payer classes: Medicare; Medicaid; private/commercial; uninsured/self-pay; and worker’s compensation. Each payer class category includes at least one study that suggests positive scoring bias and at least one study that suggests negative scoring bias. Despite mixed results across studies, several payer class groups demonstrate a stronger trend toward either positive or negative bias.

Medicare patients were found to give higher satisfaction scores in seven included studies, five of which were statistically significant. Only two studies had statistically significant findings of Medicare patients giving lower satisfaction scores. The studies indicating that Medicare patients confer a positive bias (n=24,306) outnumber those indicating a negative bias (n=498) in sample size as well (Table 3). Odds ratio meta-analysis of Medicare patient satisfaction scores across studies relative to private/commercial patient satisfaction scores resulted in a weighted average odds ratio of 1.3102 (Figure 1), with Medicare patients consistently giving higher scores. The combined 95% confidence interval lies entirely above the 1.0 line, indicating a statistically significant difference in scores between Medicare and private/commercial insurance groups. Despite some mixed results, there is strong evidence that Medicare is associated with higher satisfaction scores.

Figure 1. Forest plot of Medicare patient satisfaction relative to private/commercial patient satisfaction results. Bible et al(11); Buie et al(14); Johnson et al.(16); Mordhorst et al.(21); Rane et al.(17); Rane et al.(18); Rogo-Gupta et al.(15); Wells et al.(20).

Conversely, patients classified as either uninsured/self-pay or worker’s compensation were found to be associated with lower satisfaction scores (Table 3). Studies indicating that uninsured/self-pay patients confer a negative scoring bias (n=11,600) outnumber those indicating a positive bias (n=5616). Studies indicating that worker’s compensation patients confer a negative scoring bias (n=3563) outnumber those indicating a positive bias (n=748) as well. Although the results are mixed, the data suggest that patients classified as uninsured/self-pay and/or worker’s compensation give lower satisfaction scores.

Discussion

Patient satisfaction is increasingly used as a quality metric for healthcare institutions as well as individual physicians. MACRA bolsters the practice of using patient satisfaction scores to determine physician reimbursement via the value-based payment model. This literature review and meta-analysis evaluates the implications of this practice since MACRA was passed by compiling outpatient satisfaction and insurance data from 2015 through today.

If it is true that these payer classes inherently give lower patient satisfaction scores, physicians serving these populations will be financially penalized.

Our results support previous findings that nonmodifiable patient factors affect patient satisfaction scores. Specifically, there is evidence of insurance status as a determinant of patient satisfaction. Although data from outpatient clinic settings is limited, some common trends are found across most studies. Medicare patients were associated with higher satisfaction scores, whereas uninsured/self-pay and worker’s compensation patients were associated with lower satisfaction scores.

These trends corroborate concerns of lower patient satisfaction among certain domains of the underserved population. Uninsured/self-pay and worker’s compensation patients tend to be underserved, because these groups are largely composed of low-income, minority, and blue-collar patients.(23,24) If it is true that these payer classes inherently give lower patient satisfaction scores, physicians serving these populations will be financially penalized. This provides a disincentive for physicians and healthcare institutions to care for underserved patients, which may worsen health disparities. In contrast, physicians with a predominantly Medicare payer mix may receive higher satisfaction scores and increased reimbursement.

Although this literature review includes only outpatient studies, previous studies in hospital settings have found similar results. For example, large academic or public hospitals that provide care for a higher proportion of underserved patients also have been found to receive lower patient satisfaction scores.(5) Using the same patient satisfaction standards to compare these institutions with smaller community hospitals may lead to unfair financial penalties based on payer class. This may strip resources from already underfunded hospitals and contribute to worsening health disparities.

Due to these potential biases, the validity of using patient satisfaction as a healthcare value measure should be further analyzed and reexamined. Several studies have already shown that high patient satisfaction ratings do not correlate with improved health outcomes.(25,26) Potential solutions to account for inherent biases caused by nonmodifiable patient factors should be considered as well. One possible solution is improved case-mix adjustment, which adjusts satisfaction scores based on patient demographic factors to compare different healthcare institutions fairly.(27) However, case-mix adjustment based on nonmodifiable patient factors remains controversial, because response tendencies of certain demographic groups is not universal, and adjustments may disguise disparate care delivery based on the same patient factors.(28,29)

Although our literature review highlights trends such as the association between Medicare status and higher patient satisfaction scores in outpatient settings, there is no clarity regarding causation. It is unclear whether the treatment Medicare patients receive is different in any way, whether satisfaction can be attributed to the age demographic (previous studies have shown older patients to be more satisfied), or whether other demographic factors impact Medicare patients’ satisfaction. Similarly, there is evidence of a relationship between worker’s compensation and uninsured/self-pay patients and low satisfaction. However, the exact reasons for these phenomena are unknown. More research into this topic is necessary to determine the exact relationship between these variables.

In conclusion, these data are important for policymakers, physicians, healthcare institutions, and patients. Policymakers should be aware of any possible inequities inherent in the use of patient satisfaction in value-based payment models and make appropriate adjustments. Physicians and healthcare institutions also should be cognizant of how their payer mix may be impacting their reimbursement. Further research on nonmodifiable determinants of patient satisfaction and other value-based payment scores is necessary to ensure an equitable reimbursement and evaluation structure.

References

1. Library of Congress. Medicare Access CHIP Reauthorization Act of 2015. www.congress.gov/bill/114th-congress/house-bill/2 . Accessed November 28, 2016.

2. Centers for Medicare and Medicaid Services. Quality Payment Program. https://qpp.cms.gov . Accessed November 28, 2016.

3. Barber EL, Bensen JT, Snavely AC, et al. Who presents satisfied? Non-modifiable factors associated with patient satisfaction among gynecologic oncology clinic patients. Gynecol Oncol. 2016;142:299-303. doi:10.1016/j.ygyno.2016.06.009.

4. Abtahi AM, Presson AP, Zhang C, et al. Association between orthopaedic outpatient satisfaction and non-modifiable patient factors J Bone Joint Surg. 2015;97:1041-1048. doi:10.2106/JBJS.N.00950.

5. McFarland DC, Ornstein KA, Holcombe RF. Demographic factors and hospital size predict patient satisfaction variance—implications for hospital value-based purchasing. J Hosp Med. 2015;10:503-509. doi:10.1002/jhm.2371.

6. Post DM, McAlearney AS, Young GS, et al. Effects of patient navigation on patient satisfaction outcomes. J Canc Educ. 2015;30:728-735. doi:10.1007/s13187-014-0772-1.

7. Compton J, Glass N, Fowler T. The effect of workers’ compensation status on the patient experience. JB JS Open Access. 2019;4(2). e0003. https://doi.org/10.2106/JBJS.OA.19.00003

8. Chang BH, Hoaglin DC. Meta-analysis of odds ratios: current good practices. Med Care. 2017;55:328-335. doi:10.1097/MLR.0000000000000696.

9. Harrer M, Cuijpers P, Furukawa T, et al. dmetar: Companion R Package For The Guide “Doing Meta-Analysis in R”. R package version 0.0.9000. 2019; URL http://dmetar.protectlab.org .

10. Chinn S. A simple method for converting an odds ratio to effect size for use in meta-analysis. Stat Med. 2000;19:3127-3131. doi:10.1002/1097-0258(20001130)19:22<3127::aid-sim784>3.0.co;2-m.

11. Bible JE, Kay HF, Shau DN, et al. What patient characteristics could potentially affect patient satisfaction scores during spine clinic? Spine. 2015;40:1039-1044. doi:10.1097/BRS.0000000000000912.

12. Nieman CL, Benke JR, Boss EF. Does race/ethnicity or socioeconomic status influence patient satisfaction in pediatric surgical care? Otolaryngol Head Neck Surg. 2015;153:620-628. doi:10.1177/0194599815590592.

13. Martin L, Presson AP, Zhang C, et al. Association between surgical patient satisfaction and nonmodifiable factors. J Surg Res. 2017;214:247-253. doi:10.1016/j.jss.2017.03.029.

14. Buie J, de Riese W, Sharma P. Inherent biases in patient-reported satisfaction with care in the outpatient setting. J Med Pract Manag. 2018;34:108-119.

15. Rogo-Gupta LJ, Haunschild C, Altamirano J, et al. Physician gender is associated with Press Ganey patient satisfaction scores in outpatient gynecology. Women’s Health Issues. 2018;28:281-285. doi:10.1016/j.whi.2018.01.001.

16. Johnson BC, Vasquez-Montes D, Steinmetz L, et al. Association between nonmodifiable demographic factors and patient satisfaction scores in spine surgery clinics. Orthopedics. 2019;42(3):143-148. doi:10.3928/01477447-20190424-05.

17. Rane A, Tyser A, Presson A, et al. Patient satisfaction in the hand surgery clinic: an analysis of factors that impact the Press Ganey Survey. J Hand Surg Am. 2019;44:539-547.e1. doi: 10.1016/j.jhsa.2019.03.015.

18. Rane A, Tyser A, Kazmers N. Evaluating the impact of wait time on orthopaedic outpatient satisfaction using the Press Ganey Survey. JBJS Open Access. 2019;4:e0014. doi:10.2106/JBJS.OA.19.00014.

19. Tisano BK, Nakonezny PA, Gross BS, et al. Depression and non-modifiable patient factors associated with patient satisfaction in an academic orthopaedic outpatient clinic: is it more than a provider issue? Clin Orthop Relat Res. 2019;477:2653-2661. doi:10.1097/CORR.0000000000000927.

20. Wells J, Batty M, Box H, Nakonezny PA. Effect of patient body mass index, recommendation for weight modification, and nonmodifiable factors on patient satisfaction. J Am Acad Orthop Surg. Published online August 21, 2019. doi:10.5435/JAAOS-D-19-00330.

21. Mordhorst TR, McCormick ZL, Presson AP, et al. Examining the relationship between epidural steroid injections and patient satisfaction. Spine J. 2020;20:207-212. doi:10.1016/j.spinee.2019.09.024.

22. Davis-Dao CA, Ehwerhemuepha L, Chamberlin JD, et al. Keys to improving patient satisfaction in the pediatric urology clinic: a starting point. J Pediatr Urol. Published online March 29, 2020. doi:10.1016/j.jpurol.2020.03.013.

23. Morelli V, Zoorob R, Heidelbaugh JJ. Primary Care of the Medically Underserved, An Issue of Physician Assistant Clinics. [e-book]. Elsevier Health Sciences; 2018.

24. Facts + Statistics: Workplace Safety/Workers Comp | III. Accessed June 30, 2020. www.iii.org/fact-statistic/facts-statistics-workplace-safety-workers-comp .

25. Lobo Prabhu K, Cleghorn MC, Elnahas A, et al. Is quality important to our patients? The relationship between surgical outcomes and patient satisfaction. BMJ Qual Saf. 2018;27(1):48-52. doi:10.1136/bmjqs-2017-007071.

26. Xiang X, Xu WY, Foraker RE. Is higher patient satisfaction associated with better stroke outcomes?. Am J Manag Care. 2017;23(10):e316-e322.

27. O’Malley AJ, Zaslavsky AM, Elliott MN, et al. Case-mix adjustment of the CAHPS Hospital Survey. Health Serv Res. 2005;40(6 Pt 2):2162-2181.doi:10.1111/j.1475-6773.2005.00470.x.

28. de Boer D, van der Hoek L, Rademakers J, et al. Do effects of common case-mix adjusters on patient experiences vary across patient groups? BMC Health Serv Res. 2017;17(1):768. doi:10.1186/s12913-017-2732-z.

29. Howell EA, Egorova NN, Balbierz A, et al. Site of delivery contribution to black-white severe maternal morbidity disparity. Am J Obstet Gynecol. 2016;215:143-152. doi: 10.1016/j.ajog.2016.05.007.