Evaluating Performance of Payer Program Offerings Using a Data Science-Driven Approach

Healthcare payers are navigating through an evolving market landscape. Macroeconomic forces including the conclusion of the COVID-19 public health emergency, rising healthcare costs attributed to inflation, and the ongoing trend in provider consolidation, all contribute to a market of uncertainty. This uncertainty has also manifested across government programs as the expiration of the expanded enrollment rules for Medicaid and Affordable Care Act (ACA) coverage is likely to require consumer reenrollment, prompting a potential shift in coverage to a different payer. The decision to retain or switch coverage to a competing payer is highly predicated on an individual or group’s overall member experience and satisfaction with existing coverage, reinforcing the need for healthcare organizations to design and offer an enriched consumer experience across all of their programs.

Furthermore, the Centers for Medicare & Medicaid Services (CMS) projected a 5.4% average annual growth in National Health Expenditure (NHE) over the 2022-2031 period. This coincides with the National Association of Insurance Commissioners (NAIC) report indicating that profit margin for the health insurance industry declined by almost 38% to 3.3% in 2023 as compared to 5.3% in 2020. With rising healthcare costs and shrinking profit margins, customer retention becomes paramount. To navigate this challenge, payers must assess the value generated by each of their various programmatic offerings, prioritizing those that enhance the consumer experience while remaining cost-effective.

Payers benefit from evaluating a broad range of healthcare programs, including ancillary products, supplemental benefits and care management programs, among others, as highlighted in Figure 1. These insights can help inform critical decisions to optimize value creation (e.g., budget allocation, benefit design, care management program offerings, consumer retention strategies, future potential investments).

Today, many payers use outdated methods, relying only on cost measures or comparison of absolute values to assess their programs’ value, which may lead to misleading results. In evaluating program value, payers need to ensure that the measured value from a program is uniquely attributed to the program design and not the underlying attributes of membership, employer groups, providers, or other factors independent of program design. Data science methods rooted in statistical analysis help isolate and quantify the value contributed by different program offerings while controlling for factors that may influence program assessment results.

Figure 1. Sampling of Healthcare Program Offerings

Evaluating Healthcare Program Value

To assess the quantitative value of program offerings, payers can study the impact of program participation on various metrics, including health outcomes, cost outcomes, utilization rates, member retention, Net Promoter Score (NPS) and Care Management (CM) engagement rates. Statistical matching methods can be an effective, versatile approach to reduce the bias from other influencing factors and ultimately, drive decisions to modify, expand, reduce, or terminate program offerings. While widely applicable, this approach also enables payers to address a wide variety of questions concerning healthcare program effectiveness, as outlined in Figure 2.

Figure 2. Examples of Healthcare Program Performance

The ability to effectively evaluate the value generated by various benefit programs can further aid payers in optimizing budget allocation, future investments, benefit design, vendor strategy, member experience, and sales strategy.

Challenges to Performance Evaluation

Evaluating the value delivered from a program offering requires isolating the impact on relevant outcome measures. This ensures any observed improvement can be definitively attributed to the program itself. There are several factors in a healthcare program offering that may influence the outcome, such as product structure (i.e., HMO vs. PPO), which can influence member behavior and patterns for seeking care. Similarly, a higher proportion of older and high-risk members could inflate overall costs in a specific cohort. Bundled program benefits make it difficult to isolate the impact and evaluate the value of each individual benefit.

Additionally, program performance evaluation involves analyzing datasets which provide information about member enrollment, medical claims, pharmacy claims, customer satisfaction survey results, marketing campaigns and sales data. These datasets, often extracted from disparate sources, add significant complexity to program evaluation.

Key Components of a Data Science-Driven Approach

The following outlines the key components of a robust data science-driven approach that leverages industry expertise to measure program value. The approach applies statistical matching methods to evaluate the effectiveness of healthcare programs while controlling for confounding variables that may unintentionally impact the outcome measures of interest.

Value Proposition Analytics (VPA) follows a five-step process (see Figure 3) and is described in the subsequent sections.

Figure 3. Value Proposition Analytics Overview

Step 1: Data Wrangling

The data wrangling process validates and prepares incoming data for subsequent analysis. Healthcare data such as eligibility files, medical claims, pharmacy claims, member survey results, and provider information is identified, ingested, validated, structured, explored, and transformed to create refined datasets (see Figure 4).

Figure 4. Data Wrangling

Step 2: Segmentation

The output from data wrangling is segmented into two member cohorts based on their participation in the program under evaluation: participants (i.e., the test group) and non-participants (i.e., the control group).

Figure 5. Segmentation into “Test” vs. “Control” Groups

Step 3: Creation of Control Groups

Each of the confounding variables identified in this analysis is binned into discrete categories. Subsequently, each conceivable combination of these categories creates a “bucket” of members who exhibit the same characteristics. We leverage a statistical matching method that utilizes random sampling to create a control group sample such that the proportion of its members in each of these buckets is the same as those in the corresponding buckets of the test group. In the example illustrated in Figure 6, Bucket A is comprised of 2.1% of members in the test group, who are in the age range of 50 to 60 years, reside in the east region, and have opted for an HMO health coverage plan. The nonparticipating members in the control group are then sampled in such a way that 2.1% of the resulting control group sample have the same characteristics as defined by Bucket A.

Figure 6. Creation of a Control Group Sample

Step 4: Measure Calculations

The similarity of member attributes across the test group and control group sample then allows for any significant differences in outcomes between them to be more directly attributable to program participation. This process is repeated multiple times to create a set of control group samples consisting of non-participating members who share the same characteristics as the participating members in the test group. Outcome measures relevant for a given program evaluation are then calculated for each of these samples to generate as many outcome values per measure as the number of control group samples in the set (see Figure 7). In this example, the outcome measure is Inpatient (IP) Utilization which is calculated for each of the 30 control group samples to generate 30 values of IP Utilization.

Figure 7. Control group samples and outcome measures

Step 5: Significance Testing

The values calculated for each of the 30 control group samples are used to estimate the range in which a member not participating in the program is expected to fall in for a given outcome measure of interest. If the outcome measure for the participating member is within a similar range then the program participation is estimated to not have any significant impact on the outcome.

However, the program participation is expected to have a significant impact when the outcome measure calculated for the participating members in the test group is outside the range calculated using the control group samples. Based on the relevance of any differences observed in the outcome measures between the participating and non-participating members, payers can assess the value delivered by the program under evaluation. Collaborating with industry experts in identifying potential confounding variables and outcome measures that quantify the value delivered by a healthcare program further increases the applicability of such a method in driving business decisions.

A Holistic View for Performance Evaluation

While cost measures have traditionally been central to healthcare program evaluation and subsequent decision-making, a value-centered approach (e.g., VPA) offers a broader, more holistic view. It goes beyond medical cost reduction and considers the program’s full value with factors including service utilization, member retention, member satisfaction, new member growth, length of stay, and member engagement.

VPA is a robust and powerful tool that helps assess the impact of healthcare programs. It analyzes and synthesizes data from both participating and non-participating members to determine whether a program is achieving its intended objectives. This approach informs and drives strategic planning and budget allocation decisions on healthcare program offerings, including programs to grow, refine or discontinue. VPA’s versatility is broad and can be applied to a wide range of business applications to evaluate performance across clinical care programs, vendor solutions, value-based care programs, and ancillary and supplemental benefits.

HealthScape Can Help

HealthScape’s proprietary approach goes beyond basic data analysis. Our solution combines advanced analytics with extensive industry expertise to unlock hidden potential within health plans’ programs.