Background

Intermittent shortages of chemotherapeutics used to treat curable malignancies are a worldwide problem. Over the past decade, developed and developing countries have reported numerous shortages, as have those with and without nationalized health systems.^1–4 A 2019 survey found that 2 in 3 US hospitals experienced sporadic chemotherapy scarcity, and current tracking data notes 19 active shortages.^4,5 Strategies to improve consistent chemotherapy access vary by country and include assessing the burden of disease, expanding healthcare coverage and pharmaceutical production, increasing supply chain redundancy and transparency, and incentivizing manufacturers.^3,6–8 Related research has involved forecasting chemotherapeutic need,¹ retrospectively assessing shortage-related morbidity and mortality,^9–14 evaluating stakeholder experiences,^15–17 and proposing allocation guidelines.^18–20 Published guidelines consider prioritizing the following patient groups as potential shortage management strategies: the youngest, those whose disease is most responsive to the scarce medication, and those who would do worst without it.^19–23

Although allocation guidelines have been published by some oncology societies,¹⁹ their application to shortage management is limited because multiple competing strategies are considered acceptable.^19,22 Moreover, prospective study is limited by the unpredictability and heterogeneity of shortages. Because of the complexity of supply chains and supplier contracts, a single shortage may severely impact some hospitals but leave others with normal supply. In addition, hospitals with more resources may have pharmacists dedicated to tracking and preemptive stockpiling. Allocation during these situations therefore remains uncoordinated and variable.^4,11,24 There are few data to help decision-makers understand how different ethically acceptable strategies impact survival, when allocation may meaningfully improve outcomes, and which strategies produce improvements over the standard practice of first-come, first-served.

To address these critical clinical dilemmas, we constructed a hospital-level microsimulation model to evaluate the effects of allocation during intermittent chemotherapy shortages and applied it to the recent shortage of vincristine in the United States.²⁵ This model follows others that have successfully addressed intermittent ICU bed and organ scarcity. Collectively, such models allow decision-makers to evaluate allocation schemes before patients are subjected to them, reducing the risk of unintended consequences, bias, and treatment disparities.^26–30 Using retrospective patient and guideline-based inputs, validated against national outcomes data, we aimed to assess the impact of different strategies across varying levels of vincristine supply, thereby creating a flexible structure that can be adapted to future shortages as they arise.

Patients and Methods

We developed a simulation model to evaluate the effect of different prioritization methods on overall survival during chemotherapy shortages. The model used agent-based simulation characteristics, where autonomous agents make individual decisions based on a set of rules,³¹ and discrete-event characteristics, where agents pass through a process in a stepwise fashion and wait in queues based on resource availability.³² We evaluated the model by imposing vincristine shortages on a simulated cohort of patients instantiated using demographic, disease, and treatment data from all patients treated using vincristine at our institution from 2015 through 2019, and survival data with risk adjustments from applicable trials of vincristine-based regimens and available alternatives. The model was constructed in AnyLogic university software, version 8.7.0 (The AnyLogic Company), and statistical analyses were performed using STATA/SE, version 15.1 (StataCorp LLC). Further details for each methods section can be found in supplemental eAppendix 1, available with this article at JNCCN.org.

Model Structure

A schematic of the discrete-event portion of the model is shown in Figure 1. The model is prefaced on a utilitarian calculus (ie, maximizing a selected outcome), because utilitarianism is an accepted normative framework favored by stakeholder groups and is commonly used during healthcare scarcity.^19,22,26,33 Patients requiring scarce treatment entered the model at the time of treatment decision (diagnosis, relapse, surgery, treatment choice) and entered a queue, where an initial allocation score was calculated:

$\mathit{\text{allocation}}\hspace{0.17em}\mathit{\text{score}}=\mathit{\text{w}}\left(\mathit{\text{waitlist}}\hspace{0.17em}\mathit{\text{time}}\right)-\mathit{\text{x}}\left(\mathit{\text{age}}\right)+\mathit{\text{y}}\left(\mathit{\text{efficacy}}\right)+\mathit{\text{z}}(\mathit{\text{alternatives}}),$

where waitlist time was the time the patient had been in the queue, age was the patient’s age when entering the model, efficacy was the survival probability associated with administering the scarce drug (survival per mg of vincristine), and alternatives was the absolute survival probability difference between the scarce-drug regimen and the best alternative regimen. Score terms were selected because of their operational equivalence to accepted ethical allocation principles^15–17,19 and were normalized against each other. The w, x, y, and z coefficients could vary between 0 and 1, which allowed each term to be included, excluded, and independently weighted. This formula was chosen for its ease of use and flexibility and to avoid implicit preference toward any strategy.

View Full Size

Schematic of the discrete-event portion of the model.

Citation: Journal of the National Comprehensive Cancer Network 20, 4; 10.6004/jnccn.2021.7047

As new patients entered and left the queue, scores were recalculated and patients were reordered. Queued patients could then (1) receive their scarce drug treatment regimen if supply was available when they reached the head of the queue; (2) receive their best alternative regimen if a predefined, cancer-specific, maximal wait time expired before they reached the head of the queue; or (3) die while awaiting treatment. Patients alive when leaving the queue received their assigned treatment with survival probabilities based on their regimen’s trial-based outcome and applicable risk adjustment.

Model Inputs

A description of model inputs and references can be found in Table 1. Demographic, disease, treatment, and treatment timing data for 1,689 patients treated with vincristine were abstracted from an institutional database and constituted the cohort of simulants in the model. Cohort demographics are described in supplemental eTable 1. Survival probabilities, risk adjustments, and vincristine doses for the regimens used were abstracted from the literature. Trials cited in the NCCN Clinical Practice Guidelines in Oncology (NCCN Guidelines) for each disease and subtype were used (NCCN.org/guidelines/category_1). NCCN Guidelines were also used to abstract nonvincristine alternative regimens with the greatest survival probabilities applicable for each disease, as well as disease risk adjustments that were not otherwise accounted for by the choice of regimen. Specific trials and risk adjustments can be found in supplemental eTables 2 and 3.

Table 1.

Model Inputs and References

View Table

Results

Scenario Evaluation

Allocation by the efficacy score term alone, but not the alternative or age terms alone, resulted in significant differences in 3-year survival compared with standard practice (waitlist time term; Figure 2A) over a wide range of possible shortages (23.3% and 91.1% of adequate vincristine supply; P<.01 at each interval supply level). Improvements were clinically meaningful (>5% absolute mean survival improvement) when the supply was between 43.0% and 77.9% of the level needed to treat all patients (Figure 2A). The reduction in the risk of death at 3 years for allocation by the efficacy term compared with the waitlist time term is shown in Table 2; the number needed to treat across all possible supply levels was 40.3 (99% CI, 38.9–41.9).

View Full Size

Probability of 3-year survival and percentage change in survival. (A) Probability of 3-year survival by score terms. Highlighted sections indicate significant differences compared to waitlist time. Colored blocks indicate the range over which a strategy maintains survival similar to models with full supply. Minimum supply thresholds are listed in each block. (B) Percentage change in survival normalized against survival by the waitlist time-based allocation.

Citation: Journal of the National Comprehensive Cancer Network 20, 4; 10.6004/jnccn.2021.7047

Table 2.

Risk Reductions in Death at 3 Years

View Table

Sensitivity testing showed potential survival improvements for both the efficacy and alternative terms compared with the waitlist time term (supplemental eFigures 11 and 12), and therefore a score that included both terms was assessed (see “Scenario Evaluation” in supplemental eAppendix 1). Results of this efficacy-alternatives score showed synergistic improvements in 3-year survival; a score with equal coefficients (allocation score = 1[efficacy] + 1[alternatives]) produced the greatest improvements (Figure 2A). Significant differences between this efficacy-alternatives score and the waitlist time score occurred when the drug supply was between 6.7% and 93.2% of the amount needed to treat all patients (P<.01 at each interval supply level); it maintained clinically meaningful survival improvements over 56.7% of possible supply levels (threshold values: 24.4% and 81.1%; Figure 2B). The reduction in the risk of death at 3 years for allocation by the efficacy-alternatives score compared with the waitlist time score is shown in Table 2; the number needed to treat across all possible supply levels was 31.4 (99% CI, 30.6–32.4). No other score term combinations showed consistent survival improvements over the waitlist time term (supplemental eFigure 13).

The minimum supply level at which each strategy maintained survival rates similar to survival without shortages are represented by the blocks in Figure 2A. The efficacy-alternatives score maintained survival at rates similar to those of models with shortages until the supply went below 72.2% of the amount needed to treat all patients. Scores based on the efficacy, alternatives, age, and waitlist time terms alone maintained similar rates of survival only until 81.1%, 88.8%, 88.9%, and 94.3% of the needed supply was available, respectively.

The mean age of the surviving patients and mean time in the queue for all patients were then modeled (supplemental eFigures 14 and 15). When compared with an age-based score, the increase in the mean age of patients surviving according to the efficacy-alternatives score was <2.5 years at all supply levels. The efficacy-alternatives score led to mean times in the queue that were lower than the waitlist time–based score for supply levels between 47.9% and 100% (all P<.01). Across all supply ranges, the reduction in time in the queue for this score was 1.1 days per patient.

Discussion

We successfully constructed a model for assessing the impact of allocation strategies on patient survival during intermittent chemotherapy shortages. In a vincristine shortage scenario, we found that the use of certain allocation strategies improved survival rates during clinically relevant supply reductions compared with standard practice. A strategy that prioritized patients according to higher drug efficacy per volume and greater alternative treatment efficacy differences produced the greatest survival improvement (relative risk reduction = 7.5%) and maintained statistical significance over a wide range of possible shortages. Compared to the standard practice of first-come first-served, this strategy also maintained survival rates when shortages were approximately 5 times more severe (72.2% of adequate supply vs 94.3% for standard practice). This model provides a flexible structure by which responses to intermittent shortages can be evaluated as they arise. These data suggest that significant differences in outcomes can be seen among acceptable strategies and that allocation itself can ameliorate the impact of some shortages.

Our calibration, validation, and sensitivity analyses confirm that the model performed logically and that its outputs faithfully reflected real-world experience. For example, rates that impact the size of the queue can easily be calibrated to mimic actual institutional arrival rates. In addition, survival is dependent on queue length, even when controlling for supply and bounding rates by those seen in the cohort. Furthermore, relative allocation score sensitivities are appropriately minimized in edge cases, where allocation choices decrease near the upper and lower limits of supply. Outcomes with adequate supply also matched those seen in SEER data, indicating that modeled regimen probabilities and risk adjustments are comparable to real-world outcomes. These procedures are easily adaptable within the model, and external data sources are either publicly available or locally accessible from most medical records systems such that validation can be replicated for new scenarios. Because the overall goal of this model is for it to be used by the oncology community and to better evaluate allocation policy, research is ongoing to identify best practices for model distribution that encourages widespread use by decision-makers without access to programming expertise.

Because of the heterogeneous distribution of medications and the unpredictability of intermittent shortages in many countries, prior studies have been generally limited to retrospective assessments through qualitative interviews, surveys, and database abstractions.^{9–13,15–17} Impressive policy and operational work by the FDA and the American Society of Health-System Pharmacists has greatly improved the supply infrastructure and supply-side reporting of shortages^5,7; unfortunately, the transparency of hospital data and allocation practices has not matched these efforts and is limited by proprietary contracts and the taboo of institutions publicly disclosing shortages. As such, ongoing shortages, such as those of Erwinia asparaginase, are managed in varying ways using different allocation strategies (if any formal strategies are used at all).^4,11,24 Tools to evaluate ongoing or upcoming shortages have been lacking. A recent FDA report outlined potential market solutions that will require large-scale industry changes and legislative action,⁷ suggesting that shortages are likely to continue for the foreseeable future. As antineoplastic shortages continue, we are hopeful that our modeling approach can be used to help hospitals maximize patient outcomes by allowing them to estimate demand and the potential impact of different ethical allocation strategies before rationing decisions are made.

The vincristine scenario shows that survival rates over a wide range of shortages can be impacted by allocation alone. Although guidelines promote several acceptable allocation strategies, this model can provide data for each strategy and combination. In this scenario, an efficacy- and alternatives-based combination strategy improved the numbers of patients surviving and only minimally impacted other relevant outcomes. Moreover, this strategy revealed the potential to ameliorate a shortage’s impact over a large and clinically relevant range of supply reductions. It maintained survival rates until supply fell below 72.2% of the amount needed to treat all patients, below the 80% level some hospitals have experienced during some prior shortages.^5,10 This threshold may be of particular interest to suppliers, who could target this supply level for hospitals within their network until the scarcity is resolved without encouraging inappropriate stockpiling by individual hospitals. Simultaneously, our scenario showed significant survival improvements above and below that threshold (6.7%–93.2%). Because the severity of future shortages is unknown, the development of strategies that can maintain outcomes over the largest range possible is prudent.

Our model must be interpreted in the context of its limitations. First, the modeled scenario was institutional and did not allow for patient transfer when supply was short, which has been described in a small number of shortages.⁴ Although institution-level allocation is the norm in the United States, others may prefer a regional or national approach; the model structure is flexible and can accommodate any approach where the cohort is subject to a single supply. Second, once a patient was assigned treatment, that chemotherapy supply was dedicated to them, and all patients completed their assigned treatments. Although stopping and redirecting therapy midcourse is possible, a dedicated supply model was chosen because of the ethical tenet of duty to care and because the evidence base for treatment outcomes is dependent on completion of the regimen studied. Third, available data did not allow us to account for supply redistributions resulting from toxicity; future development will include estimating treatment regimen use and reductions through retrospective pharmacy data and soliciting subspecialist input on estimated rates of reduction. Fourth, because not all treatments had nonvincristine alternatives, an efficacy reduction compared with the recommended regimen was preset by assessing the mean difference of the alternatives and outcomes during similar shortages; true reductions may vary. Fifth, the model was not tested with the pediatric cohort removed, and results should not be extrapolated to centers without a pediatric population. Sixth, modeling necessarily requires some input data assumptions and produces precise results. This is both a strength, because it requires decision-makers to acknowledge and state assumptions explicitly, and a limitation, because models produce results that may create a false sense of accuracy and precision for those unfamiliar with their methods. Finally, there are necessary aspects of the input data that are specific to our cohort, and outcomes with particular strategies will vary with the drug and population assessed. However, because the purpose of the model is for future use, this limitation is not of the model itself but is inherent to the reporting of any scenario tested.

Conclusions

We developed a model capable of simulating scarce chemotherapy allocation according to different acceptable strategies. Results show the significant impact that allocation can have on survival. Although the locations from which intermittent shortages arise are diverse, modeling provides flexible means by which the outcomes associated with ethical allocation choices can be assessed transparently and objectively. It also providers decision-makers a mechanism by which to pretest allocation schemes before their implementation rather than subjecting patients to untested measures with potentially unintended consequences. The acceptability or unacceptability of any strategy and outcome is individual to the population affected by the shortage. The only universally unacceptable practice is to continue forcing clinicians to face shortages of lifesaving chemotherapeutics without objective data. This model offers useful means by which options can be explored and best practices identified before patients are put at risk.