In 2006, the WHO stated that even when health systems are well-developed and appropriately resourced, quality remains a serious concern.1 Therefore, it is important to gain insight into the care that is provided in order to monitor and effectively improve high-quality care.2,3 Quality of care can be monitored and measured by using a dashboard of valid and reliable quality indicators (QIs).4,5 A QI is defined as “a measurable element of practice performance for which there is evidence or consensus that it can be used to assess the quality, and hence change in the quality, of care provided.”6 Many QIs have already been developed for oncologic diseases7,8 and nononcologic diseases.9
Head and neck cancer (HNC) is the sixth most common cause of cancer worldwide.10 HNC and its treatment have a significant impact on a patient's well-being because HNCs grow relatively quickly in an anatomically and functionally complex location.11–14 Patients often experience problems with speech, swallowing, and physical disfiguration due to surgery, systemic therapy, radiation, or a combination of these treatments.15–19 For an optimal result, it is crucial that medical specialists and allied health professionals (AHPs) deliver integrated care throughout the care process.
In 2014, a dashboard in the form of a quality registration, with census- and evidence-based QIs, was developed for measuring the quality of integrated care for patients with HNC, (eg, the Dutch Head and Neck Audit [DHNA]).20 The start-up of a quality registration takes several years before stable data can be obtained.21 In addition to stable data, insight into determinants that influence variation in care at the patient and hospital levels can provide tools for explaining the QI scores and improving current care through medical staff learning from each other.22
This is the first article with results from the DHNA—a recently launched quality registration that collects data prospectively. We aimed to obtain insight into the quality of currently delivered HNC care in the Netherlands and some of the influencing characteristics to test the value of the QI set as an instrument for monitoring and improving clinical performance for this quality registration.23
Methods
Study Design
This is a prospective, observational, multicenter study for measuring current quality of integrated HNC care with a recently launched quality registration, including 39 census- and evidence-based QIs.20 We determined hospital variation in QI scores and explored this variation to explore whether differences at the patient and hospital levels would explain these variations. The Medical Ethical Committee of the region Arnhem–Nijmegen declared that ethical approval was not necessary.
Setting
In the Netherlands, approximately 3,000 patients are diagnosed with HNC annually. HNC care is centralized to 14 hospitals: 8 head and neck oncology centers (HNOCs) and 6 affiliated centers.24 The number of patients treated annually varies between 70 and 600 per center. Medical specialists and AHPs involved in HNC care are united in 2 national foundations: one for medical specialists and another for AHPs; there is also a Dutch HNC patient association.
Study Population
All patients with newly diagnosed HNC from December 2013 to January 2017 who were treated with curative intention in 1 of the 14 hospitals were eligible for registration in the DHNA.
QI Set
All 39 QIs of the DHNA had been systematically developed and based on national and international evidence-based guidelines. The Rand modified Delphi method was used for this purpose by the 2 national foundations for medical specialists and AHPs, and by the patient association.20 This resulted in 5 outcome indicators (survival, recurrence, complication rate, and patient-reported outcomes and experiences), 13 process indicators from the perspective of medical specialists (performance in the diagnostic, treatment, and follow-up phases, as well as aspects regarding coordination and communication), 18 process indicators from the perspective of AHPs (malnutrition screening and involvement of speech therapist), and 3 structure indicators from the perspective of AHPs (number of available case managers and nurses for each hospital). In addition, 10 determinants at patient level and 1 at hospital level were selected (see following section).
Data Collection
The 14 hospitals were encouraged to have all patients treated for HNC self-register online in the DHNA. Health professionals working in the participating hospitals used this registration system to collect the necessary data for calculating QI scores and assessing determinants at the patient and hospital levels. Determinants were subdivided into the following categories: age (continuous), sex, smoking (never, former, or current smoker), alcohol consumption (never, former, or current drinker), WHO performance status (PS; 0 vs 1–4), tumor site (oral cavity and oropharynx; larynx and hypopharynx; or other); clinical TNM stage (early [stage I–II] vs advanced [stage III–IV]); and comorbidity (yes vs no) at patient level, and the volume of HNOCs and affiliated centers (<200 patients vs ≥200 patients) at hospital level. Volume was defined as the average curatively treated patients per year in the HNOCs and affiliated centers25,26 as registered by the Netherlands Comprehensive Cancer Organisation (IKNL).27
Sample Size
Previous research shows that the precision of a QI score depends on the number of patients included (numerator).28 Therefore, only QIs with a numerator of >400 patients were included for calculating a precise indicator score with a confidence limit of ±5 percentage points.
Data Analyses
Inclusion criteria for the analysis were: (1) patients who underwent curative treatment, (2) availability of sufficient data (at least patient age and date of first treatment initiation), and (3) at least 30 patients per hospital (Figure 1).29 The last criterion assured that results were not skewed unfairly because of too few data points.
Descriptive Analyses: The following were assessed for each indicator:
Denominator: total number of patients who should have received the care as recommended in the QI
Indicator score: quotient of the number of patients who received care as recommended in the QI compared with the number of patients who needed to receive the care as recommended
Missing data: percentage of missing values per indicator. Missing data for indicators may bias the results. A percentage >25% refers to poor data quality30
Inclusion of patients in this study.
aAt least the age and date of the start of the first treatment are known.
Citation: Journal of the National Comprehensive Cancer Network J Natl Compr Canc Netw 16, 12; 10.6004/jnccn.2018.7061
Inclusion of patients in this study.
aAt least the age and date of the start of the first treatment are known.
Citation: Journal of the National Comprehensive Cancer Network J Natl Compr Canc Netw 16, 12; 10.6004/jnccn.2018.7061
Inclusion of patients in this study.
aAt least the age and date of the start of the first treatment are known.
Citation: Journal of the National Comprehensive Cancer Network J Natl Compr Canc Netw 16, 12; 10.6004/jnccn.2018.7061
Determinant Analyses: To assess determinants at the patient and hospital levels, we first studied the single relationship between indicators (eg, percentage of patients discussed in the multidisciplinary team meeting [MDT] before start of treatment) and determinants that could influence the indicator score (eg, tumor site) using a generalized linear mixed model. This model accounts for the nested structure of data, because individual patients (patient level = 1) are nested within hospitals (hospital level = 2). Determinants that had a univariate relationship of P<.20 were assumed to have a potential association with the indicator score and were selected as possible determinants for the multivariate model.
We tested all of the selected determinants together in a multilevel logistic regression model or multivariate model. Patients were automatically excluded (list-wise deletion) from the analysis if indicator or determinant data were missing. Missing was defined as the state in which data were randomly missing, and was tested by comparing the population included for the analysis with the original population of the study. We needed a minimum of 10 patients for each degree of freedom in the model to develop a reliable multilevel model. Odds ratios with 95% CIs described the association between indicator scores and determinants.
Indicator scores were recalculated for each indicator and for each part of the associated determinant. For example, an indicator could be recalculated for all patients with an early tumor stage and for all patients with an advanced tumor stage. If multiple determinants were associated with a single indicator, for example tumor stage and PS, we combined the different determinants and calculated stratified scores for the different categories.
Results
Study Population
In total, 1,667 patients satisfied the inclusion criteria for this analysis. Patients originated from 5 HNOCs and 3 affiliated centers. Median patient age was 65 years (range, 64–68 years) and most were male (68.5%; range, 60%–72%) (Table 1). Missing data for most patient characteristics varied between 0% and 32%; however, 75% of the comorbidity data were missing. Regarding volume, 3 of 8 HNOCs and affiliated centers were classified as low-volume centers (<200 curatively treated patients) (results not
Patient Characteristics
Current HNC Care and Variation
A total of 9 process indicators had a numerator of >400 patients: 7 from the medical specialist perspective and 2 from the AHP perspective (Table 3). The indicator for a case manager or nurse practitioner being present at the consultation to discuss the treatment plan had the lowest score and a high variation (adherence to QI-9: 28.9%; range, 0%–90%), and the indicator for the availability of a treatment plan had the highest score (adherence to QI-2: 100%; no range). The scores of 3 process indicators for an MDT and a treatment plan had an adherence of >90%, and the variation between hospitals was small, particularly for an MDT before treatment (QI-1: 95.4%; range, 88%–98%), availability of a treatment plan (QI-2: 100%; no range), and registration of whether the patient was treated according to protocols (QI-3: 97.2%; range, 86%–99%). The scores of 3 lead-time indicators varied between 48.4% and 82.6% adherence, with the lowest score for starting the first treatment within 30 calendar days from first consultation (QI-7: 48.4%; range, 24%–78%) and the highest score for finishing diagnostics within 21 calendar days (QI-6: 82.6%; range, 63%–100%). Variation between hospitals was greatest for malnutrition screening (QI-8: 49.9%; range, 2%–100%). Three indicators had >25% of missing data: involvement of a dental team before radiotherapy, start of first treatment within 30 days of first consult, and malnutrition screening before treatment.
Determinants
Comorbidity, nationality, and marital status had ≥75% missing data and were therefore excluded from
Patients Diagnosed From December 2013 to January 2017a
Discussion
This study aimed to assess the variation of current HNC care between Dutch hospitals and the influence of patient and hospital characteristics. Quality of current care was low for the QI regarding the presence of a case manager at the consultation to discuss the treatment: the case manager was present in nearly one-third of all hospitals. Hardly any variation was seen between hospitals regarding the QI for the percentage of patients discussed in the MDT. However, almost all hospitals scored differently, varying from 2% to 100%, for the malnutrition screening QI. Four characteristics at patient and hospital levels had a large influence on indicator scores: tumor stage, PS, tumor site, and hospital volume.
Previous research shows that care, despite national guidelines, is not always delivered the same way in different hospitals.31,32 We also found large variation in care delivery from the perspectives of medical specialists and AHPs. The largest variation was for QIs from the perspective of AHPs, perhaps because of the lack of evidence-based guidelines. Regarding patient characteristics,
Results of Indicators From Medical Specialist and AHP Perspectives
Influenced Scores of Indicators in the Multilevel Analyses and Corresponding Stratified Indicator Score
Other countries already have databases with the aim of improving quality of care and patient outcomes, such as the National Head and Neck Cancer Audit in the United Kingdom and the Danish Head and Neck Cancer database.35,36 A difference with our database is that the DHNA relies on census- and evidence-based QIs from the 3 different perspectives of medical specialists, AHPs, and patients, whereas the other databases are mostly set up from an epidemiologic perspective and form a base for clinical trials. An epidemiologic database is not primarily intended for quality registration. In addition to our initiative with quality-of-care data, de Ridder et al31 published a retrospective cohort evaluation study about the variation in HNC care in the Netherlands, showing hospital variation in volume and treatment of oropharyngeal and oral cavity cancer. However, conclusions were drawn from retrospective data that were collected for a purpose other than quality registration. The set up and prospective character of the DHNA makes it far more suitable for measuring and monitoring the quality of HNC care.
Low indicator scores do not necessarily mean that the quality of care is poor; rather, it is a signal to evaluate further. Dentler et al37 define good-quality data as being available, complete, and correct. Further, they show that data quality can influence indicator results. Our quality registration is in the start-up phase. The quality of our data does not seem optimal. Regarding completeness, it is crucial to include all patients with HNC to conclude anything about general healthcare.38 At the moment, registration is not complete, although all 14 hospitals were encouraged to register all patients with HNC treated in their hospital. Centers with low inclusion must be encouraged to include all patients. Completeness will be increased when feedback on indicator scores is actually used in hospitals and when these scores begin improving. Regarding availability, we excluded 666 patients (2,351 minus 1,685) due to insufficient data in the first step (Figure 1). This can be explained by the start-up phase of the registration in which hospitals are still figuring out how to collect data efficiently, resulting in incomplete records and incomplete patient inclusion. We assume that these patients' data are missing at random. Hereafter, the indicator with the highest number of included patients had a denominator of 1,013 patients. We assume that the difference between 1,685 and 1,013 patients is also random, due to a high registration burden and health professionals stopping halfway in the registration system. However, the percentage of missing data for the indicator scores, varying from 1.1% to 39.4%, might be due to high registration burden or to the fact that data were missing in the electronic record, such as the percentage of patients who received a malnutrition screening at predefined moments. It might be possible that health professionals skipped the section on malnutrition screening in the registration system. This type of missing data may also be random, because it would not depend on the type of patients. Regarding the missing data for the determinants, we take into consideration that most of the missing data will also be random due to the start-up phase and high registration burden. Regarding correctness, data from the DHNA will be compared with data from the Netherlands Cancer Registry, which is managed by the Netherlands Comprehensive Cancer Organisation. However, in the past years, the online self-registration system has improved due to the optimizing of registration guidelines, the learning curve of health professionals, and the exploring of links between electronic patient record systems and the DHNA database. Therefore, better data quality is expected in the future. Hereafter, all indicators could be evaluated, rather than only the best-performing indicators.
Strengths and Limitations
A strength of this study is that it enabled census- and evidence-based QI scores to be assessed for nearly 1,700 patients. Because only indicators with a numerator of >400 were taken into account for calculating a precise indicator score, insight into the total quality of care for patients with HNC was impossible. In addition, data quality criteria, such as availability, completeness, and correctness, need further improvement. We will achieve this by giving feedback about missing data and indicator scores, advising on improvement processes, and validating the registered data. Furthermore, more data are necessary to perform a reliable case mix correction. However, we assume that the case mix will be comparable between the different participating hospitals for the following reasons: (1) distribution across the 14 hospitals is largely geographic and not based on specific expertise, and (2) Dutch health insurance is based on a social system with basic assurance for each citizen, and therefore it is unlikely that there is an association between socioeconomic status and quality of care. Lastly, based on the opinions of the caregivers involved in the registration, collection of data seems to be difficult and time-consuming. Focus on less labor-intensive registration systems and more automatic data collection is needed, particularly registration at the source. Initiatives in the United States and the Netherlands have already started.39,40
Conclusions
Initiating a multidisciplinary quality registration system based on census- and evidence-based QIs is challenging; the main requirements are guidelines for developing QIs and sufficient data to evaluate the QIs, determine case mix, and assess the effects of variation on outcomes. However, our recently launched audit already visualizes the variation in current care among hospitals that deliver HNC care. One of the next steps for the DHNA will be to conduct more research to explore the association between variation in quality of care and differences in patient outcomes and to identify targets for quality improvement.
The authors have disclosed that they have no financial interests, arrangements, affiliations, or commercial interests with the manufacturers of any products discussed in this article or their competitors.
The study sponsor, CZ Health Insurer, did not have any role in study design and the collection, analysis, and interpretation of data, nor in writing the article and the decision to submit it for publication.
References
- 1.↑
World Health Organization. Quality of Care: A Process for Making Strategic Choices in Health Systems. Available at: http://www.who.int/management/quality/assurance/QualityCare_B.Def.pdf. Accessed July 25, 2016.
- 2.↑
Porter ME, Teisberg EO. How physicians can change the future of health care. JAMA 2007;297:1103–1111.
- 4.↑
Kotter T, Blozik E, Scherer M. Methods for the guideline-based development of quality indicators—a systematic review. Implement Sci 2012;7:21.
- 5.↑
Campbell SM, Braspenning J, Hutchinson A et al.. Research methods used in developing and applying quality indicators in primary care. BMJ 2003;326:816–819.
- 7.↑
Del Turco MR, Ponti A, Bick U et al.. Quality indicators in breast cancer care. Eur J Cancer 2010;46:2344–2356.
- 8.↑
Wennekes L, Ottevanger PB, Raemaekers JM et al.. Development and measurement of guideline-based indicators for patients with non-Hodgkin's lymphoma. J Clin Oncol 2011;29:1436–1444.
- 9.↑
van den Boogaard E, Goddijn M, Leschot NJ et al.. Development of guideline-based quality indicators for recurrent miscarriage. Reprod Biomed Online 2010;20:267–273.
- 12.
Shiraz F, Rahtz E, Bhui K et al.. Quality of life, psychological wellbeing and treatment needs of trauma and head and neck cancer patients. Br J Oral Maxillofac Surg 2014;52:513–517.
- 13.
Vokes EE, Weichselbaum RR, Lippman SM et al.. Head and neck cancer. N Engl J Med 1993;328:184–194.
- 14.↑
Gritz ER, Carmack CL, de Moor C et al.. First year after head and neck cancer: quality of life. J Clin Oncol 1999;17352–17360.
- 15.↑
Hammerlid E, Wirblad B, Sandin C et al.. Malnutrition and food intake in relation to quality of life in head and neck cancer patients. Head Neck 1998;20:540–548.
- 16.
De Boer MF, McCormick LK, Pruyn JF et al.. Physical and psychosocial correlates of head and neck cancer: a review of the literature. Otolaryngol Head Neck Surg 1999;120:427–436
- 17.
Bjordal K, Ahlner-Elmqvist M, Tollesson E et al.. Development of a European Organization for Research and Treatment of Cancer (EORTC) questionnaire module to be used in quality of life assessments in head and neck cancer patients. EORTC Quality of Life Study Group. Acta Oncol 1994;33:879–885.
- 18.
van den Berg MG, Rasmussen-Conrad EL, Gwasara GM et al.. A prospective study on weight loss and energy intake in patients with head and neck cancer, during diagnosis, treatment and revalidation. Clin Nutr 2006;25:765–772.
- 19.↑
Starmer HM, Gourin CG. Is speech language pathologist evaluation necessary in the nonoperative treatment of head and neck cancer? Laryngoscope 2013;123:1571–1572.
- 20.↑
van Overveld LF, Braspenning JC, Hermens RP. Quality indicators of integrated care for patients with head and neck cancer. Clin Otolaryngol 2016;42:332–329.
- 21.↑
Breedveld F, Grol R, Hoeksema J et al.. Kwalititeitregistraties als basis voor verbeteren en vertrouwen. Available at: http://www.nfukwaliteit.nl/pdf/1208.02_Notitie_NFU_Kwaliteitsregistraties.pdf. Accessed April 14, 2017.
- 22.↑
Stienen JJ, Hermens RP, Wennekes L et al.. Variation in guideline adherence in non-Hodgkin's lymphoma care: impact of patient and hospital characteristics. BMC Cancer 2015;15:578.
- 23.↑
Mourad SM, Nelen WL, Hermens RP et al.. Variation in subfertility care measured by guideline-based performance indicators. Hum Reprod 2008;23:2493–2400.
- 24.↑
Netherlands Comprehensive Cancer Organisation (IKNL). Figures on Cancer [Cijfers over Kanker]. Available at: http://www.cijfersoverkanker.nl/selecties/Dataset_1/img5528b4d09d2a2. Accessed May 22, 2017.
- 25.↑
Chien CR, Lin HW, Yang CH et al.. High case volume of radiation oncologists is associated with better survival of nasopharyngeal carcinoma patients treated with radiotherapy: a multifactorial cohort analysis. Clin Otolaryngol 2011;36:558–565.
- 26.↑
Gourin CG, Frick KD. National trends in oropharyngeal cancer surgery and the effect of surgeon and hospital volume on short-term outcomes and cost of care. Laryngoscope 2012;122:543–551.
- 27.↑
Netherlands Comprehensive Cancer Organisation (IKNL). Head and Neck Cancer. Available at: https://www.iknl.nl/oncologische-zorg/tumorteams/hoofd-halskanker. Accessed February 2, 2017.
- 28.↑
van Doorn A, Kirschner K, Bouma M et al.. Evaluatie van het onderdeel medisch handelen van de accreditering. Huisarts Wet 2010;53:141–146.
- 29.↑
Gooiker GA, Kolfschoten NE, Bastiaannet E et al.. Evaluating the validity of quality indicators for colorectal cancer care. J Surg Oncol 2013;108:465–471.
- 30.↑
Ouwens MM, Marres HA, Hermens RR et al.. Quality of integrated care for patients with head and neck cancer: development and measurement of clinical indicators. Head Neck 2007;29:378–386.
- 31.↑
de Ridder M, Balm AJ, Baatenburg de Jong RJ et al.. Variation in head and neck cancer care in the Netherlands: a retrospective cohort evaluation of incidence, treatment and outcome. Eur J Surg Oncol 2017;43:1494–1502.
- 32.↑
Doornaert P, Terhaard CH, Kaanders JHDutch National Platform Radiotherapy Head and Neck Cancer. Treatment of T3 laryngeal cancer in the Netherlands: a national survey. Radiat Oncol 2015;10:134.
- 33.↑
Tyldesley S, Roques TW, Erridge S. General practitioner assessment of stage and performance status in lung cancer patients at a population level: implications for prognosis and radiotherapy needs analyses. Lung Cancer 2007;57:381–388.
- 34.↑
Birkmeyer JD, Siewers AE, Finlayson EV et al.. Hospital volume and surgical mortality in the United States. N Engl J Med 2002;346:1128–1137.
- 35.↑
Overgaard J, Jovanovic A, Godballe C et al.. The Danish Head and Neck Cancer database. Clin Epidemiol 2016;8:491–496.
- 37.↑
Dentler K, Cornet R, ten Teije A et al.. Influence of data quality on computed Dutch hospital quality indicators: a case study in colorectal cancer surgery. BMC Med Inform Decis Mak 2014;14:32.
- 38.↑
Sorensen HT, Sabroe S, Olsen J. A framework for evaluation of secondary data sources for epidemiological research. Int J Epidemiol 1996;25:435–442.
- 39.↑
Kern LM, Kaushal R. Accuracy of electronically reported “meaningful use” clinical quality measures. Ann Intern Med 2013;159:73.
- 40.↑
Registration at the Source. Available at: https://www.registratieaandebron.nl/. Accessed April 14, 2017.
