The diagnostic performance of patient symptoms in screening for COPD

Abstract

It is recommended that screening for COPD be restricted to symptomatic individuals, but supporting evidence is lacking. We determined the performance of wheeze, cough, phlegm, and dyspnea in discriminating COPD versus non-COPD in a population-based sample of 1332 adults. Area Under the Receiver Operating Curves (AUC) indicated that symptoms had modest performance whether assessed individually (AUCs 0.55–0.62), or in combination (AUC for number of symptoms as the predictor 0.64). AUC improved with the inclusion of multiple other factors (AUC 0.71). Restricting screening to symptomatic individuals is unlikely to substantially improve the yield of general population screening for undiagnosed COPD.

Introduction

Chronic Obstructive Pulmonary Disease (COPD) is a common inflammatory lung condition that is characterized by symptoms of shortness of breath, cough, and sputum production [1]. Although COPD is under-diagnosed in the community [2], several major guidelines, including from the influential US Preventive Services Task Force, have recommended against the use of spirometry to screen for COPD in asymptomatic individuals in the general population because the number-needed-to-screen (NNS) to prevent adverse disease outcomes is prohibitively large [3, 4]. Some have advocated for case finding strategies to improve the diagnosis rate in the community in a more cost-effective manner, for example by targeting spirometry only among symptomatic individuals [5]. However, many patients with undiagnosed COPD have mild disease and may have few (if any) respiratory symptoms [6], and individuals without COPD can experience symptoms similar to those of COPD patients [7]. In addition, symptomatic COPD patients tend to be diagnosed earlier [6] and are therefore removed from the pool of cases that would be detected through a screening program. We determined the diagnostic performance of patient symptoms for screening in the general population to assess whether the yield of screening could be improved by restricting it to the symptomatic population.

Methods

We used data from the Canadian Cohort of Obstructive Lung Disease (CanCOLD) Study. CanCOLD was a prospective cohort study of 1332 adults ≥40 years who were sampled from the general Canadian population with multi-level sampling to ensure representativeness. Participants were followed for a maximum of 3 years with visits at 18-month intervals [8]. They reported their demographic information, smoking status and history, comorbidities, and respiratory symptoms at each visit using validated questionnaires. Diagnostic spirometry was performed at each visit and persistent airflow limitation was defined as post-bronchodilator FEV₁/FVC < lower limit of normal. Participants were deemed to have undiagnosed COPD if they had persistent airflow limitation but did not report previous physician-diagnosed COPD, emphysema, or chronic bronchitis. Subjects with a previous diagnosis of COPD were excluded. Information was collected on the frequency or severity of cough, phlegm, and wheeze using three questions for each symptom. The responses were coded as a variable ranging from 0 to 3 for each symptom. Breathlessness was measured using the Medical Research Council dyspnea scale. We also assessed the total number of symptoms experienced by each participant (0–4 range).

First, we determined the independent associations between individual symptoms and the presence of undiagnosed COPD (v. no COPD) using a logistic regression model with symptoms as separate independent variables and adjusting for participant demographics, comorbidities, smoking status, and pack-years. Second, we assessed the diagnostic performance of symptoms when used individually to distinguish patients with undiagnosed COPD from non-COPD subjects. We evaluated the sensitivity and specificity of each symptom at different thresholds (i.e., 0, 1, 2, or 3) for defining a patient as symptomatic. We fitted Receiver Operating Characteristic (ROC) curves to determine the Area Under the Curve (AUC) for each symptom individually, as well as their combined performance using the total number of symptoms. Finally, we used the AUC of the above-mentioned logistic regression model to assess the performance of all individual symptoms and covariates together. An AUC of 0.5 indicates the model has no discriminatory ability. Generalized estimating equations were used in all models to account for clustering of observations within individuals.

Ethics approval for CanCOLD was obtained from the relevant institutional review board at each study site. Written informed consent was obtained from all participants prior to study entry.

Results

The mean age of the sample was 67.4 years [SD 9.7], 44% were females, and 40% of the participants had three study visits. The overall prevalence of undiagnosed COPD was 26%; 95% had mild to moderate disease based on the GOLD spirometric grading system [1]. The regression model indicated that reporting wheeze, dyspnea, and cough on most days were independently related to the presence of undiagnosed COPD (Table 1). However, symptoms alone had poor performance in identifying patients with undiagnosed COPD. Almost all symptoms, regardless of the severity, had sensitivities and positive predictive values less than 50% (Table 2).

Table 1 Association between symptoms and other patient characteristics with the odds of having undiagnosed COPD (v. no COPD)

Table 2 Prevalence of each symptom severity category in the whole population (‘Prev’) across all study visits, and the prevalence of undiagnosed COPD (‘COPD+’) within that symptom severity category. Sensitivity (‘TP’), specificity (‘TN’), positive predictive value (‘PPV’), and negative predictive value (‘NPV’) of each symptom when used alone to classify undiagnosed COPD (v. no COPD) using different severity thresholds

If screening was applied to the general population (a “blind” screening approach), the NNS to detect one COPD case would be 3.8. If screening was restricted to individuals who reported symptoms, the NNS would be 3.0, but compared with the “blind” approach, an additional 17% of individuals with persistent airflow limitation would be missed.

The ROC curves indicated that wheeze had the best performance among all symptoms (AUC = 0.62), followed by cough and dyspnea (each AUC = 0.57), and phlegm (AUC = 0.55). The total number of symptoms performed marginally better than any symptom alone (AUC = 0.64). AUC improved by ~ 0.06 when each of the symptoms was combined with smoking history (measured as pack-years), resulting in an AUC of 0.67 for a model that included wheeze and pack-years. The model that included all individual symptoms and covariates improved the AUC to 0.71 (Fig. 1).

Fig. 1

Receiver operating characteristic (ROC) curves for a model with all of the symptoms and covariates included (‘All variables’), as well as for each of the symptoms individually and the total number of symptoms reported by study participants (‘Total symptoms’)

Discussion

Our results indicate that symptoms are relatively poor at discriminating undiagnosed COPD patients from non-COPD subjects in a general population. The use of symptoms for screening is unlikely to significantly improve the diagnostic yield compared with “blind” screening in the general population. These data highlight the apparent paradox in finding strong associations between symptoms and undiagnosed COPD, yet poor diagnostic performance when symptoms are used to diagnose these “hidden” COPD cases. This observation is consistent with the well-established notion that a predictor can be strongly associated with an outcome while still being a poor classifier of that outcome [9]. Association models are useful for evaluating relationships at the population level, but classification models are more relevant to decisions at an individual level, specifically whether a test (e.g., the presence of a symptom) can detect the underlying disease state (e.g., undiagnosed COPD).

Previous studies have evaluated the merits of opportunistic case detection based on patient characteristics at the point of care [10, 11]. They generally found that patient characteristics and symptoms have modest capacity in detecting undiagnosed COPD [10,11,12]. A unique feature of our study is its population-based sample, which provides new evidence on whether the yield of population-based screening can be improved if symptoms are considered in the inclusion criterion (e.g., through advertisements for referral of symptomatic individuals for lung function testing). Although the costs and benefits of early intervention between symptomatic and asymptomatic patients were not considered here, our results do not support the use of symptoms in “case finding” for COPD. Symptoms should be used in conjunction with other characteristics such as pack-years of smoking to improve the diagnostic performance. Future studies should evaluate the cost-effectiveness of this approach considering the long-term outcomes associated with earlier diagnosis of COPD.