Introduction

Surgical resection remains the most effective treatment for lung cancer, which is one of the most frequently treated malignancies worldwide [¹,²]. Pulmonary complications after lung resection continue to be a high source of morbidity [³]. The ability to predict which patient will have the highest risk of developing complications can aid the clinician in focusing efforts to reduce morbidity and mortality.

Many of the previous attempts to predict pulmonary complications after thoracic resection utilizing single variable measurements derived from pulmonary function testing and combination variables have shown only modest promise [⁴,⁵,⁶,⁷]. Pierce, for example, proposed the predictive postoperative product (PPP); [ PPP = ppoFEV1% x ppoDLCO% ] that was used to predict postoperative mortality in surgical thoracic patients [⁷]. A retrospective evaluation of the relationship between pulmonary variables and postoperative complications was used in the development of the predictive respiratory complication quotient, (PRQ); [ PRQ = (ppoFEV1%) x (ppoDLCO%)2 / A-a gradient ], with promising results [⁸]. The purpose of this study was to prospectively evaluate pulmonary variables, and the two composite formulas as predictors of pulmonary complications and to attempt to develop predicting models No study has yet evaluated these variables on a prospective basis.

Materials And Methods

A prospective study was performed on thoracic surgical patients operated in over a period on one year. The protocol was approved by the institutional review board. Patients were eligible for entry if they were considered to have a resectable tumor on the basis of preoperative clinical and radiological evaluation. All patients, once scheduled for thoracic surgery, were considered for study participation. Criteria were: age > 18 years and presence of pulmonary malignancy. Informed consent for study participation was obtained from each patient according to Memorial Sloan Kettering Cancer Center (MSKCC) research protocol. Patients having other, unrelated surgery on the same admission, patients with concomitant rib, chest wall, diaphragmatic, pericardial, and/or pleural resection were excluded. If the patient requested, he/she was removed from the study. Data collection was funded by an in-house grant.

Pulmonary function tests, arterial blood gases and A-a gradient at room air were performed prior to surgery. The predictive post operative product (PPP), the predictive respiratory quotient (PRQ) and probability risks were calculated prior to surgery. Researchers performing postoperative follow-up were blinded to PRQ, PPP and risk of complications calculation results. Quantitative V/Q scans were obtained in high-risk patients at the discretion of the medical or surgical attending. In those who did not undergo V/Q scans, predicted postoperative values were estimated using the number of pulmonary segments excised. Patients were followed during general ward and Intensive Care Unit stay. Patients discharged before 30-days were followed at home by phone.

Postoperative pulmonary complications were prospectively monitored, and were defined as the following pulmonary problems that required special treatment within 30 days after surgery. They included: respiratory failure requiring reintubation, pneumonia, atelectasis requiring bronchoscopy, and the need for supplemental oxygen (O2) at the time of hospital discharge. Pneumonia was defined as a new pulmonary infiltrate accompanied by fever that necessitated intravenous antibiotics for treatment. Wound infection, empyema, prolonged air leak, and bronchopleural fistula were considered to be surgical complications and were not defined as pulmonary complications in this study. Cardiac complications were defined as ischemia, myocardial infarction and any arrhythmia requiring intervention. The only dependent variable analyzed in this study was post-surgical pulmonary complications.

The following independent variables were analyzed in the study: forced expiratory volume (L/sec) (FEV1); its percent of predicted (FEV1%); carbon monoxide diffusion capacity adjusted (ml x mm Hg-1 x min-1)(DLCO); its percent of predicted (DLCO%); predicted postoperative FEV1% and DLCO% (ppoFEV1%, ppoDLCO%); arterial partial pressure of oxygen (mm Hg)(PO2); arterial partial pressure of carbon dioxide (mm Hg)(PCO2); alveolar-arterial oxygen gradient (room air, mm Hg) (A-a gradient); number of pack-years in smokers, number of segments excised, predictive postoperative product (PPP) and predictive respiratory quotient (PRQ).

Pulmonary variables were compared between group with and without complications with univariate parametric and non-parametric (where applicable) tests. Univariate logistic regression models were developed relating pulmonary complications to all independent variables, including the composite variables PRQ and PPP. The accuracy of PRQ, PPP and single variables as instruments for predicting postoperative complications were compared with Receiver-Operating Characteristic (ROC) curve analysis. A p value < 0.05 was regarded as statistically significant. Univariate artificial neural networks (ANN) models were developed using each of the 14 predictor variables to predict complications and ordered by their predictive capability. Subsequently, multivariate ANN models and associated probability of complication (risk) curves were created in a step-wise process as listed in the appendix

Results

One hundred and seventy-five patients with lung cancer were evaluated, only one death occurred. There were 81 males and 94 females. The mean age was 64.3 years, ranging from 30 to 84. Because there was only one death, mortality was not analyze. One hundred and forty patients had non-small cell lung cancer and the rest of patients had metastatic cancer to their lungs. The surgeries performed were as follow: 129 lobectomies, 37 segmental or wedge resections and 9 pneumonectomies. A total of 69 patients had complications, 35 patients had surgical and 26 patients had cardiac complications. Forty-three (24.57%) patients had pulmonary complications. Pulmonary complications included: atelectasis in 6 (15%) of cases, pneumonia in 29 (48%), need for supplemental O2 at the time of discharge in 9 (21%), and respiratory failure in 1 (4%). The mean LOS for all patients was 7.20.3 days. The range of LOS was 3 to 8 days for 75% of the patients. Age, pack-years, arterial blood gases (PO2, PCO2), FEV1, ppoFEV1%, DLCO, ppoDLCO%, A-a gradient, PRQ and PPP were not different comparing patients with and without pulmonary complications, all the parameters except PCO2 were statistically significantly different between groups on univariate tests (Table 1)

Figure 1

Table 1: Univariate comparison between groups with and without pulmonary complications

a) Measurements are reported as mean ± standard deviation (SD) in first columns and median values in second columns. p-values are t-test values comparing means if the data are normally distributed; otherwise they are Mann-Whitney Rank Sum p-values comparing medians. b) Statistically significant difference between means of data sets that are normally distribute c) Statistically significant difference between medians of data sets that are not normally distributed. FEV1 = forced expiratory volume in 1 second (liters); FEV1% = percent FEV1; DLCO = carbon monoxide diffusion capacity adjusted (mL x mm Hg-1 x min-1); DLCO% = percent DLCO; PO2 = arterial partial pressure of oxygen (mm Hg); PCO2 = arterial partial pressure of carbon dioxide (mm Hg); PPOPFEV1% = predicted postoperative FEV1%; PPOPDLCO = predicted post-operative DLCO%; A-a gradient = alveolar-arterial oxygen gradient (room air) in mm Hg; PPP = predictive postoperative product; PRQ = Pulmonary Resection Quotient.

Eighty percent of patients suffering pulmonary complications had a ppoFEV1% <76%, ppoDLCO% 61%, PPP<3850, PRQ<10 000. Odds ratios of pulmonary complications when the parameter was higher than the specified value compared to the parameter lower than this value were: 0.21 (95% CI 0.097-0.457) for PRQ; 0.236 (95% CI 0.11-0.51) for PPP; 0.39 (95% CI 0.18-0.83) for ppoDLCO% and 0.39 (95% CI 0.18-0.85) for ppoFEV1%. Validated areas under ROC curves were calculated for logistic regression models for all 14 predictor variables taken singly. PRQ was the best single predictor of complications with ROC area of 0.74, followed closely by pack-years (ROC=0.72), the A-a gradient (ROC=0.70), and DLCO (ROC=0.70). ROC area of PPP index was 0.684. None of these models and variables gave validated ROC areas exceeding 0.76. Figure 1 shows patients' population with different composite values in groups with and without complications. The Pierce index, PPP could not be validated secondary to the single death in our cohort. A composite ANN model with ROC curve area of 0.821 [SE 0.041; 95% CI 0.74-0.90] was built (Figure 2). The variables in it, rank-ordered by importance, were: FEV1%, PAKSYEAR, DLCO, PO2, ppoDLCO%, ppoFEV1%, PPP, FEV1, DLCO%, PRQ, PCO2, AGE, AAGRAD, gender. Univariate ANN models were built using each of the variables listed in methods and evaluated using ROC curve areas (Table 2).

Figure 2

Figure 1: Cumulative distribution functions for patients with and without pulmonary complications.

Figure 3

Figure 2: Validated ROC curve for composite ANN model (175 cases dataset).

Figure 4

Table 2: Single variables used for ANN modeling rank-ordered by performance.

Discussion

The goal of this prospective study was to evaluate a number of pulmonary variables in their ability to predict pulmonary complications after thoracic surgery and among them, to test 2 composite indexes, PPP and PRQ. These two composites incorporate predicted postoperative FEV%, predicted postoperative DLCO% and the latter one incorporates A-a gradient as well. In the original studies, the PPP and the PRQ were shown to be able to differentiate patients who suffered pulmonary complications from those who did not, and using a specific cutoff point [⁷,⁸]. The present study aimed to validate the choice of a cutoff value and any of the formulas on a prospective database. This analysis was performed on 175 patients subject to pulmonary resections. The original PRQ study was performed on a database that included 61 patients and the PPP on 54 patients.

All the 14 variables tested, including age, gender, smoking history, pulmonary function tests, predicted postoperative values of FEV1 and DLCO, significantly differed between the groups with and without pulmonary complications. PCO2 was the only exception. Eighty percent of the population with pulmonary complications had ppoDLCO% less than 60% and ppoFEV1% less than 76%. FEV1% and DLCO% were found to be predictive of pulmonary complications in a number of studies [⁹,¹¹,¹³]. FEV1 remains a practical spirometric criteria for predicting postoperative morbidity and mortality. The guidelines for operability, such as FEV1 < 2 L, or less than 80% predicted for pneumonectomy, are still widely used in clinical evaluation, but clinicians are suggesting surgery for the patients with impaired lung functions as well. The usual cutoff point used in studies for both variables was around 70%. In one study it was shown that ppoFEV1% does not predict pulmonary complications in patients with FEV1 lower than 70% [¹³]. The authors concluded that ppoFEV1 should not be used alone as a selection criteria for operation in these high-risk patients. Some studies have examined lung resection in lung cancer patients with moderately and severely compromised lung function for example Cerfolio examined 85 lung cancer patients with mean FEV1 of 44% predicted undergoing resection and quoted a mortality rate of 2.4% and complication rate near 50%. Temeck examined Seventy-three lung cancer patients with a mean FEV1 of 42% predicted and noted a mortality rate of 1.4% and a morbidity rate of only 4%. A smaller study of lung resection in 32 lung cancer patients with FEV1 < 1.0 L without perioperative mortality was described by Miller and Hatcher [¹⁴,¹⁵,¹⁶]. Our results show that both ppoFEV1% and ppoDLCO% are useful in assessing postoperative pulmonary complications but the construct formula (PRQ) was the best predictor, because it incorporates not only lung function, but also physiologic responses but not good enough. A number of investigators have agreed that no single variable has the sufficient power to predict pulmonary complications or death in a thoracic surgical population. Therefore the importance of a composite formula is relevant in assessing risk in severely compromised patients [⁹]. This study confirms the association between an impaired ppoDLCO% and ppoFEV1% and an increased risk of pulmonary complications after lung resection, and demonstrates that a combination of ppoDLCO%, ppoFEV1% and A-a gradient is to some extent similar to single variables in predicting pulmonary complications.

In some studies, but not all, it has been found that advanced age is an important risk factor for early death [⁹]. In our study advanced age was not a significant predictor of complications and it supports the view that advanced age is not a contraindication to a thoracic surgical treatment, if the main concern is postoperative complications. Other risk factors for major complications after lung resections have been identified in studies, namely: male sex, pneumonectomy, operation duration, preoperative corticosteroids use and previously defined concomitant disease. Operation duration and preoperative corticosteroids were not analyzed in our study. Gender was not a factor in predicting complications in our study. Differences in definitions of complications and considering different endpoints can contribute to the apparent inconsistency in reported results in different studies. As an example, the PPP index was developed to predict pulmonary mortality, while PRQ was developed to predict pulmonary morbidity. The PPP was a good predictor of surgical mortality [⁷], but it was not shown to be a good predictor of pulmonary complications in the present study. The risk of death was estimated by Pierce in the following way: patients with PPP less than 1650 had a risk of 33% and above 2300, of 20%. In our patient population only 20 patients had PPP less than 1650 and 45% of these patients had pulmonary complications. Among patients with a PPP above 2300, 37% had complications. Prediction of deaths was not possible in our patient population, since there was only one death. The patient that died was 63 year- old man with a previous history of coronary artery disease, who postoperatively had a cardiac event and a pulmonary embolus. He had a calculated a PPP of 4128 and should have survived. Results from the original PRQ retrospective study the probability of pulmonary complications were high if the PRQ values less than 2,200. In this study only 20% of patients experiencing complications post-surgery fell below a PRQ value of 2,200. Consequently, a composite number was not proved to be a valid cutoff point for classifying patient outcomes based on this database. Other indicators of severity of disease and variables predicting complications may probably be found and validated with larger databases or different methods of analyses

The present study did not validate any single preoperative variable as a predictor of pulmonary complications. Even though some variables correlate with complications; overall, none of them predict pulmonary complications successfully in high-risk patients. The use of any cutoff point suggested by the composite formulas should not be used, because they do not predict post-thoracotomy pulmonary complication in thoracic surgical patients with much more accuracy than simple pulmonary function tests.

Appendix

The artificial neural networks (ANN) models transformed the original pre-surgery, patient-specific, variable values for thirteen variables into a single model output value whose magnitude correlated with the risk of complication.

The ANN model output values were ordered by magnitude and binned into a number of groups, either ten or twenty.

A risk of complication curve was developed. It employed the average model output value for each group as the x-value, and the true proportion of complications for that group plotted as the y-value. The curve was developed as the set of straight-line segments connecting these points. The x-axis represented the ANN model output value; the y-axis the risk of complication.

Confidence bounds were placed on the point-estimate risk of complication as a function of the sample size of each bin.

A check on the predictive capability of the neural net models over the entire risk spectrum was made employing the Hosmer-Lemeshow goodness-of-fit test. [¹⁴]

ANN models were developed for classifying patients pre-surgery into two groups - those who will suffer pulmonary complications and those who will not. The best models were determined by their respective classification accuracies, their Receiver-Operator Characteristic (ROC) curve areas, and their robustness (ability to generalize well to unseen data). Numerous ANN models were developed with different randomizations of the data. A random selection of 75% of the data was used to train each model. The remaining 25% was used to validate the models. A model's robustness was a criterion in selecting useful models. ROC curve areas for validation datasets were evaluated for all models to assess model performance. For models that generalize well, the validated ROC curve areas were used to compare and select superior models. A model was considered to have generalized well with a less than or equal to 0.05 difference in ROC curve areas between training and validation sets. A model was considered to have overfitted the data with greater than 0.05 differences in ROC curve areas, hence was not robust, did not generalize well, and was discarded.

The standard error (SE) of the validated ROC curve area was also derived to address the inherent variability from different randomizations of the data and, in combination with the point estimate of the ROC area, was used to place confidence bounds on the ROC area. When models were validated, the entire data set was used to determine the ROC area point estimate and its SE.

These composite ANN model results on the 175-patient database were compared with the results of three statistical models from our companion paper: SM1=cutoff PRQ value model, SM2= the equation PRQ model, and SM3=the interval PRQ model [¹]. The models were compared using the ROC curve areas. Accuracy was assessed by recording risk estimates for each patient. Risk estimates greater than or equal to a conversion value (either 0.5 or 0.4 herein) were converted to complications; those below the conversion value were converted to non-complications. These results were compared with actual outcomes to determine the proportion predicted correctly. Corresponding sensitivities and specificities, positive and negative predictive values and were also calculated. Subsequently, the composite ANN model was applied to the data of additional prospective 58 patients and the results compared with the results of SM1, SM2 and SM3 on the same 58 patients.

Finally, a new ANN model was trained and validated as described above on the combined database of 294 patients, consisting of 175 patients from the companion study, 61 patients from the original study [²] and 58 prospective patients (all recruited with the same protocol). The results of this new ANN model and SM1, SM2 and SM3 were compared.

Correspondence to

Rafael Barrera, Long Island Jewish Medical Center. New Hyde Park, New York. Tel: (718) 470 7645 Fax: (718) 962 2239 Email: rbarrera@lij.edu