Introduction

The verb 'to audit' dates from sixteenth century when it meant 'to make an official systematic examination of accounts'. It has retained this meaning till to-date and if the words 'of accounts' are omitted, it is exactly what clinical audit means.₁ Attempts to analyze the results of treatments have been made since antiquity though the analyses have often been biased by reconceptions or faulty logic. The Royal College of Surgeons of England has defined audit as the 'systematic appraisal of the implementation and out come of any process in the context of prescribed targets and standards.'₂

Audit has three components - structure, process, and outcome. In 1982, Sheldon defined clinical audit as 'A study of outcome of part of the structure, process and outcome of medical care carried out by those personally engaged in the activity concerned, to measure whether set objectives have been attained and thus assess the quality of care delivered'. The audit of structure is essentially administrative. The audit of process is that, if correct steps are taken in correct order, the outcome measured in goods or services will be satisfactory and the audit of outcome is self-explanatory.₃

There are many scoring systems that predict the risk of mortality with varying degrees of accuracy. However morbidity is almost universally ignored. The best known and most widely used scoring system is APACHE II (Acute Physiology And Chronic Health Evaluation) which is ideal in the intensive care patient but requires 24 hours of observation and weighing tables for individual disease states. Also, APACHE III scoring system has been recently introduced. While all these scoring systems are used in generally sick patient, none are exclusively for surgical patients. POSSUM (Physiological and Operative Severity Score for the enUmeration of Mortality and morbidity) is the only scoring system which is meant for exclusive use in surgical cases.₄ Of late, ‘Portsmouth' modification of POSSUM (P-POSSUM) scoring system has been introduced which predicts mortality more accurately than the former.₅

In this prospective study, an attempt has been made to determine the accuracy of P-POSSUM prediction of in-patient mortality in operated general surgical patients in our hospital (elective and emergency cases), to assess the mortality in six surgical units and compare `raw' mortality (percent of observed deaths) with risk-adjusted mortality rates.

Patients And Methods

All in-patients admitted and operated by the six surgical units of the Department of General Surgery at Kasturba Medical College Hospital, Manipal, India from 15th January 2000 to 15th May 2000 (four months) were included in the study. All the necessary data was collected prospectively. The physiological data was entered in proforma sheets at admission in emergency cases and a day before in elective patients or when the results of tests were available. Necessary investigations were done for all the patients. Operative data was obtained from the records and by personal communication with the operating surgeon, when required. The scoring system used to classify patients was similar to that of Copeland et al.⁴

The data was analyzed using the P-POSSUM formula for mortality i.e., Ln [R (1-R)] = -9.065 + (0.1692 * physiological score) + (0.1550 * operative severity score), where R is the predicted risk of mortality. For a given range of risk, the number of operations within that range was given together with the mean risk for the operations and the predicted number of deaths was calculated (number of operations * mean risk). This was compared with the actual number of occurring deaths.⁵

Results

A total of 493(n) patients were operated during the period of study, in six general surgery units of which 308 were males and 185 females. The types of surgeries done were as indicated in Table 1. Of these, 103 were emergency surgeries. Among 493 patients operated, 26 patients died.

Figure 1

Table 1. Type of surgeries done

The distribution according to the range of predicted mortality rate (%) was as in Table 2. The results were evaluated by χ² test. Mean of 493 patients physiological score was 18.93(S.D. 8.05) and mean of operative score was 10.54 (S.D. 5.24). Table 2 also shows a significant lack of fit ( χ²=0.73, 4 d.f, p>0.5), i.e., observed values are not significantly different from predicted value, hence the formula predicts deaths accurately.

Figure 2

Table 2. Predicted and reported deaths using Portsmouth POSSUM equation in 493 patients.

χ²=0.73; 4 d.f., p>0.5 *Rounded off to nearest whole number.

Of the 26 patients who died, 14 had under gone laparotomy for benign conditions (4 elective, 10 emergency), 8 for malignant conditions (4 each, elective and emergency), 2 had undergone major amputation (emergency). One patient died of myocardial infarction (MI) following elective hernia repair and 1 died following emergency minor amputation. Mean physiological score of 390 patients undergoing elective surgery was 17.46 (S.D.± 6.35) and mean operative score was 9.34 (S.D.± 4.21). The mean of physiological score of patients undergoing emergency surgery was 24.48 (S.D.±10.94) and operative score was 15.07 (S.D.±6.20). The higher physiological and operative score among emergency patients was reflected in higher mortality in these patients.

The mortality ranged from 3.38% to 7.2%. The use of ‘raw' mortality rates to compare and audit between the units is not justified because the patient population groups may not be the same (Table 3).

Figure 3

Table 3. Total number of operations done and the `raw' mortality in each unit

Even after adjusting for the different population groups using P-POSSUM predictor equation, we noted that the O:E ratio ranged from 0.66 in unit V to 1.25 in unit IV. Though there was large difference in the raw mortality between unit II and unit VI, on adjusting the risk, the O:E ratio between them was all most equal, i.e., 0.83 and 0.8 respectively (Table 4).

Figure 4

Table 4. Risk adjusted mortality rates. (Using P-POSSUM logistic regression equation)

*Rounded off to nearest whole number.

The mean physiological score ranged from 18.1 (S.D.±7.6) in unit II to 21.2 (S.D.±8.5) in unit VI. The mean operative score ranged from 9.6 (S.D.±4.4) in unit IV to 11.3 (S.D.±5.8) in unit VI (Table 5).

Figure 5

Table 5. Mean physiological and operative severity score in the six units.

The distribution of cases operated by each unit is as in Table 6, they ranged from 114 (unit I) to 55 (unit VI).

The type of cases done in each of the units is shown in Table 6. The criteria used to classify them in to minor, moderate, major and major+ surgeries is as per recommendations of Copeland et al.⁴

Figure 6

Table 6. Distribution of type of surgeries done in each of the six units

Discussion

The interpretation of the Observed:Expected (O:E) death ratio is that, if it is more than one, the formula over predicts death. If it is less than one, it under predicts deaths and if it is equal to one, it correctly predicts deaths.

When the patients were divided according to range of risk (Table 2) and observed and predicted deaths were studied, they matched with each other. Surgical unit V had the least O:E death ratio (0.66) and unit IV had the highest (1.25). Low O:E ratio in unit V may be because of two reasons, either under reporting of deaths or they have expertise to treat patients to cut down mortality to such a low level. Cases must be individually scrutinized before accepting the latter view. A very high O:E ratio in unit IV may be because of over reporting deaths or they lack the expertise so as to cut down the mortality. They probably will have to analyze their methods of treatment and change / modify them.

Midwinter MJ et al ₆ evaluated mortality and morbidity in vascular surgery using POSSUM and the P-POSSUM. In 221 patients undergoing elective and emergency major arterial surgery under a single consultant, observed morbidity and mortality rates were compared with the rates predicted by POSSUM and P-POSSUM using linear method of analysis. The mortality rate predicted by POSSUM was significantly higher ( χ² =24.04, 6 d.f., p<0.001) than the observed rate. The mortality rate predicted by P-POSSUM was not significantly different from the observed rate ( χ² =9.00, 6 d.f., p=0.17). Therefore, he concluded that P-POSSUM predicts deaths more accurately than POSSUM.

Jones DR et al₇ evaluated POSSUM against APACHE II scores in 117 consecutive admissions to a high-dependency unit after major surgery with respect to one month mortality and morbidity rates. Eleven percent died and 50 percent developed a post-operative complication. Receiver operating characteristic curve analysis showed POSSUM to have good predictive value for mortality (area under curve 0.75) and morbidity (area under curve 0.82). They concluded that APACHE II scores had a significantly inferior predictive value for mortality (area under curve 0.54) (p <0.002). POSSUM was superior to APACHE II in prediction of mortality in-patients admitted to a high-dependency unit after general surgery. Prediction of post-operative complications by POSSUM is accurate and may be useful for audit. Midwinter J et al ⁶ have shown that P-POSSUM predicts deaths more accurately than POSSUM. Therefore we can probably say that P-POSSUM predicts deaths more accurately than APACHE II, however there are no other studies to indicate the same.

Prytherch DR et al⁵ came out with a modification of POSSUM equation to over come the problem of over prediction and they called it Portsmouth-POSSUM (P-POSSUM). They collected necessary data from 10000 general surgical interventions. The POSSUM mortality equation was applied to the full 10000 surgical episodes. The 10000 patients were arranged in chronological order and the first 2300 were used as training set to produce the modified POSSUM i.e., P-POSSUM predictor equation. This was then applied prospectively to remaining 7500 patients. The predicted deaths in these groups of patients showed close agreement to those observed. The necessary details of the last 1500 general surgical cases were published in their article. The results are compared below with our results (Table 7). The O:E ratio in all the risk ranges match with one another and is nearing one.

Figure 7

Table 7. Comparing our results with the results of Prytherch DR et al5 (last 1500 cases). Values in brackets are the results of our series using P-POSSUM.

Midwinter-MJ et al ⁶ evaluated mortality and morbidity in vascular surgery using POSSUM and the P-POSSUM. In 221 patients undergoing elective and emergency major arterial surgery under a single consultant, observed morbidity and mortality rates were compared. The mortality rate predicted by P-POSSUM was not significantly different from the observed rate ( χ² =9.00, 6 d.f., p=0.17). The results are compared with our study in Table 8. The O: E ratios in all the risk ranges agree with one another and are nearing one.

Figure 8

Table 8. Comparison of observed and predicted deaths from P-POSSUM logistic regression equation (Midwinter et al 6 study). Values in brackets are the results of our series.

*Rounded off to nearest whole number.

Conclusions

In conclusion, this study has demonstrated that Portsmouth-POSSUM i.e., P-POSSUM predictor equation for mortality predicts death accurately in general surgical patients and comparing risk adjusted mortality rate is more meaningful than comparing ‘raw' mortality rate. It can be used as an audit tool to compare the pear group results and learn from one another.

Correspondence to

Dr. Gabriel Rodrigues, MS., DNB. Assistant Professor of Surgery, 157, KMC Quarters, Madhavanagar, Manipal – 576 104. Karnataka, INDIA. Tel: +91-820-2572647 Fax: +91-820-2570061 Email: gabyrodrigues@excite.com