A Likert style survey form was circulated to 81 anaesthetists (30 consultant, 51 trainees), 100 critical care nurses and 26 other members of staff including recovery nurses at the Manchester Royal Infirmary, Manchester, UK. Six questions with up to twelve choices in each question, concerning attitudes to the use of and characteristics of computers in anesthesia and critical care were asked. The questionnaire included notes that were intended to give staff unfamiliar with the concept of computer-based decision support systems sufficient background information to answer the questions.

A separate section on the questionnaire posed demographic questions about profession, grade, experience, age and gender. Subjects were also asked to score their access/use of computers and computer literacy on two scales of 0-4. Non-responders to the questionnaire were not followed up individually because initial prototyping of the questionnaire indicated that direct contact lead to discussions about the details of potential decision support systems that could bias the subject’s responses. As a result, the rate of return of questionnaires was low with a mean of 26.6% and a range from 39.2% (trainee anesthetists) to 19.0% (critical care nurses).

To ensure that responders were representative of the population of staff sampled a “demography only” questionnaire was sent to the same target groups after the close of acceptance of full questionnaire returns. This second questionnaire asked for replies only from those who had not responded initially.

All responses were pooled and the relative rank of choices within questions was examined using Friedman rank analysis. This analysis gave an overall measure of the approval/disapproval rating of the choices by the responders. To study differences in attitudes between different user groups, responses were divided into two in two different ways. Firstly, medically trained responders were separated from others to test for differences between professions. Secondly, responders were divided according to those who said they specialized in general anesthesia and those who included intensive care in their activities. Differences between the relative ranking of choices within questions between groups were tested using the Mann Whitney U statistic. The pooled data was examined using factor analysis by maximum likelihood to detect underlying “issues” identified by responders within questions. Clustering of responses was not considered significant unless the maximum likelihood weighting factor for cluster membership exceeded 0.5. Finally, responses were tested for correlation with the self-assessed computer access or use and computer literacy scores using Pearson rank correlation.

No statistical differences between sample or analysis groups in age, years post original professional qualification, years in grade, computer literacy or computer access were detected using T-testing or Mann Whitney U testing where appropriate (p<0.05). Statistically significant differences in gender ratio were found between the groups, with significantly higher proportions of male responders in both the medical analysis group and general anesthesia analysis group.

Question 1: What words should characterize the function of a decision support system?

This question consisted of a list of 12 words suggesting different general characteristics for any decision support system design. The results are shown in table 1 in descending order of rank. The key to all the tables is noted at the end of the paper. No statistically significant differences between the anesthesia (A) and critical care (CC) groups were seen. Statistically significant differences medical (M) and non-medical (NM) groups were found for 4 words “direct”, “manage”, “command” and “insist”. In all these cases the non-medical group gave higher rankings to these words than did their medical colleagues.

Figure 1

Table 1: Results for the responses to question 1

Three groupings in the words were identified by factor analysis. “Insist”, “command”, “order” and “manage” were separated from “suggest” and “advise”. The word “direct” was also separated as a group. There was a significant correlation of the word “insist” with computer access scores (r=0.34, p=0.016). Responders indicated a clear preference for systems that advised, suggested or guided, rather than controlled.

Question 2: What clinical tasks should a computer system perform?

In this question choices were given relevant to the types of functional tasks a computer based decision support system might make. Generally the choices would be considered to be general questions of design philosophy or specific areas of overall system design. Table 2 shows the rank order of the responses to questions. Two questions showed differences in responses between the medical and non-medical group. Again, the non-medical group ranked the responses to choices more highly than did their medical colleagues. One statistically significant difference was seen between the anesthesia and critical care groups, with the latter giving a higher ranking to choices concerned with offering advice when requested. Factor analysis found four groups. In group I responders identified offering advice to juniors and compensation for lack of experience as being logically linked, though they showed a certain ambivalence in the ranking of these two responses.

Figure 2

Table 2: The results for the responses to question 2

A second grouping consisted of the choice concerned with offering advice when asked combined with that compensating for failures of concentration. Both these responses were given reasonably high overall ranks. However, on the face of it their grouping by responders is somewhat illogical. How can a system compensate for failures of concentration or vigilance and yet still only offer advice when asked? As will be seen in the other results, responders tended to prefer better alarm systems that would compensate for concentration failure but would not offer detailed advice on line. This strategy would make it possible to compensate for failure of vigilance and offer advice off line. The question on lock out received a very low rank, although the very high statistical difference indicated between the medical and non-medical groups suggests that non-medical staff are prepared to restrict the actions of clinicians more than are the clinicians themselves. Taking direct action was given a low rank and seen as a different type of feature design. No correlation with any demographic data and responses was demonstrated.

Question 3: What type of information should a decision support system give?

This question was aimed at eliciting details of how information gathered by a decision support system should be used. Table 3 shows the responses in overall rank order. The highest rank was given to the combination of offering advice with alarms. This was significantly higher than choices limited to advice only. No significant differences with any choice and between groups was demonstrated and only one correlation between the choice “give detailed procedure specific advice” was found with computer literacy score (r = -0.38 and p = 0.004).

Figure 3

Table 3: Results for responses to question 3

Factor analysis indicated that 7 choices could be segmented into four groups. The choices “Offer advice and alarms” and “Give general advice only” were only seen as distinctly different to other choices. As would be expected, responders grouped two of the choices concerned with recording performance details. However, again a certain ambivalence was shown in the ranking of the two choices grouped. It appeared that staff were almost evenly divided between those who would ask that no performance details of the individual operator were ever recorded and that performance should be recorded for junior staff for the perusal of senior staff. The fact that the responders did not group these two questions with the choice concerning a private performance record probably indicates that it is “publication” of the performance that is really the issue of concern. The last group consisted of three responses about how guidance information should be presented. Responders showed a marginal preference for step by step guidance offered off-line at the request of the operator.

Question 4: What sort of expert information should be used in a decision support system?

Present computer technologies offer novel ways of obtaining expert information. It would be possible to design a decision support system that would gather its own information in a structured fashion as it was used and make that the basis of advice. Such a system would be the antithesis of a system where the rules were laid down by a panel of experts or a single senior clinician inside a hospital. Question 4 was aimed at detecting differences in attitudes to different strategies for information gathering.

Figure 4

Table 4 : Results for the responses to question 4

Table 4 shows that three responses reached statistically significant difference between the medical and non-medical group with the non-medical group again prepared to give higher aproval. The overall rankings indicated that people preferred opinions and information to be based on as broad a basis as was possible. They were very much less keen on information derived from an individual, even when they themselves were that individual. A relatively high ranking was given to inclusion of economic criteria in the expert information used by the system. Factor analysis indicated four groups. One was concerned with where data would be gathered from if it was the system that gathered the data. A second was concerned with the authority giving the information. A third connected questions of economic criteria with information about individual practice, presumably showing the underlying feeling about how economic judgment should be fed back to individuals. Lastly responders differentiated between international advice and advice from UK based sources.

Question 5: How much control should a Decision Support System have?

This question describes choices of design for difference alarms, e.g. general or condition specific with different methods of alarm (sounds, lights, etc). Different combinations of different types were presented as choices.

Figure 5

Table 5: Results of responses to question 5

Table 5 shows the overall ranking to responses. Choices suggesting high levels of automatic control were ranked poorly. Response to the choice “Set off a general audible alarm and put up a condition specific visual alarm” showed a significant statistical difference with the general anesthesia and critical care groups. The critical care group were very much more against the strategy than were the anesthesia group. This response was to be expected, as general audible alarms are perceived as being an annoying problem in critical care. Factor analysis indicated that responders identified three distinct issues. Responses were grouped in accordance with combinations of alarms. Responders considered that the details of audio alarm systems were important. They also identified the ability to change the patient’s treatment as a distinctly different feature. The idea of a private audible alarm was seen as being a separate strategy as well scoring reasonably high on overall ranking. Two significant correlations with computer literacy score and choice were demonstrated. The choice “Set off a non-specific audible alarm heard by other staff and patients” gave r=-0.27, p=0.04, and the choice “Change the patients treatment if alarm is not cancelled and the vital signs show an unequivocally dangerous condition”, gave r=-0.29, p=0.04. A significant correlation with computer access scores was demonstrated for the choice “Under no circumstances change the patients treatment automatically” (r=-0.41, p=0.002). The overall responses to question 5 indicate that staff have very subtle preferences for alarm types and combinations which it would be as well for designers to take into consideration when designing physiological measurement equipment.

Question 6: How should a computer decision support system communicate with the clinician?

This question was aimed at obtaining more information about detailed choices between methods of communication that were not predominantly alarm based, though some alarm related choices were given for comparison purposes. The results are summarised in Table 6. The second highest rank given to the choice that corresponds to the Paterson sounds alarm strategy was negatively correlated with computer literacy (r=-0.33, p=0.019). In other words the more computer literate the responders the less they liked the idea of structured alarm sounds as recommended in international standards. Factor analysis found four groups: offering graphical animated displays, giving instructions using flow diagrams, having spoken alarms and providing extra physiological measurement displays. The overall rankings within these groups indicate that spoken alarms have a low acceptability. Flow diagrams appeared to be the most popular communication method.

Figure 6

Table 6: Results of responses to question 6

* In this case the critical care specialist group ranked this choice significantly below that of the general anesthesia specialists

Offering of extra physiological instrument displays received overall high ranking. This is somewhat at odds with indications from human factors analysis which would suggest that there is more than enough physiological information and numbers inside either anesthesia or intensive care at the present time. Indeed, extra physiological instruments and numbers might be considered to be potentially detrimental to good human factors performance and safety.