Introduction

Out-of-hospital cardiac arrests (OHCA) remain a public health crisis, with over 350,000 events nationally.¹ In part, this continued problem is the result of the modern opioid epidemic’s third wave, now estimated to cause over 100,000 deaths annually in the USA alone.² Through their effects on the central nervous system, opioids cause respiratory depression, which, in excess, can cascade into cardiopulmonary arrest. Regardless of the cause, bystander cardiopulmonary resuscitation (CPR) is among the most significant means of improving OHCA survival rates.³ Though upwards of 76% of OHCA cases in large metropolitan areas receive bystander CPR, mortality rates remain over 80%, likely owing to under a 3% rate of civilian CPR certification.^1,4 Given the low rate and high necessity of CPR-trained USA citizens, any immediately available and easy-to-use CPR educational tool could be lifesaving in countless cases of cardiopulmonary arrest.

There are numerous CPR training tools and smartphone apps that aim to pre-train users in CPR skills and knowledge; however, few follow peer-reviewed protocols.⁵ Further, we find no evidence of a smartphone app that was designed to direct an untrained user through the steps of CPR in real-time during an active OHCA with the efficiency needed to sustain life. Such applications exist internationally and have demonstrated effectiveness but are unavailable within USA app stores.⁶ To fill this need, we have created a free, publicly downloadable smartphone application that provides CPR instructions meeting such criteria. The application also instructs users in the proper requisition and use of intranasal naloxone, a step not conventionally part of CPR, but which recent American Heart Association (AHA) publications have reported to be meritorious.⁷

The effectiveness of this application for real-time CPR guidance is actively being studied by our research group in a large-scale trial. This report focuses on detailing the methodology and merits of integrating medical simulation into ‘beta-testing’ used for app development. At the University of California San Diego School of Medicine Simulation Center, we placed university students and staff in simulated OHCA scenarios with the application as a guide. We recorded both clinically relevant objective measures of CPR success and subjective participant feedback. This data was used to iteratively refine the application. Herein, we present the methodology and results obtained throughout ‘beta-testing’ to demonstrate the merits of this approach and expected effectiveness of the ‘Rescue Me CPR!’ smartphone application.

Methods:

The smartphone application was developed in both Kotlin for Android and Swift for iOS. The design stepped through the AHA’s recommended Basic Life Support (BLS) protocol by providing each step on a distinct app view. The design focus was simplicity over aesthetic appeal so that information could be presented to an untrained bystander in an immediately usable form. The initial application workflow is displayed in Figure 1. Written instructions were supplemented with audio reading the instructions and brief-looping videos in GIF format of each correctly performed CPR step. Transitions between views were accomplished with buttons answering basic questions in the affirmative or negative.

Figure 1

Flowchart of ‘Rescue Me CPR!’ app views. The initial version of the application (left) went through three revisions in response to subject feedback and CPR performance to generate the app view arrangement of version 4 (right), the first published version.

University of California, San Diego undergraduate students without CPR training were recruited as our testing group. Medical staff with CPR certification from the University of California San Diego medical system were recruited as our control group. Testing group subjects were provided with an Android smartphone with the experimental version of the Android app loaded onto it and informed of the app’s existence. Control group subjects were provided no app. Next, subjects were read a prewritten prompt describing that they had just found an acquaintance with a known opioid use disorder unconscious at a social gathering, instructed that they would need to perform BLS, and moved to a room containing a CPR training mannequin (Adult Series 2000 CPR Mannequin, Prestan) on the floor (Figure 2). A timer was started as subjects entered the room. The mannequin was sterilized between each use using bleach germicidal wipes. Additionally, as a precaution against COVID-19, a new plastic barrier was placed over the mannequin’s mouth for each subject. The intermediate-fidelity mannequin in use reported whether appropriate compression rates and depths were being achieved but the COVID-19 precautions obstructed its airflow sensor, limiting data collection on adequate breath volume. If subjects physically felt for a pulse, air movement from the mannequin’s mouth, or chest rise on the mannequin, they were verbally informed of what the respective vital sign was at the time by the test proctor. After performing the first cycle of CPR, if requested by the participant, they were provided an imitation container of nasal Naloxone. After two cycles of CPR, the next time the patient assessed the mannequin’s pulse or breathing they would be informed that they had recovered.  

Figure 2

Experimental setup. Subjects were read a prompt outside the room then allowed to enter. A timer was started as they entered. They did two cycles of CPR on a Prestan CPR mannequin while the study proctor recorded pertinent timepoints and performance metrics.

While each subject performed CPR, we recorded timed and quantitative metrics of CPR success listed in Table 1. At the conclusion of trial, we also performed a ‘debrief’ in which we asked for unprompted feedback pertaining to the application’s usability. Analysis of the objective results and subjective feedback was used to identify numerous areas of improvement in the app’s design. In total, three iterations of the app were assessed during the pilot study, producing version 4, which we published on both the Android and iOS app stores.

Table 1

List of recorded metrics and unprocessed subject data. The above metrics were recorded on each subject. Subject 2 did not start a second cycle of CPR so length of cycle was not calculated (*). Subjects 7 and 12 performed rescue breaths before chest compressions (**). Subject 11 did not survey situation or address patient, skipping to checking vitals (***).

Results

We recruited 4 CPR certified subjects (i.e. medical professionals) and 8 novice subjects. Unprocessed data for each subject is displayed in Table 1. The control group (CPR certified subjects) assessed patient vital signs after 15 ± 1 seconds (average ± 95% confidence interval), requested outside assistance 75% of the time, requested naloxone 0% of the time, requested an AED 25% of the time, and called 911 before starting CPR 75% of the time. They performed the first cycle of chest compressions after 32 ± 3 seconds, the first cycle of rescue breaths after 42 ± 12 seconds, and had CPR cycle lengths (including compressions, rescue breathing, use of naloxone if available, and reassessing vitals) of 48 ± 14 seconds. After CPR they placed the mannequin in a safety position 50% of the time. Compression rates and depth for all four subjects was >120bpm and >2 inches.

For the experimental group (untrained in CPR), we used 4, 2, and 2 subjects with version 1, 2, and 3 of the application during the experimental scenario respectively. Subjects using version 1 assessed patient vitals after 22 ± 9 seconds, requested outside assistance 75% of the time, requested naloxone 50% of the time, requested an AED 50% of the time, and called 911 before starting CPR 100% of the time. They performed the first cycle of chest compressions after 60 ± 20 seconds, the first cycle of rescue breaths after 90± 24 seconds, and had CPR cycle lengths of 58 ± 19 seconds. After CPR, they placed the mannequin in a safety position 0% of the time. Compression rate and depth for all four subjects was 100-120bpm and >2 inches. All four subjects reported some degree of confusion pertaining to the written instructions, though no two reported difficulties with the same text. Two subjects requested app visual improvement and app beautification, one specifically stating the instructional gifs blend in with the app background too much. Two subjects had difficulty using the app due to simultaneous written and audio instructions being presented while the other two reported specifically liking the synchronous presence of both. Lastly, two subjects reported ignoring the naloxone and AED instructions because they did not know what those two items were. Version 1 of the application was updated to clarify every instance of confusing text, have a new background color that contrasted the instructional gifs, have a clearly visible mute button for the audio instructions, and contain brief descriptive information about naloxone and AED’s usefulness in an OHCA situation.

Subjects using version 2 assessed patient vital signs after 22 ± 15 seconds, requested outside assistance 100% of the time, requested an AED 100% of the time, and called 911 before starting CPR 100% of the time. They performed the first cycle of chest compressions after 45 ± 32 seconds, the first cycle of rescue breaths after 77 ± 44 seconds, and had CPR cycle lengths of 48 ± 12 seconds. After CPR, they placed the mannequin in a safety position 100% of the time. Compression rate for both subjects was 100-120bpm and >2 inches. Neither subject had negative feedback about the application’s appearance, text, or audio. However, both subjects had multiple accidental button presses that delayed CPR while they re-navigated the app. They subsequently attributed these accidental presses to small button size and unclear button labeling. Version 2 of the application was updated to include larger buttons with each button label being either “Yes” or “No” in response to a question directly above it in each view.

Subjects using version 3 assessed patient vitals after 18 ± 7 seconds, requested outside assistance 100% of the time, requested an AED 100% of the time, and called 911 before starting CPR 100% of the time. They performed the first cycle of chest compressions after 39 ± 19 seconds, the first cycle of rescue breaths after 67 ± 12 seconds, and had CPR cycle lengths of 80 ± 21 seconds. After CPR they placed the mannequin in a safety position 100% of the time. Compression rate for both subjects was 100-120bpm and >2 inches. Both subjects reported noticing slight discrepancies between written and auditory instructions (due to text being modified in previous revisions but audio not being revised), but no suggestions for improvements were consistent between both subjects. Version 3 of the application was updated to version 4 with improved audio to match the textual instructions, and the patient vitals reassessment instructions were split into two views to reduce complexity. This version of the application was uploaded to both the Android and iOS app stores for public download. The version 4 app workflow is displayed in Figure 1 and screenshots of this application’s views are displayed in Figure 3.

Figure 3

Four key screen views of version 4 of the iOS ‘Rescue Me CPR!’ app. Colored background, grayscale gifs, updated text, the mute button, and enlarged yes/no buttons are among the changes visible in these views.

From the first to the 3rd version, the subjects’ time to compressions was reduced from 60 ± 20 seconds to 39 ± 19 seconds, as compared to 32 ± 3 seconds for the control group (Figure 4). Likewise, time to rescue breaths decreased from 90 ± 24 seconds to 67 ± 12 seconds, as compared to 42 ± 12 seconds for the control group (Figure 4). Iterative app changes also improved rates of participants calling for naloxone, an AED, and 911, as well as the rate of participants putting the mannequin in a safety position after CPR concluded. The changes made to the application did not affect CPR cycle length.

Figure 4

Comparative time to initiating compressions and initiating rescue breaths per app version. Statistically significant differences existed with both metrics between novices using version 1 of the app and CPR certified subjects. As app versions developed both metrics demonstrated a decreasing trend, with time to starting compressions given by novices using version 3 of the app becoming statistically equivalent to CPR certified subjects.

Discussion

With the growing annual rate of OHCA, owed in part to increasing high-potency synthetic opioid overdose rates, there is a growing need for a CPR-capable population.8 With population CPR certification rates remaining below 3% despite increased access to training in the last decade, less than 20% of individuals capable of carrying out effective CPR 6 months after CPR certification training, and less than 12% of CPR certified individuals capable of carrying out effective CPR, novel methodologies of providing effective CPR education are needed.9,10 An in-the-moment CPR tutorial application targeting the ubiquitous smartphone seems an obvious answer, especially if designed with naloxone distribution instructions to combat the increasing rate of opioid-induced OHCA. However, the highly time-sensitive and life-threatening nature of OHCA necessitate any functional application be capable of reliably guiding users in performing CPR within speed and effectiveness at least approximating that of a skilled healthcare professional.

As the primary goal of BLS is maintaining oxygenation and perfusion, time to initiating compressions, quality of compressions (i.e. appropriate rate and depth), and time to initiating rescue breaths are measures of CPR effectiveness. The medical professionals who participated in this study performed their first round of chest compressions and rescue breaths 32 and 42 seconds respectively after seeing the mannequin, with a cycle of compressions, rescue breaths, naloxone/AED use, and vital assessments occurring every 48 seconds. While the depth of their compressions met AHA guidelines, every CPR certified professional we assessed performed compressions at a rate far exceeding 120bpm. Our mannequin did not report exact compression rates, only that they were over the 100-120bpm recommended range. However, calculating compression rate from the time between compressions and rescue breaths described above suggests professionals performed compressions at an average of roughly 180bpm, well in excess of recommendations. While the users of version 1 of the application did not match CPR pacing of CPR certified professionals, users of version 3 approached this pace. They performed the first round of chest compressions and rescue breaths at 39 and 67 seconds respectively. Further, their adherence to AHA guidelines was better than that of a professional, with a higher ratio of version 3 users requesting an AED, calling 911, and doing compressions at both the appropriate rate and depth (100-120bpm). This improvement in performance between versions, as well as tight adherence to AHA guidelines, acts as validation to our approach of applying the medical simulation to app development.

A particularly difficult challenge we came across during application development was varying user opinion on the inclusion of audio guidance. A roughly equivalent number of users reported finding the audio recitation of the app’s text instructions necessary to its effectiveness or  entirely disorienting. Our solution, which is vindicated by the post-fix performance results detailed above for version 2 and 3 of the application, was to have the audio on by default but include a mute button. The variable with the largest persistent difference between app-guided-novice and CPR-certified-medical-professional performance was CPR cycle time. App users’ cycle times did not improve with our iterative application changes and remained 61 ± 19 seconds compared to 48 ± 18 seconds for professionals. While these values technically statistically overlap, we suspect with larger sample sizes they would not. The likely cause for this persistent difference is that, unlike compression or rescue breathing instructions, which require no “interpretation,” or assessments of patient consciousness, which is just a binary conscious/unconscious interpretation that is public knowledge, the assessment of patient breathing and pulse that is performed each cycle requires the participant to integrate complex information, leading to delays as they doubt their own judgment. To address this challenge, our revisions to version 3 of the application included splitting the assessment instructions into multiple views to simplify what was being asked of the participants. In future app updates, we may make vitals assessments less frequent to further reduce this delay and more adhere with AHA guidelines, which recommend vitals assessment every 2 minutes (4 cycles).

Previous studies have assessed the advantages and disadvantages of various modalities of BLS guidance tools.⁶ Smart phone applications published outside the United States have been shown capable of guiding technically sound CPR, at the cost of substantial time delay, an unacceptable cost in a real OHCA scenario.⁶ The ‘Rescue Me CPR!’ app was developed to mitigate this limitation through a series of small ‘scientific trials,’ each designed to obtain relevant CPR performance metrics accompanied by subjective feedback from users regarding what they felt slowed down their use of the application. By using a standardized medical simulation as an unchanging variable, and recording predetermined performance data, as is becoming increasingly routine within medical education, we could rapidly obtain the data needed to iterate the application’s design. The full development of this application as well as the conduct of this study occurred on a scale of weeks to months. We recommend this approach, including publication of the data obtained, to any developers seeking to design medical applications that require certain performance standards prior to publication. Increased transparency throughout the development and testing phase of smartphone app creation serves to collectively prevent multiple development or research teams from facing the same challenges.

There are several limitations to this study that must be considered. Firstly, a byproduct of this study’s focus on rapid iteration is small subject sample size, resulting in high error in every reported value. Objective measures reported within this report should be assessed only in relation to one another, not as absolute measures of either professional or guided-novice CPR performance. Even with smaller sample sizes, statistically significant differences in metrics of CPR performance were observed between users of different versions of the application. Secondly, the population studied in this trial was undergraduate college students at UCSD. There is no guarantee that individuals of differing socioeconomic, educational, or cultural backgrounds will interact with the app in a similar way. Thusly, there may exist other limitations to the usability of the ‘Rescue Me CPR!’ app which were not identified and then iterated on within this study.  Lastly, no smartphone application will ever be a substitute for CPR certification training, as developed by the American Heart Association. Neither this nor any future study should be utilized as justification of such a false equivalency.

Conclusion

We created the ‘Rescue Me CPR!’ smartphone application, a tool designed to allow bystanders who have never been trained in BLS to perform CPR and administer naloxone in an OHCA scenario. To our knowledge it is the only smartphone application available in the United States aiming to provide in-the-moment CPR guidance. The application was developed using simulated medical scenarios and rapid iteration based on both objective metrics from the simulation and user feedback, with design focused towards mitigating the largest limitation of smartphone CPR applications: increased delay before the start of compressions. Users following the final version of the application demonstrated the ability to perform CPR at rates and qualities approaching that of CPR certified medical professionals, while other aspects of BLS were even better. This study serves to demonstrate the methodology and effectiveness of  ‘Rescue Me CPR!’s’ unique development path, which may have applications for other medically related smartphone apps. A full trial of the app using the most recently revised version of the app is currently underway.