S Northam, J Lakomy
S Northam, J Lakomy. Promoting Advanced Practice APRNS’ Buy-In For Evidence Based Practice. The Internet Journal of Advanced Nursing Practice. 2008 Volume 10 Number 1.
To promote safety and progress in health care, advanced practice APRNs need to base practice on best evidence. Evidence-based practice remains more an ideal rather than a reality. This article reviews a guide for reading and critiquing quantitative research to help APRNs make clinical decisions based on the appraisal of the evidence for relevance, safety, and applicability to their practice. Quantitative and qualitative data from nurses indicate the critique enhances their understanding of the quality and utility of research. The step-by-step process of doing the critique in small parts builds confidence as each section is completed and fits in with busy APRN time schedules. Finally, scoring the research article on a 100 point scale helps nurses determine the quality of the research. (122 words)
Reading and understanding research is challenging and is cited as a barrier to research application (). Generally novices require more time to read, understand, and critique than experienced readers (), but many find the research critique to be a helpful learning activity (). Lack of familiarity with terms and confidence in ability to understand the findings can make the experience of reading and critiquing research intimidating and frustrating. Thus it is not surprising that few nurses read research; and alarmingly, Wood () recently noted that trepidation, antipathy, and apathy toward research has not improved in the past two decades (). But regular reading to gain insight into best evidence useful for practice is essential if nursing is to meet the goal of improving health care through the implementation of evidence based practice. Thus, strategies to improve the skills of APRNs in reading and understanding research and make the experience less overwhelming, time consuming and frustrating are important. Recent findings have shown that reading research increases critical thinking skills (), thus it benefits not only patients but also nurses. This article reviews a guide devised, used and refined for 12 years by nurses who evaluate it positively. Data are included to support their evaluations. Understanding research is an essential first step towards evidence based practice ().
This guide fills a major gap in critique guides. This guide is short (4 pages), easily printed with space for responses, and contains the first scoring criteria since Duffy's criteria was published in 1985 (). The point allocations for this critique were derived by the authors who are experienced clinicians and researchers. While other critique guides exist, the ones in research texts are generally long, presented as text boxes within chapters and lacks space for readers to include notes as they critique. Polit and Beck (2004) use 138 questions within 18 text boxes to guide readers in evaluating published quantitative research reports. Burns and Grove (2003) include 38 sections with 150 questions: 19 sections with 56 questions in the comprehension guide; 11 sections with 86 questions in the comparison and analysis guide; and 8 questions in the evaluation guide. The Burns and Grove guide includes more directing questions for the comprehension literature review, separates evaluation of variables from measurement and validity assessment, and includes questions about missing study elements. The reader is also guided through the process of article review and critique steps twice rather than once. Both Polit and Beck (2004) and Burns and Grove (2003) recommend that readers summarize study strengths and weaknesses but do not score research nor are they easily used by nurses in practice. Recently Daggett, Harbaugh and Collum (2005) published a single page worksheet devised to teach baccalaureate nursing students to critique research by briefly noting parts of the research. It is utilitarian in its simplicity but is more geared to noting the parts of the research rather than critiquing the study or determining the utility for evidence based practice. Cutcliffe and Ward (2007) argue there is no one best guide to reading and critiquing research that would suit all nurses. Their book reviews various methods of critique, notes the strengths and limitations of each, and ends with a presentation of a new approach used by the Network for Psychiatric Nursing Research (NPNR) Journal club. The NPNR Journal club approach can be used for both qualitative and quantitative research and includes eleven guidelines focusing on a holistic critique. It lacks guidelines in reviewing the scientific method of quantitative research, derives no scoring of the research report but encourages nurses to support a critique of strengths and limitations by fitting it within the context of relevant material.
In many areas of life from kindergarten to delivering advanced patient care, steps are often created that break the task into small parts. Small parts create the impression the whole task is not overwhelming and provide a mental advantage and organization for completion. The critique guide accomplishes that goal of breaking the task of reading and understanding research into small parts.
Throughout the critique guide, point allocations are noted among the steps of research to enable readers to calculate an overall critique score, which has not been done since Duffy's critique in 1985 (). While Duffy's maximum score was 306, the maximum on this guide is 100, a more traditional score. The higher the score, the better the nurse appraises the parts of the research study and evaluates its strengths overall. A rating over 80 would indicate the research has high quality and should be highly considered when the APRN is making a clinical decision in the practice setting. A future article will provide readers with an example of a critique with a high score and utility for implementation in evidence based practice by advance practice nurses.
The major steps of the research process
As simple as it sounds, discerning the research problem is often difficult and may be confused with the purpose. While the purpose of the research may be to identify ways to safely see more patients in a busy clinic setting, the problem may be long wait times to get lab results. Discerning the problem allows for a much more focused way to view how to study, measure, and improve the situation. The problem is the one clearly addressed by the research study rather than an expansive area such as efficient health care delivery. Identifying the problem also lets advanced practice APRNs see if this research study is relevant to problems or issues in their own sphere of influence and practice.
Review of the Literature
Generally the initial narrative of the research article, just after the abstract, lays the groundwork for what is known about the topic. The discussion of existing literature cites prior research and helps the APRN understand what is already known about the problem. Some journals restrict the length of articles and thus the discussion of existing literature may be brief. Classics are well known works in an area. For example, if the research is on dreams and their meaning, it is likely some mention of Freud would be included since he wrote extensively on dreams. Beginning readers may be unfamiliar with classics, but this section serves as a reminder that there is classic research on most issues in nursing and allows the APRN to make a judgment call as to whether or not the review of the literature is adequate to give this study the strength it needs if changes in practice are being considered. Areas or topics where little research exists may explain limited recent publications that are cited within the review of literature or dependence on literature from another discipline.
Just like a long road trip requires a map, a research study needs an organizational structure or framework, commonly called the theoretical framework. Quantitative research involves deductive reasoning flowing from the framework to a testable hypothesis designed to support or refute theory. As the reader contemplates the research problem and framework, it is important to evaluate the reasoning or conclusions that were reached by the researcher. Strong research clearly explains the underlying theory to be tested by the research and sometimes supports the theory with a visual model.
Research Questions and Hypotheses
All quantitative research involves either research questions and/or hypotheses, and all research reporting significance involved statistics to test hypotheses. Frequently the research questions are noted at the end of the article's introduction. A research question is the researcher's underlying probe as to why something is happening (or not happening) in certain situations. It could be as simple as ““Will providing elderly patients with pedometers increase strength and endurance in walking exercises?”” A hypothesis flows from the theoretical framework is more of a speculation about what will happen if certain situations occur. The researcher has more of a burden with hypotheses which must be supported or not. An example of a hypothesis would be ““Providing pedometers to elderly clients will increase the amount and quality of walking by 20% over a six month period.”” Beginning readers may find it helpful to select journals that include a hypothesis heading within the abstract. If the hypothesis is not clearly stated it may take some sleuthing to discern them. A quick check in the results section may report the hypothesis along with the type of statistic used to test it. Knowing the research question or hypothesis will help the APRN understand what to look for in the discussion of findings.
Quantitative research generally progresses along a continuum. When little is known about a phenomenon, researchers may simply aim to describe it and are thus at the descriptive end of the continuum. When variables are known and surveys can be designed or existing surveys can be used to examine relationships among variables, the design is at the survey point, sometimes also called correlation research. Research in which the researcher directly plans the manipulation of an independent variable and measurement of a dependent variable is at the experimental end of the continuum. This distinction is important as APRNs try to decide whether or not to recommend a change in practice based on the appraisal of the evidence. Descriptive and correlation research can show relationships and situations that are similar to those APRNs face each day and can offer some alternatives. However, before implementing evidence based practice, the APRN must appraise the evidence for soundness, evaluate the sample size was adequate, conclude the findings are strong enough to warrant a consideration of changing the ways things are done, and determine the proposed intervention is congruent with patient preferences. Obviously, many studies and experts would be consulted before changes were made that affect patient and nurse safety, but all change begins with knowledge and understanding of alternative ways of doing things. Quasi-experimental means just what it sounds like, ““almost, but not quite, experimental.”” When research is done on people, sometimes all of the variables cannot be controlled like they can with lab mice. So, using quasi-experimental does not mean it is not sound research. Whether or not it is quasi-experimental research is dependent on the use of randomization and/or a control group. This section of the critique includes brief questions about the design that help the APRN to understand it by applying common research terms.
Research texts generally restrict the population to the group from which the sample was drawn. Research rarely uses findings for the exact same population but rather evidence based research efforts are geared to moving research to more widespread, yet appropriate, use. This section is designed to get the APRN to think about the demographics of the population and who could benefit from the findings. Clearly, research done on one or two ethnic or racial groups could hardly be generalized to all. The challenge of this century is to recognize best evidence and improve practice through the appropriate generalization of research.
In order to determine the usefulness and appropriateness of applying this research in the clinical setting, the APRN must determine what is being tested. These variables must be defined. One person might describe ““lifting risk”” as the chance of dropping a patient on the floor; another might describe it as the chance the APRN will get a back injury. Both are essentially right, but the reader must know what is being discussed, tested and recommended; therefore, quantitative research involves both conceptual and operational definitions. The conceptual definitions are the ones mentioned previously; the operational definition is how the researcher measured it, such as a patient fall risk scale or a back injury scale. It is up to the research consumer to decide if the measurement of the variable makes sense and truly reflects the concept of interest. This discussion pertains to construct validity. APRNs can often discern the variables by referring back to the research question and / or hypothesis. The title also generally includes the key variables. The table within the critique provides a place to list the name of the variable (column 1), then note how it was measured or operationalized (column 2), and if that method of measuring the variable is valid. More than yes or no answers are necessary and require critical thinking. Actual surveys are not included in publications but sometimes sample survey items are included. The APRN can evaluate the clarity and appropriateness for the population of sample items. The total number of survey items can also be noted. More items are generally felt to improve the ability to capture the concept, but certainly expansive, time consuming surveys are likely to undermine completion due to fatigue. Evaluating validity can be difficult, but practice increases the APRN’s skill.
The term reliability has the same basic meaning in research that it does in other areas such as employment, appliances, and automobiles, among other examples. But the myriad of types of reliability, particularly the common internal consistency, takes the term beyond the general understanding. This section in the critique guide was designed to narrow down reliability to the three types and includes a section for readers to note any actions taken to enhance reliability. It is generally the second most challenging part of the critique, after the validity section.
Researchers often do a dry run of a research study to work out any kinks before undertaking a large scale, expensive study. The pilot may simply involve testing surveys using a small group to discern item clarity and the time required to complete them. Pilots are also helpful when effect sizes are unknown and initial data can assist in examining for an effect size to determine the sample required in a larger study to discern the effect.
The number of individuals who actually participated and provided data are considered the sample and is generally included in every research study. When surveys are used, individuals may elect to answer some items but not others and this explains when the n (sample size notation) varies within tables or the article narrative. Randomization may occur in either selection or assignment. Randomized controlled designs (RCT) are the classic experimental research design and involve random assignment into groups. Sample demographics, often presented in a table, can be compared to the demographics of the population to evaluate if the sample is representative.
The actual steps involved in gathering data are generally addressed very briefly within the research article, primarily to save space. The reader may be unable to discern who collected the data or the circumstances of data collection. Acknowledgements at the beginning or end of the article may note the names of individuals who gathered data but all authors may not have actively participated in the data collection. As much information as possible can be included in this section but incomplete insight into the who, what, when, where and how is common.
The limitations section primarily pertains to internal and external threats to validity. Discerning threats helps to evaluate rival hypotheses, and alternate explanations for changes in the dependent variable other than the independent variable. Controls are any actions taken by the researcher to understand or minimize the threats to validity.
Preparing Data for Analysis
This section was devised as a step before considering the statistical analysis. The variables listed here should directly mirror each variable listed in section 7 of the critique guide where the variables, their measurement and validity were presented. Now the goal is to think of what the data will yield in terms of a number and whether an average is feasible. Non-parametric statistics are used when no average score is possible or meaningful such as with demographic variables of race or marital status. Parametric statistics are used when an average score is possible and meaningful with the caveat that other assumptions for each parametric statistic must also be met. The questions within this section are designed to increase the APRN’s confidence in reading statistical reports.
Many research reports include a variety of statistics from descriptive statistics used to report demographics to inferential statistics used to test hypotheses. At this point in the critique the APRN should refer back to the hypothesis and look for reported significant or non-significant findings. The pre-set alpha is the level of error the researcher felt reasonable before beginning the research and is often omitted from the research report. Since a 5% chance of error is common, any reported p values less than .05 that are termed significant (sometimes using an asterisk) imply a preset alpha of 0.05. Sometimes notations under tables help to discern the preset alpha.
Interpreting the Results
The APRN should think about the deductive process involved in quantitative research and determine if the results support the hypothesis and theory and solve the problem. It would be appropriate at this point to go back through the critique and put the parts together mentally.
Communication of Findings
Internationally, researchers strive to reduce the time from research to implementation into practice. While all research cannot be easily implemented, this section prompts the APRN to evaluate who should know about the research and whether or not implementation is appropriate. The actions should clearly be derived from the research.
The four basic tenets of research ethics are provided for evaluation and comment. Any reference to grant funding, particularly US government funding from institutions like the National Institute of Health or the American Cancer Institute, imply IRB approval since it is required at the time of proposal submission. The use of a consent form also implies IRB or HSRB approval.
Risks are rarely enumerated within research article reports so the reader must consider the risks and weigh them against the benefits.
The qualifications of the researchers are determined by noting the educational and professional credential notations after the author names or within a footnote about the authors.
Anonymity exists if the researcher never saw the research subject. More often anonymity did not exist within the research study but confidentiality was protected.
Interpreting the Critiquing Score
Once the APRN has completed the critique guide, a score is obtained. If the score is less than 70, no change is recommended for practice. If the score is between 71 – 79, the APRN should utilize this research cautiously in the practice setting. If the score is 80 or greater, the APRN is recommended to include the research study into the clinical decision process for evidence based practice.
Nurses Evaluation of the Critique
This critique guide was devised to help APRNs read and evaluate research. It has been very positively evaluated and APRNs suggested this article as a way to share it. To gather some recent evaluation data, an anonymous online evaluation was completed by APRNs (N=12), 98% strongly agreed and 8% agreed that the guide assisted their understanding of research. Qualitative comments included: 1) I enjoyed the critiques--they changed my way of reading professional articles and studies. They made me think a little more critically; 2) It has helped me to become more comfortable reviewing research and discussing it with others. Information I have seen before now makes more sense to me 3) Even though I started out fearing research, writing the critiques made me understand the process sooooo much better. Now, I look for articles to see if they are as great as they say they are. What a great learning experience.
Quantitative research involves a deductive process outlined within this critique. The goal is to break the process down into small parts with questions to guide APRNs in doing critical appraisal of the evidence. Even when a section or two are challenging, the overall ease of completion fosters the understanding and confidence that are essential for APRNs striving to improve their skills in reading and using best evidence in practice.
Special thanks to Dr. Lynn Wieck for editorial input and Dr. Cheryl Beck for review and input.