G Duke, B Haas, C Hawkins, S Northam
advanced practice nurses, critique, evidence-based nursing practice, nursing research, research
G Duke, B Haas, C Hawkins, S Northam. The APRN Critique Guide Used To Advance Evidence-Based Practice. The Internet Journal of Advanced Nursing Practice. 2008 Volume 10 Number 2.
Reading and critiquing research are essential skills for advanced practice nurses. This article builds upon a critique guide published in volume 10, issue 1 of this journal. The purpose of this companion article is to facilitate reading, understanding, and possible implementation of research by providing nurses with an example of reading and critiquing research. The critique presented was conducted by one of the authors who is an advanced practice nurse. The process is demonstrated using a published nursing research article, and each section is discussed. Sections of the critique are scored and summed to create a composite score useful in overall evaluation of the article. This method of critiquing research has been used by faculty members in a graduate nursing program, and practicing nurses have evaluated the guide as helpful in breaking the critique process down into manageable sections.
This article builds on the critique guide presented in ‘Promoting Advanced Practice APRNS’ Buy-in for Evidence’ article published in volume 10, number 1. The goal of these two companion articles is to facilitate reading and understanding of research by providing nurses with a brief guide to improve their skills. This article includes an actual critique done by one advanced practice registered nurse (APRN) to illustrate use of the guide. The critique guide and the APRN’s critique are noted in bold. Discussion of the critique is non-bolded. The critique is based on research conducted and published by Schroeder and Pridham concerning the development of relationship competencies through guided participation for mothers of preterm infants.(1)
1. Identify the problem. Briefly state in your own words: (5/6 points)
Parent education in Neonatal Intensive Care Units (NICU) has focused on developing skills for specific tasks for care giving, letting the parent get acquainted with the infant, and supporting understanding of the infant as they develop. Little literature has been published on how to help clinicians assist parents in developing a relationship with their preterm infant. The focus is developing relationship competencies with the mother by caring for the preterm infant through guided participation (GP).
The initial information on the critique guide simply notes the article in a reference format. Then the actual critique begins with a brief statement of the problem and a scoring of 5 out of 6 points indicating the APRN thought the problem discussed within the article was good but not perfect. The APRN noted the gap filled by the research and succinctly noted the problem.
2.Review of the literature. (4/6 points)
At the beginning of the article, did the author review other research studies that have been done on this topic?
Yes, references were made to studies by Blackburn(2) and Robinson, Pirak, and Morrell(3) that demonstrated that the ability of the mother to take on the role of parent in the NICU likely affects home care. Additionally, Brazy, Anderson, Becker, and Becker(4) indicated wide spread recognition of education as critical to the parenting of premature infants and suggested that programs to support parents are needed. Studies by Brazy et al.(4) and Melnyk, Feinstein, and Fairbanks(5) that focused on mothers’ coping and knowledge of infant development and behavior were also addressed. Finally, the author stated that little literature exists to guide clinicians in helping parents build a relationship with the infant, including care-giving competencies and competencies in relating to the infant.
Did it appear that ‘classics’ had been reviewed?
No classic theories or studies were identified, but a focus was present on the basic concepts of the study. A citation to Bowlby(6) described a working model that focuses on constructing an attachment relationship. Rogoff(7, 8) was cited with regard to the guided participation process, stating a novice in a socially important activity develops competency through participation.
Were the majority of references recent? (# <5 years / total # references)
Only 21%, or 9 of the 43, articles were recent (<5 years) when the article was published. Four articles were 5 years old at the time of publication, leaving only 11.6% less than 5 years old in 2007.
The APRN assigned 4 of 6 points for the second section, indicating that the review of literature (ROL) had strong points and some weaknesses. Specifically, the author of the reviewed work noted studies that establish the known research related to mother-infant relationships and interventions for improving those relationships. In identifying these studies, the author was able to establish a gap in the literature. The primary weaknesses addressed in the critique are the outdated references and the lack of ‘classics’ reviewed. Regarding the references, a lack of recent literature in the field may explain why 79% of the articles cited were more than five years old at the time of publication. However, because this possibility was not expressly addressed, the reader would likely conclude that more current scientific research was available to help strengthen the knowledge base of the study concepts. Finally, no classic sources were identified in the ROL which is not as essential as identifying the relevant and recent research. Of primary importance is the fact that the author identified the gaps in the literature.
3. Theoretical/Conceptual Framework: (5/6 points)
Is it clear how this research fits into nursing?
As the clinicians who spend the most time with the infants and mothers, nurses are available to teach, guide, and foster relationship and caregiving competencies. Thus, nursing staff potentially have the greatest impact on the relationship developed between the mother and infant.
Does the literature review provide a clear, sound groundwork for the study?
The groundwork was vague in areas, and the APRN had difficulty seeing where the researcher was headed without rereading multiple times to follow the concepts. However, the literature explains the difference between standard teaching, in which the clinician transmits information to the mother, and guided participation, in which the clinician focuses on building the mother’s parenting skills. As the goal of the study was to examine the effects of guided participation versus standard care teaching, the literature review did effectively explain the basic components of the study.
As noted by the APRN reviewer, studying the effects of guided participation in relationship competencies is clearly relevant to nursing practice. The authors identified how the proposed intervention of guided participation may improve competencies for developing healthy maternal-infant relationships. However, the researchers used numerous other conceptual terms that seem to “muddy” the direction of this study. Ideally, the reader should be able to link the concepts in a theoretical framework to the variables in the research questions/hypotheses, and as the APRN points out, this connection is not clear in the article.
4. Formulation of research question(s) or hypothesis(es): (6/6 points)
Are there research questions? If yes, write it (them) here:
Are there hypotheses? If yes, write it (them) here:
Hypothesis 1: “mothers experiencing GP will demonstrate greater attunement and adaptability of their expectations and intentions concerning their relationship with their infant”(1, p. 364)
Hypothesis 2: “mothers experiencing GP will demonstrate greater gain in this attunement and adaptability”(1, p. 364)
Hypothesis 3: “mothers experiencing GP will demonstrate greater competency in being with the infant and in knowing and relating to the infant as a person” (1, p. 364)
Are they null, non-directional or one-way?
Directional or One-Way. The hypotheses stated that GP will demonstrate greater attunement and adaptability of their expectations and intentions; greater gain in this attunement and adaptability; and greater competency in being with the infant and in knowing and relating to the infant as a person. The hypotheses stated a positive relationship between outcomes and GP versus standard care teaching.
(Note: All quasi-experimental & experimental research MUST have hypotheses, even if one needs to figure out what the hypotheses are based upon the stated problem.)
The APRN assigned a perfect score to this section, as was appropriate given the experimental design of the study. Remember that research questions can be easily implied with a clearly stated hypothesis. Directional hypotheses were also appropriate given the evidence provided in the ROL.
5. Selecting the research design: (6/6 points)
Underline the terms that apply to the research design of this study:
1. Cross-sectional OR
2. Descriptive OR Survey OR Quasi-Exp. OR
Design stated to be part of a larger study using spilt-plot factorial repeated measures design.
The design was between two groups. Polit and Beck(9) state features of factorial design include “experimental manipulation of more than one independent variable; permits a test of main effects for each manipulated variable and interaction effects for combinations of manipulated variables” (p. 261) .
As the APRN noted, the authors earned a perfect score for the design, because a longitudinal model was appropriate for the purpose of the study and met all of the conditions for an experimental study: an intentional intervention, randomization to groups, and use of a control group. Having a longitudinal design permitted data collection before and after the intervention, thus allowing changes to be observed that were due to the intervention being tested.
6. Specifying the population: (5/6 points)
Who is the population? (Include some demographics in your answer)
(Note: The population is not the same as the sample. The sample is drawn from the population. Therefore, the answer should not include a number)
The population was mothers from two Level 3 neonatal intensive care units in the Midwestern United States. Mothers of prematurely born, medically stable, very low-birth weight infants were eligible for the study. The mothers were at least 18 years old and were able to read and write in English. Infants were born at 28 weeks gestational age and had appropriate weight for gestational age. Infants included predominately Caucasian males and females, with some inclusion of Asian, African American, Latino, and Native American races. The study included both breast and bottle fed infants. Medical diagnoses included anemia, apnea, jaundice, and retinopathy all related to prematurity, feeding problems and respiratory distress syndrome. Enrollment for the study occurred for 12 months.
The authors wrote a thorough description of the population and in so doing, demonstrated the appropriateness of the population characteristics to the potential benefit of the findings for this group.
7. Operationalizing and measuring the research variables: (6/8 points)
List each variable here and explain how they measured (operationalized) each variable. Then critique the validity of each variable.
What is/are the independent variable(s)?
Guided Participation and Standard Care Teaching
What is/are the dependent variable(s)?
Working Model Relationship Score (hypothesis 1 & 2) and Relationship Competencies Assessment (hypothesis 3).
This section was rated as 6 out of 8 possible points. The APRN noted that the authors demonstrated clarity in explaining how all of the variables were operationalized, or measured, in the study. Furthermore, the authors included samples of tool items, which allows the reader to have an idea of the type of questions used and can be helpful in future research. Points were deducted because no instrument validity measures were reported.
8. Reliability: (All of the following questions pertain to reliability) (8/8 points)
a)What Cronbach’s Alpha was reported for the instrument? (= Internal consistency). What do you think of the Cronbach’s Alpha?
b) The Working Model Relationship Score (Hhypotheses 1 and 2) was .86, .89, and .89 which reflected high internal consistency. The subparts of the instrument are reliable.
c) High internal consistency was also demonstrated in the Relationship
d) Competencies Assessment (Hypothesis 3) with coefficients of .81 to 1.0.
e) The subparts of the instrument are reliable.
f)Was a test-retest r reported? (=Stability) If yes, what was the # ___and what do you think of that?
h) Was any inter-rater r reported? (= Consistency) If yes, what was the #____ and what do you think of that?
i) Working Model Relationship Score (hypothesis 1 & 2), intercoder reliability was used for 10 of 48 interviews with 80% agreement. A consensus code was used. Coefficients of .80 are highly desirable according to Polit and Beck(9).
j) Relationship Competencies Assessment (hypothesis 3) was used through concurrent observations of the research nurse and one APRN for 10 or 96 learning sessions. Agreement was 89%. Coefficients of .80 are highly desirable according to Polit and Beck(9).
k) If another researcher
l) Based on Cronbach’s alpha showing high internal consistency and the high inter-rater reliability, a good chance exists that the researcher would get the same results.
m) The researcher:
n) a. used tools with high internal consistency (Cronbach’s alpha).
o) b. assessed the inter-rater reliability of each test through comparisons of coding and observations of interactions.
p) c. controlled external factors including age of infant, interviewers/data collectors, and similar maternal demographics which impact validity.
q) d. protocol integrity was assessed through observation of learning sessions.
r) The full 8 points were awarded for the reliability section because the authors clearly addressed the issues and the study used controls that enhanced reliability.
9. Pilot: Was a pilot done? (2/2 points)
Yes, a pilot was conducted. Three mothers who participated in the pilot had responses used to determine coding categories.
A perfect score was appropriate since a pilot was conducted.
10. Sample: (3/6 points)
What is the sample size? ___
What do you think of that sample size? (Support your answer)
This number is a small sample and can result in less accurate estimates than larger ones. The population is basically homogeneous which may make the small sample more adequate. More analysis based on race and previous parenting could occur with a larger sample size. The researcher stated that a lack of available literature was found to determine an accurate estimate of effect size.
How was the sample selected? (
The process was non-random selection but random assignment. The sample size was selected based on the number of families available in the one year enrollment period from two Level III NICUs in Midwest hospitals. Forty- three (43) mothers were asked to participate with 27 refusing. Ten (10) did not have transportation or family issues preventing frequent visitation, 11 had infants who were medically unstable or died prior to recruitment, three infants were transported to other facilities, and 3 declined participation. The participants were randomly assigned to groups.
Is the sample
The sample is not representative of the NICU population. The study had a predominately white population and only focused on two hospitals in the Midwest. Additionally, low income mothers who did not have transportation may not have been part of the study.
Only 50% of the points possible were awarded for several reasons. Although a power analysis was difficult to calculate beforehand because the effect size was not known, a post hoc analysis could have been done and, depending on the results, given the reader more confidence in the findings. The concern that the sample lacks minority representation, which may or may not reflect the population from which the sample was drawn, was correctly identified by the APRN. Approximately 10% of the eligible population was unable to participate due to lack of transportation, which may have excluded lower income eligible participants.
11. Data collection: (5/6 points)
How were the data collected? (e.g., Were surveys used? Was it self-report or were subjects observed?)
Data were collected by self report, observation, audio recorded interview and infant’s medical record review.
What were the circumstances of data collection?
The researcher collected all data. Baseline data were obtained from self report, observation and audio-recorded interview when the infant was 29 weeks postconceptual age. Self report included demographic and attribute information. Observation data were from the mother’s verbal and behavioral expressions of competencies with the infant. The infant’s birth and medical history were obtained from the medical record. Two interactions were video-taped and were used a week later during an interview concerning relationship with the infant. Flexibility in interviewing was made to meet the mother’s life circumstance needs including using the mother’s home for interview in six cases.
How many researchers were present?
One researcher interacting for data collection.
The assigned score was appropriate, and probably would have been perfect if the procedures were not so complex. The authors were very detailed in their explanation of who did what, when, where, and how.
12. Limitations: (5/6 points)
List and evaluate the controls used in the design and data collection:
Randomization: A coin flip was used to assign patients to groups.
Repeated Measures: Used in assessing the mothers during baseline, at 33 weeks post conception age (PCA) and 35 weeks PCA.
Crossover: Not applicable
Homogeneity: Used in determining who would participate in the study. Mothers had to be able to read and write English and had to have transportation to come to the NICU, which limited the people who could participate. Also infants were born at 28 weeks PCA or less.
Blocking/Stratification: Not used.
Matching: Not used. Use of this method might have reduced the incidence of first time mothers primarily being in one group.
Comment on each of the following validity threats:
History: Some infants being discharged prior to the end of the study was a threat because of possible influence of the mother infant interaction. The researcher did not identify the participants from each group who were discharged.
Maturation: Some of the mothers going home before the end of the study were a threat because the interaction with the infant could be impacted.
Mortality: Did not occur in this study. All participants who started completed the study.
Self-selection: Not applicable, groups were randomly assigned.
Testing: A baseline assessment was done which may impact the subsequent two assessments. Each group was given a baseline test so the impact should be comparable.
Instrumentation: The instrumentation remained the same through the study.
Hawthorne Effect: The mothers knew they were being observed which could have impacted the interaction with the infants and therefore been a threat.
The near perfect score assigned was appropriate since the authors were attuned to details of control and potential threats to study validity. The only detail missing was that of homogeneity between the intervention and control groups. Though they did not begin the study, one has to wonder how different the results might have been if the 10% of the eligible sample were able to have transportation and participate.
13. Preparing data for analysis: (4/4 points)
What tables or graphs helped you understand the data?
Table 5 (Hypotheses 1 and 2): showed the mean and standard deviation comparing the GP and SCT groups at baseline, 33 weeks and 35 weeks.
Figures 1 (Hypothesis 1) and Figure 2 (Hypothesis 3): graphically show the differences between the GP and standard caregiving teaching groups at the different assessment periods and compares the two.
Table 6 (Hypothesis 3): shows the progression of each of the two groups and compares based on mean score and standard deviation from baseline through each learning session.
What common terms like `means' or percentages were used?
Mean and Standard Deviation
Was the data understandable?
Yes, a higher mean score indicated greater attunement and adaptability.
Again, all of the points were assigned for the data preparation section. The presentation of means and standard deviations indicated the data were continuous and interval level. The variable level was noted here to help the reader think about the statistical tests appropriate for analyzing the hypothesis. The interval level data indicated that parametric statistical testing was appropriate if the assumptions of the parametric test are met.
14. Analyzing the data: (6/6 points)
What statistical technique was used to answer the key research question(s)?
The effect of the intervention over time by group was evaluated using ANOVA which is appropriate to examine differences in three or more groups.
What was the pre-set alpha? ___
Were the results statistically
Hypothesis 1: The higher scores of the GP group in attunement and adaptability of expectations were supported. WMRS for GP group:
Hypothesis 2: Greater gains in development of attunement and adaptability were not supported.
Hypothesis 3: Both groups increased over time with demonstrating greater competency in being with the infant and knowing and relating to the infant as a person. All mothers had a significant change over time. Relationship Competency Assessment for both groups over time:
The preset alpha was .05 and each hypothesis and the results of testing using ANOVA were appropriately related.
15. Interpreting the results: (2/4 points)
Tables were helpful in visualizing the results. Each hypothesis was broken down. Hypothesis 1 and Hypothesis 3 has a clear description of the results. Hypothesis 2 results were more difficult to determine.
Did the results make sense?
The data were clear that the GP group had higher relationship competencies in interacting with their infant. The differences between each of the three hypotheses were harder to interpret.
Were the authors’ findings congruent with expectations?
Yes, attunement and adaptability of relationship with the infant would likely be higher with the GP group since the focus was interaction with the infant. Traditional teaching toward discharge care does not focus on relationship competencies.
The APRN felt that the authors were unsuccessful at clarifying how these findings apply to clinicians. Theoretical framework and procedures are somewhat complex and potentially difficult for some clinicians to grasp clearly. As a result, data analysis and interpretation of findings may also be unclear.
16. Communicating the findings: (6/6 points)
Who should be told about these results?
Results should be conveyed to NICU care teams, with a focus on the NICU and pediatric nurses, who work with preterm infants and families. This information could be used to develop programs to enhance the relationship competencies of the mother and be incorporated with daily care of the infant in the NICU.
Is this research relevant to practice?
Developing positive parenting competencies should be a goal in the care of families of NICU babies. These babies’ special needs can impair interaction between the infant and parent. By developing relationship competencies while developing care competencies, positive parenting could be assisted.
Is it safe to use this research in practice?
More description on the exact program and interaction used to receive the results is needed. The article did not cover the specifics of GP, which is necessary to implement the findings of the intervention. Further, inappropriate interaction with parents could result in harm to the relationship between the mother and infant. Therefore, the intervention is not ready for broad scale implementation.
If appropriate, formulate a nursing intervention based on this research. Be sure the intervention is based upon the research and is specific. Not all research is ready to be used in practice.
17. Critiquing the ethics of this research: (8/8 points)
IRB/HSRB: The study was approved by the human subjects review boards of the participating academic institutions and hospitals.
Informed consent: The researcher stated informed written consent was obtained.
Risk versus Benefit: The researcher did not specifically address risk versus benefit. No care was withheld from either group. Standard care teaching was done with one group without a specific focus on relationship competencies. An argument could be made that the standard care teaching group did not have the advantage of relationship competency building, which could impact the mother-infant relationship. No significant risk was introduced and benefit might be obtained by studying this subject in relation to positive parenting.
Qualified researchers: Both researchers are registered nurses. One researcher is a PhD prepared pediatric clinical nurse specialist and the other is a PhD prepared professor at a major university-based school of nursing.
Anonymity/Confidentiality: Patients cannot be linked to the data through this article. No patients were identified, hospitals were not identified, and the time of the data collection was not identified. Information was reported in aggregate.
Full points were given to this section by the APRN. The researchers were careful not to provide details that would identify participants and the study was approved by review boards.
What is the level of evidence?
Level II8 single random controlled trial using an intervention based on formal protocols. One group had standard care teaching and the other had guided participation.
The concept of the GP article was interesting. Improving parenting skills for parents of children in the NICU should be a goal. The focus on the sick infant can influence the interaction with the infant. The strengths include a well designed study with a focus on design validity and reliability of the tools. Ethical components of the study were addressed. A weakness of the study included a small sample size that had limited participation by minorities and may have excluded lower income families. The focus of the study was mothers and did not consider the effect on other family members. The study was difficult to read on first review but was easier to follow after each element was explored. The results of the study were promising, demonstrating that Guided Participation impacts attunement and adaptability in the relationship with the preterm infant. More description of interventions is needed to bring Guided Participation into clinical practice.
Total Score: 86
The APRN summarized the review and derived a score of 86, indicating the research study was well done. This APRN summarized the key points including the strengths and weaknesses of the research. Thus, the step by step critique process ends with the nurse generally reviewing the entire article. The use of the guide to review research reports breaks the process down into small steps and has been evaluated as very helpful by graduate nursing students and practicing nurses for many years. The confidence gained by regular reading and critique of research is essential in building the skills necessary to evaluate studies before incorporating findings into practice.