HESI APRN Standardized Testing Results Guided by Third Voice
D Martinez-Anderson
Keywords
advanced practice nurse, hesi aprn exam, standardized testing, standardized testing results, third voice
Citation
D Martinez-Anderson. HESI APRN Standardized Testing Results Guided by Third Voice. The Internet Journal of Advanced Nursing Practice. 2021 Volume 18 Number 1.
DOI: 10.5580/IJANP.54937
Abstract
Standardized testing is an established credible and reliable educational assessment tool that measures Advanced Practice Registered Nurse (APRN) pre-professional certification exam preparedness. While the literature is plentiful related to standardized testing use in undergraduate education, limited evidence related to graduate level (APRN) nursing education can be found. Furthermore, no existing literature explores a strong theoretical underpinning that fits the role that standardized test results play in the educational process. Guided by the concepts of the three voices in the Reconceptualization of Vygotsky’s Theory of the Zone of Proximal Development (ZPD) leading to a specified goal outcome (attainment). The first two voices represent the educator, student, respectively, while standardized test results are conceptualized as the third voice. When the HESI APRN Exam is viewed through the conceptual framework of Third Voice, this strategy provides educators a clear plan to utilize testing results to direct remediation for APRN certification success.
Introduction
Standardized testing has become an accepted credible and reliable educational tool1, used as a familiar strategy by nursing educators in assessing core competencies and clinical knowledge in preparation for licensure and certification. (2) Some, like Sullivan (3), believe standardized testing “holds students accountable for their performance and nursing schools accountable for meeting accreditation standards”.3, p.47 Furthermore, standardized tests provide evidence of pedagogical rigor for faculty and gauge knowledge of advanced clinical skills, professional practice competencies for students, and readiness for national certification exam.4,5 The HESI APRN Exam is an evidence-based testing product designed to match the national certification testing blueprint that measures advanced practice nursing core competencies and clinical knowledge in preparation for national nurse practitioner certification.1 Multiple methods of implementation are used for standardized testing, however, Mee & Hallenbeck discovered there are no established standards or guidelines related to standardized testing policies, including use of testing results, in nursing education.6 A conceptual framework underpinning standardized testing can assist nurse educators in implementing evidence-based education.7
Several secured, computerized testing products are available specifically for graduate nursing (APRN) faculty to use, such as Elsevier’s HESI Exit Exam, Advanced Practice Education Associates (APEA) Predictor Exams, Barkley & Associates, Inc. and the National League of Nursing (NLN) End of Program Exam (EPE) and Rosh Review Nurse Practitioner (NP) Comprehensive Exam. To understand the influence standardized testing has on nursing knowledge, conceptual frameworks must be identified and established. A review of the literature was completed to identify the conceptual models currently used to underpin how standardized testing is used in graduate nursing (APRN) educational programs.
Methods
A systematic review was undertaken to identify conceptual models in studies that utilized standardized testing in nursing preparation for licensure and certification. Four evidence sources were searched, including CINAHL complete, Medline complete, Cochrane, and ERIC. Search tems used were computerized educational testing, educational testing, educational assessment, educational policy, exit exam OR exit test. Terms were searched with the Boolean connector “and” producing no results. Using the Boolean connector “or” produced 3,597 articles. The inclusion criteria applied included publication dates from 2010 to 2018, English language which yielded 21 articles. Duplicates were removed after reading titles and abstracts. Twelve articles were identified after applying the inclusionary criteria of implementation of standardized testing, and report of outcomes. A separate search within the Virginia Henderson Global Nursing Online eRepository produced a Sigma Theta Tau presentation by Willson, and the presentation was included as grey literature.8 Thirteen reports are displayed in Table 1, showing conceptual frameworks used, types of nursing programs, findings, and discussions.
|
Citation: author(s), date of publication & title/ Level of evidence |
Conceptual Framework |
Nursing Program Type |
Findings |
Discussion |
|---|---|---|---|---|
|
Willson P, Goodman J, 2015, Standardized Testing to predict APRN credentialing success: What is the science? Level V |
No theory |
Advanced Practice Registered Nurse (APRN) N=35 Graduate nursing program deans and directors Response rate= 26% |
Scores from 700-900 was 100% pass rate. Benchmark criteria is based on evidence presented that <699- scores were 56% pass rate, while >700 had 100% pass rate. SD= 113.39 Predictive validity score mean 796. Accuracy of scoring |
Most schools administer the HESI APRN at the end of the program of study. Graduate faculty have limited track record of using predictive standardized exams, even if using them in undergraduate education. This is a building block for educational science on APRN predictive testing and should be strengthened with further study. |
|
Barton L, Willson P, Langford R, Schreiner B, 2014, Standardized predictive testing: Practices, policies, and outcomes Level IV |
Vygotsky’s Theory of Zone of Proximal Development [Goal Attainment] |
Registered Nurse (RN) N=471 Stratified random sample of diploma, Associate Degree in Nursing (ADN), and Bachelor of Science in Nursing (BSN) nursing program deans and directors in the United States. |
HESI Exit exam score of 900 or above has a 98.26% pass rate. Table 4 summarizes findings related to a variety of HESI Implementation Policies. Four policy components that point to improved student outcomes; -achievement of a mandatory benchmark score -required participation in prep plan - required re-testing -require remediation after failing to meet benchmark Standardized testing policies need to be crafted to include certain components that are related to standardized exam success. Further study longitudinally for testing policy, implementation is necessary. |
The results of this study demonstrate that developing and implementing policy to support standardized testing in school of nursing is an integral part of student success. Third voice is representative in zone of proximal development. Limitations to generalizability of conclusions due to randomized sample is from single test product database. Also due to retrospective, non-experimental nature of study, no effort to discern or verify school policies implementation. |
|
Stonecypher K, et al, 2015, Faculty experiences developing and implementing policies for Exit Exam testing Level IV |
No theory |
RN N=15 programs 9=BSN programs 6= ADN programs (11=Public and 4=Private) |
Negative triggers that will generate the need for testing policy change. Then once it occurs, policy modification gives it value, then reaction to change of testing policy is measured. |
Although resistance was experienced with change, ultimate goal of higher passage rates were well accepted. As the worth of the policy is demonstrated, acceptance or improvement are made. Continuation to monitor student success and attend to improving curricula, teaching strategies, and evaluation processes is needed by faculty. |
|
Lauer M, Yoho M, 2013, HESI Exams: Consequences and remediation Level IV |
Classical Test Theory by Crocker and Algina Critical Thinking Theory by Paul |
RN N=66 programs N=3758 students 16% designated a benchmark score as 900, 72% designated 850 5% designated 850-900 |
The schools that established a mandatory consequence and remediation policy showed significantly higher HESI Exit scores than those who did not. |
Based on the findings, it is best to attach consequences to standardized testing and to require remediation for those students who do not achieve the faculty-designated benchmark score. If no consequences or remediation is applied, then students will devalue or interpret as unimportant. |
|
Nibert A, Morrison S, 2013, HESI Testing- A history of evidence-based research. Level V |
Classical Testing Theory |
RN N=9 validity studies in last 13 years N=49115 students |
These studies investigated the accuracy of the E2 in predicting success in the NCLEX-RN. Additional variables related to standardized testing that were also investigated included the effects of monitoring, the predictive accuracy of repeated testing with parallel versions of E2, and the impact of HESI Specialty exams throughout curriculum. |
The research findings are to serve as supplement that have undergone rigorous peer review and inspires further expansion of science that supports the dissemination of knowledge related to standardized testing. |
|
Young A, Willson P, 2010, Predicting NCLEX-PN success with the HESI Exit Exam. Level IV |
No theory |
Licensed Practical Nurse/Licensed Vocational Nurse (LPN/LVN) N=72 programs N=4383students |
All three versions of the E2 were found to have predictive validity above 90%. The most common faculty designated benchmark score was 850. Most schools required students to retest until benchmark was reached. |
This study emphasizes the predictive accuracy of the E2-PN HESI for Practical Nurses. Findings are consistent with four previous studies that evaluated the accuracy of the E2-PN HESI at predicting NCLEX-PN success. |
|
Langford R, Young A, 2013, Predicting NCLEX-RN success with the HESI Exit Exam: Eighth Validity Study. Level IV |
Critical thinking theory by Paul and Classical test theory described by Crocker and Algina |
RN N=66 programs N=3758 students |
The HESI E2 Exit Exam continues to be highly accurate 94.93-98.32% in predicting NCLEX-RN success with first time testing and 2 re-tests. The majority of participating nursing programs reported establishing policies related to HESI E2 performance and most set 850 as their benchmark score. |
Predictive accuracy of first time NCLEX-RN takers based on E2. Scores between 850-899 on a repeat testing had significantly greater risk of failing NCLEX-RN with each successive retesting, so continued remediation is essential for students who score 850-899; but of concern for those who require repeat retesting to achieve 850. |
|
Young A, Rose G, Willson P, 2013, Online case studies: HESI Exit Exam scores and NCLEX-RN outcomes. Level IV |
No theory |
RN N=72 programs N=4383 students |
Mean E2 scores and NCLEX-RN pass rates of students who used case studies and those who did not were compared. Both the E2 scores and NCLEX-RN pass rate were significantly higher for those who used case studies compared to those who did not. |
Based on the study, directors and deans are utilizing case studies for remediation and exam preparation. However, this study highlights the significant relation of utilizing Elsevier’s case studies in preparing for the E2 exam and ultimately for the licensing exam. |
|
Zweighaft E., 2013, Impact of HESI Specialty Exams: The Ninth HESI Exit Exam Validity Study. Level IV |
Classical Test Theory by Crocker and Algina Critical Thinking Theory by Paul |
RN N=63 programs N=3790 students Timeline: took E2 between 2008-2009 |
Findings continued to indicate the predictive accuracy of the HESI E2 Exams leading to NCLEX-RN success at 96.61%. Of the eight HESI Specialty exams investigated, scores on the Critical Care, Pediatrics, and Med-Surgical Specialty exams were most predictive of NCLEX-RN success. |
Based on its pattern of predictability, the HESI E2 can confidently be used by faculty as part of curriculum or student remediation, thus decreasing risk of failing the licensure exam and increasing NCLEX-RN pass rates. E2 and Specialty Exams can also be tracked over time as an outside measure of curriculum evaluation and faculty effectiveness for accreditation reports. |
|
Willson P, 2011, Electronic Standardized Testing for Advanced Practice Registered Nursing (APRN) programs. Sigma Theta Tau, International Presentation Level II |
No theory |
APRN N=9 programs N=141 APRN students For 2008-2009 academic year |
Electronic standardized APRN predictive exams provide valuable student competency data that allow faculty to address student knowledge gaps with remediation and clinical experience focus. Also, provide outcome measures for curricular programing, testing policy development and reporting for accreditation. |
This national multi-site US sample of graduate nursing programs found that electronic standardized end of program assessments are highly accurate at predicting APRN certification success. Findings are consistent with preliminary benchmark setting studies for APRN HESI minimum scores of 750 or higher. Faculty’s implementation strategies, testing policies, and curricular evaluation benefitted from results. |
|
Santo L, Frander E, Hawkins A, 2013, The use of standardized exit examinations in baccalaureate nursing education. Level II |
No theory |
RN N=1 program N= # of students, not provided |
Study looks at interventions to implement to address ethical, cultural, socioeconomic, and technological aspects to their standardized testing policy revision. |
Overview of controversy surrounding the use of standardized testing. Discusses the potential for ethical, legal, cultural, socioeconomic and technological consequences. A look at their program’s exit exam policy, pass rates, and recommendations are presented. |
|
Carr S, 2011, NCLEX-RN pass rates peril: one school’s journey through curriculum revision, standardized testing and attitudinal change. Level II |
No theory |
RN N=1program N= 41 students |
Private nursing program discussed and identified multifaceted problems cause poor NCLEX-RN pass rates. Gaps in curricular content, student attitudes, delays in taking exam, inadequate student preparation. |
Strategies to address the low NCLEX-RN pass rates were necessary of this program included: addressing the areas of weakness in the curriculum, revising courses, engaging students, changing standardized exit exams, employing a mid-curricular and implementing remedial courses for poor performers. |
|
Willson P, Martinez-Anderson D, Throckmorton T, 2018, Use of standardized testing in Advanced Practice Registered Nurse (APRN) education. Level III |
Reconceptualized Theory of Vygotsky’s Zone of Proximal Development (goal attainment) |
APRN N=114 programs N=497 students |
Aim to establish predictive validity of one set of standardized exams for APRN certification success in a national U.S. sample and to determine implementation strategies used by faculty. | A national multi-state sample of graduate nursing programs indicate that computerized standardized end of program tests are highly accurate at predicting APRN certification success and give support for faculty to set testing policies and benchmarking scores. Test summary reports, using the conceptual framework of third voice, inform both the faculty and student for programmatic evaluation. |
Note: NCLEX-RN=national council licensure examination; E2 PN HESI= HESI Exit Exam for practical nurses; HESI=Health Education Systems, Inc.; APRN HESI=standardized test for advanced practice registered nurse; HESI E2= HESI Exit Exam. Level of evidence: I= Systematic reviews of randomized-controlled trials or non-randomized trials; II=Single randomized- control trial or single non-randomized; III=Systematic review of correlational or observational study; IV= Single correlational or observational study; V=Systematic review of descriptive, qualitative or physiologic studies (adapted from: Polit & Beck, 9th edition, 2012).
Findings
Eighty percent (n=10) of the articles focused on undergraduate nursing standardized testing using HESI NCLEX-RN, while two articles and one presentation highlighted APRN graduate standardized testing using the HESI APRN Exam. Fifty percent (n=6) of the articles do not make reference to using a conceptual framework,2,5,9,10,11,12 while the other half, described using a specific conceptual framework when relating to standardized testing results and practices.1,4,13,14,15,16 The three commonly mentioned conceptual frameworks related to standardized testing and used in undergraduate nursing education were identified as: 1) the Classical Test Theory by Crocker and Algina17, 2) the Critical Thinking Theory by Paul18, and 3) the Reconceptualization of Vygotsky’s Theory of the Zone of Proximal Development by Eun, Knotek, and Heining-Boynton19. Nibert and Morrison1 used the Classical Test Theory only. The other studies used a combined Classical Test Theory with the Critical Thinking Theory as a framework.4,14,15 While these theories give foundation to test item construction, development and validity, only two articles were found, in the literature, that address the implementation of standardized testing results for faculty and students. This first article by Barton, Willson, Langford and Schreiner (2015)13 utilized Eun, Knotek and Heining-Boynton’s Reconceptualization of Vygotsky’s Theory of the Zone of Proximal Development that lead to goal attainment19 as a conceptual framework, in a sample of diploma, ADN, and BSN nursing programs in relation to a policy for standardized testing. The second article, by Willson, Martinez-Anderson, and Throckmorton (2018)16 also referenced the work of Eun and colleagues,19 in their longitudinal study exploring HESI APRN predictive validity for national certification success among 114 graduate nursing (APRN) programs to underpin the conceptual framework of standardized testing research for their study. The 13 studies included two Level V systematic reviews and seven Level IV studies which demonstrate a strong quality of research design for evidence-based nursing education.20
Third Voice
The defining attributes of the Reconceptualized Theory the Zone of Proximal Development (Goal Attainment) by Eun, Knoetek, and Heining-Boynton19, include dialogue, development, and evaluation that can occur at any level in a variety of disciplines and circumstances. In this theory, three participants exist as “voices.” First voice represents the more experienced participant, such as the educator, second voice represents the inexperienced participant, such as the student, and third voice is the silent or inaudible participant, representing the objective influencer, conceptualized here, as standardized testing results. In the educational environment, the interaction or dialogue between the first (educator) and second voices (students) is considered active development, that is, where teaching and learning occur. The specified measurable space between attaining understanding and knowledge, defines the student’s transition from novice to mastery or goal attainment is called the Zone of Proximal Development (ZPD).19 At the time of evaluation, a standardized test is presented to the student by the educator, this is when Third Voice is awakened and manifests as the inaudible educational phenomenon that reveals the objective and authoritative results of the development (i.e., teaching and learning) that occurred or did not occur. The phenomenon of Third Voice expresses its duality in education and speaks to both, the educator and the student through its unbiased mechanism of results. Student outcomes, has been widely accepted as a principle driver for teaching. When applied to teaching, Third Voice offers educators and students a formative evaluation of their investment in the teaching and learning collaboration. Third voice provides a concise and strategic map for educators for guided remediation as they assist students towards their goal of attaining successful APRN certification. Simultaneously, informing the student of specific content strengths and weaknesses they need to address while in the zone of proximal development.
Third Voice Applied
The phenomenon of Third Voice, has been experienced for many years, however, it was not clearly identified. The breakthrough work of Eun, Knotek, and Heining-Boynton’s Reconceptualized Theory of Vygotsky’s Zone of Proximal Development (ZPD)19 now identifies Third Voice and recognizes its role as an active, but invisible, inaudible participant in the educational process. The defining attributes of dialogue, development, and evaluation can occur at any level and in a variety of circumstances along the educational process continuum.
Figure 1
Dialogue between first and second voice reap rewards as teaching-learning occurs in the ZPD. However, dialogue must be tested to verify if development occurred. With standardized testing, objective results (ie, Third Voice) is evidence that development did or did not occur. The behaviors that follow by the educator and student are based on Third Voice. For the educator, Third Voice is the formative evaluation of the status of understanding and development of the student outcomes. Third Voice gives the educator a model that can be a step by step map to successful use of such tools as one-on-one tutoring/remediation, systematic self-study strategies, review books, targeted clinical experiences, and simulation to assist students to succeed.8,9,12 The interaction between Third Voice and first voice (the educator) moves the educator through their own zone of development to enhance their own knowledge and provide a focused approach for student development and goal attainment of passing the APRN certification exam.
For students, Third Voice revealed in the HESI APRN Exam Student Report is an individualized report that identifies APRN national standards of the profession and student’s rank compared to national norms.1 This report categorizes and highlights areas of need for strategic self-remediation that translates to an overview of student readiness prior to APRN certification. Similarly, the interaction between Third Voice and second voice (the student) moves the student through the ZPD to achieve developmental goal attainment of APRN certification success.
Additionally, Third Voice can be used as a continuous objective, authoritative appraisal of the effectiveness of the nursing program’s curriculum, informing the program administrators through the ZPD where program improvement can be addressed without bias if an objective weakness is found.21 Third Voice provides a guide or roadmap for educators, students and nursing programs to achieve a final developmental goal of APRN certification success.
Discussion
The primary role of APRN education is to ensure that students meet the National Organization of Nurse Practitioner Faculties (NONPF) and American Association of Colleges of Nursing (AACN) Master’s Essentials criteria to become safe and competent primary care providers.22,23 Third Voice contributes objective evidence that acquisition of knowledge, skills, competency, teaching, curriculum, and evaluation of graduate education has been successfully achieved. Using evidence-based theoretical frameworks like Third Voice, empower both the educators in teaching and the students in learning for enhanced development that produces success. The challenge put forth in this article is to recognize, Third Voice as an ever-present partner in the educational process. It serves as an essential source of independent evidence to enhance the use of standardized APRN test results in APRN education and fosters student certification success.
Third Voice’s impact on nursing standardized testing outcomes warrants further exploration. With Third Voice as a guiding framework, evidence-based outcomes for testing products, directed remediation strategies, and timing for predictive testing should be measured not only for the field of nursing but for the various healthcare disciplines requiring licensing and certification testing.5
Acknowledgement:
Dr. Pamela Willson for her mentorship and guidance through this project.
