Introduction
In today’s society, health professionals need to remain competent and adapt to the growing healthcare landscape, to provide safe and high-quality services that effectively meet the public’s health needs. To achieve this, education and training for health professionals must meet these demands and ensure continuous updates in knowledge. Nursing education enhances nurses’ knowledge and skills [1].
Nurses must interpret and apply knowledge to deliver patient care based on the best available evidence, rather than relying on tradition or authority. The ability to critically appraise relevant published literature is a crucial skill for nurses and is considered the benchmark for healthcare by medical professionals. Critical appraisal involves evaluating the quality and relevance of research within a specific setting or context [2].
Anzeige
Critical appraisal skills are essential in nursing because they enable practitioners to sift through extensive information and select the most relevant research for their practice. These skills also empower nurses to participate in evidence-based practice by identifying areas for improvement, implementing necessary changes, and evaluating the outcomes of those changes in their practice [3].
Nursing competence involves the knowledge, skills, and behaviors needed to effectively carry out roles and responsibilities. It is crucial for delivering safe and high-quality patient care [4]. Enhancing postgraduate nursing students’ competency in critical appraisal ensures that nurses provide the most effective and up-to-date care, resulting in improved patient outcomes and higher quality healthcare [5]. However, a major challenge for higher education and the ongoing education of healthcare professionals is to equip them with the skills, knowledge, and values necessary to critically and responsibly consume, adapt, and apply research evidence in their practice [6].
Journal Clubs (JCs) have long been utilized as an educational tool to teach critical appraisal, research methodologies, and biostatistics, enhance reading habits, and encourage evidence-based nursing practice [7]. A journal club (JC) is an educational meeting where participants discuss and evaluate published articles, often serving as a central part of postgraduate training [8]. Traditionally, journal club (JC) activities were held as face-to-face sessions. However, with the onset of the pandemic, many educational activities, including JCs, transitioned to online platforms [9].
In recent years, there has been a swift embrace of different virtual journal club (VJC) formats utilizing Twitter and other social media platforms [10]. These virtual journal clubs (VJCs) can be conducted either synchronously using platforms like Google Meet or Microsoft Teams or asynchronously through online forums such as Twitter. Asynchronous VJCs allow participation at any time and from any location, enabling contributions during a set period without needing all participants to be online simultaneously, often occurring on an Internet blog. Both synchronous and asynchronous VJCs facilitate collaboration across national and international boundaries by overcoming geographical limitations [11].
Anzeige
The use of VJCs is expected to increase as videoconferencing technology advances, creating a more favorable environment for online meetings [12]. However, there is a surprising lack of published information on virtual journal clubs (VJCs) specifically related to postgraduate nursing education. This scarcity is unexpected given the significant expansion of nursing education in recent decades, evidenced by the increasing number of global master’s programs in health professions education, the diverse range of related research papers and scientific journals, and the growing number of nursing education organizations and academic institutions [13]. Therefore, this study aimed to investigate the impact of implementing VJC on developing critical appraisal competencies among postgraduate nursing students. We hypothesized that participants who underwent VJC sessions would have improved their critical appraisal competencies.
Method
Research design and study setting
A one-group pretest-posttest quasi-experimental research design was used in this study. An empirical study with this type of design seeks to determine the impact of an intervention in a particular population without using random assignment [14]. The study has been reported according to the Transparent Reporting of Evaluations with Nonrandomized Designs (TREND) checklist [15]. The study was conducted at the community health nursing department, Faculty of Nursing, Mansoura University, Egypt, during February and December 2023. The nursing department was established in 1994 and serves as a specialized academic and research hub. The department provides a comprehensive environment for postgraduate education, offering a range of advanced courses and programs designed to enhance the knowledge and skills of nursing professionals. The department provides master’s and doctoral programs in community health nursing. In addition to specialized master’s degrees in prevention and control of infection, and evidence-based health care.
Sample size calculation and participants
The required sample size to achieve the aim of the study was determined using G*Power Program® Version 3.1.9.4. The sample size determination took into account the correlation test, which assessed the correlation among scales, assuming an α level of 0.05, a β level of 0.20, and a desired power of 80%, with an assumed mean difference of 1.5 before and after the intervention, and an effect size of 1 [16]. The G*Power computation indicated that 40 participants were required for this study to ensure meaningful analysis. This number was expected to be sufficient to detect a meaningful mean difference for knowledge retention 3 months and six months post-VJC after allowing for 25% for any loss during follow-up. However, we tried to reduce participant dropout by making all follow-ups brief, flexible, and convenient for participants, sending routine reminders to schedule follow-ups, and maintaining detailed contact information of the participants. A convenience sampling technique was used to recruit the participants for this study, adjusting for baseline characteristics. Random sampling was not feasible because of the small number of post-graduate students who met the study’s inclusion requirements and were prepared to take part in it. The recruitment was from the community health nursing department affiliated with the faculty of nursing at Mansoura University, Egypt. The specific inclusion criteria were as follows: (1) postgraduate students, including master’s and doctorate students with similar entry dates; (2) being proficient in using simple online meetings and video conferencing software; and (3) being willing to participate in the study and sign the informed consent form. To address any confounding factors and eliminate any biases from their performance and outcomes, the participants had no prior experience with critical appraisal and journal club from their undergraduate program to their current position. The exclusion criteria included not completing the questionnaires, missing more than one educational session, and taking part in identical training programs. A screening procedure was done to confirm the demographic data of the participants and to evaluate their willingness, availability, and motivation to take part. Additionally, to exclude any participants who have a conflict of interest, such as being aware of the intervention of the study beforehand.
Outcomes
Post-graduate students’ knowledge acquisition, critical appraisal competencies, and attitudes toward VJC were anonymized and evaluated pre-, immediately, and after 6 months of the VJC implementation.
Study tools
The data-collection tool was uploaded electronically to the Google platform. The content validity of the tool was tested by submitting it to five experts in the field of “Community Health Nursing and EBP.” Face validity was tested by conducting a pilot study on 10% of the study sample (n = 4), who were not included in the study. Based on the collected data, the necessary modifications were made, some questions were added, and others were clarified or omitted. The tool consists of four sections constructed by the researchers:
I.
Personal and occupational data included age, gender, years of work experience, academic background, affiliation, membership in a professional organization, job category, previous experience, and numbers of training times on research and EBP.
II.
The knowledge test consisted of 13 multiple-choice questions to assess participants’ self-perceived knowledge of critical appraisal and appraisal of cohort and case-control studies, pre-, 3, and 6 months after the intervention. The tool was constructed by the researchers based on data retrieved from [17]. Participants received a score of (1) for correct answers and (0) for incorrect answers. The scores were averaged to obtain the arithmetic mean and standard deviation. The lowest possible score was zero, and the highest score was 48. Higher scores indicated greater knowledge. The tool’s reliability was assessed using test-retest and internal consistency for all items. The calculated Cronbach’s alpha for the tool was 0.81, which indicates an acceptable reliability.
III.
The skills in applying the critical appraisal scale were used to assess the participants’ skills in applying critical appraisal for research studies (case-control and cohort studies), pre-, 3, and 6 months after the intervention. The tool was constructed by the researchers based on data retrieved from [17]. Based on predefined criteria, participants’ competencies were quantitatively assessed online by three of the authors (RA, ES, & NN) on a three-point Likert scale (competent, improving, and incompetent) [18]. A score of three for competent, two for improving, and one for incompetent was assigned to each item. There were a total of 17 competencies. Consequently, one had the lowest possible score and 51 had the highest score. Greater competencies were recognized by the higher scores. The tool’s reliability was assessed using the test-retest and internal consistency for all items. The calculated Cronbach’s alpha for the tool was 0.79, which indicates an acceptable reliability.
IV.
The attitude scale of the VJC was used to measure the participants’ attitudes towards VJC pre-, immediately, and 6 months after the intervention. The scale was constructed by the researchers based on data retrieved from [17]. Participants used a five-point Likert scale, with 1 denoting strongly disagree and 5 denoting strongly agree, to indicate how much they agreed or disagreed with statements. Fifteen attitudes were present with a possible score of 75. A higher score reflected more positive attitudes. The scale’s reliability was assessed using the test-retest and internal consistency for all items. Cronbach’s alpha for the tool was 0.78 indicating acceptable reliability.
Study implementation process
Pre-VJC Session Preparation
This stage took place over 4 weeks in February 2023. Having computer access was a requirement for participation, and participants were specifically asked to have an email and to download the Microsoft Teams (MS Teams) application. Microsoft Teams is a user-friendly platform that allows for live audio and video streaming, slide presentations, chat, desktop sharing, whiteboard functionality, live Q&A sessions, and polling. Participants were encouraged to share videos and use their cameras to promote active discussions and engagement, although they had the option to keep their cameras off if they preferred. Attendees joined from various locations, with some connecting from separate rooms at work and others from their homes.
Upon enrollment, initial data regarding participants’ personal and occupational characteristics as well as their knowledge and attitudes toward VJC were evaluated using an online questionnaire designed in Google Forms® (Google, Mountain View, CA, USA). The questionnaire was sent via email and was open for two weeks, with a reminder sent after one week. It included a brief explanation of the study’s aim, procedures, the voluntary nature of participation, and assurances of confidentiality and anonymity. Participants were informed that they could withdraw from the study at any time without consequences, and any collected data could be removed upon request. Participation required signing and returning an informed consent form via standard mail.
Additionally, all participants received online materials via email one week before the first meeting. These materials included login information and instructions in a PowerPoint presentation on how to use the video-conferencing software for the study. The participants also received an assignment to evaluate their critical appraisal skills in the form of two research articles and standardized SIGN checklists to guide the critical appraisal process of cohort and case-control studies. They were instructed to read and critique the articles using the attached checklist to present their appraisals and be evaluated in the first meeting. Two virtual meetings were held to evaluate the participants’ skills in critical appraisal individually. One meeting focused on evaluating their skills in appraising cohort studies and the other focused on appraising case-control studies. Three authors (RA, ES, & NN) evaluated each participant online using an observational checklist with clear rubrics outlining the criteria for evaluating critical appraisal skills, such as identifying study design, bias, validity, and reliability [18]. Each time critical appraisal skills were assessed, the appropriate assessor made note of any incorrect executions and offered feedback once the participant finished the test and it was graded. These data served as the baseline for future comparisons. The educational content was developed based on the results of the initial assessment and a review of relevant literature on critical appraisal [19‐22]. The material was carefully selected to include a variety of research articles tailored to the participants’ specialties and interests.
Conducting the VJC Session
We conducted a series of VJC sessions for three months, with one session per week held via a virtual platform (Microsoft Teams) in March, April, and June 2023. The study participants were divided into four teams, each consisting of 10 members, to promote equal participation, encourage discussions and debates, and enhance interactions. One week before the first VJC session, participants received materials via email. These materials included the journal club dates, times, meeting links, session topics, duration, article citations, agendas, and contact information. Additionally, they received online reading resources, on critical appraisal techniques, covering key concepts such as study design, bias detection, and evidence evaluation. Before each VJC session, a few ground rules were reviewed to remind everyone that the VJC is intended for professional and scholarly discussions.
We conducted eleven synchronous VJC sessions in an online room using MS Teams Live technology, as shown in Box 1. These sessions were designed to enhance critical appraisal competencies among participants, a crucial skill for evaluating and applying evidence in healthcare practice. The VJC sessions were meticulously planned with a clear aim, specific goals, and relevant content. They incorporated educational strategies that emphasized the role of participants as co-educators. Specific objectives included (a) Understanding the principles and importance of critical appraisal; (b) Identify and utilize key critical appraisal tools for different types of research studies; (c) Develop skills to assess the validity, reliability, and applicability of research evidence; (d) Apply critical appraisal skills to make informed decisions in clinical and research settings. Each session was structured with guided questions to encourage critical thinking. Five sessions were dedicated to teaching the theoretical aspects, while six sessions focused on the practical components. Each session lasted between 60 and 90 min. The theoretical sessions included introductory PowerPoint presentations covering the concepts, significance, procedures, and tools of critical appraisal. These presentations were designed to provide a comprehensive overview of the fundamental aspects of conducting, analyzing, and appraising case-control and cohort studies.
Box 1
Virtual journal club sessions
Content of VJC Sessions | Methods |
|---|---|
1st session: Introduction to Critical Appraisal - Definition and purpose of critical appraisal. - The role of critical appraisal in evidence-based practice. - Overview of different types of research studies (e.g., RCTs, cohort studies, qualitative studies). - Key concepts: validity, reliability, bias, and applicability. | - Interactive lectures - Group discussions - Presentations - Questions and answers |
2nd to 4th session: Introduction to Cohort and Case-Control Studies - Definitions and key differences between cohort and case-control studies. - Purpose and applications of each study design in healthcare research. - Study design structure: prospective vs. retrospective, selection of participants, and exposure/outcome assessment. - Examples of landmark cohort and case-control studies. | - Interactive lectures - Group discussions - Presentations - Questions and answers |
5th and 6th sessions: Appraisal Frameworks and Tools - Introduction to appraisal tools (e.g., CASP, STROBE guidelines). - Key appraisal questions: assessing validity, reliability, and bias. - Critical appraisal checklists: detailed breakdown and application. - Case studies for practical application using these tools. | - Interactive lectures - Group discussions - Presentations - Questions and answers - Critical appraisal checklists and templates - Scientific articles |
7th and 8th sessions: Biases in Cohort and Case-Control Studies - Common biases in cohort studies: selection bias, information bias, and loss of follow-up. - Common biases in case-control studies: recall bias, selection bias, and misclassification. - Strategies to minimize biases in research design and conduct. - Analyzing case studies to identify and evaluate biases. | - Interactive lectures - Group discussions - Presentations - Questions and answers - Critical appraisal checklists and templates - Scientific articles |
9th session: Statistical Interpretation and Results Analysis - Measures of association: relative risk (RR) and odds ratio (OR). - Understanding confidence intervals and p-values in the context of these studies. - Assessing the strength of associations and causality. - Practical exercises: interpreting and critiquing statistical results from real studies. | - Interactive lectures - Group discussions - Presentations - Questions and answers - Critical appraisal checklists and templates - Scientific articles |
10th session: Application to Clinical Practice - Translating study findings into clinical guidelines and practice. - Evaluating the applicability of study results to specific patient populations. - Case scenarios: applying evidence from cohort and case-control studies to clinical decision-making. - Group discussions on integrating evidence into practice. | - Interactive lectures - Case-based learning - Peer learning - Group discussions - Presentations - Questions and answers - Critical appraisal checklists and templates - Scientific articles |
11th session- Interactive Appraisal Sessions - Participants present their appraisal of selected cohort and case-control studies. - Peer review and constructive feedback sessions. - Reflective discussions on challenges faced during appraisal. - Final assessment of critical appraisal skills. | - Group discussions - Presentations - Debriefing - Reflection |
Anzeige
The practical sessions focused on evaluating the validity, reliability, and applicability of scientific cohort and case-control articles provided electronically, and on demonstrating how to apply this evidence to clinical decision-making. Participants received an electronic copy of a peer-reviewed medical journal article, selected in advance by the researchers using critical appraisal tools. The article was chosen by consensus for its relevance and appropriateness for the target cohort of the trial. It was distributed to all participants two weeks before the scheduled VJC session, with a reminder sent one week before the session to encourage reading of the article.
All participants entered the online room and input their preferred names, which appeared in the upper left corner of the screen. The session moderator addressed the participants while displaying PowerPoint slides in the center of the screen. Questions and comments from participants were shown on the left side of the screen. Participants had choice to speak by activating their microphone. Despite the platform allowing for easy session recording, we chose not to record the journal club discussions to prevent participants from feeling hesitant about sharing their analyses and opinions.
Each session started at 3:30 pm Central Time with a 5-minute scripted welcome and an overview of the agenda. Participants could hear the presenter and see the welcome and agenda slides on the screen. Following this, a 5-minute review of the technology features was conducted. After the introduction, the researchers presented for 30–45 min, with their PowerPoint slides visible on the screen. The presentation included an overview of the selected article, covering its background, study design, and significance. The researcher(s) summarized the article, emphasizing its methodology, results, and conclusions while highlighting key aspects for appraisal, such as the study’s strengths, weaknesses, and potential biases.
After the presentation, the assigned researchers guided participants through the critical appraisal process using the selected tool for 25–30 min. The appraisal was conducted with the SIGN checklist, adapted from the Scottish Intercollegiate Guidelines Network, tailored to the study design. The whole teams were provided with the same article. This process involved evaluating the appropriateness of the study type for the research question, the study design, sources of bias, and the reliability and validity of the chosen outcome measures, as well as the applicability of the results. Each session focused on a single article, with the researchers encouraging active participation by posing open-ended questions and facilitating discussion. They reviewed each section of the article, assessing aspects such as validity, reliability, and applicability.
Anzeige
The session was delivered using a hands-on approach, interactive methods to encourage participation, and collaboration to gauge participants engagement with the material in real-time and identify any gaps, challenges, or questions. Following this, participants had 10 min to ask questions to the authors either by using a microphone or typing in the online room. After the Q&A session, the authors signed off, and a 20-minute moderated discussion began. In the final 5 min, participants were thanked and reminded about the next session. Each session ended at 5:00 pm.
Whenever possible, VJC sessions were facilitated by the same panel of moderators to minimize variations during scheduled VJC sessions. The effectiveness, relevance, and satisfaction of the intervention, as well as its impact on participants’ engagement, were assessed using feedback collected before, during, and after the sessions to guide any necessary modifications or improvements. Additionally, attendance tracking, email reminders, and completion rates were monitored to gauge participants’ engagement with the intervention. The researchers of the study documented the attendance rate at each VJC, the number of questions answered by participants, the duration of each VJC session, and the participant attrition rate from the beginning to the end of the session to assess the feasibility. A WhatsApp group was created to facilitate ongoing discussion for up to one week after the live meeting and to provide a summary at the conclusion of the discussion. WhatsApp was chosen because all members were regular users, and the postgraduate students were already part of an academic communication group on the platform.
Post-intervention phase
Participants received e-mail notifications to complete knowledge tests and attitude scale on their personal computer or smart phone using the same pre-test online questionnaire with different wording to reduce memory bias. They also received a set of two research articles, including cohort studies, and case-control studies related to their current clinical practice and real-world issues, with standardized SIGN checklists for evaluating these studies. Furthermore, two online meetings were scheduled using the MS Teams platform to evaluate their skills in critical appraisal in the same manner as a pre-test. One meeting focused on evaluating their skills in appraising cohort studies and the other focused on appraising case-control studies. Participants’ knowledge retention, attitudes toward VJC, and critical appraisal skills were evaluated immediately and after six months of the implementation of the VJC sessions. All participants started and completed the study intervention and follow-up.
Statistical analysis
Version 25 of the Statistical Package for Social Sciences (SPSS) (Armonk, NY, USA) was used to analyze the data after visualization, organization, removing the outliers from the data set, and tabulation of the collected data. The Kolmogorov-Smirnov test was used to determine whether the data were normal, and descriptive statistics in the form of frequencies and percentages were used to show the results. The variables were examined for anomalies, missing data, and possible errors. Every statistical analysis involved the verification of assumptions. For continuous variables, arithmetic mean and standard deviation were employed, whereas percentages were used for categorical variables. The differences between the pretest and post-test were measured using the student’s t-test. Repeated measures analysis of variance (RM-ANOVA) was used to compare the means at different follow-up durations. By contrasting participants’ knowledge, skills, and attitudes before and after the intervention, the impact of the intervention was shown. A second relationship was examined to examine pertinent sociodemographic factors. The Cronbach’s alpha coefficient test was utilized to determine the reliability (internal consistency) of the questionnaires utilized in the study. An acceptable level of internal consistency was defined as having a Cronbach’s alpha between 0.6 and 0.7, good internal consistency as 0.7–0.9, and outstanding internal consistency as > 0.90 [23, 24]. Statistical significance was set at P < 0.05.
Anzeige
Results
Table 1 summarizes the participants’ personal and occupational characteristics. The mean age was 30.6 ± 6.9 years, with over half (60%) being male. In terms of academic background, 65% had postgraduate qualifications, and half (50%) were affiliated with governmental hospitals. Additionally, over one-third (40%) completed their thesis/dissertation, and the majority (95%) had attended training sessions on research methods and critical appraisal.
Table 1
Distribution of the personal and occupational characteristics of the participants and their history of prior exposure to Research and EBP
N = 40 | % | |
|---|---|---|
Age (Years) Mean ± SD (30.6 ± 6.9) | ||
< 30 | 22 | 55.0 |
30–40 | 10 | 25.0 |
> 40 | 8 | 20.0 |
Gender | ||
Male | 24 | 60.0 |
Female | 16 | 40.0 |
Experience (Years) | ||
< 5 | 10 | 25.0 |
5–10 | 20 | 50.0 |
> 10 | 10 | 25.0 |
Academic background | ||
Master qualification | 14 | 35.0 |
Postgraduate qualification | 26 | 65.0 |
Affiliation | ||
Governmental hospital | 20 | 50.0 |
Community health | 9 | 22.5 |
Academic setting | 11 | 27.5 |
Membership in professional associations | 22 | 55.0 |
Job Category | ||
Preparing for thesis/dissertation | 8 | 20.0 |
Finishing thesis/dissertation | 16 | 40.0 |
Preparing/working on research paper | 9 | 22.5 |
Supervision on thesis/dissertation | 7 | 17.5 |
Previous experience | ||
Personal conduction of research | 28 | 70.0 |
Previous attendance of bio-statistics courses | 34 | 85.0 |
Previous performance of epidemiology training | 34 | 85.0 |
Previous performance of research methods training | 38 | 95.0 |
Previous attendance of critical appraisal workshop | 38 | 95.0 |
Number of training times on research and EBP (n = 38) | ||
Single lecture (1–3 h) | 2 | 5.3 |
Weekend course (3–10 h) | 2 | 5.3 |
University course (> 20 h) | 34 | 89.5 |
Table (2) illustrates the distribution of the participants according to their knowledge and skills in critical appraisal before, immediately after, and six months following the implementation of the VJC. It was noted that 7.5% had good knowledge of critical appraisal during the pre-VJC sessions. However, 87.5% and 82.5% showed good knowledge immediately and six months after VJC session implementation, respectively. The difference was statistically significant (P ≤ 0.001).
Table 2
Distribution of knowledge domains of critical Appraisal among the studied students
Items | Pre–Intervention | Immediate Post | Follow Up | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Poor Knowledge | Fair Knowledge | Good Knowledge | Poor Knowledge | Fair Knowledge | Good Knowledge | Poor Knowledge | Fair Knowledge | Good Knowledge | ||||||||||
n | % | n | % | n | % | n | % | n | % | n | % | n | % | n | % | n | % | |
Concept of critical appraisal | 18 | 45 | 16 | 40 | 6 | 15 | 1 | 2.5 | 2 | 5 | 37 | 92.5 | 2 | 5 | 3 | 7.5 | 35 | 87.5 |
Steps of critical appraisal | 26 | 65 | 10 | 25 | 4 | 10 | 1 | 2.5 | 1 | 2.5 | 38 | 95 | 1 | 2.5 | 3 | 7.5 | 36 | 90 |
Appraising case-control study | 26 | 65 | 12 | 30 | 2 | 5 | 2 | 5 | 3 | 7.5 | 35 | 87.5 | 4 | 10 | 3 | 7.5 | 33 | 82.5 |
Appraising cohort study | 26 | 65 | 14 | 35 | 0 | 0 | 2 | 5 | 4 | 10 | 34 | 85. | 4 | 10 | 6 | 15 | 32 | 80 |
Total Knowledge Score | 24 | 60 | 13 | 32.5 | 3 | 7.5 | 2 | 5 | 3 | 7.5 | 35 | 87.5 | 3 | 7.5 | 4 | 10 | 33 | 82.5 |
X2 = 68.185 P ≤ 0.001* | ||||||||||||||||||
In terms of critical appraisal skills, 100% of the participants had poor critical appraisal skills during pre-VJC sessions. However, 85% and 87.5% showed good skills immediately and six months after VJC session implementation, respectively. There was a statistically significant difference (P ≤ 0.001) (Table 3).
Table 3
Distribution of skills domains of critical Appraisal among the studied students
Items | Pre–Intervention | Immediate Post | Follow Up | |||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Incompetent | Improving | Competent | Incompetent | Improving | Competent | Incompetent | Improving | Competent | ||||||||||
n | % | n | % | n | % | n | % | n | % | n | % | n | % | n | % | n | % | |
Appraising cohort study | 40 | 100 | 0 | 0 | 0 | 0 | 2 | 5 | 4 | 10 | 34 | 85 | 2 | 5 | 4 | 10 | 34 | 85 |
Appraising case-control study | 40 | 100 | 0 | 0 | 0 | 0 | 2 | 5 | 3 | 7.5 | 33 | 82.5 | 2 | 5 | 2 | 5 | 36 | 90 |
Total Skills Score | 40 | 100 | 0 | 0 | 0 | 0 | 2 | 5 | 4 | 10 | 34 | 85 | 2 | 5 | 3 | 7.5 | 35 | 87.5 |
X2 = 103,872 P ≤ 0.001* | ||||||||||||||||||
Figure (1) displays the distribution of the participants based on their attitudes towards VJC before, immediately after, and six months following the implementation of VJC. It was observed that 77.5% of the participants had negative attitudes towards VJC. However, 60% and 82.5% showed positive attitudes towards VJC immediately after and six months after VJC session implementation, respectively. The difference was statistically significant (P ≤ 0.001).
Fig. 1
Distribution of Attitudes toward VJC P: *Significance
×
![]()
Table 4 presents the correlation between knowledge, critical appraisal skills, and attitude scores at pre-intervention, immediate post-intervention, and follow-up. A positive correlation with highly statistically significant differences between the participants ' total knowledge score levels, total practice score levels, and total attitude score levels in the immediate post was found (P = 0.038,0.006), respectively. Furthermore, there was a positive correlation with highly statistically significant differences between the participants ' total knowledge score levels, total practice score levels, and total attitude score levels in the follow-up (P = 0.012, 0.001), respectively.
Table 4
Correlation between knowledge, critical Appraisal skills, and attitudes scores at Per-Intervention, Immediate Post, and Follow-Up
Items | Knowledge | Critical Appraisal Skills | Attitudes | |||
|---|---|---|---|---|---|---|
r | p | r | p | r | p | |
Pre–Intervention | ||||||
Knowledge | - | - | 0.091 | 0.575 | ||
Critical Appraisal Skills | - | - | - | - | ||
Attitudes | 0.091 | 0.575 | - | - | ||
Immediate Post | ||||||
Knowledge | 0.329 | 0.038* | 0.425 | 0.006* | ||
Critical Appraisal Skills | 0.329 | 0.038* | 0.405 | 0.010* | ||
Attitudes | 0.425 | 0.006* | 0.405 | 0.010* | ||
Follow–Up | ||||||
Knowledge | 0.392 | 0.012* | 0.518 | < 0.001** | ||
Critical Appraisal Skills | 0.392 | 0.012* | 0.441 | 0.004* | ||
Attitudes | 0.518 | < 0.001** | 0.441 | 0.004* | ||
The associations between the demographic and occupational characteristics of the participants and the total knowledge score are shown in Table 5. A significant relationship was revealed between the mean score of knowledge of the participants and their age, years of experience, membership in professional associations, and job category in the follow-up assessment. However, no relationship was found between the mean knowledge score of the participants and their gender, academic background, and affiliation.
The associations between the demographic and occupational characteristics of the participants and their total critical appraisal skills score are shown in Table 6. A significant relationship was revealed between the mean score of critical appraisals of the participants and their age, years of experience, membership in professional associations, and job category in the follow-up assessment. However, no relationship was found between the mean knowledge score of the participants and their gender, academic background, and affiliation.
Table 5
Association between the demographic and occupational of the participants and the total knowledge score
Items | Pre–Intervention | Immediate Post | Follow–Up |
|---|---|---|---|
Mean ± SD | Mean ± SD | Mean ± SD | |
Age (Years) | |||
< 30 | 12.6 ± 5.2 | 26.8 ± 8.4 | 23.5 ± 10.8 |
30–40 | 15.0 ± 3.9 | 33.4 ± 5.6 | 31.7 ± 8.5 |
> 40 | 12.3 ± 5.8 | 34.0 ± 6.7 | 33.8 ± 7.3 |
One-way ANOVA [F, P] | F = 0.957, P = 0.393 | F = 4.169, P = 0.023* | F = 4.503, P = 0.018* |
Gender | |||
Male | 12.8 ± 4.8 | 29.2 ± 6.6 | 23.2 ± 9.4 |
Female | 15.1 ± 6.5 | 32.4 ± 9.5 | 28.1 ± 9.6 |
Student’s T–Test [T, P] | T = 1.287, P = 0.205 | T = 1.259, P = 0.215 | T = 1.601, P = 0.117 |
Experience (Years) | |||
< 5 | 12.6 ± 5.3 | 27.2 ± 7.2 | 22.1 ± 8.5 |
5–10 | 13.4 ± 6.1 | 29.5 ± 8.2 | 24.1 ± 11.5 |
> 10 | 14.0 ± 6.5 | 35.8 ± 7.1 | 33.2 ± 6.5 |
One-way ANOVA [F, P] | F = 0.136, P = 0.872 | F = 3.449, P = 0.042* | F = 3.689, P = 0.035* |
Academic background | |||
Educational Qualification | 13.4 ± 5.8 | 29.2 ± 7.4 | 24.2 ± 7.7 |
Postgraduate qualification | 13.0 ± 5.6 | 31.2 ± 8.2 | 28.3 ± 9.2 |
Student’s T–Test [T, P] | T = 0.212, P = 0.832 | T = 0.760, P = 0.451 | T = 1.419, P = 0.164 |
Affiliation | |||
Governmental hospital | 12.0 ± 5.1 | 29.6 ± 7.2 | 23.3 ± 10.5 |
Community health | 16.2 ± 6.5 | 33.4 ± 10.6 | 28.4 ± 11.0 |
Academic setting | 12.7 ± 6.0 | 29.8 ± 6.8 | 25.8 ± 6.4 |
One-way ANOVA [F, P] | F = 1.744, P = 0.188 | F = 0.767, P = 0.471 | F = 0.895, P = 0.416 |
Membership in professional associations | |||
Yes | 14.2 ± 6.0 | 33.3 ± 8.8 | 29.8 ± 9.6 |
No | 11.9 ± 5.5 | 27.6 ± 6.9 | 23.5 ± 9.2 |
Student’s T–Test [T, P] | T = 1.251, P = 0.218 | T = 2.240, P = 0.031* | T = 2.103, P = 0.042* |
Job Category | |||
Preparing for thesis/dissertation | 11.7 ± 5.1 | 26.6 ± 8.9 | 20.0 ± 9.2 |
Finishing thesis/dissertation | 12.0 ± 5.7 | 29.7 ± 5.3 | 23.2 ± 10.5 |
Preparing/working on research paper | 13.5 ± 5.8 | 34.0 ± 2.7 | 26.7 ± 5.8 |
Supervision on thesis/dissertation | 17.6 ± 6.9 | 36.6 ± 7.9 | 34.0 ± 12.6 |
One-way ANOVA [F, P] | F = 1.737, P = 0.176 | F = 4.087, P = 0.013* | F = 2.915, P = 0.047* |
The associations between the demographic and occupational characteristics of the participants and total attitude scores are presented in Table 7. A significant relationship was found between the mean score of critical appraisals among the participants and their age, years of experience, membership in professional associations, job category, academic background, and affiliation in the follow-up assessment. However, no relationship was found between the mean knowledge score of the participants and their gender.
Table 6
Association between the Demographic and occupational of the participants and the total critical Appraisal skills score
Items | Immediate Post | Follow–Up |
|---|---|---|
Mean ± SD | Mean ± SD | |
Age (Years) | ||
< 30 | 2.0 ± 0.6 | 2.9 ± 0.7 |
30–40 | 2.6 ± 0.5 | 2.5 ± 0.5 |
> 40 | 2.7 ± 0.7 | 2.9 ± 0.8 |
One-way ANOVA [F, P] | F = 7.034, P = 0.003* | F = 7.282, P = 0.002* |
Gender | ||
Male | 2.6 ± 0.5 | 2.6 ± 0.7 |
Female | 2.4 ± 0.8 | 2.7 ± 0.8 |
Student’s T–Test [T, P] | T = 0.975, P = 0.335 | T = 0.418, P = 0.678 |
Experience (Years) | ||
< 5 | 2.1 ± 0.5 | 1.9 ± 0.7 |
5–10 | 2.4 ± 0.7 | 2.4 ± 0.6 |
> 10 | 2.9 ± 0.8 | 3.0 ± 0.8 |
One-way ANOVA [F, P] | F = 3.664, P = 0.035* | F = 6.607, P = 0.003* |
Academic background | ||
Educational Qualification | 2.1 ± 0.5 | 2.0 ± 0.6 |
Postgraduate qualification | 2.6 ± 0.7 | 2.8 ± 0.7 |
Student’s T–Test [T, P] | T = 2.361, P = 0.023* | T = 3.615, P < 0.001** |
Affiliation | ||
Governmental hospital | 2.6 ± 0.5 | 2.5 ± 0.6 |
Community health | 2.6 ± 1.0 | 2.4 ± 0.7 |
Academic setting | 2.5 ± 0.5 | 2.9 ± 0.8 |
One-way ANOVA [F, P] | F = 0.096, P = 0.908 | F = 1.664, P = 0.203 |
Membership in professional associations | ||
Yes | 2.8 ± 0.6 | 2.9 ± 0.7 |
No | 2.3 ± 0.7 | 2.2 ± 0.7 |
Student’s T–Test [T, P] | T = 2.432, P = 0.020* | T = 3.146, P = 0.003* |
Job Category | ||
Preparing for thesis/dissertation | 2.0 ± 0.5 | 1.9 ± 0.5 |
Finishing thesis/dissertation | 2.4 ± 0.5 | 2.5 ± 0.8 |
Preparing/working on research paper | 2.8 ± 0.5 | 2.7 ± 0.7 |
Supervision on thesis/dissertation | 2.8 ± 1.1 | 3.0 ± 0.6 |
One-way ANOVA [F, P] | F = 2.913, P = 0.047* | F = 3.431, P = 0.027* |
Table 7
Association between the Demographic and occupational of the participants and the total attitudes score
Items | Pre–Intervention | Immediate Post | Follow–Up |
|---|---|---|---|
Mean ± SD | Mean ± SD | Mean ± SD | |
Age (Years) | |||
< 30 | 41.5 ± 6.6 | 48.1 ± 11.9 | 48.5 ± 9.9 |
30–40 | 44.9 ± 13.0 | 54.8 ± 12.1 | 56.9 ± 16.0 |
> 40 | 45.9 ± 11.2 | 60.8 ± 13.6 | 60.0 ± 9.4 |
One-way ANOVA [F, P] | F = 0.841, P = 0.439 | F = 3.406, P = 0.043* | F = 3.674, P = 0.035* |
Gender | |||
Male | 42.6 ± 6.4 | 52.2 ± 13.1 | 52.8 ± 13.5 |
Female | 45.6 ± 10.9 | 58.0 ± 9.2 | 56.6 ± 6.8 |
Student’s T–Test [T, P] | T = 1.097, P = 0.279 | T = 1.533, P = 0.133 | T = 1.093, P = 0.281 |
Experience (Years) | |||
< 5 | 41.2 ± 6.1 | 46.2 ± 11.9 | 59.5 ± 9.3 |
5–10 | 43.2 ± 9.6 | 56.4 ± 12.5 | 56.6 ± 12.7 |
> 10 | 47.2 ± 11.2 | 60.8 ± 12.0 | 64.1 ± 9.4 |
One-way ANOVA [F, P] | F = 1.093, P = 0.345 | F = 3.840, P = 0.030* | F = 4.252, P = 0.021* |
Academic background | |||
Educational Qualification | 40.6 ± 6.5 | 42.9 ± 7.6 | 50.4 ± 12.7 |
Postgraduate qualification | 44.2 ± 10.3 | 55.5 ± 9.4 | 63.6 ± 7.0 |
Student’s T–Test [T, P] | T = 0.212, P = 0.832 | T = 3.193, P = 0.030* | T = 4.258, P < 0.001** |
Affiliation | |||
Governmental hospital | 40.5 ± 5.6 | 48.0 ± 12.9 | 53.1 ± 14.4 |
Community health | 46.8 ± 9.7 | 53.1 ± 12.1 | 58.6 ± 7.0 |
Academic setting | 46.0 ± 12.6 | 60.8 ± 11.2 | 65.2 ± 7.2 |
One-way ANOVA [F, P] | F = 2.183, P = 0.126 | F = 3.857, P = 0.030* | F = 4.006, P = 0.026* |
Membership in professional associations | |||
Yes | 44.0 ± 10.4 | 59.2 ± 12.6 | 62.2 ± 12.9 |
No | 42.3 ± 8.3 | 52.8 ± 11.9 | 53.8 ± 9.2 |
Student’s T–Test [T, P] | T = 0.562, P = 0.577 | T = 2.163, P = 0.037* | T = 2.400, P = 0.021* |
Job Category | |||
Preparing for thesis/dissertation | 39.2 ± 5.9 | 43.5 ± 12.1 | 47.9 ± 13.7 |
Finishing thesis/dissertation | 42.8 ± 11.1 | 51.1 ± 12.2 | 54.2 ± 9.0 |
Preparing/working on research paper | 44.7 ± 9.4 | 55.9 ± 13.0 | 62.1 ± 4.6 |
Supervision on thesis/dissertation | 51.0 ± 10.6 | 60.9 ± 8.9 | 65.9 ± 10.1 |
One-way ANOVA [F, P] | F = 1.916, P = 0.144 | F = 3.013, P = 0.042* | F = 5.707, P = 0.003* |
Discussion
Enabling the virtual delivery of journal clubs (JCs) could facilitate centralized, wide-scale participation in specialized JCs delivered by content experts to enrich the learning experience and maximize value [25]. A critical appraisal of a research article as part of journal club sessions is an essential aspect of postgraduate training across different medical institutions in various settings [26]. These journal club sessions play a significant role not only in exposing the participants to recent advances but also in the research methodology and skills of critical appraisal, which are essential for determining the validity of the study findings [27‐29].
The delivery of a VJC using a learning platform is simple, educationally valuable, and offers several advantages over traditional face-to-face journal clubs [30]. In the context of EBP, this method is a valuable format for professional development and is not only applicable to the nursing profession but also to all types of healthcare professionals [31]. VJCs can be easily replicated across all nursing specialties and provide opportunities for asynchronous activities, meaning that each member can participate at a time and venue of convenience [32].
The current study revealed significant gains in knowledge, skills, and positive attitudes toward critical appraisal among postgraduate nursing participants after implementing a virtual journal club. These results were achieved because the objective of VJC is kept in mind in all sessions, considering aspects such as using personalized tutoring, structured learning strategies, and materials such as checklists for critical evaluation, integrating topics and activities with clinical practice, and using a multifaceted approach to teaching and learning. This chat feature was extremely beneficial, although it could lead to simultaneous conversations. These results coincide with those of [33] who found that online journal clubs appear to increase awareness and uptake of journal article results and are considered a useful tool by participants. Additionally [34], concluded that VJC sessions improve self-reported confidence in critical analysis skills, and feedback from participants is very positive.
Virtual journal clubs create opportunities for nurses and nursing participants to learn collaboratively while discussing EBP knowledge at a time and place of convenience [35]. Concerning knowledge about critical appraisal, the current study revealed significant gains in EBP knowledge among participants who attended VJC sessions. Participants gained knowledge primarily during the critical appraisal step of EBP. The findings of this study follow a study that demonstrated that VJC impacts knowledge among nurses and creates an opportunity to meet learner needs while discussing information that promotes EBP knowledge and implementation [36]. The same results were found by [37], who stated that VJCs, compared to JCs, equally increased students’ perceptions of their skills in reading, analyzing, and critiquing scientific literature and decreased their perceived levels of stress and frustration.
Furthermore [38], found that VJC was very well received by all participants. The participants found the information current and engaging, the journal club strengthened their knowledge base, enhanced communication and teaching skills, and offered an ideal setting to teach participants how to evaluate and employ EBP. In addition [39], found that participants increased their knowledge of EBP and shared evidence with stakeholders. They intended to use evidence in practice.
Developing a VJC is one approach to using research to improve clinical practice and to facilitate critical review of research to change nursing practice based on empirical findings [32]. Regarding critical appraisal skills, the present study revealed significant gains in EBP skills among participants who attended VJC sessions. The participants gained skills primarily by asking an answerable question, searching for evidence, critical appraisal, applying, and evaluating EBP. Skills maintenance and development over time were achieved by advising the participants to continue participating in the VJC beyond the course or semester, even after completing formal training. Furthermore, starting with simpler studies and gradually introducing more complex research designs and critical appraisal tasks. This scaffolding approach allows students to build confidence and expertise over time. In addition, choosing articles related to current clinical practice and real-world issues; makes the appraisals applicable and meaningful. This keeps students motivated as they can directly see the impact on their future work. Moreover, emphasizing that critical appraisal is not a one-time skill but a lifelong competency that needs continual refinement. The findings of this study agree with several studies that demonstrated that participation in online nursing journal clubs has been correlated with increased familiarity among nurses with research processes and outcomes as well as improved ability to critique research studies and apply evidence from the literature to practice [40, 41]. Additionally [42], concluded that VJC not only exposes nursing staff to scientific evidence to support changing their practice but may also lead to institutional policy changes that are based on best practice evidence in the literature. Moreover, the virtual nursing journal club fostered nurses’ reading of nursing research studies, as stated by [43].
Likewise, a study by [44] reflects a gradual ability to critically review the article under critique and a mastery of the process through an online journal club. Added to [45], who concluded that there was a significant increase over 8 weeks in EBP use and behavior with a hybrid model journal club. Furthermore [46], found that social media-facilitated journal clubs offer an efficient way to continue developing critical appraisal skills in nursing participants, as it increases students’ attention and engagement with the presented EBP activities. In addition [47], found that it promotes multidisciplinary global scientific discussions and research dissemination.
Concerning the students’ attitudes toward virtual journal clubs, a significant increase in positive attitude scores was observed after the implementation of the VJC. This result agreed with the results of a systematic review conducted by [48] that concluded high satisfaction and equivocal or increased preference for online JCs compared to face-to-face due to ease of access, diverse participation, and less time and cost spent traveling. Online JCs were found to be educationally valuable, aiding the development of critical appraisal skills, and promoting change in practice. The same results were found by [44, 49], who stated that an online journal club was enjoyable, informative, and helpful for fostering professional relationships. In addition [34, 40, 50], mentioned that the participants agreed with the significance of staying up to date with recent research and that the online journal club gave them a good opportunity to enhance their competence in clinical practice, apply evidence from the literature to practice, and enable participants to take considerable responsibility for their learning. Furthermore, the results of this study agree with those of [39, 43, 46], who reported that nurses reported high satisfaction with VJC participation and fostered their interaction with other nurses. One of the most successful features was connecting the authors of the articles directly to participants. However, our study participants encountered issues such as an unstable internet connection, limited internet quota, poor sound quality, non-working microphones, and difficulty interacting and communicating with moderators and classmates. To address these challenges, they were advised to make use of free internet data packages and ensure they were using functioning microphones.
Conclusion
The results obtained suggest that the development of a supportive and interactive environment for VJC resulted in significant improvements in the competencies of critical appraisal of published research articles among postgraduate students. They became more knowledgeable and skilled in formulating questions, searching for literature, critically appraising research articles, and applying the best evidence in clinical practice. These improvements in knowledge and skills were retained for up to six months after attending the journal club sessions. Overall, the degree of improvement demonstrated in this study provides sufficient evidence to support VJC as a medium for improving critical appraisal competencies. In VJC, learning becomes much more individualized (adaptive learning) while at the same time promoting the learner’s interaction with others (collaborative learning). Virtual journal clubs equip participants with advanced knowledge and competencies in critical appraisal, allow for technology adaptation and usage, and eventually prepare them to integrate evidence-based procedures into patient care for the advantage of patients and the community. The study results will help to inform the future provision of postgraduate medical education using virtual technology.
Strengths, and limitations
The strength of this study lies in the concept of a novel idea for improving the utility of VJC in augmenting critical appraisal skills and the use of different questions on both the pre-and post-tests, which improved the validity of our study. Although the target participants were Egyptian postgraduate nursing students in a unique cultural and academic environment, the findings of this study can be extrapolated to other undergraduate nursing students with various cultural contexts as this study’s intervention was not culturally sensitive. However, we believe that the principles and processes of developing critical appraisal skills among post-graduate nursing students, as presented in this study, are transferable to other contexts. While cultural differences may influence specific learning approaches or educational preferences, the core elements of critical appraisal—such as evaluating research methodology, assessing bias, and interpreting findings—are universal competencies in nursing education. These skills are grounded in evidence-based practice, which transcends cultural boundaries. Nevertheless, we agree that adaptations may be necessary to align with different cultural and educational settings. Future studies could explore the integration of critical appraisal training in other regions to assess whether cultural or institutional factors affect the outcomes. We will ensure to highlight this limitation in the discussion section of the manuscript and suggest this as an area for future research.
When analyzing the results, a few constraints must be considered. First, bias may have been present because of the study’s non-probability convenience sample with a small sample size. Second, relying solely on participants’ self-reported replies could result in over-reporting or information bias, which weakens our belief about the impact of the intervention. Third, students who choose to participate in the VJC may have a higher level of intrinsic motivation and interest in critical appraisal, which might not represent the broader student population. Fourth, the virtual nature of the VJC may limit opportunities for in-depth, face-to-face interactions and spontaneous discussions, which can be valuable for learning and skill development. Finally, without a control group, the quasi-experimental methodology employed in this study prevented us from establishing causation. Therefore, additional validation of our findings is needed using appropriate control groups in future controlled studies. The study’s authors tried to reduce bias by using a non-biased scorer to consistently assess the pre-and post-tests to overcome these restrictions.
Recommendations
We suggest that virtual journal clubs should be used regularly to enhance critical appraisal competencies in an academic setting. Furthermore, virtual journal clubs should be integrated into academic organization policies to improve behavioral changes for continued professional development. It is also essential to incorporate multiple social media strategies when developing a VJC with a platform for distance learning that is flexible, simple, practical, and up-to-date.
Acknowledgements
The authors are thankful to the Deanship of Graduate Studies and Scientific Research at the University of Bisha for supporting this work through the Fast-Track Research Support Program. Also, we acknowledge all the faculty members and the postgraduate participants for their active and enthusiastic participation and support during the conduct of the study.
Declarations
Ethical approval and consent to participate
The researchers carried out the study in compliance with the guidelines and moral standards set forth by the Research Ethics Committee of the Faculty of Nursing, Mansoura University (Ref. No. P.0296). This study was performed following the ethical standards laid down in the 1964 Declaration of Helsinki and its later amendments. All participant in the study gave their informed consent after explaining the study’s aim and ensuring data confidentiality, and it was a completely volunteer, non-profit endeavor. Additionally, participants were made aware that they could leave the study at any moment without giving a reason. Every study participant had their data entered into a password-protected electronic database via a secure computer and internet connection. Only specific members of the study team had access to the electronic database. Regular automated backups were conducted to track the data entry. Throughout the study, all participant records were the responsibility of the chief investigator and were kept secure.
Consent for publication
Not applicable.
Competing interests
The authors declare no competing interests.
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
