Lasater clinical judgment rubric in nursing education: a Turkish validity and reliability study

This was a cross-sectional observational methodological study of instrument translation and validation.

For the cultural adaptation and psychometric evaluation of the rubric, video recordings of the simulation experiences of 3rd and 4th -year students enrolled in the nursing department of a foundation university were used. The data for the study were collected between April and August 2024. Simulation experience videos of first- and second-year students were not included in the study because they did not yet have sufficient clinical knowledge. Randomly selected simulation experience videos from third-year students were used in the pilot phase. The randomization process was conducted by a faculty member other than the researcher using a simple random sampling method, resulting in the selection of 8 videos. For fourth-year students, all 64 videos included in the scope of the Public Health Nursing course were examined in the primary validity and reliability processes. Public Health Nursing was chosen as it provides comprehensive simulation scenarios that align with the rubric’s focus on evaluating clinical decision-making skills, while also considering the evaluator’s expertise in this field.

The nursing education curriculum at the university where the study was conducted includes laboratory and clinical practices carried out through simulation-based education. Additionally, there is a Center of Advanced Medical Simulation and Education at the university where the research was conducted. This center is a multidisciplinary medical simulation facility that provides high-quality education using advanced simulation systems and various educational methodologies. Furthermore, the researchers have certificates indicating that they have completed training on simulation applications in nursing education provided by the university.

The translation and back translation processes of this rubric followed the recommendations of the International Society for Pharmacoeconomics and Outcomes Research. This process included preparation, forward translation, reconciliation, back translation, back translation review, harmonization, cognitive interview, review and finalization of the results of the cognitive interview, and final reading [24].

Permission was obtained from the author of the rubric, and studies on the subject were investigated. Two independent faculty members and a linguist who was fluent in both languages translated the rubric into Turkish. The translations were compared, and ambiguous expressions and inconsistencies between languages were evaluated by the researchers. Spelling and typographical errors were also checked. A linguist whose native language was English and who lives in Turkey translated the Turkish version back into English, and consistencies were evaluated. The translated measurement tool was sent to the scale author to evaluate it for meaning and similarity. For content validity, the Turkish-translated rubric was submitted to the opinions of 10 faculty members who completed their doctorate degree in different fields of nursing (nursing education, fundamentals of nursing, surgical nursing, medical nursing, pediatric health and diseases nursing, medical education) and provided simulation-based education. The experts were asked to evaluate each item in terms of its suitability for the purpose and comprehensibility, and to provide their opinions on improving the items, following the Davis technique.

The rubric, which was adapted to the target language consistent with the opinions received from the experts, was piloted with eight randomly selected simulation experience videos included in this study. In the pilot application, the faculty members who watched the simulation videos of the student nurses were informed about the rubric and asked to evaluate using this rubric. After the pilot study, necessary adjustments were made to the rubric consistent with the feedback from the instructors. An instructor who had not previously seen the translation of the rubric was provided the rubric to examine it for spelling, punctuation, and reading errors. All data were recorded during the language and cultural adaptation processes of the rubric, which was made ready for use in evaluating student nurses’ clinical judgment competency in simulation-based education practices.

The Lasater Clinical Judgment Rubric (LCJR) was developed to provide a common language for nurse educators and students to assess clinical judgment competence in simulation-based education and to describe the trajectory of clinical judgment development. The rubric includes 11 dimensions that evaluate students’ clinical judgment abilities, with performance scored across four levels of development: beginner (1 point), developing (2 points), proficient (3 points), and exemplary (4 points). The LCJR categorizes clinical judgment into four aspects: noticing, interpreting, responding, and reflecting. Noticing involves focused observation, recognition of deviations from expected patterns, and information seeking. Interpreting includes prioritizing data and making sense of the data. Responding highlights maintaining a calm and confident manner, clear communication, well-planned interventions with flexibility, and being skillful. Reflecting focuses on evaluation/self-analysis and commitment to improvement. Students’ overall clinical judgment is determined by their total score across the 11 dimensions, categorized as beginning (11 points), developing (12–22 points), successful (23–33 points), and exemplary (34–44 points) [17, 25, 26].

In this study, fourth-year nursing students, who constituted the sample group, participated in three different scenario applications requiring clinical decision-making during home visits organized as part of the public health nursing course within the curriculum. To ensure comprehensive sampling, all three scenarios, which were of the same level, were included. A total of 64 students participated in the study. The scenario was designed in accordance with the Healthcare Simulation Standards of Best Practice™ of the International Nursing Association for Clinical Simulation and Learning (INACSL) [27].

The students were provided theoretical training by the instructor of the course for the learning objectives, and relevant resources were shared. Before the scenario implementation, a 15-min prebriefing was conducted [28]. In this preliminary information session, information about the patient’s background, learning objectives, and role of the nurse were presented. Before the SBE, the students provided consent that audio and video recordings would be taken and that these data could be used for educational and research purposes. The students participated in the scenario in groups of two, with each scenario lasting approximately 10 min. One student assumed the role of the nurse, while the other took on the role of the student nurse. During the evaluation process using the rubric, the students who played the nurse role were assessed, as the decision-making process was centered on the nurse. During the scenario, audio and video recordings were taken. The student who completed the scenario moved to the debriefing room indicated in the prebriefing.

After completing the scenarios, the debriefing session was conducted in groups of 8–10 students in 30–40-minute sessions, as recommended by the INACSL [29]. A moderator, who received training on simulation-based training in nursing education provided by the institution and was certified, observed the scenario during the implementation and led the debriefing session. The debriefing for the promoting excellence and reflective learning in simulation method was used in the debriefing session [29].

The recordings of the clinical simulation scenarios wherein the students participated were taken from the Learning Management System archive used by the Clinical Simulation Training Center. In the video recordings, the students were assigned codes; subsequently, two researchers independently evaluated the students using the Lasater Clinical Judgment Rubric- Turkish (LCJR-T) version. In case of inconsistency, the third researcher acted as an independent evaluator, and these inconsistencies were resolved by consensus. The researchers evaluating the videos are instructors in the nursing department and are not responsible for grading the students.

Permission was obtained from the author of the scale via e-mail to adapt the scale into Turkish and conduct validity and reliability studies. Before data collection began, ethics committee approval (approval no.: 2023-21/737) was obtained from the Medical Research Ethics Committee of the institution where the study was conducted, and written institutional permission was obtained from the university administration.

The data obtained in this study were evaluated in a computer environment using Statistical Package for the Social Sciences (SPSS) version 22.0 and Analysis of Moment Structures (AMOS) statistical programs. To determine whether the research variables were normally distributed, kurtosis and skewness values were analyzed. The kurtosis values of the variables that showed normal distribution were between + 1.5 and − 1.5, and the skewness values were between + 2.0 and − 2.0.

Interobserver agreement was analyzed using kappa analysis and intraclass correlation coefficient (ICC). To assess the construct validity of the scale, confirmatory factor analysis (CFA) was performed. Scale reliability was analyzed using Cronbach’s alpha. Item analysis, internal consistency, and scale discrimination were analyzed using independent samples t tests between the lower and upper 27% groups.

For the validity analysis of the rubric, language, content, and construct validities were analyzed.

This expert opinion step was performed to evaluate the content validity of the Turkish rubric. To calculate the content validity index (CVI) of the rubric, the Davis technique (Davis, 1992) was used. The rubric was sent to 10 experts, comprising academicians. Expert opinions were analyzed, and the CVI was calculated. The CVI value of the rubric was 0.94, and the content validity ratio (CVR) value was 0.97.

In this study, CFA was performed to examine whether the factor structure of the translated rubric was compatible with that of the original scale. CFA results showed that the factor loadings of the rubric items ranged between 0.537 and 0.998, and the factor loadings for all the items were statistically significant (p < 0.001) (Table 1; Fig. 1).

In this study, the following were the fit indices of the scale: chi-square/degrees of freedom (χ2/df), 1.70; goodness-of-fit index (GFI), 0.90; comparative fit index (CFI), 0.94; root mean square error of approximation (RMSEA), 0.06; and root mean square residual (RMR), 0.03 (Table 2).

The rubric content significantly differed between the lower and upper 27% groups (p < 0.05). The means of each item were compared between the highest and the lowest 27% score groups using an independent groups t test. The students in the upper 27% group had a significantly greater total score than those in the lower 27% group (p < 0.01).

Internal consistency (Cronbach’s alpha and item‒total score correlation) and interobserver agreement reliability methods were used in the reliability analysis of the measurement tool.

The overall Cronbach’s alpha value of the rubric was 0.96, the Spearman–Brown coefficient was 0.921 (equal length), and the Guttman split-half coefficient was 0.912. The Cronbach’s alpha values of the noticing, interpretation, responding, and reflection subdimensions of the rubric were 0.84, 0.82, 0.83, and 0.80, respectively (Table 3). Analyzing the item‒total correlations revealed that the correlation values ranged between 0.64 and 0.90 (Table 4).

To evaluate the agreement between the two observers, kappa and ICC analyses were performed. The kappa values ranged between 0.72 and 0.92, indicating significant agreement for all the items (p < 0.05). The ICC values ranged between 0.70 and 0.90 and were statistically significant for all the items (p < 0.05). The highest ICC value was 0.90 for item 8 (Table 4.).

Content validity is an indicator of the extent to which each item of the scale and the whole scale serve a purpose [30]. This study used the Davis technique for content validity. Regarding the appropriateness of the questions, the opinions of 10 experts with at least a PhD degree and experience in simulation-based training were obtained, and CVI and CVR values were calculated using the data obtained. The lowest CVI and CVR value was 0.80 [31]. In this study, no items with CVI and CVR values of < 0.80 were noted, and the average CVI and CVR values of the total scale items were 0.97 and 0.94, respectively. This value is consistent with that reported in the literature, and the fact that both values are > 0.80 in this rubric indicates that content validity is considered sufficient [32]. This finding indicates that the rubric items represent the population targeted by this study.

Here, CFA was used to examine the construct validity of the rubric. CFA is performed to confirm the factors identified in measurement tools adapted from a foreign language [33] . χ2/df, GFI, IFI, CFI, RMSEA, RMR, and SRMR are the indices used for evaluating the goodness-of-fit of factors in CFA [34, 35]. The acceptable goodness-of-fit level for the GFI, IFI, and CFI is ≥ 0.90, and a value of ≥ 0.95 is considered good. The acceptable goodness-of-fit level for χ2/df is between 3 and 4, and values between 0 and 3 are considered good fits. The acceptable agreement level for the RMSEA is between 0.05 and 0.08, and a range of 0–0.05 indicates good agreement. The acceptable agreement level for the SRMR and RMR ranges from 0.05 to 0.10, and 0–0.08 is considered a good agreement level [35]. In this study, CFA was used to evaluate the four-factor structure. In the validity and reliability study conducted by Shin et al. for the Korean version, the four-factor model provided a better fit than the second-order factor and independence models [22]. In this study, the following were the fit indices of the rubric: χ2/df, 1.70; GFI, 0.90; AGFI, 0.90; CFI, 0.94; RMSEA, 0.06; and RMR, 0.03. The validity and reliability of the rubric slightly differ between the findings obtained in this study and the fit indices in the Spanish, Portuguese and Korean adaptation studies [18, 21, 22]. These differences were believed to be due to the different languages and cultural characteristics wherein the scales were developed and adapted; however, despite the differences in Korean, Portuguese and Spanish, they demonstrated similarity in terms of showing an acceptable level of agreement. The obtained data revealed that the four-factor structure of noticing, interpreting, responding, and reflecting showed an acceptable and good fit.

The CFA results revealed that noticing, interpreting, responding, and reflecting factor loadings were 0.72–0.88, 0.54–0.68, 0.54–0.76, and 0.93–1.00, respectively. The critical values and significance levels of the items indicate that all the scale items are necessary. In the literature, scale items with factor loadings of < 0.30 should be removed from the scale [36]. In this study, the fact that the factor loadings of the items were > 0.30 and within acceptable limits shows that the rubric has a valid structure for the sample in Turkish and that each item can accurately measure the feature in the dimension to which it belongs.

Reliability is the ability of a measurement instrument to consistently measure a desired characteristic [37]. In this study, the reliability of the rubric adapted into Turkish was evaluated using internal consistency and invariance analyses. Internal consistency indicates that all the items and subdimensions of a scale measure the same concept. Cronbach’s alpha reliability coefficient and item–total score reliability are the most frequently used methods for determining the internal consistency of a measurement tool [38]. The Cronbach’s alpha value of a measurement tool is interpreted as “not reliable,” “low reliability,” “reliable,” and “highly reliable” when it has a range of 0–0.40, 0.40–0.60, 0.60–0.80, and 0.80–1.00, respectively [27, 32]. In this study, the Cronbach’s alpha coefficient of the LCJR-T was 0.912, and each item had a high Cronbach’s alpha coefficient. In the validity and reliability studies of the original version of the rubric, the Cronbach’s alpha coefficient ranged 0.86–0.97 [22, 39, 40]. The rubric was noted to have a high reliability level.

Analyzing the item‒total correlations revealed that each item had a strong relationship with the scale. Although there is no consensus on the range of the correlation coefficient, it should be between 0.30 and 0.50 at a minimum [41]. The correlation values ranging between 0.64 and 0.90 indicate that all the items are in good agreement with the general structure of the scale.

In this study, interrater reliability was used for evaluating invariance. The kappa values ranged 0.72–0.92, and a significant agreement for all the items was observed (p < 0.05). The kappa values are high for the items, indicating that the agreement between the observers is relatively consistent. Similarly, the ICC values indicate that the agreement between the observers is good. The ICC values ranged 0.70–0.90 and were statistically significant for all the items (p < 0.05). Kappa and ICC analyses revealed that the observers’ assessments were consistent and statistically significant.