Association between game metrics in a simulation game and nursing students’ surgical nursing knowledge – a quasi-experimental study

Surgical nursing care is demanding since the patient’s situation may change suddenly, especially in the postoperative phase, and nurses must make complex decisions in time-constrained situations [1, 2]. Cognitive skills needed in surgical nurses include early recognition of physiological deterioration, confirming physiological deterioration, initiating rescue, securing medical assistance, and rescuing the patient [1]. Especially among newly graduated nurses, the early recognition of physical deterioration is difficult due to limited exposure to patients with complex problems [1]. Therefore, students must gain experience in coping with challenging situations in clinical practice already during their training. With the current global shortage of nurses [3], it is even more important to create conditions for the successful learning experience in clinical practice so that students remain in the nursing field [4].

To prepare nursing students for challenging clinical situations, online interactive software tools that combine theoretical pre- and postoperative nursing care content and practical skills are needed [5‐8]. In recent years, teaching and learning methods for surgical nursing have diversified. Recent scoping review revealed that teaching pre- and postoperative content has included human patient simulations, virtual simulations, web-based learning, and case studies [5]. Interactive and motivating virtual simulations may benefit students in their clinical preparedness since they promote systematic patient assessment [9] which is important skill for surgical nursing. In addition, a desktop virtual reality application for learning preoperative handover enables undergraduate nursing students to be active in the learning activity, which facilitates perceived learning outcomes [10].

In addition to the benefits related to the learning process, it has been shown that simulation games are effective for acquiring surgical nursing knowledge during education (Anonymised, accepted for publication). Previously, we conducted a quasi-experimental study, including a simulation game group (n = 140) and a control group (n = 136) and pre- and post-test measurement of the level of surgical nursing knowledge with a surgical nursing knowledge (SNK) test (Anonymised, accepted for publication). The results revealed that both groups had better surgical nursing knowledge after the intervention than before the intervention, but the change was greater in the simulation game group than in the control group.

One benefit of using simulation games in education and research is related to the information they store about students’ behaviour while playing and the information it produces to evaluate learning outcomes. This information, so-called game metrics, can be used in performance evaluation in education. Game metrics measure all actions taken by a player during playing, for example, the player’s location in the game environment and interactions with other game characters [11]. In educational research, game metrics have mostly focused on playing time [11‐13], the number of games played, the number of completed tasks, and points [12‐14].

The educational value of game metrics lies in their providing information to students and teachers about students’ skills and knowledge [14, 15] and students’ learning and competency [16]. Game metrics could also be used to determine the optimal playing time to achieve good performance. In a learning context other than nursing education, one study demonstrated that the quality of the learning outcome was positively correlated with the time spent and the number of objects placed during gaming [13] but by contrast, another study found no correlation between time spent on execution and learning effectiveness [17]. In nursing education, it has been found that nursing students’ mean scores in a nursing simulation game were significantly associated with clinical reasoning skills [14]. Further, it has been reported that total time spent by students on the simulation game correlated with learning [18] and the more time students spent on playing a scenario the better their mean score was [15]. This implies that students achieve higher scores when engaging for a longer time in the scenarios, thus reflecting better performance.

However, although virtual simulations and games are increasingly used in nursing education, published research on the game metrics stored in the systems and their connection to learning outcomes is lacking. The experimental work presented here is one of the first studies to fill this knowledge gap.

In this study, a slight majority of students (58.6%) had completed the Finnish Matriculation Examination. More than half of the students (60%) had less than 1 year of work experience in social and health services.

This study was undertaken to evaluate the association between game metrics in a simulation game and nursing students’ surgical nursing knowledge. These experiments confirmed that surgical nursing knowledge of nursing students was better after the intervention than before the intervention, as expected, also supporting our previous findings (Anonymised, accepted for publication). Using a knowledge test to measure surgical knowledge objectively demonstrates an increase in knowledge, although it must be noted that educational interventions generally have the desired effect on learning outcomes. In this study, the association between background information and game scores and knowledge test was not investigated but work experience in the social and health services that some of the students had may have had a slight influence on their scores in the game and knowledge test. Although it is unlikely, it is worth investigating more in the future.

Nevertheless, these results add to the rapidly expanding data for the educational value of virtual simulations [5, 7] in learning key competence areas of nursing such as surgical nursing. The contents of the simulation game scenarios corresponded to the cognitive skills required from surgical nurses [1], and thus, it could be cautiously stated that the simulation game supported nursing student knowledge in these cognitive areas. The scenarios also enabled students to practice a certain systematic framework or protocol, as in other Nordic studies [10, 25], and repeating the same content in a virtual simulation has been found to promote the learning of nursing students [9, 26].

As in many earlier reports [11‐13], the game metrics in this study focused on scores and time spent playing. Regarding mean score and learning outcomes, the results corroborate the findings of a previous study reporting that mean scores were significantly associated with learning outcomes, in that case clinical reasoning skills [14]. This study showed that by playing a simulation game it is also possible to acquire knowledge and therefore, in addition to traditional ways of acquiring knowledge, such as lectures and written assignments, simulation games can be used in nursing education to strengthen student knowledge base on surgical nursing. This, in turn, can contribute to successful learning experiences in clinical practice which can potentially engage students in the nursing field [4].

Regarding mean playing time, these results did not confirm the association between learning outcome and time spent playing, unlike an earlier study [13], but did confirm the findings of where no correlation was noted between time spent playing and learning effectiveness [17]. On the other hand, in our earlier study (Anonymised, accepted for publication), where we investigated the association between time spent playing and the mean scores achieved by students in different scenarios, the result showed that the more time students spent playing a scenario the better their mean score. Taken together, research results are inconsistent and warrant further investigations before conclusively stating that time spent playing is associated with positive learning outcomes. However, shorter playing time may also reflect better initial knowledge, i.e. superior knowledge at the outset will speed up the playing required to learn something new. Thus, more research is needed to determine the optimal playing time to achieve good performance.

A strength of the study is that two objective measures were used: a knowledge test and game metrics. Both have been used in previous studies [15] (Anonymised, accepted for publication). This avoided the bias that can arise from self-evaluation. However, these results may be biased due to the purposive sampling and the study being limited to three UAS in one country. In assessing the validity of the game metrics, it is essential to pay attention to data collection and interpretation of the results. The correct information must be collected about the player’s behaviour in the game. A reliable interpretation of the results requires a clear definition of the investigated objects, i.e. an explanation of what is to be shown with the measured objects. In this study, we determined in advance that information on the mean scores and mean playing time would be collected so that their connection to scores on the knowledge test could be examined. By combining game metrics with knowledge test data, the validity of the research result could be improved. However, one limitation of the study was the one-sided use of game metrics, with only mean playing times and mean scores being evaluated. A factor that weakens the validity of the findings is that some of the students played the scenarios for a very short time, reflected in the large standard deviation in playing times. These results, therefore, need to be interpreted with caution, and in the future game metrics should be used in a more versatile manner in the evaluation of learning outcomes.

The main finding of this study was that the higher the mean score in the simulation game was the better the students’ surgical nursing knowledge was in the knowledge test. However, this study did not show that the time spent playing had an impact on students’ post-playing knowledge. The results show that it is possible to acquire surgical knowledge with playing the interactive simulation game. In addition, the results indicate that game metrics can be used to objectively evaluate performance in education. Taken together, these findings suggest that simulation games could be used to improve nursing students’ cognitive skills for challenging preoperative and postoperative clinical situations and game metrics could be used to evaluate learning outcomes.