Virtual patient simulation to improve nurses’ relational skills in a continuing education context: a convergent mixed methods study

This study aimed to quantitatively and qualitatively assess the acceptability of a VP simulation to improve nurses’ relational skills in a CE context. This study is part of a doctoral dissertation [20]. Acceptability is defined as “a multi-faceted construct that reflects the extent to which people delivering or receiving a healthcare intervention consider it to be appropriate, based on anticipated or experienced cognitive and emotional responses to the intervention” [21].

Our specific research objectives were: a) to measure the extent of the VP simulation nurses’ perceived acceptability in regards of the simulation design elements, of the global system quality and the technology acceptance, of the role simulation plays in supporting nurses’ professional practice, and of the achievement of learning objectives (quantitative objective); b) to explore nurses’ learning experience (qualitative objective); c) to deepen understanding of how the VP simulation contributed to nurses’ uptake of relational skills, to overall learning and its transfer into practice (mixed methods objective).

To fill the gap in this type of educational intervention available for nurses and to address the communication roadblock [7], we developed a VP simulation [22] aimed to improve nurses’ relational skills. The VP simulation was informed by a strengths-based nursing approach [23], motivational interviewing (MI) [6], and adult learning theories [24]. The clinical case scenario is the story of Mr. Wilson, an HIV-positive man having difficulty taking his antiretroviral therapy. The clinical content is informed by MI [6]: MI is  one technique HCP can use to ensure the effective relational skills they need to open up conversation with patients about behavioural changes. MI is defined as a person-centered, collaborative, and guiding communication style (rather than a directive one) that elicits people’s motivation and commitment to change. The core of the simulation is then focused on applying relational skills consistent with MI (e.g., asking open-ended questions, using reflective listening), rather than acquiring HIV-specific knowledge. The screen-based simulation allows users to interact with a two-dimensional animated VP character (see Fig. 1). The simulation includes a prebriefing video and text, an electronic patient record, a glossary, and a simulated nurse-patient consultation. The narrative-simulated consultation encompasses quizzes and feedback loops, in which the learners’ choices and decisions can influence the VP’s speech. Green and red labels were also used as visual and theoretical cues to qualify the nurse-patient dialogue and acted as feedback. Key elements of the simulation are described elsewhere [22]  and are summarized in Additional file 1 in the CONSORT-EHEALTH [25].

We conducted a convergent mixed methods study (Fig. 2). This mixed methods design is defined as the collection and analysis of quantitative and qualitative data integrated for results comparison and complementarity [26, 27]. Both types of data were collected separately and analyzed independently. A comparison strategy was then used to combine quantitative and qualitative results [28, 29], to interpret how the merged results agreed (correspondence, similarities), offered complementary information, or contradicted (disagreement or dissonance) [26]. First, we carried out a pre-experimental study with a one-group post-test design [30] to measure nurses’ perceptions of the VP simulation. Second, we used a qualitative exploratory design [31] to describe nurses’ learning experience and to further nuance and deepen our understanding of the acceptability of the intervention by using complementary topics that were not covered by the quantitative component. The purpose of the integration of quantitative and qualitative findings was to compare and contrast both components to provide a comprehensive picture of the VP simulation’s contribution to nurses’ learning. The samples from the quantitative and qualitative components were interdependent as participants were required to complete the VP simulation and the post-test survey before being invited to take part in the focus group. Each method (quantitative and qualitative) aimed to answer complementary research questions. Some data collection questions targeted the same broad content while using different wordings: the role of the simulation to support nurses’ professional practice (quantitative component) and the concrete implications of having participated in the simulation for nursing practice (qualitative component). Other data collection questions were different such as the acceptance of the technology (quantitative component) and the motivation in participating in the whole study (qualitative component). One qualitative question was formulated from one quantitative statement to get a deeper understanding and explain what was meant for nurses to have gotten a useful learning experience helped to the VP simulation. During the integration phase of quantitative and qualitative findings, we identified content areas represented in both data sets [26] gained from survey and focus group - in order to compare and contrast them.

This scale development was informed by Simulation Design Scale [36] to measure nurses’ perceptions of the simulation design elements (23 items): 1) context of the VP simulation / prebriefing (3 items); 2) glossary (4 items); 3) electronic patient record (1 item); 4) quizzes, feedback, and labels (11 items); 5) fidelity (4 items). Fidelity is the extent to which the VP simulation approaches reality [10].

We slightly adapted the French-language version [37] of the Technology Acceptance Model [35] to measure these two main dimensions (total: 27 items): 1) global system quality and 2) technology acceptance. Global system quality is divided into the following constructs: system quality (5 items); information quality (3 items); service quality (3 items); and, user interface design quality (3 items). Technology acceptance includes: perceived usefulness (3 items); perceived ease of use (3 items); perceived enjoyment (3 items); and, intention to use (4 items). The higher the score (range of averages: 1–4), the greater the overall acceptance. The reliability of the original instrument [35] is demonstrated by a Cronbach alpha of 0.70 to 0.96 while our adapted version of the scale showed these psychometric properties: 0.68 to 1.00.

Inspired by a validated French translation [38] of the Role of Simulation in Nurse Education Questionnaire [39] targeting the clinical preparation of students graduating from nursing programs, we developed a 22 item-tool. It evaluated the actual and anticipated impact of simulation on the nurse-patient relationship and on nurses’ communication skills, learning, and confidence in their ability to transfer the relational skills into practice.

This tool was developed to assess nurses’ agreement with the achievement of learning objectives (8 items) following their participation in the simulation.

Once quantitative and qualitative data were collected and analyzed separately, both components were integrated using a comparison of results strategy [48]. We first used a weaving technique, inspired by Fetter et al. [49, 50], that aims to narratively group both quantitative and qualitative findings under a mixed methods interpretation. For this interpretation process, we utilized the four stages of the pillar integration process (Fig. 3) [28] to visually compare quantitative and qualitative components and integrate them into a joint display (Additional file 3).

We reported the quantitative component, including the simulation description, with CONSORT EHEALTH [25] (Additional file 1). We used CHERRIES for Internet e-surveys [51] (Additional file 4); COREQ for the qualitative component [52] (Additional file 5), and GRAMMS for the mixed methods study [53] (Additional file 6).

The participant flow chart of completers (n = 27) and non-completers (n = 22) is presented in Fig. 4.

Most of the completers held a bachelor’s degree. They had been working as nurses for an average of 18 years. Eighteen nurses (66.67%) had experience as HIV nurses. Eight nurses (30%) had previous MI training. Majority of participants (25/27, 93%) reported being confident in their computer skills.

From these 27 completers, five nurses took part in the qualitative component: two men and three women. They worked with different clienteles, including people living with HIV (PLHIV). Four nurses were trained in MI. Additional completers and non-completers’ characteristics are presented in Table 3.

Participants were recruited in person (28/49, 57.14%), i.e. by being informed by a colleague or by the student-researcher; by e-mails sent by the Quebec order of nurses (11/49, 20%) and the HIV mentoring program (10/49, 20%).

The detailed quantitative findings are presented in the additional files: the simulation design elements (Additional file 7), the global system quality and technology acceptance (Additional file 8), the role of the simulation (Additional file 9), and the learning objectives achievement (Additional file 10). Highlights are presented in each subsection.

Four main themes are presented: 1) Motivations to engage in the simulation-based research; 2) Learning in a realistic, immersive, and non-judgmental environment; 3) Perceived utility of the simulation; and, 4) Perceived difficulty in engaging in the simulation-based research.

All the nurses mentioned the simulation’s capacity to promote mistake-based learning through quizzes and feedback loops:

It was fun because it’s like action/reaction. It was immediately obvious if you asked the question wrong, you could see the effect. I found it interesting because if you took a wrong action, you could get back on track. That way, we could understand why it was a mistake. (Female nurse-manager)

This participant, who did the entire virtual simulation twice, reported a progression of his learning, building on the mistakes he had made:

The first time, I made a lot of mistakes because I told myself that I was going to go with my knowledge and experience. The second time, I did it with my new knowledge. It gives you parallel vantage point onto yourself, onto your own beliefs. (Male nurse case-manager)

The simulation thus allowed participants to reflect and take a critical look at themselves and their practice, becoming aware of past mistakes and the impact of their interventions on their relationship and interactions with patients:

You’re never neutral in a MI. Yes, you’re the care provider, but you’re a person. It can set certain limits or can even make you get stuck in it. [The simulation] makes you aware of who you are through all this. (Male nurse case-manager).

Look, if patients don’t react or aren’t motivated, well, maybe it’s because I too am playing a part as the care provider: maybe I am not addressing them in the right way, maybe I am not considering them in their entirety, according to their beliefs and values. (Female nurse-manager)

The interactivity inherent to the simulation supports this reflexive process, which in turn can lead to transferring learning to real practice, and thus improve it:

When you’re one-on-one [with a young person], sometimes you’ll answer off the cuff because you’re in a hurry. If you’ve practiced [the situation] in simulation, you’re going to know that whatever you said was not so great, you know, you’re going to question yourself. So, you’re going to be more careful when a similar situation occurs in reality [...] I’m going to try saying it differently to help the person get a little further. It makes you better. (Female school nurse)

This participant questioned his past interventions, in which he hastily presumed the cause of non-adherence (e.g. relapse, substance abuse) when interacting with his clientele. After participating in the simulation, this nurse stated his intention of changing his way of intervening so that he better understands the patient’s situation, before drawing conclusions:

Do I go too fast sometimes? Telling myself that, well, he didn’t take it [his treatment], that he must have relapsed, always jumping to my conclusions first. Don’t I miss things sometimes, too? I was thinking that maybe now I will be more careful and try to understand the patient’s reasons and stop just saying ‘Ah, well, he didn’t take it.’ (Male assistant head nurse)

The simulation helped to underscore the importance of listening to patients. This meant being present, available during the consultation and living in the “here and now”:

It helps nurses understand or realize that it’s important to listen, to be there in the here and now. More and more, we have our electronic medical records, we write in the record and don’t even look at the patient. We no longer take the time to actually look at the patient because we are so busy on our computer... It’s really worth it to sit down and look at the patient and just be present with them. (Female nurse-manager)

The simulation had a positive influence on revisiting ways of communicating and asking patients the right questions to support them in reflecting and identifying their own solutions:

I’d say it’s more in the way the questions are asked. It’s really focused on open-ended questions, and solutions that come from the patient. We [nurses] may have solutions, but they have to come from them [the patients], and that’s when they are most effective [...] How can we ask questions that bring out the best in the patient? (Male nurse case-manager)

The simulation alerted the nurses and raised their awareness of how they relate to patients, creating optimal conditions for successful relational practice and mobilizing communication skills that allow patients to express themselves and, especially, to find their own solutions.

One participant with no prior MI training considered the simulation to be an effective and efficient way to achieve intensive learning:

I’d read a little about MI, but I’d never done any training. I didn’t expect to learn so much in such a short time. (Female nurse-researcher)

Moreover, for another participant, who had received training in MI and who does not practice directly with patients, the key lays in putting theoretical elements into action with the VP. Consequently, the simulation-facilitated practice helped reinforce her knowledge and feelings of competence in applying MI:

I had already had some MI training. [The simulation] reassured me a bit that, actually, I was competent and that I would have been good, face-to-face, with a patient. So, it just confirmed this for me. Because there’s always a doubt about MI being this huge thing. But in the end, you know, we just lack practice. And I found that the platform meant that I was able to strengthen my nursing practice and my past theoretical learning, since I don’t see patients every day. (Female nurse-manager)

For the other three participants who had previous MI training, the simulation helped them better understand the theory and refresh their knowledge, as well as learn how to better apply it. Simulation as a learning modality thus seemed to benefit nurses with various levels of MI training and knowledge.

Using the pillar integration process (see Additional file 3), we have combined both quantitative and qualitative data and categories (findings). The operationalization of this process as well as an excerpt of the joint display that provides a side-by-side comparison of both types of data and categories is presented in Additional file 11. Examples are provided to support the identification of the four mixed method interpretation findings resulting from our analysis: 1) Influence of the simulation’s fidelity on nurses’ impression of getting a real practice and of having an immersive learning experience; 2) Simulation’s perceived flexibility, efficacy, and control over one’s learning led to a positive learning experience; 3) Taping self-awareness and reflection in relational practice, 4) Acquiring new knowledge and building self-confidence. These findings describe how the VP simulation quality, its element designs and its role contributed to nurses’ learning experience.

Overall, nurses perceived that VP simulation is highly acceptable, if we consider that the great majority of means were above 3, on the 4-point Likert scale. The quantitative results were highly consensual in favour of simulation design elements, global system quality and technology acceptance, the simulation’s role in supporting nursing practice, and the achievement of learning objectives. The qualitative results nuanced, deepened, and even added new elements (e.g. motivation to participate and difficulties encountered) to the quantitative results. The integration of quantitative and qualitative findings drew a full portrait of the continuum of the nurses’ simulation-based experience, the VP elements that contributed to their immersive learning, and its potential transfer to their practice. Four mixed method interpretations were described: 1) Influence of the simulation’s fidelity on nurses’ impression of getting a real practice and having an immersive learning experience; 2) Simulation’s perceived flexibility, efficacy, and control over one’s learning led to a positive learning experience; 3) Taping self-awareness and reflection in relational practice; 4) Acquiring new knowledge and building self-confidence.

The qualitative theme motivations to engage in the simulation-based research brings a new element to the quantitative results, which could not capture this perspective. We created and reinforced what Moore et al. [54] call a teachable moment in order to influence the enrollment, participation, and engagement of nurses in this learning activity. To do so, we shared information about the project via different channels, with a view to reaching a variety of nurse profiles. The majority of nurses who completed the VP simulation was recruited through in-person strategy (16/27). The social influences, such as recommendations by colleagues, are recognized as facilitators to the self-directed learning of physicians in CE, alongside with affective attitude (e.g. professional interest, motivation, willingness) and training accreditation [55]. Learners’ characteristics/qualities, like perseverance and the desire to finish what has been started, are also considered to be elements that can affect both learning and the motivation to engage in the simulation [36, 56].

VP simulation led to immersive, realistic, active, and constructive learning experiences. Prebriefing was planned in the VP simulation codevelopment process [22] and is considered a best practice [57, 58]. Prebriefing is known as a facilitation method and a preparatory activity to ready learners for the simulation-based experience. The red and green labels were cues that served as feedback, and were also used as a facilitation method to orient learners through the simulation [57].

Feedback is considered to be the most important feature of effective learning [16, 59]. Indeed, it can support learners’ self-assessment of their skills and allow the progression, development, and maintenance of those competencies [59]. In the quantitative results, participants reported appreciating the timing of the synchronous feedback. The timing of feedback can indeed play a role in learning, as can its source (how it is provided, by whom) and type (i.e. outcome or process-based). One other meaningful feature of the simulation is the opportunity of deliberate and repetitive practice, which can impact learning [59, 60], unlike other educational interventions, such as conferences or lectures in which learners are often passive recipients of “inert” information [61].

In light of our results and when comparing with the literature, it is reasonable to believe that different modes of fidelity, be it physical, conceptual, emotional/experiential [62, 63], not only affected participants’ learning, but also their engagement, overall experience, and immersion in the simulation [59, 62‐64]. Physical fidelity refers notably to the VP’s appearance and the nurse’s virtual office. In the VP simulation, conceptual fidelity is illustrated by the if/then concept [62, 63]: if the learner adopts a relational skill consistent with MI (e.g. a guiding style of counseling vs a directing one), then it will open the dialogue with the patient. One participant clearly expressed this mode of fidelity by the “action/reaction principle.” Finally, participants in the focus group reported the sense of having truly practiced; they found that simulation was an effective learning approach, reflecting the emotional and experiential mode of fidelity [62]. This mode refers to the learner’s emotions, feelings, and beliefs relating to their entire experience of participating in the simulation [62, 63].

It would seem that the benefits of the simulation extend beyond the learning of the recommended communication techniques to encompass a relational component: being present, taking the time to listen carefully and understanding patients without falling prematurely into professional preconceptions. This relational component is aligned with partnership, one of the vital aspects of MI [6] that has to do with seeing the world through the patient’s eyes without imposing the nurse’s view. Another important finding of this study is the reflective learning developed during nurses’ simulation-based experience. Reflection was indeed central to the learning experience because it played a double role, as both a reflective methodology (i.e. synchronous feedback used to promote reflection and to learn from one’s mistakes) and a learning outcome [65] (i.e. VP simulation allowed nurses to develop reflective learning). Reflection is considered to be a means of supporting professional development [66] and has the potential to transform experience into new learning [67]. A meaningful finding of our study is that nurses raised their awareness of themselves, and of others (i.e. patients), by acknowledging how some elements (such as their own communication styles) could influence positively or negatively therapeutic relationships with patients. Perceptual skills have been reported as a communication skills learning outcome in a qualitative synthesis (n = 168) targeting health professions workers [1]. Perceptual skills (self-awareness, awareness of others and context) accounted for 9% of learning outcomes throughout the whole papers and were considered of importance in reflective, self-regulating health providers. Bennett-Levy [61] sheds light that interpersonal perceptual skills are fundamental to effective therapeutic practice, and include these three attributes: empathy, mindfulness and reflection-in-action. By bringing into the table these notions of reflection and perceptual skills, in agreement with Denniston et al. [1], we argue that relational skills are complex. Training interventions aimed to address relational skills, including communication between healthcare providers and patients, should consider an array of relational skills learning outcomes in their evaluation design.

Our findings also corroborate those of a qualitative studies [11, 68] and those reported in an integrative review (n = 38 articles) [18] that explored how VP simulation influenced the non-technical skills (such as communication) of undergraduate nursing students [11] and undergraduate health professional education [18]. The findings suggest that students acknowledged the importance of communication and listening to their patients. The VP “opened their eyes” to the impacts either effective or poor communication had on healthcare [18]. VP simulation exerted a positive influence on students, by reinforcing or teaching new communication skills, providing opportunities for practicing those skills and for building their confidence in applying them, developing specific verbal and nonverbal communication skills [11] and developing awareness of non-technical skills including but not limited to communication [68].

As far as we are aware, this is the first study to examine the acceptability of a VP simulation informed by MI to improve nurses’ relational skills in a CE context. This mixed methods study led to a gain in complementary and rich data, providing a comprehensive picture of nurses’ learning experience. However, two main challenges were encountered during the integration. The first was dealing with huge numbers of items and data in the quantitative survey (n = 80 items) and comparing these to the qualitative data that was focused on circumscribed thematic. The second was the unequal sample size (27 in the quantitative component; and 5 in the qualitative part). Use of the joint display was helpful in putting together data and results that can be compared and discussed. Limitations of one method associated with the present of the other method are described in further details in Additional file 6.

The convenience sampling and snowball approaches used in both quantitative and qualitative components allowed us to recruit nurses with various profiles who work in different settings, thus adding to the richness of the findings. However, those sampling approaches have their own limitations. The findings may have been tainted with participation bias, given our sample had to complete 100% of the simulation and the post-test survey. It means that only the volunteers and the most “motivated” and willing nurses participated. This could explain the consensual findings in favour of the VP simulation. It would have been useful to also explore the reasons why nurses abandoned the simulation; but this was unfortunately not possible. The nonrepresentativeness of this captive sample of nurses may have limited the transferability of the findings to other contexts and population [69]. In addition, the five-participant sample size was relatively small (focus group).

An important limitation of our study is that we did not follow the best practices for scale development and validation, such as those suggested by Boateng et al. [70]. This is consistent with the exploratory lens of our study. With the exception of the Global System Quality and Technology Acceptance instrument [35], the other tools were not validated. However, to ensure content clarity and ease of navigation, we conducted two rounds of survey pre-tests with potential end users before launching the study.

The variety of the nurses’ profiles provided insight into the transferability of the VP simulation beyond the field of HIV care. In the post-test survey, nurses recommended the simulation to other healthcare professionals. Further research could focus on simulation acceptability for a variety of providers. Subsequent research would be necessary to explore the influence of contextual enablers/barriers (e.g. resources, structure, organizational culture) and those that are related to healthcare professionals themselves (e.g. role, work structure, habits, competing demands) when using VP simulation to apply the resulting MI-inspired relational skills. Considering that reflective learning was an important finding, this aspect could be deepened by exploring the underlying causal mechanisms that lead users to improve their relational skills and to put these latter into practice. Future work could be guided by these research questions: What are the contexts and mechanisms that allow healthcare professionals to integrate MI-consistent relational skills into their professional practice? How does the VP simulation produce different outcomes? Additional research could be conducted to examine how simulation-based intervention change nurses’ practice and how practice-change behaviour translates into patient outcomes (e.g. comfort and quality of life, empowerment, medication adherence).