Development of clinical competence – a longitudinal survey of nurse practitioner students

The aim of the present study is to analyse the development of NP students’ self-assessed clinical competence from the beginning of their education (baseline) to after completion of their clinical studies (follow-up).

The following research questions regarding the students’ education were addressed:

A longitudinal survey design is applied in this study, and the ProffNurseSAS II questionnaire employed for the NP students’ to self-assess their clinical competence at baseline and follow up.

The PROFFNurseSAS II questionnaire used in this study aims to measure the clinical competence of nurses at different educational levels from a holistic and lifelong learning perspective [15]. The questionnaire builds on the validated questionnaire PROFFNurseSAS I [18], which consisted of six components: direct clinical practice, professional development, ethical decision-making, clinical leadership, cooperation and consultation, and critical thinking. Wangensteen [15] sought to improve the questionnaire and developed a modified version containing 50 items subdivided along two scales: the A-scale for self-assessed clinical competence, and the B-scale for self-assessed need for further training [15].

The PROFFNurseSAS II questionnaire has been evaluated for content validity [15] and reliability (the Cronbach’s alpha value for the A-scale was 0.936) [16]. Since the aim of the present study addresses the development of clinical competence, it reports findings on the A-scale.

The survey involved the recruitment of a sample of RNs attending an NP programme at a Norwegian university. The first inclusion criterion was enrolment as a first-year student in the programme during their first semester. The second criterion centred on the students’ completion of their clinical studies (450 h) and the passing of their Objective Structured Clinical Assessment (OSCA). A total of 46 NP students from four cohorts that met the inclusion criteria were invited to participate. Among these, 36 were included in the present study, a response rate of 78%.

A power analysis was performed to evaluate sample size. The standard deviations of the total A-score from the first and second investigations were 0.90 and 0.74, respectively. It is possible, given that there were 36 APN students in the follow-up investigation, that the test power will be at least 90% if the mean difference in the total A-score between the first and second estimations is at least 0.50. Thus, we are satisfied that 36 students appears to be an appropriate sample size for the present study.

The students completed the questionnaire twice: initially at the beginning of their NP education (baseline), and then later after they had completed their clinical studies (follow-up). The interval between surveys was approximately 2 years. The NP education is a part-time programme, in which all the students are assigned an individual study plan stating when and where their clinical studies are conducted, according to their needs and wishes. This provides the students with some flexibility, enabling some to spend more time completing their clinical studies than others. For this reason, there are slight differences between the two measuring points, although an approximate 2-year period applied to all students.

Data were collected using the questionnaire PROFFNurseSAS II, which includes 50 items with responses ranged on a scale from 1 to 10, where 1 indicates a poor, and 10 an excellent, level of clinical competence. The questionnaire also includes an option for scoring ‘entirely missing competence’, which is quantified as zero. The questionnaire also includes the option ‘competency not covered in the programme’, which was treated as an invalid value and not included in the analysis. The number of students who selected this option is set out in the Tables. A number of sociodemographic variables were also collected, including gender, age, years of clinical work experience as an RN, area of work experience (i.e., as a specialist or in primary health care), and previous higher education qualifications above a Bachelor’s degree in nursing, as recognised by the ECTS.

The students were first invited to participate in the baseline investigation by means of a printed handout of the questionnaire distributed during an NP programme lecture. As part of the second invitation, to the follow-up investigation, the students were given the opportunity to respond to the questionnaire online. Data collection was carried out between August 2015 and May 2020 by two of the present authors (IT and LF).

The project was approved by the Norwegian Centre for Research Data (NSD: approval no. 52648). Information was provided to the informants both at baseline (orally and on a written handout) and follow-up (via e-mail). The students were also informed about participant anonymity and their right to withdraw from the study at any time, without giving any reason. An ethical issue arises in situations where the nurse educators are also researchers conducting the study, and their students are participants. Such students may feel under some duress for fear that non-participation will impact on their progress or learning experience within the programme [19]. In order to address this concern, a PhD student with no teaching affiliation to the faculty (IT) assumed responsibility for collecting all the data, except in the case of one of the cohorts for which a former education programme leader (LF) collected data at baseline. It was emphasised to all the students that their responses would not affect their teachers’ evaluations or their examination grades. A scrambling key was created that enabled directly identifiable information to be stored separately from the data. Only the first author (IT) had access to the scrambling key.

The software IBM SPSS® Statistics 26.0 for Windows was used for data analysis. Of the total number of participants, 79% (n = 31) responded to all 50 items in the questionnaire. The baseline and follow-up investigations had total response rates of 85% (n = 33) and 87% (n = 34), respectively. Only participants with fewer than 10 missing items (18% of the total number of items) were included in the study. This criterion resulted in the exclusion of three participants. Thus, among the 39 NP students who responded to the questionnaire (a response rate of 80%), 36 were included in the data analysis. The case mean substitution technique is recommended in self-assessment studies [20] and it was thus used to replace missing data for participants who met the inclusion criteria (n = 5).

The data were expressed as frequencies, percentages and means to summarize the students’ demographical variables, items and total mean. Paired sample t-tests (two tailed) were conducted on participants’ total mean scores and individual items during comparisons of baseline and follow-up scores. An independent sample t-test was used when comparing NP students with high and low self-assessed clinical competencies.

Multiple linear regression analyses were conducted in situations where the total mean score for self-assessed clinical competence at follow-up was used as a dependent variable. The independent variables were as follows; (a) total mean score for the students’ self-assessed clinical competence at baseline, (b) age, (c) years of clinical work experience as an RN (overall, in primary health care and specialist health care, respectively) and (d) previous level of higher education above a Bachelor’s degree in nursing (measured in ECTS credits). Forced entry was used for all the predicting variables [21], which means that all independent variables were entered simultaneously. A backward variable selection method was subsequently applied by which predictors were removed one by one when they were not statistically significant, until all the remaining predictors were recognised as significant. The aim of this process was to build a model containing only statistically significant predictors [21]. Follow-up scores were adjusted for the baseline when linear regressions were performed, as recommended by Vickers and Altman [22],

It is important in this study to recognize a potential ceiling effect resulting from selection of the highest value at baseline, which may lead to an erroneous conclusion of no improvement [23]. In the present study, a ceiling effect was considered to be present at baseline if more than 15% of the participants self-assessed themselves with the highest possible score.

The aim of the present study is to analyse the development of NP students’ self-assessed clinical competence from the beginning of their education (baseline) to after completion of their clinical studies (follow-up). Among the 50 items in the PROFFNurseSAS II questionnaire, 45 increased significantly. The total clinical competence score increased by 1.39 points, resulting in a total score of 8.22 (of a maximum of 10) after the students had completed their clinical studies. This finding is consistent with that of Wangensteen [15], who found that APN students self-assessed their clinical competence at 8.08 on completing their education.

We found that, among the 10 items that increased the most between the beginning of the students’ NP education and following the completion of their clinical studies, 8 items concerned the direct clinical practice of history-taking, physical examination, differential diagnosis and medication. These items refer to medically-related clinical skills that highly proficient RNs must acquire in order to become NPs [1, 24]. This finding is in line with previous research, which has shown that direct clinical practice was what NP/APN students most desired to learn [15, 16], and what NP students perceived as the most important aspect of their clinical competence [25].

Clinical decision-making is an important aspect of an NP’s clinical competence. Tiffen et al. ([26] p.400) developed the following definition: ‘Clinical decision-making is a contextual, continuous, and evolving process, where data are gathered, interpreted, and evaluated in order to select an evidence-based choice of action’. Taylor, Bing-Jonsson, Johansen, Levy-Malmberg, and Fagerström [27] found that while NP students demonstrated some structured history-taking and physical assessment techniques, they struggled to demonstrate decision-making in their OSCE (first clinical exam) for a pre-clinical course that they were required to pass prior to starting their clinical studies. In the present study, all the participating students had completed their clinical studies. They had also passed their OSCA (a second clinical exam), during which they were asked to assess clinical preceptor-selected patients who had given their consent. Item no. 8 in the questionnaire – ‘I interpret, analyse and reach alternative conclusions about patients’ health conditions after a detailed mapping of health history and health assessment (physical examination)’ – is consistent with the aforementioned definition of clinical decision-making, and was ranked second among the items that increased its scores the most in the follow-up part of the present study. This skill is specific and essential to NPs, as is clearly stated in the APN Nordic model [11].

Previous research has shown that medication (interactions and side-effects) represented the item about which NP/APN students most wanted to learn [15, 16]. The present study found that this item was ranked third among the items in the questionnaire that increased its scores the most. A study has shown that RNs working in nursing homes are in need of a deeper understanding of the inter-complexity of age-specific diseases and medication in order to ensure that their clinical judgments and actions are appropriate and safe [28]. Another study revealed that among 243 graduating nursing students and 203 RNs who took a multiple choice test, 25% of the answers to questions related to medicines management indicated a high risk of error [29]. We thus regard it as a promising finding of the present study that the item related to medication was ranked third among those that increased its scores the most. Despite the fact that NPs in Norway have yet to be granted prescription rights, our NP programme course in pharmacology is extensive and compares well with those offered in countries where NPs are granted such rights. The findings in this study indicate that the students had developed their clinical competence for direct clinical practice in line with the ICN’s [1] NP definition and the intended learning outcomes of the NP Master’s programme, specifically in relation to advanced health and physical assessment, advanced pathophysiology and advanced pharmacology.

We found that the 10 items that increased the least between the start of students’ NP education and after the completion of their clinical studies concerned fragmented aspects of clinical competence such as responsibility, cooperation with the physician, decision-making, improvements in the workplace, and the use of electronic devices such as telephones and e-mail when assessing the patient. All of these items coincided with similarly fragmented aspects of clinical competencies set out in the Nordic APN-model, i.e., consultation, coaching and guidance, collaboration and leadership [11].

In the case of item 19 in the questionnaire (‘I improve routines/systems that fail to meet the needs of patients at my workplace’), the students reported average scores at the beginning of their education, and scores for this item did not increase significantly on completion of their clinical studies. This item is considered to be important as a measure of a student’s clinical competence in relation clinical leadership in the workplace. According to the ICN [1], NPs are clinical leaders who can influence health service delivery and the profession at large. Thus, an assessment of the development of students’ clinical competence at advanced nursing level must include an evaluation of their ability as clinical leaders. Indeed, leadership has been reported to be a key factor in an advanced practitioner’s ability to influence innovation, improve clinical practice and health care delivery, and advance the nursing/midwifery professions [30]. Moreover, the leadership aspect of the APN role is embedded in the statutory regulations governing APN Master’s programmes offered in Norway [31], which conform in most respects to national policy and competency standards relating to NP education in countries such as Australia, England, the United States and South Africa [32‐35]. However, the finding in the present study may indicate that the NP students did not develop sufficiently with regard to clinical competence in some aspects of clinical leadership.

The students’ self-assessment of item no. 46—‘I give health promotion advice and recommendations to patients by telephone, e-mail or other electronic devices’—was the second-lowest at the beginning of their education. This item did not increase significantly on completion of their clinical studies. This outcome is in line with previous findings showing that NP/APN students consistently reported the lowest scores for this item [15], and were neutral in their attitudes as to whether this was an important factor in the further training [16]. We note in passing that experience from the Covid-19 pandemic has demonstrated that digital health care is more relevant today than it has ever been.

A ceiling effect was observed for seven of the ten items that increased their scores the least. Since a ceiling effect can be indicative of an incomplete scale [23], it may be argued that some of the items in the PROFFNurseSAS II questionnaire have limitations regarding their value as measures of clinical competence at an advanced level. However, a ceiling effect may reflect other than purely statistical issues [23] and NP students may have been reporting a maximum score for items relevant to their experiences as an RN. There are distinct differences between the levels of clinical competence among RNs and NPs. NPs have a broader degree of autonomy due to their advanced in-depth critical decision-making skills [1]. Due to the recent introduction of the NP role in Norway, many of the students in the present study did not associate with other NPs who could act as role models and indicators of what might be expected in terms of clinical competence at advanced NP level. Thus, in this respect, the present study has some limitations with regard to its measurement of some aspects of clinical competence at NP level.

When selecting the ten items with the lowest mean scores at the beginning of the students’ education, the high- and low-clinical competence groups revealed surprising similarity. With the exception of two items, those selected were identical in both groups. Even more surprisingly, in the case of 22 items, students with a low clinical competence at baseline increased their scores significantly more than those starting with a high clinical competence. A ceiling effect was observed for 14 of the 22 items, but not for those relating to direct clinical practice, which represent the central competencies set out in the Nordic APN model [11]. This means that the students with low clinical competence increased their clinical competence of direct clinical practice more that the students with higher clinical competence. In short, the differences between students with high and low levels of clinical competence at baseline were levelled out during their education. As a consequence, the students evolved into a more homogenous group with similar levels of clinical competence in relation to direct clinical practice after the completion of their clinical studies.

It is important to implement set standards in NP education as a means of achieving consistency in the educational preparation and authorization of NPs [36]. However, there are both positive and negative aspects in having all students develop their clinical competence to the same level. Stensaker and Prøitz [37] have expressed the concern that our perception of quality in education is conflicted. On the one hand, educational programmes are broad-based and generalised in an attempt to promote inclusivity. Others, on the other hand, education policy evolve as narrow and elitist in pursuit of promoting excellence. A challenge currently facing NP education is the fact that programmes seek simultaneously to educate students to be safe practitioners at an advanced level [38] while also encouraging them to be highly autonomous pioneers with the ability to make a difference to the quality of patient care [39]. We argue that NP education should aim to level up differences in clinical competence in order to ensure that the NP role represents a sustainable standard that contributes to improvements in patient safety.

Previous research has not found an association between the clinical competence and work experience of RNs in primary health care [40], or among RNs and critical care nurses working in intensive care units [41]. This is in line with the present authors’ previous research [16], during which we also did not establish work experience in primary health care as a significant predictor of clinical competence among APN/NP students at the beginning of their education. This finding is also supported by that of Wangensteen [15]. However, in the present study, we found that work experience in primary health care was a statistically significant, yet minor, predictor of the total mean score reported by students after they had completed their clinical studies, following adjustment for the total mean score reported at baseline. This could mean that, even though previous work experience was not associated with clinical competence at the beginning of their education, the students who had worked within primary health care utilized their work experience in a way that enabled them to develop clinical competence. Knowles, Holton, Swanson, and Robinson [42] emphasise that the richest resources for learning reside in the adult learners themselves, and recommend teaching techniques that tap in to the learners’ own experiences. Offering students the opportunity to draw on their prior work experience during their Master’s education has been shown to promote learning, especially when educators facilitate the students’ critical reflections on their experiences and provided feedback on their performance [43]. Students who participated in the present study were specifically encouraged to apply their previous work experience during the programme’s coursework and lectures. Our findings may therefore indicate that the NP Master’s programme is well suited to students who have prior work experience in primary health care.

The finding that prior work experience in primary health care is associated with clinical competence development is interesting with regard to NP programmes that include clinical work experience as an entry requirement. According to the ICN [1], entry requirements for NP programmes in terms of work experience differ markedly between countries. For example, Gardner, Dunn, Carryer and Gardner [44] found that entry requirements across the 14 programmes they studied varied from zero to 5 years of experience. In the present authors’ previous study [16], we did not find evidence to recommend having work experience as an entry requirement. Due to the findings in the present study, that are significant yet minor, we remain inconclusive to a recommendation for having working experience as an entry requirement.