The Professional Nurse Self-Assessment Scale: Psychometric testing in Norwegian long term and home care contexts

Throughout the world, health care services are undergoing continuous and rapid changes related to demands for cost effective yet safe and high quality health care. In that adverse health care events threaten the realization of high quality care in all care settings [1], it is imperative to improve patient safety [2]. The importance of nurses’ roles and competence in ensuring patient safety has been confirmed in several studies [3, 4]. Naylor et al. [5] found positive linkages between nursing and patient care quality, and nurses hold an important and leading role in improving health outcomes [6]. A global, well-educated nurse workforce is needed.

Many countries are facing shortages in the health care workforce [7, 8]. This not only relates to the ability to maintain a sufficient number of care staff but also to the ability to provide a suitable mix of nursing competence, with the aim to ensure patient safety in all care contexts. The systematic assessment of nurses’ actual competence levels in diverse care settings has therefore become ever more crucial and of interest to educators, healthcare managers, and politicians on all levels. To ensure the clinical competence of nurses, constant monitoring and supervision is needed. Nurses themselves need to be cognizant of whether their own competence levels meet the standard required for their actual roles. Systematic assessment also enables the tracking of competence throughout an individual career and the assessment of whether nurses’ competence levels meet the requirements of the health care services. Thus measuring nurses’ competence may be useful for a variety of reasons.

During the past decades, advanced practice nurse (APN) roles have been successfully established (e.g., in the USA, Canada, Great Britain, Holland, New Zealand, and Australia). The APN role includes educational advancement, specialization, and role expansion [9] and is shaped by country and context specific characteristics [10]. It is nevertheless recommended that the role be held by individuals with a master’s level degree in nursing [10‐12] that also includes a concentration in an APN role [9, p. 79]. When the APN role is implemented, promising effects of the redistribution of professional roles are seen: where certain tasks, including responsibility, are transferred from physicians to APNs. The fact that APNs provide care and treatment of equal or even better quality than physicians is of interest [13, 14]. APN roles and models are emerging in the Nordic countries at the moment. Even at this early stage, it is possible to discern an emphasis on the importance of clinical competence [15, 16] and that the role transition from registered nurse (RN) to APN appears to be a maturation process that encompasses a broader and deeper holistic view of the patient’s state of health [17]. A recent Nordic study also revealed that top-level managers and politicians emphasize that the acute and complex needs of ill older people will require nurses who possess an advanced competence, relative to both medical treatment and nursing care [18].

During the last decade, interest in assessing nurses’ competence has clearly increased [cf. 19‐26]. Still, Watson et al. [27] report that until 2002 the concept “competence” was poorly defined and that a lack of rigor in the instruments used for its assessment existed. We carried out a comprehensive literature search of the Medline and CINAHL databases using the following key terms: research/assessment/measurement combined with clinical competence/nursing competence/advanced clinical competence/advanced clinical practice, and instrument/tool/scale. The literature search was augmented by an examination of the collected articles’ reference lists. The search process resulted in several instruments for nurses’ self-assessment of competence.

In Jordan, the Competency Evaluation Questionnaire was developed to assess the competence levels of nursing graduates [28]. In China, the Competency Inventory for Registered Nurses, which compares nursing competency and organizational climate [29], was developed and validated as having the potential for cross-cultural application [30]. In Taiwan, three instruments were developed: the Clinical Nursing Competence Questionnaire [31], the Public Health Nurse Professional Competency Scale [32], and the Clinical Competence Questionnaire [23]. In Japan, two instruments were developed: the Holistic Nursing Competence Scale [33] and the Competence Scale for Senior Clinical Nurses [19]. To detect differences in competence across countries in Europe, the European Health Care Training and Accreditation Network developed the EHTAN Questionnaire Tool (EQT), a nurse competence self-assessment tool for general nurses [20, 34].

In Finland, the generic Nurse Competence Scale (NCS) was developed [24]. The NCS includes 73 items and is based on Benners’ domains of clinical practice [35]. The NCS has been used to assess the competence of graduating nurses [22] and to measure or compare nurse competence in different work environments [25, 36‐39]. The NCS has also been translated into various languages and used across cultures and countries, for example in Lithuania [21], Iran [40‐42], the USA [43], Norway [44], and Australia [45]. The NCS has furthermore been validated in an Italian study [46]. Still, even though Meretoja et al. [24] provide extensive documentation of the development of the NCS, during testing of the German version in Switzerland Müller [47] reported that the original seven-factor structure of the NCS was not confirmed. A recent psychometric test of the NCS in Norway has also concluded that the original NCS factor structure was not confirmed [48].

In Sweden, a new instrument has been developed based on formal national competence requirements: the Nurse Professional Competence (NPC) Scale [26]. The NPC was developed for use prior to graduation and among practicing registered nurses over time.

Nilsson et al. [26] maintain that only a few instruments are psychometrically sound. To date, instruments have mostly been developed for use in hospital contexts. In general, the focus of instruments has been the assessment of nurses’ clinical competence at the generalist level (basic level qualifications or a bachelor’s degree). However, when the development of new APN roles started in Finland and Sweden at the turn of the 21st century, a clear need existed for a new instrument for the self-assessment of clinical competence at different educational levels and across specialties and countries.

Wilkinson [49] maintains that without the right tools to assess competency it is difficult to know if nurses are safe to practice. To our knowledge, no instrument as yet exists that measures nurses’ clinical competence at different educational levels. Therefore, the aim of this study was to test the reliability and construct validity of the new Professional Nurse Self-Assessment Scale (ProffNurse SAS) questionnaire in long term and home care contexts in Norway.

The overall response rate (N = 704) was 52.7 % (n = 371). The response rate for the first region was 51.% (n = 166) and the second region 54.% (n = 205). Of the total sample 80.7 % were nurses in direct care (n = 300), 11.3 % were nurse managers (n = 42), and 1.6 % were administrative nurses (n = 6). 95.4 % were women (n = 354). The respondents’ mean age was 41.5 years (range 22–68), with average working experience 9.8 years (range 0–32). Of the total sample 40.2 % (n = 149) possessed education above the generalist level, with 2.9 % (n = 11) educated on the master’s level.

In the second EFA 49.2 % of the variance was explained, and there were 680 (25 %) non-redundant residuals with absolute values >0.05. The Structure Matrix (Table 1) shows correlations between variables and components before rotation. Table 1 demonstrates that 28 items had loadings ≥0.4 to one component, 20 items to two components, 17 items to three components, and 3 items to four components. Six items had no loading exceeding 0.4 to any of the components.

The Pattern Matrix (Table 2) shows the unique contribution of a variable to a component. Two items (items 29 and 62) have loadings ≥ 0.4 to more than one component, while the remaining items with loadings ≥ 0.4 only loaded to one component. Table 2 demonstrates that 19 items had loadings ≥0.4 to component one, 11 items to component two, 11 items to component three, 7 items to component four, 2 items to component five, and finally 6 items to component six (loadings ≥0.4 for all items). Two items loaded >0.4 to more than one component, while all the other components, with the exception of 18 items with no loading ≥ 0.4, loaded only to one of the components.

The eighteen items with loadings <0.4 were excluded from further analysis. High levels of non-response may identify problem items. Deletion may be an option when missing values on variables are not central to the analysis [60], and recommendations for item deletion range from 15 %–40 % [64]. We decided to exclude three items due to internal missing items and employed limits to determine which should be excluded. Items 49 (8.4 % missing), 71 (12.6 % missing), and 72 (17.6 % missing) were subsequently excluded. Although the chosen limits for exclusion may seem rather rigid, these three items had greater missing values than the other, included items. In total 21 items were excluded. An overview of these items and why they were excluded are shown in Table 3.

The pattern matrix of this final EFA (not shown in table) demonstrated that no items loaded more than 0.4 to more than one component. Nineteen items loaded ≥0.4 to component one, 5 items to component two, 11 items to component three, 6 items to component four, 6 items to component five, and 2 items to component six. Two items were excluded due to low loadings in the final EFA (item 51—highest loading -.335 and item 14—highest loading -.307). Despite loadings of <0.4 two items (59 & 62) were kept in component six after discussions in the research group. Reliability tests were performed with and without each of these items, and the alpha values were higher when these items were included. The final version of the ProffNurse SAS is shown in Table 4 and consists of 51 items sorted into six components.

The number of items in each component varies between 4 (lowest) and 19 (highest). The names assigned the components were derived from the content of the items with highest loadings in each component [65] and are as follows: Direct Clinical Practice (19 items), Professional Development (5 items), Ethical Decision-Making (11 items), Clinical Leadership (6 items), Cooperation and Consultation (6 items), and Critical Thinking (4 items). Factors with ≥ 5 items and factor loading >0.5 are considered “solid factors” whereas factors with <5 items are considered “unstable” [66]. With respect to internal consistency, the highest Cronbach’s alpha value was found for Direct Clinical Practice (0.940) and the lowest for Critical Thinking (0.772). Grove et al. [65] defined 0.70–0.79 as moderate Cronbach’s alpha values for newer instruments, and 0.70 is also reported as being acceptable [58]. Accordingly, the internal consistency of the ProffNurse SAS may be considered good.

The results of the Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy showed appropriate intercorrelations for the 74 scale items to explore the underlying structure. The factorality of the data was good [cf. 57]. The PCA (Principal Component Analysis) revealed a six-component structure reducing the items from 74 to 51. Reliability is a premise for validity [65]. For this factor structure Cronbach’s alpha values ranged from 0.737–0.940, leading to the estimation that the ProffNurse SAS is reliable [67].

The first component, Direct Clinical Practice, contains 19 items with a Cronbach’s alpha of 0.940. Bland and Altman [68] describe 0.90 as a minimum and 0.95 as a desirable value for Cronbach’s alpha. Though highly correlated items may make a scale overlong with the possibility of over-emphasizing some aspects, items that are too similar may be redundant [69]. Tavakol and Dennick [67], however, recommend a maximum Cronbach’s alpha of 0.90; they maintain that Cronbach’s alpha exceeding this maximum is an indication that redundant items may be present and suggest shortening the test if such occurs.

The first component consists of 19 items, which may be considered broad. Nevertheless, all of the items in this component loaded between 0.412 (lowest) and 0.791 (highest) and only to this component. As mentioned previously, this instrument was intentionally developed to strengthen the measuring of nurses’ clinical skills. It is therefore important to include all these items in order to capture the complexity of nurses’ clinical competence at all levels, even with the risk that some of the items might be redundant. Defining and finding consensus for the concept “competence” is still under debate, but agreement seems to be emerging [70]. Competence is dynamic and relational, and according to Takase and Teraoka [33] it is a synthesis of knowledge, attitudes, values, and skills; when a holistic approach is applied, ethics and context are included [71, 72]. Clinical competence can be described as “knowledge in actions” when based on the Aristotelian view of knowledge [cf. 73]. Nurses’ clinical competence is essential to ensuring patient safety and high quality nursing care in all caring areas. Therefore this first component, Direct Clinical Practice, covers important aspects of nurses’ clinical practice at different educational levels.

The epicenter of nurses’ clinical competence is the dynamic and mutual nurse-patient relationship: the core of clinical competence is truly “knowing the patient”. The formation and fostering of such a therapeutic relationship is also the core of person-centered care [74]. In this term the word “person” involves all those who are engaged in caring and is underpinned by mutual respect, respect for others as human beings, the right to self-determination and understanding. The third dimension, Ethical Decision-Making, is built on these values and reveals that taking care of patients’ physical, social, mental, and spiritual needs is a holistic as well as a moral commitment in relation to nurses’ clinical competence.

The theoretical framework of the ProffNurse SAS supports all six components. These components encompass the eight domains seen in the Nordic APN model [52], which strengthens the validity of the instrument. To some extent, the sixth component (Critical Thinking) is relevant to the domain Research and Development seen in the Nordic APN model. To incorporate nursing knowledge into practice, critical thinking is necessary [cf. 75, 76]. We perceive critical thinking as a crucial part of nurses’ clinical competence, which implicitly includes how nurses in the Aristotelian manner integrate their scientific knowledge (epistêmê) into their practical doing (technê) in order to provide high quality and safe nursing care to the best for their patients (phronêsis) - whether in an acute care setting or not. The sixth component (Critical Thinking), therefore, corresponds to the content of the domain Research and Development seen in the Nordic APN model.

Item 48 (“I focus on relatives’ need for support and guidance”) had too low loading to be included. A patient’s relatives are important partners for nurses in regard to cooperation, and an investigation of this item should be included in further testing of the instrument. Items 64 and 65 (“I report all ‘near incidents’” and “I report all incidents in accordance with the actual patient safety system”) also had to be excluded because of low loadings. Item 37 (“I give health promotion and illness preventive recommendations in accordance with national guidelines to patients”) and item 72 (“I give health promotion advice and recommendations to patients by telephone”) were not included because of low loadings (see Table 3). Nevertheless, because of the importance of these items to the dimension coaching and guidance as defined in the theoretical framework, these items should be included in the next step of the development of the ProffNurse SAS. Also items 48, 64, and 65 should be evaluated for inclusion in larger samples in other contexts in further development of the instrument.

The six ProffNurse SAS components (Direct Clinical Practice, Professional Development, Ethical Decision-Making, Clinical Leadership, Cooperation and Consultation, and Critical Thinking) are both reliable and valid in this actual study. The ProffNurse SAS is therefore a promising instrument. Nevertheless, we recommend further psychometric testing of the ProffNurse SAS in other countries and contexts and the inclusion of larger samples of nurses at various levels of education, particularly master’s level APNs.