Psychometric evaluation of the Ethical Caring Competency Scale in nursing

Here, we first give an overview of the philosophical foundations of the moral competence scales that have been developed and used thus far and then explain the conceptual framework of the ECCS.

Kohlberg defined moral judgment competence as ‘the capacity to make decisions and judgments which are moral (i.e., based on internal principles) and to act in accordance with such judgments’. [25] Colby et al. developed the Moral Judgment Interview (MJI) [26], and Lind developed the Moral Judgment Test [27] to measure moral judgment/reasoning based on Kohlberg’s theory of moral development. Additionally, Rest developed the Defining Issues Test (DIT) [28] based on Kohlberg’s theory. Rest indicated that moral behavior is formulated by four psychological components, namely, ‘moral sensitivity’, ‘moral judgment’, ‘moral motivation’, and ‘moral character’ [28]. The DIT has been used in several studies in more than 40 countries, especially in the 1970s and 1980s [28]. The MJI and the DIT have been mainly utilized in nursing studies [29, 30].

However, the results of nursing studies that have used these tools have indicated that nurses and nursing students have consistently lower than expected levels of moral reasoning [31]. Furthermore, the results regarding the relationships among the variables of moral judgment, education level, and ethical behavior for nurses or nursing students are unclear [29, 30, 32, 33]. In response to these results, many nursing researchers have criticized Kohlberg’s theory for focusing on a justice-oriented conception of morality, which is used more frequently by men [29, 30, 32, 34]. A justice-oriented conception of morality is a theory that judges an act based on whether or not the act is consistent with a specific ‘ethical obligation’. These authors have also suggested that Gilligan’s sex-related theory [35] should be considered; consequently, the use of MJI and DIT has declined in nursing research.

Koskenvuori et al. introduced the four most recent scales in a scoping review of health care professionals’ ethical competence [36]. These are the Moral Competence Scale for Home Care Nurses (MCSHCN) [20], the Moral Competence Questionnaire for Public Health Nurses (MCQ-PHN) [21], the Moral Competence Scale (MCS) [37], and the Moral Skills Inventory (MSI) [38]. In three of these scales (the MCSHCN, the MCQ-PHN, and the MSI), the structure of the instrument follows the four-component model for determining moral behavior described by Rest [28]. The other scale does not follow any previous model. These facts mean that, despite the criticisms of Kohlberg’s theory and suggestions to consider Gilligan’s sex-related theory, no such measuring tool has yet been developed.

Gilligan’s theory is also known as ‘ethics of care’ or ‘care ethics’. Ethics of care is the opposite of a justice-oriented conception of morality. Ethics of care focuses on the responsibilities of multiple people in disagreement and their networks and explains these responsibilities in a contextual and narrative way of thinking. Recently, Toronto has shown four ethical elements of caring, namely, attentiveness, responsibility, competence, and responsiveness [39]. Although the ECCL was derived from a qualitative and descriptive analysis of the data obtained from interviews, it has been shown to fit very well into these four ethical elements. Therefore, the ECCL is currently viewed as a rare list of competencies for health care professionals with a theoretical background in ethics of care.

In assessing nursing competence, it is important to consider the context within which it is to be used [19, 40]. This is especially important for moral competence assessment, as such assessment should reflect the actual behavior of nurses in ethical situations [34]. Additionally, the nursing competence required for effective performance in nursing practice has been mainly defined as an integrated set of knowledge, skills, traits, and attitudes [41‐45]. Even though Fry and Johnstone [46] acknowledged the importance of ethical practice in producing quality care, and moral competence has been described as one of the professional components in nursing [47], a highly applicable scale to measure nurses’ ethical competence has not been developed.

The draft ECCS, which is based on the concept of caring ethics, was created by Katayama et al. through a qualitative and descriptive study and has confirmed content validity and criterion validity [23, 24]. The data used for qualitative descriptive analysis were obtained from interviews held with 15 nurses (mean age, 41.5 ± 5.2 years; mean work experience as a nurse, 19.9 ± 5.2 years). The draft ECCS consists of the four core competencies and a total of 22 items of competency. The consistency between Toronto’s theory and the ECCS has been confirmed as follows. Toronto’s first element, namely, ‘attentiveness’, is included in ECCS items such as ‘Estimates the patient’s subjective distress from physical assessment’ and ‘Feeling conflicted and uncomfortable about situations where good care is not being provided’. The second element, namely, ‘responsibility’, is included in ECCS items such as ‘Expresses values about good care in individual cases’ and ‘Explores diverse values and awareness without sticking to own values’. The third element, namely, ‘competence’, is included in ECCS items such as ‘Introduces evidence into practice with appropriate procedures’ and ‘Disseminates and raises issues without ignoring the challenges of performing good care’. The fourth element, namely, ‘responsiveness’, is exactly the same as the ECCS item of ‘The evaluation of care is based on reactions such as the words and behavior of the patient and/or their family’. The draft ECCS can be used as an action guideline for ethical care. Moreover, the draft ECCS can be used in the creation, evaluation, and operation of an in-service education program for each difficulty level of evidence-based nursing ethics after statistical verification.

Katayama et al. also suggested that it is necessary to select the 22 competency related items by difficulty level according to the proficiency level of nurses, referring to the Dreyfus model [48]. This is because nursing skills change qualitatively according to experience and proficiency, as seen in the application of Benner’s Dreyfus model of the acquisition of nursing skills [49]. The ECCS can be used as a behavioral guideline for ethical care. In addition, the draft ECCS will be available for the creation, evaluation, and operation of evidence-based nursing ethics nurse education programs after statistical validation.

Little has been written regarding establishing a basis for an instrument to evaluate health professionals’ concrete ethical caring competence. If a concrete ethical competence list could be developed, such a list could guide educators, as well as managers, in health care in supporting the development of ethical conduct in health care, which could lead to the possible assessment of ethical competence. Although the draft ECCS has confirmed content validity and criterion validity, its reliability and validity as an evaluation scale for each difficulty level have not yet been confirmed. In addition, statistical verification is required to examine whether the conditions of the Rubric Scale of Ethical Caring Competency (RECC) are satisfied.

Therefore, the research goals under consideration in this study were as follows:

The aim of this study was to verify the reliability and validity of the difficulty setting of the items found in the draft ECCS and to obtain suggestions for the use of the scale as a RECC.

This cross-sectional instrumental psychometric study determined the reliability and validity of the ECCS and used an anonymously self-rated questionnaire.

The participants consisted of a total of 1157 nurses working in two public hospitals, specifically, 486 nurses in August 2018 and 489 nurses in July 2019 from Hospital A and 182 nurses in September 2020 from Hospital B. All the participants were regularly employed nursing staff who worked in the ward at the target hospitals and were not on leave. Hospital A was investigated in two different years, but the participants were treated as different groups because many staff members changed over time. The authors recruited as many participants as possible. The reason was that, in general, 3-10 times as many participants as measurement items were needed, and Gorsuch has suggested that ‘the minimum number is 100 and the maximum is as many as possible’. [50] For the test-retest, we invited 10 nurses for whom no personnel changes were planned at that time from one specific department of Hospital An in 2018.

Research collaborators at Hospitals A and B collected completed questionnaires that had been sealed by the participants and delivered them directly to the researchers unopened.

The questionnaire consisted of the following instruments.

The analysis was carried out according to the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) Risk of Bias checklist [53] and the Health Measurement Scale: A Practical guide to Their Development and Use, Fifth Edition [54]. SPSS for Windows ver. 26 and Amos for Windows ver. 26 were used for the analysis. For all statistical analyses, a two-sided p < .05 was considered significant.

This study was approved by the Clinical Research Ethics Committee of Hamamatsu University School of Medicine (Decision no. 18-267). Nurses were informed that participation in this study was voluntary and that refusing to participate would not incur any negative consequences. They were also told that their answers would be anonymous and confidential and would not be used for any purpose other than this study. Consent was confirmed by filling in the consent confirmation column for participation in the questionnaire and posting it in the collection box. The questionnaires were returned in a sealed envelope to a collection box located at each of the participating hospitals. The participants had the freedom to not answer questions and were allowed to return incomplete or blank questionnaires.

Of the total 1157 people who were sent the questionnaire, 962 provided valid responses (valid response rate = 83.1%). Of these, 98.1% were regular nurses, and approximately half of them had less than 10 years of experience as a nurse. Regarding the highest academic credentials earned, 68.2% had attended a 3-year nursing college (Table 1). The Kaiser–Meyer–Olkin measure of sampling adequacy was .931, and Bartlett’s test of sphericity was <.0001.

None of the 22 ECCS items showed a ceiling effect or floor effect. The item-total correlation ranged from .529 to .736. The Cronbach’s α coefficient when the item was eliminated ranged from .926 to .930 (Table 2). In an exploratory factor analysis (principal factor method, promax rotation), the highest value of an item factor loading was ≥0.4 for all items, and the number of factors also converged well at four. The Cronbach’s α coefficients of the four factors all showed ≥.70. Therefore, no items were excluded.

The Cronbach’s α coefficient of the four core competencies ranged from .717 to .890. We also confirmed r = .900 (p < .001) by the test-retest method.

As a result of verifying the structure of the four factors by confirmatory factor analysis, the goodness-of-fit index (GFI) was .841, the adjusted goodness-of-fit index (AGFI) was .801, the comparative fit index (CFI) was .850, and the root mean square error of approximation (RMSEA) was .092 (Fig. 1).

The competency score for ‘yes’ respondents regarding making an effort to reduce the use of physical restraint was 74.5 points, which was significantly higher than the 71.3 points scored for ‘no’ respondents (p < .001). The competency score of the respondents who had participated in training on nursing/medical ethics was 75.0 points, which was significantly higher than the 71.9 points scored for the respondents who answered ‘no’ (p < .001). Those with a longer duration of experience as a nurse tended to have a higher competency score than did those with a shorter duration (p < .001 for multiple combinations) (Table 3).

Pearson’s correlation coefficient r between the score of the four core competencies and the score of work motivation ranged from .232 to .367 (in either case p < .001). In addition, the scores of the four core competencies had a higher correlation coefficient with ‘future work motivation’ than with ‘current work motivation’ (Table 4).

According to the relative ranking of the average scores of each of the 22 items, three difficulty levels ranging from level A to C were composed within four core competencies (Table 2). The difference in the total score of the items at the three difficulty levels in the four core competencies was significantly different in all pairs (A > B > C) (p < .001, in all pairs) (Table 2). In addition, the competency scores according to the levels of the expert group (n = 168), the proficient group (n = 407), and the competent group (n = 387) showed significant differences in most pairs (Table 3).

The number of participants was 962, which was at least ten times larger than the 22 items included in the ECCS and ≥ 100; therefore, the sample size was suitable for factor analysis [54]. The Kaiser–Meyer–Olkin measure of sampling adequacy [55] and Bartlett’s test of sphericity showed goodness of fit.

Nursing manager competencies are believed to increase with years of experience [24, 56]. It has also been shown that ethical sensitivity, which is defined as the ability to recognize perceived ethical discomfort as a ‘problem’, is also associated with increased nursing experience [24, 57, 58]. Based on this, it can be estimated that the competencies evaluated by the ECCS may differ qualitatively and quantitatively depending on the number of years of experience as a nurse. Therefore, it is considered that approximately half of the participants had less than 10 years of experience as a nurse, which was suitable for verifying the difficulty level of the ECCS.

Furthermore, although the new fiscal year in Japan starts in April, this study was conducted in August, November, and June; since these are times when workloads are relatively stable, the measurement time chosen was most likely appropriate.

Since neither a ceiling effect nor a floor effect were shown, it was judged that there were no items with biased scores. The item-total correlation showed that all items exceeded .3. Five of these items exceeded .7 (II-4, III-1, III-4, III-6, IV-2), but all are necessary and indispensable items for evaluating ethical caring competency [24]. In addition, since the number of items was 12 or more, the sample size exceeded 300, and the Cronbach’s α coefficient when an item was eliminated exceeded .9 for all the items, it was confirmed that the reliability of the items was high [53]. All the items reached the criteria for an item-factor loading ≥0.4, a Cronbach’s alpha ≥.70 in the exploratory factor analysis (principal factor method, promax rotation), and had the possibility of a factor interpretation. Therefore, no items were excluded.

The r in the test-retest reliability was as high as 0.9, thereby confirming the reproducibility of the ECCS. The Cronbach’s α coefficients of the four core competencies were also above the criteria values. Based on these results, the reliability of the ECCS was confirmed.

Structural validity was confirmed to be almost suitable by confirmatory factor analysis (GFI < .84, GFI > AGFI, CFI < .85, RMSEA >.10) [53].

The ECCS score was significantly higher for those who were working to reduce the use of physical restraint, those who had learning experience in nursing/medical ethics, and those who had many years of experience. Routine ethical care practices and learning are thought to increase one’s level of ethical sensitivity [59]. Competencies increase with years of experience [56], and years of experience affect the ethical sensitivity of nurses [57, 58]. Based on this, it is reasonable that the competency evaluated by the ECCS also showed a high score for those who are making an effort to reduce their use of physical restraint, those who have learning experience in nursing/medical ethics, and those who have many years of experience. Therefore, the validity of the ECCS can be verified based on the characteristics of the participants.

In addition, the four core competencies were significantly associated with current and future work motivation. This outcome is consistent with the competency rudder concept, which shows that proficiency that continues to mature while reflecting on experience affects work motivation and job satisfaction [51]. Therefore, it was confirmed that the ECCS can appropriately evaluate the competencies of ethical care in nursing practice. Ethical caring competency in the clinical nursing setting is defined as follows: nurses who have learned the principle of ethics discuss that what is best for the patient is being hindered in daily care. It is also a competence that can be displayed through a series of actions that work indirectly and directly to promote consensus building for what is best for the patient and continue to mature when one reflects on one’s experience [24]. Convergent validity was confirmed by testing the hypothesis that there is a positive correlation between the four core competencies and work motivation.

The total score for each of the three difficulty levels for each of the four core competencies showed a significant difference. This result indicates that the difficulty level of the evaluation sheet items set in this study was statistically supported. This outcome not only verified the reliability and validity of the ECCS as a general evaluation scale but also suggested that the scale could be used as a rubric format evaluation sheet.

A rubric is a tabular tool that sets up several evaluation viewpoints for the evaluation of a certain task and shows the achievement level for each stage in short sentences. Rubrics have been used for self-evaluations and program evaluations in recent years [60]. Rubrics have the property of concretely exemplifying and expressing the content when considering whether the required quality is achieved [61]; therefore, it can be said that it is a suitable format for the evaluation of good care targeted by the ECCS.