Development and assessment of the psychometric properties of a compassionate care questionnaire for nurses

The present study used an exploratory sequential design to develop an instrument in two phases- qualitative and quantitative-from February 2016 to October 2018 [21]. The study was conducted in one of the major cities in the south east of Iran. The objective of the qualitative phase was to identify the concept of compassionate nursing care and its dimensions and sub-dimensions based on the experiences of nurses, hospitalized patients and their family caregivers. In this phase we used the conventional content analysis approach recommended by Graneheim and Lundman (2004). Content analysis approach was used to interpret the content of textual data to gain a deep understanding of the concept of compassionate nursing care [22]. After the meaning of compassionate nursing care and its constituent parts was established, the researchers used the blueprint to create the initial pool of questions based on the categories and subcategories extracted from the definition of compassionate nursing care. Subsequently, an extensive review of literature was conducted to complete the items of the questionnaire. Thus, the initial draft of the questionnaire was created. In the quantitative phase, the psychometric properties of the questionnaire, including its validity and reliability, were assessed. The validity of the questionnaire was measured according to its face validity, content validity, and construct validity. In the quantitative phase, COSMIN (Consensus-based Standards for the selection of health Measurement Instrument) criteria, consisting of 9 measurement properties in the three domains of reliability, validity, and responsiveness, were employed to evaluate the psychometric properties of the compassionate nursing care questionnaire [23].

In the qualitative phase of the study, 20 nurses (18 clinical nurses and 2 nurse instructors), 8 patients, and 6 family caregivers were selected via purposeful sampling and according to the inclusion criteria from various departments (internal, surgical, emergency, CCU, ICU, and hemodialysis) of university hospitals. After being selected, the subjects were interviewed individually. In addition, two focus group interviews were conducted with one group consisting of 6 nurses from internal, emergency, CCU, and ICU departments and another consisting of 6 nurse instructors.

The inclusion criteria for the nurses were having at least a bachelor’s degree in nursing; working in a fixed ward; not being in charge of a critically-ill patient; suffering from physical or emotional fatigue as a result; having manageable workload and appropriate physical and mental status, and being prepared for a 45-min interview as confirmed by the interviewee.

The inclusion criteria for the patients were being over 18 years old, being hospitalized at least for 3 days, being in good physical conditions (being able to walk to a private place to be interviewed for 45 min), not having taken a sedative or any other medicine which influences consciousness, not having a history of a known psychological disorder, and being declared by their nurses to be physically and emotionally fit for an interview.

The inclusion criteria for the family caregivers (family members or relatives) consisted of being over 18 years old, being actively involved in their patient’s care, not having a known metabolic or psychological disorder, not having taken any medicine which affects the mind, not being physically or emotionally fatigued as a result of caring for their patient, and being prepared for a 45-min interview as confirmed by the interviewee. It was also necessary that the subjects be willing to participate and answer the questions to be included.

Data were collected through individual interviews, focus interviews, and field notes. Accordingly, 34 in-depth, semi-structured interviews were conducted with 34 participants on a face-to-face basis. Also, 2 focus group interviews were conducted with 2 groups consisting of 6 clinical nurses. All the interviews were carried out in the lecture halls of the hospitals or the nursing school with prior arrangements with the participants. The individual and focus group interviews lasted 45–70 and 60–90 min, respectively. The researchers also carried out observations in the hospital departments. Following each interview, the nurses’ interactions with the patients and family caregivers were observed and recorded. Moreover, during the interviews, the interviewees’ non-verbal communication was noted. Each field observation session lasted from 2 to 8 h and all the work shifts -morning, afternoon, and night- were included. In total, 6 observations, which lasted about 48 h, were carried out. The observations consisted of descriptions of the subjects, events, and nurses’ interactions with the patients and family caregivers.

Each interview (with the nurses, patients, and family caregivers) began with the general question “What is your understanding of the word “compassion?” followed by more specific questions. The specific questions for the nurses included: “What are your experiences of compassionate nursing care?”, “What are some examples of your caring behaviors which demonstrate compassion?”, “Can you talk about the role of compassion in your caring for patients during a work shift?”, “How does compassionate care influence your interactions with the patients and their family caregivers?”, and “When you speak about compassionate care, what are you reminded of?”

The specific questions for the patients included: “What are your experiences of compassionate nursing care during your stay in the hospital?”, “How do you feel when you receive nursing care combined with compassion?”, “How do you feel when you receive nursing care which is not accompanied by compassion?”, and “Based on your experiences, how do you define compassionate nursing care?”

The specific questions for the family caregivers included: “What are your experiences of compassionate nursing care during your patient’s stay in the hospital?”, “What is an example of compassionate nursing care given to your patient?”, “How do you feel when your patient receives nursing care combined with compassion?”, “How do you feel when your patient receives nursing care which is not accompanied by compassion?”, and “Based on your experiences, how do you define compassionate nursing care?” The researchers also asked some follow-up questions, e. g. “Can you explain further?”, “What do you mean by that?” and “Can you give an example?” in order to collect more information toward reaching the research objectives.

To ensure the rigor of the data collected in the qualitative phase, the researchers used the criteria suggested by Lincoln and Guba [24]. For credibility, the researchers used prolonged engagement with data, member checking, peer debriefing, triangulation of individuals (nurses, patients and family caregivers of different genders and age groups), and maximum variation sampling based on contrasting evidence. Dependability and confirmability were ensured through checking the accuracy of the transcripts and the extracted codes and categories by a panel of experts. To increase transferability, the researchers provided accurate and comprehensive descriptions of the concept in question, the participants’ characteristics and the manner of data analysis along with documented examples of the participants’ statements.

At the end of the qualitative phase, items for the questionnaire were developed based on the collected data. Next, using this template, the researchers created a pool of items based on the domains and sub-domains of the concept of compassionate nursing care (inductive approach). Also, the researchers conducted a review of literature and relevant questionnaires (deductive approach). The research team then merged the overlapping items, and the initial 80-item version of the questionnaire was considered for psychometric analysis. The initial draft was designed as a self-report questionnaire for measuring the nurses’ compassionate care.

The second phase of the study was an assessment of the psychometric properties of the instrument. Face and content validities were measured using qualitative and quantitative methods. Also, construct validity and reliability were measured.

For qualitative evaluation of face validity, the researchers assigned 12 nurses with different specialties, 3 nurse instructors, and 2 language experts to evaluate the items in terms of difficulty level, ambiguity, and syntax in face-to-face interviews. Their comments resulted in additions to the questionnaires’ content, but no item was deleted. Next, the quantitative method, based on the impact scores of the items, was used to evaluate the validity of the questionnaire. Accordingly, 10 nurses (2 from ICU, 2 from CCU, 1 from hemodialysis, 3 from internal, and 2 from surgical departments) who were working in the hospital were asked to comment on the importance of each of the items on a 5-point Likert scale (5 = very important, 4 = important, 3 = relatively important, 2 = not very important, 1 = not important at all). The impact score of each item was calculated and scores more than 1.5 were considered to be satisfactory [25]. The impact score coefficient was calculated using the formula below:

To test the content validity qualitatively, the researchers assigned 15 expert nurses (10 clinical nurses who were in practice in special care, internal and surgical departments and 5 doctor nurses who had extensive knowledge and experience in the field of instrument development and nurse education) to evaluate each item in terms of syntax, use of appropriate words, placement of the items and scoring and record their detailed comments in writing.

For quantitative evaluation of content validity, the content validity ratio (CVR) of each item was calculated to determine the necessity of that item. The content validity index (CVI) was used to examine the relevance of each item to the concept of compassionate care [26]. The Kappa coefficient for measuring agreement between the evaluators was calculated using the total content validity index (S-CVI) [27]. Content Validity Ratio (CVR) was rated on a 3-point Likert Scale (necessary, useful but not necessary, not necessary). According to Lawshe’s table, items with a score equal to or greater than 0.49 were retained [28]. The CVR of each item was calculated using the formula below:

Content Validity Index (CVI) was calculated through Waltz and Bausell’s (2010) approach. Accordingly, 15 expert nurses were asked to evaluate the items in terms of relevance, simplicity and clarity on a 4-point Likert scale. The cutoff point for the CVI was set at 0.78 and higher [28]. Additionally, the Kappa statistics were calculated to determine the extent of agreement between the evaluators [27]. The mean of the content validity index (S-CVI) was used to calculate the total content validity index (S-CVI) [29]. The CVI of each item was calculated using the formula below:

Item analysis was performed to assess the Cronbach’s alpha coefficient for initial reliability and identify the items that affected the reliability [30, 31]. Before exploratory factor analysis, the items were analyzed with a sample of 40 clinical nurses who were selected via convenience sampling from special care (ICU, CCU, and emergency), internal and surgical departments. Most studies suggest a sample size of 30–50 subjects for item analysis [32]. The purpose of item analysis is to determine whether the items in a questionnaire are relevant to what it has been designed to measure or not [33]. Item analysis also aims to assess the correlation coefficients between the items: if an item does not have a correlation coefficient of at least 0.2–0.3 with at least another item, it should be omitted [34]. If an item has a correlation coefficient of more than 0.7 with another item, one of them should be omitted or they should be merged. Items with a total correlation coefficient of under 0.3 can also be omitted [35].

In this study, the construct validity of the questionnaire was assessed via exploratory factor analysis. To determine the required sample size for factor analysis, 5–10 people per item have been recommended though larger sample sizes have also been suggested [36].

Boateng et al. (2018) suggest that the minimum sample size should be 300 to 450 subjects [37]. In the present study, 450 nurses were selected from different departments of university hospitals via convenience sampling. To collect data, the first researcher (BT) visited various departments (surgery, internal, ICU, CCU and emergency) of the hospitals on different days and at different shifts (morning, afternoon, and night). After obtaining permission from the supervisors and head nurses, the first researcher asked the nurses who met the inclusion criteria and were willing to participate in the study to complete the self-report compassionate care questionnaire. Response rate was 93.33%; of the 450 qualified nurses, 10 refused to participate due to work overload and fatigue, 6 were not willing to participate, and 14 failed to answer all the items on the questionnaire. Thus, in the end, 420 questionnaires were available for data analysis. The study population consisted of all the nurses who were in practice in the above-mentioned departments.

The inclusion criteria for the nurses were having at least a bachelor’s degree, willingness to participate in this research, those with manageable workload, and those in good physical and mental status. Those who failed to complete the questionnaires were fully excluded.

The exploratory factor analysis was performed using the Kaiser-Meyer-Olkin Index (KMO) and the Bartlett’s test of sphericity, main component analysis, scree plot and varimax Rotation with a sample size of 420 nurses. To determine the number of constructs, the researchers used initial eigenvalues and scree plot [38]. In the next step, exploratory factor analysis was performed using varimax rotation. The factor loading of each item in the factor matrix and the rotated matrix should be at least 0.4 [39].

In the second stage of evaluation of construct validity, to assess the final model of the factor construct of the questionnaire, the researchers conducted confirmatory factor analysis with a second sample consisting of 300 nurses. The analysis was completed using means and variance-adjusted weighted least square (WLSMV) in Mplus 6.1.

Confirmatory factor analysis is based on a theory and hypothesis test about the factor construct in question and is usually performed after determination of the correlation matrix or factor construct. In the present study, the most common goodness of fit models based on the accepted threshold were considered. The Chi-square goodness of fit, root mean square error of approximation (RMSEA), Tucker-Lewis Index (TLI), and comparative fit index (CFI) were calculated [40].

To evaluate convergent validity, we simultaneously distributed the present questionnaire and Caring Behaviors Inventory (CBI-42) developed by Wolf et al. (1998) among 100 nurses selected via convenience sampling from various departments (emergency, CCU, ICU, internal and surgery) of the university hospitals, and the correlation between their scores was measured.

The internal consistency and stability of the questionnaire were measured to assess its reliability. Internal consistency was assessed with a sample of 420 nurses. A Cronbach’s alpha of 0.7 to 0.8 indicated satisfactory internal consistency [31]. The test-retest method was used to assess the consistency of the questionnaire with 50 nurses over a two-week interval. The scores of the two tests were determined by calculation of the intra-class correlation coefficient (ICC) for each of the sub-domains and the whole questionnaire. Burns and Grow (2014) recommend that the stability of a questionnaire should be assessed over a period of 2 weeks in a month [41]. An ICC index rating of above 0.8 confirmed the stability of the instrument [42]. The ease of use of the questionnaire, as well as the ceiling and floor effects, were also studied. The latter were investigated using the same sample size used for evaluation of construct validity (420 subjects). Data analysis was performed using SPSS v.16.

After the CVR of the items had been calculated, 40 items had scores of more than 0.49 and were, therefore, retained. Given the cutoff point of 0.78 for the content validity index, no item was deleted as all of them were found to be above the minimum.

Also, the Kappa coefficient score for the 40 items was excellent. Mean score of the content validity index (SCVI/Ave) was 0.91, which is considered to be excellent.

Before exploratory factor analysis, item analysis was performed with a sample size of 40 nurses. The reliability of the questionnaire was found to be 0.94 as measured through calculation of the Cronbach’s alpha coefficient. The results of item analysis to assess the correlation coefficients between the items and the total score led to omission of 2 items as decided by the research team. Then, the reliability of the questionnaire increased to 0.94. Thus, 38 items were left. As for the other items, each item correlated with at least one other item (0.2–0.3), and no item was deleted.

The mean age of the nurses was 27.58 ± 6.17 years and their mean work experience was 6.26 ± 4.77 years. The correlation coefficient of each item with the other items on the questionnaire was found to range between 0.21 and 0.7 and none of the items had a correlation coefficient of more than 0.7 with another item. Accordingly, none of the items was omitted or merged with another item in this stage. The correlation score of all the items with the total score of the questionnaire was found to be above 0.30.

To examine the convergent validity of the instrument, the researchers simultaneously distributed the present questionnaire and the Caring Behaviors Inventory (CBI-42) developed by Wolf et al. (1998) among 100 nurses and measured the correlation between the respondents’ scores [43]. The correlation between the scores obtained with these two questionnaires was moderate (p < 0.001 and r = 0.68). The means of age and work experience of the nurses who participated in this stage were 28.3 ± 4.69 and 7.29 ± 5.77 years, respectively.

To assess the ease of use of the questionnaire, we calculated the average length of time needed for completing the questionnaire and the percentage of individuals who did not respond to each item. The average time needed to complete the questionnaire was found to be 4 min, with a range of 3–5 min. Also, for all items, the non-response rate should be 0–5%, as was the result with the present questionnaire.

The final version of the self-report compassionate nursing care questionnaire consists of 28 items. All the items are scored positively on a 5-point Likert scale: “always” (5 points), “often” (4 points), “sometimes” (3 points), “rarely” (2 points), and “never” (1 point) (Additional file 2). The score range of the questionnaire is between 28 and 140. The score range of each subscale is as follows: professional performance = 9–45, continuous follow-up = 6–30, patient-centered performance = 7–35, and empathetic communication = 6–30. Based on the differences between the subscales, the scores in the low third (28–65) are considered as poor, the scores in the middle third (66–103) as average, and those in the high third (104–140) as satisfactory.

The final version of the questionnaire consisted of 28 items on a 5-point Likert scale. Thus, the highest and lowest possible grades were 140 and 28 respectively. On a three-section range, evaluation of the scores was determined as follows:

Accordingly, the cut-off point for the present questionnaire was set at approximately 37; this amount was added to the minimum score (28) to determine the ranges. Thus, the scores in the lower third (28–65) were considered to be poor, those in the middle third (66–103) were regarded as average, and the scores in the top third (104–140) were considered to be satisfactory.

The development and evaluation of the psychometric properties of the present questionnaire were conducted in Iran. Therefore, it is recommended that further studies be carried out in other cultures, languages and contexts for cultural adaptation and more accurate evaluations of the reliability and validity of the questionnaire.