Assessing the Consultation and Relational Empathy (CARE) Measure in sexual health nurses’ consultations

The overall aim of the current study was to determine the validity and reliability of the CARE Measure as a tool for assessing patients’ views on the quality of healthcare encounters with sexual health nurses working in drop in clinics that provide a range of integrated specialist genito-urinary medicine and reproductive health services [36]. Nurses providing care in ‘drop in’ services are less likely to have serial consultations with patients and hence have limited opportunity to establish empathic relationships with them over time. The specific research questions addressed in the study were:

This was a cross-sectional questionnaire study. The study setting was the public specialist genito-urinary medicine and reproductive drop-in clinics of one Scottish Health Board. All 70 nurses working in the service were invited to participate. The nurses were introduced to the study during a staff meeting that the researchers (AB and BF) attended and an email about the study was sent to all nurses working in the clinics. Recruitment stopped when 20 nurses (from eight clinics) volunteered to participate. All participating nurses gave written, informed consent. The intention was to collect 50 patient questionnaires for each nurse. This number was based on previous studies in doctors that showed that 40–50 questionnaires were required to give a reliable estimate of mean practitioner score [24]. As it was not always possible to determine beforehand whether a patient would consult a nurse who was participating in the study, the receptionists gave questionnaires to all patients who reported at reception. Participating nurses were then asked to write their study identification number on the questionnaire at the start of each consultation so that the questionnaire could be linked to the nurse.

The study questionnaire was similar to the ones used previously. A description of the questionnaire including the CARE Measure is shown in Table 1.

Patient participation in the study was voluntary and the questionnaire did not elicit any identifiable information about patients or any information about their medical conditions. As in previous studies the patients were asked to fill in the questionnaire immediately after their consultation with the nurse and place it in a sealed box in the waiting area. Data collection was between April 2013 and October 2013. In 2013 the number of visits across all the clinics of the service was 110,300 made by 60,800 individuals (of which 70% female).

Through descriptive statistics the data were summarised, the CARE Measure and Patient Enablement Instrument (PEI) scores calculated, and the variability in the data assessed. Differences between groups were assessed through non-parametric tests, because the data distributions were skewed. To test whether the CARE Measure measured relational empathy in the way that the Measure was designed to measure this concept we assessed its validity in sexual health nurse consultations in several ways. The perceived relevance and face validity [37] (i.e. whether the patients perceived the CARE Measure to measure empathy) of the CARE Measure were examined by calculating the number of not applicable and missing values for each of the ten CARE Measure items, and the patients’ rating of the importance of the ten items. Construct validity [37] (meaning the extent to which the CARE Measure predictably corresponds or not corresponds with other constructs) was assessed through factor analysis (principal component analysis with varimax rotation and Kaiser normalisation and correlations (Spearman’ rho) between the CARE Measure items, and PEI (divergent validity, meaning expect no relation between the constructs) and the overall satisfaction measure (convergent validity, meaning expect the constructs to relate). To assess that the CARE Measure can be interpreted consistently with sexual health nurse consultations we assessed the reliability of the CARE Measure. Internal reliability of the CARE Measure was examined by using Cronbach’s alpa that calculates the inter-correlations of the 10 CARE Measure items. Finally, the inter-rater (patient) reliability of the CARE Measure in providing stable levels of relational empathy to sexual health nurses was assessed by Generalisability-theory (G-Theory). The software packages SPSS (version 21) and urGENOVA via its associated wrapper program GS4 [38, 39] were used to analyse the data as was done in our previous studies [25, 26].

In total, there were 238 ‘not applicable’ and 9 missing item responses, representing 2.6% and 0.1% respectively of the total number of responses for all 10 items. The higher number of ‘not applicable’ responses was mostly explained by two items which asked patients to rate how [good] the nurse was at ‘helping you to take control’ (item 9) and ‘making a plan of action with you’ (item 10) (Table 3).

Almost three-quarters (72%) reported that the CARE Measure items were very important to the consultation. The perceived relevance of the Measure increased with age (p = 0.020) (Table 4). No significant association was found in perceived relevance of the CARE Measures scores in relation to the other patient characteristics.

Factor analysis on the 10 CARE Measure items, 6 PEI items and 3 satisfaction measures (overall satisfaction, satisfaction with waiting time, and satisfaction with consultation time) identified three factors (Table 5). The CARE Measure items loaded on one factor with high loadings (0.804 to 0.891), suggesting that the Measure has a robust internal structure. The PEI items loaded on the second factor, and the satisfaction measures on the third factor. The three factors explained 73.2% of the variance.

The CARE Measure correlated moderately with overall satisfaction 0.377, p = 0.001, thereby supporting convergent validity. As anticipated the CARE Measure correlated less with (divergent) patient enablement (Spearman’s rho 0.232, p = 0.001).

The Cronbach’s alpha coefficient of 0.95 indicated high internal reliability of the CARE Measure. Nurses achieved very high CARE Measure scores that varied little between nurses (range of CARE Measure mean scores between individual nurses is 47.1–49.1). The very high scores and the consequent limitation of range impacted on the other forms of reliability analyses beyond that of internal consistency. For example, the analysis of inter-rater (i.e. inter-patient) reliability is dependent upon variance in nurse scores for its numerator. Given a limited range of mean individual question scores for the participating nurses it was not possible to show reliability dependent on differences between nurses. As a result, the inter-rater reliability could not be established in the current study.

As anticipated, the number of patients who had seen the sexual health nurse previously (17%) was much lower than those in a similar study with practice nurses (76%) who are more likely to have consultations with the same patients. However, the mean CARE Measure scores for all sexual health nurses was slightly higher (47.8) than in a similar study with practice nurses (45.9) [23]. It was also higher than the CARE Measure scores with GPs (40.9) [24].

Face validity was similar to that found in previous studies, with high level of perceived relevance of the CARE Measure items to practice nurse consultations (73%) and GP consultations (76%), and low numbers of not applicable responses in both studies. Studies outside general practice [25, 26] and in international settings reported similar findings [27, 28]. That older patients (45 years or more) were more likely than younger patients to score the CARE Measure items as important was also found in previous studies [24] as well as that item 9 (helping you to take control) and item 10 (making a plan of action with you) had the highest instances of ‘not applicable’ responses [24‐26, 28, 30]. A possible reason for this could be that most participants rated themselves as healthy and that within the context of their visit as healthy individuals they did not perceive that they needed help to take control or that they did not need further follow up or intervention.

In terms of construct validity, the current study found lower correlations between the CARE Measure, PEI and overall satisfaction than in previous studies [7, 22, 23, 26]. The lower correlations are likely to be due to the limited variation in the CARE Measure scores in this sample. Moreover, in the current study, the CARE Measure could not differentiate between the individual nurses, while in a previous study with practice nurses the inter-rater reliability indicated that 60 patient questionnaires per practice nurse provided a stable basis to evaluate the quality of the encounter for educational and quality improvement purposes [23]. The estimated number of completed CARE Measure was lower (50 CARE Measure questionnaires) in the study with GPs because there was more variability in the data [24].

This means that less patients are needed in order to differentiate between the GPs on relational empathy. As mentioned before, in the current study there was hardly any variability in the CARE Measure data. Therefore, it was not possible within this sample to calculate the required number of patient questionnaires on which feedback to the individual sexual health nurse practitioner should be based.

Previous studies have consistently demonstrated a strong association between continuity of care and high CARE Measure scores. In the present study, sexual health nurses who are less likely to have serial encounters with patients than other practitioner populations studied achieved higher CARE Measure scores. For the reasons previously discussed, it is possible that continuity of care is not important if the purpose of the consultation is very focused for both patient and practitioner.

A benefit was that the study followed on from previous validity and reliability work on the CARE Measure [23‐26], thereby expanding the scope of the CARE Measure across different healthcare professions and settings. Limitations of the study related to the participating nurses being volunteers, and the participating nurses being asked to write their study identification number on the questionnaire at the start of each consultation. As discussed above, we were not able to ascertain how many patients refused to participate or were excluded by the reception staff and this may have introduced sample bias [37]. Another limitation was that only 15 out of the 20 nurses collected the required 50 questionnaires and this could have affected the inter-rater reliability as more questionnaires would have increased the possibility to be able to differentiate between the individual nurses [24].