A Spanish adaptation of the Quality in Psychiatric Care—Inpatient (QPC-IP) instrument: Psychometric properties and factor structure

Recent decades have seen a growing interest worldwide in measuring the quality of healthcare, not least in relation to mental health services [1]. However, these developments have not taken place to the same extent in all countries, nor even in different regions of the same country. One reasons for this is the lack of standardized measurement instruments for identify areas in need of improvement and for making national and international comparisons [2]. It has been observed in a recent systematic review [3] that quality instruments in mental health have psychometric properties with highly variable results and it is recommended to take into account those with high quality standards of results. For example, in the area of mental health the Quality of Psychiatric Care–Inpatient (QPC-IP) instrument [4] was developed with the aim of reduce this gap. The QPC-IP is part of a family of instruments that comprise both a common core and context specific: the QPC-OP for the psychiatric outpatient care [5], the QPC-FIP for psychiatric forensic inpatient care [6], the QPC-DA for daily activities in community-based services for people with psychiatric disabilities [7], and the QPC-H for the quality of community housing support [8]. Each QPC instrument also has a staff version, of which the one for psychiatric forensic inpatient care (QPC-FIPS) [9] has currently been validated. Adaptations of the QPC-FIP have been carried out in Denmark [10] and the QPC-IP and QPC-IPS in Indonesia [11, 12].

In Spain, legislation on the quality of public healthcare (Law 16/2003) has led to the creation of regional bodies with responsibility for assessing the quality of health services, especially mental health. In Catalonia, the Agency for Health Quality and Assessment (AQuAS) fulfils this role using validated satisfaction instruments. However, the concept of patient satisfaction does not encompass all aspects of quality of care. Questionnaires of satisfaction often involve questions that reflect the concerns of managers, such as health outcomes, rather than focusing on aspects which patients themselves might see as important [13]. Indeed, patient satisfaction is not necessarily synonymous with quality from the patient’s perspective [14], since measuring satisfaction in terms of service indicators is not the same as exploring patients definition of quality of care and what enable their recovery [15]. The essential components that make up this quality are the therapeutic setting, the therapeutic relationship and support, assessment, professional performance, assessment of practice, and environmental health [16].

Faced with this problem, the QPC-IP has the potential be a useful tool for assessing the quality of mental health services in Spanish-speaking countries. However, the QPC-IP was developed in the Swedish context, and according to the 6-D model of national culture [17], Spain and Sweden are culturally distinct, especially on the dimensions of uncertainty avoidance and masculinity-femininity, although they are similar in terms of power distance, individualism-collectivism, and long-term orientation. Given these cultural differences, a rigorous adaptation of the instrument is required, including analysis of its psychometric properties.

This research is part of an extensive study designed to adapt the instrument to other international contexts, analyze the psychometric properties and dimensions of the different instruments of the QPC-IP, and detail and compare the quality of hospital mental health care in the different territories.

The adaptation process and the psychometric evaluation was carried out in the context of two psychiatric services in Catalonia, one in the city of Barcelona and the other in the nearby town of Sant Boi de Llobregat. Meetings were held with staff to inform them about the purpose and nature of the study. Nurses and occupational therapists recruited patients who were willing to participate and who met the inclusion criteria. After obtaining informed consent, the staff instructed the patients on how to complete the questionnaire. Patients were included consecutively until the required sample size was reached.

The study was approved by the Clinical Research Ethics Committee of our hospital (CEI PIC− 128 − 15) and permission was granted by the coordinators and supervisors of the respective psychiatric units. All questionnaires were confidential, and all the patients signed informed consent in accordance with existing Spanish legislation. Their participation was voluntary.

The purpose of this study was to develop and examine the psychometric properties and factor structure of a Spanish adaptation of the QPC-IP instrument. The results show that the Spanish QPC-IP has adequate psychometric properties in terms of internal consistency, temporal stability (test-retest), content and construct validity (confirmatory factor analysis) and convergent validity. The CFA revealed an adequate fit of the six-factor structure consistent with the original Swedish version.

It was determined that the instrument was adequate and could be self-administered after conducting a pilot test and a cognitive interview with 30 patients. The results of this phase were positive and there were no problems in the comprehension or administration of the questionnaire. Although 13.3% of the sample did not complete comprehensive school, no support was needed for the patients to complete the instrument.

Internal consistency was acceptable (α ≥ .70) for the total scale and for five of the six dimensions. The highest alpha value was found for the Encounter dimension and the lowest for Secluded environment dimension (α = .67). Since Cronbach alpha is highly influence by the number of items in a dimension, the low alpha value of Secluded environment is probably due to the fact that it only include three items. However, the internal consistency of Secluded environment in other language versions of the QPC-IP, such as the Indonesian one [12], also shows a low Cronbach alpha value, suggesting that this dimension might be more cultural specific than the other dimensions in the QPC-IP. Overall, the magnitude of the alpha values for individual items showed a reasonable degree of variation, suggesting that none of the items was perceived as confusing or upsetting. It should be noted that the alpha values are adequate according to established criteria [23] and are higher than those reported for the original version [21] and in other studies that have used the OPC-IP [11]. It is also worth pointing out that the reliability results obtained for the Spanish QPC-IP are similar to other quality of care measures reported in a recent systematic review [3]. Specifically, instruments yielding similar results to those of the Spanish QPC-IP are the SEQUenCE (Service user Quality of CarE) instrument [28], with a Pearson correlation coefficient of .65 and a Cronbach’s alpha of .87, the MQOC (Menninger Quality of Care) measure [29], with an alpha of .92, and the QPC-IPS with an alpha 0.92 [30].

Regarding test-retest reliability, the value of the intraclass correlation coefficient obtained in the present study (ICC = .69) was acceptable. The Discharge dimension (ICC = .681) showed the lowest ICC values. One of the reasons for the low ICC values may be that patients discharge from ward were not scheduled in advance.

The convergent validity of the Spanish QPC-IP was examined by calculating the Spearman rho correlation coefficient with respect to a 10-point numerical satisfaction scale. The result was interpreted according to the criteria proposed by Martínez González, Sánchez Villegas, Toledo Atucha, and Faulin-Fajardo [31], namely: a value of zero, no correlation; <.30, weak association, ≥.31 ≤ .70, moderate association; >.71, strong association. The value obtained (rho = .49, p < .001) indicates a moderate association, showing that the higher the level of satisfaction the more positive is the patients’ rating of quality of care.

Regarding construct validity, the CFA yielded a six-factor model consistent with the original QPC-IP [21], thus confirming the multidimensional nature of the concept of quality [32]. All the item loadings were above .30, which is considered an acceptable minimum [33]. The goodness-of-fit indices also indicated a reasonably good fit of the model [25]. Overall, the results of the CFA support the validity of the Spanish QPC-IP and are in line with those reported for the QPC-IP [21], the QPC-OP [5] the QPC-FIPS [32] the Danish QPC-FIPS [34], and the Indonesian QPC-IP [12] and the Indonesian QPC-IPS [11].

It should be noted, however, that some differences were observed with respect to the recently validated Indonesian adaptation of the QPC-IP [12], possibly due to cultural differences with respect to our country.

However, given the present results, we can see that the patients in this study rated the highest quality in the Encounter dimension, which is in the line with previous studies on inpatient psychiatric care [12, 21]. The Secluded Environment was rated the lowest in terms of quality of care whereas in the study by Schröder et al. [21] performed in Sweden the Secluded Environment dimension was the second highest in terms of quality. One possible explanation may be that in Spanish psychiatric units there are no individual rooms available, except for isolation rooms for emergency crisis. However, it should be noted that the Secluded environment dimension in the Spanish QPC-IP fitted the original Swedish model whereas in the study by Lunqvist et al., [12], performed in Indonesia, the Secluded Environment dimension did not fit the model of the original scale. This result indicates that the concept of secluded environment is fairly similar in Spain and Sweden but distinctly different between Indonesia and Sweden, and possibly Spain too.

The extensive validation process in adapting the QPC-IP to the Spanish situation is a strength of the present study. However, a number of limitations also need to be acknowledged. First, although we estimated the minimum sample size needed for the present analysis, further studies with larger samples are required to confirm the factor structure obtained here. Second, the instrument has been developed with the specific purpose of assessing quality of care from the psychiatric inpatients’ perspective. Further studies are therefore needed to adapt the QPC instrument for use with Spanish-speaking populations in other contexts, such as outpatients. It would also be of interest to analyze the predictive capacity (sensitivity and specificity) of the Spanish QPC-IP instrument on patient recovery.

The Spanish QPC-IP is a simple and easily administered tool for measuring various aspects of quality in psychiatric inpatient care from the patients’ perspective. The six-factor structure and psychometric property of the Spanish QPC-IP are consistent with those of the original instrument, supporting its use as a measure of quality of care in Spanish-speaking populations. In this respect, it has the potential be used in cross-cultural comparative studies of quality of care in mental health. The results of these studies can be used to improve the quality of the provided service. Future studies will need to look at the psychometric properties of this instrument in relation to other variables and other samples of patients, both in the community and in other settings.