Spanish adaptation of the quality in psychiatric care-outpatient (QPC-OP) instrument community mental health patients’ version: psychometric properties and factor structure

At the international level, it has been observed that investing in the provision of services related to mental health (prevention, intervention and health promotion) produces benefits in terms of health and social factors. In addition, economic costs associated with mental health problems are reduced [1]. Current trends support a community model of mental health care which replaces long, expensive hospital stays with treatment of patients in their community so that they do not lose their family and social connections [2‐4]. These services should be developed following a recovery ethic and should focus on the person and their autonomy, along with the therapeutic relationship [2, 5]. Coordinated, comprehensive care of people with mental health problems in the community where they live is associated with a perception of greater satisfaction in this population [6].

Wilson and Daly, promoted the move away from the medical model of care to encourage a spirit of recovery in which patients and caregivers are recognized as equal partners in decision-making and certain aspects of planning, such as the provision of mental health services [7]. A focus on individual preferences and patient participation in decisions is vital to the development of a recovery-oriented system [8, 9].

Patients’ perceptions are relevant indicators when assessing interventions and the quality of the health service in question [10, 11] and, in some cases, the capacity to respond to their expectations is considered a performance parameter of the health system itself [12]. The area of community mental health is no exception, given that patients’ perceptions of the quality of psychiatric care received have become an important component in assessments [13] and have even been used as an indicator of quality [14].

Quality of care is a fundamental component of the right to health and essential in ensuring equity and dignity among patients of health services [15]. Services that meet quality of care criteria can lead to expected health outcomes [16] and better quality of life [17]. However, deficits in quality of care can contribute to dissatisfaction, lack of adherence and even an increase in patient mortality [16].

There is no widely accepted definition of quality of care. This term refers to a multidimensional concept [18] that is perceived by the mental health patient as positive [19]. As distinct from the term satisfaction, quality of care includes the perspectives of all parties involved [20]. It is important to conduct research into those factors that may affect the rating of psychiatric care received by patients, given that a better understanding of the factors that have a negative impact on such ratings will allow both professionals and the administration to improve the patients’ experience [21].

There is a lack of intercultural studies that compare the perceptions of professionals and patients on the quality of care [22] and this is mainly due to the very few standardized instruments with adequate psychometric properties available for the assessment of this care [23].

A recent systematic review reported that there is a scarcity of instruments for the evaluation of satisfaction and quality of psychiatric care that present an acceptable validation process and adequate psychometric properties [24]. Some instruments are based on the professionals’ point of view on care quality [25, 26], despite the fact that the perceptions of professionals and patients frequently differ on what constitutes good quality of care [27]. In fact, there are authors who question the validity of assessing the quality of psychiatric care without taking the patients’ perspective into account [28] as these patients are an essential component in the development of measurement instruments [29, 30].

The systematic review mentioned above highlights two instruments within the field of community mental health [24]. One is the Psychiatric Out-Patient Experiences Questionnaire (POPEQ) which assesses experiences solely from the patients’ point of view [29]. The other instrument is Quality in Psychiatric Care – OutPatient (QPC-OP) [31], an instrument of Swedish origin that evaluates the quality of psychiatric care in the community environment from the perspectives of both professionals (QPC-OP Staff) and patients (QPC-OP). This instrument forms part of a battery of Quality in Psychiatric Care tools which assess the quality of psychiatric care in community [31], hospital [32] and forensic settings [33]. The definition of quality used to create the instrument was developed from a phenomenographic study [19], assessed for apparent validity in a pilot study and tested in a sample of patients admitted to psychiatric hospitalization units in Sweden [30].

This study is part of a wider international project to adapt the QPC-OP patient version of the instrument in different countries, test its psychometric properties and the equivalence in dimensionality of the different versions by language, and describe and compare the quality of psychiatric care in the community setting in various countries.

The aim of this study was to adapt the Quality in Psychiatric Care Outpatient (QPC-OP) instrument into Spanish and analyze its reliability and validity.

To generate the adapted Spanish version of the QPC-OP instrument, a translation/back-translation process was carried out. Other studies of QPC instruments [53‐62] have obtained versions in other languages through a similar process. The results in this phase were satisfactory and no difficulties were observed with respect to degree of comprehension or administration of the instrument.

Regarding psychometrics, construct validity (CFA), internal consistency and temporal stability (test-retest) values were adequate.

The confirmatory factor analysis (CFA) carried out indicated that the Spanish version has the same 8 quality of psychiatric care factors as the original Swedish QPC-OP version [31] and no items required modification. In addition, an EFA was performed to determine the factorial model with the best fit to the Spanish sample analyzed, revealing that it could behave in a one-dimensional manner. The EFA also demonstrated an adequate fit for the model with the 4-factor structure in the sample analyzed.

Factor 1 in this new model groups items that make up dimensions F2.Participation-empowerment and F3.Participation-information of the original structure. This is reasonable as it brackets items related to users’ participation in their care.

Furthermore, factor 2 of the four-factor structure fully groups the items that make up F8.Accessibility of the original structure and an item from F4.Discharge. This makes sense as the item belonging to F4.Discharge measures the perception of the user regarding help received from professionals to look for work or other occupations and the items that form F8.Accessibility represent the potential difficulties that users perceive in contacting either the professionals or the center.

Separately, factor 3 entirely consists of items from F5.Support and F6.Environment, along with some items from F1.Encounter, F4.Discharge and F7.Next of Kin. This is logical as these items assess the user’s perception of safety and respect (not doing harm to oneself or others, feeling safe with the other users and with the center itself, professionals’ respect for users and their relatives, etc.).

Finally, factor 4 seems to include some items from F1.Encounter, F2.Participation-empowerment, F3.Participation-information, F4.Discharge and F7.Next of kin of the original structure. These items assess the user’s perception of treatment received (interest shown, understanding and concern for users’ feelings and care received) as well as respect in decision-making.

Analysis of the reliability of the instrument with the 8 original factors was performed using Cronbach’s alpha coefficient. Globally, the instrument showed a Cronbach’s alpha of 0.951, and in 5 of 8 factors values greater than 0.70 were obtained; an adequate value according to Nunnally and Berstein [63]. The global Cronbach’s alpha is identical to that of the original instrument [31], with a higher value than the Spanish professional versions in both the hospital and community settings [53, 59]. This value is also higher than the Spanish version for patients in the hospital setting [58], Indonesian hospital versions [56, 57], the original hospital version for professionals [64], and the community version for Norwegian professionals [65]. However, it showed a somewhat lower value than other versions of the QPC instrument [32, 33, 54, 66, 67]. Of all the factors, F7 (Next of Kin) showed the lowest reliability, as was the case in the original Swedish version [31]. This may be due to the fact that this factor consisted of only two items and despite the fact that these results were not optimal, it was considered a priority to preserve the original structure of the instrument in the Spanish version.

Temporal stability or test-retest was calculated within the original 8-factor structure. This value was not determined in either the original Swedish QPC-OP version [31] or in other original community, patient’s hospital or either forensic QPC versions [32, 33, 66‐68] or their translated versions [54‐57, 65], with the exception of the original hospital version for professionals [64], Spanish hospital versions [53, 58] and the Spanish community version for professionals [59]. The ICC for the total instrument and for each of the factors demonstrated good agreement [34, 37], with values higher than 0.70 except in F2 (0.589), F3 (0.689) and F8 (0.684). This global value is lower than that of the original hospital version for professionals [64] and that of the rest of the Spanish versions although it should be pointed out that these also showed values lower than 0.70 in some factors. Specifically, the Spanish hospital versions showed values lower than 0.70 in F6. Discharge [53, 58], while in the Spanish community version for professionals [59], it was F5. Support that showed a lower value than we would have preferred. This was not the case in the original hospital version for professionals, where all the factors showed values over 0.70 [64]. However, the four factor structure presented based on the EFA showed greater reliability in each of the dimensions analyzed.

As a limitation we should stress that it was not possible to assess the sensitivity to change or predictive validity due to the cross-sectional design of the study design. A potential limitation could result from the use of a traditional method to calculate the sample size although, given the complexity of the sample, a sample size of 200 participants was established to carry out the confirmatory factor analyses. It may be the case that the sample is not entirely representative of the study population although the participants were selected from a wide geographical area so that the results can be extrapolated. For the Spanish sample, the unidimensionality or the four dimensions model of the instrument can be considered, although studies with a larger sample size are required to determine whether the one or four-dimensional behavior is maintained. In fact, a larger sample size is highly recommended for further research in general. Another limitation, regarding recruitment, was due to the difficulties arising from the COVID-19 pandemic at health facilities. Future research should consider the collection of other variables that could influence the patient’s perception of quality in psychiatric care to ensure that all such factors are taken into account, as well as the type I intensity of the mental health disorder. Longitudinal designs are recommended for future research to assess sensitivity to change. The data necessary to allow calculation of convergent validity were not collected and this determination is recommended in future research. These limitations should be taken into account in the design of future studies.

The Spanish version of the QPC-OP instrument is a simple, useful tool for the measurement of various aspects of quality of psychiatric care from the perspective of patients in the community mental health setting. Its 8-factor structure and its psychometric properties are consistent and in accordance with the original version, thus allowing this instrument to be used to assess the quality of services provided from the perspective of Spanish-speaking patients. However, the model that best fit the Spanish sample analyzed revealed a unifactorial or four-factor structure that should be considered in future studies. We would like to emphasize the importance of understanding the perception of psychiatric care received in the community setting given the mainly voluntary connection that patients have with these services.