Cross-cultural adaptation and validation of the Hospital Survey on Patient Safety questionnaire for a Chilean hospital

An instrumental study was conducted to adapt and validate the HSOPS 2.0 questionnaire [8] in a Chilean hospital. First, the translation and cross-cultural adaptation of the questionnaire was carried out. Second, the psychometric properties were studied. Specifically, content validity was studied with experts, and construct validity and reliability were analyzed using a cross-sectional study at the Carlos Van Buren Hospital. It is a highly complex hospital center belonging to the healthcare network of the Valparaíso-San Antonio health service that has 526 beds.

The Hospital Survey on Patient Safety (HSOPS 2.0) comprises 32 items grouped into ten dimensions. These dimensions are D1 Teamwork, D2 Staffing and Work Pace, D3 Organizational Learning-Continuous Improvement, D4 Response to Error, D5 Supervisor, Manager, or Clinical Leader Support for Patient Safety, D6 Communication About Error, D7 Communication Openness, D8 Reporting Patient Safety Events, D9 Hospital Management Support for Patient Safety, and D10 Handoffs and Information Exchange. The original version 2.0 from the United States has good psychometric properties and reliability, measured with Cronbach’s Alpha, which ranges between 0.67 and 0.89 [8]. The HSOPS V2 has been cross-culturally validated and adapted in hospital nurse populations from China [9], South Korea [10], Turkey [11], Indonesia [12] and Brazil [13], always obtaining good results regarding their psychometric properties. The AHRQ provided written permission via email (CRM:00350728) to use the HSOPS 2.0 in English and carry out cross-cultural adaptation to the Chilean population.

The original version of the HSOPS 2.0 was translated in the sequence: direct translation, reverse translation, and final version into the official language of the destination country, following the recommendations of the literature [16]. See Fig. 1 for a detailed translation sequence.

Firstly, the original version (OV) was directly translated independently by two bilingual translators whose mother tongue was the official language of the country of destination, obtaining the translation in the original language into the target language. Subsequently, the researchers and the questionnaire author met to discuss the two translated versions, taking the items from the original version as a reference. At this meeting, a single version translated into the official language of the destination country (VTD) was agreed upon.

Secondly, two bilingual translators whose native language was the language of the country of origin of the questionnaire performed the reverse translation of the VTD version into the original language of the questionnaire. The process was developed blindly and independently (without knowing the original version). Subsequently, researchers and translators agreed on the reverse translation version (VTI) in a meeting. Finally, each item from the OV and VTI versions was evaluated and compared to look for inconsistencies in the VTI version. Two bilingual people with experience translating health sciences texts developed this step. The pertinent modifications were made to those items of the VTD version, and the item statement was faithfully written in the original language.

Thirdly, the researchers met with the author of the original questionnaire to identify translation alterations and confirm the final translated version (VTF) [17]. Similarly, a pilot test and cognitive pretest were carried out on 35 nurses and nursing assistants to check the questionnaire’s understanding, response, coding, duration in time, and functioning [18].

The content validity of the VTF was studied through a group of 12 experts made up of 6 academic nurses with a master’s degree and at least five years of teaching experience in healthcare management areas or research and six nurses with at least five years of experience in quality management and patient safety units in the hospital setting. They evaluated the Final Translated Version of the HSOPS 2.0 questionnaire using a 4-level Likert-type scale and provided feedback on the wording. Content validity was evaluated using Polit, Beck, and Owen’s methodology, with an Item-Content Validity Index (I-CVI) score of over 0.78, indicating good validity [19]. I-CVI is the proportion of experts giving an item a relevance rating of 3 or 4. Also, the Scale-Content Validity Index (S-CVI) average (also nominated as S-CVI/Ave) was computed for each dimension and the whole instrument. S-CVI is calculated by averaging I-CVI scores for all items on the scale [20]. Two rounds of evaluation were conducted to achieve acceptable levels of content validity. The final version, translated into Spanish, is linked to this article in File 1 Supplementary Material.

A cross-sectional study was conducted to determine the construct validity and reliability of the Chilean version of the HSOPS 2.0. The study population comprised nurses and nursing assistants from the Carlos Van Buren Hospital in Valparaíso (Chile). Nursing assistants were included because they are vital members of nursing teams in Chile, working alongside nurses to provide care to patients [21]. In medical-surgical units, they are responsible for delivering basic nursing care to hospitalized individuals, including hygiene and comfort, monitoring vital signs, administering medications, providing oxygen, and preventing pressure ulcers. They always work under the supervision of the charge nurse.

The sample size was set between 5 and 10 subjects per questionnaire item, according to the recommendations in the literature. As the HSOSP 2.0 questionnaire has 32 items, a sample between 160-320 participants was considered sufficient. Data was collected between May and July 2021 through the Google Forms application sent by email from the hospital’s nursing care management subdirectorate to 370 nursing and nursing assistants employed by the hospital. The online form included the version of the HSOPS 2.0 obtained after content validity. The response period was 60 days, and a weekly reminder was sent. Forms that were not wholly answered were excluded. A total of 259 participants responded to the questionnaire, reaching a ratio of 8 participants for each item [22], an adequate response rate of 70% [23] and reaching a minimum sample of 200 participants necessary to apply factor analysis [24], following the recommendations of the literature. The sample (n = 259) consisted of 148 (57.1%) nurses and 111 (42.9%) nursing assistants. 90% (n = 233) were women, 9.3% (n = 24) were men, and 0.7% (n = 2) identified as Others. No questionnaire was excluded. Table 1 presents a complete descriptive analysis of the sample.

Data was collected between May and July 2021 through the Google Forms application sent by email from the hospital’s nursing care management subdirectorate. The online form included the version of the HSOPS 2.0 obtained after content validity. The response period was 60 days, and a weekly reminder was sent. Forms that were not wholly answered were excluded.

Construct validity was analyzed by applying a structural equation model to validate the model of the original instrument, composed of 32 items grouped into ten dimensions, with data from the Chilean cultural context [25]. The Diagonally Weighted Least Squares (DWLS) estimator was used as ordinal variables were involved. The overall fit of the model was evaluated using the chi-square ( \(\chi ^2;\chi ^2/df\le 2\) ) [26] and the goodness-of-fit indicators Comparative Fit Index (\(CFI>0.95\) indicates good fit) and Tucker Lewis Index (\(TLI>0.9\) indicates good fit) to compare the existing model with a null model.

Additionally, Root Mean Square Error of Approximation (\(RMSEA<0.6\); considered acceptable) and Standardized Root Mean Residual (\(SRMR<0.08\) considered acceptable) were used to confirm whether the sample fit the proposed model [25]. Furthermore, the collinearity between the dimensions was studied with the covariance matrix.

Because the adjustment of the original model did not obtain satisfactory values, the search for a modified structure began, for which it was decided to perform an exploratory factor analysis. To ensure that the data met the assumptions required for this procedure, the Kaiser-Meyer-Olkim (KMO) test was used for the instrument, and each variable, an overall Measure of Sampling Adequacy (MSA) value close to 1, suggests that factor analysis would be useful because correlations between variables are strong. Also, variables with MSA lower than 0.8 were removed since they show weak correlations with the other variables. Additionally, Bartlett’s Test of Sphericity was used to determine whether the variables were sufficiently correlated to perform a factor analysis.

With these improvements in the original data, exploratory factor analysis was conducted to observe how the items were grouped into different factors or dimensions. The number of factors was determined with a parallel analysis [27]. A threshold of +1 factors was also considered according to the recommendations of Lim and Jang [28]. Factor loading was explored using the polychoric correlation matrix between items since they were ordinal variables. A factor loading greater than 0.3 points was considered to maintain an item in a particular dimension [24]. Finally, the new factor structure was confirmed by running a new structural equation model on the same sample. Reliability was studied with McDonald’s Omega for categorical variables (adequate reliability if \(\omega >0.8\)) [29]. Statistical analysis was performed with R statistical software version 4.0.5. The Laavan package version 0.6-9 was used for the analysis of Structural Equations [30], the psych package for the factor analysis [31], and the semTools package for reliability analysis [32].

The study was favorably evaluated by the scientific, ethical committee of the Valparaíso San Antonio Health Service in Chile on August 21, 2019, with record 04/2019, and by the deontological commission of the Universitat Jaume I on September 13, 2019, with file CD /43/2019. In addition, the ethical considerations provided in Law 20,585 on access to public information in Chile [33] and the principles of the Declaration of Helsinki [34]. Both the expert evaluators and all the nursing team participants electronically signed the informed consent, previously clarifying that their participation was completely voluntary and anonymous.

The items that required greater consensus due to the differences produced in the translation stages could be consulted in supplementary material Table S1. A brief comment on the solution adopted is indicated.

In the process of cultural adaptation, carried out by the group of consulted experts, nine items were modified, considering the suggestions of the cultural interpretation of some words or terms, in addition to the form of wording, for example, of written questions in negative that have a particular connotation in the Chilean population (See Table S2 in supplementary material). After this process, the final version of the HSOPS 2.0 was obtained, translated, and adapted to the Chilean context. The wording of the items, grouped into categories, can be consulted in Table 2.

The HSOPS 2.0 obtained excellent overall content validity with a S-CVI=0.982. Similarly, all dimensions obtained S-CVI higher than 0.94. Table 2 presents global content validity results by dimensions and items. It is observed how all the items obtained an I-CVI greater than 0.78, with universal agreement (ICVI=1) in 81.3% (n = 26) of the items. Similarly, five of the ten dimensions also gain universal agreement.

The goodness-of-fit indicators of the structural equation model indicated that the original structure of the questionnaire was no better than a model with 0 variance ( \(\chi ^2/df\le 2.26\), CFI = 0.781 and TLI = 0.741). In addition, the model’s residuals were analyzed to determine what percentage of the variance could not be explained by the model in the sample studied. It was observed that the model did not fit the data (RMSEA = 0.089; SRMR = 0.102). The details of the values of each item can be consulted in Table S3 of the supplementary material.

Because it was not possible to confirm that the original questionnaire structure was better than a model with zero variance (See supplementary material for details), an exploratory factor analysis was run. Previously, items A5, A6, A11, A13, A14, B2, F4, F5, and F6 were eliminated from the original version adapted to the Chilean context after running the KMO test because they had MSA values lower or equal to 0.8 (see also polychoric correlations in supplementary material Figure S1). After this improvement, the resulting dataset shows adequate for factor analysis since the KMO value for the instrument is 0.91. Also, Bartlett’s Test suggests that variables are sufficiently correlated (\(\chi ^2=3958.921\), \(df=496\), p-value \(<0.001\)). Additionally, Table 3 presents a descriptive analysis of items to analyze their distribution.

The exploratory factor analysis grouped the 23 items into seven factors that explained 71% of the variance. Figure 2 shows the factor loading diagram with seven factors.

Factor F3 explained 14% of the variance and grouped all the items (A1, A8, and A9) of the original dimension D1 “Teamwork” and two of the four items (A7 and A10) of the original dimension D4 “Response to Error”. Factor F4 explained 13% of the variance and grouped the same items (C1, C2, and C3) as the original dimension D6, “Communication About Error”. Factor F5 explained 11% of the variance. It grouped two items (B1 and B3) from the original dimension D5 “Supervisor, Manager, or Clinical Leader Support for Patient Safety”, in addition to items A12 and C6 that in the original instrument were in dimension D3 “Organizational Learning-Continuous Improvement” and in dimension D7 “Communication Openness”, respectively.

Factor F2 explained 9% of the variance and included all the items (F1, F2, and F3) of the original dimension D9 “Hospital Management Support for Patient Safety”. Factor F1 explained 9% of the variance and included all items (D1 and D2) of the original dimension D8 “Reporting Patient Safety Events”. Factor F6 explained 7% of the variance. It included two of the four items (A2 and A3) of the original dimension D2 “Staffing and Work Pace”, in addition to adding item A4, which was initially found in dimension D3 “Organizational Learning-Continuous Improvement”. Finally, factor F7 explained 7% of the variance and grouped three of four items (C4, C5, and C7) of dimension D7, “Communication Openness” of the original instrument. The factor loading diagram with seven factors can be consulted in Fig. 2.

Of the original instrument’s ten dimensions, dimensions 6, 7, and 8 were precisely preserved. Dimension 1 added half of the items from dimension four, while dimensions 5 and 7 remained the same except for one item that was eliminated or moved to another dimension. Dimension 2 retained half of its items, dimension three was distributed across other dimensions, and dimension 10 was eliminated. Table 4 presents the results of the exploratory factor analysis.

Subsequently, a new structural equation model was carried out, with the structure that resulted from the exploratory factor analysis, composed of 23 items grouped into seven dimensions (See Table 5). Goodness-of-fit indicators indicated that the proposed model was better than a model with 0 variances (CFI=0.995; TLI=0.994) and that the data adequately fit the model (RMSEA=0.048; SRMR=0.064).

Furthermore, the goodness-of-fit indicator \(\chi ^2/df=1.46\) (\(p < 0.001\)) also confirmed the model’s validity. Figure 2 presents the model using a path diagram that includes item variances, factor loading, and factor covariances.

The results of McDonald’s Omega indicated a high overall reliability of the instrument (\(\omega =0.9325\)). Cronbach’s alpha was also calculated for comparison with other studies (\(\alpha =0.9313\)). The validated HSOPS 2.0 instrument adapted to the Chilean population, with the items’ wording grouped into new categories, can be consulted in Table S4 of the supplementary material.