Development and psychometric testing of quality nursing care scale in Mongolia

We implemented a two-phase exploratory sequential mixed-method design to develop and validate QNCS-M. During the phase I of this project, we dedicated time and effort to develop an instrument merited on the 5-step proposed by DeVellis [12] while, phase II was dedicated to ascertaining psychometric performance of QNCS-M.

This phase began by drafting the blue-print of QNCS-M which was based on the overall concept of QNC, first proposed by Kunaviktikul [7]. However, the components of QNCS-M and its 5 dimensions were established based on the results of a previously implemented qualitative study in Mongolia and conducting a comprehensive literature review [13]. We then proceeded with establishing the operational definition of each of the five dimensions of QNCS-M. These operational definitions were unanimously agreed by the members of our research team. We defined the Physical Care as the interventions provided by the nursing professionals with the objective of symptom management and assistance with activities of daily life. The operational definition of Psychological Care was focused on activities provided by nursing professionals that encourage patients in assuming self-directed activities to relieve mental distress. We defined the operational operation of Emotional Care as supports and activities provided by the nursing professionals to understand and to support the frame of minds and emotional states of patients because of their illness. We explicated the operational definition of Social Care as the process of building positive social relationships with the patients and creating a positive and supportive ambiance to prevent depression among patients due to their sense of isolation. Finally, the operational of definition of Spiritual Care was established as recognition of and respect for patients’ religious beliefs and proclivity. In addition, we included the practice of acceptance and respect for patients’ cultural diversity by the nursing professionals in the operational definition of Spiritual Care. This step was followed by drafting and scaling of a total of 66 items in English language. Each item was scaled based on a five-point Likert scale, ranging 0–4. The sum of scores reflected the level of quality of nursing care with value of 0 indicating the lowest quality of nursing healthcare services, while the value of 264 indicated the highest possible level.

The English translated version of QNCS-M was reviewed and evaluated by a panel of seven academicians from US, Thailand and Mongolia with expertise in instrument development and content validity assessment. Per the recommendations of the panel, seven questions were deleted because the item-content validity index (I-CVI) < .78 [14]. Item-content validity indices (I-CVIs) of the remaining items ranged from 0.85 to 1.0. The revised version of QNCS-M was translated back to Khalkha dialect and was reviewed by a panel of 10 nursing academicians and directors in Mongolia. The panel assessed our instrument for its clarity, readability, accuracy and its length. The panel of reviewers flagged 12 items as ambiguous. These 12 items were revised and restructured to ascertain clarity in Khalkha dialect. The latest version of this 59-item instrument was pretested among 20 nursing professionals working in the State Central Third Hospital in Mongolia. The Cronbach’s alpha coefficients of the instrument was .94.

Assessment of the psychometric performance of QNCS-M consisted of two steps. During step one, we focused on field testing of the 59-item instrument for its construct validity and internal consistency reliability. Field Testing of Instrument was conducted in nine public hospitals in city of Ulaanbaatar. These hospitals with 984 strong nursing professionals are secondary and tertiary level hospitals and serve as the referral system for the entire country [15]. A total of 485 nursing professionals using multi-stage sampling technique were randomly identified and recruited to take part in our study [16]. First, we applied a proportional random sampling approach to determine the number of participants; our sampling approach was adjusted for the size of the nursing staff in each hospital. In the second step, proportionate stratified sampling method was used to select the number of nurses from different departments in each hospital. Finally, prospective study participants randomly were selected from the name list of each nursing department. Only those who met the eligibility criteria were recruited to participate in our study. We have outlined location of the participating hospitals in the city of Ulaanbaatar, Mongolia (Fig. 1, Additional file 1: Appendix A). Additionally, the list of hospitals and percentage of sample size from each hospital are provided in Additional file 1: Appendix A.

In calculating the total sample size, we assumed participant-to-item ratio of 7:1 [17], we further inflated our sample size by 18% to account for the sample attrition due to no-response. This approach was justified based on previous reports suggesting a sample size of 300–500 as adequate for factor analysis procedure [12, 18]; additionally, the relatively small number of the nursing professionals (n = 985) working in the nine public hospitals was another justification for our sample size calculation.

The nursing professionals were eligible to participate in our study if: A) They were actively employed by the public hospital system in the city of Ulaanbaatar; B) They were performing their professional duties in one of the nine accessible secondary or tertiary care hospitals in Mongolia; C) They had at least 1 year of work experience in direct patient-care health services and D) They were willing to participate in our study. Nurses who had participated in the pre-testing stage were not eligible to participate. This restriction was imposed to reduce the likelihood of information bias.

A trained research assistant introduced the objective of the study to the prospective study participants and obtained their signed informed consent forms. The research assistant then distributed copies of our 59-item QNCS-M and Marlowe-Crowne Social Desirability Scale (MCSDS) and Demographic questionnaires to the study participants and instructed them to return the completed instruments within 2-weeks in sealed envelopes in the drop-boxes which were placed in the Nursing Departments of the participating hospitals. Returned surveys were anonymous. This section of our study started in February of 2019 and was completed in December 2019.

We used descriptive statistics to summarize the demographic characteristics of the study participants and distributions of responses to each item listed in our 59-item instrument. Inter-item correlation was examined by using the Spearman rank-order correlation. Item-total correlation was calculated by using reliability analysis. The criterion for selecting qualified items to constitute a consistent scale included item-total correlation and a corrected item-subscale correlation of .30 or higher [18]. Inter-item correlations ranged from .30 to .70 [19]. The internal consistency reliability was analyzed using Cronbach’s alpha coefficient. A reliability coefficient above .70 is considered acceptable for a new scale [12]. Finally, we applied Spearman rank-order correlation used to assess the relationship between MCSDS score and each item of QNCS-M.

We then proceeded with calculating the Kaiser-Meyer-Olkin (KMO) value to ascertain the adequacy of our sample size. Additionally, we performed Bartlett’s test of sphericity to evaluate for item redundancy of QNCS-M [20]. The criteria for determining the factor solution of factor extraction included: 1) a factor with an eigenvalue of 1.00 or above, 2) item with a factor loading of .40 or above, 3) no factor with fewer than three items [21]. Finally, we performed exploratory factor analysis (EFA) to assess construct validity of QNCS-M. This approach permitted us to investigate if the internal structure of the scale of QNCS-M fit with the original theoretical definition and conceptual framework [7].

In this step, we assessed the empirical relationship between QNCS-M and the criterion tool, the Quality of Nursing Care Questionnaire-Registered Nurse (QNCQ-RN) [22]. The QNCQ-RN has been widely to assess quality nursing care instruments in hospital settings [23, 24]. Permission was obtained from the authors; QNCQ-RN was translated into Khalkha dialect and then was translated back to English to ascertain the validity of the translated version.

We recruited 54 nursing professionals from State Central First Hospital, Bayangol District Health Center and Bayanzurkh District General Hospital. We imposed three inclusion criteria:1) Providing healthcare services in the role of a nursing professional and 2) Having at least 1 year of work experience in direct patient-care health services and 3) Willingness to participate in our study. The sample size of 54 was based on the expected correlation coefficient value of 0.4, power of 80% and, alpha = .05; our calculation yielded a minimum sample size of 46 [25]; however, we inflated our sample size by 18% to account for non-responders.

The lead investigator introduced the objective of this segment of our study and copies of QNCS-M and the QNCQ-RN were distributed among the study participants. The study participants were instructed to return the completed instruments within 2-weeks in sealed envelopes in the drop-boxes which were placed in the Nursing Departments of the two participating hospitals. All the 54 participants returned their anonymous completed questionnaires; the returned questionnaires were evaluated by the lead investigator for completeness; data were entered into the databases for statistical analysis. To assess the validity scale of QNCS-M, we calculate the correlation between QNCS-M and the QNCQ-RN using the Spearman rank-order statistical technique [26]. Our calculation yielded a statistically significant correlation between the two instruments (r = .28, P < .03).

This study was approved by the Research Ethics Committee of the Faculty of Nursing, Chiang Mai University (Approval-020/2019), the Bio-Medical Research Ethics Committee of Mongolian National University of Medical Sciences, and the participating hospitals in Mongolia. Informed written consents were obtained from all participants. Confidentiality of study participants were respected by the research team. All personal identifiers were deleted after the final data quality and assurance. Data were stored in a password protected database behind the IT firewall of the Mongolian National University of Medical Sciences.

Of the 485 study participants, 456 (94%) returned their questionnaires; we excluded 16 (3.2%) of the returned questionnaires because of missing information. Questionnaires from a total of 440 (90.7%) participants were complete and deemed as acceptable for further analysis. The mean age of study participants was 36 years (SD = 10; range 22–59). The majority of the nurses were female (n = 431, 98.0%), and more than half of the subjects were married (n = 313, 71.1%). More than half of the nursing professionals (n = 262, 59.5%) who contributed to our study, had earned a bachelor’s degree in nursing science (Table 1).

The main strength of our study is its design approach which involved both the nursing professionals and patients in generation of items. Our study has two limitations. First, the instrument was validated only by the nursing professionals. Second the study was implemented in the hospital systems, owned and operated by the Mongolian government. Therefore, its applicability might be limited to other settings such as private for profit or not-for-private hospitals.

QNCS-M is the segue to improvement of quality of healthcare services delivered by the nursing professionals in Mongolia. Although QNCS-M was designed and developed for application in Mongolian society, this newly developed instrument can be utilized as a guideline in other countries with healthcare systems similar to Mongolia. Finally, QNCS-M can be used in academic nursing focusing as an educational tool in teaching the concept of quality nursing healthcare services.