Measuring nurses’ perception of work environment: a scoping review of questionnaires

Researchers refer to the work environment as, for example: working conditions, practice environment and job characteristics. In this review, we used Lake and Friese’s definition of the nursing work environment: “characteristics of a work setting that facilitate or limit nursing practice ([11] p.2)”.

A literature review conducted by Bae [7] synthesized various work conditions and their respective associations with patient outcomes. The work conditions were grouped in 10 concepts; autonomy, philosophy emphasizing quality of clinical care, nurse participation, supportive managers, collaborative relationships with physicians, collaborative relationships with peers, staffing, decentralization, patient-centred climate, and busyness. Bae’s synthesis indicated that there is some degree of convergence in the topics. All studies were conducted in acute hospital settings and seven out of eleven studies included in that review used a version of the Nursing Work Index (NWI) for data collection. The NWI is a frequently used questionnaire for measuring nurses’ work environments. It was first developed in the USA for hospital registered nurses in 1989 [12], and there are several versions adapted and revised for different settings and different contexts [13‐16]. It has been pointed out that the instruments’ properties are unstable [17, 18]. This is acknowledged by the authors of the NWI, who later developed the Essentials of Magnetism (EOM) [19] process measurement tool and subsequently its revised version EOMII [20, 21]. The EOM tool was developed using the 14 Forces of Magnetism [22] as a framework, together with an extensive participant observation and a qualitative interview study, making the tool reflect a more contemporary nursing practice and the practice environments [19].

Our overall goal was to find questionnaires to measure the work environment in long-term care but in the process of conducting the review, we expanded the criteria to include questionnaires that were used in acute care settings, because these questionnaires contain domains of interest that are also applicable to long-term care nursing work environments. We believe that our review is of interest and useful to stakeholders in other areas of nursing practice. In addition, when choosing topics for a questionnaire, it is necessary to prioritize in order to balance the response burden and information needs. The questionnaires identified in the review provided an excellent opportunity to map the work environment domains that were prioritized by a number of authors.

The research questions guiding this study were:

The description of work environment questionnaires of interest referred to in research question 1 is presented in more detailed under Screening – Inclusion and Exclusion.

A literature search strategy was designed with a basis in research question 1, and criteria described under Screening – Inclusion and Exclusion. The initial source was electronic databases, limited to articles published in peer-reviewed journals in the English or Scandinavian languages. The search was conducted with support from a research librarian. A test-search was first executed in order to identify relevant keywords representing the study topics. An extensive search was performed in October 2015, and updated in December 2016. The following databases were searched: Embase (1974-) Ovid MEDLINE(R) In-Process & Other Non-Indexed Citations, Ovid MEDLINE(R) Daily, Ovid MEDLINE(R) and Ovid OLDMEDLINE(R) (1946 -); PsycINFO (1806-); CINAHL and SweMed + .

We used the keywords and searched in title, index terms and author’s keywords. Several keywords in different combinations, endings, spelling, grammatical forms and synonyms were included in the extensive search. The search strategy was tailored to the best possible fit for each database. We provide the strategy used to search MEDLINE as an example (Table 1). The complete list of search terms can be found in Additional file 1.

The reference lists of relevant articles were manually searched for additional literature. This was followed by a “snowball” procedure: when a citation in an article appeared relevant, we read the cited article. Figure 1 shows the final extensive search process illustrated in a flowchart. Our searches in electronic databases and reference lists were supplemented by targeted internet searches. Based on our familiarity with the field, we screened internet sites and publications of organizations that had previously done work in these specific or neighbouring areas, e.g. the Norwegian Association of Local and Regional Authorities (KS), the Swedish Association of Local Authorities and Regions (SALAR) and the Agency for Healthcare Research and Quality (AHRQ).

All citations from the literature search were imported into an EndNote™ library, after which duplicates and conference abstracts were manually removed. After the study selection, the citations were imported into a spreadsheet and screened for relevance and quality.

The relevance of the studies in the search was assessed using a two-step screening process. The inclusion criteria in the first step were: a) Articles in which the questionnaire in use were tested on nurses working in direct and basic nursing care in general hospitals, nursing homes or home healthcare. That is, everyday nursing care that does not require special education, and that may be performed by less qualified personnel under registered nurses’ supervision. Hence, questionnaires that could be suitable for both registered nurses, practical nurses, and nursing assistants were of main interest. In this review, the term “nurses” include the three groups. b) Self-reported survey questionnaires constructed to measure work environment characteristics that are assumed to facilitate or limit nursing practice. This refers to nursing practice outcomes related to patient and/or organizational outcomes; the latter are highly relevant in human resource management. In order to be useful in quality development, the questionnaires of interest should describe characteristics of the work environment that are amenable to change. c) Articles presenting an assessment of the measurement properties of a questionnaire were included because measurement properties are central aspects when choosing a questionnaire.

Exclusion criteria included: a) Study setting was countries with health systems and cultures that differed greatly from Norway, such as Iran or China. b) Conference abstracts, books, reports and dissertations.

The first-step screening was conducted by the first author based on the titles in the publications. If more information was needed, the abstract was browsed. In the case of uncertainty regarding inclusion, the literature was included for further screening in the second step.

In the second step, the eligibility criteria were determined on a post-hoc basis, as the two authors independently judged the relevance by screening the abstracts. Because difficulties with recruitment and turnover among nurses were identified as a global concern in the early 2000s [27], we made a choice to restrict the included literature to not older than 20 years. Review articles [28, 29] were not included, but they were examined for citations. Articles explicitly stating that a usage fee or licence was required [30] were not included. We also excluded articles describing a questionnaire that measured only one work environment characteristic such as leadership [31]. This was done because the work environment is considered a multidimensional phenomenon, and in order to have a questionnaire of a reasonable length, we excluded questionnaires that went deep into one characteristic, measuring only one dimension.

Articles were included for full-text reading and charting of contents, if one author found it potentially relevant. Additional articles that presented quality assessment of the questionnaire in the article were read thoroughly in order to clarify the development, use or properties of the questionnaire. For example, if an article referred to a qualitative study that described the content development of a questionnaire or the psychometric properties of the particular questionnaire, this article was included in the appraisal of the questionnaire and charting of data.

The results are mainly presented in tables as recommended by Khalil et al. [25]. Table 2 is a presentation of the characteristics of the questionnaires in order to compare and assess their relevance. If the questionnaire was used in several studies, we present these studies together.

First, we identified the study author(s) and country of development, name of the questionnaire used in the study and the main study object, i.e. the main dimension the questionnaire is designed to measure. The number of items in the questionnaire represents a total count, including questions not concerning work environment issues, but excluding sociodemographic questions. We recorded the target population in the study; this may differ from the population the questionnaire was originally developed for. The workplaces of the participants in each study and the response format used in the questionnaire are also presented in the table. In case of revisions, we extracted data from the latest version known to us.

Daudt et al. [26] suggested that scoping reviews should include some form of quality assessment for included studies. Therefore, we performed a brief appraisal by recording relevant information about psychometric properties that were presented with the questionnaire. This assessment was based on a very short customized version of the COSMIN checklist [32, 33]. For example, whether the content development was described, if reproducibility or internal consistency was tested and if it had acceptable results. The appraisal also included a global rating of scientific quality and of the overall face validity for basic nursing in long-term settings. The appraisal scores were summed, and the questionnaires were categorized in three groups according to their appraisal scores (nine with high scores in group one; thirteen in group two with medium scores; twelve in group three with low scores).

The methodological quality or risk of bias was not assessed in the included articles. This is in line with how scoping reviews are usually conducted [34].

In regard to research question 2; the questionnaire content mapping was conducted by mapping the content domains of the nine questionnaires in group 1, represented by the labels assigned to them by the authors. We decided, a priori, to build on Bae’s [7] review of working conditions. The first author did a qualitative interpretation of the domains in the questionnaires and their concurrence with Bae’s synthesised domains. This was done in order to map the domains and labels used in the questionnaires and possibly expand the range of domains already identified by Bae.

The first literature search was conducted in October 2015 and yielded 4305 unique articles. The update search conducted in December 2016 provided 750 new articles, after duplicates were removed. Figure 1 illustrates the search and selection. After the first relevance screening, 263 articles remained. For the final synthesis of full-text articles, 50 articles were included. Three more questionnaires measuring different dimensions of the work environment were found by screening references in the included literature [35‐37]. We included 12 articles that elaborated on properties of any of the identified questionnaires; the final selection consisted of 65 articles comprising 34 questionnaires. We searched government and organization websites and found generic work environment questionnaires currently in use in many different types of services; for example the QPS-Nordic [38] and the 10-faktor [39].

Table 2 shows the questionnaires found in our search and is sorted first by the appraisal group and then by the author’s name.

The number of items in the questionnaires varies considerably from 14 [40, 41] to 105 [42]. The questionnaires were developed for and tested in health personnel subgroups, for example, questionnaires developed for nurse aides [43‐45], practical nurses [46], nurse practitioners [47], all employees [35], all care workers [48‐53], and specific versions developed for different groups of workers [36, 54]. Most questionnaires were developed and tested for registered nurses [14, 16, 37, 53, 55‐57].

Most questionnaires were tested in a general hospital setting. Six questionnaires were developed specifically for use in nursing home settings [35, 44‐46, 49, 58], and one [59] for home healthcare. However, four questionnaires were adapted and modified from a hospital setting for use in nursing homes [48, 51, 60, 61]. Three questionnaires were developed for use in a hospital setting but were used in the long-term care setting without modification, or with just minor changes in wording to fit the new setting [3, 62‐65].

The Nursing Work Index [63, 66] and the Job Content Questionnaire [67, 68] are used in both home healthcare and nursing homes settings. In these studies, only registered nurses were included.

The most frequently used response format was a Likert-type four- or five-point scale.

There was considerable variation as to which outcomes and work environment dimensions were measured by the questionnaires. These were, for example, quality of care [53], job satisfaction [45, 46, 51, 69], safety attitudes or safety culture [58, 70, 71], creative climate [50], barriers [44], person-centred care [48], or evidence-based practice [36]. In regard to measuring only the practice environment, the most frequently used questionnaire that we found in this review is the Nursing Work Index (NWI). In our findings, the NWI is also the questionnaire that has been most revised. The nurses’ practice or work environment as the main study object was also found in a questionnaire named the Brisbane Practice Environment Measure (B-PEM) [55], which is similar to the NWI in terms of contents. Some of the questionnaires have the work environment in a specific context or setting as the main study object, such as: Work environment and perceived work effectiveness [35], Patient care associates’ practice environment in the acute care settings [43], Nurses’ practice environment (person-centred practice framework) [56], Essentials of a productive nurse work environment identified by nurses practicing in Magnet hospitals [37], and work environment as perceived by nurses [16].

The Job Content Questionnaire (JCQ) [67, 68] and the Creative Climate Questionnaire (CCQ) [50] are generic questionnaires, developed to be used in any professional group.

Table 3 shows the nine questionnaires in appraisal group 1, with attention paid to the questionnaire contents. The ten domains synthesized in Bae’s review [7] are presented in the top row in Table 3.

The content domains are labelled differently. For example, Bae uses the concept “supportive managers” but supposedly similar domains identified in eight out of nine questionnaires were labelled “perceptions of management” [71], “professional support” [51], “relationship with organization” [59], “leadership” [35, 36], “management support” [55] and “ward leadership” [42]. The concept of “supportive managers” was split into two domains (relationships and development) in one questionnaire [69].

The concept domain of collaborative relationships with peers was present in almost all questionnaires, but the label varied. The labels in the questionnaires were: “a climate of community” [40], “teamwork climate” [71], “team spirit” [51], “relationship with peers” [59], “informal interactions” [36], “relationships” [69] and “staff cohesion” [35] and “professional relationship amongst nurses” [42]. The collaborative relationships with the physicians’ domain were less prevalent than the relationships with peers. The label was called: “relationship between nurses and medical staff” [42] and “relationship with physician” [59]. In one questionnaire, the label was “formal interactions” [36] and included different healthcare providers, not only physicians.

The perception of busyness is also a topic in the majority of the questionnaires. These are labelled: “stress and workload” [59], “workload” [51, 55], “staff organization” [42], “organizational slack-staff”, “organizational slack–time” [36], “nature of work” [69] and “perceived work effectiveness” [35].

The domains of autonomy, participation and involvement were labelled “personal satisfaction” [51], “professional pride” and “autonomy” [59], “influence on timing of ward and patient events”, “influence on ward management” and “influence on human and financial resources” [42] and “culture” [36] in the questionnaires.

The domain of patient-centred climate was present in one questionnaire, labelled “relationship with patients” [59]. Person-centred climate was also the overall phenomenon to be measured in one questionnaire [40].

Four out of nine questionnaires contained a domain related to professional practice and education. These were labelled “professional practice” [42], “training” [51], “professional development” [55], and “education” [69]. One questionnaire had “evidence-based practice” as an overall phenomenon to be measured [36]. Professional development was not included in Bae’s [7] synthesis.

Other domains that were not present in Bae’s review were the physical surroundings and availability of resources. In the questionnaires identified in the present study, these were labelled: “ward facilities” [42], “a climate of everydayness” [40], “structural and electronic resources”, “organizational slack–space” [36] and “resources” [69].

A domain including salary, benefits and rostering was also present in the questionnaires, labelled “rostering” [55], and “salary and benefits” [59], as was patient safety, labelled “ward layout” [42], “a climate of safety” [40] and “safety climate” [51].

The Nursing Work Index (NWI), and modified versions of it, stand out as the most frequently used instrument for measuring the work environment of registered nurses in this review. Because the NWI was developed in the USA over 25 years ago [12], the content of the NWI may be decreasingly relevant for contemporary work settings. However, The Essentials of Magnetism (EOM) [19] process measurement tool and subsequently its revised version EOMII [20, 21] were developed with a basis in the NWI and assess more contemporary aspects of importance for a productive nursing work environment. The healthcare sector is constantly under transformation. New management structures and cost containment have been prominent features in recent years [72], as have the change from profession-centredness to patient-centredness and patient-safety focus. Taking a broader view on the work environment, the questionnaires include a varied selection of constructs and operationalisations intended to represent the work environment domains of nurses. The domains we mapped in the questionnaires are to some extent overlapping, often with little consistency in terminology. Nurses’ perceptions of their work environment may include a range of different phenomena that are not necessarily directly related to one another, but indirectly or directly comprise the environment in which the nurses work [73, 74]. Some domains are more prevalent in the reviewed questionnaires, but it is premature to conclude that these are more significant than others for measuring the work environment. Some elements may been the subject of less attention or research and therefore not measured in the questionnaires we found in this review, such as relationships with other professionals or relatives, as opposed to relationships with physicians and peers, which are the commonly measured domains.

Our findings illustrate the importance of clarifying and defining the outcome one intends to measure. When measuring a broad construct, in our case “nursing work environment”, the subdomains of relevance for the target population and in the specific context need to be defined [33, 75]. In our review, several questionnaires seem to measure the same or overlapping domains, but under different labels and uniquely operationalised. For example, the domain “autonomy” is a common work environment domain included in work environment surveys. It has been argued that the concept of autonomy can be theoretically differentiated into two discrete concepts – autonomy related to the nurses’ clinical practice and autonomy in relation to work [76]. When measuring a domain such as “autonomy”, one needs to clarify the theoretical construct, and be aware that a measure of a construct in one questionnaire may not be used interchangeably with another construct of the concept in a different questionnaire. This means that a theoretical consideration of how nurses’ work environments are conceived needs to be made, and made explicit, when choosing among questionnaires and in the design of a study [7].

Based on a review of national frameworks of long-term care quality policy documents and analytic frameworks in the academic literature, the Organization for Economic Co-operation and Development (OECD) stressed three aspects as generally accepted and critical underpinnings of the quality of long-term care: patient-centredness, care co-ordination, and safety effectiveness [77]. The dimension of patient-centred climate was not a common dimension in the first group of the reviewed questionnaires. One questionnaire measured person-centred climate as a sole topic [40]. Patient-centredness has become a healthcare quality hallmark and may represent something slightly different in long-term care settings compared to acute care settings. Nurses in long-term care deem social relationships with residents to be an important factor of their work environment and in their assessment of quality of care and their intent to remain in their work [78], as well as a motivating factor that is important for their job satisfaction [79]. This is supported by previous research, which found that residents in nursing homes find the relationships with nurses to be an important factor in their wellbeing and in high-quality care [80, 81].

The dimension of collaborative relations with peers was an aspect in almost all nine questionnaires. Researchers found that team collaboration and performance are associated with higher levels of quality of care and functional outcomes among residents in nursing homes [82, 83]. Nursing home staff’s perceptions of better team climate were related to better-perceived quality of care in a recent study, and the researchers concluded that team climate was an important factor to consider when trying to improve quality of care [84]. It is also argued that there is a possible association between improved teamwork and reduced work stressors and less care left undone [85].

The last aspect OECD stresses as important to quality of long-term care is “safety” [77]. We found this domain in the questionnaires. Nursing care is provided to patients in complex care environments that can generate errors and cause harm. Patient safety is also considered an indicator of high-quality nursing care. This can be seen as related to nurses’ direct roles in integrating care, detecting possible errors and preventing harm and adverse events [86]. A failure in fulfilling these roles may result in errors in patient care as well as adverse events. The aspect of safety climate may therefore be among the most important factors when measuring the work environment in long-term care settings.

An important phenomenon that was absent in the questionnaires was the relationship with relatives. As the residents in nursing homes need more complex care, in cases of, for example, dementia, the relatives’ role as “spokespersons” will be crucial for patient-centredness [87]. The relatives’ role may grow in importance and become more central among the prerequisites that facilitate good nursing practice.

The literature about the work environment is large and complex, with a wide variety of constructs and operationalisations to represent the nursing work environment, often with little consistency in the use of terminology [74]. This means that there may be terms that pertain to the work environment domains that we did not include in the search. As a result, questionnaires may have been neglected by the procedures we followed. The first screening was done by reading the title, and browsing the abstract in case of uncertainty. The precision of this procedure is entirely dependent on the terminology used in the titles and the abstracts. There is a risk that relevant articles may have been overlooked for this reason. We directed our search to factors that are amenable to change and that pertain to the aspects of professional nursing practice. Consequently, questionnaires may have been filtered out because we found that the main content concerned personal or psychosocial characteristics, while, in fact, a part of the questionnaire may have fitted our aim. Conversely, some of the questionnaires included in the review may have domains relating to psychosocial or personal aspects. There may also be questionnaires used by governments and organizations that our internet searches did not find. The search was also conducted with terms including measurement properties (included in Table 1), i.e. articles that did not present measurement properties could have been sorted out, even though they may fit our criteria.

The appraisal in this review should not be seen as a complete quality assessment, rather an appraisal of the questionnaires’ “fit” to a generic group of nursing personnel and setting.

Our results may also be influenced by some degree of dissemination bias [88], because questionnaire developers may be less willing to publish results that are unfavourable in terms of the psychometric properties of a questionnaire.