Purpose/Objectives: To examine oncology nurses’ attitudes toward and reported use of the Edmonton Symptom Assessment System (ESAS) and to determine whether the length of work experience and presence of oncology certification are associated with their attitudes and reported usage.
Design: Exploratory, mixed-methods study employing a questionnaire approach.
Setting: 14 regional cancer centers (RCCs) in Ontario, Canada.
Sample: Oncology nurses who took part in a larger province-wide study that surveyed 960 interdisciplinary providers in oncology care settings at all of Ontario’s 14 RCCs.
Methods: Oncology nurses’ attitudes and use of ESAS were measured using a 21-item investigator-developed questionnaire. Descriptive statistics and Kendall’s tau-b or tau-c test were used for data analyses. Qualitative responses were analyzed using content analysis.
Main Research Variables: Attitudes toward and self-reported use of standardized symptom screening and ESAS.
Findings: More than half of the participants agreed that ESAS improves symptom screening, most said they would encourage their patients to complete ESAS, and most felt that managing symptoms is within their scope of practice and clinical responsibilities. Qualitative comments provided additional information elucidating the quantitative responses. Statistical analyses revealed that oncology nurses who have 10 years or less of work experience were more likely to agree that the use of standardized, valid instruments to screen for and assess symptoms should be considered best practice, ESAS improves symptom screening, and ESAS enables them to better manage patients’ symptoms. No statistically significant difference was found between oncology-certified RNs and noncertified RNs on attitudes or reported use of ESAS.
Conclusions: Implementing a population-based symptom screening approach is a major undertaking. The current study found that oncology nurses recognize the value of standardized screening, as demonstrated by their attitudes toward ESAS.
Implications for Nursing: Oncology nurses are integral to providing high-quality person-centered care. Using standardized approaches that enable patients to self-report symptoms and understanding barriers and enablers to optimal use of patient-reported outcome tools can improve the quality of patient care.
Early systematic assessment and management of cancer-related symptoms has been shown to improve patient- and system-level outcomes (Berry et al., 2011; Gilbert et al., 2012; Kearney et al., 2008). Oncology nurses, in collaboration with the interdisciplinary team, play a fundamental role in identifying, assessing, and managing physical and psychological symptoms that patients experience throughout their cancer journey. Nurses are also most available to care for patients and their families. However, the literature shows that patients continue to experience unmet needs and distress throughout their illness trajectory, resulting in poorer quality of life (Barbera et al., 2012; Chang, Hwang, Feuerman, & Kasimis, 2000; Deshields, Potter, Olsen, Liu, & Dye, 2011).
Symptom burden may be underestimated because of the under-recognition of symptoms by nurses and other healthcare providers. In a prospective study of 1,933 patient–healthcare provider dyads across 11 European countries, physicians and nurses underestimated symptom intensities in about 1 in 10 patients with cancer and overestimated it in 1% of patients (Laugsand et al., 2010). Although the study found that nurse assessments were more in alignment with patient ratings than physician assessments, concerns regarding oncology nurses’ acknowledgement of patients’ symptoms remain an identified gap in care. Another prospective study examined the accuracy of oncology nurses’ recognition of supportive care needs and symptoms of their patients undergoing chemotherapy and found that nurses significantly under-recognized patients’ psychological symptoms, support needs, and general physical symptoms, such as constipation, insomnia, dyspnea, and pain (Nakaguchi et al., 2013). Other studies have found that oncologists and nurses may not always be accurate in identifying patients with significant distress (Mitchell, Hussain, Grainger, & Symonds, 2011; Mitchell & Kakkadasam, 2011).
To address this gap, initiatives such as patient-reported outcome (PRO) tools have become a focus of routine clinical practice (Greenhalgh, 2009; Snyder et al., 2012). Indicating scores for outcomes on symptoms that matter to patients has been shown to be more reliable than clinician identification of symptoms (Lipscomb et al., 2007) or relying on patients to call between visits to voice particularly bothersome or worrisome complaints (Basch & Abernethy, 2011). In addition, the use of PRO tools has been found to help improve patient–clinician communication and symptom detection and management (Basch et al., 2005; Chen, Ou, & Hollis, 2013; Cleeland et al., 2011; Seow, Sussman, Martelli-Reid, Pond, & Bainbridge, 2012), leading the way toward optimal and timely symptom management (Basch & Abernethy, 2011).
The Edmonton Symptom Assessment System (ESAS) (Bruera, Kuehn, Miller, Selmser, & Macmillan, 1991) is a valid and reliable standardized symptom screening patient-reported tool that was first developed for patients receiving palliative care and is being used in its revised form in several large oncology settings as part of standard clinical care (Basch & Abernethy, 2011; Dudgeon et al., 2012). ESAS is used to screen for the intensity of nine common symptoms experienced by patients: pain, tiredness, nausea, drowsiness, depression, anxiety, appetite, shortness of breath, and well-being (Bruera et al., 1991; Watanabe, Nekolaichuk, & Beaumont, 2012). It offers nurses and other providers with a snapshot of a patient’s physical and psychological symptom profile, including severity rating (Fitch, Howell, McLeod, & Green, 2012). Use of ESAS as a screening tool was implemented by Cancer Care Ontario in 2006 and is recommended for use in all patients with cancer visiting the 14 regional cancer centers (RCCs) across Ontario, Canada (Dudgeon et al., 2012). The tool can easily be accessed from the organization’s website (http://bit.ly/2gkh83l) alongside guidelines for use (http://bit.ly/2gVethB). ESAS has also been implemented across a number of partner hospitals in Ontario, Canada.
Some literature has explored the perspectives of Ontario healthcare providers toward the use of ESAS; however, these studies focused on a single center (Bainbridge et al., 2011) or physician-only group (Chasen, Bhargava, Dalzell, & Pereira, 2015) within the Ontario cancer system. Bainbridge et al. (2011) saw variable adoption of the tool throughout a cancer center (particularly among physicians), with Chasen et al. (2015) reporting a more favorable attitude toward regular use by medical oncologists and general practitioners in oncology compared to radiation oncologists at a different site.
A review of the literature suggests that attitudes of nurses toward a topic may be influenced by various factors, such as age, gender, education level, years of nursing experience, and education in the specific topic area (Cevik & Kav, 2013; Ford & McInerney, 2011; Lange, Thom, & Kline, 2008). Khader, Jarrah, and Alasad (2010) found that older nurses feel more comfortable talking about end-of-life issues than younger ones, and women are more open to death-related thoughts and feelings than men. Integration of ESAS into nursing school curricula and continuing education programs, along with individual experiences, may enhance nurses’ positive attitudes toward caring for dying patients.
Oncology nurses play an important role in symptom management (Canadian Association of Nurses in Oncology, 2012; Fitch et al., 2012; Mick, 2008), and the study team hypothesized that nurses’ years of work experience and the designation of CON(C) (Certified in Oncology Nursing in Canada) may affect their attitudes toward the use of systematic assessment tools. In Canada, CON(C) national certification is granted by the Canadian Nurses Association following completion of work in the specialty and successful results on the national examination. However, to date, little has been done to understand the use of a standardized symptom assessment tool from a nursing perspective. This could help to address gaps in nursing practice and promote person-centered care for patients.
The objectives of this study were twofold: to examine attitudes toward and reported use of ESAS among oncology nurses and to identify whether a relationship exists between nurses’ attitudes toward ESAS and the length of their work experience or the presence of oncology certification.
The current study analyzed responses from oncology nurses who took part in a larger province-wide study that surveyed a total of 960 interdisciplinary providers in oncology care settings at all of Ontario’s 14 RCCs where standardized systematic symptom assessment is routine (Pereira et al., 2016). A multipronged approach, including a provincial project plan and management and evaluation framework, was undertaken in 2006 to implement the Interactive Symptom Assessment and Collection—including ESAS—alongside expert coaching and guidance of healthcare practitioners (Dudgeon et al., 2012; Pereira et al., 2014).
Potential participants currently practicing in ambulatory settings within one of the 14 RCCs were asked by the Ontario Cancer Symptom Management Collaborative center-specific leads to participate in the current study. Participants were invited via email to complete an anonymous online survey; they were given four weeks to complete the survey. Nonresponders were sent as many as three reminder emails during a four-week study period until they responded or opted out.
The surveys were completed from April to June 2014. The study was approved by the Ottawa Health Science Network Research Ethics Board as the primary site, and all participants provided informed consent prior to completing the questionnaire.
[[{"type":"media","view_mode":"media_original","fid":"28166","attributes":{"alt":"","class":"media-image","height":"343","typeof":"foaf:Image","width":"366"}}]]
Participant characteristics were obtained from the demographic data section of the questionnaire, which included gender, years of practice, and presence of oncology certification among the respondent nurses (see Table 1). Attitudes toward and reported usage of ESAS (see Tables 2 and 3) were measured using the ESAS in clinical care section of the overall 21-item questionnaire, with 17 closed-ended (dichotomous, Likert-type scale, or frequency) and 4 open-ended questions. The design of this survey also included an opportunity to gather qualitative data; participants were asked to provide additional comments after certain questions to better understand their perspectives.
[[{"type":"media","view_mode":"media_original","fid":"28171","attributes":{"alt":"","class":"media-image","height":"1058","typeof":"foaf:Image","width":"545"}}]]
[[{"type":"media","view_mode":"media_original","fid":"28176","attributes":{"alt":"","class":"media-image","height":"457","typeof":"foaf:Image","width":"544"}}]]
With permission, the survey was modeled after those used in two previous Ontario-based studies (Chasen et al., 2015; Seow et al., 2012). It was checked for face and content validity by the principal investigators and then transferred to an electronic format (FluidSurveysTM).
Quantitative data were analyzed using SPSS®, version 23.0. Descriptive statistics were used to generate frequencies and percentages for demographic variables and individual scale items. Kendall’s tau-b or tau-c test was used to explore nurses’ attitudes and use of ESAS in relation to their years of practice and presence of oncology certification. A stratified analysis was performed using the same test to assess the presence of CON(C) national certification by length of work experience and to investigate whether a difference in attitudes existed.
An examination of missing data did not reveal any systematic or nonrandom patterns across measures. Pairwise deletion was used to handle missing responses. The Mahalanobis distance method was used to determine if multivariate outliers existed in the data (Hazewinkel, 2001).
[[{"type":"media","view_mode":"media_original","fid":"28181","attributes":{"alt":"","class":"media-image","height":"942","typeof":"foaf:Image","width":"759"}}]]
Free-text comments were analyzed using a qualitative content analysis approach (Priest, Roberts, & Woods, 2002; Weber, 1990) with QSR® NVIVO 10. Free-text comments were analyzed in context of the companion quantitative questions and response type (agree or disagree) when appropriate. Strongly agree and agree free-text comments for each question were collapsed and coded together (agree), and the same was done for strongly disagree and disagree comments (disagree). Analytic consensus was reached through a process of coding review and discussion of main themes. Additional details on the qualitative analysis approach are presented in Pereira et al. (2016).
RNs (N = 1,012) were identified across 14 RCCs in Ontario, and 361 (36%) completed the online survey. Of the 361 responses, 8 (2%) were identified as outliers (using a critical point of 31.264 with the Mahalanobis distance method) and excluded from data analysis. The current analyses included responses from a final sample of 353 oncology nurses, representing 37% of the total sample in the larger study.
Of the 353 RNs who participated in the current study, 41 (12%) were advanced practice nurses with master’s-level education (clinical nurse specialists and nurse practitioners), and 178 (50%) were certified with CON(C). Ninety-six nurses (27%) had practiced more than 20 years in an oncology setting at the time of assessment.
Collaborative responsibility: A majority of nurses (n = 318, 90%) agreed or strongly agreed that managing symptoms was within their scope of clinical responsibilities. Of those who reported agreement and provided a free-text comment (n = 40, 11%), some reported that symptom management was a collaborative responsibility together with other providers, acknowledging the role of referral to other providers when appropriate. As one nurse stated, “Managing some symptoms is within my scope of practice—not all of them. The team contains people who together we can [address the symptoms].”
Standardized tools as best practice: Many nurses (n = 286, 81%) agreed or strongly agreed with the statement: “Generally, in everyday practice, the regular use of standardized, valid instruments to screen for and assess symptoms should be considered best practice.” Of the nurses who were in agreement and also provided a free-text comment (n = 85, 24%), some reported that standardized tools allowed for all patients to be assessed in a consistent manner, and others noted the role of tools in helping to focus the visit on patient needs, as the following comment illustrates: “[Standardized, valid instruments] allow consistency in approach to patient problems and help direct care to issues identified as most important to the patient.”
Of those who disagreed and also provided a free-text comment (n = 17, 5%), some nurses felt that a standardized tool, such as ESAS, may not be suitable or sufficient for all populations of patients with cancer. One participant said, “[Standardized tools] should be used as a guideline. . . . Depending on disease site or treatment, a symptom may be missed in the standardized tool because it is not covered.”
Patients reporting symptoms with the Edmonton Symptom Assessment System: Of all nurses participating, 261 (74%) agreed or strongly agreed with the statement, “The ESAS helps patients report their symptoms.” Some participants commented that ESAS helps to ensure that symptoms are not missed. As one nurse noted, “Most patients forget to mention symptoms and feel better having the list of symptoms in front of them to ensure that all symptoms are reported and addressed.” However, of those who disagreed and also provided free-text comments (n = 28, 8%), some reported that certain patients may be confused with aspects of the ESAS tool, with one nurse stating that many patients fill it out incorrectly.
The Edmonton Symptom Assessment System and efficiency: More than half of the nurses (n = 209, 59%) agreed or strongly agreed that the ESAS improves the efficiency of the meeting with the patient. Of those in agreement, 61 participants (17%) also provided a free-text comment, with many reporting that ESAS helped to focus the assessment on what was important to the patient. As one nurse reported, “[With ESAS], we are more able to pinpoint the [patient’s] most distressing symptoms. . . . It makes the assessment more efficient.”
In contrast, of the participants who disagreed and also provided a free-text comment (n = 33, 9%), some reported that ESAS slowed down clinics because clinicians may have to wait for the patient to fill it in or that patients often reported issues that may not be related to their cancer. One nurse stated, “[ESAS] may actually lengthen [the visit], as some symptoms are not related to the current visit.”
Symptom screening and assessment: Although only 230 of the nurses (65%) agreed or strongly agreed that ESAS improves symptom screening, 297 (84%) agreed or strongly agreed that ESAS served as a useful starting point to assess patients’ symptoms, with 52 participants (15%) also providing a free-text comment. Some nurses emphasized that ESAS helped to open the discussion but was a starting point and made up only part of the assessment, as this comment illustrates: “That’s just it. . . . [ESAS] is a starting point. It gets the [patient] thinking about how they are feeling and opens up the conversation for further assessment.”
Person-centered care: When asked what they liked best about ESAS, many nurses reported that it served as a patient-driven tool, allowing the patient’s voice to be heard. Many felt that ESAS helped open the discussion with the patient, allowing the visit to focus on the patient’s main concerns, as expressed in the following: “[ESAS] gives the patient a voice and ability to focus on the symptoms that are their concern.”
Nurses also liked the feature of tracking patients’ symptoms over time. For example, they reported that tracking symptoms allowed them to monitor treatment effectiveness. One nurse said, “When there are specific symptoms that you need to follow up with, the documentation of previous numbers can easily be seen to monitor effectiveness of treatment down the road.”
Areas for improvement: When asked what they would like to change about ESAS, nurses emphasized the need for improved patient instructions on how to complete ESAS, citing several areas of confusion that patients may experience. For example, “Appetite—many people think if they have a good appetite, they put a 10.” Suggestions were also made on how to amend the tool. For example, to have “more site specific symptoms depending on the patient’s primary or multiple cancer diagnosis.” In addition, nurses reported the need to add additional symptoms to ESAS, such as constipation, which was noted as “a frequent symptom that is not reported.”
A majority of nurses (311 of 348, 89%) indicated that they often or always reviewed their patients’ ESAS scores in clinic, 291 of 344 (85%) talked to patients about their ESAS scores, and 255 of 340 (75%) incorporated symptom management in response to ESAS in their care plan.
A statistically significant difference was found in attitudes toward ESAS between nurses with different lengths of work experience. Compared to nurses with more than 10 years of work experience, nurses who had 10 years of experience or less were more likely to agree that: (a) in their practice, the regular use of standardized, valid instruments to screen for and assess symptoms should be considered best practice (p = 0.049); (b) ESAS improves symptom screening (p = 0.013); and (c) ESAS enables them to better manage patients’ symptoms (p = 0.014). No statistically significant difference was found between oncology-certified RNs and noncertified RNs on attitudes toward ESAS. Regarding reported usage of ESAS, no difference was found between those with less than or more than 10 years of practice experience or between those who are or are not oncology certified.
Results from the stratified analysis revealed that no statistically significant correlation (using Kendall’s tau-b) was found between years of work experience and attitudes toward standardized symptom management and ESAS, regardless of CON(C) designation.
Implementing a population-based symptom screening initiative is a major undertaking (Dudgeon et al., 2012; Pereira et al., 2014). As ESAS and other PRO tools are being introduced in clinical settings, understanding how nurses view and use these instruments will add value to nursing care. To date and to the authors’ knowledge, the current study is the first to examine oncology nurses’ attitudes and reported use of ESAS in multiple cancer centers. Results of the current study revealed that many nurses value a standardized approach to assess and intervene on symptom issues. When compared with nurses, only 66% of physicians in the larger study considered the use of standardized tools to screen for symptoms as best practice (Pereira et al., 2016). In addition, most nurses encourage their patients to complete ESAS, which may suggest that nurses value the patient voice. These findings further support that nurses appreciate that appropriate tools are useful to improve patient care and the patient experience.
The role of nursing in symptom management is critically important. Oncology nurses play an integral role in providing high-quality, comprehensive patient care by screening for and assessing symptoms across the care continuum and settings. Although most nurses agreed or strongly agreed that managing symptoms is within their scope of practice and clinical responsibilities, many also acknowledged the need to work collaboratively with other providers in managing patients’ symptoms. This acknowledgement may indicate that interdisciplinary communication is occurring in the clinic, which is a critical success factor in implementing symptom screening and assessment as a programmatic approach (Fitch et al., 2012). More importantly, it demonstrates that all providers have a role in symptom management.
Since the beginning of this century (Ferguson & Day, 2005), in Canada and other countries, greater emphasis has been placed on the application of using evidence-based practice in nursing education. As such, nurses who are relatively new in the field may be more likely to engage in evidence-based practice and value the standardized approach to symptom management. This hypothesis aligns with the finding from the current study, in which nurses with 10 years of experience or less were found to have more positive attitudes toward standardized symptom management and the use of a PRO tool (i.e., ESAS). Although the authors hypothesized that attitudinal differences may exist between oncology-certified nurses versus those without certification, this was not observed in the current study. The current study also found no statistically significant difference between the presence of oncology certification and years of experience on attitudes; a larger study with higher response rates may reveal more differences between these groups.
In the current study, more than half of the nursing participants agreed that ESAS enables them to better manage patients’ symptoms. However, a common concern was that ESAS may lack specificity because it does not address disease-specific symptoms. This concern was also raised by physicians in a study exploring multidisciplinary healthcare professionals’ perceptions of ESAS for patients with cancer (Bainbridge et al., 2011). To address some of the coverage issues, Cancer Care Ontario is undertaking initiatives related to the implementation of disease-specific PRO measures.
Similar to previously reported concerns from healthcare providers (Carli Buttenschoen, Stephan, Watanabe, & Nekolaichuk, 2014; Watanabe, McKinnon, Macmillan, & Hanson, 2006), oncology nurses in the current study were also concerned about the patients’ ability to understand or complete the tool correctly. These findings reinforce the need for targeted knowledge translation and educational activities for providers and patients to promote uptake and appropriate use of ESAS in clinical practice.
Conducting a large survey-based study has challenges, and, as illustrated with the findings, the response rate was low (36%). One factor that may have contributed to the low response rate may be survey fatigue, a concept that is well known among researchers (Porter, Whitcomb, & Weitzer, 2004). Although the survey instrument was built on previous questionnaires (Chasen et al., 2015; Seow et al., 2012), time and resource limitations prevented pilot testing. Other study limitations include its cross-sectional nature and that the attitudes of those who declined or did not respond are unknown. Response bias is also possible because the nonresponders may, in general, have different attitudes toward symptom screening.
Oncology nurses are integral members of the cancer team in clinical and community settings and remain key to the success of patients and family caregivers in providing high-quality, person-centered care. Nurses understand the needs of patients and value the opportunities to use a standardized, consistent approach that enables patient reporting of symptoms and distress. The better the benefits, facilitators, and barriers to using PRO tools in clinical practice are understood, the more the use of those tools can be optimized to improve the quality of patient care.
[[{"type":"media","view_mode":"media_original","fid":"28186","attributes":{"alt":"","class":"media-image","height":"226","typeof":"foaf:Image","width":"365"}}]]
Using a population-based screening approach provides the opportunity to improve the patient experience on a global scale, but challenges exist to implementing it effectively. The current study revealed findings that can be used by other practice settings as PRO tools are introduced. Additional interventions should explore and provide strategies to improve the limitations of ESAS, and provide a focus on clinic service design to integrate PRO data collection and response systems into workflow.
The authors gratefully acknowledge Sean Molloy, MHSc, Serena Kurkjian, MBA, Wenonah Mahase, MBA, and Reena Tabing, MA, for their work and support on this study.
[[{"type":"media","view_mode":"media_original","fid":"28191","attributes":{"alt":"","class":"media-image","height":"617","typeof":"foaf:Image","width":"364"}}]]
Bainbridge, D., Seow, H., Sussman, J., Pond, G., Martelli-Reid, L., Herbert, C., & Evans, W. (2011). Multidisciplinary health care professionals’ perceptions of the use and utility of a symptom assessment system for oncology patients. Journal of Oncology Practice, 7, 19–23. doi:10.1200/jop.2010.000015
Barbera, L., Seow, H., Husain, A., Howell, D., Atzema, C., Sutradhar, R., . . . Dudgeon, D. (2012). Opioid prescription after pain assessment: A population-based cohort of elderly patients with cancer. Journal of Clinical Oncology, 30, 1095–1099. doi:10.1200/jco.2011.37.3068
Basch, E., & Abernethy, A.P. (2011). Supporting clinical practice decisions with real-time patient-reported outcomes. Journal of Clinical Oncology, 29, 954–956. doi:10.1200/jco.2010.33.2668
Basch, E., Artz, D., Dulko, D., Scher, K., Sabbatini, P., Hensley, M., . . . Schrag, D. (2005). Patient online self-reporting of toxicity symptoms during chemotherapy. Journal of Clinical Oncology, 23, 3552–3561. doi:10.1200/JCO.2005.04.275
Berry, D.L., Blumenstein, B.A., Halpenny, B., Wolpin, S., Fann, J.R., Austin-Seymour, M., . . . McCorkle, R. (2011). Enhancing patient-provider communication with the electronic self-report assessment for cancer: A randomized trial. Journal of Clinical Oncology, 29, 1029–1035. doi:10.1200/jco.2010.30.3909
Bruera, E., Kuehn, N., Miller, M.J., Selmser, P., & Macmillan, K. (1991). The Edmonton Symptom Assessment System (ESAS): A simple method for the assessment of palliative care patients. Journal of Palliative Care, 7(2), 6–9.
Canadian Association of Nurses in Oncology. (2012). Standards and competencies for cancer chemotherapy nursing practice. Retrieved from http://van-cf.cano-acio.ca/~ASSETS/DOCUMENT/CANO_Chemotherapy_Standards…
Carli Buttenschoen, D., Stephan, J., Watanabe, S., & Nekolaichuk, C. (2014). Health care providers’ use and knowledge of the Edmonton Symptom Assessment System (ESAS): Is there a need to improve information and training? Supportive Care in Cancer, 22, 201–208. doi:10.1007/s00520-013-1955-8
Cevik, B., & Kav, S. (2013). Attitudes and experiences of nurses toward death and caring for dying patients in Turkey. Cancer Nursing, 36, E58–E65. doi:10.1097/NCC.0b013e318276924c
Chang, V.T., Hwang, S.S., Feuerman, M., & Kasimis, B.S. (2000). Symptom and quality of life survey of medical oncology patients at a veterans affairs medical center: A role for symptom assessment. Cancer, 88, 1175–1183. doi:10.1002/(SICI)1097-0142(20000301)88:5<1175::AID-CNCR30>3.0.CO;2-N
Chasen, M., Bhargava, R., Dalzell, C., & Pereira, J.L. (2015). Attitudes of oncologists towards palliative care and the Edmonton Symptom Assessment System (ESAS) at an Ontario cancer center in Canada. Supportive Care in Cancer, 23, 769–778. doi:10.1007/s00520-014-2411-0
Chen, J., Ou, L., & Hollis, S.J. (2013). A systematic review of the impact of routine collection of patient reported outcome measures on patients, providers and health organisations in an oncologic setting. BMC Health Services Research, 13, 211. doi:10.1186/1472-6963-13-211
Cleeland, C.S., Wang, X.S., Shi, Q., Mendoza, T.R., Wright, S.L., Berry, M.D., . . . Vaporciyan, A.A. (2011). Automated symptom alerts reduce postoperative symptom severity after cancer surgery: A randomized controlled clinical trial. Journal of Clinical Oncology, 29, 994–1000. doi:10.1200/jco.2010.29.8315
Deshields, T.L., Potter, P., Olsen, S., Liu, J., & Dye, L. (2011). Documenting the symptom experience of cancer patients. Journal of Supportive Oncology, 9, 216–223. doi:10.1016/j.suponc.2011.06.003
Dudgeon, D., King, S., Howell, D., Green, E., Gilbert, J., Hughes, E., . . . Sawka, C. (2012). Cancer Care Ontario’s experience with implementation of routine physical and psychological symptom distress screening. Psycho-Oncology, 21, 357–364. doi:10.1002/pon.1918
Ferguson, L., & Day, RA. (2005). Evidence-based nursing education: Myth or reality? Journal of Nursing Educucation, 44, 107–115.
Fitch, M.I., Howell, D., McLeod, D., & Green, E. (2012). Screening for distress: Responding is a critical function for oncology nurses. Canadian Oncology Nursing Journal, 22, 12–30. doi:10.5737/1181912x2211220
Ford, R., & McInerney, F. (2011). An evaluation of aged-care workers’ knowledge of and attitudes toward the palliative approach. Research in Gerontological Nursing, 4, 251–259. doi:10.3928/19404921-20101103-01
Gilbert, J.E., Howell, D., King, S., Sawka, C., Hughes, E., Angus, H., & Dudgeon, D. (2012). Quality improvement in cancer symptom assessment and control: The Provincial Palliative Care Integration Project (PPCIP). Journal of Pain and Symptom Management, 43, 663–678. doi:10.1016/j.jpainsymman.2011.04.028
Greenhalgh, J. (2009). The applications of PROs in clinical practice: What are they, do they work, and why? Quality of Life Research, 18, 115–123. doi:10.1007/s11136-008-9430-6
Hazewinkel, M. (2001). Mahalanobis distance. Retrieved from https://www.encyclopediaofmath.org/index.php/Mahalanobis_distance
Kearney, N., Miller, M., Maguire, R., Dolan, S., MacDonald, R., McLeod, J., . . . Wengstrom, Y. (2008). WISECARE+: Results of a European study of a nursing intervention for the management of chemotherapy-related symptoms. European Journal of Oncology Nursing, 12, 443–448. doi:10.1016/j.ejon.2008.07.005
Khader, K.A., Jarrah, S.S., & Alasad, J. (2010). Influence of nurses’ characteristics and education on their attitudes towards death and dying: A review of literature. International Journal of Nursing and Midwifery, 2, 1–9.
Lange, M., Thom, B., & Kline, N.E. (2008). Assessing nurses’ attitudes toward death and caring for dying patients in a comprehensive cancer center. Oncology Nursing Forum, 35, 955–959. doi:10.1188/08.ONF.955-959
Laugsand, E.A., Sprangers, M.A., Bjordal, K., Skorpen, F., Kaasa, S., & Klepstad, P. (2010). Health care providers underestimate symptom intensities of cancer patients: A multicenter European study. Health and Quality of Life Outcomes, 8, 104. doi:10.1186/1477-7525-8-104
Lipscomb, J., Reeve, B.B., Clauser, S.B., Abrams, J.S., Bruner, D.W., Burke, L.B., . . . Trimble, E.L. (2007). Patient-reported outcomes assessment in cancer trials: Taking stock, moving forward. Journal of Clinical Oncology, 25, 5133–5140. doi:10.1200/JCO.2007.12.4644
Mick, J. (2008). Factors affecting the evolution of oncology nursing care. Clinical Journal of Oncology Nursing, 12, 307–313. doi:10.1188/08.CJON.307-313
Mitchell, A.J., Hussain, N., Grainger, L., & Symonds, P. (2011). Identification of patient-reported distress by clinical nurse specialists in routine oncology practice: A multicentre UK study. Psycho-Oncology, 20, 1076–1083. doi:10.1002/pon.1815
Mitchell, A.J., & Kakkadasam, V. (2011). Ability of nurses to identify depression in primary care, secondary care and nursing homes—A meta-analysis of routine clinical accuracy. International Journal of Nursing Studies, 48, 359–368. doi:10.1016/j.ijnurstu.2010.05.012
Nakaguchi, T., Okuyama, T., Uchida, M., Ito, Y., Komatsu, H., Wada, M., & Akechi, T. (2013). Oncology nurses’ recognition of supportive care needs and symptoms of their patients undergoing chemotherapy. Japanese Journal of Clinical Oncology, 43, 369–376. doi:10.1093/jjco/hyt003
Pereira, J., Green, E., Molloy, S., Dudgeon, D., Howell, D., Krzyzanowska, M.K., . . . Macdougall, L. (2014). Population-based standardized symptom screening: Cancer Care Ontario’s Edmonton Symptom Assessment System and performance status initiatives. Journal of Oncology Practice, 10, 212–214. doi:10.1200/jop.2014.001390
Pereira, J.L., Chasen, M.R., Molloy, S., Amernic, H., Brundage, M.D., Green, E., . . . Klinger, C.A. (2016). Cancer care professionals’ attitudes toward systematic standardized symptom assessment and the Edmonton Symptom Assessment System after large-scale population-based implementation in Ontario, Canada. Journal of Pain and Symptom Management, 51, 662–672. doi:10.1016/j.jpainsymman.2015.11.023
Porter, S.R., Whitcomb, M.E., & Weitzer, W.H. (2004). Multiple surveys of students and survey fatigue. New Directions of Institutional Research, 121, 63–73. doi:10.1002/ir.101
Priest, H., Roberts, P., & Woods, L. (2002). An overview of three different approaches to the interpretation of qualitative data. Part 1: Theoretical issues. Nurse Researcher, 10, 30–42. doi:10.7748/nr2002.10.10.1.30.c5877
Seow, H., Sussman, J., Martelli-Reid, L., Pond, G., & Bainbridge, D. (2012). Do high symptom scores trigger clinical actions? An audit after implementing electronic symptom screening. Journal of Oncology Practice, 8, e142–e148. doi:10.1200/jop.2011.000525
Snyder, C.F., Aaronson, N.K., Choucair, A.K., Elliott, T.E., Greenhalgh, J., Halyard, M.Y., . . . Santana, M. (2012). Implementing patient-reported outcomes assessment in clinical practice: A review of the options and considerations. Quality of Life Research, 21, 1305–1314. doi:10.1007/s11136-011-0054-x
Watanabe, S., McKinnon, S., Macmillan, K., & Hanson, J. (2006). Palliative care nurses’ perceptions of the Edmonton Symptom Assessment Scale: A pilot survey. International Journal of Palliative Nursing, 12, 111–114. doi:10.12968/ijpn.2006.12.3.20694
Watanabe, S.M., Nekolaichuk, C.L., & Beaumont, C. (2012). The Edmonton Symptom Assessment System, a proposed tool for distress screening in cancer patients: Development and refinement. Psycho-Oncology, 21, 977–985. doi:10.1002/pon.1996
Weber, R.P (1990). Basic content analysis (2nd ed.). Newbury Park, CA: Sage.
Green is the director at the Canadian Partnership Against Cancer in Toronto; Yuen is an analyst at Cancer Care Ontario in Toronto; Chasen is the medical director of palliative care at William Osler Health Services in Brampton, Ontario; Amernic is a research associate at Cancer Care Ontario; Shabestari is an assistant professor in the Institute of Health Policy, Management, and Evaluation at the University of Toronto; Brundage is a professor and director in the Department of Oncology at Queen’s University in Kingston, Ontario; Krzyanowska is an associate professor in the Institute of Health Policy, Management, and Evaluation at the University of Toronto; Klinger is a postdoctoral fellow in the Faculty of Medicine at the University of Ottawa in Ontario; Ismail is a manager at Cancer Care Ontario; and Pereira is the director of research at the College of Family Physicians of Canada in Toronto, Ontario. No financial relationships to disclose. Green, Chasen, Brundage, Krzyzanowska, Klinger, and Pereira contributed to the conceptualization and design. Green, Chasen, and Pereira completed the data collection. Yuen, Shabestari, and Ismail provided statistical support. All of the authors contributed to the analysis and the manuscript preparation. Green can be reached at esther.green@partnershipagainstcancer.ca, with copy to editor at ONFEditor@ons.org. Submitted December 2015. Accepted for publication June 3, 2016.