Translate this page into:
Validation of the Comprehensive Needs Assessment Tool in Patients with Advanced Cancer
This is an open access journal, and articles are distributed under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 License, which allows others to remix, tweak, and build upon the work non-commercially, as long as appropriate credit is given and the new creations are licensed under the identical terms.
This article was originally published by Wolters Kluwer - Medknow and was migrated to Scientific Scholar after the change of Publisher.
Abstract
Aims:
The 59-item Comprehensive Needs Assessment Tool (CNAT) for cancer patients is an English language survey developed in South Korea. The objective of this study was to validate the English version of CNAT in advanced cancer patients in Singapore.
Methods:
This was a cross-sectional survey where advanced cancer patients completed the CNAT in English. Confirmatory factor analysis was used to assess construct validity. For known groups validity, independent samples t-test was used to compare CNAT scores based on the Karnofsky performance status and outpatient versus inpatient setting. Cronbach's alpha was used to measure internal consistency.
Results:
A total of 328 advanced cancer patients were recruited. The mean age was 59.6 years and 49.1% were male. Majority (68.0%) were Chinese, 20.4% were Malay, 7.9% were Indian, and 3.7% were of other ethnicities. The 7-factor model previously established in Korea showed sufficient construct validity with root mean square error of approximation 0.037 and comparative fit index 0.944. All 59 items had a factor loading ≥0.5. Group invariance test showed no difference in the pattern of factor loadings between ethnic Chinese and other ethnic groups (P = 0.155). For known groups validity, there were significant differences in CNAT scores by performance status and outpatient versus inpatient setting. The CNAT total and factor scores showed good internal consistency with Cronbach's alpha of between 0.80 and 0.937.
Conclusions:
The CNAT showed construct and known-group validity and internal consistency in this study sample and can be used to assess the unmet needs of advanced cancer patients in the Singapore context.
Keywords
Advanced cancer
measurement properties
needs
patients
INTRODUCTION
Cancer is a leading cause of death and a highly burdensome disease for patients, their families, and the health-care system.[1] Previous studies indicate that patients with advanced cancer have high levels of unmet need that significantly contribute to the health-care burden at the end of life.[2] Therefore, needs assessment in these patients is important so as to identify patients requiring greater input from as well as aid service planning by health-care organizations.
The assessment of patients’ need for care is a vital step in achieving patient-centered care. However, these needs can be significantly affected by factors related to the socioeconomic-cultural setting and health-care system.[34567] Therefore, tools related to patient-reported outcomes developed in one setting may not be entirely or readily usable in another, giving rise to the need to validate assessment tools for the appropriate setting.[8]
The 59-item Comprehensive Needs Assessment Tool in Cancer (CNAT) is an English language survey for needs assessment developed and validated in South Korean cancer patients.[9] It has been used as the tool to describe the prevalence of needs in cancer patients but has not be specifically validated elsewhere.[1011] There is also a corresponding version of CNAT for caregivers and the potential for both the patient and caregiver versions to be used concurrently make the CNAT attractive.[1213] The aim of this study was to validate the CNAT in advanced cancer patients in Singapore.
METHODS
Study setting and participants
Singapore is a multiethnic society in Southeast Asia, where English is the working language and medium of instruction in schools. Most Singaporeans are bilingual in English and a mother tongue. According to the census in 2010, 74.1% of the residents are of Chinese descent.
This was a cross-sectional study. English-speaking participants were recruited from specialist outpatient clinics in the National Cancer Center of Singapore or inpatient oncology wards in Singapore General Hospital. The patient inclusion criteria were those aged 21 years or older, intact cognition as deemed by the attending physician at study entry, able to read or understand English, diagnosed with metastatic or locally advanced solid cancer that was not receiving curative treatment, Karnofsky Performance Status (KPS) >20, and able to give informed consent. This study was approved by the SingHealth Centralized Institutional Review Board (reference number 2013-082-A). Written informed consent was obtained before research participation.
Study instruments
The CNAT is a 59-item questionnaire with seven factors as follows: (1) information and education, (2) psychological problems, (3) healthcare staff, (4) physical symptoms, (5) hospital facilities and services, (6) social and religious/spiritual support, and (7) practical support. These seven factors were established by exploratory factor analysis (principal component analysis with varimax rotation).[9] Items are scored on a 4-point scale of severity according to the level of need. The original questionnaire was found to be reliable, valid, and acceptable in South Korean cancer patients.[9]
In this study, the CNAT was modified (with written permission from the CNAT developers) to replace the word “cancer” with the word “illness.” This modification was important, as it is not uncommon for patients to be unaware of their diagnosis in the Singapore context.[1415] Similar modifications have been previously reported for the adaptation and validation of the Functional Living Index-Cancer questionnaire in Singapore.[16]
Demographic and clinical data were obtained from eligible consented patients as well as medical records at the study entry. The CNAT was administered once to eligible participants by trained interviewers.
Statistical analysis
Participants rated each CNAT item according to their level of need for help – 0, (no need for help), 1 (a little), 2 (moderately), and 3 (a lot). Higher scores indicated higher levels of unmet need.[9] The factor scores were obtained by computing the mean score of items in each factor. The total CNAT score was obtained similarly by computing the mean score of all 59 items. The factor and total scores then rescaled to a 0–100 scale.
The percentage of respondents who obtained the minimum (floor) and maximum score (ceiling) were tabulated. Less than 30% of participants on the floor or ceiling were considered acceptable levels of floor or ceiling effects.[17]
To assess construct validity, we conducted confirmatory factor analysis (CFA) of the 7-factor model for the 59 items previously proposed by Shim et al.[9] We used the robust-weighted least squares method for the CFA.[18] The root mean square error of approximation (RMSEA) and Comparative Fit Index (CFI) were used for model selection and assessment of goodness-of-fit.[1920] While there is no golden rule to determine what cutoff values are optimal, we consider RMSEA 0.05 and CFI 0.95 as approximate reference values, which are often discussed in the literature.[1819] We tested group invariance in factor loadings between ethnic Chinese (n = 223) and non-Chinese (n = 105) participants.[18]
Known-groups validity was assessed based on the KPS and study setting. Independent samples t-test was used to compare CNAT factor and total scores for those with KPS 0–60 (requires assistance) versus KPS 70–100 (able to care for self), and those in the outpatient setting versus in the inpatient setting.
Internal consistency was measured using Cronbach's α. Test–retest reliability was not planned, as there may be rapid changes in functional status and psychosocial well-being due to complications from cancer progression resulting in different responses unrelated to the measurement properties of the CNAT questionnaire. Cronbach's α values of ≥0.7 were considered acceptable.[21]
Sample size considerations
For validity assessment by known-group comparison using the independent samples t-test, with group size that may vary by up to a 2:3 ratio, a total sample size of at least 275 gives 90% power, at 5% two-sided Type 1 error rate, to detect an effect size of 0.4 standard deviation (SD) in the mean difference between the two groups. Furthermore, the recommendation of Comrey and Lee[22] is that, for factor analysis, a sample size of 300 is “good.” The study sample size satisfied both criteria.
RESULTS
A total of 328 patients were recruited. The mean age was 59.6 years, 49.1% were male, and most (74.1%) were married with living with their spouse. Majority (68.0%) were Chinese, 20.4% were Malay, 7.9% were Indian, and 3.7% were of other ethnicities [Table 1].
| Characteristic | Frequency (%) |
|---|---|
| Age (years), mean (SD) | 59.6 (10.7) |
| Ethnicity | |
| Chinese | 223 (68.0) |
| Malay | 67 (20.4) |
| Indian | 26 (7.9) |
| Others | 12 (3.7) |
| Gender | |
| Male | 161 (49.1) |
| Female | 167 (50.9) |
| Marital status | |
| Single | 50 (15.2) |
| Married and living with spouse | 243 (74.1) |
| Married and living separately from spouse | 5 (1.5) |
| Divorced | 10 (3.1) |
| Widowed | 20 (6.1) |
| Highest education level | |
| Primary | 29 (8.8) |
| Secondary | 168 (51.2) |
| Postsecondary | 130 (39.6) |
| Employed | 114 (34.8) |
| Clinical setting | |
| Inpatient | 131 (39.9) |
| Outpatient | 197 (60.1) |
| Cancer type | |
| Lung | 54 (16.5) |
| Breast | 57 (17.4) |
| Colorectal | 62 (18.9) |
| Others | 155 (47.3) |
| Receiving chemotherapy | 166 (50.6) |
| Receiving radiotherapy | 26 (7.9) |
| Karnofsky performance status 70 and above (able to care for self) | 198 (60.4) |
SD: Standard deviation
On a 0–100 scale, the mean CNAT total score was 14.1 [Table 2]. Less than 2% of participants reached the ceiling for any of the domains. Substantial (>30%) floor effect was observed in 5 of 7 factors; the highest floor effect was seen in factor 6 “social and religious/spiritual support” (58.2%).
| Mean (SD) | Floor score, n (%) | Ceiling score, n (%) | |
|---|---|---|---|
| Factor 1: Information and education (10 items) | 19.5 (22.6) | 130 (21.7) | 9 (1.5) |
| Factor 2: Psychological problems (10 items) | 16.4 (21.5) | 200 (33.3) | 3 (0.5) |
| Factor 3: Healthcare staff (8 items) | 11.0 (18.3) | 302 (50.3) | 2 (0.3) |
| Factor 4: Physical symptoms (12 items) | 13.3 (16.6) | 159 (26.5) | 0 (0) |
| Factor 5: Hospital facilities and services (6 items) | 12.9 (17.5) | 238 (39.7) | 2 (0.3) |
| Factor 6: Social and religious/spiritual support (5 items) | 7.9 (14.1) | 349 (58.2) | 1 (0.2) |
| Factor 7: Practical support (8 items) | 13.4 (17.4) | 217 (36.2) | 2 (0.3) |
| Total score (59 items) | 14.1 (14.7) | 23 (3.8) | 0 (0) |
SD: Standard deviation
CFA of the 7-factor model gave RMSEA 0.037 and CFI 0.944, indicating approximately sufficient fit. All 59 items had a factor loading ≥0.5. The 7-factor structure is, therefore, valid in this study. We tested for group invariance between the ethnic Chinese participants and participants of other ethnicities. The difference testing gave P = 0.155, demonstrating lack of difference in factor structure between the two groups.
When comparing participants with different KPS, there was a statistically significant difference in the CNAT total scores, 13.0 (16.2) for those with good performance status versus 20.0 (17.5) for those with poor performance status (P < 0.01) [Table 3]. The effect size was approximately 0.4 SD. There was also the good separation of scores for Factor 1 (information and education), Factor 2 (psychological problems), Factor 4 (physical symptoms), Factor 5 (hospital facilities and services), and Factor 7 (practical support). There was no significant difference in scores for Factor 3 (health-care staff) and Factor 6 (social and religious/spiritual support).
| Mean (SD) | Pa | ||
|---|---|---|---|
| KPS 70 and above (able to care for self), (n=198) | KPS 60 or less (needs assistance), (n=130) | ||
| Factor 1: Information and education (10 items) | 20.6 (25.3) | 26.8 (25.6) | 0.03 |
| Factor 2: Psychological problems (10 items) | 14.7 (21.0) | 22.5 (25.1) | <0.01 |
| Factor 3: Health-care staff (8 items) | 11.3 (20.5) | 15.4 (21.9) | 0.08 |
| Factor 4: Physical symptoms (12 items) | 11.6 (18.3) | 18.4 (18.5) | <0.01 |
| Factor 5: Hospital facilities and services (6 items) | 10.5 (17.8) | 19.6 (20.7) | <0.01 |
| Factor 6: Social and religious/spiritual support (5 items) | 7.8 (15.8) | 10.6 (16.7) | 0.13 |
| Factor 7: Practical support (8 items) | 10.1 (16.3) | 20.8 (22.0) | <0.01 |
| Total score (59 items) | 13.0 (16.2) | 20.0 (17.5) | <0.01 |
aP value from independent samples t-test. KPS: Karnofsky performance status, SD: Standard deviation
When comparing participants in the outpatient versus inpatient setting, there was a statistically significant difference in the CNAT total scores – 11.4 (13.9) for those in the outpatient setting and 22.2 (19.3) in the inpatient setting (P < 0.01) [Table 4]. The effect size was approximately 0.6 SD. There was also good separation of factor scores.
| Mean (SD) | Pa | ||
|---|---|---|---|
| Outpatient setting (n=197) | Inpatient setting (n=131) | ||
| Factor 1: Information and education (10 items) | 19.4 (24.3) | 28.6 (26.5) | <0.01 |
| Factor 2: Psychological problems (10 items) | 13.1 (19.8) | 25.0 (25.5) | <0.01 |
| Factor 3: Health-care staff (8 items) | 8.6 (17.1) | 19.4 (24.7) | <0.01 |
| Factor 4: Physical symptoms (12 items) | 9.9 (15.1) | 21.0 (21.3) | <0.01 |
| Factor 5: Hospital facilities and services (6 items) | 8.9 (15.3) | 21.9 (22.4) | <0.01 |
| Factor 6: Social and religious/spiritual support (5 items) | 7.1 (13.5) | 11.6 (19.4) | 0.01 |
| Factor 7: Practical support (8 items) | 9.2 (13.9) | 22.2 (23.6) | <0.01 |
| Total score (59 items) | 11.4 (13.9) | 22.2 (19.3) | <0.01 |
aP value from independent samples t-test. SD: Standard deviation
Overall, the internal consistency was high, with a Cronbach's alpha of 0.97 for the total score. All factors had Cronbach's alpha of between 0.80 and 0.93 [Table 5].
| Factor | Cronbach’s alpha |
|---|---|
| Factor 1: Information and education (10 items) | 0.92 |
| Factor 2: Psychological problems (10 items) | 0.93 |
| Factor 3: Health-care staff (8 items) | 0.91 |
| Factor 4: Physical symptoms (12 items) | 0.90 |
| Factor 5: Hospital facilities and services (6 items) | 0.80 |
| Factor 6: Social and religious/spiritual support (5 items) | 0.81 |
| Factor 7: Practical support (8 items) | 0.84 |
| Total score (59 items) | 0.97 |
DISCUSSION
This is a validation study of the English version of the CNAT for advanced cancer patients. The results show little ceiling effect but considerable floor effect in this study population. Despite the floor effect, we found good levels of construct validity in terms of CFA results, known-groups validity in relation to performance status and setting of care, and internal consistency.
Significant floor effect was observed for a few factors and may be due to the usage of a narrow 4-point response scale. In the context of a tool that measures unmet needs, the floor effect is of lesser concern clinically as needs-based interventions aim to target those with higher CNAT scores indicating more unmet needs. However, the significant floor effect could have suppressed the difference in CNAT scores by KPS – the two factors with the highest floor effect were the same two factors that did not have significant separation of scores by KPS. The minimal ceiling effect observed is a desirable property as a change from a high baseline level of unmet need can then be monitored.
Factor 6 (social and religious/spiritual support) had a particularly high floor effect with 58.2% endorsing the floor score. This may be due to the lower prevalence of problems and unmet needs with regard to spiritual issues in Asian countries compared to Western countries.[23] Dimensions of spirituality may also have varying salience in different cultures. In Asian culture, spirituality may be experienced more in terms of making sense of one's situation and role rather than as a matter of one's relationship with God.[2425] For example, a study on spiritual needs of cancer patients in China found that “to be accepted” and “to take responsibility” were more important than “to talk to someone,” “to pray,” and “to feel God with me.”[26]
Factor 3 (health-care staff) also had a high floor effect with 50.3% endorsing the floor score. The items in this factor related to the interactions with doctors and nurses, for example, to be “respected and treated as a person by my doctor” and to have “nurses to promptly attend to my discomfort and pain.” Having a low floor score may indicate that these aspects of professional care are routinely provided as part of standard care. Nonetheless, it is still a domain that is important to include in a comprehensive assessment of unmet needs, to identify and promptly address any gaps, where present.
A limitation of this study is that it was a cross-sectional study, and thus, we were unable to assess sensitivity to change or test–retest reliability. A longitudinal study was not done as our target sample of patients with advanced cancer were likely to have rapidly deteriorating functional status, and consequently, a high drop-out rate for a follow-up assessment. The low levels of unmet needs expressed by participants also make validity assessment difficult. Future studies may explore a shorter version of the CNAT, which will increase the feasibility of using this tool in the clinical setting.
CONCLUSION
The CNAT showed construct and known-group validity and internal consistency in this study sample and can be used to assess the unmet needs of advanced cancer patients in the Singapore context.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
Acknowledgments
GMY received salary support from the Lien Centre for Palliative Care.
REFERENCES
- Ministry of Health Singapore. Principal Causes of Death. Available from: https://www.moh.gov.sg/resources-statistics/singapore-healthfacts/principal-causes-of-death
- Race, treatment preferences, and hospice enrollment: Eligibility criteria may exclude patients with the greatest needs for care. Cancer. 2009;115:689-97.
- [Google Scholar]
- Concepts within the Chinese culture that influence the cancer pain experience. Cancer Nurs. 2008;31:103-8.
- [Google Scholar]
- Gender and ethnic differences in cancer patients’ needs for help: An internet survey. Int J Nurs Stud. 2008;45:1192-204.
- [Google Scholar]
- Dying from cancer in developed and developing countries: Lessons from two qualitative interview studies of patients and their carers. BMJ. 2003;326:368.
- [Google Scholar]
- Racial and ethnic differences in preferences for end-of-life treatment. J Gen Intern Med. 2009;24:695-701.
- [Google Scholar]
- U.S. Department of Health and Human Services Food and Drug Administration, editor. Guidance for Industry Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. Silver Spring, MD: Food and Drug Administration; 2009. p. :35.
- Comprehensive needs assessment tool in cancer (CNAT): The development and validation. Support Care Cancer. 2011;19:1957-68.
- [Google Scholar]
- Comprehensive unmet needs and correlations with quality of life in Chinese cancer patients. Eur J Cancer Care (Engl). 2018;27:e12813.
- [Google Scholar]
- Evaluation of psychometric properties of needs assessment tools in cancer patients: A systematic literature review. PLoS One. 2019;14:e0210242.
- [Google Scholar]
- The development of a comprehensive needs assessment tool for cancer-caregivers in patient-caregiver dyads. Psychooncology. 2011;20:1342-52.
- [Google Scholar]
- Cross-cultural adaptation and psychometric assessment of the Chinese version of the comprehensive needs assessment tool for cancer caregivers (CNAT-C) Qual Life Res. 2015;24:1607-14.
- [Google Scholar]
- The practice of nondisclosure of advanced cancer diagnosis in Singapore: A continuing challenge. Singapore Med J. 2013;54:255-8.
- [Google Scholar]
- Is therapeutic non-disclosure still possible? A study on the awareness of cancer diagnosis in China. Support Care Cancer. 2011;19:1191-5.
- [Google Scholar]
- Validation of the English and Chinese versions of the quick-FLIC quality of life questionnaire. Br J Cancer. 2005;92:668-72.
- [Google Scholar]
- Understanding Health Care Outcomes Research. Burlington, MA: Jones and Bartlett Publishers; 1997.
- Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling. 1999;6:1-55.
- [Google Scholar]
- In search of golden rules: Comment on hypothesis-testing approaches to setting cutoff values for fit indexes and dangers in overgeneralizing Hu and Bentler's (1999) findings. Struct Equ Modeling. 2004;11:320-41.
- [Google Scholar]
- Comparison of problems and unmet needs of patients with advanced cancer in a European country and an Asian country. Pain Pract. 2015;15:433-40.
- [Google Scholar]
- Quality of life domains important and relevant to family caregivers of advanced cancer patients in an Asian population: A qualitative study. Qual Life Res. 2015;24:817-28.
- [Google Scholar]
- An exploration of spiritual needs of Taiwanese patients with advanced cancer during the therapeutic processes. J Clin Nurs. 2011;20:950-9.
- [Google Scholar]
- Spiritual needs and their associated factors among cancer patients in China: A cross-sectional study. Support Care Cancer. 2018;26:3405-12.
- [Google Scholar]
