INTRODUCTION
The rapid increase in elderly populations leads to escalations in health problems. With an older population that continues to grow worldwide, global expenditure on health problems is expected to increase to $18 trillion by 2040, and this encourages policymakers to invest in proper countermeasures [1]. According to the World Health Organization (WHO) statistics, disability-adjusted life-years (DALYs), a measure of disease burden expressed as the number of years lost because of disability, have increased in non-communicable diseases 16% over the last 10 years [2]. Mental illness has been reported as one major cause for the increase in disease burden and medical expenditure. The WHO estimates that 6.6% of DALYs among the elderly can be attributed to neuropsychiatric disorders, such as dementia and depressive disorders [3]. DALYs have also increased 13.5% because of mental illness over the past decade [4]. Early detection and proper management of mental illness in the elderly are critical.
For this reason, our research team has tied to develop and implementation of effective psychosocial interventions for cognitive decline and depressive disorders via a community mental health service [5-7]. However, we have experienced some difficulties in the screening of elderly individuals who need community mental health services. First, conventional screening tools are somewhat complicated and time-consuming for population-based applications. Of course, widely used screening tests such as Beck’s depression inventory and Mini mental status exam, do not take much time to perform just one test each. However, there are needs to do multiple screening tests about each disesases to many subjects in community mental health service setting, and it takes a lot of time eventually. Previous studies have reported that complicated screening tools may not only lower response rates but also reduce the quality of the responses [8,9]. In addition, a comparison of a binary and a Likert-scale version of the Short-Form 36 Health Survey showed that replacing the Likert-scale version with a binary option did not decrease validity; but significantly reduced the time needed to complete the survey [10]. There has therefore been an increasing need for the development of briefer screening tools that are both easier to understand and to complete. Second, elderly people are likely to have several mental illnesses at the same time. It is estimated 50–90% of elderly individuals who have depressive disorder or an anxiety disorder have another comorbidity [11,12]. With limited financial and human resources of community mental health services, however, it is difficult to carry out number of conventional screening tools simultaneously.
Considering the above difficulties, we have developed a 14-items binary response (yes/no) screening questionnaire for most common mental illnesses of the elderly, called BS4MI-elderly (brief screening for four mental illnesses of elderly). The BS4MI-elderly was designed to assess four common and burdensome mental illnesses of the elderly in South Korea: dementia, depressive disorder, sleep disorder and the anger syndrome “hwa-byung.” The main objective of the present study was to develop a brief but comprehensive screening questionnaire to identify elderly individuals who need further assessment in community mental health service, and examine its reliability and validity.
METHODS
Participants
The study was conducted from February 2018 to November 2018 at the Suwon Geriatric Mental Health Center and the outpatient clinic of the Institute on Ageing at Ajou University Hospital, South Korea. The Suwon Geriatric Mental Health Center, which was established in 2008 and comprises outreach sites throughout the Suwon city’s districts, operates specialized mental health services targeting the elderly population. The center, which is run in cooperation with psychiatrists, social workers, nurses, and mental health professionals, provides case management for health promotion in the elderly, and offers various services including disease screening, basic counseling, education and resource linkage. Two hundred fifty-four participants with a mean age of 76.1±8.0 years (70.4% women) were recruited. Total 38 people were recruited from hospitals, and 216 from the community. Inclusion criteria were: 1) age higher than 65 years; 2) made written informed consent. Exclusion criteria were: 1) a history of psychotic disorders such as schizophrenia or mood disorder with psychotic symptoms; 2) a diagnosis of receptive or expressive language impairments that could interfere with the study; 3) a diagnosis of substance use disorder; 4) a history of neurological disorder, such as brain tumors, intracranial hemorrhage, subarachnoid hemorrhage, epilepsy, hydrocephalus, encephalitis, metabolic encephalopathy, or other neurologic conditions that could interfere with the study; and 5) a history of acute or severe medical illness, such as acute kidney or liver failure or metastatic cancer.
The study was approved by the Institutional Review Board of Ajou University (approval number: AJIRB-MED-SUR-17-436). All participants provided written informed consent.
Assessments and measurements
The BS4MI-elderly
BS4MI-elderly scores were the primary measurement in our study. The questionnaire was developed as a screening tool for four common mental illnesses (dementia, depressive disorder, sleep disorder, and hwa-byung). 6 psychiatrist and 1 psychologist made a committee and had regular meetings to make items of BS4MI-elderly. Other 2 psychiatrist and 3 social workers reviewed whether the items were simple enough and clinically useful. Globally, the prevalence of dementia, depressive disorder and sleep disorder is high, and South Korea also shows a similar pattern, increasing disease burden and decrease the quality of life in elderly [13-18]. Hwa-byung, culturerelated “anger syndrome” diagnosed in South Korea, is also a common and burdensome disease of elderly [19].
It is characterized by unique symptoms that include subjective suppressed anger and somatic and behavioral manifestations of expressed anger [20], such as: subjective anger, external anger, feelings of unfairness, heat sensations, pushingup sensations in the chest, guilty feelings, anxiety, insomnia, and agitation. Hwa-byung has a high comorbidity with other psychiatric diseases such as generalized anxiety disorders, major depressive disorder and anger disorders [21]. The prevalence of hwa-byung in South Korea is estimated to be around 4.1% overall, and higher in women. The condition is listed in the Glossary of Culture-bound syndrome of Diagnostic and Statistical Manual of Mental Disorders (DSM), fourth edition.
The BS4MI-elderly consists of 12 binary response symptom questions, with three items for each disease, and two additional binary response questions to evaluate the course and duration of the illnesses (Table 1). Two of the three symptom items are related to core symptoms of the disease and one item asks about additional symptoms of the disease. We formulated questions about the key components of the disease that many other screening tools ask in common, but we did not use the exact same question of other test tools.
For dementia, memory functions are evaluated with two items and one item assesses instrumental activities of daily living (IADL). For depressive disorder, two items ask about mood symptoms, and one item assesses suicidal ideation that could have the most serious consequences. In the case of hwabyung, two items assess the core symptoms of the disease, anger, and one item is related to the somatic symptoms. For sleep disorder, two items ask about the degree of sleep disturbances, one item assesses restless leg syndrome, which is a relatively common cause of sleep disorder in the elderly [22].
Screening tools usually divide subjects into a normal group that is, not in urgent need of medical service, and a risk group, that may need further evaluation and early intervention. We designed the BS4MI-elderly to divide subjects into three groups- a “normal” group, a “risk” group, and a “disorder” group, the added disorder group as those individuals who may have more urgent problems and need medical attention. Participants who reported to have no symptoms were classified as the normal group. Individuals who had at least one symptom and not satisfying disorder group criteria are classified as risk group. Participants who had symptoms that had lasted more than 1 month and were getting worse were classified as the disorder group.
Conventional scale for each disease
Conventional scales for each disease were also used. Dementia was assessed using the Korean dementia screening questionnaire (K-DSQ), which consists of 15 items that can detect changes in early cognitive decline. In the K-DSQ, each item is scored from 0 to 2, with higher scores indicating poorer function. A community-based study in South Korea suggested an optimal cut-off score for screening of 11 or higher on the K-DSQ [23].
Depressive disorder was measured using the Korean version of the 15-item geriatric depression scale-short form (SGDS-K), introduced by Sheik and translated into Korean by Bae and Cho. SGDS-K scores range from 0 to 15 and are highly correlated with the 30-item original Korean version [24]. Previous studies suggest that the optimal cut-off point for screening major depressive disorder is a SGDS-K score of 8 or higher.
Sleep disorder was measured using insomnia severity index (ISI), consists of seven items, with each item rated on a scale from 0 to 4, and a higher score indicating greater sleep disorder severity. The index was developed by Sohn and translated into Korean by Cho et al., [25] and an ISI cut-off score of 15 is used as a threshold for significant sleep disorder.
Hwa-byung was measured using a hwa-byung symptom scale, developed and validated by Kwon et al. in 2008 and, simplified by Choi et al. in 2015 [26,27]. The hwa-byung symptom scale consists of 15 items with item scores ranging from 0 to 4. Higher scores indicates more severe symptoms, and a score of 30 is clinically useful as a cut-off score.
Scales for general mental health
Scales not related to specific disease but measuring general mental health were also used. Subjective happiness was measured using the concise measure of subjective well-being (COMOSWB), which was developed by Seo and Koo and, consists of nine items with each items rated on a scale from 0 to 7 [27]. The total score of the COMOSWB can range from -21 to 42, because it also includes three items for negative emotions that have negative scores. The 12-item general health questionnaire (GHQ-12) is a short version of the general health questionnaire (GHQ), asking common psychopathologies to screen mental illness in primary care settings [28]. Not only the original version, but also the short version and the Korean versions have been validated in multiple studies [29,30]. GHQ scores ranged from 0 to 12, with higher scores indicating higher levels of pain. The brief resilience scale (BRS) was developed to assess participant’s ability to bounce back or recover from stress [31]. The BRS consists of three positive items and 3 negative items. Each question has a maximum score of 5 points, with total scores thus ranging from -15 to 15 points. To measure stress levels in our subjects, we used a scale that consisted of one single item, the stress level that the participant experienced during the past week, rated from “very severe” (10 points) to “not at all” (0 points). The effectiveness of such a single-item stress test tool has been verified, mainly in the field of stress in working environment [32].
Data analysis
Sensitivity and specificity rates, as well as the predictive value of positive test and the likelihood ratio of positive test, were estimated from 2×2 tables. Homogeneity of variance was examined using the Levene test and, as variances were not homogenous, Welch’s robust test and the Dunnett’s T3 test were used to test the validity of the BS4MI-elderly, by evaluating conventional scale score differences among participant groups divided by the BS4MI-elderly. Cronbach’s α was used to evaluate the internal consistency of the BS4MI-elderly. Pearson’s correlation was used to examine the correlation between the BS4MI-elderly and the scales for general mental health. SPSS 20.0 for Windows (IBM Corp., Armonk, NY, USA) was used to perform all statistical analysis.
RESULTS
Participant characteristics
The participants’ demographics were described using descriptive statistics (Table 2). The mean ages of all participants was 76.1 years. Females accounted for a higher percentage of participants than men, at 73.1%. More than half of the participants had below elementary school education, and the rate of participants with a university level education was below 10%. Participants’ mean K-DSQ score, for screening dementia, was 7.0±5.6 and the percentage of participants whose K-DSQ score was higher than the cut-off point was 22.0%. Participants’ mean SGDS-K score was 7.1±4.8, and the percentage of participants with a score above the cut-off point was 43.7%. 26.4% of participants had a hwa-byung scale score higher than the cut-off, and the mean score of the hwa-byung scale was 20.9±13.7. The mean ISI score was 9.6±72, and the percentage of participants whose ISI score was higher than the cut-off point was 25.2%.
Validity of the BS4MI-elderly
To test the validity of the BS4MI-elderly, we analyzed the sensitivity, specificity, predictive value of positive test and likelihood ratio of positive test. Considering limited financial and human resources in community mental health services, testing the validity of the BS4MI-elderly using clinical diagnoses did not seem practical. We therefore designed our study to validate the BS4MI-elderly’s efficacy by screening out participants whose conventional scale score was above the cut off point. Because the BS4MI-elderly has two cut-off points that subdivide three groups, the sensitivity and specificity for each cut-off point are presented separately in Table 3. The sensitivity of the first cut-off point that divides the normal group from the risk group and the disorder group was determined as follows: dementia: 0.600, depressive disorder: 0.881, insomnia: 0.857, hwa-byung: 0.939. The specificity of the second cut-off point that divides the disorder group from the normal group and the risk group was determined as follows: dementia: 0.923, depressive disorder: 0.943, insomnia: 0.856, hwabyung: 0.876.
Comparisons of mean conventional scale scores for the groups classified using the BS4MI-elderly are depicted in Table 4. The mean conventional scale score of the normal group was significantly lower than those of the risk and disorder groups for each disease; and the mean conventional scale score of the disorder group was higher than those of the normal and risk groups for all diseases; however, the K-DSQ dementia score difference between the risk group and the disorder group was not statistically significant.
Internal consistency of the BS4MI-elderly
To test the internal consistency of the BS4MI-elderly, Cronbach’s α was analyzed, as shown in Table 5. Cronbach’s α for the three dementia items in the BS4MI-elderly was 0.587, which indicates moderate internal consistency. Item 1 was less correlated with the other items. Cronbach’s α for the depressive disorder items and hwa-byung items was 0.727 and 0.757, respectively indicating high internal consistency for both diseases. Cronbach’s α for the sleep disorder items was 0.570, indicating moderate internal consistency, with item 12 being less correlated with the other sleep disorder items. Potential reasons for the moderate internal consistency are described in the Discussion section below. In order to confirm the associations between all BS4MI-elderly scores, Cronbach’s α was evaluated for all four disease scores separately.
Correlation analysis between general mental health scales and BS4MI-elderly
To test correlations between the BS4MI-elderly and scales assessing general mental health, for example, resilience, subjective happiness, and stress, Pearson’s correlation was used, as shown in Table 6. As expected, the BS4MI-elderly score was negatively correlated with the Brief Resilience score (Pearson coefficient r=-0.47) and COSMOSWB score (Pearson coefficient r=-0.59); and positively correlated with the stress score (Pearson coefficient r=0.58) and the GHQ-12 score (Pearson coefficient r=0.64). All correlations were statistically significant.
DISCUSSION
We studied the validity of the BS4MI-elderly, a brief but comprehensive binary screening test designed to assess four common and burdensome diseases of the elderly. We examined the sensitivity, specificity, likelihood ratio of positive test, and positive predictive value of BS4MI-elderly cut-off points. The cut-off points that divides normal participants from those in the risk and disorder groups had high sensitivity. The cut-off points that divides the disorder group from the risk and normal group had high specificity. These results are consistent with the design purpose of the scale. We also tested the validity of the BS4MI-elderly by comparing the mean conventional scale scores of the normal, risk and disorder groups divided by the cut-off points of the BS4MI-elderly. The conventional scales scores have positive correlation with BSQ4MI score in all diseases, and all the conventional scale score differences between groups divided by BS4MI-elderly are significant except in dementia. Cronbach’s α was used to evaluate internal consistency for each disease. Depressive disorder and Hwa-byung showed high internal consistency. The BS4MI-elderly was compared with scales that evaluate general mental health rather than with specific diseases, and all of these scales showed significant correlations.
This study has following strong points. First, we tried to examine the validity of the BS4MI-elderly precisely by comparing the mean scores of conventional scales between groups divided using the BS4MI-elderly via a multiple comparison analysis, and also using sensitivity and specificity values. Sensitivity was moderate to high for the cut-off point dividing the normal group from risk group, disorder group, suggesting that the scale is appropriate for the screening it was designed for. The cut-off points dividing disorder groups from risk group, normal group had high sensitivity, thereby potentially providing a basis for faster intervention. In the multiple comparison analysis, the differences in the mean scores of the conventional scale between the normal group and the other groups were large and significant. Although the difference was not significant for all diseases, the mean conventional scale scores of the disorder group were always higher than those of the other groups. This finding shows that the classification of subjects using the BS4MI-elderly is in line with the conventional scales.
Second, we evaluated the correlation between the BS4MI-elderly and general mental health. The BS4MI-elderly showed significant correlations with the Brief Resilience scale, the COMOSWB scale, and the GHQ-12 scale, indicating that people with mental illness feel less happy, less resilient, and more stressed, consistent with previous research [33]. The GHQ, developed to screen mental illness in primary care settings, also correlated with the BS4MI-elderly, with the strongest correlation coefficient among general mental health scales. The GHQ was developed with similar purpose as the BS4MI-elderly, so strong correlation is to be expected. Unlike the GHQ, however, that consists of questions about common psychiatric pathology, the BS4MI-elderly targets specific diseases that are most common in the elderly. The BS4MI-elderly has another strong point, in that it allows a comprehensive assessment by measuring disease duration and deterioration in comparison with the GHQ. Community mental health services not only targets the elimination of psychopathology, which is the ultimate goal of clinical treatment, but also has larger and more inclusive goals, such as the well-being of neighbors in a community and the prevention of and rehabilitation from diseases. We thus broadened the scope of the BS4MI-elderly, by confirming the relationship between the BS4MI-elderly and various mental health measures.
Third, we evaluated the internal consistency of the items used in the BS4MI-elderly by examining Cronbach’s α. Depressive disorder and hwa-byung showed a high α-value, as expected. In accordance with previous studies, a high overall α value for the diseases in the BS4MI-elderly indicates that there is a high tendency for individuals to have more than one disease at the same time. As for the insomnia items, item 9 shows small internal consistency with other insomnia items. Unlike the questions about other diseases, item 9 asks about restless leg syndrome (RLS), a completely different pathology from other questions about sleep disorders. It was thus predictable that this item would be less related to other items.
This study has several limitations. First, the lack of a comparison between the BS4MI-elderly and final clinical diagnoses by clinical expert can be a limitation of this study. However, we sought to obtain objectivity through careful comparison with well-proven conventional scales. Development purpose of BS4MI-elderly was not to replace clinical diagnoses, but to replace performing multiple conventional screening tools, which would cost a lot of resources. Second, we collected few demographic data from the participants. Third, participants were recruited from community mental health care centers and university hospitals, and therefore represent a rather inhomogeneous group with varying characteristics. However, patients with a sufficient number of diseases were needed to validate the screening tools, and hospital recruitment was therefore necessary. Fourth, sensitivity of the cut-off point dividing normal group from risk group, disorder group in dementia was moderate at 0.60. This implies symptoms asked in BS4MI-elderly’s dementia items are too severe for screening. It results from our intention to prevent the false positives as previous study suggests participants’ lower education level was associated with higher false-positives in brief cognitive assessments [34]. Furthermore, as previous studies have pointed out, self-reporting of dementia is likely to lead to some underestimation [35].
In conclusion, our study confirms the utility of a brief but comprehensive binary response screening questionnaire available to elderly populations in community mental health services. We expect this questionnaire to be used extensively in elderly populations who have difficulties in performing screening tests in community mental health services in the future.
