INTRODUCTION
Stigma is classically defined as “pertaining to the shame that a person may feel when he or she fails to meet other people’s standards, and to the fear of being discredited—which causes the individual not to reveal his or her shortcomings [1].” Social psychology characterizes stigma as the disapproval of a person based on physical or behavioral characteristics that distinguish them from others. Stigma is frequently attached to individuals with mental disorders and is associated with negative consequences including discrimination and rejection [2]. Traditionally, stigma research has focused on schizophrenia, one of the most stigmatized mental disorders [3]; however, depression has garnered increasing research attention [4-7]. Depression is a common mental disorder and one of the leading causes of disability worldwide [8]. Stigma related to depression is associated with various disadvantages in social participation and vocational integration [7].
Stigma is most often measured in terms of public attitudes toward specific scenarios rather than through the experiences of individuals with mental health problems [9]. The Perceived Devaluation and Discrimination Scale [10] and the Self-stigma of Mental Illness Scale are examples of tests that measure public attitudes [11]. The primary limitation of this approach, i.e., the absence of direct assessment of the stigmatized person’s experience, is that stigma may be overestimated, particularly in terms of problems related to knowledge (ignorance) and to attitudes (prejudice) [2]. Instruments that measure experienced stigma in people with mental health problems include the Consumer Experiences of Stigma Questionnaire [12] and the Rejection Experience Scale [13]. However, most measures of experienced stigma have not been psychometrically validated [14].
The Discrimination and Stigma Scale (DISC 12) was developed to fill this research gap and focuses on the responses and experiences of stigmatized persons [15]. The psychometric properties of the DISC 12, including reliability, validity, acceptability, and feasibility, have been demonstrated [14]. Several studies have used the DISC 12 to assess the overall patterns of experienced discrimination and anticipation [16,17]. The DISC 12 has been translated into several languages and has been used in studies of schizophrenia in 27 countries [15] and of depression studies in 30 countries [7].
No Korean version of the DISC 12 has been available; therefore, we developed and standardized a Korean version of the test (DISC 12-K) for use in Korean patients with depressive disorders.
METHODS
Study protocol
This study is a sub-study of the MAKE Biomarker discovery for Enhancing anTidepressant Treatment Effect and Response (MAKE BETTER) trial, which was undertaken to develop a treatment–response prediction index of biomarkers for patients with depressive disorders. Details of the study have been published as a design paper [18] and registered at cris.nih.go.kr (identifier: KCT0001332). Briefly, the naturalistic 2-year prospective study was designed to identify biomarkers for predicting treatment response in real-world settings. Therefore, participants were enrolled regardless of depression subtype or physical comorbidity. Treatment interventions were performed naturalistically using the type, dose, and regimen of antidepressant and other medications preferred by the patients and prescribed by the clinicians; however, the protocol was guided by pre-planned assessments at given time points. The DISC 12-K was administered 1 year after the initiation of antidepressant therapy to assess the level of stigma experienced in the prior 12 months. Demographic and clinical characteristics were obtained at the same time as assessment tests were administered. The DISC 12-K was re-administered 4 weeks later to assess test–rest reliability. All data were obtained using a structured clinical report form (CRF) completed by clinical research coordinators who were blinded to the treatments. The clinicians were trained by the research psychiatrists on the use of the CRF and data collection methods. The study was approved by the Chonnam National University Hospital Institutional Review Board (CNUH 2013-163).
Participants
Participants in the MAKE BETTER study who attended the outpatient psychiatric department of Chonnam National University Hospital and had been followed for 1 year were consecutively enrolled in our study between November 2013 and August 2017. The research psychiatrists used the MiniInternational Neuropsychiatric Interview (MINI) [19], a structured diagnostic psychiatric interview based on the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) criteria, to diagnose depressive disorders [20]. Inclusion criteria were 1) diagnosed with major depressive disorder (MDD), dysthymic disorder, or depressive disorder not otherwise specified (NOS); 2) aged 18–70 years; 3) able to complete questionnaires, understand the objective of the study, and sign the informed consent form. Exclusion criteria were 1) unstable or uncontrolled medical condition; 2) unable to complete the psychiatric assessment or comply with the medication regimen due to a severe physical illness; 3) current or lifetime DSM-IV diagnosis of bipolar disorder, schizophrenia, schizoaffective disorder, schizophreniform disorder, psychotic disorder NOS, or other psychotic disorder; 4) history of organic psychosis, epilepsy, or seizure disorder; 5) hospitalization for any psychiatric diagnosis apart from depressive disorder (e.g., alcohol/drug dependence); and 6) electroconvulsive therapy received for the current depressive episode. All participants reviewed the consent form and provided written informed consent.
Original and Korean versions of the DISC 12
The DISC 12 is a structured interview for assessing discrimination experienced by individuals with a mental disorder. The DISC 12 contains 32 questions concerning aspects of everyday life including work, marriage, parenting, housing, and leisure and religious activities divided into four subscales: 1) Unfair Treatment (items 1–21) measures experienced discrimination (e.g., “Have you been treated unfairly in making or keeping friends?”); 2) Stopping Self (items 22–25) measures anticipated discrimination (e.g., “Have you stopped yourself from applying for work?”); 3) Overcoming Stigma (items 26 and 27) assesses the coping strategies patients use to overcome discrimination (e.g., “Have you been able to use your personal skills or abilities in coping with stigma and discrimination?”); 4) and Positive Treatment (items 28–32) assesses positive treatment received as a result of a mental health problem (e.g., “Have you been treated more positively by your family?”). Responses were rated on a four-point Likert scale (0=not at all, 1=a little, 2=moderately, and 3=a lot) with a “not applicable” response option for items that the patients judged as not relevant to their situation. Scores on the Positive Treatment subscale were reverse coded so that a high score indicated a lack of positive treatment or occasions when the person received help or support due to their mental health problem that was not available to others. The total score for each subscale was generated by counting the number of items on which the score was 1, 2, or 3 (disadvantage scores), with higher scores indicating greater stigma.
Formal permission to translate the DISC-12 into Korean was obtained from its developer (Prof. G. Thornicroft). The standard translation procedure was used, which included forward and backward translation as well as pilot tests for acceptability and feasibility. The final DISC 12-K is available in the online supplement. The DISC 12-K is administered and scored in the same way as the original DISC 12. We used the DISC 12-K to assess experienced stigma during the 12 months after the initiation of antidepressant treatment.
Patient characteristics
The sociodemographic characteristics obtained included age, sex, years of formal education, marital status (currently married or not), cohabitation status (living alone or not), religion (religious affiliation or no religious preference), occupation (currently employed or not), and annual income (above or below $2,000 USD). The clinical characteristics included duration of illnesses, number of previous depressive episodes, history of psychiatric hospitalization, and awareness of illness (agree or disagree with diagnosis).
Assessment scales
Internalized Stigma of Mental Illness (ISMI) scale
The ISMI scale is a 29-item self-report measure designed to assesses mental health service users’ subjective experience of internalized stigma [21]. The scale is composed of five subscales: alienation, stereotype endorsement, perceived discrimination, social withdrawal, and stigma resistance, with higher scores indicating greater stigma. The reliability and validity of the ISMI scale were established in a sample of 127 mental health outpatients [21], and the scale has been formally standardized in Korean [22].
Other psychometric assessment scales
Several psychometric scales were used to assess participants’ symptoms and level of functioning. Depressive symptoms were assessed objectively using the Hamilton Depression Rating Scale (HAMD) [23] and subjectively using the Beck Depression Inventory (BDI) [24]. Level of functioning was assessed using the Social and Occupational Functioning Assessment Scale (SOFAS) [20], self-esteem using the Rosenberg Self-Esteem Scale (RSES) [25], and quality of life using the EuroQol-5D (EQ-5D) [26]. Higher scores on the HAMD, BDI, RSES, and EQ-5D and lower scores on the SOFAS indicated more severe symptoms. All of the assessment scales have been formally standardized in Korean [27-31].
Statistical analysis
Descriptive data are expressed as mean (standard deviation, SD), minimum–maximum values, or as number (percentage), as appropriate. The proportion of “Agree” responses for each DISC 12-K item is expressed as the combined total of the “disadvantage” response categories (a little, moderate, and a lot). Reliability and validity of each DISC 12-K subscale were assessed. Reliability was assessed as internal consistency using Cronbach’s alpha coefficient and by estimating the intercorrelation of items and corrected item-total correlations. The interrater reliability and test-retest reliability were assessed using intraclass correlation coefficients at the item and subscale levels. The concurrent validity of the DISC 12-K with the other assessment scales was assessed using Spearman’s correlation coefficient. All statistical analyses were performed using SPSS software version 21.0 (IBM Corp., Armonk, NY, USA).
RESULTS
Patient characteristics
The study included 230 patients with depressive disorders. Patient sociodemographic and clinical characteristics are shown in Table 1. The proportion of responses for each DISC 12-K item is shown in Table 2. Disadvantage was frequently reported (>20%) on items 1 (making or keeping friends), 3 (dating or intimate relationships), and 7 (family) of the Unfair Treatment subscale; on items 24 (close personal relationship) and 25 (concealed or hidden mental health problem) of the Stopping Self subscale; and on all five items of the Positive Treatment subscale. In contrast, the frequency of reported disadvantage was low (5–6%) for the two items on the Overcoming Stigma subscale. Moreover, on each of the DISC 12-K items, many participants (25.7–59.6%) selected the “not applicable” option. Scores for all of the psychometric scales are shown in Table 3. The mean scores on the HAMD (5.9), BDI (8.0), and SOFAS (80.3) tests indicated that most of the participants were stable.
Reliability analyses
The findings of the reliability analyses of the DISC 12-K subscales are shown in Table 4. Cronbach’s alpha coefficients were high for all of the DISC 12-K subscales (0.919–0.987). The item intercorrelations were high for the Unfair Treatment, Stopping Self, and Overcoming Stigma subscales and slightly low for the Positive Treatment subscale. The corrected itemtotal correlations were high for all subscales. The clinical research coordinators assessed the interrater reliability of the DISC 12-K in a subsample (n=30) of patients. The intraclass correlation coefficients for interrater agreement were high at the individual item and subscale levels. The DISC 12-K was readministered to 200 patients 1 month after they completed the initial test to evaluate test-retest validity. The intraclass correlation coefficients for test-retest stability were high at the item and subscale levels.
Validity analyses
The correlations among DISC 12-K subscale scores and between the DISC 12-K total and other assessment scale scores are shown in Table 5. The only significant correlations were between the Unfair Treatment and Stopping Self subscale scores. Comparison of the DISC 12-K and ISMI scales revealed that the Unfair Treatment and Stopping Self subscale scores were significantly correlated with all of the ISMI subscale scores. Furthermore, a significant inverse relationship was found between the Overcoming Stigma subscale and the ISMI Stigma Resistance subscale. The Positive Treatment subscale was not correlated with any of the ISMI subscales. The DISC 12-K Unfair Treatment and Stopping Self subscale scores were significantly correlated with the HAMD, BDI, SOFAS, EuroQol-5D, and RSES scores; thus, higher stigma scores were associated with more severe pathology. Scores on the Overcoming Stigma and Positive Treatment subscales were significantly correlated with RSES scores, but not with the HAMD, BDI, SOFAS, or EuroQol-5D scores.
DISCUSSION
Our standardization study of the DISC 12-K in Korean patients with depressive disorders found that the four DISC 12-K subscales had high reliability. The validity of the Unfair Treatment and Stopping Self subscales was good; however, the Overcoming Stigma and Positive Treatment subscales had only fair validity.
The DISC 12 is widely used to assess experienced stigma in patients with mental disorders [9]. Although the DISC 12 has been translated into several languages and is frequently used in international research [7,15], only a few validation studies have been conducted [14,32]. Thus, our study is one of the few studies to validate the DISC 12. Cronbach’s alpha coefficients and inter-item correlations were high for all of the subscale scores, indicating that the questionnaire construction was consistent across the subscale items. The interrater reliability was good, suggesting that the DISC 12-K items could be rated uniformly and consistently by assessors. The test–retest reliability was also good, suggesting that the DISC 12-K reflected particular individual characteristics consistently over time.
The scores on the Unfair Treatment and Stopping Self subscales of the DISC 12-K were highly correlated, indicating that the subscale items were based on similar constructs and measured related aspects of stigma. Moreover, the Unfair Treatment and Stopping Self subscales were significantly correlated with all of the ISMI subscale scores and the HAMD, BDI, SOFAS, RSES, and EuroQol-5D scores. These findings indicate that the experienced stigma measured by two DISC 12-K subscales reflects depressive symptoms per se as well as social functioning, self-esteem, and quality of life, which are multi-dimensional aspects of depression. Together, our findings suggest that the Unfair Treatment and Stopping Self subscales of the DISC 12-K are valid instruments for measuring experienced stigma in patients with depressive disorder.
Scores on the Overcoming Stigma and Positive Treatment subscales had good reliability but only fair validity. These subscale scores were not correlated with the Unfair Treatment and Stopping Self subscales scores or with any of the ISMI subscale scores. Although the Overcoming Stigma and Positive Treatment subscale scores were correlated with the RSES score, they were not significantly associated with the HAMD, BDI, SOFAS, or EureQol-5D scores. These findings suggest that the Overcoming Stigma and Positive Treatment subscales are not valid measures of experienced stigma in patients with depressive disorders. Previous studies have reported similar results; only the Unfair Treatment and Stopping Self subscales have been used in international research [7], and a previous DISC 12 validation study found that the Unfair Treatment subscale was the only valid measure of experienced discrimination [14]. Taken together, our findings and those of previous studies support the use of the Unfair Treatment and Stopping Self subscales as valid measures of stigma.
However, it should be noted that the proportion of “disadvantage” responses to individual items in the Overcoming Stigma and Positive Treatment subscales was low (5 items had scores >20%) (Table 3) compared with responses reported previously in patients with depression (10 items scored >20%) [7]. This difference may be explained by the fact that we administered the DISC 12-K when the participants were in the continuation or maintenance phase of antidepressant therapy (i.e., 1 year after the initiation of treatment), so most were stable. However, data concerning the depression severity and antidepressant treatment status of the patients in the previous study was not available. Nevertheless, despite differences in study design and the proportion of “disadvantage” responses, our findings and those of the previous study indicate that higher levels of experienced stigma were associated with poor social functioning.
The strengths of our study include the sample size (n=230), which was larger than those of previous DISC 12 validation studies (n=86–89) [14,32], and the structured research protocol and use of well-established, standardized scales. The limitations of our study include the fact that participants were recruited from a single center, which may limit the generalizability of our findings, although a single-center study has potential strengths in terms of consistency in evaluation and treatment. Furthermore, participants included patients with dysthymic disorder and depressive disorder NOS as well as MDD, whereas previous studies focused on patients with MDD [7,32]. However, our findings were similar to those previously reported, indicating that the DISC 12 is a useful assessment tool for a broad range of depressive disorders.
In summary, the new DISC 12-K is a reliable and valid instrument for assessing stigma in patients with depressive disorders. In particular, the Unfair Treatment and Stopping Self subscales have good reliability and validity. Recently, stigma has been recognized as target for depression treatment, and the effect of stigma reduction on social rehabilitation has received considerable research attention [33]. The DISC 12-K offers a standardized and efficient approach to assessing stigma and helping patients reach treatment goals. We recommend that this instrument be administered as a complementary tool to existing assessment scales, as it may provide unique information that could be critical for comprehensive assessment in depressive patients. Future studies are needed to assess the validity of the DISC 12-K in other mental disorders. In addition, consequences of stigma should also be investigated since stigma may have negative impacts in patients with mental disorders [34].
