INTRODUCTION
Emotion is a crucial factor that allows individuals to pay attention to external stimuli, adjust decision-making, and shape their behavior. It is also very beneficial in coordinating social interactions. However, emotions can be problematic when the type, intensity, and duration are inappropriate [1]. Therefore, appropriate emotion regulation is central to human adaptation. Previous studies have confirmed that the ability to regulate emotions successfully is critical to many domains of functioning. In contrast, difficulties in emotion regulation act as a key risk factor for psychopathological symptoms [2]. Thus, accurate assessment and intervention for emotion regulation in clinical settings are essential, and developing and validating reliable measures of emotion regulation is required.
According to Gross, emotion regulation refers to the “processes by which individuals influence which emotions they have, when they have them, and how they experience and express these emotions.” [3] In his process model of emotion, emotion regulation is conceptualized as selecting, modifying, paying attention, and appraising situations to yield particular responses over time [4,5]. One of the most widely adopted scales for emotion regulation is the Emotion Regulation Questionnaire (ERQ) [4], which is based on the model by Gross [5]. The ERQ addresses two main strategies of emotion regulation; cognitive reappraisal and expressive suppression. Cognitive reappraisal is a form of cognitive change and is among the antecedent-focused emotion regulation process. It involves reevaluating the emotion-eliciting situation to regulate the emotional impact. Cognitive reappraisal is often used to decrease negative emotions and is associated with positive mental health [6]. Whereas expressive suppression is a response modulation and refers to the inhibition of emotion-expression behavior after generating an emotional response. Expressive suppression is known to be associated with poor memory and less positive relations with others [7]. The ERQ consists of 10 items and has been validated in several languages and cultures including, Italy, Spain, and Taiwan [8-10]. Its brevity, and excellent reliability and validity, make the questionnaire one of the most widely used tools in measuring emotion regulation.
Unfortunately, although the ERQ has also been commonly used within the Korean population [11,12], its psychometric properties have not been investigated yet. The lack of cross-cultural validation is problematic, especially when cultural values play a critical role in the experience of emotions and the employment of specific regulation strategies. Indeed, a few validation studies that have been administered in other countries have failed to replicate the original structure of the two independent factors in Gross and John [4]. For instance, the original ERQ factor structure could not be confirmed using a community sample in Australia and the United Kingdom [13]. The researchers found that two items showed high covariance with each other and proposed a 9-item version of the ERQ by removing one item. Another comparable finding was also observed in a study investigating the psychometric properties of the ERQ using an Italian undergraduate and community sample [14]. They found a poor model fit when testing the original factor structure and consequently removed two items from the scale as the items had high error covariance. Furthermore, Wiltink and Glaesmer [15] demonstrated that the factor structure of the original ERQ could not be supported in a German community sample as the result of the confirmatory factor analysis showed inadequate fit, with one item from the cognitive reappraisal scale loading onto both factors (cognitive reappraisal and expressive suppression). Therefore, to achieve a good model fit, they modified the model so that the one item could cross-load on both factors. Overall, this discrepancy in factor structure demands cross-cultural evaluation, and there is currently limited evidence on the tool’s utility with the Korean population.
Besides the potential cultural differences, other factors may contribute to the variation in the factor structure between the samples. Such as, in regards to the low fit indices of confirmatory factor analysis obtained in one Italian sample [16], the authors suggest that this may be due to the methodological issues associated with the translation of the items. More specifically, they addressed that the Italian translation of the ERQ provides more examples that allow the respondents to distinguish between positive and negative emotions better than the German version of the ERQ.
Another noteworthy point is that even though the original scale was developed using a non-clinical sample, an increasing number of studies have utilized the ERQ to examine the specific impairment in emotion regulation within different mental disorders [17-20]. Recent studies using the ERQ include individuals with experience of trauma or posttraumatic stress disorder symptoms [21-24], eating disorders [25,26], bipolar disorders [27], social anxiety disorder [28], and obsessive-compulsive disorder [29]. However, the psychometric properties of the ERQ in the clinical setting have not been fully verified yet. To the best of our knowledge, only one study has examined the factor structure of the ERQ using a clinical sample. Marco et al. [30] analyzed the psychometric properties of the Spanish version of the ERQ in participants with personality disorder, with the majority having a borderline personality disorder. Although this study demonstrates the instruments’ validity outside the non-clinical population, it is limited in that the sample is focused narrowly on personality disorders, providing only partial empirical support for the use of ERQ in clinical settings.
The ERQ is also commonly used in clinical and research contexts in Korea as one of the primary instruments in assessing emotion regulation [11,31-35]. Studies utilizing the Korean version of the ERQ in a clinical sample include investigating the mental health of youths with traumatic experience [31], emotion regulation of patients with bipolar disorder in relation to their working memory [32], and the characteristics of adolescents diagnosed with depression [33]. There are also studies involving non-clinical samples. These include research examining the cultural and gender differences in emotion regulation between Korean and American college students [11], emotion regulation of male adults in association with their brain activity [34], and the contribution of emotion regulation on attentional processing of healthy adults [35].
In summary, considering the ERQ’s common usage with psychiatric patients internationally and the heterogeneity between community and clinical sample on emotion regulation [19,36], it is imperative to test the reliability and validity of the ERQ in the clinical population. The present study investigates the psychometric properties of the Korean version of the ERQ (K-ERQ) and evaluates whether the two-factor structure is also replicated in the Korean population. In particular, we were interested in individuals with mental disorders as there is limited data on whether the assessment is also suitable outside the community and the student sample. Therefore, the current study explored the factor structure, internal consistency, and concurrent validity of the K-ERQ using a sample of psychiatric patients.
METHODS
Participants
This study was approved by the Institutional Review Boards of the corresponding author’s medical center (HYUH 2021-03-058-004). The data were obtained from patients who visited the psychiatric department of a university medical center from February 2020 to March 2021. Our sample consisted of 193 patients with an average age of 29.90 (age range=18 to 79, SD=13.58), and 69.4% of the total sample were male. Most participants (96.4%) have at least graduated from high school as 53.9% of the participants reported a high school diploma as their highest degree of education, and this was followed by a college degree (23.3%), associate degree (13.0%), graduate degree (6.2%), and middle school diploma or lower (3.6%). Primary diagnoses were made using a Structured Clinical Interview for DSM-5 [37]. The majority of the patients had multiple diagnoses, and the average number of diagnoses was 2.00 (SD=0.93). The exclusion criteria were current diagnosis of 1) mood disorders with psychotic features, 2) schizophrenia spectrum disorders, 3) intellectual disability, or 4) neurocognitive disorders. The detailed sample characteristics are presented in Table 1.
Among the mood disorder subgroups, the majority of the patients met the criteria for a current episode. Within the major depressive disorder (MDD) group, 4 participants were in partial remission while 67 participants had a current episode of MDD and the severity of their episode were as follows: mild (n=7), moderate (n=43), and severe (n=17). For persistent depressive disorder (PDD) group, the subtypes are as follows: with pure dysthymic syndrome (n=1), with persistent major depressive episode (n=5), with intermittent major depressive episodes, with current episode (n=13), and with intermittent major depressive episodes, without current episode (n=2). There were 3 participants with bipolar I disorder and 16 participants with bipolar II disorder in the bipolar disorder group (n=19). For patients with bipolar I disorder, the subtypes were: current or most recent episode depressed, moderate (n=1), current or most recent episode manic, mild (n=1), current or most recent episode manic, in full remission (n=1). Lastly, for patients with bipolar II disorder, the subtypes were: current or most recent episode depressed, moderate (n=9), current or most recent episode depressed, severe (n=2), current or most recent episode depressed, in partial remission (n=2), current or most recent episode hypomanic, severe (n=2), and current or most recent episode hypomanic, in full remission (n=1).
Materials
The Korean version of the Emotion Regulation Questionnaire (K-ERQ)
The ERQ [4] is a 10-item measure that assesses the respondent’s habitual use of cognitive reappraisal and expressive suppression. The cognitive reappraisal subscale consists of six items and contains questions such as “When I want to feel more positive emotion (such as joy or amusement), I change what I’m thinking about.” The expressive suppression subscale is four items and includes questions such as “I keep my emotions to myself.” Both subscales require participants to evaluate on a 7-point Likert-type scale (1=strongly disagree, 7=strongly agree). The present study utilized the K-ERQ translated by Shon [38]. This translation is the widely used version of the ERQ by many researchers and clinicians in Korea (for example, Oh et al. [32] and Lee and Jang [39]). The internal reliability in Shon’s [38] study for cognitive reappraisal and expressive suppression subscale was 0.85 and 0.73, respectively.
The Korean version of the Beck Depression Inventory-II (K-BDI-II)
The BDI-II [40] is a 21-item questionnaire that measures the severity of self-reported depressive symptoms within the past two weeks. It reflects the major depressive disorder criteria in the Diagnostic and Statistical Manual of Mental DisordersFourth Edition (DSM-IV). Respondents are asked to rate each item on a 4-point Likert type scale ranging from 0 to 3, with higher scores indicating more severe depressive symptoms. The Korean version of the BDI-II was validated by Lim et al. [41], showing good internal reliability (Cronbach α=0.89) and test-retest reliability of 0.90.
The Korean version of the Anxiety Sensitivity Index-3 (K-ASI-3)
The Anxiety Sensitivity Index-3 [42] is an 18-item self-report instrument assessing three dimensions of anxiety sensitivity; physical, cognitive, and social concerns. Each item is rated on a 5-point Likert type scale ranging from 0 to 4. The internal consistency (α) of the Korean version of the ASI-3 was 0.87 [43].
The Korean version of the Posttraumatic Stress Disorder Checklist for DSM-5 (PCL-5-K)
The PCL-5 [44] assess the posttraumatic stress disorder (PTSD)-related symptoms, and the Korean version of PCL-5 was validated by Kim et al. [45]. It consists of 20 items that correspond to the criteria of PTSD in the Diagnostic and Statistical Manual of Mental Disorders-Fifth Edition (DSM-5), and the respondent rate each item using a 5-point Likert type scale. The Cronbach’s alpha of the Korean version of the PCL-5 for reexperiencing, avoidance, negative alterations in cognitions and mood, and hyperarousal subscale were 0.92, 0.91, 0.93, and 0.93, respectively [45].
The Korean version of the Alcohol Use Disorders Identification Test (AUDIT-K)
AUDIT is a brief 10-item measure with a 5-point Likert-type scale that aims to identify individuals with alcohol problems. The Korean version of the AUDIT was validated by Joe et al. [46] The psychometric properties of the Korean version showed excellent test-retest reliability of 0.93.
Data analysis plan
Prior to statistical analyses, the normality of the data was examined by calculating the skewness and kurtosis of the K-ERQ. The skewness values of the K-ERQ items ranged from -0.38 to 0.35, and the kurtosis values ranged from -1.16 to -0.90, suggesting a normal distribution according to the limits recommended by Kline [47].
Confirmatory factor analysis (CFA) using maximum likelihood estimation was conducted using Mplus version 7.0 (Muthén & Muthén, Los Angeles, CA, USA) [48]. First, we tested the two-factor structure by Gross and John [4], where cognitive reappraisal factor consisted of six items (item 1, 3, 5, 7, 8, 10) and expressive suppression factor consisted of four items (item 2, 4, 6, 9). The two latent factors were allowed to correlate freely. The factor structure was evaluated using the comparative fit index (CFI), the Tucker-Lewis Index (TLI), the root-mean-square error of approximation (RMSEA), and the standardized root-mean-square residual (SRMR). The recommended cut-off values in evaluating the factor structure are as follows. CFI and TLI values over 0.90, SRMR under 0.08 [49], and RMSEA value under 0.08 represent an acceptable fit [50]. In case of a poor model fit, modification indices (MI) and factor loadings were used as a reference to propose alternative models, and these models were compared to the original model to determine the one with the best fit to the data. Then, the model with the best fit was used for subsequent analyses.
The descriptive statistics of the sample, internal consistency, t-test analysis, one-way ANOVA, and correlation analysis were carried out using SPSS version 22.0 (IBM Corp., Armonk, NY, USA) [51]. The internal consistency of the K-ERQ was assessed using Cronbach’s alpha estimates. Independent sample t-test analysis was performed to explore the gender differences between the two subscales of the K-ERQ. One-way ANOVA was used to examine the differences in the two subscales of the K-ERQ between the diagnostic groups. These diagnostic groups consisted of five groups, which were made based on the most prevalent mental health disorders found in our patient sample (MDD, PDD, bipolar disorders [BDs], attentiondeficit/hyperactivity disorder [ADHD], and PTSD). Pearson’s correlations were calculated between K-ERQ, K-BDI-II, K-ASI-3, PCL-5-K, and AUDIT-K to examine the relationship between K-ERQ and psychiatric symptomatology that have been confirmed in previous studies, and a p-value of less than 0.05 was considered statistically significant.
RESULTS
Confirmatory factor analysis
The CFA results suggest that the two-factor structure by Gross and John [4] is not supported within the Korean clinical sample. The fit indices did not meet the recommended cutoff values, indicating a poor model fit (CFI=0.89, TLI=0.85, RMSEA=0.10, SRMR=0.06). We examined the modification indices (MI), and high covariance was found between item 1 and item 7 (MI=27.65, parameter change=0.81). Both items belong in the cognitive reappraisal subscale, and the two items are very similar (Item 1=When I want to feel more positive emotion [such as joy or amusement], I change the way I’m thinking about. / Item 7=When I want to feel more positive emotion, I change the way I’m thinking about the situation.). Subsequently, alternative models were proposed for comparison to determine the model with the best fit to the data. Three models were compared: 1) model 1: original two-factor model by Gross and John [4], 2) model 2: modified two-factor model excluding item 1, 3) model 3: modified two-factor model allowing covariance between item 1 and 7. For model 2, item 1 was removed as item 7 loaded more strongly to the cognitive reappraisal subscale (item 1=0.59, item 7=0.63), and the two items showed a high degree of redundancy in terms of their content.
The fit indices for all three models were computed (Table 2). The model 2, the modified two-factor model excluding item 1, provided the best fit to the data: χ2=48.23, p<0.01, CFI=0.96, TLI=0.94, RMSEA=0.07, SRMR=0.05. The nine items showed significant and salient factor loadings ranging from 0.43 to 0.83 (Figure 1). Therefore, model 2 was selected for further analysis.
Reliability
The Cronbach’s alpha was calculated to evaluate the internal consistency of the K-ERQ. In our sample, Cronbach’s alpha was 0.82 for cognitive reappraisal and 0.75 for expressive suppression.
Differences between gender and diagnostic groups
To examine the gender difference in the emotion regulation strategies measured by the K-ERQ, independent sample t-test analysis was conducted. The results indicated no significant gender difference in the K-ERQ total score (t=0.09, p= 0.93). Similarly, there was also no gender differences in each of the subscales (cognitive reappraisal, t=-0.12, p=0.91; expressive suppression, t=0.33, p=0.75).
One-way ANOVA and Scheffe’s post-hoc analysis were used to compare the use of specific emotion regulation strategies between diagnostic groups (Table 3). Excluding the groups categorized as other diagnoses, five diagnostic groups (MDD, PDD, BD, ADHD, PTSD) were examined for any differences in the K-ERQ factors. The results indicated significant differences in the cognitive reappraisal subscale among the five diagnostic groups. Scheffe’s post hoc analysis showed that patients with PTSD reported significantly higher scores than patients with PDD or ADHD. No significant group differences were found for the expressive suppression scale.
Concurrent validity
To ensure that the K-ERQ has concurrent validity, correlations with other psychiatric symptoms were examined as well. As a result, both scales showed significant correlations with the relevant psychiatric symptoms (Table 4). Cognitive reappraisal was negatively correlated with depression (r=-0.26, p<0.001) and alcohol use disorder-related symptoms (r=-0.14, p<0.05). Whereas expressive suppression was positively correlated with depression (r=0.25, p<0.001), anxiety sensitivity (r=0.20, p<0.01), PTSD-related symptoms (r=0.21, p<0.01), and alcohol use disorder-related symptoms (r=0.15, p<0.05).
DISCUSSION
Our study aimed to examine the psychometric properties of the K-ERQ using a clinical sample. Our results indicated that a two-factor structure of the K-ERQ, comprised of nine items, provided the best fit to our data. This differs from the original two-factor structure, which has ten items suggested by Gross and John [4]. However, this slight discrepancy in factor structure has also been reported in other validation studies, and minor modifications were made to the original scale to better suit the intended population.
Specifically, the studies that have suggested an adjustment to the original scale include samples outside the student population. Such as, in a German validation with a large community sample [15], the original factor structure was not confirmed, with item 8 loading onto both factors. In two separate studies involving community samples [13,14], a high covariance was found between items 1 and 3, leading researchers to omit either one or two items from the original scale. Our findings also demonstrated an inadequate fit for the original structure, with two of the items showing high covariance (items 1 and 7). Therefore, in the present study, two alternative models were compared with the original two-factor model by Gross and John [4]. The two alternative models include a model that excludes one item (item 1) from the two items that showed high covariance (items 1 and 7) and another model that allows a covariance between the two items (items 1 and 7). The results showed that the model that discarded one item demonstrated the best model fit. Consequently, following the previous validation studies that have removed items from the original scale, we also altered the scale by eliminating one item from the pair of items (item 1 and 7) that showed high covariance for the following reasons.
First, one possible explanation for the high covariance is that the two items arguably ask the same aspect of emotion regulation (Item 1: When I want to feel more positive emotion [such as joy or amusement], I change what I’m thinking about. Item 7: When I want to feel more positive emotion, I change the way I’m thinking about the situation.). Items 1 and 7 in the cognitive reappraisal subscale share similar content and wording, with both items measuring the extent to which individuals change their thoughts to feel more positive. To some extent, including redundant items in a scale is favorable as they measure a construct in somewhat different ways, but nearly identical items hold a slight advantage [52]. Problems relating to the redundancy of the ERQ items are also evident in a study using an Australian and UK sample [13] as well as in another study involving an Italian sample [14], and these researchers have decided to discard one or two items from the scale. Likewise, we have also removed one item from the original scale to suggest a psychometrically improved version of the questionnaire specifically for the Korean clinical population. Future research might benefit from adopting item response theory, which can improve the accuracy of scores and allow researchers to increase the instrument’s efficiency by including only the discriminative items [53]. Perhaps, as some items on the ERQ are almost identical in their meaning, a further psychometric analysis might help determine and eliminate redundant items. After careful consideration from a statistical and theoretical perspective, item 1 was deleted from the K-ERQ in our study. The K-ERQ with nine items presented an excellent fit with salient loadings on each item.
The 9-itemed K-ERQ also presented good internal consistency reliability, and its concurrent validity was also confirmed through correlations with other measures as expected in the clinical population. Expressive suppression showed significant positive correlations with depression and anxiety sensitivity, and cognitive reappraisal showed significant negative correlations with depression. This is consistent with prior studies that have explored the psychometric properties of the ERQ in non-clinical populations [13,15,54]. These similar trends in correlation analyses among validation studies provide evidence that cognitive reappraisal is related to positive functioning and expressive suppression is related to negative functioning regardless of culture and diagnostic status. Moreover, previous studies have shown that cognitive reappraisal is a more effective strategy in regulating negative emotions [55]. While, in an experimental study, expressive suppression was associated with increased blood pressure during an emotionally taxing conversation [56]. Thus, our findings are generally in line with previous work involving non-clinical populations.
In addition to these measures of affective symptoms, we have also examined the associations between emotion regulation strategies and PTSD-related and alcohol use disorder-related symptoms. Alcohol use disorder-related symptoms had significant correlations with both subscales, while PTSD-related symptoms were associated with expressive suppression only. Further, the present study provides additional evidence on the two-factor model posited by Gross and John [4]. In this model, cognitive reappraisal and expressive suppression are independent constructs associated with different outcomes. Our results have also indicated that the subscales of the ERQ are two independent factors, and the two subscales have shown contrasting patterns when exploring their associations with various psychopathological symptoms. Future studies should also consider the contextual factors as they play a crucial role in an individual’s preference for emotion regulation. Indeed, unlike the first generation of studies that have broadly explored emotion regulation strategies in terms of their adaptiveness, the second generation of studies on emotion regulation has highlighted that the outcome of a specific strategy can vary depending on the context [57]. Thus, it would be worthwhile to explore the relationship between the two emotion regulation strategies and the relevant psychiatric symptoms in the context of everyday life.
We also examined the differences between men and women in their use of emotion regulation strategies. Previous studies involving non-clinical samples have reported that men tend to suppress their emotions more frequently than women [4,58,59]. However, the results of our study indicate that no notable difference exists between men and women in regard to their habitual use of expressive suppression. This is in line with the prior findings of Campbell-Sills et al. [60] that, contrary to the control group, no gender difference was found in the suppression of negative emotion in a clinical group involving patients with anxiety and mood disorders. Also, empirical evidence on patients with MDD suggests that depressed patients suppress their emotions more frequently than the healthy control group [61,62]. As a large percentage of our participants were diagnosed with depressive disorders, excessive use of suppression may be an essential characteristic of our sample that goes beyond the inherent gender differences in terms of emotion regulation.
There were also significant differences between the diagnostic groups regarding the cognitive reappraisal subscale. Patients with PTSD were more likely to employ cognitive reappraisal than patients with PDD or ADHD. Regarding this group difference, one previous study [63] showed that cognitive reappraisal was not associated with PTSD-related symptoms in a group of trauma-exposed individuals. In contrast, our study showed somewhat different results as this group exhibited the highest level in their use of cognitive reappraisal. In addition, previous research has shown that patients with MDD exhibit increased suppression of emotions [61] and decreased use of cognitive reappraisal [62]. Concerning bipolar disorders, patients with bipolar I disorder endorsed significantly lower levels of cognitive reappraisal than healthy controls [32], and euthymic bipolar patients reported decreased use of cognitive reappraisal and increased use of expressive suppression compared to the control group [64]. Impairments in emotion regulation are also found in ADHD, with patients engaging in expressive suppression more frequently and cognitive reappraisal less frequently than the healthy controls [65,66]. Therefore, one possible explanation on why the PTSD group scored significantly higher on the cognitive reappraisal scale than few of the other diagnostic groups could be that patients with PTSD may not differ in the self-reported frequency of cognitive reappraisal when compared to the non-clinical group. However, they may exhibit less efficient usage, or their level of cognitive reappraisal combined with other types of emotion regulation could be associated with a negative outcome. Previous studies have noted that cognitive reappraisal may not be an adaptive regulation strategy per se. Factors such as contextual influences and other emotion regulation strategies should be considered when evaluating its adaptiveness. For instance, interaction between low emotional clarity and high cognitive reappraisal was associated with problematic cannabis use [67], and cognitive reappraisal was negatively associated with psychological functioning in the context of oppression for individuals of ethnic minority groups [68]. Moreover, considering that PTSD-related symptoms were significantly correlated with expressive suppression, but not with cognitive reappraisal, the key maladaptive emotion regulation strategy in the maintenance of PTSD could be avoidance or emotional suppression, as suggested by studies that revealed a significant relationship between PTSD symptoms and avoidance or suppression [69-71]. However, as no comparison between healthy controls was administered in the current study, future studies could benefit from recruiting clinical and non-clinical samples to examine the group differences and the combined effect of cognitive reappraisal with other emotion regulation strategies that contribute to their clinical symptoms.
Lastly, differences in the emotion regulation strategies depending on the mood states among patients with mood disorder could not be carried out due to the small size of our sample. Regarding this, research on the emotion regulation of participants currently or previously diagnosed with MDD and healthy controls have shown that patients currently diagnosed with depression exhibited less use of cognitive reappraisal than remitted patients with depression [72]. Therefore, additional research investigating the associations with mood states may help yield more information on the emotion regulation of the clinical population.
Limitations and conclusion
The present study has several limitations that should be taken into consideration. First, our sample is relatively small, and most of the participants were diagnosed with depressive disorders. This may limit the generalizability of our findings to other mental illnesses. Future studies should replicate the results of our research using a more diverse group. Second, the lack of a healthy control group makes it difficult to conclude whether our findings are limited to the clinical population. It would be interesting to include healthy subjects to examine whether the factor structure of nine items is stable across the Korean population. Third, test-retest reliability was not obtained in our study. Therefore, the temporal stability of the KERQ should be examined in future studies.
Nonetheless, the present study is the first, to our knowledge, to examine the psychometric properties of the Korean translation of the ERQ using a clinical sample. The 9-itemed K-ERQ is a reliable and valid tool for assessing the emotion regulation strategies of adults with mental disorders. As the psychometric properties of the ERQ in clinical samples have not been sufficiently investigated yet, our study adds to the growing literature by providing preliminary evidence on its usefulness in clinical settings.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation
Print
