The Development of the Brooding Scale

Ji-Hyun Kim; Yanhong Piao; Woo-Sung Kim; Jeong-Jae Park; Nam-In Kang; Keon-Hak Lee; Young-Chul Chung

doi:10.30773/pi.2019.04.16

Psychiatry Investig > Volume 16(6); 2019 > Article

Kim, Piao, Kim, Park, Kang, Lee, and Chung: The Development of the Brooding Scale

Original Article

Psychiatry Investigation 2019;16(6):443-449.

Published online: June 25, 2019

DOI: https://doi.org/10.30773/pi.2019.04.16

The Development of the Brooding Scale

Ji-Hyun Kim¹

, Yanhong Piao^2,³, Woo-Sung Kim^3,⁴, Jeong-Jae Park³, Nam-In Kang⁵, Keon-Hak Lee⁵, Young-Chul Chung^1,³

¹Department of Psychiatry, Chonbuk National University Hospital, Jeonju, Republic of Korea

²Department of Psychiatry, Chonbuk National University Medical School, Jeonju, Republic of Korea

³Research Institute of Clinical Medicine of Chonbuk National University-Biomedical Research Institute of Chonbuk National University Hospital, Jeonju, Republic of Korea

⁴Department of Medical Science, Chonbuk National University, Jeonju, Republic of Korea

⁵Department of Psychiatry, Maeumsarang Hospital, Wanju, Republic of Korea

Correspondence: Young-Chul Chung, MD, PhD Department of Psychiatry, Chonbuk National University Hospital, 20 Geonji-ro, Deokjin-gu, Jeonju 54907, Republic of Korea Tel: +82-63-250-2185, Fax: +82-63-275-3157, E-mail: chungyc@jbnu.ac.kr

Received October 5, 2018 Revised February 25, 2019 Accepted April 16, 2019

This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0/) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Objective

The purpose of this study was to develop a Brooding Scale (BS) and to confirm its psychometric properties.

Methods

A preliminary questionnaire was developed based on a literature review and face-to-face interviews with healthy subjects. To evaluate reliability and construct validity, a 15-item BS was administered to 124 healthy subjects. Convergent validity was tested by assessing the relationship between the BS and the Ruminative Response Scale (RRS). Discriminant validity was confirmed in 58 patients with schizophrenia.

Results

The internal consistency for the BS was excellent. An exploratory factor analysis yielded two factors: the emotional (six items) and cognitive (five items) domains, which explained 33.83% and 23.69% of the variance, respectively. The BS total score and scores for factors 1 and 2 showed significant positive correlations with the RRS. The total score and sub-factor scores of the BS were significantly higher in patients with schizophrenia than in healthy subjects.

Conclusion

The BS can be used as a reliable and valid tool to assess brooding in healthy adults. In addition, it had good discriminant validity for patients with schizophrenia.

Keywords: Brooding Scale, Reliability, Ruminative Response Scale, Schizophrenia, Validity

INTRODUCTION

Rumination is a thinking style that is perseverative, self-focused, and negatively valenced [1-3]. Many definitions and very different measures of rumination have been proposed (for reviews see Segerstrom et al. [4] and Nolen-Hoeksema et al. [5]), including assessments focusing on the frequency of thinking about depressive symptoms [6], the intrusiveness of thoughts about a distressing event [7], and the degree to which individuals search for the meaning of negative experiences [8]. In addition, rumination has been characterized as a stable, negative, broadly construed way of responding to discrepancies between one’s current status and one’s target status [9]. In the past 20 years, rumination has emerged as a central concept in the understanding of psychopathology. Rumination is a well-established risk factor for the onset of major depression and anxiety symptomatology. Numerous studies suggest that the tendency to ruminate is associated with elevated and prolonged sad mood [6], vulnerability to and maintenance of depression [10], and metacognitive aspects of depression [11]. Longitudinal prospective studies have found that people who are prone to rumination also have higher levels of general anxiety and posttraumatic stress symptoms [8,12-14]. In the field of psychosis or schizophrenia, several studies were undertaken to explore the relationship of rumination with depression [15], suicidality [16], negative symptoms [17], and positive symptoms [18]. In the cognitive model for persecutory delusion [19], “a search for meaning” for anomalous experiences contribute to the formation of delusion. We hypothesized that “a search for meaning” is a similar process to rumination.

Currently, various types of tools are available to measure rumination such as the Emotion Control Questionnaire [20], Response Style Questionnaire [13], Response to Intrusions Questionnaire [21], Response to Stress Questionnaire [22], Retrospective Ruminations Questionnaire [23], Rumination On Sadness Scale [24], Ruminative Response Scale (RRS) [25], and the Stress Reactive Rumination Scale [26]. Researchers have identified two components of rumination [27]: brooding and reflection. Whereas brooding refers to a passive and contemplative attitude that compares the current situation with a standard, unachieved situation from a perspective of lamentation and complaint, the reflection component refers to introspection that aims to find a cognitive solution to the problem [27]. Previous studies have shown that the brooding component has a powerful predictive association with the development of depression [28-31] and of general anxiety in adolescents [32] and adults [33,34].

Despite the important role of rumination in the development of various symptoms, no questionnaire for rumination exists in Korea except the Korean version of the RRS [35]. We hypothesized that development of a scale targeting brooding would be more clinically relevant given its close associations with depression and anxiety. The objectives of the present study were to develop a new Brooding scale (BS) and to confirm its reliability and validity.

METHODS

Participants

This study enrolled two groups of participants: healthy volunteers and patients with schizophrenia. Healthy volunteers were recruited from subjects attending mental health workshops and via advertisements in local newspapers. They underwent a psychiatric interview using the screening module of the Structured Clinical Interview for DSM-IV (SCID)-NP [36,37]. The inclusion criteria were age between 18 and 60 years, the ability to give written informed consent, and the ability to read and write in Korean. Prospective participants were excluded if they had any of the following: current or past diagnosis of a psychiatric or neurological disorder, substance use disorder (except for nicotine), or other significant medical conditions. To determine divergent validity, patients were recruited from the outpatient Department of Psychiatry at Chonbuk National University Hospital, Jeonju, South Korea. They were considered eligible to enter the study if they had a DSM-IV diagnosis of schizophrenia, were between the ages of 18 and 60 years, and were able to read and write in Korean. We excluded patients who reported alcohol or drug dependency within the past 6 months and those with intellectual disabilities (IQ less than 80), a current or historical neurological illness, or a serious medical illness. This study was approved by the institutional review board of Chonbuk National University Hospital (IRB No. CUH 2014-11-002). All participants provided written informed consent after the procedure had been fully explained.

Measures

Preliminary questionnaire for the Brooding Scale (BS)

Based on reviews of the relevant literature by two psychiatrists and one psychiatric nurse, we defined brooding as passive, intrusive, and repetitive thoughts dwelling upon negative past events or memories, and we developed a questionnaire comprising a preliminary set of 20 items measuring brooding. To refine the questionnaire further, face-to-face interviews were conducted with participants in mental health workshops and outpatients with schizophrenia using the 20 preliminary items and some additional questions such as, “What makes you ruminate or brood about past negative events?” or, “Why is it so difficult to be free from rumination or brooding?” In several discussions, we carefully evaluated whether the preliminary items fit with the definition of brooding and whether they overlapped. The final preliminary BS, consisting of 15 items using a 4-point Likert scale (0=never; 1=a little; 2=somewhat; 3=a lot), was administered to the healthy volunteers.

Ruminative response scale (RRS)

The Korean version of the RRS, which has a Cronbach’s α of 0.89, was used [35]. The RRS consists of 22 items and three sub-factors (12 depression-related items, 5 brooding items, and 5 reflection items). It assesses an individual’s tendency to ruminate when faced with depressive symptoms. Participants are asked to indicate what they “generally do when feeling down, sad, or depressed” using a 4-point Likert-type scale representing frequency. The RRS consists of items measuring how often people engage in responses that are self-focused, those that are symptom focused, and responses focused on the causes and consequences of having a depressed mood. A higher score on the RRS indicates a predominantly ruminative response style.

Beck depression inventory (BDI)

The BDI, a 21-item self-administered questionnaire, was developed to assess the severity of subjective depressive symptoms [38]. Each response is scored from 0 to 3, with 3 representing the greatest severity. The Korean version of the BDI was shown to have good psychometric properties (Cronbach’s α 0.93, test-retest reliability coefficient r=0.91, consistency coefficient=0.85) [39].

Statistical analysis

Chi-square and independent t-tests were used to compare demographic data from healthy volunteers with those from patients. To determine the reliability of the BS, Cronbach’s alpha coefficients and item-total correlations were calculated. We also conducted an exploratory principal component analysis with varimax rotation. The number of factors was determined by the eigenvalues and the amount of variance explained by each factor. In addition, we followed the rule for factor loading cutoff that satisfactory variables 1) load onto their primary factor above 0.60, 2) load onto alternative factors below 0.40, and 3) demonstrate a difference of 0.20 between their primary and alternative factor loadings [40,41]. Convergent validity between the BS and other measures (RRS and BDI) was evaluated using Pearson’s correlation coefficients. To examine discriminant validity, the scores on t he BS were compared between the two samples. All data were analyzed using SPSS 18.0 for Windows (SPSS Inc., Chicago, IL, USA) The level of significance was set at p<0.05.

RESULTS

Demographic characteristics

In total, 124 healthy volunteers (approximately half of whom were university students) and 58 patients were initially recruited for this study. We excluded the subjects who had missing data for at least one item on the BS, RRS, or BDI, yielding final samples of 119 healthy volunteers and 52 patients. Demographic characteristics of the two samples are shown in Table 1. Among these, education differed significantly between the two samples (p<0.001).

Reliability and factor analysis (Table 2)

In healthy volunteers, the Cronbach’s α values for the BS total and factors 1, 2, and 3 were 0.89, 0.89, 0.76, and 0.50, respectively, and the item-total correlation ranged from 0.34 to 0.74. Before factor analysis, the Kaiser-Meyer-Olkin (KMO) value, calculated as an indicator of data coherence, was 0.86, and the Bartlett test was found to be significant (χ²=751.43, df=105, p<0.001). The minimum KMO value recommended to perform factor analysis is 0.60 [42]. The principal component analysis identified three factors that explained 58.45% of the total variance. All of these factors had eigenvalues greater than 1.0. Factor 1 consisted of eight items and explained 28.71% (eigenvalue, 4.31) of the variation. Factors 2 (five items) and 3 (two items) accounted for 19.26% (eigenvalue, 2.89) and 10.48% (eigenvalue, 1.57) of the variance, respectively. The factor loads of items varied between 0.53 and 0.82 (Table 2A). The factor 3 had only two items and low reliability. The item 8 had a cross loading on factor 2 above 0.4 and difference of factor loading for item 10 was less than 0.2. Therefore, we repeated the factor analysis after removing related items. This analysis produced a two-factor structure which was chosen as a final scale: factors 1 and 2 explained 33.83% (eigenvalue 3.72) and 23.69% (eigenvalue 2.61) of the variation (Table 2B).

Convergent and discriminant validity

The total score on the BS and scores on factors 1 and 2 of the BS were all significantly correlated with the total and subfactor scores for the RRS and with the total BDI score (Table 3). The mean score of total sum score on the BS and mean scores on factors 1 and 2 of the BS differed significantly between healthy volunteers and patients with schizophrenia (Table 4). After controlling for education, only factor 1 remained significant.

DISCUSSION

In the present study, a new BS was developed using data gathered from healthy adults, and psychometric characteristics of the scale were examined. We found that the BS exhibited good reliability and validity.

As for the reliability, the Cronbach’s α for the 11-item BS was excellent (0.87). The correlations between individual items and the total score for the BS were moderate to large, demonstrating high inter-item consistency. In addition, the internal consistency for the sub-factors of the BS was considered adequate, which also means that the overall reliability was good. Therefore, it is expected that the sub-factors of the BS would be useful as indicators of sub-components of brooding in future studies. Exploratory factor analysis by principal component analysis revealed that the BS is composed of two factors. The first factor, which accounted for 33.83% of the variation, was represented by six items on the questionnaire, mostly describing emotional neglect, anger, inferiority, betrayal, alienation, lowered self-esteem, and so on. This suggests a link to response styles theory, which suggests that rumination is negative repetitive thinking about the causes, consequences, and symptoms of one’s negative affect, especially depression. Therefore, it was deemed part of the emotional domain of brooding. The second factor consisted of five items: “My thinking is very rigid and inflexible, which keeps me dwelling on negative things”; “Not being broad minded, I tend to think deep and long”; “It is hard to let go of negative things that happened to me”; “I often reproach myself too long for a mistake”; and “Oftentimes, I feel mad at myself for not coping well with regard to negative events.” These items highlight a traditional malignant characteristic of patients with schizophrenia, i.e., concrete and inflexible thinking. Relatedly, trait rumination was associated with poor performance on tasks thought to index cognitive control [43]. We identified this factor as the cognitive domain of brooding. In relation to first analysis, the third factor consisted of two items, “When I am in trouble, I tend to dig into things endlessly” and, “Faced with conflict, I must solve it at all costs.” It has been suggested that a factor containing fewer than three items, called a trivial factor, might decrease generalizability [44] and that at least three items are required for factor identification [45,46]. Based on this recommendation, we removed factor 3. Furthermore, items 8 and 10 violated the factor loading cutoff which was subsequently deleted in the second analysis. Therefore, we recommend using 11-item BS as a final scale: factor 1 (emotional domain) consisting of items 2, 3, 4, 11, 12, and 15 and factor 2 (cognitive domain), items 6, 7, 9, 13, and 14.

The correlations observed between the BS scores and the RRS scores demonstrate the convergent validity of the BS. It is interesting to note that the strength of association between the BS (total and factors 1 and 2) and RRS was greater for the brooding component of the RRS than for the reflection component. These results suggest that factors 1 and 2 of the BS are more likely to capture the brooding component of the RRS and have good convergent validity with the RRS. Comparison of the BS scores between healthy subjects and patients with schizophrenia showed that patients with schizophrenia were more likely to brood about the negative things that happened to them. However, when education was controlled, differences in the BS total score and factors 2 were no longer significant. These findings imply that the emotional domain of the BS, factor 1, may be tapping a unique and strong characteristic of patients with schizophrenia, and may have better ability to differentiate patients with schizophrenia from healthy subjects.

Taken together, the results for internal consistency and those for the construct, convergent, and discriminant validities indicate that the BS is a reliable and valid tool. Psychometric properties of the Korean version of the RRS or RRS-revised have been explored in university students [35], adolescents [47] and patients with major depressive disorder [48]. However, they did not address the issue of differentiating brooding from other components or the question of divergent validity in patients with schizophrenia. The strengths of the present study are that we developed an original scale targeting brooding and tested its discriminant validity in patients with schizophrenia. Brooding is conceptualized as a negative and evaluative focus on the self, and it is proposed to be maladaptive [27]. Accurate evaluation of brooding and the development of interventions designed to decrease brooding would have an important impact with regard to alleviating diverse psychopathologies and leading to good outcomes in patients with mental illnesses. In this regard, development of the BS offers an important contribution. In future studies, it is hoped that the BS will be used to explore the role of brooding in diverse symptoms among patients with psychosis.

The present findings must be cautiously interpreted considering the following limitations. First, even though we tried to develop a scale measuring brooding specifically, it was found to have significant correlations with depression and with the reflection component of the RRS. Further refinement of the scale is needed in the future. Second, as we have deleted original four items, validity of the BS should have tested in new samples. This should be further tested. Third, given the importance of rumination in depression, validity of the BS should be explored in the patients with depression. Fourth, discriminant validity for schizophrenia was weakened after education, a confounding factor, was controlled. This should be considered in interpreting BS results for patients with schizophrenia in future studies. In conclusion, the BS has good psychometric properties, and it can be used as a reliable and valid tool to assess brooding in healthy adults. In addition, it has good discriminant validity for patients with schizophrenia. Further investigations are needed to evaluate the BS fully, including its application to the general population, to psychiatric inpatients, and to those with depression.

ACKNOWLEDGEMENTS

This study was supported by a grant of the Korean Mental Health Technology R&D Project, Ministry of Health & Welfare, Republic of Korea (HM14C2608) and a grant of the Korea Health Technology R&D Project through the Korea Health Industry Development Institute (KHIDI), funded by the Ministry of Health & Welfare, Republic of Korea (HI18C2383).

The authors have no potential conflicts of interest to disclose.

Author Contributions

Conceptualization: Young-Chul Chung. Data curation: Jeong-Jae Park. Formal analysis: Jeong-Jae Park. Funding acquisition: Young-Chul Chung. Investigation: Ji-Hyun Kim, Yanhong Piao, Woo-Sung Kim. Methodology: Ji-Hyun Kim, Yanhong Piao, Woo-Sung Kim. Project administration: Young-Chul Chung. Resources: Young-Chul Chung. Software: Jeong-Jae Park, Woo-Sung Kim. Supervision: Young-Chul Chung. Validation: Nam-In Kang, Keon-Hak Lee. Visualization: Nam-In Kang, Keon-Hak Lee. Writing—original draft: Ji-Hyun Kim. Writing—review & editing: Young-Chul Chung.

Table 1.

Demographic characteristics of healthy volunteer and patient groups

Variable	Healthy volunteer (N=119)	Patient (N=52)	p-value
Sex (M/F)	59/60	26/26	0.960
Age (years)	32.50±11.11	32.38±10.13	0.947
Education (years)	14.76±1.84	13.06±2.14	<0.001

Values are presented as mean±standard deviation or number

Table 2.

Factor load values, and item total correlation values obtained from exploratory factor analysis of the Brooding scale in healthy subjects: A) Original scale; and B) Modified scale excluding factor 3 (item 1 and 5), item 8 and item 10

Scale item no.	F1	F2	F3	Item-total correlation
A) Original scale
2	0.821	0.131	0.046	0.735
12	0.813	0.068	0.069	0.690
15	0.755	0.223	0.048	0.694
11	0.732	0.118	0.325	0.700
4	0.643	0.309	0.165	0.641
8	0.632	0.48	0.05	0.662
3	0.576	0.337	0.237	0.601
10	0.527	0.344	0.125	0.545
7	0.098	0.81	-0.137	0.588
14	0.39	0.634	0.007	0.545
13	0.372	0.629	0.197	0.586
6	0.027	0.608	0.316	0.446
9	0.289	0.602	0.126	0.477
5	0.222	-0.06	0.808	0.335
1	0.092	0.239	0.728	0.335
Cronbach’s α (total: 0.89)	0.886	0.759	0.5
Eigenvalue	4.306	2.89	1.572
Explained proportion (%) (total: 58.451%)	28.71	19.264	10.477
B) Modified scale
2	0.837	0.127		0.749
12	0.791	0.118		0.685
11	0.774	0.162		0.693
15	0.745	0.223		0.663
4	0.673	0.316		0.620
3	0.629	0.358		0.588
7	0.054	0.809		0.588
6	0.066	0.678		0.446
13	0.39	0.644		0.586
9	0.279	0.614		0.477
14	0.391	0.605		0.545
Cronbach’s α (total: 0.872)	0.867	0.759
Eigenvalue	3.721	2.606
Explained proportion (%) (total: 57.519%)	33.827	23.692

Item-total correlation: all p<0.05

Table 3.

Correlations among total and sub-factor scores of the Brooding Scale and other scales in healthy subjects

	BS (N=119)			RRS (N=119)
	Total	Factor 1	Factor 2	Total	Depression	Brooding	Reflection
BS (N=119)
Factor 1	0.915^**
Factor 2	0.847^**	0.561^**
RRS (N=119)
Total	0.645^**	0.569^**	0.574^**
Depression	0.623^**	0.568^**	0.530^**	0.952^**
Brooding	0.595^**	0.487^**	0.580^**	0.828^**	0.742^**
Reflection	0.392^**	0.345^**	0.350^**	0.722^**	0.542^**	0.399^**
BDI_total (N=119)	0.417^**	0.299^**	0.461^**	0.487^**	0.528^**	0.421^**	0.206^*

^* p<0.05,

^** p≤0.001.

BDI: Beck Depression Inventory, BS: Brooding Scale, RRS: Ruminative Response Scale

Table 4.

Comparison of the Brooding scale scores between healthy volunteer and patient groups

Variable	Healthy volunteer (N=119)	Patient (N=52)	p-value^*	p-value^†
BS
Mean	1.24±0.58	1.72±0.78	0.001	0.06
Factor 1	1.33±0.68	1.79±0.85	<0.001	0.034
Factor 2	1.13±0.62	1.65±0.81	<0.001	0.074

Values are presented as mean±standard deviation or number.

^* T-test,

^† ANCOVA.

BS: Brooding Scale

REFERENCES

1. Nolen-Hoeksema S. Responses to depression and their effects on the duration of depressive episodes. J Abnorm Psychol 1991;100:569-582.

2. Trapnell PD, Campbell JD. Private self-consciousness and the five-factor model of personality: distinguishing rumination from reflection. J Pers Soc Psychol 1999;76:284-304.

3. Watkins ER. Constructive and unconstructive repetitive thought. Psychol Bull 2008;134:163-206.

4. Segerstrom SC, Stanton AL, Alden LE, Shortridge BE. A multidimensional structure for repetitive thought: what’s on your mind, and how, and how much? J Pers Soc Psychol 2003;85:909-921.

5. Nolen-Hoeksema S, Wisco BE, Lyubomirsky S. Rethinking rumination. Perspect Psychol Sci 2008;3:400-424.

6. Nolen-Hoeksema S, Morrow J, Fredrickson BL. Response styles and the duration of episodes of depressed mood. J Abnorm Psychol 1993;102:20-28.

7. Horowitz M, Wilner N, Alvarez W. Impact of event scale: a measure of subjective stress. Psychosom Med 1979;41:209-218.

8. Fritz HL. The role of rumination in adjustment to a first coronary event. Diss Abstr Int B Sci Eng 1999;60(1-B):0410

9. Smith JM, Alloy LB. A roadmap to rumination: a review of the definition, assessment, and conceptualization of this multifaceted construct. Clin Psychol Rev 2009;29:116-128.

10. Just N, Alloy LB. The response styles theory of depression: tests and an extension of the theory. J Abnorm Psychol 1997;106:221-229.

11. Papageorgiou C, Wells A. Depressive Rumination: Nature, Theory, and Treatment. Chichester: Wiley; 2004.

12. Nolen-Hoeksema S. The role of rumination in depressive disorders and mixed anxiety/depressive symptoms. J Abnorm Psychol 2000;109:504-511.

13. Nolen-Hoeksema S, Morrow J. A prospective study of depression and posttraumatic stress symptoms after a natural disaster: the 1989 Loma Prieta Earthquake. J Pers Soc Psychol 1991;61:115-121.

14. Segerstrom SC, Tsao JCI, Alden LE, Craske MG. Worry and rumination: repetitive thought as a concomitant and predictor of negative mood. Cognit Ther Res 2000;24:671-688.

15. Thomas N, Ribaux D, Phillips LJ. Rumination, depressive symptoms and awareness of illness in schizophrenia. Behav Cogn Psychother 2014;42:143-155.

16. Ahrens B, Linden M. Is there a suicidality syndrome independent of specific major psychiatric disorder? Results of a split half multiple regression analysis. Acta Psychiatr Scand 1996;94:79-86.

17. Halari R, Premkumar P, Farquharson L, Fannon D, Kuipers E, Kumari V. Rumination and negative symptoms in schizophrenia. J Nerv Ment Dis 2009;197:703-706.

18. Badcock JC, Paulik G, Maybery MT. The role of emotion regulation in auditory hallucinations. Psychiatry Res 2011;185:303-308.

19. Freeman D, Garety PA, Kuipers E, Fowler D, Bebbington PE. A cognitive model of persecutory delusions. Br J Clin Psychol 2002;41:331-347.

20. Roger D, Najarian B. The relationship between emotional rumination and cortisol secretion under stress. Pers Individ Dif 1998;24:531-538.

21. Clohessy S, Ehlers A. PTSD symptoms, response to intrusive memories and coping in ambulance service workers. Br J Clin Psychol 1999;38(Pt 3):251-265.

22. Connor-Smith JK, Compas BE, Wadsworth ME, Thomsen AH, Saltzman H. Responses to stress in adolescence: measurement of coping and involuntary stress responses. J Consult Clin Psychol 2000;68:976-992.

23. Luminet O. 10 Measurement of Depressive Rumination and Associated Constructs. Chichester: Wiley; 2004.

24. Conway M, Csank PA, Holm SL, Blake CK. On assessing individual differences in rumination on sadness. J Pers Assess 2000;75:404-425.

25. Nolen-Hoeksema S, Larson J, Grayson C. Explaining the gender difference in depressive symptoms. J Pers Soc Psychol 1999;77:1061-1072.

26. Alloy LB, Abramson LY, Hogan ME, Whitehouse WG, Rose DT, Robinson MS, et al. The Temple-Wisconsin Cognitive Vulnerability to Depression Project: lifetime history of axis I psychopathology in individuals at high and low cognitive risk for depression. J Abnorm Psychol 2000;109:403-418.

27. Treynor W, Gonzalez R, Nolen-Hoeksema S. Rumination reconsidered: a psychometric analysis. Cognit Ther Res 2003;27:247-259.

28. Burwell RA, Shirk SR. Subtypes of rumination in adolescence: associations between brooding, reflection, depressive symptoms, and coping. J Clin Child Adolesc Psychol 2007;36:56-65.

29. Cova F, Rincón P, Melipillán R. Reflexión, rumiación negativa y desarrollo de sintomatología depresiva en adolescentes de sexo femenino. Terapia Psicológica 2009;27:155-160.

30. Mezulis A, Simonson J, McCauley E, Vander Stoep A. The association between temperament and depressive symptoms in adolescence: brooding and reflection as potential mediators. Cogn Emot 2011;25:1460-1470.

31. Yang J, Zhang C, Yao SQ. The impact of rumination and stressful life events on depressive symptoms in high school students: a multi-wave longitudinal study. Acta Psychol Sinica 2010;42:939-945.

32. Verstraeten K, Bijttebier P, Vasey MW, Raes F. Specificity of worry and rumination in the development of anxiety and depressive symptoms in children. Br J Clin Psychol 2011;50:364-378.

33. McEvoy PM, Brans S. Common versus unique variance across measures of worry and rumination: predictive utility and mediational models for anxiety and depression. Cognit Ther Res 2013;37:183-196.

34. Roelofs J, Huibers M, Peeters F, Arntz A, van Os J. Rumination and worrying as possible mediators in the relation between neuroticism and symptoms of depression and anxiety in clinically depressed individuals. Behav Res Ther 2008;46:1283-1289.

35. Kim S, Kim J, Youn S. Validation of the Korean-Ruminative Response Scale (K-RRS). Korean J Clin Psychol 2010;29:1-19.

36. First MB, Spitzer RL, Gibbon M. New York: Biometrics Research Department, New York State Psychiatric Institute; 1997.

37. Han O, Hong J. Structured Clinical Interview for DSM-IV Axis I Disorders, Research Version. Seoul: Korea Hana Medical Publishing; 2000.

38. Beck AT, Steer RA. Internal consistencies of the original and revised Beck Depression Inventory. J Clin Psychol 1984;40:1365-1367.

39. Rhee M. A Standardization study of Beck Depression Inventory I-Korean version (K-BDI) : Reliability and factor analysis. Korean J Psychopathol 1995;4:77-95.

40. Hooper D. Exploratory Factor Analysis. In: Chen H, editor. Approaches to Quantitative Research-Theory and its Practical Application: A Guide to Dissertation Students Cork. Ireland: Oak Tree Press, 2012.

41. Howard MC. A review of exploratory factor analysis decisions and overview of current practices: what we are doing and how can we improve? Int J Hum Comput Interact 2016;32:51-62.

42. Tabachnick BG, Fidell LS. Using Multivariate Statistics. Boston: Pearson Education; 2013.

43. Joormann J, Yoon KL, Zetsche U. Cognitive inhibition in depression. Appl Prevent Psychol 2007;12:128-139.

44. Gorsuch RL. Factor Analysis. Hillsdale, NJ: Lawrence Erlbaum Associates; 1983.

45. MacCallum RC, Widaman KF, Zhang S, Hong S. Sample size in factor analysis. Psychol Methods 1999;4:84-99.

46. Zwick WR, Velicer WF. Comparison of five rules for determining the number of components to retain. Psychol Bull 1986;99:432-442.

47. Shin KM, Cho SM, Kim KH. A validation study of the Korean-ruminative response scale in Korean adolescents. Psychiatry Investig 2015;12:508-515.

48. Lee S, Kim W. Cross-cultural adaptation, reliability, and validity of the revised Korean version of ruminative response scale. Psychiatry Investig 2014;11:59-64.