Park, Rogers, Bloch-Elkouby, Richards, Lee, Galynker, and You: Factor Structure and Validation of the Revised Suicide Crisis Inventory in a Korean Population

Abstract

Objective

Because of the exceptionally high suicide rates in South Korea, new assessment methods are needed to improve suicide prevention. The current study aims to validate the revised Suicide Crisis Inventory-2 (SCI-2), a self-report measure that assesses a cognitiveaffective pre-suicidal state in a Korean sample.

Methods

With data from 1,061 community adults in South Korea, confirmatory factor analyses were first conducted to test the proposed one-factor and five-factor structures of the SCI-2. Also, an exploratory factor analysis (EFA) was performed to examine possible alternative factor structure of the inventory.

Results

The one-factor model of the SCI-2 resulted in good model fit and similarly, the five-factor model also exhibited strong fit. Comparing the two models, the five-factor was evaluated as the superior model fit. An alternative 4-factor model derived from EFA exhibited a comparable model fit. The Korean version of the SCI-2 had high internal consistency and strong concurrent validity in relation to symptoms of suicidal ideation, depression, and anxiety.

Conclusion

The SCI-2 is an appropriate and a valid tool for measuring one’s proximity to imminent suicide risk. However, the exact factor structure of the SCI-2 may be culture-sensitive and warrants further study.

INTRODUCTION

Predicting who, when, and how an individual will attempt suicide is challenging even for suicide researchers and clinicians [1]. This is because suicidal ideation (SI) oscillates [2] and the desire to act on those thoughts can appear rapidly and unpredictably [3,4]. One of the most common methods that has long been used to evaluate suicide risk has been by directly assessing thoughts of death or suicide [5]. Although these assessments can provide valuable clinical information [6], previous studies have highlighted high rates of nondisclosure of SI [7,8], resulting in clinicians missing many individuals in need of care. Indeed, a significant number of suicide decedents denied SI or intent to healthcare providers prior to their deaths [9,10]. As such, reliance on SI in assessing suicide risk does not seem optimal, and instead novel tools that identify individuals at imminent risk for suicide should be implemented [1,11,12].
One such approach is to assess the Suicide Crisis Syndrome (SCS), which is thought to emerge proximate to imminent suicide risk. The syndrome was developed based on empirical predictors of imminent risk for suicidal behaviors and includes cognitive and affective characteristics that potentially precede a suicide attempt. Specifically, the SCS comprises five components and is divided into two criteria: criterion A, frantic hopelessness/entrapment; and criterion B, which includes four symptom domains, affective disturbance, loss of cognitive control, hyperarousal, and social withdrawal [13,14]. A chief difference between the SCS and contemporary suicide risk assessments is that it does not rely on self-reports of SI that are often known to be significantly biased [2,15]. Together, the SCS is posited to be a unidimensional diagnosis of one’s near-term suicidal mental state [16].
Frantic hopelessness/entrapment, identified as one of the central predictors of suicidal behavior [17,18] as well as a core feature of other models of suicide (e.g., the integrated motivational-volitional model) [19], is characterized by feelings of hopelessness, defeat, and the perception that death is the only alternative because life adversities seem inescapable [20]. Feelings of entrapment can be experienced as a result of external (e.g., relationship problems) and internal (e.g., inner thoughts of defeat) circumstances that prompt one’s desire to escape their current state [21]. Frantic hopelessness/entrapment is thought to be the core symptom of the SCS.
Criterion B includes affective disturbances, loss of cognitive control, hyperarousal, and social withdrawal. Affective disturbance incorporates four features: emotional pain [22], rapid spikes of negative emotions [23], extreme anxiety [24,25], and acute anhedonia [26]. Loss of cognitive control includes rumination [13], cognitive rigidity [27], ruminative flooding [25], and failed thoughts suppression [28,29]. Hyperarousal includes components of agitation [24,30], hypervigilance, irritability [31-34], and insomnia [35-37]. Hyperarousal, especially the association between hyperarousal symptoms and suicide attempts, has especially been noted among military service members with posttraumatic stress disorder where they are continuously wary and on edge about external events that may be of threat to them [38]. Finally, social withdrawal reflects individuals’ interactions and feelings of connectedness with the environment surrounding them [39,40]. Each of the aforementioned factors has been linked to suicidal thoughts and/or behaviors.
The SCS is assessed using the Suicide Crisis Inventory-2 (SCI-2), a 61-item self-report assessment, which is the revised version of the former Suicide Crisis Inventory [12]. The psychometric properties of the inventory have been supported across multiple studies showing strong reliability, validity and internal consistency [41,42]. One- and five-factor structures have been identified as strong model fits [41]; hence, it is necessary to examine and confirm the reliability of the identified factor structures and whether they are applicable in different cultures. Clinical utility and predictive validity among high-risk population have also been supported where both inpatient and outpatient populations who fulfilled the complete criteria of the SCS were more likely to attempt suicide in the near future compared to those who met partial criteria [16,43,44]. Validating SCI-2 in Korean would aid implementation of near-term suicide risk assessment even without clear disclosure of SI [11].
The present study sought to 1) translate the SCI-2 into Korean and validate it in a Korean sample, 2) evaluate the factor structure and the psychometric properties of the SCI-2 through confirmation of previously identified structures and exploration of alternative possibilities, 3) examine the internal consistency and reliability of the SCI-2, and 4) inspect its concurrent validity with measures of SI, depression symptoms, and anxiety symptoms. We hypothesized that the Korean version of the SCI-2 would support both the one-factor and five-factor structures as outlined by the original version of the scale. No a priori hypotheses were made regarding alternative structures, given that these analyses were intended to be exploratory. We also predicted that there will be strong correlations between factors as it is measuring cognitive and affective elements that are known to predict suicidal behaviors, and that the SCI-2 would exhibit concurrent validity with measures of depression, anxiety, and SI.

METHODS

Participants and procedures

This study was conducted as part of a larger international collaborative research project examining suicide during the coronavirus disease (COVID-19) pandemic. All study procedures were approved by the Chungbuk National University Institutional Review Board (CBNU-202007-HR-0120). Korean participants who were 19 years or older and were residing in South Korea during the outbreak of the pandemic were recruited to complete an online Qualtrics survey between August 2020 and October 2020. The survey was dispersed to various sites on social media platforms (e.g., Facebook, Instagram, University websites). Individuals who were interested in participating in the study signed an online consent form before starting the survey. To ensure participant safety, national mental health and suicide prevention resources were provided during the survey. Participants who fully completed the survey were compensated with a gift card worth 3,000 KRW (approximately 3 dollars).

Measures

SCI-2

The SCI-2 [41] is a 61-item self-report measure that assesses the severity of the SCS over the past several days. Items are rated on a 5-point scale ranging from 0 (not at all) to 4 (extremely), and evaluate feelings of entrapment, levels of affective disturbance, loss of cognitive control, hyperarousal, and social withdrawal. The SCI-2 had excellent internal consistency (α=0.97) as well as strong concurrent validity in previous research [41].

Columbia-Suicide Severity Rating Scale screener

The Columbia-Suicide Severity Rating Scale (C-SSRS) screener is a validated 6-item self-report measure that evaluates the severity of SI and suicidal acts [45]. With response options yes or no, 5 items regarding SI were used for this study, anchored to both the past month and lifetime; the highest score was used to reflect participants’ lifetime and past-month severity of SI, with higher scores reflecting more severe SI. In addition to the C-SSRS screener, items asking lifetime suicide attempt (i.e., Have you ever attempted suicide/tried to kill yourself?) and suicide attempt in the past month (i.e., Have you attempted suicide/tried to kill yourself in the past month?) were included. Reliability and validity of the scale has been demonstrated in multiple studies and diverse populations [45,46], including a Korean sample [47].

Patient Health Questionnaire-9

The Patient Health Questionnaire-9 (PHQ-9) is a 9-item self-administered assessment that evaluates the severity of depression during the past two weeks [48]. The questionnaire is evaluated on a 4-point Likert scale with total scores ranging from 0 to 27: 0 (not at all), 1 (several days), 2 (more than half the days), and 3 (nearly every day). The Korean version of the PHQ-9 had good reliability, validity and internal consistency (α=0.95) [49].

Generalized Anxiety Disorder-7

The Generalized Anxiety Disorder-7 (GAD-7) is a 7-item self-report scale that measures severity of generalized anxiety disorder [50]. Participants reported the frequency of anxiety symptoms they experienced during the past two weeks with the response options 0 (not at all), 1 (several days), 2 (more than half the days), and 3 (nearly every day). Total score ranges from 0 to 21, with higher scores indicative of more severe generalized anxiety symptoms. The GAD-7 had good psychometric properties in a Korean population with an internal consistency of α=0.93 [51].

Translation of the SCI-2

Standard procedures were followed in the process of translating the English version of the SCI-2 into Korean [52]. The inventory was first translated into Korean by a doctoral-level psychologist and a graduate student in clinical psychology. A professional translator was then hired to back translate the items into English, its original language. Dr. Igor Galynker, the author of the inventory, reviewed the back translations to see whether items relevantly reflected their original ones. After making necessary modifications, the authors of the current study revised the entirety of the inventory until the Korean version of it was deemed acceptable. Finally, graduate students in clinical psychology participated in the pilot-testing process of the translated questionnaire. Students reported phrases or words that were difficult to understand, which were changed if consensus of word choice was met.

Data analytic strategy

The Kaiser-Meyer-Olkin (KMO) measure of sampling adequacy [53] and Bartlett’s test of sphericity [54] were used to establish that these data were suitable for factor analyses. Confirmatory factor analyses (CFA) were conducted to test the proposed one- and five-factor structures of the SCI-2. Specifically, in the one-factor model, all items were set to load onto a single factor. In the five-factor model, items were set to load on their respective subscale factors of the original SCI-2: frantic hopelessness/entrapment, affective disturbance, loss of cognitive control, hyperarousal, and social withdrawal. Because items were ordinal (i.e., rated on a 5-point Likert scale), diagonally weighted least squares estimation was used.
The exploratory factor analysis (EFA) of the 61 items in the English version of the SCI-2 with direct oblimin rotation, was then conducted on a randomly selected half of the full sample (n=530). Items that significantly loaded onto factors with loadings ≥0.45 were retained. Although 0.40 is a frequently employed standard, we decided to retain items ≥0.45 to diminish cross-loadings and remove items that were weakly associated with their respective factors [55,56]. Items with significant loadings on multiple factors (with factor loadings ≥0.35) were removed from the scale to maintain a simpler factor structure with minimal cross-loadings. The number of suitable factors for the Korean version of the SCI-2 was determined through examination of model fit via the chi-square difference test, balanced with examination of significant loadings across models to establish which model resulted in interpretable factors. Next, CFAs of the remaining items were conducted in the other half of the full sample (n=531) to test the best fitting model yielded from the EFA. Items were set to load on their respective factors yielded from the EFA.
Model fit was evaluated using recommended guidelines [57], including the chi-square statistic (χ2), comparative fit index (CFI), Tucker-Lewis Index (TLI), root mean squared error of approximation (RMSEA), and standardized root mean residual (SRMR). Namely, good model fit was indicated by a non-significant χ2 statistic, CFI ≥0.95, TLI ≥0.95, RMSEA ≤0.08, and SRMR ≤0.08. Comparison of the one- and five-factor models was computed using the chi-square difference test; however, because the four-factor model yielded from EFA had a different number of items and cases, model comparison was not feasible.
Cronbach’s alpha values were evaluated to establish internal consistency of the scale, while correlation analyses were conducted to examine the scale’s concurrent validity. There were no missing data, as only participants who completed the full study were included in the dataset. Analyses were conducted using psych packages in R, specifically the lavaan [58], semTools [59] packages for CFA analyses, MPlus for EFA analyses (Muthén & Muthén, Los Angeles, CA, USA), and SPSS Version 27.0 for internal consistency/validity analyses (IBM Corp., Armonk, NY, USA).

RESULTS

Preliminary results

In total, 1,061 participants completed the online Qualtrics survey where 72.2% (n=766) were women and 27.8% (n=295) were men. Ages ranged from 19 to 68 years (30.56±8.49). A majority of the participants were single (69.1%) and had completed their bachelor’s degree (59.8%). Among participants, 29.5% were students, 36.3% had a full-time job, and 5.1% were unemployed. Demographic information of the sample is presented in Table 1. A total of 189 participants (17.8%) had endorsed thoughts of suicide sometime during their life, 405 participants in the past month (38.2%), 67 participants (6.4%) had a history of suicide attempt and among them, 8 participants (11.9%) made an attempt in the past month. Approximately 13.2% were receiving treatment for mental health (e.g., outpatient therapy, group therapy, and medication).

One-factor and five-factor models

CFA

Results of the one-factor CFA of the Korean version of the SCI-2 resulted in good model fit (χ2[1769]=16150.33, p< 0.001, CFI=0.99, TLI=0.99, RMSEA=0.08, SRMR=0.06). Similarly, the five-factor CFA demonstrated strong model fit (χ2[1759]=11603.66, p<0.001, CFI=0.99, TLI=0.99, RMSEA=0.07, SRMR=0.05). Comparison of the one-factor and fivefactor models indicated that the five-factor model demonstrated superior model fit to the one-factor model (Δχ2[10]=4546.67, p<0.001). The results of CFA are presented in Table 2.

Exploration of alternative factor structures

EFA

The KMO statistic (0.98) and Bartlett’s test of sphericity (χ2[1830]=29944.64, p<0.001) each indicated that there were significant correlations in the data for factor analysis. Results of an EFA suggested a five-factor model of the SCI-2. However, although there were interpretable loadings in the first four factors, the fifth factor consisted exclusively of reversecoded items. This often occurs among reverse-coded items due to possible shared covariances [60] and is often considered to reflect poor model fit [61]. Accordingly, the four-factor model was retained for subsequent analyses. The full factor structure, including each factor’s corresponding items (in English), is presented in Table 3. Three items (items 12, 29, and 36) significantly loaded on multiple factors, whereas two items (items 3 and 28) did not load on any factor; thus, these items were removed from the scale and subsequent analyses. Additionally, three reverse-coded items (items 24, 34, and 37) negatively loaded onto a factor, inconsistent with their theoretical conceptualization; these items were also removed from the resultant scale. All factors were significantly correlated with each other, r=0.62–0.85, ps<0.01, consistent with the conceptualization of the SCS as a strongly related construct.
The first factor, called hopelessness and overwhelming distress, included 9 items assessing features of cognitive disturbances including uncontrollable thoughts, hopelessness, and inescapable feelings as well as insomnia and emotional pain. The second factor, named affective, cognitive, and physical disturbances, included 32 items that loaded various items related to a range of symptoms including entrapment, emotional pain, agitation, anhedonia, irritability, rumination, and failed thoughts suppression. The third factor incorporated 5 items measuring aspects of anxiety and bodily symptoms and was thus called extreme anxiety. Lastly, the fourth factor contained the same 5 items from the original questionnaire and was labeled social withdrawal as it evaluated one’s desires of and responses to social interactions.

CFA

Results of the four-factor CFA, with factors derived from the results of the EFA exhibited good model fit (χ2[1319]=3132.56, p<0.001, CFI=1.00, TLI=1.00, RMSEA=0.05, SRMR=0.04). Standardized factor loadings for each model are presented in Table 3. All latent factors in the four-factor model were significantly related to each other (r=0.70–0.92, ps< 0.001).

Reliability and validity of the SCI-2

Internal consistency

Internal consistencies of the one-factor, five-factor, and four-factor models were evaluated with Cronbach’s alpha and are presented in Table 4. The full 61-item scale for the one-factor model had excellent reliability with an alpha value of 0.98. For the five-factor model, each of the factors presented high reliability with α=0.95 for entrapment, α=0.93 for affective disturbance, α=0.86 for loss of cognitive control, α=0.93 for hyperarousal, and α=0.90 for social withdrawal.
The full 53-item Korean version of the scale also presented excellent reliability with an alpha value of 0.98. Each of the factors also had high reliability with α=0.93 for hopelessness and overwhelming distress, α=0.98 for affective, cognitive, and physical disturbances, α=0.89 for extreme anxiety and α=0.90 for social withdrawal.

Concurrent validity

To assess the concurrent validity of the questionnaire to examine whether the full 61-item Korean version of the SCI-2 similarly measures the construct of suicidality, we evaluated correlations between the current scale with lifetime and past month suicidality employing the C-SSRS. There was a significant, yet moderate, correlation with coefficients r=0.35, p< 0.01 and r=0.33, p<0.01, respectively. PHQ-9 and GAD-7 scales were also significantly correlated with the SCI-2 total score with correlations of r=0.69, p<0.01 and r=0.72, p<0.01, accordingly. Negligible correlations were discerned between extreme anxiety with lifetime (r=0.28, p<0.01) and past month (r=0.18, p<0.01) SI. Detailed results are presented in Table 5.

DISCUSSION

The aim of this study was to examine the factor structure and the validity of the SCI-2 in a sample of South Korean participants. As posited in our first hypothesis, CFA results of the one-factor and five-factor models exhibited strong model fits which indicate that SCI-2 can be used in its current 61-item version in Korea as in the United States. To examine alternative factor structures, we conducted an EFA in which results yielded a four-factor model with 53 items and showed similarities with the results of the SCS network analysis [42] where the second factor incorporated components of helplessness and loss of cognitive control. Finally, internal consistency, reliability, and validity of the inventory were examined as appropriate.
According to our EFA results, the first factor mainly embodied aspects of hopelessness (e.g., Did you feel hopeless?) and overwhelming thoughts that are difficult to control (e.g., Did you feel that your head could explode from too many thoughts?) with one emotional pain item (Did you have a sense of inner pain that was too much to bear?) and one insomnia item (Did you wake up from sleep tired and not refreshed?). Together, we titled it hopelessness and overwhelming distress. Hopelessness is a psychological construct that has been identified as one of the major red flags for suicide risk [62-65]. In a 10-year longitudinal study, Beck et al. [63] found that intense hopelessness significantly accounted for future suicide. In addition, in various theories of suicide, such as the Three-Step Theory (3ST) [66] and the interpersonal theory of suicide [39,40,67], hopelessness is regarded as one of the key contributors to suicidal thoughts and behaviors. In addition, having uncontrollable, overwhelming thoughts (also known as ruminative flooding) is another major risk factor for suicide [18]. As regards to the original SCI-2, this first factor was comprised of hopelessness items from the entrapment factor and ruminative flooding items from loss of cognitive control factor. On the other hand, items related to helplessness (e.g., Did you feel helpless to change?) from the entrapment factor loaded under factor 2 along with emotional, cognitive, and physical disturbances. A core difference of frantic hopelessness/entrapment from hopelessness and overwhelming distress is that the former does not incorporate symptoms of ruminative thoughts and emphasizes an intense sense of doom [12,68]. Consistently, a network analysis of the SCS indicated that entrapment and ruminative flooding are strongly connected to other SCS symptoms [42], suggesting that they are central or facilitating factors among other crisis symptoms. As such, a combined state of hopelessness and overwhelming distress may be a subsyndrome of suicide crisis in a Korean sample. Similar to the network analysis of the SCS [42], uncontrollable hopelessness manifested in factor 1 may be a core suicide crisis symptom cluster across the culture.
The second factor had the most item loadings encompassing diverse symptoms of the helplessness part of entrapment, emotional pain, rapid spikes of negative emotions, acute anhedonia, irritability, agitation, hypervigilance, and rumination. It was challenging to develop a name of its own as it incorporated a mixture of affective and cognitive disturbances, coupled with hyperarousal manifestations. A possible explanation for this is that certain symptoms co-occur and interact with others during states of suicidal crisis. By integrating these features, we decided to label the factor as affective, cognitive, and physical disturbances. In fact, the second factor included most of the items from the affective disturbance, loss of cognitive control, and hyperarousal factors of the original SCI-2. We believe that cultural differences play role in the assemble of these affective characteristics. As De Vaus et al. [69] indicated, Eastern culture and Western culture differ in how people interpret and react to negative emotions. Unlike Westerners, Easterners embrace a holistic style and process information integrating the whole context. In Korea particularly, a cultural syndrome of “hwa-byung” exists, in which people experience a mixture of emotional, cognitive, and physical symptoms [70-72]. As such, the factor structure of the English version of the SCI-2 loads items by the different emotional states while the Korean version coalesces many of the affective disturbance characteristics together. In addition, with regards to cognitive disturbances, we noticed a subtle difference among the items that were divided into factor 1 and factor 2. Among the items relevant to ruminative flooding, those in factor 1 asks about the repercussions of ruminating (e.g., Did you have a decreased ability to think, concentrate or make decisions due to too many thoughts?), whereas items in factor 2 inquires one’s physical conditions (e.g., Did you feel pressure in your head from thinking too much?).
The third factor, extreme anxiety, incorporated symptoms of anxiety accompanying panic-like symptoms and unusual or strange sensations. These items were originally a subscale that belonged to the affective disturbance factor in the SCI-2 English version; however, they loaded as a separate factor in our analysis. The content in the factor is related to frantic worry accompanied by physical symptoms or sensory distortion (e.g., Did you feel that ordinary things looked strange or distorted?). It is notable that only extreme anxiety subscale resulted in a distinct symptom cluster from the other three subscales of affective disturbance of the SCS diagnostic criteria, which were mostly loaded on factor 2. One potential explanation for this outcome is that our data was based on a community sample whereas the SCI-2 English version was validated in clinical samples. Further, this factor was originally developed based on research that panic-like symptoms and sensory distortions predict suicide attempt among psychiatric patients [43,73]. Further examination of the utility of this factor is warranted.
The final factor, titled social withdrawal, is the only factor that loaded identical items as the original questionnaire evaluating respondent’s relationship with others (e.g., Did you feel isolated from others?). As claimed in the interpersonal theory of suicide, lacking sense of belongingness is a detrimental risk factor for suicidal behavior [39]. The two constructs in the theory, thwarted belongingness and perceived burdensome, both elaborate on the social triggers that engender suicidal desires [67]. Examples include not receiving appropriate reciprocal care from families and friends as well as the incorrect interpretation of the need to sacrifice to prevent being a burden to others. Overall, the factor analysis suggests that four distinct factors are inherent in the Korean version of the SCI-2.
Reliability and concurrent validity of the inventory were examined through correlations and Cronbach’s alpha values, respectively. Interestingly, the correlation between the SCI-2 and the C-SSRS, both assessing levels of suicidality, was relatively moderate. One possible explanation for this observation is that the questions from the C-SSRS that were selected for this study assess SI while the SCI-2 is theorized to measure one’s risk for imminent suicidal behavior. A bulk of former studies have claimed that not everyone who harbors SI necessarily attempts suicide [74-76] and the recent validation study of the SCI-2 suggested that this measure is a better predictor of suicidal behaviors [41]. Additionally, the timeframe that the two assessments base their questions on are also different: the SCI-2 inquires about one’s state for the past several days, whereas the C-SSRS asks about one’s ideation over the past month and across one’s lifetime. The results of the current study are promising in that novel tools are being developed to assess near-term suicide risk. Such instruments could be more clinically informative and effective in ameliorating rates of deaths by suicide.
Our CFA results indicate that the model fit of the one-factor, original five-factor, and four factor structures are all relevant with excellent fit scores. Conceptually, the original subfactors of emotional, cognitive, physical, and social disturbances of the SCI-2 are well defined [41]. Empirically, however, clinical manifestations of those disturbances could differ across nations, society, or cultural contexts. Considering that suicide is an outcome of complex and heterogeneous factors, the single factor solution may provide a stable construct across countries, while different factor models may help enhance our understanding of culture-specific clinical manifestations of the suicide crisis. As this study is the first validation study of the SCI-2 in an Eastern population, further examination and replication studies are needed to generalize these findings.
Limitations are inherent in the current study. First, our sample was not representative of a diverse group since demographic characteristics ratios were not proportionate. For example, data from older participants (e.g., over 40) was limited. Since data collection was entirely conducted online, it may have been cumbersome for participants who are not familiar with social media to complete it. In this context, these results may not be generalizable to those who are unfamiliar with social media and technology. Also, our results are based on data from a community sample and therefore are not generalizable to clinical samples. Second, considering that our data was collected during the outbreak of the COVID-19 pandemic when people experienced drastic changes in their daily lives, it is probable that levels of depression, anxiety, and suicidality were reported relatively higher than they would have been without the pandemic extant throughout the study [77]. Third, due to the cross-sectional nature of the data, the current study was unable to examine the predictive validity of the SCI-2. Thus, further study with a longitudinal study design would advance current findings. Finally, the study was short in providing sensitivity, specificity, and cutoff point by the SCS diagnosis as it was a cross-sectional study. Future studies should attempt to examine these aspects for it will elevate improved near-term suicide risk assessment.
Moving onward, we suggest that future studies attempt to validate the Korean version of the SCI-2 in a range of community and clinical samples in order to examine the generalizability of the inventory. In addition, it would be beneficial if researchers could identify possible subconstructs that is inherent in factor 2, affective, cognitive, and physical disturbances, as it consists of the most items. Upon inspecting the items that are loaded under this factor, it may be viable, for instance, to divide affective disturbances (e.g., Did you feel relentless, agonizing emotional pain?) from cognitive disturbances (e.g., Did you want your troubling thoughts to go away but they wouldn’t?) and physical disturbances (e.g., Did you feel tensed or keyed up?). As such, further investigating and refining the inventory will be valuable in supporting the clinical utility of the SCI-2.
In conclusion, the proposed one-factor and five-factor models of the SCI-2 present strong model fits. Our four-factor model also helps delineate potential critical symptoms that must be flagged when assessing risk for suicide in Korea. This alternative factor structure constitutes a total of 53 items that are organized under hopelessness and overwhelming distress (factor 1), affective, cognitive, and physical disturbances (factor 2), extreme anxiety (factor 3), and social withdrawal (factor 4). The factor structure of the inventory may be culture-sensitive and thus requires further study. Although the results are exploratory in nature, our efforts in redefining traditional suicide risk assessments (e.g., detecting presence of SI or history of suicide attempts) seem promising. It is critical to identify symptoms, or possibly profiles, that are immanent among suicidal individuals in a more objective manner as not everyone is transparent about their suicidal desires. Especially, given the low rates of mental health service utilization in Korea, it would be ideal to develop a scale that is applicable for both patients and non-patients. Developing tools to discern individuals with imminent suicide risk will significantly contribute to the suicide literature. Furthermore, it will be clinically beneficial such that psychologists will be able to acknowledge individuals who are at high-risk and thus implement necessary preventive measures in advance to impede one from attempting suicide.

Notes

Availability of Data and Material

The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors have no potential conflicts of interest to disclose.

Author Contributions

Conceptualization: Ji Yoon Park, Sungeun You. Data curation: Ji Yoon Park, Jenelle A. Richards, Sungwoo Lee. Formal analysis: Ji Yoon Park, Megan L. Rogers. Investigation: Sarah Bloch-Elkouby, Igor Galynker. Methodology: Ji Yoon Park, Megan L. Rogers, Sungeun You. Supervision: Igor Galynker, Sungeun You. Funding acquisition: Sungeun You. Writing—original draft: Ji Yoon Park, Megan L. Rogers. Writing—review & editing: all authors.

Funding Statement

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF2020S1A5A2A03044181).

Table 1.
Demographic characteristics of the sample
Characteristics Value
Gender
 Men 295 (27.8)
 Women 766 (72.2)
Age (yr) 30.56±8.49
Education
 Did not complete high school 3 (0.3)
 High school 82 (7.7)
 Some college/2-year college (associate degree) 96 (9.0)
 4-year college (bachelor’s degree) 635 (59.8)
 Master’s degree 215 (20.3)
 Doctorate 30 (2.8)
Marital status
 Single/never married 733 (69.1)
 Married 305 (28.7)
 Cohabitating but not married 15 (1.4)
 Separated 1 (0.1)
 Divorced 6 (0.6)
 Widowed 1 (0.1)
Job status
 Full-time job 385 (36.3)
 Part-time job 159 (15.0)
 Self-employed 36 (3.4)
 Unemployed 54 (5.1)
 Retired 6 (0.6)
 Housewife 64 (6.0)
 Student 313 (29.5)
 Other 43 (4.1)

Values are presented mean±standard deviation, number (%)

Table 2.
Confirmatory factor analysis of the Suicide Crisis Inventory-2: a comparison of one and five-factor models (N=1,061)
χ2 df CFI TLI RMSEA SRMR
One-factor model 16,150.33 1,769 0.99 0.99 0.08 0.06
Five-factor model 11,603.66 1,759 0.99 0.99 0.07 0.05

CFI, comparative fit index; TLI, Tucker-Lewis index; RMSEA, root mean square error of approximation; SRMR, standardized root mean residual

Table 3.
Exploratory factor analysis and the factor structure of the Suicide Crisis Inventory-2 (N=530)
Item (English) Factor
1 2 3 4
Factor 1 – Hopelessness and overwhelming distress (9 items)
61 Did you have many thoughts in your head? 0.669* 0.051 -0.097 0.080
59 Did you feel that your head could explode from too many thoughts? 0.603* 0.185 0.198 0.014
17 Did you feel that it was hard for you to stop worrying? 0.564* 0.337 -0.022 0.015
2 Did you feel there was no exit? 0.542* 0.103 0.124 0.195
35 Did you feel hopeless? 0.522* 0.264 0.134 0.136
18 Did you have a decreased ability to think, concentrate or make decisions due to too many thoughts? 0.513* 0.269 0.044 0.086
21 Did you wake up from sleep tired and not refreshed? 0.507* 0.033 0.108 0.115
39 Did you feel that there was no way out? 0.481* 0.154 0.249 0.183
44 Did you have a sense of inner pain that was too much to bear? 0.466* 0.261 0.218 0.101
Factor 2 – Affective, cognitive, and physical disturbances (32 items)
26 Did you want your troubling thoughts to go away but they wouldn’t? 0.153 0.882* -0.201 -0.049
1 Did you feel a sense of inner pain that had to be stopped? 0.052 0.825* 0.031 -0.030
33 Did you feel that ideas kept turning over and over in your mind and they wouldn’t go away? 0.192 0.822* -0.144 -0.059
48 Did you feel powerless to stop the thoughts that were upsetting you? 0.074 0.813* -0.050 0.012
16 Did you feel you wanted to crawl out of your skin? 0.050 0.766* -0.044 0.013
13 Did you feel nervousness or shakiness inside? 0.081 0.761* 0.015 0.029
38 Did you feel dissatisfied or bored with everything? -0.013 0.744* -0.026 0.055
8 Did you feel any unusually intense or deep negative feelings or mood swings directed towards someone else? -0.079 0.739* 0.048 0.059
19 Did you feel that there were no good solutions to your problems? -0.069 0.734* 0.116 0.062
45 Did you feel any unusually intense or deep negative feelings or mood swings directed towards yourself? 0.002 0.718* 0.047 0.067
54 Did you feel that your emotional pain was unbearable? 0.062 0.711* 0.085 0.089
7 Did you feel you were constantly watching for signs of trouble? 0.038 0.705* -0.084 0.081
27 Did you feel doomed? 0.043 0.688* 0.008 0.129
49 Did you feel so restless you could not sit still? -0.071 0.684* 0.188 -0.046
43 Did you feel that the urge to escape the pain was very hard to control? -0.017 0.684* 0.222 0.028
32 Did you feel that if you didn’t stay alert and watchful, something bad would happen? -0.084 0.668* 0.024 0.093
14 Did you feel pressure in your head from thinking too much? 0.218 0.661* 0.022 -0.007
57 Did you feel like you were getting a headache from too many thoughts in your head? 0.200 0.653* 0.038 0.006
4 Did you feel yourself thinking that things would never change? -0.030 0.627* 0.183 0.143
15 Did you feel trapped? 0.137 0.612* 0.103 0.084
60 Did you feel so stirred up inside you wanted to scream? -0.018 0.610* 0.089 0.049
47 Did you feel tensed or keyed up? 0.214 0.608* -0.099 0.028
56 Did you feel there is no escape? 0.000 0.607* 0.184 0.127
53 Did you feel easily annoyed or irritated? 0.100 0.601* -0.239 0.213
41 Did you have temper outbursts that you could not control? 0.010 0.574* 0.109 0.059
25 Did you feel helpless to change? 0.145 0.557* 0.037 0.146
9 Did you feel your views were very consistent over time? -0.138 0.550* 0.233 0.063
11 Did you feel bothered by thoughts that did not make sense? 0.099 0.532* 0.238 0.000
46 Did you feel relentless, agonizing emotional pain? 0.248 0.507* 0.126 0.132
10 Did you feel you had lost your interest in other people? -0.108 0.503* 0.048 0.289
20 Did you feel that most people could not be trusted? -0.054 0.457* 0.066 0.297
42 Did you get into frequent arguments? -0.129 0.450* 0.272 0.133
Factor 3 – Extreme anxiety (5 items)
50 Did you feel unusual physical sensations that you have never felt before? 0.240 -0.081 0.675* 0.098
22 Did you feel strange sensations in your body or on your skin? 0.013 0.129 0.655* 0.045
6 Did you feel suddenly frightened to such an extent that you developed physical symptoms or had a panic attack? -0.035 0.189 0.640* 0.106
5 Did you become afraid that you would die? 0.192 -0.046 0.602* 0.136
30 Did you feel that ordinary things looked strange or distorted? 0.326 0.147 0.495* -0.006
Factor 4 – Social withdrawal (5 items)
55 Did you evade communications with people who care about you? 0.022 -0.068 -0.017 0.919*
40 Did you push away people who care about you? 0.005 -0.048 0.023 0.891*
31 Did you feel you did not open up to members of your family/friends? 0.045 0.100 -0.077 0.737*
52 Did you interact less with people who care about you? 0.020 0.075 -0.005 0.662*
23 Did you feel isolated from others? -0.021 0.223 0.043 0.614*
Retracted items
29 Did you have trouble falling asleep because you were having thoughts that you could not control? 0.363 0.345 0.108 0.079
58 Did you feel that the world was closing in on you? 0.095 0.182 0.413 0.289
51 Did you feel your thoughts were racing? 0.209 0.441 0.285 -0.031
12 Did you feel blood rushing through your veins? 0.035 0.358 0.456 0.025
28 Did you find pleasure in your hobbies and pastimes? 0.176 -0.053 -0.226 0.029
3 Did you enjoy being with your family or close friends? 0.102 -0.156 -0.086 0.253
36 Did you feel a lot of emotional turmoil in your gut? -0.124 0.465 0.392 0.086
24 Did you often change your mind? 0.185 -0.540 -0.262 0.029
37 Did you feel you could easily change your mind over things that bother you? 0.203 -0.386 -0.289 -0.017
34 Did you feel you could change your mind once you’ve come to a conclusion? 0.052 -0.429 -0.115 -0.021

Rotation method: direct oblimin rotation.

* factor loadings greater than 0.45

Table 4.
Reliability of the Suicide Crisis Inventory-2 (N=1,061)
Number of items Cronbach’s α
One factor model 61 0.98
Five factor model
 Factor 1 – Entrapment 10 0.95
 Factor 2 – Affective disturbance 18 0.93
 Factor 3 – Loss of cognitive control 15 0.86
 Factor 4 – Hyperarousal 13 0.93
 Factor 5 – Social withdrawal 5 0.90
Four factor model
 Factor 1 – Hopelessness and overwhelming distress 9 0.93
 Factor 2 – Affective, cognitive, and physical disturbances 32 0.98
 Factor 3 – Extreme anxiety 5 0.89
 Factor 4 – Social withdrawal 5 0.90
Table 5.
Concurrent validity of the Korean version of the Suicide Crisis Inventory-2 (N=1,061)
1 2 3 4 5 6 7 8 9 10 11 12 13 14
1 SCI-2 total score (61-item) -
2 SCI-2 F1 0.95* -
3 SCI-2 F2 0.96* 0.91* -
4 SCI-2 F3 0.85* 0.83* 0.79* -
5 SCI-2 F4 0.95* 0.88* 0.89* 0.78* -
6 SCI-2 F5 0.78* 0.73* 0.74* 0.51* 0.73* -
7 EFA SCI-2 f1 0.88* 0.88* 0.81* 0.88* 0.83* 0.62* -
8 EFA SCI-2 f2 0.98* 0.94* 0.94* 0.87* 0.96* 0.75* 0.85* -
9 EFA SCI-2 f3 0.84* 0.78* 0.90* 0.58* 0.73* 0.62* 0.67* 0.76* -
10 EFA SCI-2 f4 0.78* 0.73* 0.74* 0.51* 0.73* 1.00* 0.62* 0.75* 0.62* -
11 C-SSRS lifetime 0.35* 0.36* 0.26* 0.23* 0.35* 0.35* 0.30* 0.36* 0.28* 0.35* -
12 C-SSRS past month 0.33* 0.34* 0.23* 0.28* 0.34* 0.34* 0.34* 0.35* 0.19* 0.33* 0.53* -
13 PHQ-9 total score 0.69* 0.67* 0.67* 0.49* 0.67* 0.60* 0.60* 0.68* 0.59* 0.60* 0.46* 0.39* -
14 GAD-7 total score 0.72* 0.69* 0.67* 0.52* 0.71* 0.58* 0.62* 0.72* 0.60* 0.58* 0.42* 0.33* 0.83* -
M 118.68 15.68 30.35 37.63 21.51 7.37 17.68 52.61 6.67 7.37 0.49 1.14 7.27 5.53
SD 67.96 10.64 14.67 8.65 12.41 5.20 8.87 33.31 5.81 5.20 1.20 1.66 5.79 4.91

* p<0.01.

C-SSRS lifetime and past month indicate the highest score of five items assessing suicidal ideation severity of the C-SSRS, respectively. SCI-2, Suicide Crisis Inventory-2; F1, entrapment; F2, affective disturbance; F3, loss of cognitive control; F4, hyperarousal; F5, social withdrawal; EFA, exploratory factor analysis; f1, hopelessness and overwhelming distress; f2, cognitive, affective, and physical disturbances; f3, extreme anxiety; f4, social withdrawal; C-SSRS, Columbia-Suicide Severity Rating Scale; PHQ-9, Patient Health Questionnaire-9; GAD-7, Generalized Anxiety Disorder-7; M, mean; SD, standard deviation

REFERENCES

1. Franklin JC, Ribeiro JD, Fox KR, Bentley KH, Kleiman EM, Huang X, et al. Risk factors for suicidal thoughts and behaviors: a meta-analysis of 50 years of research. Psychol Bull 2017;143:187–232.
crossref pmid
2. Kleiman EM, Turner BJ, Fedor S, Beale EE, Huffman JC, Nock MK. Examination of real-time fluctuations in suicidal ideation and its risk factors: results from two ecological momentary assessment studies. J Abnorm Psychol 2017;126:726–738.
crossref pmid
3. Anestis MD, Soberay KA, Gutierrez PM, Hernández TD, Joiner TE. Reconsidering the link between impulsivity and suicidal behavior. Pers Soc Psychol Rev 2014;18:366–386.
crossref pmid
4. Millner AJ, Lee MD, Nock MK. Describing and measuring the pathway to suicide attempts: a preliminary study. Suicide Life Threat Behav 2017;47:353–369.
crossref pmid
5. Silverman MM, Berman AL. Suicide risk assessment and risk formulation part I: a focus on suicide ideation in assessing suicide risk. Suicide Life Threat Behav 2014;44:420–431.
crossref pmid
6. Giddens JM, Sheehan DV. Is there value in asking the question “do you think you would be better off dead?” in assessing suicidality? A case study. Innov Clin Neurosci 2014;11:182–190.
pmid pmc
7. Podlogar MC, Rogers ML, Chiurliza B, Hom MA, Tzoneva M, Joiner T. Who are we missing? Nondisclosure in online suicide risk screening questionnaires. Psychol Assess 2016;28:963–974.
crossref pmid
8. Richards JE, Whiteside U, Ludman EJ, Pabiniak C, Kirlin B, Hidalgo R, et al. Understanding why patients may not report suicidal ideation at a health care visit prior to a suicide attempt: a qualitative study. Psychiatr Serv 2019;70:40–45.
crossref pmid
9. Berman AL. Risk factors proximate to suicide and suicide risk assessment in the context of denied suicide ideation. Suicide Life Threat Behav 2018;48:340–352.
crossref pmid
10. Luoma JB, Martin CE, Pearson JL. Contact with mental health and primary care providers before suicide: a review of the evidence. Am J Psychiatry 2002;159:909–916.
crossref pmid pmc
11. Menon V, Bafna AR, Rogers ML, Richards J, Galynker I. Factor structure and validity of the revised Suicide Crisis Inventory (SCI-2) among Indian adults. Asian J Psychiatr 2022;73:103119
crossref pmid
12. Galynker I, Yaseen ZS, Cohen A, Benhamou O, Hawes M, Briggs J. Prediction of suicidal behavior in high risk psychiatric patients using an assessment of acute suicidal state: the Suicide Crisis Inventory. Depress Anxiety 2017;34:147–158.
crossref pmid
13. Rogers ML, Joiner TE. Rumination, suicidal ideation, and suicide attempts: a meta-analytic review. Rev Gen Psychol 2017;21:132–142.
crossref
14. Schuck A, Calati R, Barzilay S, Bloch-Elkouby S, Galynker I. Suicide crisis syndrome: a review of supporting evidence for a new suicide-specific diagnosis. Behav Sci Law 2019;37:223–239.
crossref pmid
15. Forkmann T, Spangenberg L, Rath D, Hallensleben N, Hegerl U, Kersting A, et al. Assessing suicidality in real time: a psychometric evaluation of self-report items for the assessment of suicidal ideation and its proximal risk factors using ecological momentary assessments. J Abnorm Psychol 2018;127:758–769.
crossref pmid
16. Barzilay S, Assounga K, Veras J, Beaubian C, Bloch-Elkouby S, Galynker I. Assessment of near-term risk for suicide attempts using the Suicide Crisis Inventory. J Affect Disord 2020;276:183–190.
crossref pmid
17. Li S, Galynker II, Briggs J, Duffy M, Frechette-Hagan A, Kim HJ, et al. Attachment style and suicide behaviors in high risk psychiatric inpatients following hospital discharge: the mediating role of entrapment. Psychiatry Res 2017;257:309–314.
crossref pmid
18. Li S, Yaseen ZS, Kim HJ, Briggs J, Duffy M, Frechette-Hagan A, et al. Entrapment as a mediator of suicide crises. BMC Psychiatry 2018;18:4
crossref pmid pmc
19. O’Connor RC. The integrated motivational-volitional model of suicidal behavior. Crisis 2011;32:295–298.
crossref pmid
20. Gilbert P, Allan S. The role of defeat and entrapment (arrested flight) in depression: an exploration of an evolutionary view. Psychol Med 1998;28:585–598.
crossref pmid
21. De Beurs D, Cleare S, Wetherall K, Eschle-Byrne S, Ferguson E, B O’Connor D, et al. Entrapment and suicide risk: the development of the 4-item entrapment scale short-form (E-SF). Psychiatry Res 2020;284:112765
crossref pmid
22. Hendin H, Maltsberger JT, Szanto K. The role of intense affective states in signaling a suicide crisis. J Nerv Ment Dis 2007;195:363–368.
crossref pmid
23. Hendin H, Al Jurdi RK, Houck PR, Hughes S, Turner JB. Role of intense affects in predicting short-term risk for suicidal behavior: a prospective study. J Nerv Ment Dis 2010;198:220–225.
pmid
24. Busch KA, Fawcett J, Jacobs DG. Clinical correlates of inpatient suicide. J Clin Psychiatry 2003;64:14–19.
crossref
25. Yaseen ZS, Chartrand H, Mojtabai R, Bolton J, Galynker II. Fear of dying in panic attacks predicts suicide attempt in comorbid depressive illness: prospective evidence from the national epidemiological survey on alcohol and related conditions. Depress Anxiety 2013;30:930–939.
crossref pmid
26. Hawes M, Galynker I, Barzilay S, Yaseen ZS. Anhedonia and suicidal thoughts and behaviors in psychiatric outpatients: the role of acuity. Depress Anxiety 2018;35:1218–1227.
crossref pmid
27. Perrah M, Wichman H. Cognitive rigidity in suicide attempters. Suicide Life Threat Behav 1987;17:251–255.
crossref pmid
28. Cukrowicz KC, Ekblad AG, Cheavens JS, Rosenthal MZ, Lynch TR. Coping and thought suppression as predictors of suicidal ideation in depressed older adults with personality disorders. Aging Ment Health 2008;12:149–157.
crossref pmid
29. Pettit JW, Temple SR, Norton PJ, Yaroslavsky I, Grover KE, Morgan ST, et al. Thought suppression and suicidal ideation: preliminary evidence in support of a robust association. Depress Anxiety 2009;26:758–763.
crossref pmid
30. Rogers ML, Ringer FB, Joiner TE. A meta-analytic review of the association between agitation and suicide attempts. Clin Psychol Rev 2016;48:1–6.
crossref pmid
31. Chu C, Klein KM, Buchman-Schmitt JM, Hom MA, Hagan CR, Joiner TE. Routinized assessment of suicide risk in clinical practice: an empirically informed update. J Clin Psychol 2015;71:1186–1200.
crossref pmid
32. Jha MK, Minhajuddin A, Chin Fatt C, Kircanski K, Stringaris A, Leibenluft E, et al. Association between irritability and suicidal ideation in three clinical trials of adults with major depressive disorder. Neuropsychopharmacology 2020;45:2147–2154.
crossref pmid pmc
33. Malhi G, Bell E, Das P, Outhred T. Relating irritability and suicidal ideation using mood and anxiety. Evid Based Ment Health 2019;22:95–99.
crossref pmid pmc
34. Trivedi MH, Morris DW, Wisniewski SR, Nierenberg AA, Gaynes BN, Kurian BT, et al. Clinical and sociodemographic characteristics associated with suicidal ideation in depressed outpatients. Can J Psychiatry 2013;58:113–122.
crossref pmid
35. Chan JW, Lam SP, Li SX, Yu MW, Chan NY, Zhang J, et al. Eveningness and insomnia: independent risk factors of nonremission in major depressive disorder. Sleep 2014;37:911–917.
crossref pmid pmc
36. Li SX, Lam SP, Yu MW, Zhang J, Wing YK. Nocturnal sleep disturbances as a predictor of suicide attempts among psychiatric outpatients: a clinical, epidemiologic, prospective study. J Clin Psychiatry 2010;71:1440–1446.
crossref pmid
37. Pigeon WR, Pinquart M, Conner K. Meta-analysis of sleep disturbance and suicidal thoughts and behaviors. J Clin Psychiatry 2012;73:e1160–e1167.
crossref pmid
38. Stanley IH, Rogers ML, Hanson JE, Gutierrez PM, Joiner TE. PTSD symptom clusters and suicide attempts among high-risk military service members: a three-month prospective investigation. J Consult Clin Psychol 2019;87:67–78.
crossref pmid
39. Joiner TE. Why people die by suicide. Cambridge, MA: Harvard University Press; 2005.

40. Van Orden KA, Witte TK, Cukrowicz KC, Braithwaite SR, Selby EA, Joiner TE Jr. The interpersonal theory of suicide. Psychol Rev 2010;117:575–600.
crossref pmid pmc
41. Bloch-Elkouby S, Barzilay S, Gorman BS, Lawrence OC, Rogers ML, Richards J, et al. The revised Suicide Crisis Inventory (SCI-2): validation and assessment of prospective suicidal outcomes at one month follow-up. J Affect Disord 2021;295:1280–1291.
crossref pmid
42. Bloch-Elkouby S, Gorman B, Schuck A, Barzilay S, Calati R, Cohen LJ, et al. The suicide crisis syndrome: a network analysis. J Couns Psychol 2020;67:595–607.
crossref pmid
43. Yaseen ZS, Hawes M, Barzilay S, Galynker I. Predictive validity of proposed diagnostic criteria for the suicide crisis syndrome: an acute presuicidal state. Suicide Life Threat Behav 2019;49:1124–1135.
crossref pmid
44. Ying G, Cohen LJ, Lloveras L, Barzilay S, Galynker I. Multi-informant prediction of near-term suicidal behavior independent of suicidal ideation. Psychiatry Res 2020;291:113169
crossref pmid
45. Posner K, Brown GK, Stanley B, Brent DA, Yershova KV, Oquendo MA, et al. The Columbia-Suicide Severity Rating Scale: initial validity and internal consistency findings from three multisite studies with adolescents and adults. Am J Psychiatry 2011;168:1266–1277.
crossref pmid pmc
46. Brown LA, Boudreaux ED, Arias SA, Miller IW, May AM, Camargo CA Jr, et al. C-SSRS performance in emergency department patients at high risk for suicide. Suicide Life Threat Behav 2020;50:1097–1104.
crossref pmid pmc
47. Jang H, Park E, Jon D, Park H, Hong H, Jung M, et al. Validation of the Columbia suicide severity rating scale in depression patients. Kor J Clin Psychol 2014;33:799–814.
crossref
48. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001;16:606–613.
crossref pmid pmc
49. An J, Seo E, Lim K, Shin J, Kim J. Standardization of the Korean version of screening tool for depression (Patient Health Questionnaire-9, PHQ-9). J Korean Soc Biol Ther Psychiatry 2013;19:47–56.

50. Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med 2006;166:1092–1097.
crossref pmid
51. Ahn JK, Kim Y, Choi KH. The psychometric properties and clinical utility of the Korean version of GAD-7 and GAD-2. Front Psychiatry 2019;10:127
crossref pmid pmc
52. Chapman DW, Carter JF. Translation procedures for the cross cultural use of measurement instruments. Educ Eval Policy Anal 1979;1:71–76.
crossref
53. Kaiser HF, Rice J. Little jiffy, mark IV. Educ Psychol Meas 1974;34:111–117.
crossref
54. Bartlett MS. The effect of standardization on a χ2 approximation in factor analysis. Biometrika 1951;38:337–344.
crossref
55. Costello AB, Osborne J. Best practices in exploratory factor analysis: four recommendations for getting the most from your analysis. Pract Assess Res Eval 2005;10:7

56. Osborne J. Best practices in quantitative methods. Thousand Oaks, CA: SAGE Publications, Inc; 2008.

57. Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. Struct Equ Model 1999;6:1–55.
crossref
58. Rosseel Y. lavaan: an R package for structural equation modeling. J Stat Softw 2012;48:1–36.

59. Jorgensen TD, Pornprasertmanit S, Schoemann AM, Rosseel Y, Miller P, Quick C, et al. semTools: Useful Tools for Structural Equation Modeling 2021 [Internet]. Available at: https://cran.r-project.org/web/packages/semTools/index.html. Accessed August 13, 2021.

60. Williams LJ, Ford LR, Nguyen N. Basic and advanced measurement models for confirmatory factor analysis. In: Rogelberg SG, editor. Handbook of research methods in industrial and organizational psychology. Hoboken, NJ: Blackwell Publishing Ltd, 2004, p. 366–389.

61. Marsh HW. Negative item bias in ratings scales for preadolescent children: a cognitive-developmental phenomenon. Dev Psychol 1986;22:37–49.
crossref
62. Beck AT, Brown G, Berchick RJ, Stewart BL, Steer RA. Relationship between hopelessness and ultimate suicide: a replication with psychiatric outpatients. Focus 2006;4:291–296.
crossref
63. Beck AT, Steer RA, Kovacs M, Garrison B. Hopelessness and eventual suicide: a 10-year prospective study of patients hospitalized with suicidal ideation. Am J Psychiatry 1985;142:559–563.
crossref pmid
64. Minkoff K, Bergman E, Beck AT, Beck R. Hopelessness, depression, and attempted suicide. Am J Psychiatry 1973;130:455–459.
crossref pmid
65. Wolfe KL, Nakonezny PA, Owen VJ, Rial KV, Moorehead AP, Kennard BD, et al. Hopelessness as a predictor of suicide ideation in depressed male and female adolescent youth. Suicide Life Threat Behav 2019;49:253–263.
crossref pmid pmc
66. Klonsky ED, May AM. The Three-Step Theory (3ST): a new theory of suicide rooted in the “ideation-to-action” framework. Int J Cogn Ther 2015;8:114–129.
crossref
67. Chu C, Buchman-Schmitt JM, Stanley IH, Hom MA, Tucker RP, Hagan CR, et al. The interpersonal theory of suicide: a systematic review and meta-analysis of a decade of cross-national research. Psychol Bull 2017;143:1313–1345.
crossref pmid pmc
68. Yaseen Z, Katz C, Johnson MS, Eisenberg D, Cohen LJ, Galynker II. Construct development: the Suicide Trigger Scale (STS-2), a measure of a hypothesized suicide trigger state. BMC Psychiatry 2010;10:110
crossref pmid pmc
69. De Vaus J, Hornsey MJ, Kuppens P, Bastian B. Exploring the east-west divide in prevalence of affective disorder: a case for cultural differences in coping with negative emotion. Pers Soc Psychol Rev 2018;22:285–304.
crossref pmid
70. American Psychological Association. Hwa-byung [Internet]. Available at: https://dictionary.apa.org/hwa-byung. Accessed January 1, 2021.

71. Lee J, Wachholtz A, Choi KH. A review of the Korean cultural syndrome hwa-byung: suggestions for theory and intervention. J Asia Pac Couns 2014;4:49–64.
crossref pmid pmc
72. Mim SK. Treatment and prognosis of hwabyung. Psychiatry Investig 2004;1:29–36.

73. Yaseen ZS, Briggs J, Kopeykina I, Orchard KM, Silberlicht J, Bhingradia H, et al. Distinctive emotional responses of clinicians to suicide-attempting patients--a comparative study. BMC Psychiatry 2013;13:230
crossref pmid pmc
74. Borges G, Angst J, Nock MK, Ruscio AM, Kessler RC. Risk factors for the incidence and persistence of suicide-related outcomes: a 10-year follow-up study using the national comorbidity surveys. J Affect Disord 2008;105:25–33.
crossref pmid pmc
75. Fowler JC. Suicide risk assessment in clinical practice: pragmatic guidelines for imperfect assessments. Psychotherapy (Chic) 2012;49:81–90.
crossref pmid
76. Nock MK, Borges G, Bromet EJ, Alonso J, Angermeyer M, Beautrais A, et al. Cross-national prevalence and risk factors for suicidal ideation, plans and attempts. Br J Psychiatry 2008;192:98–105.
crossref pmid pmc
77. Czeisler MÉ, Lane RI, Petrosky E, Wiley JF, Christensen A, Njai R, et al. Mental health, substance use, and suicidal ideation during the COVID-19 pandemic—United States, June 24–30, 2020. MMWR Morb Mortal Wkly Rep 2020;69:1049–1057.
crossref pmid pmc