Psychiatry Investig Search

CLOSE


Psychiatry Investig > Volume 23(9); 2026 > Article
Kim, Kim, Kim, Jeon, Choi, Kim, Kim, Lee, Jhon, and Kim: Daily Language as an Objective Indicator of Depressive Affective States: A Two-Week Ecological Momentary Assessment Study in Emotional Labor Workers

Abstract

Objective

Workers performing emotional labor are at increased risk for depression, yet conventional self-report assessments often lack objectivity and temporal sensitivity. This study aimed to examine whether longitudinal analysis of daily natural language collected through ecological momentary assessment (EMA) can serve as an indicator of depressive affective states and to compare its temporal sensitivity with traditional self-report measures.

Methods

A total of 400 call center employees completed three voice-recorded free-text entries per day for two weeks using an EMA application. Transcriptions were analyzed using a lexicon-based sentiment approach (Linguistic Inquiry and Word Count [LIWC]) and three large language models (LLMs; GPT-4o-mini, Qwen, and Mistral) under zero-shot prompting. Depressive symptoms were assessed using the Patient Health Questionnaire-9, and neuroticism was measured using both self-reported questionnaires and language-inferred scores.

Results

LLM-derived sentiment scores significantly differentiated groups across levels of depressive symptom burden and consistently outperformed LIWC. Longitudinal analyses demonstrated clear group-level separation, particularly in morning entries, whereas self-reported mood ratings failed to distinguish groups and showed lower adherence over the two-week period. Language-inferred neuroticism exhibited stronger associations with depressive symptoms than self-reported neuroticism. A cumulative model based on morning sentiment scores showed progressively improved discrimination over time, reaching its highest performance on Day 10 (area under the curve=0.75).

Conclusion

Daily natural language captures meaningful longitudinal affective dynamics associated with depressive symptoms and may complement conventional self-report assessments as a low-burden and scalable indicator of depressive affective states in real-world occupational settings.

INTRODUCTION

Depression, a prevalent mental disorder, is projected to become the leading contributor to global disease burden by 2030 [1]. In particular, workers performing emotional labor face extreme stress and a high burden of mental health issues. A recent survey found that 26% of female call-center employees in Seoul had clinically significant depressive symptoms, exceeding the 9%-21% prevalence observed in the general Korean population [2]. Research on the detection and management of stress-related depression in occupations characterized by emotional labor, such as call center work, is becoming increasingly topical. Most previous studies on this population have relied on self-report surveys or interviews to identify mental health problems [2,3]. Survey-based assessments provide valuable snapshots of prevalence and risk factors; however, their cross-sectional designs are limited. Workers typically report their distress retrospectively, which can introduce recall bias and overlook day-to-day fluctuations. Recent efforts have therefore explored more objective and temporally sensitive approaches to capture stress and affective burden in real time.
Language use is widely recognized as a reflection of a person’s emotional state. Depressed individuals tend to use more negative emotion words and fewer positive ones when describing their lives [4-6]. For example, one study found that the emotional tone of brief written language responses predicted changes in depressive symptoms three weeks later, even after controlling for current mood and depression severity [5]. Similarly, Kaźmierczak et al. [7], who analyzed longer spoken narratives collected at three time points over a year, found an association between frequent use of negatively valenced words and more severe depression and anxiety symptoms. In addition, a recent ambulatory assessment study by Hartnagel et al. [8] highlighted the link between moment-to-moment language use and real-time affective states. By collecting brief spoken responses multiple times a day during a rapid mood-shifting intervention (sleep-deprivation therapy), this study showed that the use of positive and negative emotion words closely tracked concurrent depression severity. Similarly, Shin et al. [9] applied large language model (LLM) to user-generated diary texts collected over a two-week period and found that LLMderived sentiment features from free-text narratives could accurately detect depression severity.
Previous studies have employed a range of methods to analyze linguistic features associated with depression. Traditional lexicon-based approaches, such as Linguistic Inquiry and Word Count (LIWC), quantify the frequencies of psychologically relevant word categories (e.g., positive and negative emotion words and personal pronouns) and offer a rule-based framework for sentiment analysis [10]. However, these methods often fail to account for word order, negation, and contextual nuances. Recent advances in natural language processing, particularly in the use of LLMs, have enabled sentiment and symptom analyses that incorporate semantic depth and contextual meaning. LLM-based classification studies have generally focused on social media data, such as that from Twitter or Reddit, to identify depression-related language patterns [11,12]. For example, Bao et al. [11] applied GPT-3.5 in a few-shot setting on labeled datasets such as PsySym and BDI-Sen, achieving an F1 score of 0.91 in binary depression classification. Shin et al. [9] constructed a novel dataset by collecting user-generated emotional diary entries through a mobile app and tagging them with depression labels based on the Patient Health Questionnaire-9 (PHQ-9), Beck Scale for Suicide Ideation, and expert ratings. GPT-3.5 was fine-tuned and various prompt formats were tested, with the best F1 score of 0.685 achieved using chain-of-thought prompting [9]. In the clinical context, Ohse et al. [13] applied GPT-3.5 to transcribed interview text, classifying individual sentences into depression, anxiety, somatic, and insomnia symptom clusters. They achieved an F1 score of 0.82 for depression detection using PHQ-8 scores as the reference.
Neuroticism, a core personality dimension, is a well-established risk factor for major depressive disorder [14]. Individuals high in neuroticism tend to rely on maladaptive cognitive emotion-regulation strategies, which may impair their ability to effectively cope with stressful situations [15]. Neuroticism is closely associated with increased sensitivity to negative emotional stimuli [16]. This heightened affective reactivity is often reflected in language. For example, neurotic individuals have been found to use more negative emotion words [17]. Identifying linguistic markers of neuroticism in naturalistic data may enhance the predictive performance of language-based depression detection systems. Recent advances in natural language processing, particularly in LLM development, offer new opportunities for inferring personality-related traits from text. By leveraging the contextual and semantic understanding of LLMs, trait-level estimates of neuroticism could be derived directly from selfgenerated language samples.
While previous studies have demonstrated the utility of language-based approaches in mental-health assessments, many have relied on single-time-point or retrospective data and therefore lack temporal resolution. Moreover, recent studies using LLMs have focused on social media or aggregated diary data, thereby limiting their applicability in structured real-world settings. To address these limitations, this study implemented a high-frequency ecological momentary assessment (EMA) design. Prompted free-text responses and momentary mood ratings were collected three times daily over a two-week period from call-center workers. Sentiment scores were computed from each text using both LIWC- and LLM-based approaches. In addition, we evaluated whether neuroticism scores inferred from language using LLMs contributed to group differentiation indexed by depression severity. Using a repeated-measures approach, we examined whether temporal patterns in emotional language reflect differences in affective profiles indexed by depression severity and assessed the discriminative utility of these time-series sentiment features.

METHODS

Study design and participants

Participants were recruited from three call centers and one public agency. These workers performed emotional labor by providing civil consultations, customer service, sales, or administrative duties. Data were collected on-site at the participants’ workplaces between August 3, 2023 and January 25, 2024.
Figure 1A outlines the study design and participant flow. In total, 480 customer service agents participated. Participants completed in-person sociodemographic, psychiatric, and personality assessments through baseline surveys. They were asked to install an EMA app specifically developed for this study to facilitate real-time data collection. During a two-week period, participants completed three EMA sessions per day using the app, in which they reported their moods and stress levels and submitted voice recordings. Follow-up assessments, including psychiatric scales and in-person surveys, were conducted at the end of the two-week period. All participants provided written informed consent. The study was approved by the Institutional Review Boards (IRB) of CNUH and CNUHH (approval numbers: CNUH-2023-156 and CNUHH-2023-118).

EMA protocol

Figure 1B shows the structure and timing of the EMA protocol. Participants completed EMA tasks three times per day over the two-week study period. At 08:00 on each study day, participants rated their mood on a 10-point scale and recorded corresponding voice descriptions. At 13:00 and 18:00, they answered two stress-related questions and recorded voice descriptions of stressful experiences. The 18:00 session also included mood ratings, PHQ-2 items modified for daily assessment purposes within this study, and an anger-related item. All audio recordings were collected at 8 kHz.

Depressive symptoms and personality measures

Depression severity was measured using the PHQ-9 [18], a validated screening tool for depressive symptoms. The nine items are scored from 0-3, yielding a total score ranging from 0-27. Severity levels were categorized as follows: 0-4 (none), 5-9 (mild), 10-14 (moderate), 15-19 (moderately severe), and 20-27 (severe).
Personality traits were assessed using the Korean version of the Big Five Inventory-10 (BFI-K-10). This measure evaluates extraversion, agreeableness, conscientiousness, neuroti-cism, and openness, with two items for each trait rated on a 5-point Likert scale [19].

Voice transcription and text preprocessing

All voice recordings were transcribed using Vrew software [20]. To protect participants’ personal information, transcriptions were anonymized by trained annotators through the removal of identifying information (e.g., names and birthdates). Utterances by other individuals, as well as irrelevant content, were removed to ensure data quality. Text data were cleaned and standardized to enhance consistency, readability, and reliability for downstream linguistic and computational analyses. Korean spacing rules were applied, sentence boundaries were marked with periods, and grammatical and typographical errors were corrected using a spell-check tool. Nonsemantic expressions, such as filler words (“um,” “eh”), technical remarks (“Is it working?”), and nonverbal sounds (e.g., laughter), were excluded.

Final dataset

The initial dataset comprised 14,733 entries from 480 individuals. After excluding 467 entries because of unprocessable audio, 13 participants with missing PHQ-9 scores, and 67 participants whose PHQ-9 categories changed between baseline and follow-up, the final dataset included 12,418 entries from 400 participants (Supplementary Figure 1).

Analysis

Sentiment analysis approaches

Linguistic features were extracted using two complementary methods: lexical analysis using the LIWC and LLM based analysis. LIWC quantifies psychological constructs in texts based on word frequencies within predefined categories. This study used the KNU Korean Sentiment Lexicon [21], which includes general emotion words, emoticons, and abbreviations, with valence scores ranging from -2 to +2. This approach facilitated structured sentiment scoring based on established psycholinguistic theory. LLMs utilize transformer-based architectures capable of understanding contextual and semantic information in free text. This study selected Mistral Small (24 B; Mistral AI), Qwen 2.5 (32 B; Alibaba Cloud), and GPT-4o-mini (OpenAI) for LLM-based analysis in light of their performance on Korean language tasks and their instructionfollowing capabilities. Furthermore, this study aimed to incorporate a representative range of publicly available LLMs with diverse parameter scales, pre-training corpora, and training contexts. This selection was intended to mitigate potential model-specific biases and to observe varied response behaviors across different LLMs, rather than to identify a superior model (Supplementary Table 1). Zero-shot inference was performed to extract sentiment-relevant features directly from diary entries using these models, without the need for finetuning.

Sentiment and neuroticism scoring procedures

Using LIWC, the frequencies of positive and negative emotion words were calculated. In the LLM pipeline, zero-shot prompts were used to identify positive or negative emotional expressions (Supplementary Table 2), and sentiment scores were assigned on a -5 to +5 scale (Supplementary Table 3). To facilitate interpretation of the LLM-derived sentiment scores, representative diary excerpts with corresponding LLM-derived mood scores and self-rated mood scores are provided in Supplementary Table 4. These examples are included solely for illustrative purposes, highlighting cases in which numeric self-ratings appear less aligned with the emotional content of the text, and are not intended to imply ground-truth superiority of either measure.
To estimate neuroticism, each participant’s daily diary text was segmented by time of day and processed using LLMs. Scores were mapped to a scale of 1-5 for comparison with the BFI-K-10. The prompts used for this task are presented in Supplementary Table 5.

Statistical analysis of linguistic features

Several statistical approaches were used to examine the relationship between linguistic features and depression severity. First, t-tests and analysis of variance were applied to compare the frequencies of positive and negative words across depression severity groups. Second, group-level sentiment trajectories across different times of the day (morning, afternoon, and evening) were visualized to examine variations in temporal patterns in sentiment scores. Third, LLMs were used to generate neuroticism scores based on the linguistic content of participants’ diary entries, and logistic regression was applied to examine whether these scores could predict depression (PHQ-9 <5 vs. ≥5), adjusting for sex and education level.
To assess the psychological validity of LLM-derived features, two validation analyses were conducted. First, the consistency of LIWC- and LLM-derived sentiment scores with self-reported mood ratings was assessed by calculating their correlations with morning and evening entries. Separate calculations were performed as mood-related questions differed between the time periods. Second, the consistency of LLM-predicted neuroticism scores with BFI-K-10 scores was assessed by computing their correlations using both the original dataset and a version adjusted through imputation and normalization.

Modeling sentiment trajectories by depression severity

We constructed linear mixed-effects models using GPT-derived sentiment scores from Day 1-11, the period with sufficient participant coverage (n >200), to examine whether daily sentiment trajectories differed according to depression severity. This analysis collapsed the PHQ-9 depression severity classification into three categories (normal, mild, and moderate- to-severe) to enhance group-level contrasts and facilitate interpretation of trajectory differences. GPT-derived sentiment scores were selected for modeling because they demonstrated the most robust differentiation between depression severity groups in the preliminary analyses. Depression severity was modeled as a fixed effect, and a participant random intercept was included. The models were fitted across all time points as well as separately for each segment (morning, afternoon, and evening). Time was not included as a fixed or random effect.

Comparing model-derived sentiment and self-reported mood

Separate mixed-effects analyses were conducted to compare the respective abilities of LLM-derived sentiment scores and self-reported mood ratings to reflect depression severity. Only morning data were used because both self-reported mood and mood-related diary texts were available. Each score type was modeled independently to assess its ability to differentiate between the depression severity groups.

Cumulative sentiment-based approach for depression classification

A Long Short-Term Memory (LSTM) model was employed to evaluate whether morning sentiment scores can serve as distinctive features for depression classification. These scores were treated as sequential daily data, and the classification performance was assessed across cumulative monitoring durations from Days 1 to 11. A three-step preprocessing pipeline was applied to address missing data: participants with more than 50% missing values were excluded; gaps within the time series were filled using forward linear interpolation; and any remaining missing values were imputed using each participant’s average sentiment score. The data were then split into 80% training and 20% test sets using stratified sampling to preserve class distributions. Model performance was evaluated using area under the curve (AUC) scores.

RESULTS

Participant characteristics

Table 1 presents the participants’ demographic and psychological characteristics. Among the 400 call center employees analyzed, 73.2% (n=293) were female and 26.8% (n=107) were male. The participants had a mean age of 42.5 years (standard deviation=8.4). Smoking status (p=0.023), cohabitation status (p=0.006), and the four dimensions of the BFI-K-10, excluding openness (p=0.220), differed significantly between the normal and depressive groups. Notably, the depressive group had a higher proportion of smokers (29.6%) and individuals living alone (25.6%) than the normal group (19.3% and 14.2%, respectively). The depressive group also showed lower scores on extraversion, agreeableness, and conscientiousness, and higher scores on neuroticism. Regarding these group-level differences, an adjusted logistic regression confirmed that neuroticism was a significant independent risk factor for depression (odds ratio [OR]=1.105, p<0.001) (Supplementary Table 6).

Word frequency patterns by depression severity

Using LIWC and three LLMs (Mistral, Qwen, and GPT), the frequency of positive and negative words was analyzed across the depression groups, both as a binary variable (depressive group vs. normal group) and as a categorical variable based on PHQ-9 severity levels.
As shown in Table 2, individuals in the depressive group consistently used more negative words and fewer positive ones than those in the normal group. This pattern was most pronounced with GPT, which produced the most polarized distributions of emotional word usage. In contrast, LIWC showed weaker group differentiation with no significant difference in positive word frequency (p=0.294).
While the LLMs generally reflected a severity gradient across PHQ levels (i.e., higher severity correlated with more negative word use), this trend was most pronounced for GPT and Qwen but less evident for Mistral.

Temporal patterns in sentiment scores

We compared the sentiment scores generated by LIWC and LLMs across the time segments (morning, afternoon, and evening) and depression severity groups. Supplementary Figure 2 illustrates that sentiment scores were consistently lower in the depressive group than in the normal group, with LLMs showing greater separation between groups than LIWC. Among the LLMs, GPT demonstrated the clearest distinction between the depression severity groups. Temporal trends were also observed; in all groups, sentiment scores were highest in the morning and decreased in the evening. Notably, the LIWC sentiment scores remained positive throughout the day, whereas the LLM-based sentiment scores varied by group and time. Among the LLMs, GPT and Qwen produced negative sentiment scores for the depressive group, particularly in the morning, whereas Mistral showed consistently negative scores for both groups. These findings suggest that differences in sentiment according to depression severity are most detectable in the morning.

Neuroticism and depression severity: LLM-derived vs. self-reported measures

To determine whether language-derived neuroticism scores could predict depression, self-reported BFI-K-10 neuroticism scores were compared with scores predicted by LLMs based on two-week diary texts. All measures of neuroticism were significantly associated with depression. Notably, most LLMderived scores yielded larger ORs than self-reported measures, indicating stronger associations with depression. GPT produced the highest OR (1.193) among the LLMs, followed closely by Mistral (OR=1.191) and Qwen (OR=1.069). In contrast, the self-reported neuroticism score showed a slightly lower but still significant association (OR=1.119, p<0.001) (Supplementary Table 7).

Convergent validity of sentiment and personality features

Convergent validity was assessed by examining the correlations between LIWC- and LLM-derived sentiment scores and self-reported mood ratings using morning and evening entries. As shown in Supplementary Table 8, the correlations were generally low, with the strongest correlation observed between GPT sentiment and evening mood (r=0.535, p< 0.0001). The morning correlations were weaker across all LLMs. Furthermore, the LIWC sentiment scores showed minimal associations at both time points. We then assessed the correspondence between the self-reported and LLM-predicted neuroticism scores (Supplementary Table 9). The correlation coefficients were moderately low across the models. Once again, GPT showed the highest alignment after adjustment (r=0.264).

Sentiment score trajectory and depression group classification

The GPT-derived sentiment scores differed significantly across depression severity groups over the 11-day EMA period (Figure 2). The number of participant entries varied across days because of missing data (n=235-330 per day). The normal group had consistently higher sentiment scores than the mild and moderate-to-severe groups. Supplementary Figure 3 illustrates the sentiment trajectories in the morning, afternoon, and evening. Across all time segments, the Normal group had the highest sentiment scores, whereas the moderate-to-severe group scored the lowest. The group separation was most pronounced in the morning.
Consistent with these visual patterns, mixed-effects model results confirmed significant between-group differences across all time points (Normal vs. Mild: β=-0.516, p<0.001; Mild vs. Moderate-to-Severe: β=-0.736, p<0.001). Stratified analyses revealed that the morning sentiment scores provided the greatest discriminatory power (Normal vs. Mild: β=-0.551, p=0.005; Mild vs. Moderate-to-Severe: β=-0.933, p<0.001). The afternoon and evening scores also demonstrated smaller but significant effects (Table 3).

Comparing LLM-derived sentiment and self-reported mood across depression severity levels

Supplementary Figure 4 shows the self-reported morning mood score trajectories over the 11-day study period. Although the Normal group generally reported a higher mood than the Moderate-to-Severe group, there was substantial overlap with no consistent separation of groups over time. While the GPTderived sentiment scores identified significant pairwise group differences, the self-reported mood scores did not have comparable discriminatory power. The differences between the normal and mild groups (β=-0.117, p=0.494) and the mild and moderate-to-severe groups (β=-0.133, p=0.5080) were not significant (Table 3).
Notably, on average, fewer participants (mean, 256 vs. 289 per day) provided daily self-reported mood scores than voicebased sentiment responses, potentially reflecting lower adherence to numeric self-reporting.

Prediction performance of cumulative morning sentiment scores across monitoring durations

Figure 3 shows the cumulative prediction performance of the LSTM model using morning sentiment scores for up to 11 days. The AUC scores declined until Day 6 and then increased from Day 7 onward. The LSTM model achieved its highest performance on Day 10 with an AUC score of 75.1% (71.5% on Day 11). Confusion matrices are presented in Supplementary Figure 5.

DISCUSSION

This study evaluated whether LLMs, which incorporate contextual understanding, can extract sentiments and personality traits from daily free-text entries to characterize longitudinal affective patterns associated with depression severity. Compared with traditional lexicon-based approaches, LLM-derived features demonstrated superior discriminative power in differentiating affective profiles across levels of depression severity. By conducting a longitudinal analysis of two weeks of EMA data, we found that LLM-derived sentiment scores showed clearer separation between depression severity groups than self-reported mood scores. The cumulative data yielded a discrimination performance exceeding 70%, demonstrating the feasibility of this approach. These findings highlight the potential utility of LLMs for deriving objective, language-based indicators of longitudinal affective burden in real-world occupational settings. Further methodological enhancements, such as model fine-tuning and personalization, may extend this approach to support scalable monitoring of affective states in high-stress occupational groups.
The LLM-based approach outperformed LIWC in both word-frequency analysis and sentiment scoring. This may be attributed to LLMs’ superior ability to capture contextual nuances within diary entries [22]. This contextual sensitivity may also have enhanced the estimation of personality traits. The LLM-predicted neuroticism scores demonstrated stronger associations with depression severity than the self-reported BFI scores. These findings suggest that LLMs may complement conventional self-reporting instruments when analyzing naturalistic language data. Notably, we observed LLM-specific differences in emotional sensitivity, temporal trends, and alignment with psychological constructs such as depression and neuroticism. The study did not aim to compare model performance directly, owing to substantial differences in their training data, fine-tuning scope, and parameter sizes. Nevertheless, differences were observed in how each model captured affective patterns. GPT demonstrated the most distinct separation between groups with differing levels of depressive symptoms, whereas Mistral showed flattened affective distinctions and Qwen showed intermediate performance. Given its relatively stronger discriminative capacity across key outcomes, subsequent analyses to further explore model-driven insights were conducted using GPT alone.
The LLM-derived sentiment trajectories revealed clear distinctions among groups defined by depression severity across the two-week EMA period. The value of a time-series perspective was both visually and statistically evident. When examined cross-sectionally, the mild group frequently overlapped with either the normal or moderate-to-severe groups. When considering a single time point, accurate differentiation between depression severity groups was difficult. By contrast, sentiment trajectories over time showed more consistent and separable patterns across groups. This supports prior research using speech and self-reported mood scores, which has emphasized the advantages of temporal modeling. For instance, daily monitoring of speech or mood captures subtle changes in depressive symptoms more effectively than infrequent as-sessments, as demonstrated by Gumus et al. [23] and Horwitz et al. [24] Furthermore, van de Leemput et al. [25] showed that early warning patterns, such as increasing mood instability, frequently precede depressive-episode onset and that these patterns can only be detected through a time-series approach. Shin et al. [9] also showed that LLM-based linguistic analysis of aggregated diary texts can reflect depression severity. Our study extends this work by demonstrating how longitudinal daily language trajectories enhance group-level differentiation when analyzed within a time-series framework.
In the stratified analysis, the data collected during the morning session yielded the most pronounced group differences in the LLM-derived sentiment scores. This likely reflects the EMA structure in which morning questions explicitly ask about the respondent’s current mood, whereas afternoon and evening questions focus on stress-related events. These findings suggest that the question content directly influences the strength of emotional signals captured by LLMs, with moodrelated questions providing more diagnostic information. Question design and timing shape the data quality, as has been emphasized in prior research [26]. This effect is clearly demonstrated by the stronger sentiment-based group separation observed in the morning responses.
Despite the low correlations between LLM-derived sentiment scores and self-rated mood, as well as between LLMpredicted and self-reported neuroticism, both LLM approaches showed comparable or superior discriminative capacities in distinguishing between depression severity groups. This discrepancy may reflect limitations of subjective self-reporting, particularly among individuals with reduced emotional awareness or introspective accuracy. This phenomenon is also commonly observed in depression [27]. Self-reporting can additionally be affected by recall bias, situational mood fluctuations, and social desirability, especially in occupational settings with emotional display rules [28,29]. In contrast, languagebased approaches may capture latent affective patterns that are less accessible to conscious self-report. At the same time, LLM-derived signals may reflect linguistic or cultural patterns embedded in natural language rather than affective states per se, and the possibility of model hallucinations cannot be fully excluded. Future studies should validate these findings across diverse linguistic and cultural contexts. Taken together, these findings indicate that language-derived affective indicators may serve as a complementary signal to traditional self-report measures, although broader cross-linguistic and cross-cultural validation will be required.
We observed a discrepancy between the number of completed morning voice recordings and mood ratings. The number of completed morning voice recordings per day ranged from 196-328, whereas the corresponding mood ratings ranged from 191-318, indicating that the participants completed the speech task more frequently than the self-rating task. Direct mood reporting may therefore be perceived as effortful or intrusive, particularly when repeated over time. This may reflect a general tendency to avoid repetitive or high-effort self-reporting tasks. These tendencies may be amplified in individuals with depressive symptoms who are more likely to experience reduced motivation and fatigue [30]. Unlike self-reporting, LLM-derived sentiment scores are passively extracted from transcribed speech, allowing more complete and consistent data to be captured with minimal participant effort.
Cumulative data yielded an AUC of 0.75 and a macro F1 of 0.683 for depression classification by Day 10. This performance may be considered relatively modest. A recent study that applied chain-of-thought prompting to two weeks of diary text from 91 participants to predict depression achieved a comparable F1 score of 0.685 and a balanced accuracy of 0.799 using a fine-tuned GPT-3.5 model [9]. In another EMA-based study, Lei et al. [31] analyzed 25 days of twice-daily mood and stress ratings in 103 participants to predict suicidal ideation. Their AUC remained below 0.7 until Day 11 and only exceeded 0.8 after the full 25 days were included. Although neither of these studies incorporated time-series modeling, we achieved a comparable performance using a single feature type without fine-tuning. Furthermore, the marked improvement observed after Day 8 suggests that a minimum duration of approximately one week is required for group-level differentiation. This is consistent with a prior finding that at least eight days of smartphone and wearable data are needed to achieve stable prediction of depression severity [32]. Given the larger sample size and use of directly collected natural-language data in our design, there remains potential to further improve classification performance through future model optimization.
This study has several limitations. First, relatively simple prompting strategies were used, and further validation of LLMderived sentiment and neuroticism scores against established psychometric instruments is needed. Second, the study population consisted of workers in high-stress occupational environments, which may limit the generalizability of the findings to other populations. Third, although transcription and preprocessing procedures included anonymization, naturalistic language data may still contain contextual details that could be recognizable to the participants themselves or to individuals familiar with them, reflecting an inherent limitation of real-world data. Emerging on-device language models may help by enabling local processing of sensitive data and reducing manual review, thereby supporting scalable deployment. Finally, while LLM-derived features demonstrated meaningful associations with depression severity, further work is needed to evaluate their robustness across different linguistic, cultural, and modeling contexts. As vision-language models continue to advance, this framework may be extended to incorporate behavioral observations, enabling a more comprehensive multimodal assessment of affective states.
The LLM-derived sentiment and personality-related features in this study carry important clinical implications, as they enable the inference of affective patterns without directly querying individuals. This approach may facilitate more efficient and low-burden monitoring in real-world settings, particularly in contexts where repeated self-reporting is challenging. Furthermore, the ability to leverage large volumes of naturally occurring language data in daily life suggests a pathway toward scalable mental health assessment and may contribute to the development of personalized and precision psychiatry. Future work could strengthen and extend these findings through validation against clinician-administered assessments, fine-tuning to enhance sensitivity, and integration with behavioral or physiological signals for multimodal assessment across diverse populations. In this study, LLMs successfully extracted affective signals from daily free-text entries and differentiated patterns associated with depression severity more effectively than self-reported measures, with performance improving over time. To our knowledge, this is one of the first large-scale studies to apply LLMs to time-series free-text data, highlighting their potential to advance workplace mental health assessment and population-level monitoring.

Supplementary Materials

The Supplement is available with this article at https://doi.org/10.30773/pi.2026.0035.
Supplementary Table 1.
Large language model specifications used for sentiment analysis
pi-2026-0035-Supplementary-Table-1.pdf
Supplementary Table 2.
Large language model prompts for extracting positive and negative word counts from daily texts
pi-2026-0035-Supplementary-Table-2.pdf
Supplementary Table 3.
Large language model prompts for generating context-aware sentiment scores
pi-2026-0035-Supplementary-Table-3.pdf
Supplementary Table 4.
Representative diary text excerpts with LLM-derived mood scores and self-rated mood scores
pi-2026-0035-Supplementary-Table-4.pdf
Supplementary Table 5.
Large language model prompt for estimating BFI neuroticism scores from diary texts
pi-2026-0035-Supplementary-Table-5.pdf
Supplementary Table 6.
Adjusted logistic regression analysis to predict depression
pi-2026-0035-Supplementary-Table-6.pdf
Supplementary Table 7.
Comparing self-reported and large language model-based BFI neuroticism in predicting depression: adjusted logistic regression results
pi-2026-0035-Supplementary-Table-7.pdf
Supplementary Table 8.
Correlations between self-rated mood and sentiment scores derived by LIWC and large language models in the morning and evening
pi-2026-0035-Supplementary-Table-8.pdf
Supplementary Table 9.
Correlation between self-report and large language model-predicted Big Five Inventory neuroticism scores
pi-2026-0035-Supplementary-Table-9.pdf
Supplementary Figure 1.
Flowchart of data inclusion and final sample. A total of 14,733 records were collected from 480 individuals. After excluding 467 records that could not be processed, 14,266 remained. The records were further screened based on the availability of Patient Health Questionnaire-9 (PHQ-9) scores, excluding 13 individuals without PHQ-9 data and 67 individuals with extreme differences in PHQ-9 scores over time. The final sample included 400 individuals, and 12,418 records were analyzed.
pi-2026-0035-Supplementary-Fig-1.pdf
Supplementary Figure 2.
Sentiment differences by time of day, estimated by lexicon-based and large language model-based approaches. Mean sentiment scores are plotted by time of day (morning, afternoon, evening) and depression severity group using four approaches: (A) Linguistic Inquiry and Word Count, (B) Mistral, (C) Qwen, and (D) GPT. The top row presents binary group comparisons (Normal vs. Depression), and the bottom row presents depression severity group comparisons (normal, mild, moderate, moderately severe, severe; according to Patient Health Questionnaire-9 scores).
pi-2026-0035-Supplementary-Fig-2.pdf
Supplementary Figure 3.
Daily sentiment trajectories by depression severity group for the (A) morning, (B) afternoon, and (C) evening. Panels (A-C) show the sentiment scores for the morning, afternoon, and evening sessions, respectively. These scores were derived from the large language model-based language analysis of the daily ecological momentary assessment voice responses. Lines represent group means and shaded areas indicate 95% confidence intervals around the means. The groups are color-coded as follows: normal (red), mild (blue), and moderate-to-severe (orange).
pi-2026-0035-Supplementary-Fig-3.pdf
Supplementary Figure 4.
Daily morning mood trajectories by depression severity. Daily mood scores are self-reported in the morning. Lines represent group means and shaded areas indicate 95% confidence intervals around the means. The groups are color-coded as follows: normal (red), mild (blue), and moderate-to-severe (orange).
pi-2026-0035-Supplementary-Fig-4.pdf
Supplementary Figure 5.
Confusion matrices for Long Short-Term Memory (LSTM) binary classification: (A) 10-day and (B) 11-day monitoring periods. (A) presents the LSTM classification results for the test set trained with 10 days of temporal data. (B) shows the classification results using 11 days of temporal data.
pi-2026-0035-Supplementary-Fig-5.pdf

Notes

Availability of Data and Material

The data supporting the findings of this study are not publicly available due to the sensitive nature of the information, including identifiable language data and mental health-related assessments. Access to the data may be considered upon reasonable request to the corresponding author, subject to institutional review board approval.

Conflicts of Interest

Jae-Min Kim and Sung-Wan Kim, contributing editors of the Psychiatry Investigation, were not involved in the editorial evaluation or decision to publish this article. All remaining authors have declared no conflicts of interest.

Author Contributions

Conceptualization: Hye Hyeon Kim, Min Jhon, Ju-Wan Kim. Data curation: Hye Hyeon Kim, Min Jhon, Ju-Wan Kim. Formal analysis: Hye Hyeon Kim, Seok Jun Kim, Min Jhon. Funding acquisition: Hye Hyeon Kim. Investigation: Hye Hyeon Kim. Methodology: Hye Hyeon Kim, Min Jhon. Project administration: Suehyun Lee, Min Jhon, Ju-Wan Kim. Supervision: Min Jhon. Validation: Eun-Kyoung Jeon, Yoonjoo Choi, Jae-Min Kim, Sung-Wan Kim. Writing—original draft: Hye Hyeon Kim, Seok Jun Kim, Dae-Kwang Kim, Min Jhon. Writing—review & editing: Hye Hyeon Kim, Seok Jun Kim, Sung-Wan Kim, Suehyun Lee, Min Jhon, Ju-Wan Kim.

Funding Statement

This study was supported by grants from the Chonnam National University Hwasun Hospital Institute for Biomedical Science (HCRI25031) to Min Jhon and the National Research Foundation of Korea (NRF), funded by the Korean Government (MSIT) (RS-2024-00459226) to Hye Hyeon Kim.

Acknowledgments

The authors are grateful to the staff of the CNUH and CNUHH psychiatry research teams for their assistance in collecting patient data, and to Dr. Kim Jae Chang and Dr. Kim Sun Hee for providing the facilities for data analysis and collection.

Figure 1.
Study design and timeline of data collection. A: Overview of the study procedure. A total of 480 customer service agents completed baseline assessments and participated in a two-week ecological momentary assessment (EMA) using mobile app. Follow-up surveys were conducted in person. B: EMA protocol. Participants completed three app-based assessments each day at 08:00, 13:00, and 18:00. Each session included a brief survey and a voice recording. Morning sessions focused on mood, while afternoon and evening sessions assessed stress levels and difficult events.
pi-2026-0035f1.jpg
Figure 2.
Daily sentiment trajectories by depression severity group. Mean large language model-derived sentiment scores are plotted across the 33 sequential ecological momentary assessment sessions (three sessions per day over 11 days). Participants were grouped into three depression severity categories based on their Patient Health Questionnaire-9 scores: normal, mild, and moderate-to-severe. The number of participants contributing data varied over time due to missing entries (N=235-330 per time point). Shaded regions indicate 95% confidence intervals around mean scores.
pi-2026-0035f2.jpg
Figure 3.
Long Short-Term Memory (LSTM) model binary classification area under the curve (AUC) performance across monitoring durations. The AUC scores for binary classification (Normal vs. Depression) are plotted using the LSTM model. Each time point (Days 1-11) represents the length of cumulative monitoring data used as model input. For instance, “Day 8” indicates that the first eight days of morning sentiment scores were used for prediction. All AUC scores represent the model’s performance on the 20% test set.
pi-2026-0035f3.jpg
Table 1.
Demographic and clinical characteristics of participants by depression group
Total (N=400) Normal (N=197) Depressive (N=203) p
Sex 0.787
 Male 107 (26.8) 51 (25.9) 56 (27.6)
 Female 293 (73.2) 146 (74.1) 147 (72.4)
Age (yr) 42.5±8.4 43.1±9.4 41.9±7.5 0.186
Drinking >0.999
 No 145 (36.3) 71 (36.0) 74 (36.5)
 Yes 255 (63.8) 126 (64.0) 129 (63.5)
Smoking 0.023
 No 302 (75.5) 159 (80.7) 143 (70.4)
 Yes 98 (24.5) 38 (19.3) 60 (29.6)
Education 0.172
 High school 127 (31.8) 67 (34.0) 60 (29.6)
 College 81 (20.3) 31 (15.7) 50 (24.6)
 University 183 (45.8) 94 (47.7) 89 (43.8)
 Graduate school 9 (2.3) 5 (2.5) 4 (2.0)
Religion 0.829
 No 263 (65.8) 128 (65.0) 135 (66.5)
 Yes 137 (34.3) 69 (35.0) 68 (33.5)
Cohabitation 0.006
 Alone 80 (20.0) 28 (14.2) 52 (25.6)
 With family 320 (80.0) 169 (85.8) 151 (74.4)
PHQ severity <0.001
 Normal 197 (49.3) 197 (100) 0 (0)
 Mild 98 (24.5) 0 (0) 98 (48.3)
 Moderate 72 (18.0) 0 (0) 72 (35.5)
 Moderately severe 24 (6.0) 0 (0) 24 (11.8)
 Severe 9 (2.3) 0 (0) 9 (4.4)
Mood 5.0±1.3 5.4±1.4 4.6±1.1 <0.001
BFI
 Extraversion 6.0±1.8 6.4±1.7 5.7±1.8 <0.001
 Agreeableness 7.3±1.3 7.6±1.2 7.0±1.4 <0.001
 Conscientiousness 7.2±1.5 7.5±1.5 6.8±1.5 <0.001
 Neuroticism 5.7±1.8 5.0±1.6 6.5±1.7 <0.001
 Openness 6.6±1.8 6.8±1.8 6.5±1.8 0.220

Values are presented as mean±standard deviation. PHQ, Patient Health Questionnaire; BFI, Big Five Inventory.

Table 2.
Comparison of sentiment scores across depression groups using LIWC and large language models
Normal group vs. Depressive group
Average frequency of emotion words by depression severity group
Normal Depressive p Normal Mild Moderate Moderately severe Severe p
LIWC
 Positive 0.9±0.5 0.8±0.4 0.294 0.9±0.5 0.9±0.4 0.7±0.4 0.9±0.5 0.9±0.4 0.237
 Negative 1.9±0.8 2.1±0.9 0.041 1.9±0.8 1.9±0.7 2.1±0.7 2.5±1.4 2.5±1.1 0.002
Mistral
 Positive 1.6±1.2 1.2±0.6 <0.001 1.6±1.2 1.3±0.6 1.1±0.5 1.0±0.5 1.0±0.4 <0.001
 Negative 0.4±0.4 0.6±1.0 0.006 0.4±0.4 0.6±0.7 0.6±0.9 1.0±1.8 0.4±0.3 0.008
Qwen
 Positive 2.6±1.1 2.1±0.9 <0.001 2.6±1.1 2.3±0.9 2.0±0.8 1.7±0.7 2.0±1.2 <0.001
 Negative 3.3±1.3 4.3±1.8 <0.001 3.3±1.3 3.9±1.3 4.3±1.4 5.5±3.0 5.8±2.2 <0.001
GPT
 Positive 4.0±2.0 3.3±1.5 <0.001 4.0±2.0 3.6±1.4 3.0±1.3 2.8±1.3 3.4±2.6 <0.001
 Negative 3.4±1.5 4.4±1.8 <0.001 3.4±1.5 4.0±1.5 4.4±1.4 5.5±2.7 5.7±2.1 <0.001

Values are presented as mean±standard deviation. LIWC, Linguistic Inquiry and Word Count.

Table 3.
Mixed-effects model results for depression group classification using GPT-derived sentiment scores and self-reported mood scores
Time of day Group comparison Coefficient p
GPT sentiment score
 All Normal vs. Mild -0.516 <0.001
Mild vs. Moderate-to-Severe -0.736 <0.001
 Morning Normal vs. Mild -0.551 0.005
Mild vs. Moderate-to-Severe -0.933 <0.001
 Afternoon Normal vs. Mild -0.492 0.004
Mild vs. Moderate-to-Severe -0.647 0.001
 Evening Normal vs. Mild -0.535 0.003
Mild vs. Moderate-to-Severe -0.577 0.006
Self-reported mood score
 Morning Normal vs. Mild -0.117 0.494
Mild vs. Moderate-to-Severe -0.133 0.508

REFERENCES

1. Malhi GS, Mann JJ. Depression. Lancet 2018;392:2299-2312.
crossref pmid pmc
2. Cho SS, Kim H, Lee J, Lim S, Jeong WC. Combined exposure of emotional labor and job insecurity on depressive symptoms among female call-center workers: a cross-sectional study. Medicine (Baltimore) 2019;98:e14894
pmid pmc
3. Jeyapal DR, Bhasin SK, Kannan AT, Bhatia MS. Stress, anxiety, and depression among call handlers employed in international call centers in the national capital region of Delhi. Indian J Public Health 2015;59:95-101.
crossref pmid
4. Behdarvandirad S, Karami H. Depression, neuroticism, extraversion and pronoun use in first and foreign languages following mood induction. Lang Sci 2022;94:101503
crossref
5. Hur JK, Heffner J, Feng GW, Joormann J, Rutledge RB. Language sentiment predicts changes in depressive symptoms. Proc Natl Acad Sci U S A 2024;121:e2321321121
crossref pmid pmc
6. Weintraub MJ, Posta F, Ichinose MC, Arevian AC, Miklowitz DJ. Word usage in spontaneous speech as a predictor of depressive symptoms among youth at high risk for mood disorders. J Affect Disord 2023;323:675-678.
crossref pmid pmc
7. Kaźmierczak I, Jakubowska A, Pietraszkiewicz A, Zajenkowska A, Lacko D, Wawer A, et al. Natural language sentiment as an indicator of depression and anxiety symptoms: a longitudinal mixed methods study. Cogn Emot 2025;39:1693-1702.
crossref pmid
8. Hartnagel LM, Ebner-Priemer UW, Foo JC, Streit F, Witt SH, Frank J, et al. Linguistic style as a digital marker for depression severity: an ambulatory assessment pilot study in patients with depressive disorder undergoing sleep deprivation therapy. Acta Psychiatr Scand 2025;151:348-357.
crossref pmid pmc
9. Shin D, Kim H, Lee S, Cho Y, Jung W. Using large language models to detect depression from user-generated diary text data as a novel approach in digital mental health screening: instrument validation study. J Med Internet Res 2024;26:e54617
crossref pmid pmc
10. Tausczik YR, Pennebaker JW. The psychological meaning of words: LIWC and computerized text analysis methods. J Lang Soc Psychol 2010;29:24-54.
crossref pdf
11. Bao E, Pérez A, Parapar J. Explainable depression symptom detection in social media. Health Inf Sci Syst 2024;12:47
crossref pmid pmc pdf
12. Farruque N, Goebel R, Sivapalan S, Zaïane OR. Depression symptoms modelling from social media text: an LLM driven semi-supervised learning approach. Lang Resour Eval 2024;58:1013-1041.
crossref pdf
13. Ohse J, Hadžić B, Mohammed P, Peperkorn N, Danner M, Yorita A, et al. Zero-shot strike: testing the generalisation capabilities of out-ofthe-box LLM models for depression detection. Comput Speech Lang 2024;88:101663
crossref
14. Xia J, He Q, Li Y, Xie D, Zhu S, Chen J, et al. The relationship between neuroticism, major depressive disorder and comorbid disorders in Chinese women. J Affect Disord 2011;135:100-105.
crossref pmid pmc
15. Liu C, Chen L, Chen S. Influence of neuroticism on depressive symptoms among Chinese adolescents: the mediation effects of cognitive emotion regulation strategies. Front Psychiatry 2020;11:420
crossref pmid pmc
16. Klamer S, Schwarz L, Krüger O, Koch K, Erb M, Scheffler K, et al. Association between neuroticism and emotional face processing. Sci Rep 2017;7:17669
crossref pmid pmc pdf
17. Ireland ME, Mehl MR. Natural language use as a marker of personality. In: Holtgraves TM, editor. The Oxford handbook of language and social psychology. Oxford: Oxford University Press, 2014, p. 201-218.

18. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001;16:606-613.
crossref pmid pmc
19. Kim SY, Kim JM, Yoo JA, Bae KY, Kim SW, Yang SJ, et al. [Standardization and validation of big five inventory-Korean version (BFI-K) in elders]. Korean J Biol Psychiatry 2010;17:15-25. Korean.

20. Vrew. Vrew - Edit videos like a document 2024 [Internet]. Available at: https://vrew.ai/en/. Accessed August 18, 2026.

21. Park SM, Na CW, Choi MS, Lee DH, On BW. [KNU Korean sentiment lexicon: Bi-LSTM-based method for building a Korean sentiment lexicon]. J Intell Inform Syst 2018;24:219-240. Korean.

22. Lossio-Ventura JA, Weger R, Lee AY, Guinee EP, Chung J, Atlas L, et al. A comparison of ChatGPT and fine-tuned open pre-trained transformers (OPT) against widely used sentiment analysis tools: sentiment analysis of COVID-19 survey data. JMIR Ment Health 2024;11:e50150
crossref pmid pmc
23. Gumus M, DeSouza DD, Xu M, Fidalgo C, Simpson W, Robin J. Evaluating the utility of daily speech assessments for monitoring depression symptoms. Digit Health 2023;9:20552076231180523
crossref pmid pmc pdf
24. Horwitz A, Czyz E, Al-Dajani N, Dempsey W, Zhao Z, Nahum-Shani I, et al. Utilizing daily mood diaries and wearable sensor data to predict depression and suicidal ideation among medical interns. J Affect Disord 2022;313:1-7.
crossref pmid pmc
25. van de Leemput IA, Wichers M, Cramer AO, Borsboom D, Tuerlinckx F, Kuppens P, et al. Critical slowing down as early warning for the onset and termination of depression. Proc Natl Acad Sci U S A 2014;111:87-92.
crossref pmid pmc
26. Ben-Zeev D, Young MA, Madsen JW. Retrospective recall of affect in clinically depressed individuals and controls. Cogn Emot 2009;23:1021-1040.
crossref
27. DaSilveira A, DeSouza ML, Gomes WB. Self-consciousness concept and assessment in self-report measures. Front Psychol 2015;6:930
crossref pmid pmc
28. Horwitz AG, Zhao Z, Sen S. Peak-end bias in retrospective recall of depressive symptoms on the PHQ-9. Psychol Assess 2023;35:378-381.
crossref pmid pmc
29. Ebner-Priemer UW, Trull TJ. Ecological momentary assessment of mood disorders and mood dysregulation. Psychol Assess 2009;21:463-475.
crossref pmid
30. Treadway MT, Zald DH. Reconsidering anhedonia in depression: lessons from translational neuroscience. Neurosci Biobehav Rev 2011;35:537-555.
crossref pmid pmc
31. Lei C, Qu D, Liu K, Chen R. Ecological momentary assessment and machine learning for predicting suicidal ideation among sexual and gender minority individuals. JAMA Netw Open 2023;6:e2333164
crossref pmid pmc
32. Sun S, Folarin AA, Zhang Y, Cummins N, Garcia-Dias R, Stewart C, et al. Challenges in using mHealth data from smartphones and wearable devices to predict depression symptom severity: retrospective analysis. J Med Internet Res 2023;25:e45233
crossref pmid pmc


ABOUT
AUTHOR INFORMATION
ARTICLE CATEGORY

Browse all articles >

BROWSE ARTICLES
Editorial Office
#522, G-five Central Plaza, 27 Seochojungang-ro 24-gil, Seocho-gu, Seoul 06601, Korea
Tel: +82-2-537-6171  Fax: +82-2-537-6174    E-mail: psychiatryinvest@gmail.com                

Copyright © 2026 by Korean Neuropsychiatric Association.

Developed in M2PI

Close layer
prev next