In total, 20 patients (mean age 38.4 [SD=14.2]) with a depressive disorder according to ICD-10 and 20 healthy subjects matched by age and sex (mean age 38.9 [SD=15.3]) were examined in this pilot study. Their sociodemographic and clinical characteristics are presented in Table 1.

Male and female healthy subjects without psychiatric or serious somatic illness and without positive family history of mental illness were recruited for the healthy control sample. The further inclusion criteria were a verbal IQ >70, sufficient German language skills, and an ability to participate in and to consent for the study according to Helsinki and ICH-GCP requirements. The exclusion criteria of the study were severe somatic diseases or mental illnesses (including organ-brain related mental illnesses, and ethanol or drug addiction), acute suicidal or extraneous endangering behavior, and lack of consent to participate in the study. A detailed anamnesis was taken from all depressive patients and healthy volunteers in a semi-structured interview (duration 45-60 minutes). The psychometric characteristics was gathered using various questionnaires. In addition, a PC-based emotion recognition task was used, which was based on a semi-structured paradigm. In accordance with the participants, a sequence of interaction behavior was recorded.

To measure the general severity of the patients’ mental illness, the Clinical Impression Score (CGI) [29] was used. The Personal and Social Performance Scale (PSP) is a valid, reliable, and standardized instrument for measuring the psychosocial functional level [30,31]. To evaluate the scale, a PSP total value is determined; for example, the range 71-100 reflects “no or only slight difficulties.” The Beck Depression Inventory (BDI-II) [32,33] and the Hamilton Depression Scale (HAM-D) [34] were used to assess depressive symptomology. The State-Trait Anxiety Inventory (STAI-I/II) [35,36] was used to assess the severity of trait and state anxiety. The NEO-FFI according to Costa and McCrae (1989, German translation by Borkenau and Ostendorf 1993, revised 2008) [37] was used to measure the following five different personality traits: neuroticism, extraversion, openness to experience, agreeableness, and conscientiousness.

Possible impairments of emotion processing were measured using the Alexithymia-Toronto Scale (TAS-20) [38-40], which measures difficulties in identifying (TAS-1) and describing (TAS-2) feelings, and also externally oriented thinking (TAS-3). The German version of the empathy inventory (IRI) [41] is the Saarbrücken Personality Questionnaire (SPF) [42], which was used to investigate empathy and its four dimensions (i.e., perspective adoption [PT], fantasy [FS], empathic concern [EC] and personal distress [PD]) as an expression of an emotion regulation disorder. According to Paulus [43], an overall score (empathy score) can be calculated from the summation of the dimensions PT, EC, and FS as an expression of general empathy.

Finally, the German version of the Positive and Negative Affect Schedule (PANAS) [44] was used to assess positive and negative affectivity as an expression of the current emotional state. The multiple-choice vocabulary intelligence test (Mehrfach-Wortschatztest=MWT-B) [45] was used to determine verbal intelligence.

In order to study the ability to emotion recognition in relation to interactive behavior, such a test was performed on a PC using the Presentation^® software (version 17.2, Neurobehavioural Systems Inc., Albany, CA, USA). For this purpose, the participants were instructed to sit on a chair in front of a monitor at distance of approximately 60 cm. At the beginning of the test, a short briefing and an introduction at the PC took place. Afterwards, the participant first completed a test run and was able to familiarize themselves with the system and the procedures. The initial test procedure could then begin: a total of 60 facial expressions were presented for the six basic emotions as defined by Ekman and Friesen (anger, disgust, joy, sadness, fear, and surprise; 10 each) [46] and 10 images with a neutral facial expression were shown for 500 ms each. Half of the photos showed female persons and the other half male adult persons from the NimStim Set of Facial Expressions (http://www.macbrain.org/resources.htm).

After an inter-stimulus interval of 1,000 ms (white fixation cross on black background), the possible responses (Figure 1) were displayed for 8 seconds. During these 8 seconds, the subject had time to enter their response using the number pad on a conventional keyboard. The numbers on the number pad were labelled with the initial letters of the emotions. The number 5 in the middle of the pad was marked with a cross, as this was the starting point for the respondent’s input finger. If the input was made before or after the 8 seconds, then the answer was evaluated but the reaction time was not recorded. After 8 seconds or after the input, the next trial started after an inter-trial interval of 500-1,200 ms. After approximately 10 minutes and a total of 70 trials, the test was completed, and the subjects were asked to submit an evaluation of the test. The participants were able to rate the perceived difficulty of the test on a scale of 1 (not difficult at all) to 10 (very difficult). For further data evaluation, the number of correctly recognized emotion expressions and the corresponding reaction time were used.

Two patients and two healthy subjects respectively were videographed in pairs frontally, while a movie with partly humorous but also sad content is shown first (scenes from the Charlie Chaplin film “The Kid,” 1921). Watching this movie, participants were recorded with three cameras, which are installed in the videography laboratory (Figure 2) at the Research Department for Experimental Psychopathology of the LWL-University Hospital Bochum. While the participants were watching the movie, their spontaneous emotional-mimic reactions were filmed using iMotions program Affectiva/AFFDEX (I-MOTION GmbH Inc., Koblenz, Germany). Following the 5-minute movie, the participants were asked to stay in the room without receiving concrete instructions about how to behave. The participant’s spontaneous behavioral reactions during this time such as in a “couch situation” with non-instructured open possibilities were recorded using video-supported behavioral observation. In a further article, data concerning watching the Charlie Chaplin movie will be presented. Here, we focus on the interactive part of the experiment.

The recordings were then evaluated by two blinded raters (EA/MW) using the Bochum Adult Video-based Interaction Analysis Scale (BOEVIAS). In addition, the BOEVIAS was later repeated by an experienced rater (PM). The BOEVIAS is an evaluation sheet that was developed especially for this study to assess the dimensions of expression level (three items, such as relaxed or neutral expression), relationship level (three items, such as posture or verbal contact), control level (three items, such as attention or activity) and synchronicity (three items, such as verbal or non-verbal synchronicity). Each item of the scales can be rated with 1 (not present) to 5 (distinctly positive). These scales are unipolar in the same direction, so that a total value (by summarizing the subscales) can be formed individually for each of the two interaction partners (individual interaction profile, IIP; Mxpt:45). To assess the quality of the overall interaction, the dimension “deficits” is used to record difficulties in these dimensions for each of the interaction partners and abstracted from the sum value. The sum values per interaction partner corrected in this way result in the total value as a measure of the quality of the interaction (BOEVIAS-SID), whereby a high value (Mxpt. 90) corresponds to a better-quality interaction.

As a second step the interaction behavior of only the 5 minutes “couch situation” was evaluated using the Interact software (INTERACT Software Mangold VideoSync. Version17.1, 2018) (Mangold GmbH&Co Inc., München, Germany). For behavioral patterns such as eye contact, affects, verbalization and vocalization, body language, and synchronicity codes were predetermined before study begin. Using INTERACT software, blinded rater observed the video material and classified each specific behavior and situation by taking these codes. Thus, the video sequences were quantified and subsequently evaluated. This procedure was performed by trained raters in pairs (MW/SD and EA/SV) (duration 1.5 and 3 hours). The interrater-reliabilities were between Cohan’s-Kappa κ=0.68-0.93 (healthy subjects) and κ=0.73-0.95 (depressive patients), which shows good up to very good agreement values. For further statistical analyzes, the interaction coding values of EA/SV were used (values are given in frequency of events, percentage, and duration in seconds).

The IBM SPSS Statistics 26 program (IBM Corp., Armonk, NY, USA) was used for statistical analysis. For correlational analyzes Pearson and Spearman correlation was used. Parametric (MWTB, PSP, BDI, HAMD, STAI-I/II, TAS, PANAS, NEO-FFI) and non-parametric tests (IRI) were used for group comparison. Statistical significance was tested using the chi test, whereby values p≤0.05 were considered statistically significant and values p≤0.01 were considered statistically highly significant. Due to the character of a pilot study, no correction for multiple comparisons was used.

This study was approved by the Local Ethics Committee (IRB No.:18-6520) of the Medical Faculty of the Ruhr-University Bochum. All participants have given their written informed consent, after the study was explained to them in full. Patients were informed that the results of the study were published in anonymous form. This was not a clinical trail in the narrow sense and was therefore not registered.

Table 1 shows sociodemographic and clinical characteristics of the 20 patients (11 women and 9 men) and the 20 healthy subjects matched by age and gender.

In 15 patients (75%) a recurrent depressive disorder (F33.X) was present after ICD-10, while only five of the patients had their first depressive episode (ICD-10: F32.X). In total, 90% of patients were taking antidepressants. Two patients had no drug treatment at the time of the study.

Only 5 patients received monotherapy (only with a serotonin agonistic antidepressant). Most often, the patients received a combination treatment consisting of an antidepressant (e.g. SSRI) and a second generation antipsychotic.

The mean age at onset of the disease was 30.6 (SD=13.3), the mean duration of the disease was 7.9 (SD=9.3) years. The patients had severe psychiatric impairments (CGI) and showed severe functional impairments in more than one psychosocial area (PSP). The patients showed significantly elevated values of psychopathological characteristics, as can be seen in Table 1.

The findings on the affectivity/emotionality of depressive patients and healthy subjects are shown in Table 2. The patients showed significantly higher values in alexithymic features (TAS-20 dimensions) and negative affectivity (PANAS) when compared to healthy control subjects.

In regard to the empathy dimensions, there was just one significant difference in personal distress, possibly implying that the patient’s increased values expressed an existing impairment of emotion regulation. The NEO-FFI showed a significantly increased neuroticism value for the patients, while in the extraversion they achieved significantly lower values than the healthy subjects (Table2).

The results of the participants emotion recognition performance are presented in Table 3. There were no significant differences between depressed patients and healthy controls. Both the patients and the healthy controls showed difficulties in recognizing the emotions of fear and sadness. Furthermore, the patients showed a tendency to recognize the emotion surprise the worst. Self-assessment (with the exception of the emotion surprise, where patients had greater difficulty in recognizing it) did not show any significant group differences.

Based on the Interact findings, both groups showed an interaction behavior that was characterized by a predominance of neutral affect, no verbalization (silence), and gaze direction into the room (and not to look to the other person) (Figure 3). Significant differences between the patients and the healthy controls were found, as shown in Table 4. Significant group difference was observed in quality of affect and gaze direction between the two samples. The significantly different duration of affects was important in the sense that the positive effects were not only half as frequent among the depressed patients but also lasted for a shorter time. In regard to neutral affect, which was significantly more predominant among the patients (92% vs. 76% in the controls), the duration was also significantly longer (280 seconds vs. 231 seconds). Furthermore, depressed patients showed a reduced number of gazes towards the interaction partner, as well as a reduced duration of this event. Furthermore, the patients showed significantly more extended gazes into the room than the healthy controls (Table 4). In summary, patients differed significantly from the healthy subjects in terms of active willingness to interact, which was present in 65% of the patients and 100% of the healthy subjects (p=0.004). With regard to a body language that promotes interaction, only in the patient group with n=4 (20%) was a rejecting, negative posture found. Two patients showed a positive and 14 patients (70%) showed a neutral body language. Among the healthy subjects, neutral body language predominated with 75% (n=15), while five subjects showed a positive posture facing the interaction partner. There were no differences between the two groups with regard to synchronous behavior, which occurred sporadically (two dyads per group).

As can be seen in Table 4, the depressed patients showed more pronounced impairments in the dimensions of attention, interest, and activity (control level), accompanied by a less favorable individual interaction profile (BOEVIAS-IIP) and a less good dyad interaction (BOEVIAS-SID).

Correlation analysis of Interact variables and further biographical and clinical-psychometric scales showed only few significant correlations. “openness to interact” (AI) correlated inversely with anxiety (STAI-I, r=-0.464, p=0.039) and PANAS was negative (r=-0.446, p=0.048). Positive correlations existed between AI and body language (r=0.642, p=0.002), the percentage of positive affectivity (r=0.587, p=0.007), verbal communication (positive verbalization: r=0.779, p<0.001 or neutral verbalization: r=0.678, p=0.001) and eye contact with the interaction partner (r=0.521, p=0.019). In contrast, negative correlations were found between AI and gaze the other person (r=-0.23, p=0.018) or AI and silence (r=-0.785, p<0.001), as well as HAM-D and gaze to the partner (r=-0.502, p=0.024). Finally, there was also a correlation between the IRI scale “Empathy-Concern” (EC) and the percentage of positive affects (r=-0.460, p=0.041) or neutral affectivity (r=0.460, p=0.041).

Significant correlations were found between BOEVIAS-IIP and the severity of depression (BDI; r=-0.459, p=0.042), the MWTB-IQ (r=0.520, p=0.019), PANAS positive (r=0.561, p=0.010), body language (r=0.595, p=0.006) and active willingness to interact (r=0.673, p=0.001).

The influence of medication was tested by means of regression analyzes and only showed a significant relationship to emotion recognition for the dimension fear (beta:-0.343, p=0.046). In addition, significant influences were shown for the interaction-dimensions readiness to interact (beta:-0.398, p=0.043), percentage of neutral verbalization (beta:-0.493, p=0.014), and percentage of silence (beta: 0.455, p=0.022).

There were no significant correlation coefficients between ERT, INTERACT and BOEVIAS parameters neither in the examined depressed patients or in the healthy volunteers in this experimentally generated interaction situation.

Although the disorders of facial emotion recognition in depressed patients have been investigated in many cases, the findings are inconsistent. Both a global deficit in recognition of different emotions and isolated impairments in the recognition of specific emotions such as joy and sadness are reported [2,4,47]. Furthermore, some studies have not been able to detect emotion recognition deficits in depressive patients [48-50]. In contrast, Beck et al. [51] show a better performance of depressed patients. Equally to Beck et al.’s [51] results, in our study patients with depression did not differ significantly from the healthy controls in terms of emotional recognition skills. An explanation for this could be the fact that these patients were being treated. Another possible explanation for the overall diverging findings on emotion recognition in depressed patients is, as aptly formulated by Wolf et al. [5], the often non-realistic design of most studies. According to Wolf et al. [5] “remoteness from everyday life results from the isolated investigation of individual emotions with regard to either the exclusive expression, recognition or experience of this emotion” without taking the multiple contextual influencing factors into account. Complex, multilevel studies on emotion processing in depressed patients in general and in the context of successful interpersonal interaction specifically are urgently required [3]. Zwick and Wolkenstein [52] examined the ability to recognize emotions and the mimic expression of emotions in 42 acutely depressed, 43 remitted patients, and 39 healthy controls. Patients with severe depressive symptoms showed hypomimia (reduced facial expressions) and strong difficulties in the ability to recognize emotions. However, contrary to expectations, there was no correlation between the patient’s facial expression and their deficits in emotion recognition, consequently different processes of emotion processing have to be discussed. A good schema of these different processes and according to neurobiological findings in depressive patients can be found in Kupferberg et al. [15].

The abovementioned findings for ERT show the complexity of social interaction and social functioning, which can be achieved through the interaction of various skills and not only the recognition of emotions. In our study, we found that there were no correlations between emotion recognition and the affect and behavioral characteristics of the examined depressed patients recorded by INTERACT in an experimentally generated interaction situation. Depressed patients in this rather noncommittal small-talk situation showed, despite an increased negative anxiety-depressive state, an interaction behavior that was characterized by neutral affectivity, silence, and avoidance of direct eye contact, similar to the healthy subjects. The main difference was that the depressed patients showed shorter interaction-promoting behaviors (e.g., short glances at the interlocutor or short positive-verbal utterances) and they remained significantly longer in interaction-inhibiting states (e.g., glancing into the room). This can be discussed as an indication that in depressive patients, impairments of social interaction behavior are more likely to be seen in connection with depression-associated slowdown or depressive cognitive-motor inhibition (depressive inhibition). This is also supported by the fact that the patient’s individual interaction profile (BOEVIAS) showed strong deficits in the areas of concentration, attention, and activity compared to the healthy controls. These findings can be seen as an expression of psychomotor slowdown and disturbance of executive functions, and are in consensus with the literature [53-55]. Previous studies on kinematic movement analysis of the facial expressions of patients with depression have already shown that patients show significantly lower speed measures in both the voluntary and involuntary facial expression muscles than healthy people, depending on the severity of the depressive symptoms [13,14]. Troisi and Moles [56] also found a general reduction in the sense of a depressive inhibition, especially in their non-verbal interaction behavior. They examined 68 depressed patients with 72 healthy controls using scales and direct observation of behavior. Teixeira Fiquer’s research group was also able to show a reduced-inhibited interaction behavior (verbal as well as non-verbal) in patients with depression, which could be improved by treatment with transcranial stimulation (TCS) [56,57]. Interestingly, however, it was shown that the non-verbal behavior (e.g., eye contact, gestures, or shaking the head) of depressed patients changed little after this therapy measure, whereas speaking or the effort to speak increased [58]. The verbal communication behavior of depressed patients seems to have changed both quantitatively and qualitatively in the sense that more negative content is formulated [59-61].

Our findings confirm that positive-verbal communication similar to direct eye contact and body language directed towards the interlocutor is essential for successful social interaction. Therefore, it is not surprising that patients with a depressive disorder and associated impairments in these communication channels have greater difficulties in social interaction [15,18,62,63]. However, it is unclear if this social dysfunction is only a consequence of depression or, as Porcelli et al. [22] suggest in their survey work, if a trait feature has a more depressive effect. Although this aspect cannot be conclusively clarified at present, our findings allow the conclusion that depressed patients are not generally incapable of social interaction but that they need more time to adapt and react to a new situation. Radke et al. [64] reached a similar conclusion when they examined 30 depressive patients with 20 healthy controls with regard to a flexible approach-distance behavior. The depressive patients were not only conspicuous for their less dominant behavior but also for their lack of flexibility in their approaching and avoiding tendencies, and also in affective modulation [64].

Our findings on the impairment of psychosocial functioning and increased neuroticism as a predisposing depressive factor are consistent with the extensive literature on this area of research [65-70]. Elevated TAS alexithymia levels in patients with depressive disorder have frequently been reported and discussed, which suggests that there is a close relationship between this emotion regulation disorder and depression [16,71-74]. With regard to PANAS measuring negative or positive affectivity, there are existing studies on reliability and validity [44,75] but there is a lack of evaluations of this scale among depressive patients [76-78]. A recent study by Podlogar et al. [79] found that there was a close connection between an increased risk of suicide and the predominance of negative affectivity (recorded by PANAS in more than 600 anxiety-depressive outpatients).

Our investigation has several limitations. The first limiting factor is the small sample size. Furthermore, this is a monocentric study in which depressed patients were examined who were under drug treatment. Consequently, valid representative statements are possible to a limited extent. A further limiting factor is that the instrumental methods (including the experimental conditions) cannot be regarded as a standard in clinical practice, so that generally valid conclusions also appear critical here. In general, statements on the complex phenomena of behavior and emotionality in the context of social interaction derived from a tightly structured experimental framework and automated analyses are limited because they can only reflect a part of the complex human behavioral repertoire and its emotionality. Finally, only two interactive situations (i.e., watching a movieclip together and sitting together without further instructions) were investigated and were analyzed in a rather basic way. A further comparative situation, such as an instructed conversation, had not been conducted so far. In addition, it must be noted that the use of questionnaires (which were mainly designed specifically for this study and were only rated by the investigators) must be viewed critically because they have not yet been sufficiently validated.

In conclusion, within the framework of a semi-structured experimental paradigm, we were able to examine depressive patients in pairs in regard to emotion processing, their social interaction behavior, and its individual dimensions in more detail. We also made a comparison with healthy controls using newer instrumental measurement methods. Impairments of the social interaction behavior of depressive patients in the sense of reduced-inhibited interaction-promoting non-verbal and verbal activity probably seem to be less associated with disorders of basic skills of emotion recognition and they should be considered more in the context of regulatory disorders of one’s own emotionality and cognition given that they do not fundamentally prevent but significantly impede a faster adaptation to the situational conditions in depressive patients. This in turn implies specific psychotherapeutic programs for depressive patients, as called for by Weightman et al. [18], to optimize their social interaction skills.

Further studies focusing on the interactions of the various communication channels, also in the context of cognitive and metacognitive processes, appear to be necessary. In addition, the technical possibilities could be used more effectively and intensively. It seems to be useful to define more homogeneous subtypes of depressive disorders by this method. In the psychiatric-psychotherapeutic field, an instrumental analysis of interaction and behavior could be useful as a basis for differentiated behavioral treatment strategies for a large proportion of mental illnesses, which are often accompanied by impairments of social skills. In this respect, research activities need to be intensified and technical equipment needs to be established.