INTRODUCTION
Cognitive impairment is a common symptom of schizophrenia and strongly associated with functional disabilities affecting daily life. It is estimated that 90% of patients have clinically meaningful deficits in at least one cognitive domain, and 75% in at least two [1]. Several reviews have concluded that cognitive deficits show highly consistent relationships with various types of functional outcomes, including community functioning and the ability to acquire skills in psychosocial rehabilitation [2].
Pharmacotherapy for cognitive impairment represents a significant unmet medical need in schizophrenia and is a key focus of pharmacological development by government health agencies, academic researchers, and the pharmaceutical industry [3].
Detecting and following changes is essential when developing drugs to improve cognitive function in patients with schizophrenia through cognitive remediation. The representative cognitive measuring tool developed and used for that purpose is the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Consensus Cognitive Battery (MCCB), developed by the National Institute of Mental Health (NIMH) [4]. The MCCB is considered the standard tool for cognitive assessment in clinical trials, and evaluates seven domains of cognitive function using ten tests [5]. This battery is easy to administer, has high fidelity, and takes a little over 1 hour to complete on average [5].
Although objective measures of cognition using neuropsychological test batteries are considered the standard, there are occasional limitations on their use. It has been suggested that clinicians may not be able to satisfactorily evaluate changes in daily living capacity with performance-based measures alone, due to several reasons including, among others, poor compliance of agitated subjects to the procedures and practice effects. Interview-based assessments, on the other hand, may be free of these disadvantages [6]. One such assessment is the Schizophrenia Cognition Rating Scale (SCoRS), consisting of 20 items, rated on a 1–4 scale, covering all seven MCCB domains and administered to patients as well as informants [5]. One of the strengths of the SCoRS is its relatively short administration time, averaging 30 min for the entire process. Moreover, clinicians can easily monitor progress by referring to the global rating score, which effectively presents a unified yet composite representation of the patient’s cognition [7]. Keefe et al. [8] showed that the SCoRS is valid in that its global rating is strongly correlated with cognitive performance, functional outcome, and functional capacity.
Knowing the degree of association between each item of the performance-based measure and each item of the interview-based measure, and between the respective global scores, might enable clinicians to screen the most useful questions or tests to assess cognitive functions when time is limited, such as in an outpatient setting.
Several studies have been conducted to this purpose, comparing the SCoRS global and item-specific scores with performance-based cognitive tests, using either the Brief Assessment of Cognition in Schizophrenia (BACS) scale instead of the MCCB or the MCCB without the social cognition domain [9,10]. No previous study has examined the association between all MCCB domains and each SCoRS item.
Here, we therefore analyzed the correlation between all MCCB domains (performance-based) and all SCoRS items (interview-based) in patients with chronic schizophrenia, with the aim to identify the items and domains that show high correlations.
METHODS
Subjects
We analyzed the data collected from patients who participated in our previous study [11]. The study was based on observations conducted at three time points: at baseline, after 2 weeks, and after 3 months. Baseline measurements were used for the demographic and clinical data, while all three point measurements were used for the cognitive data. The total of the three point measurements was 149, including 56 baseline measurements, 53 measurements after 2 weeks, and 40 measurements after 3 months. Fifty-six clinically stable patients with schizophrenia were recruited from the day hospitals and outpatient units of three sites (one university and two mental hospitals) over a 1.5-year period between July 2015 and February 2017. The three institutions were the Inje University Busan Paik Hospital, the Sharing and Happiness Hospital in Busan, and the Busan Metropolitan Mental Hospital.
This study included patients diagnosed with schizophrenia based on the Diagnostic and Statistical Manual of Mental Disorders 5th edition [12] satisfying the following inclusion criteria: 1) aged 18–65 years; 2) clinically stable patients who visited the day hospital or the outpatient unit during the previous 3 months; 3) no changes in medications in the past 3 months; and 4) no known neurological history. We obtained signed written consent forms from all patients, and the study protocol was approved by the Institutional Review Board of Busan Paik Hospital, Inje University College of Medicine (2015-05-0002).
Clinical and cognitive assessments
Symptom evaluations were completed by psychiatrists not directly involved in the rehabilitative interventions. Neuropsychological tests were administered by trained psychiatrists external to the treatment teams who did not know the individual patients or their treatment. Data were collected from clinical records, patient interviews, and information obtained from close relatives and the day hospital case manager.
Symptom assessment
To assess clinical symptoms, psychiatrists conducted interviews using the Positive and Negative Syndrome Scale (PANSS) [15] and the Clinical Global Impression-Schizophrenia scale (CGI-SCH) [16].
The PANSS symptoms were grouped into five factor categories (positive symptoms, negative symptoms, disorganized thought, hostility/excitement, and depression/anxiety), based on a previously established model [17].
The CGI-SCH was originally developed as a brief, standardized method to evaluate the overall functions of patients with schizophrenia from a clinical perspective. The functions are categorized into four domains of positive, negative, depressive, and cognitive symptoms, each assessed using a 7-point scale [16].
We also used the Korean version of the Calgary Depression Scale for Schizophrenia (CDSS) [18], a structured interview tool composed of nine items, to assess depression.
Cognitive function evaluation
MCCB assessment
The MCCB is a standardized tool for evaluating basic cognitive functions in patients with schizophrenia, recommended for clinically testing the effects of new drugs designed to improve the cognitive functions of such patients [19].
The MCCB contains the following seven domains and tests: 1) Speed of Processing: Trailmaking Test, Brief Assessment of Cognition Symbol Coding, and Category Fluency; 2) Attention-Vigilance: Continuous Performance Test, identical pairs; 3) Working Memory: Wechsler Memory Scale-III Spatial Span and Letter-Number Span; 4) Verbal Learning: Hopkins Verbal Learning Test-Revised; 5) Visual Learning: Brief Visuospatial Memory Test-Revised; 6) Reasoning and Problem Solving: Neuropsychological Assessment Battery Mazes; and 7) Social Cognition: Mayer-Salovey-Caruso Emotional Intelligence Test (MSCEIT): Managing Emotions.
We used the Korean version of the MCCB.
The Schizophrenia Cognition Rating Scale (SCoRS)
The SCoRS is a subjective 20-item interview-based assessment of cognitive impairment and of the degree to which the impairment affects everyday activities [8]. The final component of the SCoRS requires the interviewer to rate each participant on a global rating scale, designed to summarize their overall level of subjective cognitive function, based on their responses to the previous 20 items, on a 10-point scale (1=no impairment; 10=extreme impairment). Complete administration of the SCoRS also involves an interview with an informant (someone in regular contact with the patient). We used the Korean version, SCoRS-K, presented in our previous validation study [20].
In this study, due to the insufficient number of informant and patient ratings, we used only the interviewer ratings. The interviewer’s global rating of the SCoRS was considered the main outcome measure.
To ensure inter-rater reliability, all raters successfully completed rater training before participating in the study, to achieve an interclass correlation coefficient greater than 0.75.
Statistical analysis
Comparisons among groups were performed using the Student t-test, Wilcoxon Rank Sum test, analysis of variance, and Kruskal-Wallis test for continuous measures and Pearson’s chisquared test and Fisher’s exact test for categorical measures. Spearman correlation was used to assess the correlations between each SCoRS item’s interviewer rating and each MCCB item. All tests were two-tailed, and the significance threshold was set at p≤0.05. Statistical analysis was performed using SAS software (SAS Institute, Cary, NC, USA).
RESULTS
Demographic and clinical characteristics
Demographic and clinical assessment data are presented in Table 1.
Correlations between MCCB and SCoRS scores
The correlations of the MCCB cognitive domain and composite scores with the SCoRS total score and global rating score are presented in Table 2. The correlation between the MCCB overall T score and the SCoRS global score was the strongest (r=-0.52). The SCoRS total score and the MCCB Speed of Processing score were also correlated (r=-0.48). The MCCB Visual Memory and Social Cognition domain scores were not significantly correlated with the SCoRS total score.
Of the 20 items in the SCoRS, the number of items significantly correlated with each MCCB score and the mean and median values of r are shown in Table 3. The MCCB overall T score showed statistically significant correlations with 18 out of the 20 SCoRS items, and Speed of Processing was significantly correlated with all SCoRS items.
Conversely, the number of items with significant correlation and the mean and median r values among the eight MCCB items are shown in Table 4, for each SCoRS item. The associations between the SCoRS global score and seven MCCB domains as well as the overall T score were statistically significant. The same results were obtained for SCoRS item 12 (“Do you/Does the patient have difficulty with familiar tasks?”). See Supplementary Table 1 (in the online-only Data Supplement) for the complete correlation analysis. The mean of the individual SCoRS items is shown in Supplementary Table 2 and 3 (in the online-only Data Supplement).
DISCUSSION
Keefe et al. [21] collected data in a three-site research center consortium and found that overall MCCB composite scores and SCoRS total ratings can show high correlations (r=0.46), similar to the results of this study (r=-0.44). The highest reliability was found for the sum of the 20 items, compared with the single global ratings that compose the instrument. In this study, the correlation with the global score was slightly higher. Keefe et al. [21] observed that the MCCB domain scores tended to be correlated with specific items in a broad fashion, rather than specifically with items that conceptually would seem related in terms of cognitive domains, so that many of the SCoRS items seem to evaluate multiple cognitive domains simultaneously. The results of this study also support this view.
As Keefe et al. [21] commented, sharp distinctions among MCCB domains are not well supported in factor analyses [22,23], and the domains may best be represented by a single factor [8,24], with the possible exception of Social Cognition. In agreement with such considerations and with previous studies, here, Social Cognition also showed the weakest correlation with the SCoRS items.
In the earlier study by Harvey et al. [10], the SCoRS global scores were 4.90 (2.00) and 4.90 (2.10) for the lurasidone and ziprasidone groups, respectively. As the authors did not administer the entire MCCB, they could not assess how the results would have been different if the CPT and MSCEIT were included. In our study, the entire MCCB was administered to provide information complementary to that provided by previous studies [9,10]. Our results show that Social Cognition was less correlated with the SCoRS global items than the other MCCB domains. Among the SCoRS individual items, the number of items showing statistical significance with this MCCB domain was the smallest (n=2).
Vita et al. [9] analyzed clinically stabilized patients and showed that the SCoRS interviewer’s global rating is a reliable and valid tool for easy, simple, and routine assessment of cognitive impairment, correlating satisfactorily with clinical, neuropsychological, and psychosocial functioning variables; this is also consistent with the results of the current study. The authors mentioned that the rehabilitative setting is much more similar to a real-life context where the patient may perform the activities assessed by the SCoRS interview. In our study, most of the participants were in a day hospital setting, better reflecting a real-life context. In the aforementioned study [9], Social Cognition was not included and only the SCoRS global rating was analyzed. Our study included Social Cognition and the analysis of individual items, thereby complementing the results of existing research.
In addition to the MCCB, some studies used the BACS, a test battery that is shorter than the MCCB but has similar reliability and validity [6-8]. The BACS consists of the following test and domains: List Learning Test (Verbal Memory); Digit Sequencing Task (Working Memory); Token Motor Task (Motor Speed); Category Instances Test (Semantic Fluency); Controlled Oral Word Association Test (Letter Fluency); and Tower of London Test (Reasoning and Problem Solving). Higuchi et al. [6] observed a strong correlation between the SCoRS global rating Score (interview-based measure) and the BACS composite score (performance-based measure) for their entire sample of patients, which included patients with schizophrenia (both first-episode and chronic) and subjects with an at-risk mental state (ARMS). In that study, patients with chronic schizophrenia had a SCoRS global score of 4.39 (1.93), a BACS Z score of -1.10 (0.97), and a correlation between the SCoRS global score and the BACS composite Z score of r=-0.037 (p<0.001), all points r=-0.362 (p<0.0001). In our study, the SCoRS global score was 5.13 (1.85) and its correlation with the MCCB overall T score was -0.52 (p<0.0001).
A study on the relationship between the individual domains of the BACS and the SCORS global items was conducted by Chia et al. [7] The total PANSS score of their 103 participants was 39.85, the SCoRS global score 4.53, and the BACS Z score -2.27. The BACS Z score, Working Memory, and Reasoning and Problem Solving showed correlations of -0.264 (p=0.008), -0.333 (p=0.001), and -0.232 (p=0.020) with the SCoRS global score, respectively. A similar degree of association was observed for the Working Memory domain in our study. In the study of Chia et al. [7], among the correlated domains, Working Memory and Processing Speed showed the strongest association with the global rating score. Similar results were observed in our study, where Speed of Processing was significantly correlated with every individual SCoRS item. Chia et al. [7] observed that the strength of the relationship between the SCoRS and BACS was at best modest (r=-0.264), and claimed that this appeared to be markedly weaker than the large effect size of -0.540 reported by Keefe et al. [8] Our study found an effect size that is intermediate, in between those of the two earlier studies. However, Chia et al. [7] mentioned that when the SCoRS global rating score was based solely on feedback from family and friends, its association with the BACS Z score dramatically improved to -0.662. The key point here is that the informants had the opportunity to interact with the subject before their illness, and were therefore in a better position to report the cognitive changes that occurred over the course of the illness. This suggests that if we had obtained more reliable information from the informants in our study, the correlation we found might have been higher.
Another study using the BACS conducted by Keefe et al. [8] found a Pearson correlation of r=-0.540 between the BACS composite score and the SCoRS interviewer score. The authors of this study claimed that there was considerable shared variance between these two outcome measures, suggesting that a common element of cognition is being measured; this is in agreement with the results of our research. Keefe et al. [8] argued that the SCoRS measures the aspects of cognition that are indeed relevant for real-world functioning. In future research focusing on the relationships with daily functions, a comparison with the UCSD Performance-Based Skills Assessment (UPSA) may be useful to confirm this claim.
Balzan et al. [25], using the BACS, found a Verbal Memory score of 1.88 (0.50), a Working Memory score of 2.08 (0.85), a Processing Speed score of 1.40 (0.50), a Reasoning and Problem Solving score of 1.48 (0.45), and a correlation between the SCoRS global and BACS composite scores of -0.203 (p=0.28), which is lower than the correlation found in our study.
This study has some limitations. As Keefe et al. [21] mentioned, relying on informant data, which is not available for some patients and may vary by region, is an inherent weakness of this type of studies. Chia et al. [7] claimed that in order to obtain an accurate evaluation of the patient’s cognitive profile, care should be taken to engage the appropriate informants. In future studies, more reliable informants will be needed, and the sample size should be increased. Regarding data quality, our study compared only the SCoRS interviewer ratings. In future studies, if all three kinds of SCoRS data are collected with appropriate quality, the discrepancy between cognitive function perceived by the patient themselves and by an observer may be compared with performance-based measures. Furthermore, as participants in this study were patients with chronic schizophrenia, the long-term use of antipsychotics might have affected their cognitive domains and might have led to the differences between performance- and interview-based measurements. Future similar studies in drug-naïve patients in the early stages of psychosis will give us further insights into the effects of antipsychotics on cognition.
In this study, we analyzed the association between all SCoRS items and the seven domains as well as the composite T score of the MCCB. Our results reveal clear correlations between the performance- and interview-based measurements, which suggests interview-based measurements as a useful cognitive function evaluation tool that can easily be applied in clinical settings. Confirmation of these findings in larger cohorts of patients may enable a simple item selection to evaluate cognitive function according to the patient’s condition in a clinical setting.
