Psychiatry Investig Search

CLOSE


Psychiatry Investig > Volume 20(2); 2023 > Article
Lee, Kim, Kim, Yang, Lee, Lee, and Shin: Computerized Version of the Kiddie Schedule for Affective Disorders and Schizophrenia for School-Age Children (KSADS-COMP): Development and Validation of the Korean Clinician-Administered Version

Abstract

Objective

The purpose of the present study was to develop and validate the Korean version of the clinician-administered KSADS-COMP, which is the recently updated, web-based computerized version of the Kiddie Schedule for Affective Disorders and Schizophrenia for school-age children (KSADS).

Methods

A total of 71 participants (mean age=12.04±3.86 years, female=29.57%) participated in the study. A child-adolescent psychiatrist established a diagnosis for the participant after a thorough psychiatric interview with the participant and the parent. Researchers who were blind to the diagnoses administered the clinician-administered KSADS-COMP to the parents and participants. The gold-standard diagnoses made by child-adolescent psychiatrists were compared to the current diagnoses generated by the clinician-administered KSADS-COMP. Percent agreement, Cohen’s Kappa, Gwet’s first-order agreement coefficient (AC1), sensitivity, specificity, positive predictive value, and negative predictive value were calculated.

Results

Gwet’s AC1, our preferred measure of agreement, showed excellent range between 0.78 and 1. Sensitivity, specificity, positive predicted value and negative predictive value also showed high scores.

Conclusion

The current study demonstrated excellent criterion validity of the Korean version of the clinician-administered KSADS-COMP, though the small sample size could be a limitation. The current study was the first study to examine the criterion validity of the KSADS-COMP. Due to its readily usable format and efficient and accurate diagnostic process, widely-use of KSADS-COMP is expected.

INTRODUCTION

Diagnosing mental disorders in children and adolescents could be very challenging. A thorough assessment procedure is needed for identifying their symptoms and classifying the symptoms into a specific diagnosis. The Kiddie Schedule for Affective Disorders and Schizophrenia for school-age children (KSADS) is one of the most widely used diagnostic instrument for assessing psychiatric diagnoses in children and adolescents [1]. KSADS has been translated into more than 30 languages and has been well-validated. The Korean version of KSADS has also been translated and validated [2]. KSADS has been used extensively in clinical practice and research.
KSADS-COMP is the recently updated, computerized version of the paper-and-pencil KSADS [3]. KSADS-COMP was updated to reflect Diagnostic and Statistical Manual for Mental Disorders fifth edition (DSM-5) diagnostic criteria. KSADS-COMP adopted an automated scoring algorithms which generates both categorical diagnoses and diagnosis-specific dimensional rating scales of current symptoms. Also, the scoring criteria were modified to use the same standardized 5-point rating scale for all questions regarding current symptoms [3].
KSADS-COMP has already been translated into several languages including Spanish, Dutch, and Danish. However, the validity data of KSADS-COMP was reported only in one study by the original developers [3]. The purpose of the present study was to develop the Korean version of the clinician-administered KSADS-COMP and examine the validity in Korean children and adolescents.

METHODS

Translation & development of the Korean KSADS-COMP

With the permission from the original developer, the webbased computerized versions of KSADS-COMP were translated into Korean by child-adolescent psychiatrists who were bilingual in English and Korean. Then a panel of experts which included child-adolescent psychiatrists and clinical psychologists who were also bilingual reviewed the translation. Revisions were made under the consensus of the original translators and the expert panel. Then, the translation was backtranslated into English by a professional native translator who was blind to the original sources of the measure before or during back-translation. The back-translations were sent to another psychiatrist who was bilingual to address any discrepancies between the original version and the back-translated version. The translation was proofreaded several times by the expert panel and the original translators. World Health Organization (WHO) guidelines on translation were followed during this process [4].

Participants

Children and adolescents from 6 to 18 years who were referred to the department of psychiatry in two hospitals—Ajou University Hospital and Soonchunhyang University Bucheon Hospital— were recruited from November 2021 to December 2021. Children and adolescents who presented with any kind of psychiatric symptoms such as depressed mood, anxiety or irritability and who agreed to participate in the study were included in this study. Informed consent and written assent were obtained by all parents and children, respectively. The study was approved by the Institutional Review Board of Ajou University Hospital (AJIRB-SBR-SUR-21-459) and Soonchunhyang University Bucheon Hospital (SCHBC 2021-10-0002). The trial was registered at the Clinical Research Information Service (CRIS) of Republic of Korea (Registration Number KCT0006794).

Procedures

An experienced child-adolescent psychiatrist established a clinical diagnosis according to the DSM-5 for the participant after a thorough psychiatric interview with the participant and the parent. Then researchers who were blind to the diagnoses administered the clinician-administered KSADS-COMP to the parents and participants. Researchers, comprised of psychiatric residents and clinical psychologists, were trained for the administration of the clinician version of the KSADS-COMP. The clinician-administered KSADS-COMP was completed within two weeks after the initial diagnostic interview with the child-adolescent psychiatrist.

Measures

Clinician-Administered KSADS-COMP

There are three versions of KSADS-COMP: a clinician-administered version, a self-administered youth version, and a self-administered parent version [3]. The clinician-administered KSADS-COMP is the computerized counterpart of the paper and pencil KSADS. It is a semi-structured diagnostic instrument which includes an introductory interview, the screening interview, and the supplement interview. The introductory interview obtains background information like life circumstances, adaptive functioning, treatment history and family history. The screening interview screens 2 to 4 main symptoms of each psychiatric diagnosis. The supplement interview is for the diagnosis of the disorders with above threshold scores from the screening interview. Also, self-administered pre-interview is available for both the parent and the youth to be completed before administering the clinician-administered version.
To establish interrater reliability, all six researchers from the two hospitals scored all the items on the clinician-administered KSADS by watching a mock patient interview that was video-recorded. Among 160 questions, researchers agreed on the same items for 97.5% (156 out of 160). Diagnostic concordance was 100% between the six researchers.
The parent interview was completed first if the youth was a child and the youth interview was completed first if the youth was an adolescent. After the parent and youth interview were done, final diagnoses were generated after the consensus interview which integrated all the information from the parent and youth interviews. When the researchers administered the consensus interview, both answers from the parent and the youth interviews were shown on the top of the screen.

Child Behavior Check List

Child Behavior Check List (CBCL) is a standardized parental questionnaire widely used to identify childhood emotional and behavioral problems [5]. The CBCL is comprised of 113 items which are divided into 8 syndrome subscales (withdrawn/depressed, somatic complaint, anxious/depressed, social problems, thought problems, attention problems, rulebreaking behavior, and aggressive behavior). CBCL also has a set of DSM-oriented scales: affective problems, anxiety problems, somatic problems, attention deficit hyperactivity disorder (ADHD) problems, oppositional defiant problems, conduct problems, obsessive compulsive disorder (OCD) and posttraumatic stress disorder (PTSD).

Statistical analysis

To assess the criterion validity of the KSADS-COMP, diagnoses made by the clinician-administered KSADS-COMP were compared to the gold-standard diagnoses made by the child-adolescent psychiatrists. Percent agreement, Cohen’s Kappa, Gwet’s first-order agreement coefficient (AC1), sensitivity, specificity, positive predictive value, and negative predictive values were calculated. Gwet’s AC1 was our preferred measure of agreement because it was proved to be better than Cohen’s Kappa in the previous studies [6]. T-statistics and its p-value for the test of the null that Gwet’s AC1 equals zero were calculated.
To assess the convergent validity of the clinician-administered KSADS-COMP, scores of the corresponding CBCL DSM-oriented subscales were compared between the participants with positive diagnoses and those without positive diagnoses generated by the clinician-administered KSADS-COMP. Five diagnostic categories with high frequencies were selected for the analyses for the convergent validity of KSADS-COMP: depressive disorder, anxiety disorder, OCD, ADHD, and oppositional defiant disorder (ODD). Tic disorder was also one of the diagnoses with high frequencies but was excluded because there was no corresponding CBCL subscales to compare with. Wilcoxon signed rank tests were used.

RESULTS

A total of 71 participants (mean age=12.04±3.86 years, female=29.57%) participated in the study. The clinical diagnoses of the participants made by the child-adolescent psychiatrists are shown in Table 1. Among the participants, 97.18% of the participants received a psychiatric disorder with only two participants with no diagnosis. ADHD was the most prevalent diagnosis (n=43). Tic disorders (n=23), OCD (n=22), depressive disorders (n=21), anxiety disorders (n=21), and ODD (n=19) also showed high prevalence.
The gold-standard diagnoses made by the child-adolescent psychiatrists were compared to the current diagnoses generated by the clinician-administered KSADS-COMP (Table 1). Gwet’s AC1 showed great range between 0.78 and 1. Sensitivity, specificity, positive predicted value, and negative predictive value also showed high scores. Low scores were observed in specific phobia and autism spectrum disorder (ASD). Specific phobia showed positive predictive value of 45 and ASD showed sensitivity value of 50.
Table 2 shows the mean and standard deviations of the CBCL subscale scores for depression, OCD, ADHD, ODD, and anxiety for the participants who did and did not meet criteria for the corresponding current diagnoses generated by the clinician-administered KSADS-COMP. Wilcoxon rank-sum tests indicated that for all these disorders, participants who were diagnosed with a positive diagnosis by the clinician-administered KSADS-COMP scored significantly higher on the corresponding DSM-oriented scales on CBCL than those who were not diagnosed as having that diagnosis.

DISCUSSION

The current study demonstrated excellent criterion validity of the Korean version of the clinician-administered KSADS-COMP. This was the first study to examine the criterion validity of the KSADS-COMP, which is comparing results generated from a test measure to those obtained from a criterion, or goldstandard measure. The diagnostic concordance rates between the gold-standard diagnoses made by the child-adolescent psychiatrists and the current diagnoses generated by the clinician-administered KSADS-COMP calculated by Gwet’s AC1 were statistically significant. High positive predictive values and high negative predictive values were shown in all diagnoses except for the relatively low positive predictive value observed in specific phobia. High sensitivity and specificity values were observed in all disorders except for the relatively low sensitivity shown in ASD.
Low positive predictive value was observed in specific phobia. Thus, percentage of participants who were diagnosed with specific phobia in the clinician-administered KSADS-COMP who actually have specific phobia was low. Specific phobia was the only diagnosis that had a much higher prevalence in KSADS-COMP than in gold-standard diagnoses in this study. It could mean that KSADS-COMP has a rather low threshold for diagnosing specific phobia than in clinical practice. On the other hand, it could also imply that child-adolescent psychiatrists could have left out specific phobia unintentionally when they were evaluating the participant, since it is not one of the main psychiatric diagnoses. It has been reported that specific phobias often go unidentified and left untreated in youth [7]. Given the more thorough and comprehensive coverage of the KSADS-COMP, KSADS-COMP may generate a more detailed diagnosis without overlooking any minor symptoms compared to a routine unstructured psychiatric interview by an expert.
Low sensitivity value was observed in ASD. Thus, the ability of clinician-administered KSADS-COMP to correctly diagnosis the participant as having ASD was insufficient in this study. ASD is a very heterogenous disease that is notoriously difficult to diagnose [8]. The gold-standard diagnostic tools are Autism Diagnostic Observational Schedule (ADOS) which is evaluating the patient and Autism Diagnostic Interview (ADI) which is interviewing the parent. [9] The information gathered in both tests enable the clinician to diagnosis ASD more accurately. In KSADS-COMP, questions related to ASD were only included in the parent interview. Therefore, the ability of KSADS-COMP in accurately diagnosing ASD needs further evaluation.
There are several limitations of the study that needs to be addressed. First, the relatively small sample size could be the limitation of the current study. Except for depressive disorders, anxiety disorders, OCD, ADHD, ODD, and tic disorders, the prevalence of the disorders were low. Second, the sample was comprised of clinically referred patients and 97.18% of the participants were diagnosed with a psychiatric disorder. Positive predictive value for no diagnosis showed low score of 40 in this study. Therefore, the ability of the diagnostic tool to discriminate between the disordered and the non-disordered participants needs further examination. Third, the gold-standard diagnosis used in the study was the clinical diagnoses made by a single experienced child-adolescent psychiatrist. Consensus diagnoses among multiple experts could have enhanced the reliability of the gold-standard diagnosis.
Clinician-administered KSADS-COMP has many advantages over the previous KSADS. In the previous study which validated the Korean version of the paper-and-pencil version of KSADS, Cohen’s Kappa values ranged from 0.241 to 0.695 [2]. Cohen’s Kappa values in the current study ranged from 0.58 to 0.94. Therefore, clinician-administered KSADS-COMP could have a better diagnostic accuracy than the former version. It is consistent with previous studies which reported that computerized versions of structured diagnostic instruments showed better psychometric properties compared to the paper-and-pecnil versions [10,11]. Also, clinician-administered KSADS-COMP could be much readily usable than the former pencil-and-paper version. It is faster, because of its automated scoring and reports that are ready as soon as you finish your administration. It would also require less training and reduce data-entry errors. Because large numbers of detailed response sets can be automatically collected in the database and later downloaded, it could bring benefits in studies on large study population.
In conclusion, the current study developed and validated the Korean version of the clinician-administered KSADS-COMP. The Korean version of the clinician-administered KSADS-COMP showed great psychometric properties. Due to its readily usable format and efficient diagnostic process, KSADS-COMP is expected to be even more widely used in research and clinical settings than the previous versions of KSADS. Future study could use a larger study population with a broad spectrum of the disordered and non-disordered participants.

Notes

Availability of Data and Material

The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.

Conflicts of Interest

The authors have no potential conflicts of interest to disclose.

Author Contributions

Conceptualization: Yunmi Shin. Data curation: Seong-Ju Kim, Dohyung Kim. Formal analysis: Dohyung Kim. Funding acquisition: Yunmi Shin. Investigation: Seong-Ju Kim, Sangha Lee, Areum Lee. Methodology: Jeewon Lee, Su-Jin Yang, Yunmi Shin. Software: Jeewon Lee. Validation: Su-Jin Yang. Writing—original draft: Jeewon Lee. Writing—review & editing: Seong-Ju Kim, Dohyung Kim, Su-Jin Yang, Sangha Lee, Areum Lee, Yunmi Shin.

Funding Statement

This study was supported by a research fund from the National Center for Mental Health, Ministry of Health & Welfare, Republic of Korea.

Table 1.
Criterion validity of Korean clinician-administered KSADS-COMP (N=71)
Gold-standard diagnosis frequency Clinician KSADS-COMP frequency Percent agreement Sensitivity Specificity Positive predictive value Negative predictive value Cohen’s Kappa Gwet’s AC1 t-statistic p
Depressive disorders 21 20 0.96 90 98 95 96 0.90 0.93 22.3 <0.001
Bipolar disorders 1 1 1 100 100 100 100 1 1 - -
Psychotic disorders 1 1 1 100 100 100 100 1 1 - -
Anxiety disorders 21 24 0.93 95 92 83 98 0.84 0.88 15.8 <0.001
Specific phobia 5 11 0.92 100 91 45 100 0.58 0.89 19.9 <0.001
OCD 22 20 0.92 82 96 90 92 0.80 0.86 14.4 <0.001
PTSD 5 4 0.96 60 98 75 97 0.64 0.95 33.5 <0.001
ADHD 43 35 0.89 81 100 100 78 0.78 0.78 10.3 <0.001
ODD 19 20 0.93 89 94 85 96 0.82 0.88 16.7 <0.001
CD 1 1 1 100 100 100 100 1 1 - -
Tic disorders 23 23 0.97 96 98 96 98 0.94 0.95 26.7 <0.001
Alcohol use disorder 1 1 1 100 100 100 100 1 1 - -
Eating disorders 7 8 0.99 100 98 88 100 0.93 0.98 55.8 <0.001
ASD 4 2 0.97 50 100 100 97 0.65 0.97 43.8 <0.001
No diagnosis 2 5 0.96 100 96 40 100 0.55 0.95 34.5 <0.001

Diagnostic concordance between the gold-standard diagnosis and the diagnoses generated by the clinician KSADS-COMP were examined. The last two columns present for each of selected disorder and pair of raters, t-statistics and its p-value for the test of the null that Gwet’s first-order agreement coefficient (AC1) equals zero. KSADS-COMP, computerized version of the Kiddie Schedule for Affective Disorders and Schizophrenia for school-age children; OCD, obsessive-compulsive disorder; PTSD, post-traumatic stress disorder; ADHD, attention deficit hyperactivity disorder; ODD, oppositional defiant disorder; CD, conduct disorder; ASD, autism spectrum disorders

Table 2.
Convergent validity of Korean clinician-administered KSADS-COMP: scores of corresponding scores on CBCL by current diagnoses by clinician-administered KSADS-COMP
Current negative
Current positive
Wilcoxon rank-sum test
Mean±SD N Mean±SD N Z p
Depressive disorders 63.16±8.77 51 73.8±6.27 20 -4.326 <0.001
Anxiety disorders 63.83±10.68 47 70.79±10.18 24 -2.514 0.012
OCD 65.35±9.28 51 71.9±11.05 20 -2.059 0.039
ADHD 56.44±5.94 36 68.94±12.55 35 -4.793 <0.001
ODD 58.61±9.06 51 70.2±11.12 20 -4.184 <0.001

Wilcoxon rank-sum test indicated that for all five disorders, participants who were diagnosed with a positive diagnosis by the clinician-administered KSADS-COMP scored significantly higher on the corresponding DSM-oriented scales of CBCL than those who were not diagnosed as having that diagnosis. KSADS-COMP, computerized version of the Kiddie Schedule for Affective Disorders and Schizophrenia for school-age children; CBCL, Child Behavior Check List; DSM, Diagnostic and Statistical Manual for Mental Disorders; SD, standard deviation; N, number; OCD, obsessive-compulsive disorder; ADHD, attention deficit hyperactivity disorder; ODD, oppositional defiant disorder

REFERENCES

1. Kaufman J, Birmaher B, Brent D, Rao U, Flynn C, Moreci P, et al. Schedule for affective disorders and schizophrenia for school-age children-present and lifetime version (K-SADS-PL): initial reliability and validity data. J Am Acad Child Adolesc Psychiatry 1997;36:980-988.
crossref pmid
2. Kim YS, Cheon KA, Kim BN, Chang SA, Yoo HJ, Kim JW, et al. The reliability and validity of kiddie-schedule for affective disorders and schizophrenia-present and lifetime version-Korean version (K-SADSPL-K). Yonsei Med J 2004;45:81-89.
crossref pmid
3. Townsend L, Kobak K, Kearney C, Milham M, Andreotti C, Escalera J, et al. Development of three web-based computerized versions of the kiddie schedule for affective disorders and schizophrenia child psychiatric diagnostic interview: preliminary validity data. J Am Acad Child Adolesc Psychiatry 2020;59:309-325.
crossref pmid
4. World Health Organization. Process of translation and adaptation of instruments [Internet]. Available at: http://www.who.int/substance_abuse/research_tools/translation/en/. Accessed June 1, 2021.

5. Achenbach TM, Rescorla L. Manual for the ASEBA School-Age Forms & Profiles: An Integrated System of Multi-Informant Assessment. Burlington, VT: ASEBA; 2001.

6. Wongpakaran N, Wongpakaran T, Wedding D, Gwet KL. A comparison of Cohen's Kappa and Gwet's AC1 when calculating inter-rater reliability coefficients: a study conducted with personality disorder samples. BMC Med Res Methodol 2013;13:61
crossref pmid pmc pdf
7. Ollendick TH, Raishevich N, Davis TE 3rd, Sirbu C, Öst LG. Specific phobia in youth: phenomenology and psychological characteristics. Behav Ther 2010;41:133-141.
crossref pmid pmc
8. McCarty P, Frye RE. Early detection and diagnosis of autism spectrum disorder: why is it so difficult? Semin Pediatr Neurol 2020;35:100831
crossref pmid
9. Falkmer T, Anderson K, Falkmer M, Horlin C. Diagnostic procedures in autism spectrum disorders: a systematic literature review. Eur Child Adolesc Psychiatry 2013;22:329-340.
crossref pmid pdf
10. Kobak KA, Taylor LH, Dottl SL, Greist JH, Jefferson JW, Burroughs D, et al. A computer-administered telephone interview to identify mental disorders. JAMA 1997;278:905-910.
crossref pmid
11. Brodey BB, First M, Linthicum J, Haman K, Sasiela JW, Ayer D. Validation of the NetSCID: an automated web-based adaptive version of the SCID. Compr Psychiatry 2016;66:67-70.
crossref pmid pmc
TOOLS
Share:
Facebook Twitter Linked In Google+
METRICS Graph View
  • 0 Crossref
  •   Scopus
  • 1,668 View
  • 52 Download


ABOUT
AUTHOR INFORMATION
ARTICLE CATEGORY

Browse all articles >

BROWSE ARTICLES
Editorial Office
#522, 27, Seochojungang-ro 24-gil, Seocho-gu, Seoul 06601, Korea
Tel: +82-2-717-0892    E-mail: psychiatryinvest@gmail.com                

Copyright © 2024 by Korean Neuropsychiatric Association.

Developed in M2PI

Close layer
prev next