INTRODUCTION
Diagnosing mental disorders in children and adolescents could be very challenging. A thorough assessment procedure is needed for identifying their symptoms and classifying the symptoms into a specific diagnosis. The Kiddie Schedule for Affective Disorders and Schizophrenia for school-age children (KSADS) is one of the most widely used diagnostic instrument for assessing psychiatric diagnoses in children and adolescents [1]. KSADS has been translated into more than 30 languages and has been well-validated. The Korean version of KSADS has also been translated and validated [2]. KSADS has been used extensively in clinical practice and research.
KSADS-COMP is the recently updated, computerized version of the paper-and-pencil KSADS [3]. KSADS-COMP was updated to reflect Diagnostic and Statistical Manual for Mental Disorders fifth edition (DSM-5) diagnostic criteria. KSADS-COMP adopted an automated scoring algorithms which generates both categorical diagnoses and diagnosis-specific dimensional rating scales of current symptoms. Also, the scoring criteria were modified to use the same standardized 5-point rating scale for all questions regarding current symptoms [3].
KSADS-COMP has already been translated into several languages including Spanish, Dutch, and Danish. However, the validity data of KSADS-COMP was reported only in one study by the original developers [3]. The purpose of the present study was to develop the Korean version of the clinician-administered KSADS-COMP and examine the validity in Korean children and adolescents.
METHODS
Translation & development of the Korean KSADS-COMP
With the permission from the original developer, the webbased computerized versions of KSADS-COMP were translated into Korean by child-adolescent psychiatrists who were bilingual in English and Korean. Then a panel of experts which included child-adolescent psychiatrists and clinical psychologists who were also bilingual reviewed the translation. Revisions were made under the consensus of the original translators and the expert panel. Then, the translation was backtranslated into English by a professional native translator who was blind to the original sources of the measure before or during back-translation. The back-translations were sent to another psychiatrist who was bilingual to address any discrepancies between the original version and the back-translated version. The translation was proofreaded several times by the expert panel and the original translators. World Health Organization (WHO) guidelines on translation were followed during this process [4].
Participants
Children and adolescents from 6 to 18 years who were referred to the department of psychiatry in two hospitals—Ajou University Hospital and Soonchunhyang University Bucheon Hospital— were recruited from November 2021 to December 2021. Children and adolescents who presented with any kind of psychiatric symptoms such as depressed mood, anxiety or irritability and who agreed to participate in the study were included in this study. Informed consent and written assent were obtained by all parents and children, respectively. The study was approved by the Institutional Review Board of Ajou University Hospital (AJIRB-SBR-SUR-21-459) and Soonchunhyang University Bucheon Hospital (SCHBC 2021-10-0002). The trial was registered at the Clinical Research Information Service (CRIS) of Republic of Korea (Registration Number KCT0006794).
Procedures
An experienced child-adolescent psychiatrist established a clinical diagnosis according to the DSM-5 for the participant after a thorough psychiatric interview with the participant and the parent. Then researchers who were blind to the diagnoses administered the clinician-administered KSADS-COMP to the parents and participants. Researchers, comprised of psychiatric residents and clinical psychologists, were trained for the administration of the clinician version of the KSADS-COMP. The clinician-administered KSADS-COMP was completed within two weeks after the initial diagnostic interview with the child-adolescent psychiatrist.
Measures
Clinician-Administered KSADS-COMP
There are three versions of KSADS-COMP: a clinician-administered version, a self-administered youth version, and a self-administered parent version [3]. The clinician-administered KSADS-COMP is the computerized counterpart of the paper and pencil KSADS. It is a semi-structured diagnostic instrument which includes an introductory interview, the screening interview, and the supplement interview. The introductory interview obtains background information like life circumstances, adaptive functioning, treatment history and family history. The screening interview screens 2 to 4 main symptoms of each psychiatric diagnosis. The supplement interview is for the diagnosis of the disorders with above threshold scores from the screening interview. Also, self-administered pre-interview is available for both the parent and the youth to be completed before administering the clinician-administered version.
To establish interrater reliability, all six researchers from the two hospitals scored all the items on the clinician-administered KSADS by watching a mock patient interview that was video-recorded. Among 160 questions, researchers agreed on the same items for 97.5% (156 out of 160). Diagnostic concordance was 100% between the six researchers.
The parent interview was completed first if the youth was a child and the youth interview was completed first if the youth was an adolescent. After the parent and youth interview were done, final diagnoses were generated after the consensus interview which integrated all the information from the parent and youth interviews. When the researchers administered the consensus interview, both answers from the parent and the youth interviews were shown on the top of the screen.
Child Behavior Check List
Child Behavior Check List (CBCL) is a standardized parental questionnaire widely used to identify childhood emotional and behavioral problems [5]. The CBCL is comprised of 113 items which are divided into 8 syndrome subscales (withdrawn/depressed, somatic complaint, anxious/depressed, social problems, thought problems, attention problems, rulebreaking behavior, and aggressive behavior). CBCL also has a set of DSM-oriented scales: affective problems, anxiety problems, somatic problems, attention deficit hyperactivity disorder (ADHD) problems, oppositional defiant problems, conduct problems, obsessive compulsive disorder (OCD) and posttraumatic stress disorder (PTSD).
Statistical analysis
To assess the criterion validity of the KSADS-COMP, diagnoses made by the clinician-administered KSADS-COMP were compared to the gold-standard diagnoses made by the child-adolescent psychiatrists. Percent agreement, Cohen’s Kappa, Gwet’s first-order agreement coefficient (AC1), sensitivity, specificity, positive predictive value, and negative predictive values were calculated. Gwet’s AC1 was our preferred measure of agreement because it was proved to be better than Cohen’s Kappa in the previous studies [6]. T-statistics and its p-value for the test of the null that Gwet’s AC1 equals zero were calculated.
To assess the convergent validity of the clinician-administered KSADS-COMP, scores of the corresponding CBCL DSM-oriented subscales were compared between the participants with positive diagnoses and those without positive diagnoses generated by the clinician-administered KSADS-COMP. Five diagnostic categories with high frequencies were selected for the analyses for the convergent validity of KSADS-COMP: depressive disorder, anxiety disorder, OCD, ADHD, and oppositional defiant disorder (ODD). Tic disorder was also one of the diagnoses with high frequencies but was excluded because there was no corresponding CBCL subscales to compare with. Wilcoxon signed rank tests were used.
RESULTS
A total of 71 participants (mean age=12.04±3.86 years, female=29.57%) participated in the study. The clinical diagnoses of the participants made by the child-adolescent psychiatrists are shown in Table 1. Among the participants, 97.18% of the participants received a psychiatric disorder with only two participants with no diagnosis. ADHD was the most prevalent diagnosis (n=43). Tic disorders (n=23), OCD (n=22), depressive disorders (n=21), anxiety disorders (n=21), and ODD (n=19) also showed high prevalence.
The gold-standard diagnoses made by the child-adolescent psychiatrists were compared to the current diagnoses generated by the clinician-administered KSADS-COMP (Table 1). Gwet’s AC1 showed great range between 0.78 and 1. Sensitivity, specificity, positive predicted value, and negative predictive value also showed high scores. Low scores were observed in specific phobia and autism spectrum disorder (ASD). Specific phobia showed positive predictive value of 45 and ASD showed sensitivity value of 50.
Table 2 shows the mean and standard deviations of the CBCL subscale scores for depression, OCD, ADHD, ODD, and anxiety for the participants who did and did not meet criteria for the corresponding current diagnoses generated by the clinician-administered KSADS-COMP. Wilcoxon rank-sum tests indicated that for all these disorders, participants who were diagnosed with a positive diagnosis by the clinician-administered KSADS-COMP scored significantly higher on the corresponding DSM-oriented scales on CBCL than those who were not diagnosed as having that diagnosis.
DISCUSSION
The current study demonstrated excellent criterion validity of the Korean version of the clinician-administered KSADS-COMP. This was the first study to examine the criterion validity of the KSADS-COMP, which is comparing results generated from a test measure to those obtained from a criterion, or goldstandard measure. The diagnostic concordance rates between the gold-standard diagnoses made by the child-adolescent psychiatrists and the current diagnoses generated by the clinician-administered KSADS-COMP calculated by Gwet’s AC1 were statistically significant. High positive predictive values and high negative predictive values were shown in all diagnoses except for the relatively low positive predictive value observed in specific phobia. High sensitivity and specificity values were observed in all disorders except for the relatively low sensitivity shown in ASD.
Low positive predictive value was observed in specific phobia. Thus, percentage of participants who were diagnosed with specific phobia in the clinician-administered KSADS-COMP who actually have specific phobia was low. Specific phobia was the only diagnosis that had a much higher prevalence in KSADS-COMP than in gold-standard diagnoses in this study. It could mean that KSADS-COMP has a rather low threshold for diagnosing specific phobia than in clinical practice. On the other hand, it could also imply that child-adolescent psychiatrists could have left out specific phobia unintentionally when they were evaluating the participant, since it is not one of the main psychiatric diagnoses. It has been reported that specific phobias often go unidentified and left untreated in youth [7]. Given the more thorough and comprehensive coverage of the KSADS-COMP, KSADS-COMP may generate a more detailed diagnosis without overlooking any minor symptoms compared to a routine unstructured psychiatric interview by an expert.
Low sensitivity value was observed in ASD. Thus, the ability of clinician-administered KSADS-COMP to correctly diagnosis the participant as having ASD was insufficient in this study. ASD is a very heterogenous disease that is notoriously difficult to diagnose [8]. The gold-standard diagnostic tools are Autism Diagnostic Observational Schedule (ADOS) which is evaluating the patient and Autism Diagnostic Interview (ADI) which is interviewing the parent. [9] The information gathered in both tests enable the clinician to diagnosis ASD more accurately. In KSADS-COMP, questions related to ASD were only included in the parent interview. Therefore, the ability of KSADS-COMP in accurately diagnosing ASD needs further evaluation.
There are several limitations of the study that needs to be addressed. First, the relatively small sample size could be the limitation of the current study. Except for depressive disorders, anxiety disorders, OCD, ADHD, ODD, and tic disorders, the prevalence of the disorders were low. Second, the sample was comprised of clinically referred patients and 97.18% of the participants were diagnosed with a psychiatric disorder. Positive predictive value for no diagnosis showed low score of 40 in this study. Therefore, the ability of the diagnostic tool to discriminate between the disordered and the non-disordered participants needs further examination. Third, the gold-standard diagnosis used in the study was the clinical diagnoses made by a single experienced child-adolescent psychiatrist. Consensus diagnoses among multiple experts could have enhanced the reliability of the gold-standard diagnosis.
Clinician-administered KSADS-COMP has many advantages over the previous KSADS. In the previous study which validated the Korean version of the paper-and-pencil version of KSADS, Cohen’s Kappa values ranged from 0.241 to 0.695 [2]. Cohen’s Kappa values in the current study ranged from 0.58 to 0.94. Therefore, clinician-administered KSADS-COMP could have a better diagnostic accuracy than the former version. It is consistent with previous studies which reported that computerized versions of structured diagnostic instruments showed better psychometric properties compared to the paper-and-pecnil versions [10,11]. Also, clinician-administered KSADS-COMP could be much readily usable than the former pencil-and-paper version. It is faster, because of its automated scoring and reports that are ready as soon as you finish your administration. It would also require less training and reduce data-entry errors. Because large numbers of detailed response sets can be automatically collected in the database and later downloaded, it could bring benefits in studies on large study population.
In conclusion, the current study developed and validated the Korean version of the clinician-administered KSADS-COMP. The Korean version of the clinician-administered KSADS-COMP showed great psychometric properties. Due to its readily usable format and efficient diagnostic process, KSADS-COMP is expected to be even more widely used in research and clinical settings than the previous versions of KSADS. Future study could use a larger study population with a broad spectrum of the disordered and non-disordered participants.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation
Print
