Reliability and Validity of the Korean Version of the Infant-Toddler Social and Emotional Assessment
Article information
Abstract
Objective
This article examined the psychometric properties of the Korean version of the Infant-Toddler Social and Emotional Assessment (K-ITSEA).
Methods
Translation and back-translation of the K-ITSEA were conducted after obtaining a permission. Two thousand two hundred thirty six Korean community infants (1,199 boys and 1,037 girls) between the ages of 12 and 36 months (M=34.23, SD=3.80) and 90 clinical infant samples (60 boys and 30 girls) between the ages of 12 and 36 months (M=26.84, SD=6.24) participated in the present study.
Results
Confirmatory factor analyses supported the Internalizing, Externalizing, Dysregulation, and Competence domains as well as the 17 individual scales that comprise the K-ITSEA. Young children’s sex and age differences emerged for some problem and most competence scales. All domains showed adequate intrascale reliability and test-retest reliability. Scale intracorrelation analyses and associations between the K-ITSEA and Korean version of PSI, Korean version of CBCL1.5-5 supported the validity of the assessment. Comparisons of the K-ITSEA scores for the Autism Spectrum Disorder, Psychiatric Disorders and Matched control groups supported the discriminant validity of the K-ITSEA.
Conclusion
This preliminary results indicate that the K-ITSEA would be a useful assessment for detecting the early childhood’s behavior problems and competences in Korean population.
INTRODUCTION
Early childhood from birth to three years is characterized by rapid developmental change and consequently many parents and professionals believe that early social-emotional and behavioral problems are developmentally transient and likely to diminish as children grow older [1]. However an estimated 6% to 32% of 1- and 2-year-old children and approximately 7% to 24% of 2- and 3-year-old children experience significant social-behavioral problems in western society [2]. Self-reported or teacher-reported behavioral and emotional problems in Chinese children have been found to be comparable to Western samples [3,4]. Also in Korea, a prevalence rates for social-emotional behavior problems among a community children has been reported to range from 6% to 23%, and approximately 3% to 4% of them were classified to need clinical intervention seriously [5-7]. Therefore it is now clear that a significant number of very young children exhibit psychopathological conditions around the world. In addition to increasing awareness of the presence of social-emotional and behavioral problems in young children, most researchers have stressed on the influence of young children’s social-emotional and behavioral problems to their later adaptive functioning in adolescence and adulthood.
Behavior problems in early infant and toddler period would be an important marker to expect one’s future adaptation in the aspect of developmental psychopathology approach. Early-emerging social-emotional and behavioral problems persist for a year or more [8-10] and may be a barrier preventing children from receiving needed intervention services [1]. Thus, there has been growing recognition of the importance of early detection of not only developmental delays, but also socialemotional and behavioral problems in recent years [8].
Despite a remarkable progress has been made in the conceptualization of very young children’s social-emotional problems and competences and psychopathology over the past 10 to 15 years [8,11], the assessment of very young children’s behavior problems and competences has been constrained by a lack of age-appropriate measurement. Moreover comprehensive measures appropriate for social-emotional and behavioral problem screening are limited [12]. Although Child Behavior Checklist (CBCL) 1.5-5 [13] has already been developed and widely used in researches, the CBCL not being designed to address social-emotional competences is not appropriate below 18 months of age [14] and there is a limitation for epidemiological studies related to young children’s social-emotional development and adaptation functioning because a lack of providing informations of their competences. Gathering social-emotional developmental informations from not only behavior problems but also competences is important in following several reasons presented by Carter et al. [8] First, competence in stage-salient tasks minimizes the emergence of new and maintenance of existing maladaptive patterns of behavior [15,16]. Second, assessing both positive and negative aspects of young children may minimize parental response biases [15,17]. Finally, identifying children’s social-emotional competences may facilitate the design of interventions that capitalize on their strengths [8].
The Infant-Toddler Social and Emotional Assessment (ITSEA) was developed as developmentally sensitive and empirically research-based clinical tool for identifying social-emotional strengths and weaknesses that may be areas of concern in children ages 12–36 months [14,17,18]. The ITSEA is designed as a comprehensive, multidomain, adult-report assessment of social-emotional and behavioral problems and competences [19] in four broad domains including Externalizing, Internalizing, Dysregulation, and Competence, as well as three indices including Maladaptive, Social relatedness, and Atypical Behavior.
The ITSEA was standardized and normed based on a nationally representative sample [18] and the reliability and validity of the ITSEA scale were proved in a diverse sample of children, not only a community sample in a large representative healthy birth cohort of urban and suburban families but also early intervention sample [2,14,17-19]. The ITSEA demonstrated acceptable internal consistency, test-retest reliability, and validity relative to other parent-report checklists in an ethnically and educationally heterogeneous pediatric sample [17]. A French adaptation study of the ISTEA named the Evaluation Sociale et Emotionelle de Jeunes Enfants (ESEJE) indicated the high level of reliability and validity [20]. However, adaptation and validation is required like to a French adaptation study because cultural differences may exist in parental perception of their children’s behaviors. Researchers who study cultural influence on development insist that cultural beliefs, values, and practices should be a central focus of assessments of young children [21-23], and psychiatric diseases need to be understood in the context of an illness experience, which is in a part determined by cultural interpretations of the diseases [24]. ITSEA has been verified by standardization in China [25] and Japan [26,27] for its reliability and validity for the Asian culture and proved as a valid and reliable instrument to check socialemotional problems of the Asian infants. In the traditional Korean culture, parents view unselfish devotion and sacrifice to their children as their basic role and duty because they see their children as extensions of themselves [28] and identify their children’s accomplishment and reputation with those of them. Thus, there is a necessity to confirm that the similar factor structures, reliability and validity of the ITSEA have found in Korean population. The present study is the third replication of the ITSEA reliability and validity test in Asian population, laying the groundwork for various researches studying not only epidemiological and developmental but also clinical issues of social-emotional and behavior competences among Korean infants and toddlers.
METHODS
Participants
All the participants were infants aged from 12 to 36 months (n=2,236 from community samples). The sample comprised 1,199 boys (53.6%) and 1,037 girls (46.4%). All of their mothers ranged in age from 20 to 49 years old (M=34.23, SD=3.80), and fathers from 20 to 46 years old (M=36.68, SD=8.40). The mean age of infants was 24.33 months (SD=8.71). All of respondents were mothers and most of them were housewives (49.4%). The mother’s median level of education was completion of a 4-year college degree (71.1%) and 14.2% of the sample had a high-school education or less. The father’s median level of education was completion of a 4-year college degree (70.0%) and 14.3% of the sample had a high-school education or less. The modian income cetegory was between 2,000,000 Korean Won to 5,000,000 Korean Won, reported by 76.8% of the sample. Ninety clinical infant samples (60 boys and 30 girls) between the ages of 12 and 36 months (M=26.84, SD=6.24) participated in the present study. Clinical samples were infants diagnosed with autism spectrum disorder (ASD) (n=47) and psychiatric disorders (n=43). Psychological tests including autism diagnostic observation schedule [29] were used to diagnose the clinical samples. Diagnoses of ASD and psychiatric disorders were confirmed by board-certified child and adolescent psychiatrists according to the Diagnostic and Statistical Manual of Mental Disorders, 4th edition (DSM-IV) [30].
Subjects of the K-ITSEA standardization were collected from the research project to develop a community-based mental health improvement model for infants. We provided presentations for the project at 53 daycare centers and kindergartens in Seoul, Gwangmyung, Busan, Changwon, and Incheon. They were selected in consideration of regional characteristics under the support of district offices and public health centers. Research brochures and questionnaires were first distributed to teachers of the daycare centers and kindergartens who agreed with the research purposes and applied for participation in the research and later delivered to parents of children by the teachers. When parents agreed to participate in the project after reading the brochure, they signed the consent form and answered the questionnaire which were sealed and collected by the researchers. The collection rate was 84% of the total community samples. The test-retest analysis was conducted by asking 30 parents to answer the questionnaire again approximately two weeks after answering the questionnaire. The 90 clinical samples were checked by clinical psychologists and child psychiatrists in Seoul.
This study was approved by the Korea National Bioethics Committee Institutional Review Board (P01-201507-23-003, P01-201703-22-004).
Measures
The Korean version of Infant-Toddler Social Emotional Assessment
The ITSEA assesses four domains of behavior including Externalizing, Internalizing, Dysregulation, and Competence with 169 items [18]. The Externalizing domain includes the Activity/Impulsivity, Aggression/Defiance, and Peer Aggression subscales. The Internalizing domain includes the Depression/Withdrawal, General Anxiety, Separation Distress, and Inhibition to Novelty subscales. The Dysregulation domain includes Sleep, Negative Emotionality, Eating, and Sensory-Sensitivity subscales. The Competence domain includes Compliance, Attention, Imitation/Play, Mastery Motivation, Empathy, and Prosocial Peer Relations subscales. In addition, Maladaptive, Atypical Behavior, and Social Relatedness subscales are included to assess more serious behavior problems that consist of low base rate, clinically significant behaviors. The core components of the ITSEA excepting Atypical and Social Relatedness subscales comprise 139 items and items rated on the following scale: 0) not true/rarely, 1) somewhat true/sometimes, 2) very true/often. A No opportunity code is provided for items that parents have not had the opportunity to observe certain behaviors (e.g., behavior with peers for a child who has no or very limited peer contact). The ITSEA questionnaire requires a fourth- to six-grade reading level and approximately 30 minutes to complete.
The Korean version of Parenting Stress Index Short Form (K-PSI-SF) [31]
The K-PSI-SF is a simplified parenting stress index developed by Lee et al. [31] for Korean parents by standardizing the Parenting Stress Index (PSI) which was developed by Abindin in 1995. Subscales consist of 36 questions in three domains about pain of parents, dysfunctional interactions, and difficult children. We used the 12-item Difficult Child subdomain of K-PSI-SF [29] to examine the validity of the K-ITSEA. Each question is measured by the five-point Likert scale: 1) strongly disagree, 2) disagree, 3) undecided, 4) agree, 5) strongly agree and the final scores are calculated by adding the point of each question. Reliability of the K-PSI-SF was Cronbach α=0.91.
The Korean version of Child Behavior Checklist for 1.5‒5
The CBCL1.5-5 [13] is a downward extension of the CBCL, a well-known scale that is widely used to assess symptoms of externalizing and internalizing behavior problems among older children, and is designed for young children from 18 months to 5 years old. CBCL1.5-5 has demonstrated very good reliability and validity results in various studies. In the present study, the Korean version of CBCL (K-CBCL) [32] was administrated to parents of young children older than 17 months. The K-CBCL1.5-5’s acceptable reliability, validity and multicultural robustness are demonstrated on the K-CBCL manual [32]. The CBCL1.5-5 is composed of 99 items and includes new DSM-Oriented scales, designed to parallel symptoms in DSM diagnostic areas [13].
Procedures
In order to conduct the ITSEA scale to Korean population, the ITEAS was translated from English to Korean by a developmental psychologist and two language linguisticians. Also our infant-mental-health clinicians reviewed the questionnaire to adapt the Korean language to be more colloquial. Back translation was conducted by a bilingual psychologist. For cultural adaptation of Korean version, infant-mental-health clinicians, psychiatrists, and daycare center teachers discussed the consistent level of scale items and identified an approach to make the scale items easy to understand.
Statistical analysis
Confirmatory factor analyses using AMOS 5.0 [33] software were performed on each form to evaluate the loading of each item on its corresponding hypothesized scale. Following the previous ITSEA research statistical procedures, Model fit was using three standard fit indices: the Root-Mean-Squared Error of Approximation (RMSEA), for which values of 0.80 or less are deemed acceptable, the Comparative Fit Index (CFI) and Turker-Lewis index (TLI), for which values greater than 0.90 are deemed acceptable [34-36]. ANOVA analyses and independent t-test were conducted to examine the effects of sex and age comparisons. We performed a Cronbach’s Alpha test [37] for internal consistency and a Pearson correlation for testretest reliability to determine the reliability of the K-ITSEA scores. To confirm the validity of K-ITSEA scale, domains (Externaling, Internalizing, Dysregulation, Competence) and subscales’s intercorrelations, correlations between the K-ITSEA and the K-CBCL1.5-5, and differences of the K-ITSEA scores between normal and clinical groups for construct validity were examined.
RESULTS
Three behavioral problem domains and all of subscales had RMSEA values below 0.10, CFI and TLI values were at close or above 0.90 (Table 1). Therefore model fit indices proved for same hypothesized structure of the K-ITSEA.
Confirmatory factor analysis model fit statistics for K-ITSEA scales (N=2,236)
Total behavior problem, the Externalizing, the Internalizing and Dysregulation behavior problems increased across age, especially at 24–29 months (Table 2). The Externalizing and the Internalizing behavior problem, and Competence domain showed significant main effects of infants’ sex. Boys were rated higher than girls in Externalizing and Atypical behavior problem subscales, whereas lower than girls in Internalizing and Competence subdomains.
Scores of K-ITSEA domains and scales by age group and sex (N=2,236)
Internal consistency and the test-retest reliabilities are shown in Table 3. Four broad domains (externalzing, internalizing, dysregulation, and competence) exhibited Alpha Coefficients very close (0.77 for externalizing domain) to and greater than 0.8. Test-retest reliability for domains ranged from 0.56 to 0.90 for scales and was excellent employing criteria recommended by Cicchetti and Sparrow [38].
Internal consistency and test-retest reliability of the KITSEA Scales
To estimate the scale intercorrelation, Pearson correlation analysis was performed and the result was presented in Table 4. In general, correlations between individual subscales (17 subscales) and their respective 4 domains (Externalizing, Internalizing, Dysregulation, and Competence) were moderate to strong. Correlations between the Externalizing domain score and the Externalizing subscales ranged from r=0.49 to r=0.93. Correlations between the Internalizing domain score and the Internalizing subscales ranged from r=0.46 to r=0.80. Correlations between the Dysregulation domain score and the Dysregulation subscales ranged from r=0.61 to r=0.74. Correlations between the Competence domain score and the Competence subscales ranged from r=0.59 to r=0.76. Finally, the correlations between Problem domains including Externalizing, Internalizing, and Dysregulation and Competence domain were significantly associated with one another. The Externalizing domain correlated significantly with both Internalizing and Dysregulation domains. Dysregulation domain also correlated significantly with the Internalizing domain. The Competence domain correlated significantly with Externalizing and Dysregulation behavior problem domains.
Intercorrelations between K-ITSEA domain and subscale scores
Correlations between K-ITSEA problem domain scores and K-PSI-SF Difficult child, K-ITSEA and K-CBCL1.5-5 externalizing, internalizing, and total behavior problems scores were significantly moderate to high, indicating relatively good agreement between the instruments (Table 5). In Competence domain, significant correlations with K-CBCL1.5-5 externalizing and internalizing behavior problems were found.
Correlations between the K-ITSEA, the K-PSI-SF, and K-CBCL1.5-5 (N=440)
To confirm the discriminant validity of the K-ITSEA, Comparisons of K-ITSEA for the ASD, and psychiatric disorders and matched control groups were conducted and presented in Table 6. Behavior problem scores of three main sub-domain of infants with ASD, and psychiatric disorders were significantly higher than those of normal matched infants.
Comparisons of K-ITSEA for the clinical and matched control groups
DISCUSSION
This study was intended to verify reliability and validity of ITSEA for the Korean culture where reliable social-emotional assessment does not exist for infants. The value of ITSEA was already proved in China and Japan as well as the U.S., France, Germany, and the Netherlands. Based on the factor analysis of the K-ITSEA, three behavioral problem domains and all of the subscales had the RMSEA value of below 0.10. CFI and NFI values were close to or above 0.90. Therefore, the model fit indices proved for the same hypothesized structure of the K-ITSEA. These results are consistent with the factor structure of Carter et al. [14], who developed ITSEA, the French version of ITSEA by Bracha et al. [20], and Zhang et al. [25] of China. Meanwhile, the K-ITSEA showed rather a high RMSEA value in the competence domain and additional indices, which is similar to the Chinese version of ITSEA by Zhang et al. [25] whose RMSEA value stood at 0.10. Thus, further in-depth crosscultural studies are required in the future not only for behavioral problems but also for competence of infants.
In terms of age and gender, total behavioral problems and externalizing, internalizing and dysregulation behavior problems increased across age, especially at 24 to 29 months. Boys were rated higher than girls in the externalizing and atypical behavior problem subscales and lower than girls in the internalizing and competence subdomains. In particular, the K-ITSEA showed similar results to those of the US, French, and Japanese versions. Boys were rated higher in externalizing behavior problems while girls were higher in internalizing behavior problems, implying that early intervention and prevention support are needed in consideration of the gender difference. In terms of age, however, findings of the K-ITSEA were different from those of the U.S. and French versions. Although behavioral problems increased among Korean infants at 24 to 29 months like Japanese infants [27], U.S. and French infants showed increased behavioral problems at 18 to 23 months. On the one hand, this could be attributed to late occurrence of behavioral problems among Asian children. On the other hand, however, it could be because fear of bias and prejudice of other people against children with behavioral problems in the Asian culture delayed the discovery of behavioral problems of young children rather than put focus on early recognition and prevention of mental health problems with young children. Since the ITSEA standardization of China failed to analyze age and gender difference, it is not allowed to confirm such age difference as a unique feature of the Asian culture. Cross-cultural studies of ITSEA in Korea, Japan, and China may provide in-depth explanation.
K-ITSEA scales also showed good reliability. Four broad domains (externalizing, internalizing, dysregulation, and competence) exhibited Alpha Coefficient very close to and greater than 0.80 (0.77 for externalizing domain). This is similar to the results of Carter and Briggs-Gowan [18], Carter et al. [14], and the Chinese version of Zhang et al. [25] and higher than the reliability of the French version by Bracha et al. [20] whose Alpha Coefficient was 0.73 in the internalizing domain.
Validity was checked by correlation analysis between subscales of the K-ITSEA, Pearson correlation analysis between ITSEA, K-PSI-SF [31] and the K-CBCL1.5-5 [32], and comparison of the K-ITSEA for the ASD, psychiatric disorders and matched control groups, proving that the K-ITSEA is valid enough to assess social-emotional problems of Korean infants. In general, correlations between individual subscales and their respective domains were moderate to strong. The K-ITSEA scales also showed good criterion-related validity with consistent association with difficult temperament and social-emotional problems of children. Correlation analysis between K-ITSEA, K-PSI-SF, and K-CBCL1.5-5 showed significant relationship between the difficult temperament of the K-ITSEA and KPSI-SF and behavioral problems of the K-ITSEA and K-CBCL1.5-5 [32]. This result is consistent with Carter and Briggs-Gowan [18], Carter et al. [14], and Zhang et al. [25], verifying that the K-ITSEA is a valid measure for social-emotional assessment of Korean infants.
This study is particularly meaningful in that validity of the K-ITSEA was verified by comparing the typically developing group in community sample, ASD psychiatric disorders group unlike the French, Chinese, and Japanese versions. Given that ITSEA has been used as a valid and reliable scale to assess ASD and developmental delay [8], the K-ITSEA also proves to be highly valid and reliable to assess children with ASD, psychiatric disorders. Thus, the K-ITSEA is expected to be used for early recognition and intervention for children with ASD or psychiatric disorders and children who need psychiatric consultation.
Although this study proves value of the K-ITSEA as a valid and reliable scale to assess social-emotional problems of Korean infants, it has the following limitations which need to be complemented for further research in the future: First, a multicultural comparative study is required for comparative analysis of the ITSEA scores between western and eastern cultures. Also, comparison of children in China, Japan, and Korea will allow in-depth analysis of clinical characteristics of young children in the Asian culture. Second, standardization of ITSEA for childcare provider is needed for assessment by childcare provider. As mentioned by Zhang et al. [25], the influence of bias or prejudice of parents for their children on the ITSEA cannot be ruled out. Although we recruited the participants consisted of typically developing infants from a community sample, there is a possibility that parents underestimated behavioral problems of children in the family-centered Asian culture where parents and children are treated as one rather than separate individuals. Third, longitudinal research is suggested for indepth verification of reliability, validity, and significance of the K-ITSEA. Significant correlation results between K-ITSEA externalizing problem and K-CBCL1.5-5 internalizing behavior problem were found same as results of Carter and Briggs-Gowan [18], Carter et al. [14], and Zhang et al. [25], A longitudinal research result using the K-ITSEA will be underway to examine occurrence of externalizing and internalizing behavior problems of infants and development trajectory of them in Korean infants. As reliability and validity of the K-ITSEA has been proved by this study, it is expected to contribute to prevention of and intervention in social-emotional problems to improve mental health of Korean infants.
Acknowledgements
This study was supported by a grant of the Korea Mental Health Technology R&D Project (HM15C1084).