Validity and Reliability of the Behavior Development Screening for Toddlers-Questionnaire/Parents(BeDevel-Q/P): A Korean Autism Screening Instrument for Infants aged 24–35 Months

Article information

Psychiatry Investig. 2020;17(1):47-54
Publication date (electronic) : 2020 January 25
doi : https://doi.org/10.30773/pi.2018.0271
1Department of Rehabilitation, Hanshin University, Osan, Republic of Korea
2Sewon Infant & Child Development Center, Seoul, Republic of Korea
3Department of Psychiatry, Seoul National University Bundang Hospital, Seongnam, Republic of Korea
4Department of Psychology, Sungshin University, Seoul, Republic of Korea
Correspondence: Young Il Cho, PhD Department of Psychology, Sungshin University, 2 Bomun-ro 34 Da-gil, Seongbuk-gu, Seoul 02844, Republic of Korea Tel: +82-2-920-7593, Fax: +82-2-920-2040, E-mail: yicho@sungshin.ac.kr
Received 2019 January 10; Revised 2019 July 7; Accepted 2019 September 29.

Abstract

Objective

This study tested the validity and reliability of the Behavior Development Screening for Toddlers-Questionnaire-Parents (BeDevel-Q/P), a new autism spectrum disorder (ASD) screening instrument being developed in South Korea. The parents of 24–35-month-old infants were recruited to complete the questionnaire.

Methods

The participants were 791 infants aged 24–35 months. There were 623 typically developing infants, 88 infants with ASD, and 80 developmentally delayed infants. For test-retest, the participants were surveyed every 1–4 weeks. Participants were recruited nationwide. Subjects’ parents completed the BeDevel-Q/P and concurrent validity questionnaires. The data were used for statistical analysis.

Results

A total of 24 items consisting of 16 items from factor 1 (F1), 6 items from factor 2 (F2), and 2 items from factor 3 (F3), were selected for the final BeDevel-Q/P items.

Conclusion

The factors of the screening instrument developed in this study were analyzed, and three factors were extracted, confirming the theoretical foundation of the BeDevel-Q for the parents of 24–35-month-old infants.

INTRODUCTION

Autism spectrum disorder (ASD) is a neurodevelopmental disorder which include verbal and nonverbal communication difficulty, qualitative social interaction disorder, stereotyped behaviors, and limited areas of interest [1]. Until recently, there had not been a systematic report on ASD prevalence in South Korea. However, a 2011 survey attracted attention from South Korean clinicians and parents, reporting a high prevalence (2.64%) of ASD among 7–12-year-old children [2]. ASD symptoms are observed from early childhood. Early diagnosis and intervention are considered important because, with early intervention, later developmental presentation is known to be different [3]. It has been reported that the earlier the intervention through early screening, the better the prognosis for the child’s language, emotional, behavioral, and academic development. Infants receiving ASD diagnosis and intervention after three years of age have been found not to benefit in terms of brain development plasticity. Given the known effectiveness of early screening and intervention for ASD, research on screening instruments for early childhood ASD was launched as a national project in South Korea in 2014 [4].

The process of ASD-related early recognition, screening, and intervention has tended to be slow in South Korea [5,6]. According to one South Korean study, among various types of disorders, ASD showed the largest difference between the time of discovery by parents and the time of diagnosis; further,a lack of assessment instruments for developmentally disabled infants was reported to pose significant difficulties at clinical sites [7]. A study of 84 young (one to three years) children with ASD was investigated the major problems reported by the parents. In terms of the initial recognition of the problems, 28.3% of the parents recognize the problems before age of one year, 28.3% between 13 to 18 months [8]. The reason why the problems were recognized was delayed language development among 36.2% of the infants and no eye contact among 27.6% of the infants. The major complaint was delayed language development (56%) followed by developmental delay (11%). In summary, in South Korea, delayed language development is a major symptom that causes the parents of infants with ASD to recognize developmental abnormalities. Cross-cultural comparisons are especially important for ASD, since cultural differences may impact the perception of symptoms.

In different countries including the United States, United Kingdom, Germany, and Australian, extensive research on the behavioral indicators of early childhood ASD has been conducted, and reliable and valid screening instruments have been developed. Representative diagnostic instruments include the Autism Diagnostic Observation Schedule-2 (ADOS-2), the Autistic Diagnostic Interview-Revised (ADI-R) and The Childhood Autism Rating Scale (CARS) [9-12]. Examples of existing screening instruments are The Modified Checklist for Autism in Toddlers (M-CHAT) [13], and the Social Attention and Communication Study (SACS) [14], the Child Behavior Checklist for Ages 1.5–5 (CBCL 1.5–5), the Infant-Toddler Social and Emotional Assessment (ITSEA).

Some studies have suggested that the identification of toddlers with ASD using cutoff scores may be ineffective because scores or critical items can vary across cultures. The screening instrument for ASD’s cultural validity has not been tested for South Korea. The M-CHAT have been noted regarding cultural and linguistic differences between the US, where the instrument was developed, and areas such as Spain, Sweden, India, and Sri Lanka [15]. The previous study about a Japanese version of the M-CHAT (M-CHAT-JV) showed adequate discrimination with critical items similar to those of the original instrument. In addition, a critical item from the original M-CHAT was not useful, as all toddlers later diagnosed with ASD received passing scores on this item [15]. The SACS have suggested that some key markers showing significant difference between Australian, where the instrument was developed, and South Korea [16].

The most screening tools have been developed in the United States or United Kingdom. Though South Korea has used translated foreign autism screening instruments, their clinical usefulness has not been established because of a lack of research on infants actually diagnosed with ASD. Korean parents have failed to recognize the autistic behaviors described in questionnaire items as abnormal. Other studies have similarly found that when using ASD screening tools developed in the West, parents from non-Western cultures might understand behaviors or disorders differently. Furthermore, current autism screening instruments require infants to be at least 18 months old, and the instruments are used for a wide range of ages when children are 18 months or older. However, because the initial manifestation of ASD is diverse, the trend is to develop instruments only for a specific age or instruments differentiated by age group to increase the accuracy of screening overseas. Furthermore, considering that South Korean parents recognize abnormalities in ASD infants’ development prior to 18 months, and some screening tools are administered at a specific age and many at risk children may be missed due to their age. For example, the M-CHAT is used for 18–30-month-old children. There is a need to develop Korean ASD screening instruments that can be used at an early age [17-19].

Accordingly, research on the development of an instrument for screening South Korean infants with ASD in early childhood is being conducted as a national project. Screening instruments are being developed that can be used to identify the main behavioral characteristics of infants with ASD that South Korean parents are able to recognize while also considering risk indicators by age. Among the Behavior Development Screening for Toddlers-Questionnaire/Parents (BeDevel-Q/P) currently being developed in South Korea, this study tested the validity and reliability of an instrument developed for the parents of 24–35 month-old infants.

METHODS

Participants and procedure

The participants were 791 infants aged 24–35 months. There were 623 typically developing infants, 88 infants with ASD, and 80 developmentally delayed infants. Regarding gender distribution, 473 (59.7%) participants were male and 317 (40.3%) female. Information about this study was sent to the child psychiatry departments of hospitals, clinics, and childcare centers, and the requests for participation were accepted. Written consent for participation was collected from the guardians of all participants prior to the study. The typically developing infants were from childcare centers where permission had been obtained from the director. With the cooperation of the directors of the childcare centers, information about the study was sent to the participants’ guardians. The BeDevel-Q/P for the parents of 24–35-month-old infants, along with concurrent validity questionnaires, were sent to each participant’s home only if the guardian had submitted a participation consent form. Parents completed the BeDevel-Q/P and concurrent validity questionnaires at home, sealed them in an envelope, and submitted them to their childcare center. Subsequently, the childcare center mailed the sealed questionnaires to the researchers. The total number of infants with ASD was 88, and 79 (89.7%) had been diagnosed by their hospitals or pediatric psychiatrists at clinics based on the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5). The parents of the infants with ASD were informed about the present study and were asked to visit hospitals or research institutions and complete the BeDevel-Q/P only if they consented to participate in the study. Nine infants (10.3%) in the ASD group were recruited through childcare centers. They underwent confirmatory diagnosis and were diagnosed with ASD according to the DSM-5 if they came under one of the following three selection criteria and their parents had agreed to participate in ASD diagnosis at a clinic: they exceeded the preliminary cutoff point of the BeDevel-Q/P for the parents of 24–35-month-old infants, failed two or more of the six key items of the M-CHAT, or failed three or more of the 23 M-CHAT items. The 80 developmentally delayed infants were diagnosed as having developmental delays by a pediatric psychiatrist or clinical psychologist at a hospital or clinic. The 80 developmentally delayed infants were 315.8, according to DSM-5. The developmentally delayed infants in the present study showed severe delays of two standard deviations from the mean or more in two or more developmental areas of the Mullen Scales of Early Learning. When the parents of the developmentally delayed infants visited the hospital or clinic, they completed and submitted the BeDevel-Q/P. All infants in both the ASD group and the developmentally delayed group were diagnosed with ASD or developmental delay at hospitals or clinics using the ADOS-2, parent interviews regarding developmental characteristics, CARS, and the Mullen Scales of Early Learning. For test-retest, the participants were surveyed every 1–4 weeks. Participants were recruited nationwide. The present study was conducted after obtaining approval (P01-201703-22-004) from the Institutional Review Board.

Questionnaire

Behavior Development Screening for Toddlers-Questionnaire (BeDevel-Q)

The BeDevel-Q is a questionnaire developed in the present study for 9–47-month-old infants for the early screening of ASD. The BeDevel-Q’s items are related to 1) a social communication and social interaction disorders and 2) limited and repetitive behaviors, which are the ASD diagnostic criteria in the DSM-5. Additionally, since a previous study found that South Korean parents recognized ASD through abnormalities in language development, the questionnaire includes an item related to language development. The BeDevel-Q consists of the BeDevel-Q/P (Parents), which is completed by parents or caregivers, and the BeDevel-Q/T (Teachers), which is completed by teachers. Moreover, the BeDevel-Q is divided by age into questionnaires for infants aged 9–11, 12–23, 24–35, and 36–47 months. The BeDevel-Q/P for 24–35-month-old infants used in the present study comprises 33 items: 31 items based on the ASD diagnostic criteria in the DSM-5 and two items related to language development. The questionnaire is designed such that parents answer “Yes” or “No” to each item after observing their infants’ behavior for two weeks. The higher the summed score, the more severe the ASD-related symptoms.

Korean version of the Childhood Autism Rating Scale (K-CARS)

K-CARS is based on CARS and was standardized by Shin and Kim [20]. It is a 15-item, observation-based rating scale designed to accurately differentiate infants with ASD from those with developmental delays without autism features. Each of the 15 items is rated on a four-point scale ranging from “within normal limits for that age,” which is coded as 1, to “severely abnormal for that age,” which is coded as 4 [20,21].

Brief Infant-Toddler Social and Emotional Assessment-Autism (BITSEA-A)

BITSEA is a 42-item questionnaire [22]. It was designed as a screening tool for identifying social-emotional and behavior problems and/or delays or deficits in infants aged 12–36 months. In addition to the 42 items, the BITSEA includes two questions on parental worries regarding their child’s behavior and emotions or relationships (BITSEA-A). Parents’ answers to the BITSEA-A items were used to identify autism. The questionnaire is rated on the following scale: 0=not true/rarely, 1=somewhat true/sometimes, and 2=very true/often.

Modified Checklist for Autism in Toddlers (M-CHAT)

M-CHAT is an autism screening instrument usable for 18–30-month-old infants with the parent report items of the CHAT expanded to 23 items. The questionnaire has a total of 23 items. Six of these are considered core items: “interest in peers,” “pointing behavior that shows interest,” “voluntary social initiation behavior,” “imitation behavior,” “response to name calling,” and “joint attention behavior.” The child’s primary caregiver evaluates each question as “Yes” or “No”. The screening criteria are failing two of the six core items listed above or three of the total 23 items [23].

Child Behavior Checklist for Ages 1.5–5 (CBCL 1.5–5)

CBCL 1.5–5 was developed by Achenbach and Rescorla and standardized by Oh and Kim [24]. It is composed of the problem behavior syndrome scale and the DSM diagnosis scale. The problem behavior syndrome scale includes emotional reactivity, anxiety and depression, physical symptoms, withdrawal, attention problems, aggressive behavior, sleep problems, other problems, total score of internalization, total score of externalization, and total score of problem behavior. The DSM diagnosis scale includes DSM emotional problems, DSM anxiety problems, DSM pervasive developmental problems, DSM attention-deficit hyperactivity disorder (DSM ADHD), and DSM oppositional behavior problems. The reliability coefficient, Cronbach’s α, obtained in the standardization study was 0.94 for the total score of the problem behavior syndrome scale while the reliability coefficients of the other scales ranged from 0.56 to 0.89 [24,25].

Analysis method

The present study followed the exploratory factor analysis procedures of Lee et al. [26] and also referenced Fabrigar and Wegener [27]. The tools used for analysis were SPSS (version 23.0, IBM Corp., Armonk, NY, USA), Factor 10.8, Mplusn 8.1 [28,29].

RESULTS

Frequency analysis

This study had 791 participants (Table 1). Their distribution by residential area was as follows: Jeolla-do, 256 (32.4%); Gyeongsang-do, 139 (17.6%); Seoul, 128 (16.2%); Chungcheong-do, 90 (11.4%); Incheon, 33 (4.2%); others, 8 (1.0%); Gangwon, 1 (0.1%); and no response, 34 (4.3%). When the infants were categorized into four groups by age, from 24 months to 35 months, in three-month intervals, the number of infants in each group was similar.

Demographic characteristics of the 24–35-month-old infants (N=791)

Determination of factor number and calculation of the basic structure

Correlations among items were determined by obtaining tetrachoric correlation coefficients for the 33 items. There was no item that had too strong a correlation between the items (0.8 or higher). The highest tetrachoric correlation coefficient was 0.60 between V8 and V10. It was determined that factor analysis could be performed on the sample given that the Bartlett’s chi-squared test value was 5772.2 (df=582; p<0.001), and the Kaiser-Meyer-Olkin test value was 0.89633.

A parallel test was performed to determine the number of factors. Specifically, parallel analyses based on minimum rank factor analysis (PA-MRFA) was performed; this is based on MRFA, which limits the eigen values to positive numbers [30]. In addition, a parallel test based on polychoric correlation was performed. The results of the parallel analysis indicated that two factors were appropriate because the eigenvalue distribution of random data (5.7) was larger than that of the research data for the third factor (4.7).

Next, a screen test was performed [31]. The screening test showed differences in the eigenvalues of factors 1, 2, 3, and 4. Their difference values were 8.50, 5.02, and 0.51. This can be interpreted as supporting three factors according to the sharp drop and leveled-off methods. The examination of model fit indicated that the model fit index of the three-factor model was appropriate (χ2=505.490, df=432, p=0.008, CFI=0.992, TLI=0.990, RMSEA=0.021, SRMR=0.075). Since the parallel and screen tests produced different results, the theoretical grounds along with the statistical tests were considered, and the three-factor model was selected. Because of differences between the results of the theoretical factor classification and the factor classification by exploratory factor analysis, items V2, V14, V17, V20, V23, V25, V29, V30, and V32 were deleted. Subsequently, a total of 24 items were analyzed.

Validity

Confirmatory factor analysis was performed to test construct validity with the three factors and 24 items. The results showed that the model fit index was within the acceptable range (χ2=300.380, df=249, p=0.014, CFI=0.978, TLI=0.975, RMSEA=0.023, SRMR=0.098). Table 2 presents the items for each factor and the factor loadings. The average variances extracted (AVEs) for testing convergent validity were 0.59, 0.54, and 0.81 for F1, F2, and F3, respectively. Following Fornell and Larcker’s [32] criteria for securing convergent validity (i.e., an AVE of 0.5 or higher), this scale is considered to have convergent validity. Correlation coefficients were used to determine the discriminant validity of the confirmatory factor analysis [33]. As the criterion for discriminant validity, Kline suggested that the correlation coefficient of each factor should be 0.9 or lower [34]. The correlation coefficients among F1, F2, and F3 were between 0.46 and 0.69, thus confirming discriminant validity among the factors according. Clear positive correlations were found between the BeDevel-Q/P and the BITSEA, M-CHAT, and CARS. Clear positive correlations were found between the BeDevel-Q/P and the BITSEA, M-CHAT, and CARS.

Confirmatory factor analysis of the BeDevel-Q for the parents of 24–35-month-old infants

Reliability

All 791 participants were used to test the reliability of the BeDevel-Q for the parents of 24–35-month-old infants. Internal consistency was calculated for the entire scale and for each factor. Cronbach’s α for the 24 items, 16 items of factor F1, 6 items of factor F2, and 2 items of factor F3 were 0.830, 0.823, 0.761, and 0.690, respectively. The test-retest reliability of the BeDevel-Q/Pwas estimated with 18 samples. Test-retest reliability was found to be high, with a statistically significant correlation coefficient of 0.919 at the significance level of 0.01.

Selection of the final measurement tool

A total of 24 items, consisting of 16 items from factor F1, 6 items from factor F2, and 2 items from factor F3, were selected for the final BeDevel-Q/P. The distribution of scores ranged from 0 to 24 points, and the interpretation of the score was that the higher the total score, the higher the risk of ASD. Table 3 presents the final items.

Item content

DISCUSSION

This study developed a three-factor questionnaire by adding a factor related to language development to the two areas of the ASD diagnostic criteria of the DSM-5. This was based on a previous study that found that the main reason South Korean parents recognized ASD in their infants was abnormal language development. The factors of the screening instrument developed in this study were analyzed, and three factors were extracted, confirming the theoretical foundation of the instrument.

According to current diagnostic systems, ASD is diagnosed in the same manner around the world. It appears, however, that differences are likely to arise cross-culturally because off actors such as “when a symptom is perceived, by whom, and what behavior is noticed first, as well as whether it is perceived as problematic.” This is consistent with the fact that existing autism screening instruments are either composed of two or three areas presented as the criteria of the DSM-IV-TR and DSM-5 for ASD diagnosis or composed of factors reflecting the major behaviors parents recognize as autistic in a given culture in addition to the areas presented in the DSM-5. Specifically, the Baby and Infant Screen for Infants with Autism Traits (BISCUIT), an autism screening instrument for 17–37-month-old infants, is composed of three factors—socialization/nonverbal communication, repetitive behaviors/restricted interest, and verbal communication—which is similar to the presentstudy [35-39].

The first factor is composed of 16 items that represent the ASD diagnostic criterion “A. Social communication disorders and social interaction disorders” in the DSM-5. The theoretical validity of the items belonging to the first factor was demonstrated in that they relate to reduced social smiling, reduced response to name, and reduced social interest, which are reported as characteristic behavioral indicators of deficiencies in social communication and social interactions in ASD. Nonverbal communication behaviors such as eye contact, joint attention, and gestures that are specifically used in social interactions were excluded in the present study. This was because of a finding in a qualitative study conducted during the development of this instrument to identify the recognition indicators of ASD characteristics in infants. It showed that parents initially responded “yes” to questions about “eye contact” and “joint attention” but changed their answer to “does not make eye contact” when the investigator gave examples such as “uses limited eye contact during play,” “uses limited eye contact while making requests” and “avoids eye contact in social interactions.” Additionally, the M-CHAT asks separate questions for joint attention—such as “proto imperative pointing,” “proto declarative pointing” and “point following”—while this study only included “proto imperative pointing.” South Korean parents understood the proto declarative pointing of their infants with ASD as proto imperative pointing and frequently did not notice if the child was looking in the direction the other person was pointing toward. Further, parents answered that they were uncertain about questions concerning nonverbal communication behaviors such as “nodding head” and “shaking head” because they did not pay attention to whether their infants were using such gestures. This suggests cultural differences given the high failure rates in many different cultures, including 81.5% for American parents, 54.0% for Italian parents, and 64.1% for Greek parents, regarding a question asking if their 17–37-month-old infants with ASD were “able to understand the subtleness or gestures of others.” Meanwhile, in Japan, another Asian culture, “joint attention” has been reported to be a sensitive indicator for distinguishing autism, which confirms that there are cultural differences in the major indicators parents recognize as ASD related. Additionally, while specific behaviors should be evaluated as “typical” or “atypical,” studies have suggested that South Korean parents show limit at ions in accurately understanding behavioral characteristics [40,41].

The second factor is composed of items that represent “B. Limited and repetitive behavior” in the ASD diagnostic criteria in the DSM-5. Items belonging to the second factor include “resists change,” “unusual sensory interest,” and “incessant and overactive behaviors.”

The third factor has two items for language development. These questions ask about delayed spoken language and the frequency of vocalizations or verbalizations [6]. Similarly, there is an Australian ASD screening instrument that also includes language-related items since a survey indicated that delayed language development was the first sign of ASD parents recognized in children under age two. Representative linguistic abnormalities seen in ASD include repeating the same words, phrases, or sentences frequently (e.g., media commercials) with the same intonation patterns and repeating questions verbatim. These abnormalities are present when the child is in preschool, and delayed language is more prevalent in infants and toddlers. Accordingly, it may be beneficial to focus on evaluating expressive language in delayed language development, which parents can easily observe since this instrument is meant for 24–35-month-old infants [42].

The following suggestions are made for future studies. Discrepancies between the statistical results and the theoretical grounds were found in nine items. In theory, items belonging to diagnostic criteria A and B of the DSM-5 were found to be combined with the opposite factors when factor analysis was conducted. The items were selected based on theoretical grounds; however, there is a need to investigate the cause of this phenomenon by including cultural characteristics in a follow-up validation study. The present study did not collect research data from all over the country. In particular, the amount of data collected from Jeolla-do was twice that collected from Seoul. Future research requires data collection by region. The present study reported only BeDevel-Q/P 24–35 months and future data collection and analysis will be needed in the BeDevel-Q/P 9–11, 12–23, 36–47 months.

Acknowledgements

This study was supported by the Social Services R & D Project of the Ministry of Health and Welfare of Republic of Korea (grant no.: HI14C2719).

Notes

The authors have no potential conflicts of interest to disclose.

Author Contributions

Conceptualization: Kyung Sook Lee, Suk Jin Jung, Young II Cho. Data curation: Kyung Sook Lee, Suk Jin Jung. Formal analysis: Kyung Sook Lee, Young II Cho, Suk Jin Jung. Funding acquisition: Kyung Sook Lee, Hee Jeong Yoo. Investigation: Kyung Sook Lee, Suk Jin Jung. Project administration: Kyung Sook Lee, Hee Jeong Yoo. Resources: Kyung Sook Lee, Suk Jin Jung. Software: Young II Cho, Yong Woo Shin. Supervision: Kyung Sook Lee, Young II Cho. Validation: Kyung Sook Lee, Young II Cho, Yong Woo Shin. Writing—original draft: Kyung Sook Lee, Suk Jin Jung, Young II Cho, Yong Woo Shin. Writing—review & editing: Kyung Sook Lee, Young II Cho.

References

1. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders 5th Editionth ed. Washington D.C: American Psychiatric Association; 2013.
2. Kim YS, Leventhal BL, Koh YJ, Fombonne E, Laska E, Lim EC, et al. Prevalence of autism spectrum disorders in a total population sample. Am J Psychiatry 2011;168:904–912.
3. Dawson G, Rogers S, Munson J, Smith M, Winter J, Greenson J, et al. Randomized, controlled trial of an intervention for toddlers with autism: the early start Denver model. Pediatrics 2010;125:17–23.
4. Dawson G. Early behavioral intervention, brain plasticity, and the prevention of autism spectrum disorder. Dev Psychopathol 2008;20:775–803.
5. Sunwoo HJ, Noh DH, Kim KM, Kim JH, Yoo HJ. A study onpractitioner’s perceptions in early screening of autism spectrum disorder in South Korea. J Korean Acad Child Adolesc Psychiatry 2017;28:96–105.
6. Lee JY, Moon DS, Shin HS, Yoo HJ, Suh DS. A study on the status of hospital-based early intensive intervention for autism spectrum disorder in South Korea. J Korean Acad Child Adolesc Psychiatry 2017;28:213–219.
7. Korea National Institute for Special Education. 2017 Nationwide Survey of Special Education Seoul: Korea National Institute for Special Education; 2018.
8. Lee KS, Jeong SJ, Park JA, Sin UJ, Yu HJ. Factors of early screening of young infants with autism spectrum disorder. J Korean Assoc Pers Autism 2015;15:1–24.
9. Lee KS, Yoon HS, Chung HS, Yoo HJ. A study on early screening of young infants with autism spectrum disorder in Korea and its support system. J Korean Assoc Pers Autism 2015;15:95–122.
10. Lord C, Rutter M, DiLavorne PC, Risi S, Gotham K, Bishop SL. Autism diagnostic observation schedule, second edition (ADOS-2) manual (part I): modules 1-4 Torrance, CA: Western Psychological Services; 2012.
11. Lord C, Rutter M, Le Couteur A. Autism diagnostic interview revised: a revised version of a diagnostic interview for caregivers of individuals with possible pervasive developmental disorders. J Autism Dev Disord 1994;24:659–685.
12. Kwon H, Yoo H, Kim J, Noh D, Sunwoo H, Jeon Y, Bong G. Re-adjusting the cut-off score of the Korean version of the Childhood Autism Rating Scale for highfunctioning individuals with autism spectrum disorder. Psychiatry Clin Neurosci 2017;71:725–732.
13. Kleinman JM, Robins DL, Ventola PE, Pandey J, Boorstein HC, Esser EL, et al. The modified checklist for autism in toddlers: a follow-up study investigating the early detection of autism spectrum disorders. J Autism Dev Disord 2008;38:827–839.
14. Barbaro J, Dissanayake C. Developmental profiles of infants and toddlers with autism spectrum disorders identified prospectively in a community-based setting. J Autism Dev Disord 2012;42:1939–1948.
15. Soto S, Linas K, Jacobstein D, Biel M, Migdal T, Anthony BJ. A review of cultural adaptations of screening tools for autism spectrum disorders. Autism 2015;19:646–661.
16. Lee KS, Jung SJ, Cho SW, Shin YJ, Jeon MY, Ahn SI. Application of SACS-K to Korean infants and toddlers autism spectrum disorders (ASD): comparison of markers between ASD, developmental delay, and typical children. J Korean Assoc Pers Autism 2016;16:1–26.
17. Lee KS, Jung SJ, Ahn SI. Preliminary application of ITSEA for exploring characteristics of infants and toddlers with autism spectrum disorders. J Korean Assoc Pers Autism 2016;16:111–131.
18. Matson JL, Matheis M, Burns CO, Esposito G, Venuti P, Pisula E, et al. Examining cross-cultural differences in autism spectrum disorder: a multinational comparison from Greece, Italy, Japan, Poland, and the United States. Eur Psychiatry 2016;42:70–76.
19. Ozonoff S, Iosif A, Baguio F, Cook IC, Hill M, Hutman T, et al. A prospective study of the emergence of early behavioral signs of autism. J Am Acad Child Adolesc Psychiatry 2010;49:256–266.
20. Shin MS, Kim YH, et al. Standardization study for the Korean version of the Childhood Autism Rating Scale: reliability, validity and cut-off score. Korean J Clin Psychol 1998;17(1):1–15.
21. Schopler E, Reichler RJ, Renner BR. Childhood Autism Rating Scale North Carolina: TEACCH; 1988.
22. Briggis-Gowan MJ, Carter AS, Irwin JR, Wachtel K, Cicchetti DV. The brief infant-toddler social and emotional assessment: screening for social-emotional problems and delays in competence. J Pediatr Psychol 2005;29:143–155.
23. Robins D, Casagrande K, Barton M, Chen C, Dumont-Mathieu T, Fein D. Validation of the modified checklist for autism in toddlers, revised with follow-up. Pediatrics 2014;133:37–45.
24. Oh KJ, Kim YA. Korean Version of the Child Behavior Checklist for Ages 1.5-5 Seoul: Huno Consulting; 2009.
25. Achenbach TM, Rescorla LA. Manual for the ASEBA Preschool Forms and Profiles Burington: University of Vermont, Research Center on Children, Youth, and Families; 2000.
26. Lee SM, Youn CY, Lee M, Jung S. Exploratory factor analysis: how has it changed? Korean J Psychol Gen 2016;35:217–255.
27. Fabrigar LR, Wegener DT. Exploratory Factor Analysis Oxford: Oxford University Press; 2011.
28. Lorenzo-Seva U, Ferrando PJ. FACTOR (version 10.8 01). Computer Software Tarragona: UNIVERSITA TROVIRA VIRGILI; 2017.
29. Muthen LK, Muthen BO. Mplus (version 8.1). Computer software: Los Angeles: statmodel.com; 2018.
30. Timmerman ME, Lorenzo-Seva U. Dimensionality assessment of ordered polytomous items with parallel analysis. Psychol Methods 2011;16:209–220.
31. Catell RB. The Data Box: Its Ordering of Total Resources in Terms of Possible Relational Systems. In : Nesselroade JR, Catell RB, eds. Handbook of Multivariate Experimental Psychology New York: Plenum Press; 1988. p. 67–128.
32. Fornell C, Larcker DF. Structural equation models with unobservable variables and measurement error: algebra and statistics. J Market Res 1981;:382–388.
33. Kim SY. Fundamentals and Extension of Structural Equation Modeling Seoul: Hakjisa; 2016.
34. Kline RB. Principles and Practice of Structural Equation Modeling New York: Guilford Press; 2011.
35. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders 5th editionth ed. Washington D.C: American Psychiatric Association; 2013.
36. Doernberg E, Hollander E. Neurodevelopmental disorders (ASD and ADHD): DSM-5, ICD-10, and ICD-11. CNS Spectr 2016;21:295–299.
37. Daley TC. The need for cross-cultural research on the pervasive developmental disorders. Transcult Psychiatry 2002;39:531–950.
38. Matson J, Boisjoli JA, Hess JA, Willkins J. Factor structure and diagnostic fidelity of the baby and infant screen for children with autism traits-part 1 (BISCUIT-part 1). Dev Neurorehabil 2010;13:72–79.
39. Moulton E, Bradbury K, Barton M, Fein D. Factor analysis of the childhood autism rating scale in a sample of two year olds with an autism spectrum disorder. J Autism Dev Disord 2019;49:2733–2746.
40. Soto T, Kiss IG, Carter AS. Symptom presentations and classification of autism spectrum disorder in early childhood: application to the diagnostic classification of mental health and developmental disorders of infancy and early childhood (DC:0-5). Infant Ment Health J 2016;37:486–497.
41. Soto S, Linas K, Jacobstein D, Biel M, Migdal T, Anthony BJ. A review of cultural adaptations of screening tools for autism spectrum disorders. Autism 2015;19:646–661.
42. Barbaro J, Dissanayake C. Prospective identification of autism spectrum disorders in infancy and toddlerhood using developmental surveillance: the social attention and communication study. J Dev Behav Pediatr 2010;31:376–385.

Article information Continued

Table 1.

Demographic characteristics of the 24–35-month-old infants (N=791)

Characteristics Classification Number of cases (persons) Percent (%) BeDevel-Q/P
Average Standard deviation
Age (months) 24–26 179 22.6 6.0 5.3
27–29 199 25.2 6.0 5.6
30–32 194 24.5 5.9 5.7
33–35 203 25.7 5.8 5.6
No response 16 2.0 - -
Gender Male 473 59.8 6.7 5.7
Female 317 40.1 4.6 5.1
No response 1 0.1 - -
Respondents Mother 728 92.0 5.8 5.6
Father 56 7.1 6.5 4.9
Grandmother 3 0.4 9.0 10.4
Others 1 0.1 19.0 -
No response 2 0.3 - -
Diagnosis Typical development 623 78.8 4.3 4.3
ASD 88 11.1 13.0 5.5
Developmental delay 80 10.1 10.3 5.7

BeDevel-Q/P: Behavior Development Screening for Toddlers-Questionnaire-Parents

Table 2.

Confirmatory factor analysis of the BeDevel-Q for the parents of 24–35-month-old infants

Factor Item Unstandardized Standard error Standardized
F1 V1 1.00 - 0.54
V3 1.58 0.27 0.86
V4 1.39 0.25 0.75
V5 1.44 0.29 0.78
V6 1.61 0.31 0.88
V7 1.93 0.18 0.51
V8 1.34 0.28 0.73
V9 1.86 0.19 0.47
V10 1.61 0.30 0.88
V11 1.64 0.31 0.89
V12 1.30 0.22 0.71
V13 1.70 0.31 0.93
V15 1.57 0.28 0.85
V16 1.45 0.26 0.79
V27 1.31 0.29 0.71
V28 1.50 0.28 0.81
F2 V18 1.00 0.42
V19 1.82 0.31 0.77
V21 1.61 0.30 0.68
V22 1.84 0.32 0.78
V24 1.88 0.31 0.80
V33 2.09 0.37 0.89
F3 V26 1.00 - 0.95
V31 0.90 0.08 0.85

F1 F2 F3

Correlations between factors F1 1
F2 0.46 1
F3 0.58 0.69 1

Concurrent validity BITSEA-AUTISM
M-CHAT
CARS
CBCL DSM pervasive developmental problems scale
0.622 0.479 0.756 0.501
N=686 N=687 N=48 N=48

The tests for all confirmatory factor analysis parameters were statistically significant at p<0.001. Since the first factor loading of each factor was fixed at 1 instead of estimating it, the standard error was not calculated. All correlation coefficients of concurrent validity were statistically significant at p=0.01. BeDevel-Q/P: Behavior Development Screening for Toddlers-Questionnaire-Parents, BITSEA: Brief Infant Toddler Socio-Emotional Assessment, M-CHAT: Modified Checklist for Autism in Toddlers, CARS: Childhood Autism Rating Scale, CBCL: Child Behavior Checklist for 1.5- 5

Table 3.

Item content

# Question F # Question F
1 Simultaneous combination of gazing and other behaviors to gain others’ attention F1 15 Interest in the friendly behaviors of peers F1
3 Responds to his/hername F1 16 Participation in social play F1
4 Responds to parents’ smiles F1 18 Obsessive behaviors F2
5 Responds to reuniting with parents F1 19 Resistance to change F2
6 Interest in parents’ activities F1 21 Obsessions with specific things F2
7 Spontaneous behaviors to gain parents’ attention F1 22 Aimless obsessive gazing F2
8 Pleasant responses to affection/attention from others F1 24 Hypervigilance F2
9 Immediate responses to joint attention from other people F1 26 Little or no speech F3
10 Various facial expressions F1 27 Verbal or vocal expression of various affective states F1
11 Responds to others’ facial expressions F1 28 Has conversations with other people F1
12 Imitates other people F1 31 Delayed language development F3
13 Points at what he/she wants F1 33 Uncontrolled dangerous or abrupt actions F2

#: the question number, F: is the factor number