INTRODUCTION
Autism spectrum disorder (ASD) is a neurodevelopmental disorder which include verbal and nonverbal communication difficulty, qualitative social interaction disorder, stereotyped behaviors, and limited areas of interest [1]. Until recently, there had not been a systematic report on ASD prevalence in South Korea. However, a 2011 survey attracted attention from South Korean clinicians and parents, reporting a high prevalence (2.64%) of ASD among 7-12-year-old children [2]. ASD symptoms are observed from early childhood. Early diagnosis and intervention are considered important because, with early intervention, later developmental presentation is known to be different [3]. It has been reported that the earlier the intervention through early screening, the better the prognosis for the child’s language, emotional, behavioral, and academic development. Infants receiving ASD diagnosis and intervention after three years of age have been found not to benefit in terms of brain development plasticity. Given the known effectiveness of early screening and intervention for ASD, research on screening instruments for early childhood ASD was launched as a national project in South Korea in 2014 [4].
The process of ASD-related early recognition, screening, and intervention has tended to be slow in South Korea [5,6]. According to one South Korean study, among various types of disorders, ASD showed the largest difference between the time of discovery by parents and the time of diagnosis; further,a lack of assessment instruments for developmentally disabled infants was reported to pose significant difficulties at clinical sites [7]. A study of 84 young (one to three years) children with ASD was investigated the major problems reported by the parents. In terms of the initial recognition of the problems, 28.3% of the parents recognize the problems before age of one year, 28.3% between 13 to 18 months [8]. The reason why the problems were recognized was delayed language development among 36.2% of the infants and no eye contact among 27.6% of the infants. The major complaint was delayed language development (56%) followed by developmental delay (11%). In summary, in South Korea, delayed language development is a major symptom that causes the parents of infants with ASD to recognize developmental abnormalities. Cross-cultural comparisons are especially important for ASD, since cultural differences may impact the perception of symptoms.
In different countries including the United States, United Kingdom, Germany, and Australian, extensive research on the behavioral indicators of early childhood ASD has been conducted, and reliable and valid screening instruments have been developed. Representative diagnostic instruments include the Autism Diagnostic Observation Schedule-2 (ADOS-2), the Autistic Diagnostic Interview-Revised (ADI-R) and The Childhood Autism Rating Scale (CARS) [9-12]. Examples of existing screening instruments are The Modified Checklist for Autism in Toddlers (M-CHAT) [13], and the Social Attention and Communication Study (SACS) [14], the Child Behavior Checklist for Ages 1.5-5 (CBCL 1.5-5), the Infant-Toddler Social and Emotional Assessment (ITSEA).
Some studies have suggested that the identification of toddlers with ASD using cutoff scores may be ineffective because scores or critical items can vary across cultures. The screening instrument for ASD’s cultural validity has not been tested for South Korea. The M-CHAT have been noted regarding cultural and linguistic differences between the US, where the instrument was developed, and areas such as Spain, Sweden, India, and Sri Lanka [15]. The previous study about a Japanese version of the M-CHAT (M-CHAT-JV) showed adequate discrimination with critical items similar to those of the original instrument. In addition, a critical item from the original M-CHAT was not useful, as all toddlers later diagnosed with ASD received passing scores on this item [15]. The SACS have suggested that some key markers showing significant difference between Australian, where the instrument was developed, and South Korea [16].
The most screening tools have been developed in the United States or United Kingdom. Though South Korea has used translated foreign autism screening instruments, their clinical usefulness has not been established because of a lack of research on infants actually diagnosed with ASD. Korean parents have failed to recognize the autistic behaviors described in questionnaire items as abnormal. Other studies have similarly found that when using ASD screening tools developed in the West, parents from non-Western cultures might understand behaviors or disorders differently. Furthermore, current autism screening instruments require infants to be at least 18 months old, and the instruments are used for a wide range of ages when children are 18 months or older. However, because the initial manifestation of ASD is diverse, the trend is to develop instruments only for a specific age or instruments differentiated by age group to increase the accuracy of screening overseas. Furthermore, considering that South Korean parents recognize abnormalities in ASD infants’ development prior to 18 months, and some screening tools are administered at a specific age and many at risk children may be missed due to their age. For example, the M-CHAT is used for 18-30-month-old children. There is a need to develop Korean ASD screening instruments that can be used at an early age [17-19].
Accordingly, research on the development of an instrument for screening South Korean infants with ASD in early childhood is being conducted as a national project. Screening instruments are being developed that can be used to identify the main behavioral characteristics of infants with ASD that South Korean parents are able to recognize while also considering risk indicators by age. Among the Behavior Development Screening for Toddlers-Questionnaire/Parents (BeDevel-Q/P) currently being developed in South Korea, this study tested the validity and reliability of an instrument developed for the parents of 24-35 month-old infants.
METHODS
Participants and procedure
The participants were 791 infants aged 24-35 months. There were 623 typically developing infants, 88 infants with ASD, and 80 developmentally delayed infants. Regarding gender distribution, 473 (59.7%) participants were male and 317 (40.3%) female. Information about this study was sent to the child psychiatry departments of hospitals, clinics, and childcare centers, and the requests for participation were accepted. Written consent for participation was collected from the guardians of all participants prior to the study. The typically developing infants were from childcare centers where permission had been obtained from the director. With the cooperation of the directors of the childcare centers, information about the study was sent to the participants’ guardians. The BeDevel-Q/P for the parents of 24-35-month-old infants, along with concurrent validity questionnaires, were sent to each participant’s home only if the guardian had submitted a participation consent form. Parents completed the BeDevel-Q/P and concurrent validity questionnaires at home, sealed them in an envelope, and submitted them to their childcare center. Subsequently, the childcare center mailed the sealed questionnaires to the researchers. The total number of infants with ASD was 88, and 79 (89.7%) had been diagnosed by their hospitals or pediatric psychiatrists at clinics based on the Diagnostic and Statistical Manual of Mental Disorders, fifth edition (DSM-5). The parents of the infants with ASD were informed about the present study and were asked to visit hospitals or research institutions and complete the BeDevel-Q/P only if they consented to participate in the study. Nine infants (10.3%) in the ASD group were recruited through childcare centers. They underwent confirmatory diagnosis and were diagnosed with ASD according to the DSM-5 if they came under one of the following three selection criteria and their parents had agreed to participate in ASD diagnosis at a clinic: they exceeded the preliminary cutoff point of the BeDevel-Q/P for the parents of 24-35-month-old infants, failed two or more of the six key items of the M-CHAT, or failed three or more of the 23 M-CHAT items. The 80 developmentally delayed infants were diagnosed as having developmental delays by a pediatric psychiatrist or clinical psychologist at a hospital or clinic. The 80 developmentally delayed infants were 315.8, according to DSM-5. The developmentally delayed infants in the present study showed severe delays of two standard deviations from the mean or more in two or more developmental areas of the Mullen Scales of Early Learning. When the parents of the developmentally delayed infants visited the hospital or clinic, they completed and submitted the BeDevel-Q/P. All infants in both the ASD group and the developmentally delayed group were diagnosed with ASD or developmental delay at hospitals or clinics using the ADOS-2, parent interviews regarding developmental characteristics, CARS, and the Mullen Scales of Early Learning. For test-retest, the participants were surveyed every 1-4 weeks. Participants were recruited nationwide. The present study was conducted after obtaining approval (P01-201703-22-004) from the Institutional Review Board.
Questionnaire
Behavior Development Screening for Toddlers-Questionnaire (BeDevel-Q)
The BeDevel-Q is a questionnaire developed in the present study for 9-47-month-old infants for the early screening of ASD. The BeDevel-Q’s items are related to 1) a social communication and social interaction disorders and 2) limited and repetitive behaviors, which are the ASD diagnostic criteria in the DSM-5. Additionally, since a previous study found that South Korean parents recognized ASD through abnormalities in language development, the questionnaire includes an item related to language development. The BeDevel-Q consists of the BeDevel-Q/P (Parents), which is completed by parents or caregivers, and the BeDevel-Q/T (Teachers), which is completed by teachers. Moreover, the BeDevel-Q is divided by age into questionnaires for infants aged 9-11, 12-23, 24-35, and 36-47 months. The BeDevel-Q/P for 24-35-month-old infants used in the present study comprises 33 items: 31 items based on the ASD diagnostic criteria in the DSM-5 and two items related to language development. The questionnaire is designed such that parents answer “Yes” or “No” to each item after observing their infants’ behavior for two weeks. The higher the summed score, the more severe the ASD-related symptoms.
Korean version of the Childhood Autism Rating Scale (K-CARS)
K-CARS is based on CARS and was standardized by Shin and Kim [20]. It is a 15-item, observation-based rating scale designed to accurately differentiate infants with ASD from those with developmental delays without autism features. Each of the 15 items is rated on a four-point scale ranging from “within normal limits for that age,” which is coded as 1, to “severely abnormal for that age,” which is coded as 4 [20,21].
Brief Infant-Toddler Social and Emotional Assessment-Autism (BITSEA-A)
BITSEA is a 42-item questionnaire [22]. It was designed as a screening tool for identifying social-emotional and behavior problems and/or delays or deficits in infants aged 12-36 months. In addition to the 42 items, the BITSEA includes two questions on parental worries regarding their child’s behavior and emotions or relationships (BITSEA-A). Parents’ answers to the BITSEA-A items were used to identify autism. The questionnaire is rated on the following scale: 0=not true/rarely, 1=somewhat true/sometimes, and 2=very true/often.
Modified Checklist for Autism in Toddlers (M-CHAT)
M-CHAT is an autism screening instrument usable for 18-30-month-old infants with the parent report items of the CHAT expanded to 23 items. The questionnaire has a total of 23 items. Six of these are considered core items: “interest in peers,” “pointing behavior that shows interest,” “voluntary social initiation behavior,” “imitation behavior,” “response to name calling,” and “joint attention behavior.” The child’s primary caregiver evaluates each question as “Yes” or “No”. The screening criteria are failing two of the six core items listed above or three of the total 23 items [23].
Child Behavior Checklist for Ages 1.5-5 (CBCL 1.5-5)
CBCL 1.5-5 was developed by Achenbach and Rescorla and standardized by Oh and Kim [24]. It is composed of the problem behavior syndrome scale and the DSM diagnosis scale. The problem behavior syndrome scale includes emotional reactivity, anxiety and depression, physical symptoms, withdrawal, attention problems, aggressive behavior, sleep problems, other problems, total score of internalization, total score of externalization, and total score of problem behavior. The DSM diagnosis scale includes DSM emotional problems, DSM anxiety problems, DSM pervasive developmental problems, DSM attention-deficit hyperactivity disorder (DSM ADHD), and DSM oppositional behavior problems. The reliability coefficient, Cronbach’s α, obtained in the standardization study was 0.94 for the total score of the problem behavior syndrome scale while the reliability coefficients of the other scales ranged from 0.56 to 0.89 [24,25].
RESULTS
Frequency analysis
This study had 791 participants (Table 1). Their distribution by residential area was as follows: Jeolla-do, 256 (32.4%); Gyeongsang-do, 139 (17.6%); Seoul, 128 (16.2%); Chungcheong-do, 90 (11.4%); Incheon, 33 (4.2%); others, 8 (1.0%); Gangwon, 1 (0.1%); and no response, 34 (4.3%). When the infants were categorized into four groups by age, from 24 months to 35 months, in three-month intervals, the number of infants in each group was similar.
Determination of factor number and calculation of the basic structure
Correlations among items were determined by obtaining tetrachoric correlation coefficients for the 33 items. There was no item that had too strong a correlation between the items (0.8 or higher). The highest tetrachoric correlation coefficient was 0.60 between V8 and V10. It was determined that factor analysis could be performed on the sample given that the Bartlett’s chi-squared test value was 5772.2 (df=582; p<0.001), and the Kaiser-Meyer-Olkin test value was 0.89633.
A parallel test was performed to determine the number of factors. Specifically, parallel analyses based on minimum rank factor analysis (PA-MRFA) was performed; this is based on MRFA, which limits the eigen values to positive numbers [30]. In addition, a parallel test based on polychoric correlation was performed. The results of the parallel analysis indicated that two factors were appropriate because the eigenvalue distribution of random data (5.7) was larger than that of the research data for the third factor (4.7).
Next, a screen test was performed [31]. The screening test showed differences in the eigenvalues of factors 1, 2, 3, and 4. Their difference values were 8.50, 5.02, and 0.51. This can be interpreted as supporting three factors according to the sharp drop and leveled-off methods. The examination of model fit indicated that the model fit index of the three-factor model was appropriate (χ2=505.490, df=432, p=0.008, CFI=0.992, TLI=0.990, RMSEA=0.021, SRMR=0.075). Since the parallel and screen tests produced different results, the theoretical grounds along with the statistical tests were considered, and the three-factor model was selected. Because of differences between the results of the theoretical factor classification and the factor classification by exploratory factor analysis, items V2, V14, V17, V20, V23, V25, V29, V30, and V32 were deleted. Subsequently, a total of 24 items were analyzed.
Validity
Confirmatory factor analysis was performed to test construct validity with the three factors and 24 items. The results showed that the model fit index was within the acceptable range (χ2=300.380, df=249, p=0.014, CFI=0.978, TLI=0.975, RMSEA=0.023, SRMR=0.098). Table 2 presents the items for each factor and the factor loadings. The average variances extracted (AVEs) for testing convergent validity were 0.59, 0.54, and 0.81 for F1, F2, and F3, respectively. Following Fornell and Larcker’s [32] criteria for securing convergent validity (i.e., an AVE of 0.5 or higher), this scale is considered to have convergent validity. Correlation coefficients were used to determine the discriminant validity of the confirmatory factor analysis [33]. As the criterion for discriminant validity, Kline suggested that the correlation coefficient of each factor should be 0.9 or lower [34]. The correlation coefficients among F1, F2, and F3 were between 0.46 and 0.69, thus confirming discriminant validity among the factors according. Clear positive correlations were found between the BeDevel-Q/P and the BITSEA, M-CHAT, and CARS. Clear positive correlations were found between the BeDevel-Q/P and the BITSEA, M-CHAT, and CARS.
Reliability
All 791 participants were used to test the reliability of the BeDevel-Q for the parents of 24-35-month-old infants. Internal consistency was calculated for the entire scale and for each factor. Cronbach’s α for the 24 items, 16 items of factor F1, 6 items of factor F2, and 2 items of factor F3 were 0.830, 0.823, 0.761, and 0.690, respectively. The test-retest reliability of the BeDevel-Q/Pwas estimated with 18 samples. Test-retest reliability was found to be high, with a statistically significant correlation coefficient of 0.919 at the significance level of 0.01.
Selection of the final measurement tool
A total of 24 items, consisting of 16 items from factor F1, 6 items from factor F2, and 2 items from factor F3, were selected for the final BeDevel-Q/P. The distribution of scores ranged from 0 to 24 points, and the interpretation of the score was that the higher the total score, the higher the risk of ASD. Table 3 presents the final items.
DISCUSSION
This study developed a three-factor questionnaire by adding a factor related to language development to the two areas of the ASD diagnostic criteria of the DSM-5. This was based on a previous study that found that the main reason South Korean parents recognized ASD in their infants was abnormal language development. The factors of the screening instrument developed in this study were analyzed, and three factors were extracted, confirming the theoretical foundation of the instrument.
According to current diagnostic systems, ASD is diagnosed in the same manner around the world. It appears, however, that differences are likely to arise cross-culturally because off actors such as “when a symptom is perceived, by whom, and what behavior is noticed first, as well as whether it is perceived as problematic.” This is consistent with the fact that existing autism screening instruments are either composed of two or three areas presented as the criteria of the DSM-IV-TR and DSM-5 for ASD diagnosis or composed of factors reflecting the major behaviors parents recognize as autistic in a given culture in addition to the areas presented in the DSM-5. Specifically, the Baby and Infant Screen for Infants with Autism Traits (BISCUIT), an autism screening instrument for 17-37-month-old infants, is composed of three factors—socialization/nonverbal communication, repetitive behaviors/restricted interest, and verbal communication—which is similar to the presentstudy [35-39].
The first factor is composed of 16 items that represent the ASD diagnostic criterion “A. Social communication disorders and social interaction disorders” in the DSM-5. The theoretical validity of the items belonging to the first factor was demonstrated in that they relate to reduced social smiling, reduced response to name, and reduced social interest, which are reported as characteristic behavioral indicators of deficiencies in social communication and social interactions in ASD. Nonverbal communication behaviors such as eye contact, joint attention, and gestures that are specifically used in social interactions were excluded in the present study. This was because of a finding in a qualitative study conducted during the development of this instrument to identify the recognition indicators of ASD characteristics in infants. It showed that parents initially responded “yes” to questions about “eye contact” and “joint attention” but changed their answer to “does not make eye contact” when the investigator gave examples such as “uses limited eye contact during play,” “uses limited eye contact while making requests” and “avoids eye contact in social interactions.” Additionally, the M-CHAT asks separate questions for joint attention—such as “proto imperative pointing,” “proto declarative pointing” and “point following”—while this study only included “proto imperative pointing.” South Korean parents understood the proto declarative pointing of their infants with ASD as proto imperative pointing and frequently did not notice if the child was looking in the direction the other person was pointing toward. Further, parents answered that they were uncertain about questions concerning nonverbal communication behaviors such as “nodding head” and “shaking head” because they did not pay attention to whether their infants were using such gestures. This suggests cultural differences given the high failure rates in many different cultures, including 81.5% for American parents, 54.0% for Italian parents, and 64.1% for Greek parents, regarding a question asking if their 17-37-month-old infants with ASD were “able to understand the subtleness or gestures of others.” Meanwhile, in Japan, another Asian culture, “joint attention” has been reported to be a sensitive indicator for distinguishing autism, which confirms that there are cultural differences in the major indicators parents recognize as ASD related. Additionally, while specific behaviors should be evaluated as “typical” or “atypical,” studies have suggested that South Korean parents show limit at ions in accurately understanding behavioral characteristics [40,41].
The second factor is composed of items that represent “B. Limited and repetitive behavior” in the ASD diagnostic criteria in the DSM-5. Items belonging to the second factor include “resists change,” “unusual sensory interest,” and “incessant and overactive behaviors.”
The third factor has two items for language development. These questions ask about delayed spoken language and the frequency of vocalizations or verbalizations [6]. Similarly, there is an Australian ASD screening instrument that also includes language-related items since a survey indicated that delayed language development was the first sign of ASD parents recognized in children under age two. Representative linguistic abnormalities seen in ASD include repeating the same words, phrases, or sentences frequently (e.g., media commercials) with the same intonation patterns and repeating questions verbatim. These abnormalities are present when the child is in preschool, and delayed language is more prevalent in infants and toddlers. Accordingly, it may be beneficial to focus on evaluating expressive language in delayed language development, which parents can easily observe since this instrument is meant for 24-35-month-old infants [42].
The following suggestions are made for future studies. Discrepancies between the statistical results and the theoretical grounds were found in nine items. In theory, items belonging to diagnostic criteria A and B of the DSM-5 were found to be combined with the opposite factors when factor analysis was conducted. The items were selected based on theoretical grounds; however, there is a need to investigate the cause of this phenomenon by including cultural characteristics in a follow-up validation study. The present study did not collect research data from all over the country. In particular, the amount of data collected from Jeolla-do was twice that collected from Seoul. Future research requires data collection by region. The present study reported only BeDevel-Q/P 24-35 months and future data collection and analysis will be needed in the BeDevel-Q/P 9-11, 12-23, 36-47 months.
PDF Links
PubReader
ePub Link
XML Download
Full text via DOI
Download Citation
Print
