INTRODUCTION
In 2017, there were about 14,000 orphaned and abandoned children residing in institutions in South Korea [1]. Children reared in institutions suffer many disadvantages [2]. Accumulating evidence suggests that children raised in institutions are at risk for internalizing and externalizing problems, attention deficit hyperactivity disorder (ADHD), and autism [2-4]. In fact, the incidence of externalizing problems is higher than that of internalizing problems among children living in Korean institutions [5,6]. According to several studies conducted in South Korea, up to 50% of children living in institutions have untreated symptoms of emotional and behavioral problems [1,7]. Studies conducted in other countries, such as Russia, Romania, and African countries, where institutionalization of orphans is still common, have reported similar findings [2,8-10].
A key influencing factor of mental health risk among children raised in orphanages is caregiver-child psychosocial interaction [8,9]. Smyke et al. [10] found that after controlling for various child demographic characteristics (i.e., age, gender, ethnicity, and birth weight) and the duration of institutionalization, observed caregiving interactions (i.e., both qualitative and quantitative interactions) were associated with institutionalized children’s cognitive development, competence, and negative behaviors. Previous research have also reported that when fewer caregivers consistently provided children with better socialemotional interactions, the children’s physical, behavioral, and mental development substantially improved [11,12].
Unfortunately, certain aspects of the typical institutional environment may be detrimental to caregiver-child interaction, such as the highly regimented daily schedule, high child-tocaregiver ratio, uniform and impersonal caregiving, and frequent caregiver changes due to the use of a shift work schedule [9]. These features can be ascribed to structural deficiencies and may result in caregivers’ parenting stress and a neglectful caregiving [13]. A previous study reported that the child-to-caregiver ratio in South Korea is quite inadequate to handle essential care, which can lead to high caregiving stress and reduce caregivers’ quality of care, thus hindering children’s psychological and social development [14]. Many of the staff in these institutions also do not have adequate education and must cope with challenging, exceedingly stressful working conditions, which culminates in their ignoring children and failing to respond to children’s needs and emotions, instead of providing nurturing and sensitive caregiving [14-16].
Caregivers in orphanages also often use punitive discipline to address children’s behavioral problems [16]. Lee et al. [17] found that around 40% of children raised in institutions reported being subjected to violence and physical punishment while residing there.
The coercion theory is a well-founded approach that explains how children’s externalizing behaviors develop along with their noxious interactions, especially involving behavioral aspects, with caregivers who use punitive discipline [18]. Punitive discipline may actually increase children’s problematic behaviors rather than decreasing them through two behavioral mechanisms. First, children may learn to model the aggressive coping methods that caregivers use when punishing the children. Second, children tend to respond to punishment by increasing their aggressive responses (e.g., crying, whining, and tantrums) to avoid or beat the caregivers’ aversive responses by the coercion process. This pattern of interactions may result in long term consequence of coercive interdependence between caregiver and children leading to the escalation of the children’s externalizing behaviors [19-21]. In addition, the association between neglectful caregiving, characterized by an absence of interaction, and externalizing problems has been well established. According to Knutson and colleagues, supervisory neglect has a mediating effect on predicting aggression in socioeconomically disadvantaged families [20].
These previous results suggest that interventions directly targeting positive behavioral interactions between caregivers and children might be effective in reducing externalizing problems in institutionalized children. This is because the negative behavioral aspects of child-caregiver interactions are an essential mechanism for the development of externalizing behaviors. Findings from related research on maltreatment support the efficacy of behavioral parenting training on improving caregivers’ behaviors and interactions with their children to reduce the rate of behavioral problems in chidren [21].
We considered system-wide positive behavior support (SWPBS) as one method of promoting a positive behavioral interaction for institutionalized children [22]. Positive behavior support (PBS) is an intervention originally used in schools, now spreading to juvenile justice settings [23-25], and residential treatment facilities [26,27] to deal with serious behavioral problems, under the premise that children’s behaviors are functionally related to their environment [28]. Grounded in behaviorism, PBS attempts to decrease problem behaviors and promote positive behaviors through changing the antecedents and consequences of problematic behaviors—namely, the environmental factors [29]. SWPBS is an expanded form of PBS that spans an entire system.
Previous research applying SWPBS to schools showed that it could increase positive interactions between students, teachers, and staff, as well as prompt a shift from punitive discipline to responsible and responsive discipline that is focused on positive reinforcement [30,31]. In this way, SWPBS can be an effective means of changing the coercive processes that promote aggressive behaviors in children [32].
In addition, empirical research has demonstrated the efficacy of SWPBS outside of schools, including the management of young offenders’ behavior in juvenile justice settings [23-25], boarding school students [33], and young patients with severe emotional and behavioral disorders in residential treatment facilities [26,27]. For example, a Korean study by Lee and Chung [26] reported that the implementation of a PBS intervention to address the behavioral problems of children with developmental disabilities in a rehabilitation ward significantly decreased the staff’s punitive behavior and the children’s externalizing behaviors.
The same study [26] found that providing PBS training to staff can have a positive effect on reducing their caregiving stress. Furthermore, a decrease in caregiving stress can lead to an increase in choosing an effective response to child aggression [34]. PBS might thus be an effective way to manage caregiving stress, which can lead to better disciplinary and response behaviors among caregivers.
Although SWPBS has been effective in many groups, researchers have yet to investigate its efficacy with institutionalized children with continuous psychological and emotional deprivation. The primary goal of the current study was to explore the feasibility of SWPBS in addressing the externalizing behaviors of school-aged children residing in institutions in South Korea. We hypothesized that applying SWPBS in institutions would decrease the institutionalized children’s externalizing behaviors by positively changing the caregiver-child behavioral interactions. Moreover, we also assumed that providing SWPBS training to caregivers would reduce their caregiving stress.
The following specific interventions were implemented as part of the SWPBS in this study: 1) setting rules for appropriate behaviors as an alternative to problem behaviors; 2) teaching these rules to the children; and 3) engaging in contingency management consisting of the reinforcement of appropriate behaviors and the correction of inappropriate behaviors.
METHODS
Participants
Participants included 36 school-aged boys (8 boys didn’t participate), 10 caregivers, and several staff members from an urban orphanage. At the time of data collection, the orphanage housed around 60 girls and boys. We selected caregivers and children from five of the boys’ rooms, all of whom wished to participate in the study. The participating children’s ages ranged from 5 to 13 years, with an average of 9.62 years (SD=2.73).
The orphanage consisted of several rooms in two buildings. A group of children shared a 30×49 m2 room where they ate, slept, did their homework, and played. Each room was a residential unit that included bedrooms, a living room, and a bathroom, and housed both younger and older children, so the groups were heterogeneous in terms of age. Most children entered the orphanage as toddlers and had lived at the orphanage for between 4 and 13 years, with an average of 8.72 years (SD=2.52).
Two caregivers usually cared for seven to eight children in one room in 24-hour shifts (every other day). And staff members also provided laundry services and cooked meals. All 10 caregivers were female. Their average age was 29.9 years (SD= 3.93), with a range of 25–37 years. Except for one caregiver, none were married, and all had social worker certificates. We gave all 10 caregivers about $90 for their participation in the study.
Eighteen (50%) of the participating children were concomitantly receiving psychotropic medication for ADHD. Medical or neurodevelopmental problems other than ADHD are not specifically reported here.
The two girls’ rooms were excluded because the caregivers reported that the girls did not require intervention, given that girls exhibited externalizing problems less frequently than did boys.
Measures
Problem levels in the pre-intervention phase
Before the intervention, we had caregivers administer a standardized instrument called the Korean Child Behavior Checklist for Ages 6–18 (K-CBCL 6–18; 120 items) to objectively measure children’s externalizing behaviors. The K-CBCL 6–18 was translated and validated by Oh and Kim [35] from the original by Achenbach and Rescorla [36].
We used the two broadband scales, which combine syndrome scales for internalizing and externalizing problems. Examples of items related to behavioral issues are “gets into fights” and “destroys others’ things.” The Cronbach’s alpha coefficients for the scales ranged from 0.62 to 0.82 in the standardization study of the K-CBCL.
Behavioral problems
Behavioral problems were evaluated using both direct behavioral observation and caregiver report. We developed a list of externalizing behaviors based on previous PBS research [30]. After a discussion with the caregivers, we defined the target behaviors most frequently and specifically exhibited by the participating children. These target behaviors are listed in Table 1. Caregivers tallied the frequency of each behavior among the children using a counter during the pre- and post-intervention measurement phases. Specifically, the frequencies were counted during a period lasting 30 minutes at six random times over a week for each room during each measurement phase. The 30-minute observation time was chosen as the time when the frequency of externalizing behavior being reported high by caregivers, specifically during the 30 minutes before going to school starting after breakfast (7:00 am) and the 30 minutes after dinner (5:30 pm) when children were in their respective rooms and were playing freely.
Prior to data collection, we trained the caregivers to recognize the target behaviors and reliably count their frequencies. This training continued until they reached at least 80% interobserver agreement, which is an acceptable level in PBS literature. Interobserver agreement is computed by dividing the number of agreements by the total number of both agreements and disagreements, and then multiplying the result by 100, which yields the percentage (%). In the present study, a 10-minute video clip of the orphan’s everyday lives was used for training. We could not calculate interobserver reliability during the actual study, as the director of the orphanage prohibited all guests from accessing the building outside of business hours and children were reluctant to be observed.
Caregivers were also asked to rate the level of the six target behaviors (Table 1) for each child using a 10-point Likert scale (0=not at all; 9=extremely strong). Our goal was to assess if the reduction in externalizing problems was identified not only through direct behavioral observations but also in the perceived ratings by the caregiver. Cronbach’s alpha coefficient was 0.85 for pre-measurement and 0.86 for post-measurement.
Caregiving stress
We used the Parenting Stress Scale (PSS) by Kim and Kang [37] to assess the caregiving stress experienced by caregivers. This is a self-report scale containing 24 items, each rated on a scale from 1 (not at all) to 5 (very strong). Kang [15] examined institutional caregivers’ parenting stress after excluding one unrelated factor from the original scale’s three factors and using only the remaining two factors (“typical stress due to parenting” and “pressures pertaining to the parental role and distress”), which were the same as those used in the present study (Supplementary Table 1 in the online-only Data Supplement). In Kang’s [15] study, the internal consistency of the two factors was 0.90. In this study, it was 0.93.
Intervention fidelity
For a valid and reliable intervention, the caregivers answered six items (with responses of “yes” or “no”) that we developed to assess whether they had implemented the intervention properly or not (Table 2). We used their answers to calculate the intervention fidelity, as follows.
Intervention fidelity=(number of items answered “yes”/total number of items)×100
Procedure
The caregivers and staff of the orphanage agreed to participate in the study to address orphans’ problems after learning the goal and procedure of the intervention. Subsequently, informed consent was obtained from them. This study was reviewed and approved by the Institutional Review Board of the first author’s university (1041449-201612-HR-005). The main elements of the SWPBS intervention were as follows (Figure 1). A detailed explanation of the process is provided in the Supplementary Materials (in the online-only Data Supplement).
1) The SWPBS team comprised staff, caregivers, and researchers. Team meetings were held once per week to address and solve problems encountered by the caregivers.
2) The SWPBS team developed a set of institution-wide five rules (Table 3) to reduce target behaviors. First, following a discussion, the team selected three core values (i.e., security, respect, and kindness) considered important in the orphanage. Second, based on these values, the caregivers and researchers defined five rules.
3) Subsequently, we implemented rule training for the children and caregivers in each of the five rooms, individually. The team created video clips with volunteer children to demonstrate the expected behaviors for each rule and the behaviors that violated the rules. During the rule training, the children and caregivers were instructed on rule compliance and violation behaviors using the video clips, and the children modeled the expected behaviors, practicing various examples.
4) A token contingency system was introduced to reinforce and improve the frequency of positive child-caregiver interactions (caregivers rarely initiated such interactions) and to increase rule compliance in the children. After completing the rule training, two predetermined 30-minute intervention sessions, for a total of one hour, were conducted simultaneously with the behavioral observations. During these sessions, the caregivers distributed wooden sticks as tokens for rule-compliant behaviors. The children who had earned a certain number of sticks could exchange the tokens for rewards (i.e., privileges and prizes).
5) After two weeks, the intervention was generalized across time and setting, so the caregivers distributed the tokens at all times and in all places, besides the two predetermined 30-minute sessions. The above procedure is shown in Figure 1.
RESULTS
K-CBCL 6–18 at pre-intervention phase
When the mental health status of the participants was investigated before the intervention, 28 out of the 36 (77.78%) children were found to have clinical levels of externalizing problems. Furthermore, five children (13.89%) had clinical levels of internalizing problems, and five children (13.89%) had clinical levels of both internalizing and externalizing problems. All the children with internalizing problems also had externalizing problems.
Behavioral problems
The mean frequency of problem behaviors (M=4.32) over the 30 minutes of direct observation in the post-intervention measurement represented a decrease of 73.66% compared to the pre-intervention measurement (M=16.40). Room C had rather few problems even before the intervention (M=3.40); hence, the frequency did not decrease further after the intervention (M=4.40). According to anecdotal reports from staff members, the caregivers responsible for Room C were already responsive and friendly to children and consistent in terms of their interactions. When excluding the change rate of Room C, which was overestimated, the problem behaviors showed an improvement of 78.1% in the other four rooms. Three out of the five rooms showed a decrease in problem behaviors of over 80% (Table 4).
A t-test was conducted to examine the impact of the intervention on the levels of target behaviors as rated by caregivers across the two time periods. We observed a significant decrease in the severity of children’s problem behaviors after implementing the intervention compared to before, total score: t(35)= 5.247, p<0.001. The intervention effect was significant for all types of problems. Furthermore, the size of the intervention effect (Cohen’s d) on the total score for problematic behaviors was 0.87, which indicates a large effect according to Cohen’s [38] guidelines (Table 5).
Among the 36 children, 20 (55%) showed an improvement in the total score of post-intervention measurements that was greater than one standard deviation (SD=8.30).
Caregiving stress
The results of a t-test revealed that parenting stress did not significantly differ between the pre-and post-intervention measurements (pre-intervention: M=64.20, SD=12.13; post-intervention: M=63.90, SD=17.72; t=0.10, p=0.92).
Intervention fidelity
The intervention fidelity ranged from 73.32 to 100%, with an average of 89.6%, which is considered a reasonable level of fidelity in PBS literature [39]. This result shows that the caregivers faithfully implemented the intervention according to our instructions.
DISCUSSION
We examined whether SWPBS decreased externalizing behaviors in institutionalized children in South Korea. We hypothesized that SWPBS would increase positive behavioral interactions between caregivers and children, and ultimately decrease externalizing behaviors while increasing appropriate behaviors. We also expected PBS to have a positive effect on reducing caregivers’ stress.
We found that SWPBS decreased the frequency of externalizing problematic behaviors, such as noncompliance, verbal aggression, physical aggression, general disruption, inappropriate physical contact, and emotional outbursts, in institutionalized school-aged children by around 74%. Furthermore, caregivers’ evaluations of the children’s problematic behaviors showed an improvement from pre- to post-intervention with a large effect size. These results indicate that SWPBS can also be implemented among institutionalized children in orphanages to effectively decrease behavioral problems as previous studies [25.26,31].
Of particular note is that the results were observed in a population characterized by frequent aggressive behavior. Indeed, the results of the K-CBCL 6–18 showed that approximately 80% of the children had externalizing problems above the clinical threshold before the intervention. In contrast, only around 40% of children in orphanages had externalizing problems in previous studies [2,5]. The fact that our participants had been living in the orphanage for an extended period might have contributed to the high occurrence of mental health problems. Previous research has confirmed that the critical length of exposure to such institutions ranges from 6 to 18 months in many countries, assuming that children were admitted within weeks or months after birth [9]. After the critical length of exposure, they might begin to exhibit obvious deficiencies in physical, cognitive, and psychiatric development; however, the extent of the deficiency may be affected by the institutional environment [9]. While for children from the fatally deficient orphanages, it can’t be exceed 6 months, for children from some caregiving depriving institutions could be extended to 18 months. Since the participants of this study they had lived in the orphanage for 8.72 years on average, they were well past the critical age. However, the participating children’s exact time of admission to the institution was not known, although it was confirmed that most of them were admitted as toddlers or infants. The findings of this study show that SWPBS can be a promising alternative to individual therapy or usual services for reducing externalizing problems even in aggressive school-aged orphans residing in institutions for a long term.
The positive effect of the SWPBS intervention was evidently due to the improvement in child-caregiver behavioral interactions. This improvement was particularly related to positive changes in the detrimental coercive process, which involves aversive contingencies that can inadvertently trigger and maintain aggressive behaviors in children [20]. After we established clear and positive rules and consistently provided positive reinforcement through SWPBS, the children started to exhibit appropriate behaviors incompatible with problematic behaviors. During the intervention, the caregivers engaged in high frequencies of positive behavioral interactions (in the form of providing explicit rewards and social reinforcement) whenever the children behaved appropriately, which ultimately offset their coercive and neglectful caregiving behaviors. Similarly, previous studies have found that physical, cognitive, and social developmental delays in orphanage children can be reduced when caregivers are trained to increase the frequency of positive social-emotional interactions with such children [11].
The total parenting stress score for the caregivers did not change significantly even after the intervention. Unlike biological parents, caregivers in orphanages experience additional stress due to environmental factors, such as exceedingly poor working conditions. Since our intervention did not address the working conditions of the staff, it might have been less effective in changing caregivers’ overall stress level.
The current study has various limitations. First, owing to the facility’s limitations (a policy prohibiting guests from entering the orphanage outside of business hours), interobserver reliability of direct behavioral observations could not be calculated. We attempted to overcome this problem by providing data collection training to caregivers until they reached at least 80% interobserver agreement. Second, this study only applied SWPBS for a month, which is a relatively short amount of time to decrease problematic behaviors in institutionalized children and evaluate the results. SWPBS is often applied for six months to several years [22]. Therefore, future studies should implement SWPBS in institutions for longer periods to assess the longterm effects. At institutions, caregivers often change jobs, and children often move to other rooms. No caregiver moved or left the orphanage during this one-month-long study. However, caregivers and children would be expected to move during a longer intervention. Therefore, it is necessary to investigate whether SWPBS remains effective even when significant events, such as loss of attachment, occur frequently. Third, this study focused on ensuring that SWPBS had a positive effect on reducing the children’s externalizing behaviors. Children’s emotions or attachments, however, were not assessed. Future studies should therefore evaluate the qualitative aspects of childcaregiver interactions. Fourth, the participating caregivers might have been influenced by social desirability bias (or the Hawthorne effect). To circumvent this bias, we cross-referenced their self-report data on the levels of externalizing problems with direct behavioral observation data. These data were collected by counting the occurrence of observable target behaviors using a small manual counter kept in the caregivers’ pockets;this method is less likely to result in biased over- or under-reporting. Fifth, the characteristic of the sample is also a limitation of this study. The sample size was relatively small due to that most institutions in South Korea are reluctant to have orphans examined or observed, as is true for orphanages in other countries [9]. In addition, since we were intervening on a roomby-room basis, age of participating children were relatively heterogeneous, which have resulted in difficulty to explain the effect size of the result according to developmental ages.
Our findings suggest that SWPBS may be a promising intervention for children in orphanages with serious externalizing problems. If further research confirms that SWPBS is sustainable, it would be useful to consider introducing SWPBS in an effort to alleviate the mental health problems of institutionalized children in countries where institutional accommodation is relatively common.
