A quasiexperimental evaluation of a ... - Wiley Online Library

Tropical Medicine and International Health

doi:10.1111/j.1365-3156.2010.02682.x

volume 16 no 1 pp 57–66 january 2011

A quasi-experimental evaluation of a community-based art therapy intervention exploring the psychosocial health of children affected by HIV in South Africa Joanne Mueller1, Collin Alie2, Beatrice Jonas2, Elizabeth Brown2 and Lorraine Sherr1 1 Research Department of Infection and Population Health, University College London, UK 2 Mad About Art, Knysa, South Africa

Summary

objectives To evaluate the efficacy of the Make A Difference about Art programme, a community art programme in South Africa for children affected by HIV and AIDS, which aims to reduce psychosocial problems by increasing self-esteem, self-efficacy and HIV insight. methods A quasi-experimental cross-sectional post-intervention survey of 297 children aged 8– 18 years (177 programme attendees and a control group of 120). Participants completed an inventory comprising standardized, validated psychosocial measures of depression, emotional and behavioural problems, self-esteem and self-efficacy and key sociodemographic variables potentially relevant as risk and protective factors. results Attending the intervention was predictive of significantly higher self-efficacy, but was not associated with differences in self-esteem, depression, or emotional ⁄ behavioural problems. This association remained in the multivariate analysis, controlling for potential confounders. Double parental death exerted a powerful effect on child psychosocial health, eliminating the association between intervention attendance and higher self-efficacy. However, an interaction was found between bereavement status and intervention attendance on child self-efficacy, indicating that the intervention programme may ameliorate some of the psychosocial vulnerabilities associated with becoming an orphan. Other key risk factors for poor psychosocial health in this sample were AIDS-related stigma and community and household violence. Social connection emerged as a key protective factor. conclusions Our findings suggest that such interventions may offer opportunities to increase the selfefficacy of vulnerable children to protect their psychological health. keywords evaluation, intervention, children, HIV, therapy, South Africa

Introduction The psychological health of HIV-infected and -affected children in Southern Africa is high on the research agenda (Snider 2006, Prince et al. 2007, UNAIDS & WHO 2008; JLICA 2009). Yet evidence is relatively scarce on the psychosocial health of orphaned and vulnerable children (OVC; Cluver & Gardner 2007a; Sherr & Mueller 2008). Two recent reviews suggest that HIV-affected children are at higher risk of psychological distress and to a lesser extent, behavioural problems (Cluver & Gardner 2007a; Sherr & Mueller 2008). Confusion over definitions in the literature, measurement inconsistencies and lack of control for confounding factors such as gender and HIV infection status mean that drawing conclusions is difficult (Sherr et al. 2008; Sherr et al. 2009a,b). The need for psychoso-

ª 2010 Blackwell Publishing Ltd

cial interventions for OVC is widely acknowledged and community-based programmes in Southern Africa are mushrooming (JLICA 2009; King et al. 2009). Little is known about the content of such interventions and what impact they are having on child psychosocial health; a recent Cochrane systematic review aimed to assess the effectiveness of interventions to improve the psychosocial well-being of HIV and AIDS-affected children and found no eligible studies of such interventions (King et al. 2009). Furthermore, a recent review of published and ‘grey literature’ evaluations of community interventions for OVC in Africa highlighted the variability of quality and rigour of evidence (Schenk 2009). Empirical evidence is needed to inform the development of psychosocial intervention programmes for OVC (King et al. 2009; Schenk 2009) 57



J. Mueller et al. Quasi-experimental evaluation of art therapy

HIV and ⁄ or parental death may exert an effect on child mental health both directly and via indirect routes (e.g. Wild 2001; Stein 2003; Snider 2006; Cluver & Gardner 2007a,b; Nyamukapa et al. 2010). Indirect effects of parental death due to HIV such as poverty and stigma have been identified as partial mediating factors between AIDS orphanhood and poor psychological and behavioural outcomes (Cluver et al. 2008, 2009a). Other risk factors forming an indirect route between parental death and poor child outcome identified in the existing literature include residential ⁄ caregiver instability, sibling dispersion, unequal treatment, multiple bereavement and violence. Protective factors are less well investigated but centre around peer acceptance and availability of social support (Sherr & Mueller 2008; Cluver et al. 2009b); children are particularly vulnerable to family separation and poorer social and peer support as a consequence of bereavement (Manuel 2002; Ford & Hosegood 2005; Cluver & Gardner 2006; Gilborn et al. 2006). Social support may partially mediate the association between orphanhood and psychosocial distress (Nyamukapa et al. 2006), or may mediate some outcomes and not others; peer support was associated with lower stigma and higher levels of self-concept, but not anxiety or depression (Atwine et al. 2005). In direct contrast, a recent randomized-controlled trial in Uganda (Kumakech et al. 2009) has provided some initial evidence that peer-group support interventions may reduce orphan psychological distress, but not increase self-concept. A quasi-experimental study of the impact of adult mentorship for youths heading a household in Rwanda found an increase in reports of community support and connectedness, whilst the impact on emotional well-being was less clear (Brown et al. 2007). A randomized controlled familybased social support and coping skills intervention for children and adolescents affected by HIV and AIDS in the USA had significant protective effects against substance use at a 6-year follow-up, with more social support predicting lower levels of emotional and behavioural problems (Rotheram-Borus et al. 2006; Lee et al. 2007). These studies (Brown et al. 2009; Kumakech et al. 2009; Rotheram-Borus et al. 2006) are the few methodologically rigorous, published evaluations of social support programmes and provide some evidence to support the efficacy of psychosocial interventions for OVC. Further research is needed in sub-Saharan Africa to replicate findings from the USA as HIV and poverty are widespread, treatment is less readily available and factors such as family structure may vary. Research has highlighted the importance of child resilience in the African context (e.g. Snider 2006; Rutter 2007). Resilient children have strong self-esteem, self-efficacy (Bandura 1997) and coping ability in stressful environments 58

(Rutter 1985; Bandura 1997; Luthar et al. 2000). Children with poorer self-efficacy and self-esteem are at greater risk of anxiety and depression (Muris 2001; Mann et al. 2004). A longitudinal study of HIV-infected mothers and their children in the USA found that maternal illness was a risk factor for low resiliency (Murphy & Marelich 2008). Few studies of OVC in Africa have reported resilience outcomes such as self-esteem and self-efficacy. This study aimed to evaluate empirically the effect of a community-based psychosocial intervention for children affected by HIV and AIDS using quasi-experimental crosssectional post-intervention survey design. This research also aimed to identify key psychosociodemographic factors that impact upon children’s psychosocial health in order to help structure and evaluate future interventions effectively.

Method Intervention A community-based intervention called MAD (‘Make A Difference’) About Art aims to increase children’s selfesteem and self-efficacy and thereby improve their psychosocial health. The project, for children aged 8–18 years, is based in a deprived community in South Africa since 2001. Children attend sessions for 6 months (50+ sessions), led by a team of trained and supervised ‘youth ambassadors’. The project runs art and education activities to build a sense of self-worth (self-esteem), self-concept, empowerment and emotional control (self-efficacy). These activities include children creating ‘hero’ books about their own life journey and group HIV education activities focused on selfadvocacy and empowerment. Participants Two hundred and ninety-seven children aged 8–18 years living in the Nekkies township near Knysna, South Africa, participated. Children were originally allocated to the school-based programme according to arbitrary class timetabling based on the day of the week that the project was available for this school. There were no reasons to suggest that children in those classes allocated to the intervention were different from those who were not. Two hundred and nine children were on the programme attendance register. Of these, 177 (85%) were recruited (59.6% of total sample). The comparison group was recruited from the classes of children at the school who were not allocated to the programme, and consisted of 120 of 169 eligible (71%) children. Inclusion depended on parental ⁄ guardian consent, child assent and attendance during the study period.





Study design This study employed a quasi-experimental, cross-sectional post-intervention design. Ethical and practical issues promoted our selection of this design. The established delivery of the intervention to school classes precluded the randomization of the intervention first at individual level, and secondly at class level, due to arbitrary (as opposed to random) timetabling. Ethical, practice effect and resource limitations regarding the repeated completion of the standardized questionnaire measure opposed a longitudinal design. The need for rigorous evaluation of community interventions for OVC (JLICA 2009; King et al. 2009; Schenk 2009) points to an ethical need to adopt pragmatic approaches to such research (Kirkwood et al. 1997). Similar methods have been used in previous OVC studies, cited above (e.g. Gilborn et al. 2006; Brown et al. 2009). The use of rigorous quasi-experimental designs has been supported in the research methods literature, particularly in the context of community-based interventions, for both ethical and pragmatic reasons (e.g. Kirkwood et al. 1997; Victora et al. 2004). Procedure and ethical considerations Ethical approval was gained from The University of Cape Town and University College London (1478 ⁄ 001). Informed, voluntary caregiver (parent ⁄ guardian) and personal assent from each participant was required. The information sheet emphasized voluntary participation. Children self-completed the questionnaire guided by two qualified researchers. Children were able to choose forms in English, Afrikaans or isi-Xhosa (translations by bilingual translators and back-checked for accuracy by a second bilingual translator). A drawing and colouring activity, refreshments and verbal debrief ended the session. Support structures for children after completing the questionnaire included researchers, teachers who were informed and alerted to the issues raised by the questionnaire, and availability of MAD about Art staff and youth workers. Measures Given the lack of existing psychosocial intervention evaluations, the questionnaire tool was developed specifically after a thorough literature review. The questionnaire aimed to capture psychosocial health, focusing specifically on selfesteem, self-efficacy, depression and emotional–behavioural difficulties. Self-esteem was measured using the Rosenberg SelfEsteem Scale, a 10-item Likert scale with extensive validity


and reliability data (Rosenberg 1965; Bagley et al. 1997; Gray-Little et al. 1997; Griffiths et al. 1999). This scale has not been validated in Africa. Alpha reliabilities for this scale have been previously reported between 0.72 and 0.90 (Robins et al. 2001). Self-efficacy was measured using The Self-Efficacy Questionnaire for Children, a 14-item Likert scale. The validity has been internationally demonstrated (Muris 2001; Suklo & Shaffer 2007). The emotional and social self-efficacy domains were used. Crohnbach’s alpha for the adapted scale with this sample was 0.758. Child depression was measured by an adapted version of the Child Depression Inventory (CDI; Kovacs 1992), validated for use with children and adolescents in Zimbabwe and South Africa (Snider (2006). This version had an alpha of 0.655 for the study sample, compared with an alpha of 0.64 for the original 10-item questionnaire (Snider 2006). Emotional–behavioural problems were measured using the Strengths and Difficulties Questionnaire (SDQ; Goodman 1997). The 25 items each load onto one of five subscales: Emotional Symptoms, Conduct Problems, Hyperactivity, Peer problems and Prosocial Behaviour. The SDQ is specifically designed for children of the appropriate age group to this study and has been extensively validated in several languages (including African languages) and in both developed and developing countries (e.g. Goodman et al. 2000; Mathai et al. 2003; Smedje et al. 1999; see also http://www.sdqinfo.org/). The Youth self-report version used in this study has been validated for self-completion and for adolescent populations and has a reported alpha coefficient of 0.82 (Goodman et al. 1998). HIV status for both child and parents was recorded using self-report, generating a standardized measure of perceived HIV state, but is limited in that it does not accurately reflect infection levels with a biological test. At the time of study, no standardized AIDS-related stigma scale for children had been published (Cluver et al. 2008). The scale used was adapted from Snider (2006), originally comprising six yes ⁄ no response items (e.g. ‘do you feel that people speak badly about you or your family?’). All scales from Snider’s (2006) work were developed and validated with children and adolescents of ages comparable with those in this study in Zimbabwe and South Africa. Four positively framed items were added to the scale (e.g. ‘do you feel that you have lots of good friends in your community?’) to balance the negative bias of the scale. The original scale has an alpha reliability of 0.758 (Snider 2006), and 0.531 with this sample. The adapted scale has an alpha of 0.591 with this sample, indicating increased reliability. 59




No standardized social connection scale for children in sub-Saharan Africa has been used previously in the psychosocial literature (Snider 2006). The scale used is a three-item binary-response question (yes ⁄ no) developed by Snider (2006) to measure whether children have a person (peer, other adult or caregiver) who provides them with support (e.g. ‘do you have someone in your life you can depend on to comfort you when you feel sad or sick?’). The scale has a reported alpha reliability of 0.87. Witnessing and experiencing violence questions used were those developed by Snider (2006) from the Straus Conflict Tactics Scale (P-C CTS; Straus et al. 1998) and the South African DHS (1998). Positive items were added into the questionnaire to redress the negative bias. Children’s responses were captured on a scale of frequency (weekly ⁄ monthly ⁄ less often ⁄ never), and recoded into a binary prevalence score (‘weekly ⁄ monthly ⁄ less often’ score 1; ‘never’ score 0). The risk behaviour items measuring violence towards others, substance abuse and criminality were selected from Social and Health Assessment scales, which summarize externalizing and risk behaviours (Ruchkin et al. 2004) Children responded to the three items on a scale of 0–5+ times. Items were later recoded into a binary prevalence score (ever occurred score 1; never occurred score 0). Study-specific questions measured psychosociodemographic characteristics of the sample. Demographic items included age and gender. Sociological factors included school enrolment, housing, household size and mobility. Socio-psychological parameters included caring responsibilities, feelings of belonging, illness and parental bereavement. Items were modelled on those recommended by the Snider (2006) review and similar items have been validated in previous research (e.g. Cluver et al. 2008). Statistical analyses Key psychosocial outcome variables were depression, emotional behavioural problems, self-esteem and selfefficacy. The statistical analysis followed four key steps. Differences between the intervention group and the comparison group on psychosociodemographic variables (Table 1) were calculated using Chi-squared tests and anovas. The second step investigated the association between intervention attendance and psychosocial outcomes (depression, emotional and behavioural problems, self esteem, self-efficacy) using univariate linear regression analyses (Table 2). Self-efficacy was the only psychosocial outcome to be significantly predicted by intervention attendance and therefore adjusted multivariate linear regression models are shown only for this outcome. The adjusted multivariate models (Table 3) control for psy60

chosociodemographic cofactors found to be significant predictors of self-efficacy in the unadjusted model (Table 3), or were significantly associated with the intervention group (Table 1). The first adjusted model investigates the association between intervention attendance and self-efficacy when controlling for all relevant psychosociodemographic variables except for parental bereavement. The second model additionally controls for parental bereavement. Finally, an interaction between intervention attendance and parental bereavement status on self-efficacy score was examined using multivariate linear regression analysis. Data analysis was conducted using SPSS (version 15.0). All tests were two-tailed and significance was set at the P < 0.05 level.

Results Psychosociodemographic characteristics Descriptive data on sociodemographic, community and psychosocial characteristics for both the intervention and comparison children are set out in Table 1. Children attending the intervention (mean 12.7 years) were significantly older than the comparison group [mean 11.7 years; F(1,241) = 17.9, P < 0.001]. The total sample comprised 48.1% (n = 143) girls, with significantly more girls in the comparison group [43.5% of intervention group vs. 55% comparison, F(1,285) = 7.92, P = 0.01]. The majority of children completed a questionnaire in Afrikaans (n = 279, 93.9%), five in isi-Xhosa (1.7%) and 13 in English (4.4%). Levels of bereavement were high, with 22.2% (n = 66) experiencing the death of a mother, a father, or both (25.5% of intervention group, 17.5% of comparison). No statistically significant differences were found between the intervention and comparison groups on parental bereavement status, (v2 = 1.94, P = 0.16). Few further differences were found between the intervention and comparison groups on the psychosociodemographic parameters. Overall, 12.1% (n = 36) of the sample reported that they were HIV+ (11.3% of intervention group, 13.3% of comparison), and intervention and comparison children reported similar AIDS-related stigma scores. The intervention group lived in significantly larger households [mean 8.36 people vs. 5.69, F(1,285) = 7.92, P = 0.01] and were more likely to look after younger children at their home than were comparison children (57.6% vs. 44.2%, v2 = 5.19, P = 0.02). The majority of children had looked after unwell people at home (67.7%, n = 201), with no differences between groups. Levels of violence at home and in the community were similarly high across both groups, with over a third of children reporting being physically abused at home (35%),



12.7 (12.3, 12.0) 77 (43.5) 8.36 (6.83, 9.88) 175 (98.9) 67 (37.9) 109 (61.6) 20 (11.3) 41 (23.2) 17 (9.60)

126 (71.2) 12 (6.80) 24 (13.6) 9 (5.10) 52 (29.4) 102 (57.6) 119 (67.2) 150 (84.7) 144 (81.4) 59 (33.3) 60 (33.9) 68 (38.4) 37 (20.9) 27 (15.3) 88 (49.7) 152 (85.9) 2.45 (2.32, 2.57) 3.21 (2.90, 3.53)

182 (61.3) 36 (12.1) 65 (21.9) 24 (8.10)

215 (72.4) 17 (5.70) 33 (11.1) 16 (5.40) 82 (27.6) (52.2) (67.7) (86.9) (80.1) (35.0)

155 201 258 238 104 115 (38.7) 116 (39.1) 54 (18.2) 41 (13.8) 138 (46.5) 251 (84.5) 2.48 (2.35, 2.55) 3.12 (2.91, 3.38)

(45.8) (40.0) (14.2) (11.7) (41.7) (82.5) 2.46 (2.30, 2.61) 3.04 (2.67, 3.41)

55 48 17 14 50 99

53 (44.2) 82 (68.3) 108 (90.0) 94 (78.3) 45 (37.5)

30 (25.0)

89 (74.2) 5 (4.20) 9 (7.50) 7 (5.80)

73 (60.8) 16 (13.3) 24 (20.0) 7 (5.80)

11.7 (11.5, 12.0) 66 (55.0) 5.69 (5.20, 6.19) 117 (97.5) 38 (31.7)

(n = 297) 12.2 (12.0, 12.5) 143 (48.1) 7.26 (6.33, 8.19) 292 (98.3) 105 (35.4)

Children not attending MAD programme (n = 120, 40.4%)

Children attending MAD programme (n = 177, 59.6%)

*Chi-squared tests are used to test categorical data and anovas are used for continuous variables. MAD, Make A Difference.

Demographic factors Age (years; mean, 95% CI) Female [n (%)] Household size (mean, 95% CI) Enrolled in school [n (%)] Moved home in the last year [n (%)] Believe self is HIV positive No [n (%)] Yes [n (%)] Did not want to answer HIV question [n (%)] Believe someone in family has HIV [n (%)] Bereavement Has mother or father died No [n (%)] Mother died [n (%)] Father died [n (%)] Both died [n (%)] Household and Community factors Stayed out of school to help with household duties in last year [n (%)] Look after younger children at home [n (%)] Look after unwell people at home [n (%)] Feeling of belonging at home [n (%)] Treated the same as other children in your home [n (%)] Ever been slapped, punched or hit on the head by an adult at home [n (%)] Ever seen adults at home hit each other [n (%)] Ever been attacked outside home [n (%)] Drunk or high in the past 6 months [n (%)] Arrested by the police in the past 6 months [n (%)] Threatened or beaten somebody up in the past 6 months [n (%)] Helped somebody in the past 6 months [n (%)] Psychosocial factors Social Connection (mean, 95% CI) AIDS-related Stigma (mean, 95% CI)

Comparison Group

Intervention Group

All children

Table 1 Whole-sample characteristics, and intervention and comparison group differences on psychosociodemographic variables

0.93 0.55 0.57 0.27

= = = =

= = = =

v2 v2 v2 v2

v2 v2 v2 v2

0.02 0.90 0.11 0.81 0.46

= = = = = = = = = = =

v2 v2 v2 v2 v2 v2 v2 v2 v2 v2 v2

F(1,295) = 0.01 F(1,295) = 0.49

4.29 0.08 1.92 0.50 0.55 0.07

0.91 0.48

0.04 0.78 0.17 0.48 0.46 0.80

0.63

v2 = 0.23 5.19 0.02 2.58 0.06 0.55

0.16 0.40 0.14 0.70

1.94 0.72 2.21 0.15

0.01 0.35 0.32 1.21