SRDR+

1. Review ID

(surname of first author and year first full report of study was published e.g. Smith 2001)


	Conway 2009

2. Review title


	Teaching and Learning in the Social Context: A Meta-Analysis of Service Learning’s Effects on Academic, Personal, Social, and Citizenship Outcomes

3. Date form completed


	04/08/2022

4. Initials of person extracting


	BN

5. Review funding source


	None declared

6. Possible conflicts of interest


	None declared

7. Aim of review


	to meta-analyze the existing evidence on (a) extent and types of change in participants in service learning programs; (b) specific program elements (moderators) that affect the amount of change in participants; and (c) generalizability of results across educational levels and curricular versus noncurricular service. To explore the following hypotheses: H1: We hypothesized that we would find evidence of change in participants on academic, personal, social, and citizenship outcomes. H2: We hypothesized greater change for programs, including structured reflection, than for those not including reflection. H3: We hypothesized greater change for programs of longer duration. H4:We hypothesized greater change for programs with a greater number of service hours.

8. Number of databases searched


	11

9. Names of databases searched; date ranges of databases searched


	Academic Source Premiere, Business Source Premiere, CINAHL and Pre-CINAHL, ERIC, Dissertation Abstracts, Hospitality and Tourism Index, MEDLINE, PAIS International, Professional Development Collection (education), PsycINFO, and SocINDEX

10. Date of last search


	June 2008 ?

11. Number of included studies


	103

12. Exclusion criteria for participants

(e.g age, comorbidities) 

13. Exclusion criteria for volunteering

(e.g type of volunteering, for a specific organistion/purpose)


	Service learning.

14. Exclusion criteria for study type


	To be included, a study had to meet the following criteria: (a) pretest–posttest design using identical quantitative measures for identical pre- and postsamples; (b) participation in community service between pre- and posttests; (c) sufficient information provided so we could classify the measures (e.g., as a citizenship or social outcome); and (d) reporting of pretest and posttest means, the pretest standard deviation, and sample size. In some cases, we contacted the authors to obtain information.

15. Exclusion criteria for outcome measures


	Either or a mixture of; personal, social, citizenship and academic).

16. Outcomes studied

(select all that apply)


	Psychological Physical Social General

17. Primary reported outcomes


	change in participants in service learning programs on four outcome types (academic, personal, social, and citizenship)

18. Secondary reported outcomes (if applicable)


	Specific program elements (moderators) that affect the amount of change in participants on all four types of outcomes (reflection, and intensity and duration of service).

19. Number of participants included in the review


	Only reported by outcome for meta-analysis: Self-evaluations (N = 1,819) Moral development (N = 93) Well-being (N = 274)

20. Review’s included study type (% of quant studies)


	100% quant

21. Included studies countries of publication


	Not reported

22. Range of included studies years of publication


	Not reported

23. Review’s population

(age, ethnicity, SES)


	Not reported (only coded for meta-analysis)

24. Social outcomes reported

25. Social outcomes not supported

(e.g cited as non-significant)

26. Physical outcomes reported

27. Physical outcomes not supported

(e.g cited as non-significant)

28. Psychological outcomes reported


	Self evaluations Moral development Overall, small with variability

29. Psychological outcomes not supported

(e.g cited as non-significant)

30. General outcomes reported

(i.e general health and wellbeing)


	Wellbeing: negligible effect with a small total sample siza and CI crossed 0

31. General outcomes not supported

(e.g cited as non-significant) 


	CI crossed 0

32. Interactions reported

(i.e between each other or demographic variables)


	Academic learning and social outcomes showed a significantly larger improvement than personal outcomes. Differences between programs with and without structured reflection were relatively large for personal outcomes, Md = .29 with reflection and Md = .09 without; both d values had confidence intervals that did not include zero (Q values were 11.142 and 17.916 for random and fixed effects, both ps < .05). Mean d values for number of service hours (40 or less vs. 41 or more) and number of weeks of service (15 or less vs. 16 or more) did not support our hypotheses; d values were slightly larger for smaller numbers of hours and weeks. We probed results using narrower categories and trends suggested increasing effects up to about 40 hr and 30 weeks, with lower effects for higher numbers of hours and weeks. However, the trends were not crystal clear and the categories often had very small numbers of studies. K–12 and higher education groups showed significant effects in the .20s or .30s. Q tests (both random and fixed effects) showed these to be significantly larger than mean d values for adult or mixed groups, which had mean effect sizes near zero (more effective for younger cohorts). For personal outcomes curricular service (41 studies vs. 17 for noncurricular service) had a significantly higher mean d value, .27 versus .06 (both fixed- and random-effects Q tests were significant, p < .05) (more effetive when it was part of the curriculum. = cohort, being part of the curriculum, and reflection matters.

Academic learning and social outcomes showed a significantly larger improvement than personal outcomes.
Differences between programs with and without structured reflection were relatively large for personal outcomes, Md = .29 with reflection and Md = .09 without; both d values had confidence intervals that did not include zero (Q values were 11.142 and 17.916 for random and fixed effects, both ps < .05).
Mean d values for number of service hours (40 or less vs. 41 or more) and number of weeks of service (15 or less vs. 16 or more) did not support our hypotheses; d values were slightly larger for smaller numbers of hours and weeks. We probed results using narrower categories and trends suggested increasing effects up to about 40 hr and 30 weeks, with lower effects for higher numbers of hours and weeks. However, the trends were not crystal clear and the categories often had very small numbers of studies.
K–12 and higher education groups showed significant effects in the .20s or .30s. Q tests (both random and fixed effects) showed these to be significantly larger than mean d values for adult or mixed groups, which had mean effect sizes near zero (more effective for younger cohorts).
For personal outcomes curricular service (41 studies vs. 17 for noncurricular service) had a significantly higher mean d value, .27 versus .06 (both fixed- and random-effects Q tests were significant, p < .05) (more effetive when it was part of the curriculum. = cohort, being part of the curriculum, and reflection matters.

33. Was a meta-analysis performed?

34. Number of included studies in the meta-analysis


	Self-evaluations = 32 Moral development = 4 Wellbeing = 6

35. Heterogeneity

(e.g I squared)


	(true standard deviation of difference) Self-evaluations .25 Moral development .00 Wellbeing .26

36. Pooled estimates


	(mean difference in means) Self-evaluations .26 Moral development .34 Wellbeing .17

37. Confidence intervals (95%)


	Self-evaluations .16–.37 Moral development .23–.44 Wellbeing −.07–.42

38. Key conclusions from study authors


	service learning does tend to have these effects, producing positive changes in academic, personal, social, and citizenship outcomes. Changes for personal and citizenship outcomes were small. A second major finding was that reflection was generally associated with larger effects, and a third finding was that effects tended to generalize across K–12 and higher education programs, although for adult or mixed populations therewas little evidence ofchanges in personal or social outcomes. Therewas some evidence that noncurricular service had smaller effects.

39. Review limitations


	One limitation of our meta-analysis is that most studies were from disciplines other than psychology (about 23% were from psychology or a related discipline). We conducted analyses for psychology and related disciplines versus other disciplines and found that the major results held for both categories. We are therefore confident that our findings can be applied to the teaching of psychology.

40. AMSTAR 2 quality appraisal rating


	-20

41. Quality appraisal tool used by review (if applicable)


	N/A

42. Quality of included studies (if applicable)


	N/A

43. Publication bias reported (if applicable)


	N/A

44. Was correspondence required for further study information?

45. What further correspondence was required, and from whom?


	More information on which studies were included for the meta-analysis on personal outcomes specifically, so that overlap could be calculated accurately.

46. What further study information was requested (from whom, what and when)?


	Information on the studies included for personal outcomes was requested to the contact provided on the publication: Jim Conway at conwayj@ccsu.edu

47. What correspondence was received (from whom, what and when)?


	Jim Conway provided information of the authors included in the meta-analysis for personal outcomes.

Extraction form for project: The effect of volunteering on the health and wellbeing of volunteers: an umbrella review

Design Details