The outcomes of community legal education: a systematic review, Justice issues paper 18 ( 2014 ) Cite this report
Changes in attitudes, skills, motivation and behaviour
From the two CLE studies included in the review, there is some evidence that CLE is effective in changing participants’ behaviour and outcomes in the short to medium term. This has been found for DPE programs, as measured by the use of court resources and relationship/conflict factors. The studies in our review did not specifically examine whether CLE produces changes in participants’ knowledge, skills or motivation.
Evaluations in the health sector have found that education strategies are effective in increasing knowledge, but this does not necessarily translate to behaviour change. Studies have found very mixed findings about whether health education improves all, most, some, a minority or no measures of behaviour change (as measured by health outcomes). This could reflect issues about the variability and generalisability of the findings, due to one or more of the following factors, the:
• nature of the education programs
• characteristics of the participants
• nature of the health issues addressed
• types of behaviour change required for the outcomes to be achieved.
In particular, it appears that in the heath sector education may be more effective in producing shorter-term and ‘simpler’ behavioural changes rather than longer-term, more complex and fundamental behavioural changes. In the absence of other evidence relating to CLE, this suggests that it may be best to focus on and assume the greatest impacts will be in relation to increasing knowledge and producing shorter-term, simple behavioural changes. For instance, it may be more realistic to focus on outcomes such as increased referrals to legal services rather than how people respond to their legal issues in the longer term.
What factors influence how effective an education program is?
When taken together, the CLE and health studies indicate that:
• Short programs (1 to 3 hours of education) may be just as effective as longer programs (10 or more hours). Consistently, the health education research also shows that both short programs (one or two sessions) and more intensive programs can be effective.
• Programs can be effective whether attendance is compulsory or not, but they are somewhat more effective when attendance is not compulsory. As noted above, this may be because those more willing to be trained are also more open to changing their behaviour.
• Changes in self-assessed knowledge do not necessarily correlate with changes in behaviour. This suggests the value of, for example, offering opportunities to seek advice directly after the education (which brings the opportunity to act on it close to the education), or providing education at a time and place where it can be immediately useful (e.g. divorce classes at court).
The health education research also suggests that:
• CLE may be potentially effective for a broad range of target groups, but it is helpful to tailor approaches to the needs of the target group.
• It could be useful for CLE to incorporate opportunities for participants to directly apply the information being taught or which reinforce the messages with other practices.
Evaluating CLE: a way forward
The review highlights how challenging it can be to assess the effectiveness of CLE and the need to be realistic about what is useful, sensible and feasible in evaluating CLE programs.
Outcome evaluations of CLE should only be attempted where rigorous data on effectiveness can be collected. This is generally beyond the resources and responsibility of those delivering CLE programs. As concluded by researchers in related areas such as crime prevention (see Morgan & Homel, 2013), the current practice in which the responsibility for outcome evaluations is devolved to a local organisation has not and cannot allow for the development of a strong, coordinated and independent evidence base regarding the effectiveness of interventions. Rather, they argue that central agencies will need to take the lead and a co-ordinating role with respect to the conduct of such evaluations.
At this more central level, and with appropriate co-ordination and funding, there remains valuable research that could be undertaken to better understand the impact and value of CLE in communicating different types of messages to different groups in the community. Questions include:
• Do the findings on behaviour change reported here hold for other forms of CLE and, in Australia, are these correlated with changes in knowledge, skills and/or motivation?
• How effective are different models of CLE with different groups in determining what works best for whom (e.g. according to variables such as the nature of the legal problem, the number of hours, the amount of interaction and opportunities to apply the knowledge learnt in the CLE, and whether the education is reinforced via other activities)?
However, there is also valuable evaluative work that frontline services can do to inform their practices and decision-making around CLE. There are important questions that services can gather information about, including who is reached by CLE, the types of messages received and the short term changes that may be attributed to the sessions.