 |  |
, |
Introduction
This paper reviews outreach as a model of legal service delivery to disadvantaged people with complex needs. We define outreach legal services as face to face legal assistance and advice services delivered away from the primary service/office, in places accessible to the target group. Telephone based outreach is excluded from this review. We focus on outreach to people who are marginalised or socially excluded as a result of issues such as homelessness, disability, unresolved mental health issues, Indigenous status, severe financial hardship, unemployment or remote location. Their complex needs may be over and above their legal needs.
Service providers face significant challenges in providing legal assistance to disadvantaged people with complex needs. People facing significant disadvantage often have multiple and interrelated issues, including legal issues, tend not to approach legal services for assistance, and face particular difficulties in working with lawyers to address their legal problems (Grunseit et al, 2008; Coumarelos et al, 2006; Forell et al, 2005). Issues may be at crisis point before clients reach legal assistance, and due to their sometimes chaotic lives, clients may not have the documents or the capacity required to work with the legal adviser to address the problem at hand (Grunseit et al, 2008; Forell et al, 2005). ‘Outreach’ is a strategy commonly employed by public legal services to reach and assist disadvantaged people with legal problems (see Legal Aid, 2008).
In NSW, a broad range of legal services are provided under the banner of ‘outreach’. For example, outreach legal services include legal assistance services in welfare agencies, pro bono legal advice clinics to homeless people, regular advice sessions run by community legal centres in remote locations, and civil and family law advice provided by Legal Aid to clients of Aboriginal Legal Services.
Our initial analysis of the literature indicated that a number of individual outreach legal services and programs have been evaluated, including outreach debt advice services which provide legal assistance. The purpose of this report is to draw together the best available evidence from across these studies in order to provide practical information to service providers who currently deliver outreach legal services or who may be considering this as an option.1
Objectives of this review
The objectives of this review were to appraise and synthesise the best available research evidence on:
- the effectiveness of outreach legal services in reaching disadvantaged people with complex needs, and providing the range of legal services they require in a sustainable way
- the features of effective outreach legal services.
Methodology
The findings reported here are based on a comprehensive and systematic review of research and evaluation reports on outreach legal services to disadvantaged people in Australia and overseas. A systematic review is a methodology for selecting and synthesising the results of relevant research and evaluation studies in order to provide practitioners with practical information that is based on the best available research on a specific question. This methodology:
…uses transparent procedures to find, evaluate and synthesize the results of relevant research. Procedures are explicitly defined in advance, in order to ensure that the exercise is transparent and can be replicated. This practice is also designed to minimize bias. (Campbell Collaboration, 2009)
The outcome of the process is a set of synthesised findings from research studies on the topic, which can then form the basis of best practice. Systematic reviews are increasingly being used by government policy makers to inform decision making (Government Social Research Unit, 2007).
Systematic reviews have been traditionally undertaken in the health sector and focused on the meta-analysis of experimental studies such as randomised controlled trials.2 However, more recently there has been work in the health sector to develop methodologies for systematically reviewing data collected using other research methods. In contrast, methodologies and software for reviewing and synthesising research data in the field of social research are less developed.
For this reason we based our methodology on an appraisal system developed by the Johanna Briggs Institute (JBI) to review health related research data.3 As well as conducting quantitative systematic reviews as part of the Campbell Collaboration, JBI have developed rigorous systematic review methodologies for qualitative data, economic (cost benefit) data and narrative text (expert opinion). Their rationale for developing these methods is as follows. Randomised controlled trials were traditionally the predominant form of research undertaken by medical researchers and were usually seen as the 'gold standard' for medical evidence of 'effectiveness'. However, as argued by JBI and others, different research methods are required to answer different research questions (Government Social Research Unit, 2007). For instance, qualitative research is the most appropriate methodology to explore the experiences of a participant in an intervention while an economic study best addresses the cost of the intervention. A mixed method approach may be used to explore why an intervention worked or not. JBI further argue that:
-
the results of well-designed research studies grounded in any methodological position [provide] more vigorous evidence than anecdotes or personal opinion. (JBI, 2008 p.9)
Our methodology draws upon the JBI review process for reviewing qualitative data. We took this approach for several reasons. First, nearly all of the research available on outreach legal services had a qualitative component. Second, JBI does not provide a specific review and appraisal methodology for quantitative methodologies (e.g. surveys) which are not experimental designs. Finally, as the qualitative review process is designed to be used with a range of (qualitative) methodologies, we considered that it was broad enough to provide a rigorous 'checklist' for high quality mixed method research, and an appropriate methodology for reviewing and synthesising the findings from these studies.
As an evolving methodology, some debate remains about the power of and best methods for synthesising qualitative data (JBI, 2008, p.29). Similar questions arise when considering the synthesis of findings from mixed method research. However, in our view the JBI approach gave us a robust, transparent and replicable method to trial for the review and synthesis of mixed method data. As we discuss in Appendix 1, we will learn from this review as we refine a methodology to appraise and synthesise mixed method social-legal research evidence.
The systematic review process
In broad terms, our methodology involved four stages:
- developing a research protocol which clearly articulated the research questions, inclusion criteria and search strategy (Research protocol)
- using search terms drawn from these pre-defined inclusion criteria to guide the literature search and to select relevant studies (Literature search)
- having two researchers independently assess the quality of all studies retrieved against defined and consistent criteria (see Table 2). Only those studies which pass an agreed quality standard are included in the review (Appraisal)
- having two researchers independently identify key findings with supporting evidence in each report, categorising these findings, then drawing these categories into broader 'synthesised' findings that can be applied in practice (Data extraction and synthesis).
The methodology is outlined below. Additional details are provided in Appendix 1. Readers should note that we have reviewed evaluation or research reports on outreach programs, not the programs themselves.
1. Research protocol
Before commencing the review, we developed a research protocol which defined the parameters of the systematic review, outlining the research questions, the inclusion criteria and our search strategy. The inclusion criteria identified are as follows.
Target group — disadvantaged people with complex needs
The review was restricted to studies on outreach services which targeted disadvantaged people with complex needs. 'Disadvantaged people with complex needs' were defined in this review as 'people who have multiple problems, including legal problems, but for whatever reason, are not able to access the range of social services/institutions that can be accessed by the majority of the population' (Schetzer et al, 2002, p. 15, based on a 1996 ABS definition). These disadvantaged people may also be described in the literature as 'socially excluded' or 'hard-to-reach'.4 Such groups often include Indigenous people, CALD communities, people with disability, homeless people, prisoners and those in very remote locations.
Intervention — face to face outreach legal services
Only studies on outreach legal services were included in this review. 'Legal services' include legal advice and assistance services, which may or may not provide representation. We included money and debt related advice services within our definition, where the advice provided was not just 'financial planning' but advice and legal assistance in managing existing debts.
We defined 'outreach' as 'face to face assistance (primarily advice but also minor assistance) delivered away from the primary legal service/office', in places used by target groups (e.g. homes, community (health) centres, welfare agencies, rural towns). Outreach by video-link was included within the search criteria, but telephone outreach was excluded. No stand alone studies on the effectiveness of video-link outreach services were found, though video-link services were included in one study.
Types of studies
Initially, our literature search was not limited by study design. However, we found no randomised controlled trials or case controlled studies examining the legal outcomes of outreach services. As a result, our search focused on evaluation studies, including mixed method studies that used in-depth qualitative interviews, observational methods and surveys. Some studies also drew on service data to inform their results. One study, a cost effectiveness study, used only economic and administrative data. The review was restricted to reports written in English and published after 1998. This date was selected to focus the review on the most recent and relevant studies. Studies or documents which did not meet criteria to be included as 'research' (eg documents which simply described programs or were 'expert opinion') were excluded from this review.
Outcomes of interest
We looked for evidence about the 'effectiveness' of outreach models and about the elements of these programs which were found to contribute to success (or otherwise). We defined 'effectiveness' in terms of whether outreach services:
- reached disadvantaged people with complex needs
- provided the range of legal assistance services needed by those clients
- improved clients' circumstances or addressed their legal issues
- were sustainable. The issue of costs and resources were considered here.
2. Literature search
The search strategy used, including search terms drawn from our inclusion criteria, is in Appendix 1. The databases and websites examined are listed in Appendix 2.
Our initial search of relevant databases and websites returned hundreds of potentially relevant documents. Through a staged process of rigorous assessment against the inclusion criteria described above, these were narrowed down to 16 studies. These were the only original research or evaluation reports that we located on outreach legal services to disadvantaged people with complex needs.
We believe that our search strategy was broad and deep enough to have located most published material which fits our criteria. However, while some unpublished studies were included in the review, there may be other unpublished reports that we did not identify. Table 1 below illustrates how we selected (filtered) studies through both the search and appraisal processes.
Table 1: Summary of process for selecting studies
| Initial search |  | |
| Broad search against search criteria | | Hundreds of studies identified at first search, using search terms listed. |
| Targeted search and selection | | |
| Targeted search and review of retrieved studies against inclusion criteria | | 98 studies reviewed against inclusion criteria, 39 retrieved in full text. |
| Appraisal | | |
| Review of studies against appraisal criteria | | We 'appraised' the methodology of the 16 studies which met all of the inclusion criteria, to assess whether they qualified as rigorous research or evaluation (see Table 2 below) and, in the case of Smith and Patel (2008), quality 'economic' research. |
3. Appraisal
The 16 studies that met all of the inclusion criteria (that is, were identified as relevant to the topic) were then separately appraised by two researchers for methodological quality, using the standard JBI critical appraisal criteria outlined in Table 2 below.5 On the basis of this appraisal they were then either included in the review or excluded.
Table 2: Criteria against which qualitative studies were assessed
| Criteria |
Yes
|
No
|
Unclear
|
|
| 1. | There is congruity between the stated philosophical perspective and the research methodology. |
|
|
|
| 2. | There is congruity between the research methodology and the research question or objectives. |
|
|
|
| 3. | There is congruity between the research methodology and the methods used to collect data. |
|
|
|
| 4. | There is congruity between the research methodology and the representation and analysis of data. |
|
|
|
| 5. | There is congruity between the research methodology and the interpretation of results. |
|
|
|
| 6. | There is a statement locating the researcher culturally or theoretically. |
|
|
|
| 7. | The influence of the researcher on the research, and vice-versa, is addressed. |
|
|
|
| 8. | Participants, and their voices, are adequately represented. |
|
|
|
| 9. | The research is ethical according to current criteria or, for recent studies, there is evidence of ethical approval by an appropriate body. |
|
|
|
| 10. | Conclusions drawn in the research report do appear to flow from the analysis, or interpretation, of the data. |
|
|
|
Ideally, the studies selected would have met all of these criteria. However, as is discussed further in Appendix 1, had we applied all of these criteria, we would have excluded all of the studies. Rather, we identified what we believed were the key criteria for methodological quality and only included studies which met this standard. To be included in the final review, each study had to score a 'yes' for at least criteria two to five, and criterion number ten. While no studies met all ten criteria, six met 8 or 9 criteria.
Of the 16 studies critically appraised against these criteria, eleven were included in this review. These are listed in Table 3.
Table 3: Summary of studies included in the review
| Author | | Title | | Country | | Outreach location(s) | | Legal issues | | Evaluation |
| Buck et al, 2007 | | Putting money advice where the need is: evaluating the potential for advice provision in different outreach locations | | England & Wales | | 100+ outreach locations - Welfare agencies, credit unions, housing offices, schools, community centres, family and children centres, Citizens Advice Bureaus (CABs) and prisons | | Credit and debt | | Phase 1 of evaluation - to assess different outreach locations. Early in 3 year life of projects. Interviews at 25 randomly selected outreach sites (5 of each location type). 563 interviewees. Frequency statistics from closed questions. Thematic analysis from open-ended questions. |
| Day, Collard & Hay, 2008 | | Money advice outreach evaluation: qualitative outcomes for clients | | England & Wales | | As above | | Credit and debt | | Impact evaluation nearly two years into the project. Included semi-structured interviews with 41 clients and 8 people from the target group (people eligible but did not use the service). |
| Day, Collard & Davies, 2008 | | Money advice outreach evaluation: the provider and partner perspectives | | England & Wales | | As above | | Credit and debt | | Process evaluation over 18 months, 3-9 months after start of service delivery. Semi-structured interviews with 5 key policy stakeholders, telephone survey of the 22 outreach pilots, follow up interviews with 30 project coordinators and partners, in-depth case study of 8 pilot projects and a dissemination (of interim evaluation report findings) and information exchange seminar. Thematic analysis of data. Collection and analysis of statistical monitoring information and demographics. |
| Dimos, 2008 | | Civil law ALS Outreach review | | NSW, Australia | | Outreach to 7 Aboriginal Legal Service Offices by Legal Aid | | Civil law | | Evaluation after 12 months operation. Semi-structured interviews with staff and other stakeholders (not service users), service data and literature review. |
| Gillespie et al., 2007 | | Money advice for vulnerable groups: final evaluation report | | Scotland | | 11 outreach projects - Surgeries, home and hospital visits, JCP offices, community flat in the locality, CAB offices, learning disability and welfare rights centres, youth services | | Credit and debt | | A synthesis of evaluation reports on the 11 outreach projects. Evaluation over the 2 years & 5 months development and life of the projects. Method included 111 interviews with service users and 55 with staff, focus groups, project data analysis. Included follow up interviews. Consultation with services users and staff. |
| Goldie, 2003 | | PILCH Homeless Persons’ Legal Clinic: evaluation report | | Victoria, Australia | | 6 homeless persons services | | Civil law | | Evaluation after 18 months operation. Semi-structured interviews with 10 clients and with 40 staff and host agencies reps, document/file analysis, case studies and costings. |
| Hartlepool New Deals for Communities, 2004 | | Evaluation report on the Money Advice and Debt Counselling Service Project | | England | | Based at a CAB office, but includes home visits and outreach service at other venues | | Credit and debt | | Evaluation after nearly 3 years operation. Semi-structured interviews with 7 stakeholders and 6 clients, observations, focus group, survey of service users and analysis of file, monitoring information and other data. |
| Kilner, 2007 | | Housing Legal Clinic final evaluation report | | South Australia | | 4 homeless persons services | | Civil law | | Evaluation at start of project and for first 18 months, against outcomes and KPI’s and service agreement. Interviews with project coordinator, managers and pro bono partners, postal survey to 80 clients, in depth case studies of 20 clients, workshops with clients from two clinics, surveyed and held workshops with volunteer lawyers, postal survey to other stakeholders and analysis of program data and statistics. |
| Sherr, 2002 | | A stitch in time: accessing and funding welfare rights through Health Service Primary Care. An evaluation of primary care based specialist welfare rights advice provision in Lambeth, Southwark and Lewisham | | England 3 London Boroughs | | Approx. 80 doctor’s surgeries, health care services | | Welfare entitlements, Credit and debt | | Evaluation after 15 months operation. Questionnaires completed by 79 host agency staff and 153 practitioners. Focus groups of clients (1 group) and other ‘target’ users of the health service (2 groups). Compared surgeries with advice clinic and some without. |
| Smith & Patel, 2008 | | Money advice outreach evaluation: cost and effectiveness of the outreach pilots | | England & Wales | | 100+ outreach locations - Welfare agencies, credit unions, housing offices, schools, community centres, family and children centres, Citizens Advice Bureaus (CABs) and prisons | | Credit and debt | | Analysis of 12 months data commencing after most projects had been running for at least 6 months. Data included: monthly monitoring data from participating agencies; closed case data from 17 of the 22 agencies, including 4,885 client records; records of 90,560 clients provided advice under mainstream Legal Service Commission contracts (comparison group). |
| Westwood Spice 2005 | | Evaluation report: Homeless Persons’ Legal Service | | NSW, Australia | | 6 homeless persons services | | Civil law | | Evaluation after just over a year of operation. Evaluation method: program evaluation, face to face interviews, telephone interviews, written surveys, focus groups, document analysis, including statistical reports. Consulted with clients, lawyers, welfare agency staff, other service providers. |
The fact that studies only had to meet at least half of the criteria above to be included in our review reflects a number of issues. First, these criteria were designed to assess qualitative studies reported in academic literature in the health field. Nearly all of the evaluation reports and studies we located were mixed method studies written for policy makers in the legal sector, rather than for an academic audience. So, for instance, when assessing whether there is congruity between the stated philosophical perspective and the research methodology (criterion number one, above), there was no philosophical perspective stated in some papers, and therefore we could not make this judgement. However, the number of criteria accepted for inclusion also reflects the paucity of rigorous research on outreach legal services, particularly in Australia. While we have confidence that the studies included in the review represent the best available evidence on outreach legal services to disadvantaged people, the scale and quality of the studies we included did vary.
Some of the studies included in the review were evaluation reports on single services, while other studies reviewed outreach programs over a range of sites, with a variety of client groups. It should be noted that four studies (Buck et al, 2007, Day, Collard & Hay, 2008, Day, Collard & Davies, 2008, Smith & Patel, 2008) were part of an extensive and well funded series of evaluation research projects which separately evaluated different aspects of the same very large scale program of outreach services across the UK. We made the decision to include all four, as the studies were discrete and examined quite different aspects of this program of services.
Smith and Patel (2008) is a cost-effectiveness study, drawing on economic and administrative data. We therefore also assessed the quality of this study against the JBI criteria for economic data (see http://). Smith and Patel (2008) met the key criteria for both sets of criteria.
4. Data extraction and synthesis
Once the eleven studies were selected for inclusion in our systematic review, we undertook the following data analysis to produce the findings presented in this report. To begin with, each researcher identified and extracted key findings nominated by the studies' authors, and evidence which supported those findings. Key findings are conclusions reached and reported by the author derived from a thematic analysis of the data. Evidence is the data from which the findings are derived. A quote from the report or data reported was recorded against each finding to illustrate the best evidence from the data presented to support the finding. The evidence for each finding was ranked according to three categories:
- 'unequivocal' for findings that are clearly and directly supported by the data in the body of the report
- 'credible' for findings that can be logically inferred from the data
- 'not supported' for findings that are not supported by the reported data.
Any findings which were not supported were excluded from the review. Findings which were credible were only used in collaboration with other unequivocal findings.
We then grouped the findings from all the studies into thematically similar categories. The next step was to synthesise these categories into broader findings called 'synthesised findings', which could then be used to derive best practice principles. The data extraction and synthesis was undertaken using the Qualitative Assessment and Review Instrument (QARI), a standardised JBI data extraction software tool for qualitative data (see ).
A total of 226 findings were extracted from these eleven studies, and then grouped into 53 categories. A 'thematic' meta-synthesis, based on the amalgamation of related categories, was then used to produce 19 synthesised findings (see Table A.1 in Appendix 1 for a list).
It should be noted that only one study specifically examined cost effectiveness of outreach advice services compared to other models of service delivery. While we have included findings from this study in this review, we also highlight cost effectiveness as a gap in research, particularly in Australia.
Results
The synthesized findings from the eleven studies included in the review are organised around our two research questions:
- The effectiveness of outreach legal services
- The features of effective outreach legal services.
Under these two sections, the key synthesised findings (themes) are indicated by bolded headings (italicised and otherwise). Examples are drawn from individual studies to illustrate the points.
The effectiveness of outreach advice services
Outreach services can reach target clients
The review identified that quality outreach legal services can reach clients with complex needs and who had not sought assistance before from mainstream legal service providers, or who otherwise would not have received legal assistance. However, to achieve these outcomes, outreach services need to be appropriately located and connected with target groups and their support agencies. The additional features of outreach services that did successfully reach and assist clients with complex needs are detailed below under features of effective outreach services.
-
Analysis of the monitoring data and closed case client records show that the projects were very successful in reaching clients who were both financially excluded and who had not previously sought advice. (Smith & Patel, 2008, p. 4)
Outreach legal services can provide positive outcomes for clients
The review indicated that effective outreach legal services can improve the circumstances of clients with complex needs. This could happen in a number of ways.
-
The advice services provided led to a whole range of positive outcomes for clients, including payment plans, and averting the loss of a home or utilities disconnection. In terms of measurable financial outcomes, the pilot projects gained £1.875 million as income for their clients in the one year reference period. (Smith & Patel, 2008, p. 8)
First there was substantial evidence in the studies reviewed indicating that legal assistance through outreach services to people with complex needs can provide positive legal outcomes and improve clients' circumstances, in those areas of law provided by the services. These included housing and tenancy, the management and reduction of debt, management of fine related debt and driving restrictions, access to children and other family matters, access to welfare payment and Centrelink matters. However, many clients also had legal issues which were beyond the scope and/or expertise of the outreach service (e.g. crime, family law). In these cases services tended to provide legal information or would have to refer the matter elsewhere.
The results also suggest that advice through outreach may prevent legal issues from escalating in seriousness which occurs when problems remain unaddressed. However, there is a need for further research in Australia to actually quantify the benefits (e.g. in terms of reduced debt, additional income and saved homes) of providing legal assistance to disadvantaged clients, who otherwise would not have accessed legal help.
Even when legal assistance did not result in desired outcomes, some studies found that the assistance provided to marginalised clients raised clients' self confidence, self esteem and their capacity to take responsibility for their matters. Clients spoke of the assistance provided acting as a 'circuit breaker', enabling them to 'get on with' other areas of their lives. Examples were given of clients being more able and prepared to seek further assistance for subsequent legal issues following the assistance of the outreach service.
-
The HLC had settled the mind, calmed them down, reduced stress, broken impasses and provided a new start in many cases... Others stated that the assistance provided had motivated them to help themselves more and given them a starting point. (Kilner, 2007, p. 9)
A number of studies noted the impact of the advice on clients' stress levels and overall health. These studies indicated that debt advice in particular could reduce clients' anxiety. This was chiefly so in cases where action could be taken to manage situations which were 'out of control' (e.g. threatening calls from creditors, immediate threat of eviction) and in cases where clients have long term health problems, which had been exacerbated by the stress. One study surveyed doctors with and without welfare rights advice outreach services on their premises and found that doctors with welfare services were significantly more likely to agree that "welfare rights provision improves the health and well being of their patients" (Sherr, 2002 p.68). Smith and Patel (2008) postulate that better health results for clients receiving debt advice saves money spent on the public health system.
-
The research also showed that people getting advice experienced benefits in terms of lower anxiety, better general health, relationships and housing stability. Therefore, beyond the immediate impact of the money advice, the outreach projects can be expected to have generated a range of important additional benefits for clients, and, in turn, saved associated public expenditure, for example, the demands on the health service caused by physical or stress-related ill health. (Smith & Patel, 2008, p. 47)
Impact on the target group through advocacy and law reform
Some of the outreach legal services included in our review used a model which combined case work, systemic advocacy and law reform to address the needs of very marginalised clients. The rationale for including advocacy and law reform in the work of the service was that, on the basis of what is learned through the outreach service, strategic action can be taken to improve the ways laws affect all people in the target group — including those who are so marginalised that they don't even reach outreach legal assistance. The models reviewed indicate that, if appropriately resourced to do this work, outreach legal services could bring the experiences of their constituents to advocacy and law reform.
Impact on host agency
Our review indicates the placement of outreach legal services at host agencies could be mutually beneficial for each service. Host agencies were happy to accommodate outreach legal services, finding that their own service delivery to these 'hard-to-reach' clients was streamlined and the effectiveness of referrals increased. The externally funded outreach legal services assisted the caseworkers to progress their clients' matters without being a drain on the host agencies resources.
-
The clinic relieves the burden on the agency, rather than being a burden. (Goldie, 2003, p. 67)
The co-location of the outreach service, and the relationships built between the services provided opportunities to increase the knowledge, skills and networks of the host agency workers, and lawyers alike.
Features of effective outreach services
Pre-planning and needs analyses
A feature of effective outreach legal services is that they reach clients who otherwise would not have received legal assistance. However, the review found that it is important for services to have undertaken appropriate legal needs and service gap analyses prior to the outreach service being established. This involved engaging with other organisations and services from the planning stage to ensure that the outreach legal services were not duplicating existing legal services in the area. Early engagement with other organisations had the additional benefit of streamlining referral processes both to and from the outreach legal service. This was identified as important in the studies, as clients with complex needs may need support from a range of agencies (e.g. housing, welfare, other or specialist legal services) in order to address the range of issues that may be affecting them.
Linking with clients
An overarching theme — and three key findings of this review — concerned how outreach legal services actually connect with 'hard-to-reach clients'. Outreach services need to consider how clients will find out about their service and be encouraged to use it, bearing in mind that this client group is unlikely to actively seek assistance, particularly from a service it is unfamiliar with. In the studies reviewed, three main strategies were successfully used to 'reach' clients: appropriate location, effective referral to the service, and direct marketing to the client group.
Location
Our review indicates that a defining feature of effective outreach legal services is that they are located in places that can be easily reached (e.g. by public transport) and are regularly used by target client groups. Outreach legal services included in our review were located in welfare agencies and homelessness services, community centres, doctors' surgeries, Aboriginal Legal Services,6 mental health services, community credit unions, child and family centres and prisons. The appropriateness of different sites varied with the type of clients the service was aiming to reach and the type of service which was being delivered. Overall, however, our review indicates that successful outreach locations are places:
-
Co-location with Aboriginal medical services is an option that should be considered in the development of new outreach services. However it is too simplistic to expect that by merely co-locating outreach legal services with health services, that this will of itself provide effective legal services for disadvantaged Aboriginal people. (Dimos, 2008, p. 64)
- where there is currently a gap in legal service delivery to the target group
- which the target group are familiar with and trust, such as places they already access
- where (to ensure potential client numbers) there is a flow of target-group clients through the service. For instance, more clients access a community centre or welfare agency on a given day than a hostel which has a small number of residents staying for several months
- that are physically accessible to target clients, including clients with disabilities and clients living in remote or regional areas
- that have private spaces in which clients and advisers can meet and discuss confidential issues.
However, as will be illustrated below, while placing the outreach service in an appropriate location is important, this alone will not guarantee that clients will access the service.
Effective referral pathways
Building relationships with key 'problem noticers', often in the host agency where the service was located, was a key strategy used to link potential clients with the outreach service. 'Problem noticers' was a term applied in some studies to those host agency staff, staff from other agencies and community members who may notice a client has a legal problem and refer them to the outreach legal service. It was clear from the studies reviewed that simply locating a service on-site at a host agency was usually not enough to generate sufficient and appropriate referrals to the service.
To most effectively use the host agency or community members as a source of referral, our review indicates that:
- initial and ongoing relationships need to be forged between the host agency (and/or other referring agencies) and the outreach legal service
- 'problem noticers' benefit from training by the outreach service in how to identify whether clients have relevant legal issues and what the outreach service can do for their clients
- the best sources of referrals are sources who are already trusted by the client group, such as case workers, community members or friends.
One pilot project reported that it had provided training to family support workers around how to identify clients in financial difficulty and make referrals. Earlier research into the delivery of advice in health care settings highlighted the importance of delivering formal training for staff in the role of problem noticers. (Day, Collard & Davies, 2008, p.31)
Marketing the service
The review also identified the importance of marketing the service directly to target clients and communities, and the agencies and individuals who support them. This strategy was particularly useful when host agency referrals were not sufficient to generate clients. The studies indicated that it is not only important to market the existence of a service, but also to outline what the service can do for clients and how they can contact the service.
As 'hard-to-reach' clients tend not to approach services they are unfamiliar with or do not trust, communication and marketing through existing trusted services or people appeared most effective. Indeed, services commonly reported reaching new clients through word-of-mouth referrals from existing clients, once the service became established. The review suggests that some ongoing marketing through host agencies and other local services may also be necessary, and that promotional strategies need to be appropriate to the target audience, for example, culturally appropriate and in plain language. This may involve, for example, explaining or using examples to describe more abstract concepts such as 'civil law'.
Collaboration with the host agency
Another key finding of the review concerned the importance of using both formal and informal mechanisms to sustain an ongoing relationship between the host agency and outreach legal service. Ideally, the relationship between the outreach service and host begins at the planning stage and involves strategic and operational issues. Communication is vital to support ongoing relationship-building, to maintain the flow of appropriate referrals and to address any issues that arise.
Successful strategies identified to maintain communication included shared planning days, regular staff meetings, informal communication between host agency and outreach staff (e.g. in a common tea room) and training for the host agency by the outreach staff and vice versa. Formal resolution processes for disputes between the host agency and outreach service were also put in place in some services.
-
... pilot projects agreed that ongoing work was needed to maintain good working relationships and adequate numbers of referrals from partner organisations. In practice, this meant keeping the project high profile among partner organisations, for example by advisers attending staff or team meetings on a regular basis, making presentations to partner organisations to remind them about the project and its potential benefits for their clients, or informal networking with staff. (Day, Collard & Davies, 2008, p. 31)
Host agencies were an important source of referrals for the outreach legal services (see linking with clients, above). However, in some cases, host agency staff also played an active role in the provision of legal assistance to their clients. For instance, some caseworkers sat in on client interviews, assisted clients to gather necessary documentation and ensured clients met appointment times or court dates.
Service delivery to people with complex needs
As well as actually reaching 'hard-to-reach' clients, several findings of the review identified the importance of providing legal assistance in a way which is appropriate for clients with complex needs. Clients with complex needs may have multiple intersecting legal and non-legal issues. While they may only come to an advisor or lawyer about one issue, this issue may well be bound up with other issues in their lives. In addition, due to the nature of their disadvantage, these clients tend to have difficulties in working with lawyers and dealing with their legal issues. They may have cognitive impairment or literacy issues which affect their interactions with lawyers and others. They may feel intimidated and lack trust in the prospect of dealing with lawyers and may feel embarrassed about seeking assistance (particularly for debt related problems). They may not always attend appointments, may not have necessary documentation and may be difficult to locate for follow up assistance.
Familiarity and trust, providing flexibility, timely services, consistency, confidentiality, and communicating effectively with clients were all identified as key features of effective outreach service delivery to clients with complex needs. Each of these findings is discussed below. Some studies also suggested that clients with complex needs benefited most from concrete assistance, not just advice. Assistance may include writing letters, making phone calls, preparing documentation and in some cases, providing representation in courts or tribunals.
Familiarity and trust
The studies reviewed also stressed the importance of building trust, in order to encourage clients to access the outreach legal service. Clients were more likely to engage with services and individual advisors with whom they were familiar and they trusted. This could take time to develop. As discussed above, locating services in host agencies that were already known and trusted by clients helped to facilitate relationships between clients and the outreach legal service. A number of studies also observed the benefits of high quality legal assistance not only to clients and their outcomes, but in terms of building a good reputation among clients and other services.
-
Aboriginal people need to be able to trust someone before they are accepted. When referring people, the Aboriginal Legal Service needs to be satisfied that they are being referred to someone they can trust. (Dimos. 2008, p.43)
Flexibility
All the outreach legal services included in this review focused on particular areas of law — commonly debt and money advice, or 'civil law' more broadly. However, clients who came to these services often also had legal problems which were beyond the scope of the service (e.g. criminal or family law issues). Given the barriers that people with complex needs face in accessing any service, several studies stressed the need for services to be flexible in the range of legal issues they address. Where particular issues could not be addressed by the outreach lawyers, effective referral to other specialist support was considered vital.
Appointment times or drop-in services?
The outreach services reviewed also needed to cater for the chaotic and transient nature of the lives of many of their clients. Studies found that if clients could not 'drop in' on the spot or make an appointment as soon as possible, some simply did not come back. Some services used appointments and others ran a drop-in service. Both types of service provision had mixed results. Drop-in services were reported to be less efficient for service providers. However, drop-in services were often effective in dealing with clients at the time they were ready for action to be taken. A number of services that used an appointment system had difficulties with clients not attending as scheduled.
Strategies employed by services to reduce non-attendance by clients included telephoning or sending text messages to clients on the day of their appointment, having field officers locate and provide transport for people for appointments (a strategy used in small Indigenous communities), and timing legal appointments with clients' visits to the host agency for other services.
Staying in contact with transient clients
One example of the need for and application of flexible practices concerned the strategies identified to assist transient clients. Many studies identified prompt access to assistance as a key strategy to keep in contact with transient clients. However, other strategies employed to stay in contact with clients included:
- providing the client with clear information about how to get back in contact with the service (e.g. a business card with the name of the lawyer)
- taking several different contact numbers, addresses or the details of contact people from clients at the first meeting.
Free service
Only one study specifically reported findings concerning the impact of paying (or not) for the outreach legal service provided. In that case, the fact that the legal service was free was seen to increase its accessibility. This finding was reported across a number of outreach sites reviewed in that study.7
Timeliness of advice and action
The studies reviewed reported that clients with complex needs were often slow to seek help and may well be at crisis point by the time they get to an outreach legal service. For these reasons, services ideally need the capacity to provide assistance as soon as possible. One study found that the timeliness and relevance of advice was more important to clients than their familiarity with the service, particularly when issues were at crisis point.
-
Fast action is often required due to pressing circumstances and also because the client cannot be relied upon to take follow up action or pursue referrals. (Kilner, 2007, p. 19)
Confidentiality
As outreach legal services may be provided in places where clients are known to others, confidentiality and the importance of having a private space in which to meet an advisor were identified as key issues. In the studies we reviewed, clients seeking debt related advice were particularly concerned about confidentiality.
Consistency/continuity of service
Consistency and continuity in service provision were also highly valued by clients across the studies, with clients preferring to see the same lawyer (a familiar face) each time. Consistency is easier in services with regular paid staff, compared to clinics which operate with a roster of volunteers. However, even some outreach services with paid staff had difficulties with staff turnover. Where continuity of staff is not possible, particularly if matters continue over several months, continuity of service delivery, consistency of advice and handover strategies were considered vital. The location and management of files was an important aspect of maintaining continuity (see 'record keeping' below). The use of precedents and procedure manuals assisted some services with maintaining continuity.
Effective communication between legal advisers and clients
The review indicates that clients with complex needs benefited from and appreciated friendly, approachable and respectful legal advisers, who took time to explain things to the clients in plain language.
Clients also highly valued lawyers who had the ability to listen to them. Not only does the communication between legal advisers and clients take skill, but consideration must also be given to the time that advice sessions may take (see resourcing below), particularly when there are complex issues to unravel.
-
The lawyers are always friendly and easy to understand. They simplify things. It was hard to speak to [the lawyer] at first but he made things easy. He explained things to me in an easy way. He was quite thorough. They don't make you feel like an idiot. (Goldie, 2003, p. 32)
This type of effective communication between lawyers and clients was noted to increase client confidence in the process and outcomes (even when these may not have been the outcomes they had hoped for) and their capacity to understand and address their legal issues. Effective communication was also a cornerstone of building the level of trust required to sustain both the client in dealing with their particular matter(s), and the service more broadly.
Staffing, skills and training
The review has highlighted that effective legal outreach involves more than the work of legal advisers alone. Here we discuss the range of roles involved in effective, sustainable outreach legal services and the skills/training required for these roles.
Legal advisers
The review identified skills needed by legal advisers to work in outreach with clients with complex needs. Ideally, outreach lawyers:
- have sufficient expertise in the relevant areas of law, recognising that the issues facing clients may be complex/at crisis point
- have some level of knowledge across a number of different areas of law, as clients may come to them with a range of intersecting legal problems
- are aware of where and how to refer clients who have issues beyond their expertise or role, including understanding the role and capacity of the host agency
- are skilled in communicating and working with the target group, including clients with chaotic lives, mental health issues and cognitive impairment.
The studies reviewed indicate that it can be difficult to recruit legal advisers who have all of these skills and knowledge.
-
... I sometimes do also have concern about the advice and areas of law that arise although I do think that I keep my head above water. (Lawyer quoted in Goldie, 2003, p. 82)
For this reason, it is likely that training will be required in one or more areas. In addition, cultural awareness training may be required for legal advisers working with Indigenous Australians and other culturally diverse communities.
The review also indicates that all lawyers or advice staff benefit from training in:
- the operation of that particular outreach service including administrative and record keeping requirements, particularly as they pertain to managing the files of transient clients
- appropriate referral, including information about the legal and social services available to their clients and the role and work of the host agency
- safety issues.
The review found that it may be useful to incorporate an ongoing program of training, given issues of staff turnover in a number of the outreach programs reviewed. Notably, some studies observed how the experience of working in legal outreach itself enhanced the skills of the legal staff.
Consideration must also be given to the appropriate supervision of legal work undertaken (particularly if junior lawyers are involved) and the availability of specialist advice to the lawyers.
Para-legal staff
The review identified that clients with complex needs may not always require high level 'legal' skills to address their legal needs. Some clients required para-legal help, for instance, in filling in forms or making phone calls. One suggestion to address this issue was to include either voluntary or paid para-legal staff within the service.
Coordinator
It was clear from the review that appropriately resourced coordination and administration are also intrinsic to the success of outreach legal services to people with complex needs. The tasks undertaken by coordinators included:
- developing and sustaining relationships with host agencies and other problem noticers
- developing relationships with other service providers assisting the target client group, for onward referrals
- accessing specialist support for legal staff, including (pro-bono) assistance from barristers
- managing funding, funding reporting requirements and program administration
- coordinating recruitment and training for outreach staff and training for host agency staff.
Where there is no coordinator, these tasks may need to be factored into the advisor's time.
Costs and resourcing
The findings on the costs of outreach services are not straight forward.
A feature of outreach legal services reviewed here, is that they served disadvantaged clients with complex needs. A number of studies found that it can take more time, more resources and more skills to effectively reach and assist these clients -particularly when clients have multiple legal problems and/or other complex needs. Through statistical modelling, Smith & Patel (2008) found that advice time did vary between different client groups.8 However, when the number and type of debt problems were factored into the model, the characteristics of the debt (number and type) were found to be stronger drivers of advice time than the degree of financial exclusion of the client.9 Advice time also varied depending upon the organisation at which the client received the advice.
Smith & Patel (2008) also observed that, in terms of total project funding, outreach cases cost more than mainstream advice services. However, if costs are based on a cost per hour of advice, they found that debt advice provided to disadvantaged clients through outreach can take less time per client, and therefore cost less, compared to providing debt advice to these clients through mainstream legal services. Smith & Patel (2008) conclude that the additional cost of outreach relates more to the overhead or 'fixed' costs of outreach, than the advice sessions themselves. Synthesising the results from all the studies, the fixed costs associated with effective outreach include:
- coordination and administration
- supervision and training for advice staff
- ongoing collaboration with and training for host agency staff and problem noticers
- marketing costs for the outreach service
- technology and infrastructure (telephones, internet access, intranet access where appropriate)
- in some cases, case management of clients who require ongoing or more active support.
Notably it is these very features and their associated costs which our review has identified as the elements that make outreach services more effective.
That said, resourcing requirements will also vary with the host site. For example, outreach to prisons presents particular issues in terms of access, the availability of inmate clients due to lockdown and limited access to technology by advisers (e.g. no mobile phones or internet access) when visiting prisons (see also Grunseit et al, 2008). Other issues such as non-attendance at appointments can also increase overhead costs.
Technology and infrastructure
The review identified that access to appropriate technology at outreach sites facilitated quicker and more effective service delivery. For instance, access to internet (or intranet where appropriate) enabled legal advisers to undertake research, access precedents and produce letters and documents on the spot. Access to client files from outreach sites (in hard copy or electronically) allowed for immediate assistance to transient clients, who may appear for assistance unannounced. Online file management could also assist in monitoring the quality of services provided.
Record keeping and review
Effective record keeping was identified as an important feature of successful outreach legal services. It enabled appropriate supervision of files and staff, allowed the project to be monitored and reviewed, and was used to mount a case for ongoing project funding. As discussed earlier, effective record keeping also helped the service stay in touch with and assist transient clients when they did appear.
All of the programs reviewed had received funding from external funding streams (e.g. Government grants). Appropriate monitoring and review processes were generally a condition of the funding. A key issue in the studies reviewed concerned the capacity of services to meet reporting requirements. Some studies reported lawyers not completing reporting/administration requirements as they were 'too onerous' or because they felt uncomfortable asking clients questions about issues such as disability.
Another issue raised in the review was the need to set realistic and appropriate targets when working with 'hard-to-reach' clients through outreach. As discussed earlier, outreach has a range of necessary but fixed costs associated with its set up and ongoing support. In addition, target clients may be more challenging to assist. This may translate into longer timeframes, fewer clients and higher costs.
Conclusion
'Outreach' is a strategy commonly used as a way to reach and provide legal services to very disadvantaged people isolated from legal help. To inform evidence-based practice, we have systematically reviewed the best available research evidence on outreach legal services to disadvantaged people with complex needs. We have examined whether outreach legal services can reach and assist disadvantaged people with complex needs and the features of those services which contribute to their effectiveness and sustainability.
The 'best available research evidence' on outreach legal services that we have located and reviewed in this report is a set of eleven evaluation reports on outreach services in Australia and the United Kingdom. While they do vary considerably in scale, these studies generally evaluate outreach legal services conducted over several sites. Four of the UK studies review a large scale nationwide program of outreach advice services to disadvantaged people.
The evidence we have identified in this review tells us that properly resourced, appropriately placed outreach legal services, which have solid links to their client group, with skilled advisers and strong referral networks can:
- reach disadvantaged clients with complex legal needs and who otherwise would not have received legal assistance
- assist these clients to address their legal issues, improve their circumstances and may increase their willingness to seek assistance in future
- reduce the stress and anxiety associated with having outstanding legal issues, such as debt related issues
- reduce the burden on host agencies by providing them and their clients with access to specialist support/expertise.
There is also some evidence that outreach debt advice in particular, may have direct financial benefits for clients (e.g. in terms of reduced debts, increased income and a 'clean slate'). Further, timely advice can prevent issues from (further) escalating in seriousness because they remain unaddressed.
While the exact format of any outreach service will differ depending upon the target group, the nature of the legal issues and the specific context of the program, there were a number of observations and findings made across the studies about the features of outreach services which contribute to success. These findings may assist services in the design, implementation and ongoing delivery of sustainable and effective outreach legal services to these target groups. In summary, our analysis suggests that best practice outreach legal services to disadvantaged people with complex needs have the following characteristics.
Planning and collaboration
Services are planned and are established in consultation with other legal and non-legal services which assist the target group, and/or the target group themselves. This is important to ensure the service fills a gap in services and to develop ongoing referral pathways in and out of the outreach services. The importance of maintaining ongoing formal and informal communication with any host agency and other networks was also stressed.
Linking with clients
To reach clients with complex needs, services are best located in places that are frequented and trusted by the target groups and which have a flow of clients through that service. Host locations need a private space in which the outreach service can be provided. However, appropriate location is not necessarily enough to reach clients. The reach of the service is also increased by:
- effective referral systems with agencies (including the host agency) and individuals who are known to and trusted by the target group
- services being appropriately marketed to the target group and the front line workers and community members supporting them.
Appropriate service provision
To ensure that the legal assistance provided is appropriate for the client group, services are ideally flexible in their service provision. Services need the capacity to act quickly if necessary and to spend time with clients who require more intensive support. Further, services need strategies to stay in touch with transient clients. High quality approachable services, consistency of service and confidentiality were also identified as key characteristics of effective legal outreach to 'hard-to-reach' clients.
Staffing and resourcing
A number of studies in our review noted that it can take more time, more resources and more skills to effectively reach and assist these clients — particularly when the service aims to assist clients with a range of their legal problems. However, there is also some evidence that much of the additional cost of outreach relates to the overhead or 'fixed' costs of outreach, than the length of the advice sessions themselves. Nevertheless, it is these additional fixed costs — for planning and set up, administration and coordination of the service, ongoing collaboration with host agencies and other services, and the ongoing training and supervision of legal advisers — which are needed to make outreach services work effectively. Services also appear to work more effectively with access to appropriate technology, such as access to telephones, internet and electronic filing systems.
Outreach lawyers ideally have expertise in the relevant areas of law as well as skills in working with clients with complex needs. Clients with complex needs appreciate lawyers who are approachable, respectful, skilled at explaining legal issues in plain language, and able to appropriately refer clients for additional assistance where necessary. The studies reviewed recognised the difficulty in finding staff with all the requisite skills.
Monitoring and review
In terms of monitoring and review, services and their funding bodies need to have realistic targets and expected outcomes. Funders need to have realistic expectations of the time and resources it may take to reach and assist very marginalised clients with complex needs.
Knowledge gaps
The findings reported above are drawn from the best available research evidence on the design and impact of outreach legal services to disadvantaged people with complex needs. While there was a notable consistency in the findings of these studies, the scale of the studies reported, and the outreach programs they report on, varied significantly. While this report goes some way to filling a knowledge gap on the effectiveness of legal outreach, the review process we have followed highlights the need for further high quality independent research on the design and impact of outreach legal services, particularly in Australia. In particular, further high quality research in Australia is needed on the:
- effectiveness of legal outreach for different client groups, different legal issues and in different locations
- relative costs and benefits of outreach legal assistance to 'hard-to-reach' clients and the community compared to other service delivery models
- role of community legal education in outreach services
- longer term impact of outreach legal assistance on 'hard-to-reach' disadvantaged clients.
Reflections on the research
This review has relied heavily on a methodology developed for the review of qualitative health data. The methodology has provided this review with a level of rigour and transparency which we believe increases the validity of the results reported here. However, our pilot of this methodology indicates that there is scope for this method to be refined to better define, identify and synthesise high quality research and evaluation methodologies commonly used in socio-legal research. This involves revisiting the assessment criteria to assess a broader range of methodologies (including survey results and case file and other administrative document analysis). We will give further thought to the boundaries between qualitative and quantitative information, what is understood as 'quality' in different forms of research and in the combination of different methods (e.g. through triangulation of data). There is also scope to more tightly define what is understood by and accepted as 'evidence' and as 'findings' in the qualitative research that we review.
Further, the style of report which is appropriate to inform policy in the legal sector differs to the types of reports created for academic audiences in the health field. These differences will be factored into future 'what works' research undertaken by the Foundation.
Reference list
Studies included in the review
Buck, A, Tam, T & Fisher, C 2007, Putting money advice where the need is: evaluating the potential for advice provision in different outreach locations, LSRC Research Paper no. 16, Legal Services Research Centre, London, Available at: <http://www.lsrc.org.uk/publications/Outreach.pdf>.
Day, L, Collard, S & Hay, C 2008, Money advice outreach evaluation: qualitative outcomes for clients, Legal Services Research Centre, London, Available at: <http://www.lsrc.org.uk/publications/DebtOutreachOutcomesClients.pdf>.
Day, L, Collard, S & Davies , V 2008, Money advice outreach evaluation: the provider and partner perspectives, Legal Services Research Centre, London, Available at: <http://www.lsrc.org.uk/publications/DebtOutreachProviderPartnerPerspectives.pdf>.
Dimos, D 2008, Civil law ALS Outreach review, Legal Aid NSW, Sydney, Available at: <http://www.legalaid.nsw.gov.au/data/portal/00000005/public/24799001229058541336.pdf>.
Gillespie, M et al. 2007, Money advice for vulnerable groups: final evaluation report, Scottish Executive, Glasgow, Available at: <http://www.scotland.gov.uk/Resource/Doc/171671/0047978.pdf>.
Goldie, C 2003, PILCH homeless persons' legal clinic: evaluation report, Gilbert + Tobin Centre of Public Law, Sydney.
Hartlepool New Deals for Communities 2004, Evaluation report on the money advice and debt counselling service project, Hartlepool, UK, Available at: <http://www.ndc.org.uk/documents/MoneyAdviceandDebtCounsellingService.doc>.
Kilner, D 2007, Housing Legal Clinic final evaluation report, Welfare Rights Centre SA, unpublished, Adelaide.
Sherr, L, et al. 2002, A stitch in time: accessing and funding welfare rights through Health Service Primary Care. An evaluation of primary care based specialist welfare rights advice provision in Lambeth, Southwark and Lewisham, London, Available at: <http://www.lho.org.uk/viewResource.aspx?id=8763>.
Smith, M & Patel, A 2008, Money advice outreach evaluation: cost and effectiveness of the outreach pilots, LSRC Research Paper no. 22, Legal Services Commission, London, Available at: <http://www.lsrc.org.uk/publications/DebtOutreachCostEffectiveness.pdf>.
Westwood Spice 2005, Evaluation report: Homeless Persons' Legal Service, PIAC, Unpublished, Sydney.
Additional references
Australian Government Social Inclusion Unit 2008, Social inclusion: origins, concepts and key themes, Commonwealth of Australia, Canberra, Available at: <http://www.socialinclusion.gov.au/Documents/PMC%20AIFS%20report.pdf>.
Brackertz, N & Meredyth, D 2008, Social inclusion of the hard to reach: community consultation and the hard to reach: local government, social profiling and civic infrastructure: final report, Swinburne Institute for Social Research, Hawthorn, Victoria, Available at: <http://researchbank.swinburne.edu.au/vital/access/manager/Repository/swin:9858>.
Buck, A 2009, 'Reaching further through outreach advice', in Reaching further: innovation, access and quality in legal services, Stationery Office Books, London, pp. 73-99.
Campbell Collaboration 2009, What is a systematic review? Available at <http://www.campbellcollaboration.org/what_is_a_systematic_review/index.php>.
Coumarelos, C, Wei, Z & Zhou, A 2006, Justice made to measure: NSW legal needs survey in disadvantaged areas, Law and Justice Foundation of NSW, Sydney, Available at: <http://www.lawfoundation.net.au/report/survey2006>.
Forell, S, McCarron, E & Schetzer, L 2005, No home, no justice? The legal needs of homeless people in NSW, Law and Justice Foundation of NSW, Sydney, <http://www.lawfoundation.net.au/report/homeless>.
Government Social Research Unit 2007, What do we already know? Harnessing existing research. Background paper 2 of the Magenta book: guidance notes for policy evaluation and analysis, HM Treasury, London, <http://www.gsr.gov.uk/downloads/magenta_book/Chap_2_Magenta.pdf>.
Grunseit, A, Forell, S & McCarron, E 2008, Taking justice into custody: the legal needs of prisoners, Law and Justice Foundation of NSW, Sydney, Available at: <http://www.lawfoundation.net.au/report/prisoners>.
Johanna Briggs Institute 2008, Johanna Briggs Institute reviewers' manual 2008 edition, Johanna Briggs Institute, Adelaide.
Johanna Briggs Institute Johanna Briggs Institute Qualitative Assessment and Review Instrument (QARI), Available at: <http://www.joannabriggs.edu.au/services/sumari.php>.
Legal Aid NSW 2008, 'Spotlight on outreach services', Verbals, no. 51, pp. 7–19, Available at: <http://www.legalaid.nsw.gov.au/data/portal/00000005/public/34824001223505244172.pdf>.
Schetzer, L, Mullins, J & Buonamano, R 2002, Access to justice & legal needs, a project to identify legal needs, pathways and barriers for disadvantaged people in NSW. Background paper. Law and Justice Foundation of NSW, Sydney, <http://www.lawfoundation.net.au/report/background>.
Appendix 1
SEARCH STRATEGY
A three stage search strategy was used.
First stage
The first stage of the search strategy consisted of a broad but shallow review of academic databases (legal, socio-legal, cultural studies, health, education and social science) and the websites of government bodies and research institutions using pre-defined search terms. The databases, websites and bibliographies searched are detailed in Appendix 2.
This first stage focused predominantly on locating articles or reports on legal outreach to people with complex needs. However this first stage did not, at least at first, exclude relevant looking articles on evaluations of outreach to people with complex needs in other related fields (e.g. health, education, etc).
This stage was carried out by two researchers and two librarians. At this stage only bibliographic information on each article, including a web-link and abstract, where available, were collected. Of the hundreds of reports originally identified, 98 most relevant to outreach legal services were entered into a referencing library (EndNote Version 6.0).
Search terms
The search terms used to search both databases and websites were:
| Strategy type: | Outreach, co-location, hub, one-stop shop, interagency, early intervention, legal clinic, advice clinic, Video-link advice (not court procedures) |
| Study type: | Evaluation, review, what works |
| Area of law: | Civil law, credit and debt, domestic violence/family violence/DV, family law, criminal law, financial planning, tenancy/housing, poverty law, multiple/clustered legal need/problems |
| Participants: | Complex needs, Indigenous, Aboriginal, mental illness, (intellectual) disability, homeless, caravan parks, rural, regional and remote, lone/single parent (families), disadvantaged, socially/ economically/ financially excluded, isolated/hard-to-reach, prisoners |
Use / hierarchy of search terms
We searched first on 'strategy type' and 'participants'. 'Area of law' and 'study type' were used to limit the search. The other search terms were kept in mind when we selected manually from a list of returns.
Second stage
The second stage involved a review of returns from stage 1 to identify relevant articles and reports that were specifically on evaluations of legal outreach to people with complex needs, based on the information provided in the title and abstract and full text if available. Any reports identified as relevant at this stage were retrieved in full text or abstract form, and again assessed against the inclusion criteria for relevance to the review question.
Third stage
The third stage involved a search of the references and footnotes of the most relevant reports. This involved returning to websites for more targeted searches, as well as contacting services to get copies of evaluation reports which were not publicly available. Reports on evaluations of legal outreach to people with complex needs found at this stage were retrieved and assessed against the inclusion criteria for relevance to the review question. This process is summarised in Table 1 in the main body of the report, while the studies identified at stage 2 and 3 of the search are identified in Appendix 3.
Thirty-nine reports were found to be of relevance during stages 2 and 3 and assessed against the inclusion criteria for relevance to the review question. Of these 16 were found to be evaluation reports on outreach advices services to people with complex needs, and were therefore selected for appraisal.
SYNTHESISED FINDINGS AND CATEGORIES
As described in the methodology section of the report, data synthesis involves identifying findings, grouping findings into thematically similar categories and then grouping these categories into broader findings (synthesised findings) which can then be used to derive best practice principles.
A total of 226 findings were extracted from the study, and then grouped into 53 like categories. From these 53 categories a 'thematic' meta-synthesis based on the amalgamation of related categories was used to produce 19 synthesised findings. These are listed below.
Table A1: Synthesised findings and categories
| 1 | Impact – reach | |
| a. | Outreach does reach clients with complex needs |
| 2 | Impact on clients – effectiveness | |
| a. | Consultation with lawyers can improve clients' self esteem and outlook, even when an issue can't be resolved |
| b. | Legal assistance can act as a circuit breaker and enable/motivate clients to move on with their lives |
| c. | Legal assistance can increase the willingness of clients to seek help again and to manage issues better in future |
| d. | Legal assistance can reduce anxiety |
| e. | Legal assistance to people with complex needs improves their outcomes |
| f. | Model which combines case work, systemic advocacy and law reform effectively meets the needs of client groups |
| 3 | Impact – efficiency and organisational benefits | |
| a. | Model of legal services in host agency can be mutually beneficial |
| 4 | The client group – referral and service implications | |
| a. | Client group reticent to seek help |
| b. | Clients may come to service with a range of legal problems, which may already be a crisis point |
| c. | Some clients with limited capacity, require ongoing and active support |
| 5 | Planning and set up | |
| a. | Model needs to be responsive to local conditions/needs |
| b. | Planning needs to involve legal needs/mapping exercise to ensure new service fills gaps |
| 6 | Linking with clients – location | |
| a. | Different locations have different types of clients with different needs |
| b. | Familiar 'safe' venue increases accessibility of service |
| c. | Service needs to be located in place where the target group go or visit regularly |
| 7 | Linking with clients – marketing services | |
| a. | Need to market the existence of a service, what it can do for clients and how to contact it |
| 8 | Linking with clients – identify and support pathway people | |
| a. | Community members or representatives from the client group may help facilitate contact between clients and legal services |
| b. | Host agency staff often a key link to clients with complex needs |
| 9 | Collaboration and networking | |
| a. | Model depends upon having mechanisms in place for ongoing communication/collaboration with host agency |
| b. | Service relationship with others in sector, including referral relationships essential |
| c. | Services tried to involve clients in management of service, but found this difficult |
| d. | Trust and relationship with the community/clients essential |
| e. | When planning outreach services, needs to be early engagement with partner organisations |
| 10 | Service provision – flexibility and timeliness | |
| a. | Services need to be flexible enough to cater for the range of legal issues clients likely to have |
| b. | Service needs to take action/give advice immediately/ASAP |
| c. | Services need strategies to stay in contact with transient clients |
| d. | Warm referrals may be required |
| 11 | Service provision – lawyer and client relationship (communication) | |
| a. | Clients require and appreciate friendly approachable lawyers who take time to explain things in plain language |
| 12 | Service provision – trust | |
| a. | Access increased when service is informal and non-threatening |
| b. | Clients more likely to engage with a lawyer they are familiar with and trust |
| 13 | Service provision – confidentiality | |
| a. | Confidentiality is an important way to make services accessible for 'hard-to-reach' groups, particularly with credit and debt issues - 6 findings |
| 14 | Service provision – consistency of service provision | |
| a. | Consistency/continuity in service provision valued |
| b. | Free service enhances accessibility |
| c. | High quality legal assistance builds trust and reputation |
| 15 | Training and skills – lawyers | |
| a. | Lawyers may require training about their client group and about the host agency |
| b. | Lawyers may require training in additional areas of law to cover the range of issues clients are likely to face |
| c. | Safety of legal staff to be addressed |
| d. | Training for lawyers should be ongoing |
| e. | Work at a legal outreach clinic can enhance lawyers skills |
| 16 | Training and skills – pathway workers | |
| a. | Host agency staff require training about identifying possible legal issues and referral to legal services |
| 17 | Resourcing staffing and supervision | |
| a. | High quality supervision and support for legal staff is resource intensive but important |
| b. | Model requires a coordinator position to run clinic/service, liaise with host agency and arrange training etc |
| c. | Not all issues will require a solicitor |
| 18 | Resourcing and infrastructure | |
| a. | Outreach services more resource intensive than static/office based services |
| b. | Outreach services need to be appropriately funded and supported |
| c. | Services need access to specialist legal support |
| d. | Technology seen to facilitate quicker and more effective service delivery |
| e. | Working with hard-to-reach clients is often more time consuming and costly than anticipated |
| 19 | Record keeping and review | |
| a. | If reporting/admin arrangements too onerous, they will not be completed by the lawyers |
| b. | Need to record all of clients' legal issues, outcomes of any assistance given and other relevant client information |
| c. | Quality control mechanisms need to be put in place |
| d. | Services need strategies to keep available records of clients who come irregularly to the clinic |
COMMENTS ON THE METHODOLOGY
One purpose of this systematic review was to trial the applicability of the JBI methodology as a way of reviewing the effectiveness of legal service delivery strategies. While the methodology was developed by JBI to review health related initiatives, we saw that some of the questions asked of service delivery in health were similar to the questions that we explore about legal service delivery. This is particularly the case when considering service delivery to disadvantaged or 'hard-to-reach' people. As far as we are aware, this is the first time the JBI methodology and software has been applied to socio-legal research.
However, it should be noted that the systematic review and synthesis of methodologies other than quantitative experimental designs is an evolving methodology. Just as there was disagreement and debate around the development of meta-analyses of quantitative studies, there is debate among qualitative researchers about whether qualitative data from across studies can be synthesised (e.g. integrated or aggregated) and the best methods of doing so (JBI, 2008, p. 29). We decided to trial this method as it provided us with a more robust, transparent and replicable method of reviewing and synthesising qualitative and mixed method data than a simple literature review.
We found that the prescribed approach had the benefit of ensuring the review remained focused, both in terms of the content and quality of the studies included. It was a discipline to remain true to our research protocol and more time consuming to be very thorough in the literature search. However, it was at the appraisal and synthesis stage that we were most challenged. For instance, we found that, had we insisted that each study meet all ten of the quality criteria provided to be included (that is, score a 'yes' on all ten appraisal criteria), we would have no material to review.
It should be noted, however, that the problem was not always that the standard of the studies were poor, but that many of the reports we reviewed were not written in the academic style for qualitative research that the criteria reflect. Consequently, the reports we reviewed did not always provide enough information to enable us to assess the standard. For instance, one criteria JBI use to assess qualitative research is the influence of the researcher on the research, and vice-versa, is addressed. Commonly, the research reports we reviewed did not explicitly address this, so they scored a 'no' on this criteria. There is scope within the method to define the 'standards', as the two reviewers could select the number and selection of appraisal criteria that each study had to score a 'yes' for, before they were included in the review. We insisted that studies scored a yes for what we deemed to be the 'key' criteria — that is criteria 2–5 and 10 (see Table 2). A number of the studies we included scored yes against more of these criteria. This reflected the fact that, while all of the studies we included met a certain standard, there was still variation in the scale and standard of studies which we did include.
A further issue we identified was that most of the studies we examined employed mixed methods — qualitative interviews, questionnaires, the use of administrative data and statistics. While JBI has developed separate appraisal systems for experimental quantitative data,10 qualitative data and for economic data, there are no specific criteria or review methods for 'mixed method' research of the type we identified. We took the approach of thematically analysing the mixed method data using the qualitative tool, but see scope to refine this to better address mixed methodologies, particularly in the field of socio-legal research.
We took this approach as all the reports we included have a qualitative component (with the exception of Smith & Patel, 2008). In addition, we believe that many of the qualitative appraisal criteria provide a good 'checklist' of quality research, irrespective of methodology. These 'key' criteria include:
- There is congruity between the research methodology and the research question or objectives.
- There is congruity between the research methodology and the methods used to collect data.
- There is congruity between the research methodology and the representation and analysis of data.
- There is congruity between the research methodology and the interpretation of results.
- Conclusions drawn in the research report do appear to flow from the analysis or interpretation, of the data.
The Foundation aims to build on this to develop appraisal criteria which better reflect the methodological features relevant to our field of research.
During the data analysis/synthesis stage we found a further difficulty in some reports in identifying 'findings' and 'evidence'. In some cases authors would report 'evidence' but then appear to draw no conclusions from that evidence. In other cases authors drew findings which, while logical and consistent with the results generally, did not have specific and clear evidence in the report to support those findings. In instances where there was evidence but no findings, or conversely findings but no evidence presented, we excluded those specific results from our analysis.
As we refine a review methodology for socio-legal research we will also reconsider what we understand by and accept as 'evidence' and as 'findings' in the qualitative and other mixed method research that we review.
Overall, the task of reviewing the literature in such a systematic way provided useful insight into how to best draft research reports in a more transparent and rigorous manner. The appraisal criteria provide a guide to what should be found in high quality research reports, across disciplines. More importantly, we found that this methodology provided a useful and rigorous tool for identifying and synthesising the best available research evidence on outreach legal services to people with complex needs.
Appendix 2
Databases and websites searched during stage 1 of the literature review.
Databases
LJF databases
- JAPA (Justice Access Program Alert)
Law databases
- Austlii
- Bepress Legal Repository
- Criminology via CSA Database Platform
- Australian Criminology Database (CINCH)
- APAIS
- Legal E journals
Academic search engines
- Google scholar
- Australian Policy Online
Cultural studies databases
- MAIS: Multicultural Australia and Immigration Studies via Informit Online
- Australian Public Affairs full-text
- CINCH - ATSIS
Social sciences databases
- Family & Society Plus 1980
- Women’s Resources International
- Social Science Research Network
- AGIS + text
- Humanities and social Science Collection
- APA - FT
- Informit
Health databases
- Cinch-Health
- Health & Society Database
- APAIS Health
- Rural and remote health databases
Education databases
- A + Education
Websites
Legal websites
- State Government websites (Parliaments, Departments of Justice etc)
- Commonwealth Government/Parliament websites (AGs, Parliament, Social Inclusion Unit)
- Indigenous Justice Clearinghouse http://www.indigenousjustice.gov.au/
- Scottish Executive http://www.scotland.gov.uk/Topics/
- UK Home Office http://www.homeoffice.gov.uk/
- UK Legal Services Research Centre http://www.lsrc.org.uk/index2.htm
- UK Legal Services Commission website http://www.legalservices.gov.uk/
- Campbell Collaboration http://www.campbellcollaboration.org/
- Centre for Crime and Justice Studies http://www.crimeandjustice.org.uk
- Canadian Access to Justice Network http://www.acjnet.org/nahome/default.aspx
- Department of Justice, Canada http://canada.justice.gc.ca/eng/index.html
Non legal websites
- AIHW; http://www.aihw.gov.au
- Australian Resource Centre for Health Care Innovations http://www.archi.net.au
- Australian Domestic Violence Clearing House http://www.austdvclearinghouse.unsw.edu.au/
- Australian Institute of Family Studies http://www.aifs.gov.au/
- Australian Centre for Youth Studies http://acys.info/about
- Cochrane and Joanna Briggs Institutes http://www.cohrane.org.au http://www.joannanbriggs.edu.au
- NZ Ministry for Social Development http://www.msd.govt.nz/about-msd-and-our-work/publications-resources/evaluation/index.html
Appendix 3
Stage 1:
Ninety-eight studies were identified, from an initial list of hundreds of returns as potentially relevant in the initial search. These were recorded onto an Endnote database.
Stages 2 and 3:
Thirty-nine studies which appeared to be evaluation reports on relevant outreach, were retrieved in abstract or full text, and considered in more detail against inclusion criteria. These studies were:
Abbot, S & Hobby, L 2000, 'Welfare benefits advice in primary care: evidence of improvements in health', Public Health, no. 114, pp. 324-327.
Blake Stevenson Ltd & OPM, 2003, Legal information and advice provision in Scotland: a review of evidence, Scottish Executive Social Research, Edinburgh, Available at: <http://www.scotland.gov.uk/Resource/Doc/47049/0029631.pdf>.
Buck, A, Tam, T & Fisher, C 2007, Putting money advice where the need is: evaluating the potential for advice provision in different outreach locations, LSRC Research Paper no. 16, Legal Services Research Centre, London, Available at: <http://www.lsrc.org.uk/publications/Outreach.pdf>.
Bulloch, S 2004, An evaluation of the Community Legal Service Pilot Partnerships, Research Findings no.49/2004, Scottish Executive Social Research, Edinburgh, Available at: <http://www.scotland.gov.uk/Resource/Doc/47060/0029583.pdf>.
Curran, L 2007, Ensuring justice and enhancing human rights: a report on improving legal aid service delivery to reach vulnerable and disadvantaged people, La Trobe Law, Bundoora, Vic, Available at: <https://senate.aph.gov.au/submissions/comittees/viewdocument.aspx?id=8411d778-7be5-413d-9ffb-28fdbbbb38dd>.
Day, L, Collard, S & Hay, C 2008, Money advice outreach evaluation: qualitative outcomes for clients, Legal Services Research Centre, London, Available at: <http://www.lsrc.org.uk/publications/DebtOutreachOutcomesClients.pdf>.
Day, L, Collard, S & Davies, V 2008, Money advice outreach evaluation: the provider and partner perspectives, Legal Services Research Centre, London, Available at: <http://www.lsrc.org.uk/publications/DebtOutreachProviderPartnerPerspectives.pdf>.
Dimos, D 2008, Civil law ALS Outreach review, Legal Aid NSW, Sydney, Available at: <http://www.legalaid.nsw.gov.au/data/portal/00000005/public/24799001229058541336.pdf>.
Domestic Violence & Incest Resource Centre 2004, Developing an integrated response to family violence in Victoria: issues and directions, Melbourne, Available at: <http://www.dvirc.org.au/UpdateHub/Integrated%20Respose%20Issues%20PaperNoAppendices.pdf>.
Domestic Violence & Incest Resource Centre 2005, 101 ways great and small to prevent family violence, Melbourne, Available at: <http://www.dvirc.org.au/PublicationsHub/101WaysPrevent.htm>.
European Monitoring Centre for Drugs and Drugs Addiction (EMCDDA) 2001, Guidelines for the evaluation of outreach work: a manual for outreach practitioners, Luxembourg, Available at: <http://www.emcdda.europa.eu/index.cfm?fuseaction=public.Content&nNodeID=1576&sLanguageISO=EN>.
Funston, R & Hitter, M 2006, Prisoners Legal Service review, Legal Aid NSW, Sydney, Available at: <http://www.legalaid.nsw.gov.au/data/portal/00000005/public/48618001159334571125.pdf>.
Gillespie, M, et al. 2007, Money advice for vulnerable groups: final evaluation report, Scottish Executive, Glasgow, Available at: <http://www.scotland.gov.uk/Resource/Doc/171671/0047978.pdf>.
Goldie, C 2003, PILCH Homeless Persons' Legal Clinic: evaluation report, Gilbert + Tobin Centre of Public law, Sydney.
Grace, M & Previte, J 2000, Evaluation of the Women's Justice Network, Margaret Grace and Associates, Bardon, Qld.
Hall, L, et al. 2001, 'Is untargeted outreach visiting in primary care effective? A pragmatic randomised controlled trial', Journal of Public Health Medicine, vol. 23, no. 2, pp. 109-113, Available at: <http://jpubhealth.oxfordjournals.org/cgi/reprint/23/2/109>.
Hartlepool New Deals for Communities 2004, Evaluation report on the Money Advice and Debt Counselling Service Project, Hartlepool, UK, Available at: <http://www.ndc.org.uk/documents/MoneyAdviceandDebtCounsellingService.doc>.
Kerry, N & Pennell, S 2001, San Diego Homeless Court Program: a process and impact evaluation, San Diego Association of Governments, San Diego.
Keyatta, D, Skafte-Zauss, C & Rosenfeld, M 2007, Hunter Homeless Law and Advocacy Service Pilot Project: final report, Mission Australia and Sparke Helmore.
Kilner, D 2007, Housing Legal Clinic final evaluation report, Welfare Rights Centre SA, unpublished, Adelaide.
Legal Aid NSW 2008, 'Spotlight on outreach services', Verbals, no. 51, pp. 7–19, Available at: <http://www.legalaid.nsw.gov.au/data/portal/00000005/public/34824001223505244172.pdf>.
Mackintosh, J, et al. 2006, 'Randomised controlled trial of welfare rights advice accessed via primary health care: pilot study [ISRCTN61522618]', BMC Public Health, vol. 6, no. 162, pp. 10, Available at: <http://www.biomedcentral.com/1471-2458/6/162>.
Macmillen, L & Perkins, H 2005, 'Mental health projects', in Innovation in the Community Legal Service: a review of 22 projects supported through the Partnership Initiative budget, Legal Services Commission, London.
Monro, R 2002, 'Elder abuse and legal remedies: practical realities?' Reform, no. 81, pp. 42–46.
National Pro Bono Resource Centre 2006, Regional, rural and remote pro bono: models and opportunities, Sydney, Available at: <http://www.nationalprobono.org.au/publications/documents/RRRPaper.pdf>.
Opinion Leader Research 2007, Evaluation of the Financial Inclusion Fund Face-to-Face Debt Advice Project: final report for stage one of the evaluation, Available at: <http://www.berr.gov.uk/files/file41405.pdf>.
Perkins, H & Macmillen, L 2005, 'Primary health care projects', in Innovation in the Community Legal Service: a review of 22 projects supported through the Partnership Initiative budget, Legal Services Commission, London.
Pleasence, P, et al. 2007, A helping hand: the impact of debt advice on people's lives, Research Findings 15, Legal Services Research Centre, London, Available at: <http://www.lsrc.org.uk/publications/Impact.pdf>.
Previte, J & Pini, B 2002, Women's Justice Network evaluation 2002, Legal Aid Queensland, Brisbane.
Queensland PILCH 2003, The Homeless Persons' Legal Clinic: end of pilot report, Brisbane.
Rennie, FW, Greller, W & Mackay, M 2002, Review of international best practice in service delivery to remote and rural areas, Scottish Executive Social Research, Edinburgh, Available at: <http://www.scotland.gov.uk/Resource/Doc/46997/0025635.pdf>.
Riddell, S, et al. 2006, The future delivery of advice and information services for additional support needs in Scotland: Report 2, Scottish Executive Education Department, Edinburgh, Available at: <http://www.scotland.gov.uk/Resource/Doc/140971/0034793.pdf>.
Ryan, P & Merlo, R 2005, Family Homeless Prevention Pilot: final evaluation report, RPR Consulting, Canberra, <http://www.fahcsia.gov.au/sa/housing/pubs/homelessfamilies/fhpp/Pages/default.aspx>.
Sherr, L, et al. 2002, A stitch in time: accessing and funding welfare rights through Health Service Primary Care. An evaluation of primary care based specialist welfare rights advice provision in Lambeth, Southwark and Lewisham, London, Available at: <http://www.lho.org.uk/viewResource.aspx?id=8763>.
Smith, M & Patel, A 2008, Money advice outreach evaluation: cost and effectiveness of the outreach pilots, LSRC Research Paper no. 22, Legal Services Commission, London, Available at: <http://www.lsrc.org.uk/publications/DebtOutreachCostEffectiveness.pdf>.
Steele, J & Seargeant, J 1999, Access to legal services: the contribution of alternative approaches, Policy Studies Institute, London, Available at: <http://www.psi.org.uk/publications/publication.asp?publication_id=3>.
Turley, C & White, C 2007, Assessing the impact of advice for people with debt problems, Legal Services Research Centre, London, Available at: <http://www.lsrc.org.uk/publications/bmrb.pdf>.
Westwood Spice 2005, Evaluation report: Homeless Persons' Legal Service, PIAC, Unpublished, Sydney.
Williams, T 2004, Review of research into the impact of debt advice, Legal Services Research Centre, London, Available at: <http://www.lsrc.org.uk/publications/impactlitrev.pdf>.
Reports appraised but excluded
The following reports were appraised on the basis of the rigour of their methodology (see Table A.1 above), but excluded from the review as they did not meet the criteria (due to the methodology, as reported) for inclusion as 'qualitative research'. Most of the following either did not report their methodology in sufficient detail to be assessed and used in the analysis, or were simply descriptions of projects.
Legal Aid NSW 2008, 'Spotlight on outreach services', Verbals, no. 51, pp. 7-19, Available at: <http://www.legalaid.nsw.gov.au/data/portal/00000005/public/34824001223505244172.pdf>.
Queensland PILCH 2003, The Homeless Persons' Legal Clinic: end of pilot report, Brisbane.
Steele, J & Sergeant, J 1999, Access to legal services: the contribution of alternative approaches, Policy Studies Institute, London, Available at: <http://www.psi.org.uk/publications/publication.asp?publication_id=3>.
Macmillen, L & Perkins, H 2005, 'Mental health projects', in Innovation in the Community Legal Service: a review of 22 projects supported through the Partnership Initiative budget, Legal Services Commission, London.
Perkins, H & Macmillen, L 2005, 'Primary health care projects', in Innovation in the Community Legal Service: a review of 22 projects supported through the Partnership Initiative budget, Legal Services Commission, London.
Introduction
Objectives of this review
Methodology
Results
Conclusion
Knowledge gaps
Reflections on the research
Reference list
Appendix 1
Appendix 2
Appendix 3