ContentJust Search pageLJF site navigationLeft navigation links
LJF Logo
Publications sectionJustice Awards sectionResearch sectionGrants sectionPlain language law sectionNetworks section
Just Search
 
Research Report: Outreach legal services and complex needs: what works? Justice issues paper 12
cover image

Outreach legal services and complex needs: what works? Justice issues paper 12  ( 2009 )  Cite this report



Print chapter
Search or view whole report
View PDF

Appendix 1: Additional methodological detail - search strategy and synthesised findings


SEARCH STRATEGY

A three stage search strategy was used.

First stage

The first stage of the search strategy consisted of a broad but shallow review of academic databases (legal, socio-legal, cultural studies, health, education and social science) and the websites of government bodies and research institutions using pre-defined search terms. The databases, websites and bibliographies searched are detailed in Appendix 2.

This first stage focused predominantly on locating articles or reports on legal outreach to people with complex needs. However this first stage did not, at least at first, exclude relevant looking articles on evaluations of outreach to people with complex needs in other related fields (e.g. health, education, etc).

This stage was carried out by two researchers and two librarians. At this stage only bibliographic information on each article, including a web-link and abstract, where available, were collected. Of the hundreds of reports originally identified, 98 most relevant to outreach legal services were entered into a referencing library (EndNote Version 6.0).

Search terms

The search terms used to search both databases and websites were:

Strategy type:Outreach, co-location, hub, one-stop shop, interagency, early intervention, legal clinic, advice clinic, Video-link advice (not court procedures)
Study type:Evaluation, review, what works
Area of law:Civil law, credit and debt, domestic violence/family violence/DV, family law, criminal law, financial planning, tenancy/housing, poverty law, multiple/clustered legal need/problems
Participants:Complex needs, Indigenous, Aboriginal, mental illness, (intellectual) disability, homeless, caravan parks, rural, regional and remote, lone/single parent (families), disadvantaged, socially/ economically/ financially excluded, isolated/hard-to-reach, prisoners

Use / hierarchy of search terms

We searched first on 'strategy type' and 'participants'. 'Area of law' and 'study type' were used to limit the search. The other search terms were kept in mind when we selected manually from a list of returns.

Second stage

The second stage involved a review of returns from stage 1 to identify relevant articles and reports that were specifically on evaluations of legal outreach to people with complex needs, based on the information provided in the title and abstract and full text if available. Any reports identified as relevant at this stage were retrieved in full text or abstract form, and again assessed against the inclusion criteria for relevance to the review question.

Third stage

The third stage involved a search of the references and footnotes of the most relevant reports. This involved returning to websites for more targeted searches, as well as contacting services to get copies of evaluation reports which were not publicly available. Reports on evaluations of legal outreach to people with complex needs found at this stage were retrieved and assessed against the inclusion criteria for relevance to the review question. This process is summarised in Table 1 in the main body of the report, while the studies identified at stage 2 and 3 of the search are identified in Appendix 3.

Thirty-nine reports were found to be of relevance during stages 2 and 3 and assessed against the inclusion criteria for relevance to the review question. Of these 16 were found to be evaluation reports on outreach advices services to people with complex needs, and were therefore selected for appraisal.

SYNTHESISED FINDINGS AND CATEGORIES

As described in the methodology section of the report, data synthesis involves identifying findings, grouping findings into thematically similar categories and then grouping these categories into broader findings (synthesised findings) which can then be used to derive best practice principles.

A total of 226 findings were extracted from the study, and then grouped into 53 like categories. From these 53 categories a 'thematic' meta-synthesis based on the amalgamation of related categories was used to produce 19 synthesised findings. These are listed below.

Table A1: Synthesised findings and categories
1Impact – reach
a.Outreach does reach clients with complex needs
2Impact on clients – effectiveness
a.Consultation with lawyers can improve clients' self esteem and outlook, even when an issue can't be resolved
b.Legal assistance can act as a circuit breaker and enable/motivate clients to move on with their lives
c.Legal assistance can increase the willingness of clients to seek help again and to manage issues better in future
d.Legal assistance can reduce anxiety
e.Legal assistance to people with complex needs improves their outcomes
f.Model which combines case work, systemic advocacy and law reform effectively meets the needs of client groups
3Impact – efficiency and organisational benefits
a.Model of legal services in host agency can be mutually beneficial
4The client group – referral and service implications
a.Client group reticent to seek help
b.Clients may come to service with a range of legal problems, which may already be a crisis point
c.Some clients with limited capacity, require ongoing and active support
5Planning and set up
a.Model needs to be responsive to local conditions/needs
b.Planning needs to involve legal needs/mapping exercise to ensure new service fills gaps
6Linking with clients – location
a.Different locations have different types of clients with different needs
b.Familiar 'safe' venue increases accessibility of service
c.Service needs to be located in place where the target group go or visit regularly
7Linking with clients – marketing services
a.Need to market the existence of a service, what it can do for clients and how to contact it
8Linking with clients – identify and support pathway people
a.Community members or representatives from the client group may help facilitate contact between clients and legal services
b.Host agency staff often a key link to clients with complex needs
9Collaboration and networking
a.Model depends upon having mechanisms in place for ongoing communication/collaboration with host agency
b.Service relationship with others in sector, including referral relationships essential
c.Services tried to involve clients in management of service, but found this difficult
d.Trust and relationship with the community/clients essential
e.When planning outreach services, needs to be early engagement with partner organisations
10Service provision – flexibility and timeliness
a.Services need to be flexible enough to cater for the range of legal issues clients likely to have
b.Service needs to take action/give advice immediately/ASAP
c.Services need strategies to stay in contact with transient clients
d.Warm referrals may be required
11Service provision – lawyer and client relationship (communication)
a.Clients require and appreciate friendly approachable lawyers who take time to explain things in plain language
12Service provision – trust
a.Access increased when service is informal and non-threatening
b.Clients more likely to engage with a lawyer they are familiar with and trust
13Service provision – confidentiality
a.Confidentiality is an important way to make services accessible for 'hard-to-reach' groups, particularly with credit and debt issues - 6 findings
14Service provision – consistency of service provision
a.Consistency/continuity in service provision valued
b.Free service enhances accessibility
c.High quality legal assistance builds trust and reputation
15Training and skills – lawyers
a.Lawyers may require training about their client group and about the host agency
b.Lawyers may require training in additional areas of law to cover the range of issues clients are likely to face
c.Safety of legal staff to be addressed
d.Training for lawyers should be ongoing
e.Work at a legal outreach clinic can enhance lawyers skills
16Training and skills – pathway workers
a.Host agency staff require training about identifying possible legal issues and referral to legal services
17Resourcing staffing and supervision
a.High quality supervision and support for legal staff is resource intensive but important
b.Model requires a coordinator position to run clinic/service, liaise with host agency and arrange training etc
c.Not all issues will require a solicitor
18Resourcing and infrastructure
a.Outreach services more resource intensive than static/office based services
b.Outreach services need to be appropriately funded and supported
c.Services need access to specialist legal support
d.Technology seen to facilitate quicker and more effective service delivery
e.Working with hard-to-reach clients is often more time consuming and costly than anticipated
19Record keeping and review
a.If reporting/admin arrangements too onerous, they will not be completed by the lawyers
b.Need to record all of clients' legal issues, outcomes of any assistance given and other relevant client information
c.Quality control mechanisms need to be put in place
d.Services need strategies to keep available records of clients who come irregularly to the clinic


COMMENTS ON THE METHODOLOGY

One purpose of this systematic review was to trial the applicability of the JBI methodology as a way of reviewing the effectiveness of legal service delivery strategies. While the methodology was developed by JBI to review health related initiatives, we saw that some of the questions asked of service delivery in health were similar to the questions that we explore about legal service delivery. This is particularly the case when considering service delivery to disadvantaged or 'hard-to-reach' people. As far as we are aware, this is the first time the JBI methodology and software has been applied to socio-legal research.

However, it should be noted that the systematic review and synthesis of methodologies other than quantitative experimental designs is an evolving methodology. Just as there was disagreement and debate around the development of meta-analyses of quantitative studies, there is debate among qualitative researchers about whether qualitative data from across studies can be synthesised (e.g. integrated or aggregated) and the best methods of doing so (JBI, 2008, p. 29). We decided to trial this method as it provided us with a more robust, transparent and replicable method of reviewing and synthesising qualitative and mixed method data than a simple literature review.

We found that the prescribed approach had the benefit of ensuring the review remained focused, both in terms of the content and quality of the studies included. It was a discipline to remain true to our research protocol and more time consuming to be very thorough in the literature search. However, it was at the appraisal and synthesis stage that we were most challenged. For instance, we found that, had we insisted that each study meet all ten of the quality criteria provided to be included (that is, score a 'yes' on all ten appraisal criteria), we would have no material to review.

It should be noted, however, that the problem was not always that the standard of the studies were poor, but that many of the reports we reviewed were not written in the academic style for qualitative research that the criteria reflect. Consequently, the reports we reviewed did not always provide enough information to enable us to assess the standard. For instance, one criteria JBI use to assess qualitative research is the influence of the researcher on the research, and vice-versa, is addressed. Commonly, the research reports we reviewed did not explicitly address this, so they scored a 'no' on this criteria. There is scope within the method to define the 'standards', as the two reviewers could select the number and selection of appraisal criteria that each study had to score a 'yes' for, before they were included in the review. We insisted that studies scored a yes for what we deemed to be the 'key' criteria — that is criteria 2–5 and 10 (see Table 2). A number of the studies we included scored yes against more of these criteria. This reflected the fact that, while all of the studies we included met a certain standard, there was still variation in the scale and standard of studies which we did include.

A further issue we identified was that most of the studies we examined employed mixed methods — qualitative interviews, questionnaires, the use of administrative data and statistics. While JBI has developed separate appraisal systems for experimental quantitative data,10 qualitative data and for economic data, there are no specific criteria or review methods for 'mixed method' research of the type we identified. We took the approach of thematically analysing the mixed method data using the qualitative tool, but see scope to refine this to better address mixed methodologies, particularly in the field of socio-legal research.

We took this approach as all the reports we included have a qualitative component (with the exception of Smith & Patel, 2008). In addition, we believe that many of the qualitative appraisal criteria provide a good 'checklist' of quality research, irrespective of methodology. These 'key' criteria include:

  • There is congruity between the research methodology and the research question or objectives.
  • There is congruity between the research methodology and the methods used to collect data.
  • There is congruity between the research methodology and the representation and analysis of data.
  • There is congruity between the research methodology and the interpretation of results.
  • Conclusions drawn in the research report do appear to flow from the analysis or interpretation, of the data.

The Foundation aims to build on this to develop appraisal criteria which better reflect the methodological features relevant to our field of research.

During the data analysis/synthesis stage we found a further difficulty in some reports in identifying 'findings' and 'evidence'. In some cases authors would report 'evidence' but then appear to draw no conclusions from that evidence. In other cases authors drew findings which, while logical and consistent with the results generally, did not have specific and clear evidence in the report to support those findings. In instances where there was evidence but no findings, or conversely findings but no evidence presented, we excluded those specific results from our analysis.

As we refine a review methodology for socio-legal research we will also reconsider what we understand by and accept as 'evidence' and as 'findings' in the qualitative and other mixed method research that we review.

Overall, the task of reviewing the literature in such a systematic way provided useful insight into how to best draft research reports in a more transparent and rigorous manner. The appraisal criteria provide a guide to what should be found in high quality research reports, across disciplines. More importantly, we found that this methodology provided a useful and rigorous tool for identifying and synthesising the best available research evidence on outreach legal services to people with complex needs.



Specifically, the JBI system has critical appraisal criteria for randomised controlled trials, comparable cohort/case control studies and descriptive/case series studies.

10  Specifically, the JBI system has critical appraisal criteria for randomised controlled trials, comparable cohort/case control studies and descriptive/case series studies.


CLOSE
Forell, S & Gray, A 2009, Outreach legal services to people with complex needs: what works? Justice issues paper 12, Law and Justice Foundation of NSW, Sydney