ContentJust Search pageLJF site navigationLeft navigation links
LJF Logo
Publications sectionJustice Awards sectionResearch sectionGrants sectionPlain language law sectionNetworks section
Just Search
 
Research Report: Effectiveness of public legal assistance services: a discussion paper, Justice issues paper 16
cover image

Effectiveness of public legal assistance services: a discussion paper, Justice issues paper 16    Cite this report

, 2012 , 12 p.
This paper contends that, in the context of evaluating legal assistance services, effectiveness refers to a causal link between an activity and an outcome. Accordingly, it is only through research designs which allow for precise measurement and isolation of other significant influences on outcomes that effectiveness can be accurately assessed. For this reason, research which involves appropriate sampling and an adequate comparison group will best assess the effectiveness of strategies. Other research methods may best address issues such as those relating to why an activity has succeeded or how well it has been received by clients.


Print chapter
View PDF

Introduction



Download paper: Justice Issues 16: Effectiveness of public legal assistance services: a discussion paper [PDF, 554kb]

Legal assistance services and strategies are designed to help people to resolve their legal problems. Those funding and providing services intend that these efforts actually make that difference, and do so efficiently given their limited resources. Given this aim, what does it mean to be ‘effective’ and how can this be demonstrated? As this paper will explain, measuring effectiveness is about measuring whether or not the program achieved its aim or intended outcome.

Recent interest in effectiveness is evident in many policy and strategic planning documents which include recommendations that legal assistance agencies’ activities should be evaluated in terms of the outcomes that they achieve and their associated costs. The National Partnership Agreement on Legal Assistance Services (COAG 2010) is a recent example.

There are significant benefits to be gained by evaluating activities in terms of whether they achieve intended outcomes and these are outlined in this paper. However, demonstrating that a strategy actually caused a desired outcome—that is, that the strategy rather than any other factor has actually made the difference—can be an ambitious task. Evaluating effectiveness in causal terms requires careful planning, dedicated resources, and particular expertise and methodology. This is especially the case in sectors such as the legal assistance sector, which have not, as yet, widely used this approach.

Evaluation research can also be used to examine issues other than effectiveness and cost. For example, it can inform questions of process, such as why an activity succeeded or failed, and how was it perceived and received by key stakeholders. While these questions are also important, the focus of this paper is on the evaluation of the outcomes of activities, that is, their effectiveness.

In terms of scope, this paper provides an introduction to evaluating effectiveness rather than a ‘how to’ guide. A list of resources is included at the end of this paper for readers interested in comprehensive guides to conducting outcome-based evaluation.

A list of resources is included at the end of this paper for readers interested in comprehensive guides to conducting outcome-based evaluation.

The aim of this discussion paper is to encourage and support the evaluation of effectiveness in the legal assistance sector by:
  • explaining the meaning of key terms such as effectiveness, cost-effectiveness, efficiency and evidence-based practice
  • demonstrating how evaluating activities in terms of their outcomes can challenge commonly held assumptions and provide valuable evidence to inform practice and further strategy development
  • indicating how outcomes can be identified for measurement
  • introducing relevant evaluation methods, particularly in terms of their ability to accurately measure the effectiveness (and potentially cost-effectiveness and cost-benefit) of services.
Acknowledgements

The writing of this paper was greatly assisted by input from Maria Karras and Suzie Forell, document analysis and comments on earlier versions by Megan Laufer, and by comments and/or searching for relevant source documents by Abigail Gray, Maureen Ward and Anna Russell. Thanks also to Terry Beed, Jenny Lovric and Geoff Mulherin for comments on an earlier draft of the paper.


  


CLOSE
Digiusto, E 2012, Effectiveness of public legal assistance services, Justice issues paper 16, Law and Justice Foundation of NSW, Sydney.