ContentJust Search pageLJF site navigationLeft navigation links
LJF Logo
Publications sectionJustice Awards sectionResearch sectionGrants sectionPlain language law sectionNetworks section
Just Search
 
Research Report: Legal Australia-Wide Survey: Legal need in Australia
cover image

Legal Australia-Wide Survey: Legal need in Australia  ( 2012 )  Cite this report

2. The present study



Print chapter

Method


Survey design

A copy of the survey instrument and a glossary of the terms used are presented in Appendix A1. The LAW Survey instrument was based on a revised version of the NSWLNS (Coumarelos et al. 2006).

Justiciable problem approach

Consistent with Genn’s (1999) justiciable problem approach, identification of the legal problems experienced by respondents did not require legal knowledge. The problems were presented in some detail, so that respondents could say if they had experienced these problems without being required to assess whether the problems had legal aspects or remedies. For example, rather than being asked whether they had experienced a family law problem, respondents were asked more specific questions, such as whether they had experienced any problems with residence or contact arrangements for their children. Thus, the present study minimised underreporting of legal problems by capturing problems that the respondent might not have realised were legal in nature.

For simplicity, the term ‘legal problems’ rather than ‘justiciable problems’ was adopted in the present report. The survey included as legal problems:
    • events that are generally considered to be legal problems or disputes (e.g. child support and custody issues, criminal charges)
    • problems or disputes that potentially have legal implications or remedies but may not always be recognised as such (e.g. inadequate medical treatment, disputes with neighbours).

Reference period

Surveys adopting Genn’s (1999) approach have often used reference periods of 3–5 years. The present survey, like the NSWLNS (Coumarelos et al. 2006), adopted a shorter reference period of one year to maximise accurate recall of legal problems. The LAW Survey examined legal problems that began in or continued into the 12-month period prior to interview. Thus, it measured the prevalence of legal problems rather than the incidence of new legal problems, allowing for the exploration of potentially serious, complex problems that were still ongoing after many years.

Triviality threshold

To provide a comprehensive picture of legal need, the survey did not adopt a triviality threshold based on problem severity. Departing from Genn’s (1999) triviality threshold, problems of all levels of severity were captured, regardless of whether they were ‘difficult to solve’ or ‘warranted action’. When respondents reported experiencing a specific type of legal problem, they were asked to rate the problem’s severity, and these severity ratings were explored in various analyses.

Unlike the NSWLNS (Coumarelos et al. 2006), the LAW Survey captured only legal events that were considered to be ‘problems or disputes’ and excluded events that might have legal implications but were not ‘problematic’ (e.g. buying or selling a house without complications or making a will without any problems).(2) The inclusion of non-problematic legal events may overestimate legal need. Thus, the LAW Survey is likely to have provided a more accurate measure of legal need.

Classification of legal problems

To obtain a comprehensive picture of legal need, the present study examined a broad range of legal problems covered by civil, criminal and family law. Overall, respondents were asked about 129 specific types of legal problems (see Appendix Table A2.1 for full details). As summarised below, these 129 specific types of legal problems were categorised into 12 problem groups and 27 problem subgroups,(3) and included problems or disputes regarding:

1. accidents
2. consumer
    · goods
    · services
3. credit/debt
4. crime
    · crime offender
    · crime victim
5. employment
6. family
    · children
    · relationships
7. government
    · fines
    · government payments
    · local government
    · state/federal government
8. health
    · clinical negligence
    · health services
    · mental health
9. housing
    · neighbours
    · owned housing
    · rented housing
    · other housing
10. money
    · business/investment
    · wills/estates
11. personal injury
12. rights
    · discrimination (outside work)
    · education
    · unfair treatment by police
    · other civil.

Survey instrument

The LAW Survey instrument was divided into five sections.(4) The first section, entitled ‘Introduction and screening’ (see Appendix A1, questions S1–S9), covered:
    • the purpose of the survey
    • the voluntary and confidential nature of the survey
    • the demographic characteristics used as eligibility criteria for participation (i.e. gender, age, postcode of residence, Indigenous status and languages spoken).

The second section, ‘Demographics’ (see Appendix A1, questions D1–D18), captured demographic information on family structure, housing, business, disability, employment and government payments. This information was used to determine whether each respondent had the potential to experience certain legal problems, given their life circumstances during the reference period, so they could be filtered to the relevant legal problem questions. For example, only respondents who had been tenants were asked whether they had experienced legal problems with rented accommodation.

The third section, ‘Problems or disputes’ (see Appendix A1, questions P1–P40), captured each specific type of legal problem experienced by the respondent that had started during or continued into the previous 12 months.(5) For each specific type of legal problem, the following was also captured:
    • the frequency of that type of problem in the previous 12 months
    • the severity of that type of problem in terms of its impact on the respondent’s everyday life.(6)

The fourth section of the questionnaire, ‘Action and outcome’ (see Appendix A1, questions A1–A37), involved the in-depth follow-up of a selected pool of legal problems. This pool comprised up to three of the ‘most serious’ legal problems experienced by each respondent and was determined as follows. Respondents who had experienced legal problems were asked to order the different specific types of legal problems they had experienced in terms of seriousness, and to identify the worst instance of each specific type. The worst instance of each of the three most serious specific types of legal problems was then followed up in depth. Respondents who had experienced fewer than three specific types of legal problems were asked about only the worst instance of each type and, thus, were asked about only one or two legal problems.(7)

For each of the legal problems selected for in-depth examination, the ‘Action and outcome’ section captured information on:
    • the nature of the problem
    • the adverse consequences caused by the problem
    • the types of actions taken and the reasons for not taking action
    • the advisers used
    • the most useful adviser, such as adviser type, helpfulness, mode of communication and any barriers to obtaining advice
    • the finalisation of the problem, including manner of finalisation and outcome.

Respondents’ awareness of free legal services was also captured in this section (see Appendix A1, questions D24–D25).

The final section, ‘Demographics 2’ (see Appendix A1, questions D26–D31), captured further demographic information that was not examined in earlier sections because it was not required to determine eligibility or filtering. This section captured education, income and out-of-home care.

Differences between the LAW Survey and the NSWLNS

While the current survey built on the NSWLNS (Coumarelos et al. 2006), there were important differences between the survey instruments. First, the LAW Survey improved the capture of demographic information, by refining existing questions or including new questions on:
    • languages spoken and the main language spoken at home
    • family status
    • income
    • housing type, including public renting
    • severity of any disability experienced
    • out-of-home care
    • awareness of free legal services.

Second, the LAW Survey improved the coverage of legal problems, by revising and updating the classification used in the NSWLNS (cf. Appendix Table A2.1 in the present report to Appendix Table B1, p. 279, in Coumarelos et al. 2006). The LAW Survey also improved the measurement of legal problems, because, unlike the earlier survey, it:
    • avoided overestimating legal need, by using a more appropriate ‘triviality’ threshold that excluded non-problematic events (as noted above)
    • measured the frequency and severity of each specific type of legal problem
    • captured the ‘other side’ in the legal problem or dispute.

Third, the LAW Survey used a different pool of legal problems to follow up with in-depth questions. The NSWLNS followed up the most recent legal problems experienced by respondents. In contrast, in order to gain better understanding of the strategies used and the outcomes achieved for non-trivial legal problems, the LAW Survey followed up the most serious legal problems experienced.

Fourth, the LAW Survey introduced questions about the adverse economic, health and social consequences caused by legal problems.

Fifth, the current survey improved the measurement of actions and strategies used to handle legal problems. For example, by replacing single questions with a series of more targeted questions, the LAW Survey more systematically measured:
    • self-help actions, court and tribunal proceedings, and formal mediation, conciliation and dispute resolution(8)
    • respondents’ reasons for not taking action
    • the number and types of legal and non-legal advisers consulted
    • the types of legal and non-legal help received from advisers.

Sixth, the LAW Survey captured extra information on the strategies used, such as:
    • the order in which advisers were contacted
    • the mode of communication used with the most useful adviser.

Seventh, the LAW Survey improved the categorisation of the strategies used in response to legal problems, by having a distinct category for advice received from professionals or formal advisers. That is, the survey’s ‘sought advice’ category necessitated the use of a formal adviser. Problems that did not involve consultation with a professional but involved consultation with the other side, relatives or friends were categorised as ‘handling the problem without advice’. In contrast, the NSWLNS’s ‘sought help’ did not necessitate the use of formal advisers.(9)

Finally, the LAW Survey instrument improved the measurement of the finalisation and outcomes of legal problems, by:
    • introducing a question on whether the outcome was in the respondent’s favour
    • more systematically measuring the manner of finalisation.

These methodological differences between the two survey instruments need to be taken into account when comparing the findings. For example, it would be expected that the LAW Survey’s:
    • exclusion of non-problematic events would work towards fewer respondents reporting legal problems, and a greater proportion of severe legal problems
    • in-depth follow-up of the most serious rather than the most recent legal problems would focus on less trivial problems and work towards greater use of legal advisers, and lower rates of inaction, finalisation and favourable outcomes
    • improved measurement of formal advisers, other actions and different types of help would be likely to boost recall and, thus, would work towards a greater number of advisers, other actions and help types being reported.

Sampling

The LAW Survey also differed from the NSWLNS (Coumarelos et al. 2006) in terms of the type of sample used. The LAW Survey used a general population sample, while the NSWLNS sampled from six disadvantaged areas. This difference would be expected to work towards lower prevalence of legal problems for the LAW Survey.

The LAW Survey involved 20 716 telephone interviews with household residents aged 15 years or over across Australia. At least 2000 interviews were conducted in each state and territory, to enable separate reporting for each jurisdiction (see Table 2.1). The average length of interviews was 26 minutes. Households were randomly selected using random digit dialling (RDD), and one respondent per household was interviewed.(10)

Table 2.1: Survey sample [click to enlarge]



Quotas

Within each state and territory, quotas were set for age, gender, geographical area, Indigenous status and cultural and linguistic diversity (CALD) using the 2006 Census of Population and Housing (ABS 2007a). Age-within-gender quotas were set using the following age categories: 15–17, 18–24, 25–34, 35–44, 45–54, 55–64 and 65+ years. Geographical area quotas were set so that the number of interviews in each Statistical Local Area (SLA) reflected the SLA’s share of the population. The quotas for Indigenous status were set by adjusting the census data to take into account phone availability using the National Aboriginal and Torres Strait Islander Social Survey, 2002 (ABS 2004d). This adjustment was made because landline use can be very low among Indigenous people in some areas, particularly in remote areas. It was considered unrealistic to attempt to achieve Indigenous numbers in proportion to the population in these areas.

Most interviews were conducted in English. However, to ensure a reasonable representation of people with poor English, provision was made to conduct interviews in the six most commonly used non-English languages across Australia — namely, Italian, Greek, Cantonese, Mandarin, Arabic and Vietnamese. Quotas were set so that the non-English interviews were in proportion to people in the population who speak these languages and have poor English. In addition, quotas were set so that the English interviews with people who speak both English and a non-English language were also in proportion to population numbers.

In NSW, additional quotas were set to allow for the deliberate oversampling in the LAW Survey of the six local government areas (LGAs) that formed the basis of the NSWLNS (Coumarelos et al. 2006). This oversampling was to boost the numbers available for comparisons between the two surveys.(11) Additional quotas were also set in Victoria, where both people living in remote and outer regional areas and Indigenous people were oversampled to facilitate reporting for these people in this state.(12)

Fieldwork

The interviews were conducted between January and November 2008 by Roy Morgan Research (RMR), a social research firm, using Computer Assisted Telephone Interviewing (CATI). The questionnaire and interviewing procedures were finalised following piloting in urban and rural areas of NSW and Victoria in September 2007.(13)

Weighting

The minimum of 2000 interviews per state/territory required for the separate analysis of each jurisdiction meant that states/territories had to be disproportionately sampled relative to population numbers. For example, while only about one per cent of the Australian population lives in the Northern Territory, 10 per cent of the survey sample was taken from this jurisdiction. The weighting design corrected for this deliberate disproportionate sampling of states/territories when reporting results for Australia as a whole. In addition, it corrected for the oversampling of the six LGAs in NSW and the oversampling of people from remote and outer regional areas and Indigenous people in Victoria. The weighting also involved minor corrections for age, gender and Indigenous status to accommodate instances where quota targets were not met precisely.

Unless otherwise stated, the statistics presented throughout this report series (e.g. number/percentage of respondents, number/percentage of problems, p values) reflect the appropriate statistics after weighting had been applied.(14) Given the weighting to account for the disproportionate sampling of states/territories, respondent and problem numbers in the state/territory reports do not sum exactly to the totals in the national report. In addition, as a result of rounding weighted data, the numbers and percentages reported in the text, tables and figures for respondents and legal problems sometimes do not sum precisely to the total numbers or percentages.(15)

Response rate

The response rate for the LAW Survey across Australia as a whole was calculated to be 60.1 per cent. This calculation was based on the AAPOR classification and used the Council of American Survey Research Organizations’ (CASRO) adjustment (AAPOR 2009; Smith 2009).(16)

Comparison of sample and population profile

Minimising non-response is a quality control objective in any survey. However, the response rate alone is insufficient for determining whether the sample is ‘representative’ of the population — that is, whether the sample results are likely to accurately reflect the patterns in the broader population. An understanding of the differences between those who participated in the survey and those who refused to participate is also important (AAPOR 2009; Groves et al. 1992). Confidence in the representativeness of the sample is increased when the demographic profile of the sample is comparable to that of the population.

As noted earlier, quotas were used to help to achieve a sample that would accurately reflect the demographic profile in the population. Although these quotas were broadly met, minimal weighting was used to fully align the gender, age and Indigenous proportions in the sample with population proportions. However, weighting was not applied to any of the other demographic variables. That is, weighting was not applied to disability status, education, employment status, family status, housing type, main income, main language or remoteness of residential area. Thus, in each jurisdiction, the sample profile was compared to the population profile on these demographic variables to further gauge the representativeness of the sample. These comparisons revealed that the sample was largely consistent with census data and population estimates from relevant benchmark surveys, indicating that the sample was broadly representative of the population.(17)

Despite the broad representativeness of the present sample, it should be noted that the LAW Survey is unlikely to have reached some demographic groups — namely:
    • people living in households without landline telephone access, including many Indigenous households throughout Australia, particularly in remote areas, and people living in mobile-telephone-only households(18)
    • people who were homeless and living without landline telephone access for the entire survey reference period(19)
    • people with poor English who did not speak Italian, Greek, Cantonese, Mandarin, Arabic or Vietnamese
    • people who were institutionalised (e.g. prisoners, or people in care institutions due to age or disability)
    • people living in private dwellings who were physically unable to complete a phone interview (e.g. due to age, disability, inebriation, etc.).

As a result, the LAW Survey may underrepresent these disadvantaged groups, despite providing broad representation of the general population.

Data analysis

Descriptive analyses, such as frequencies and percentages, were compiled from the survey data to address various aspects of the first six aims of the study — that is:
1. the prevalence of legal problems
2. the nature of legal problems
3. the strategies used in response to legal problems
4. the advice received for legal problems
5. the finalisation of legal problems
6. the outcome of legal problems.

Inferential statistical analyses were used to examine the demographic and problem characteristics related to each of the above. The inferential statistical analyses included both bivariate analyses (i.e. analyses involving only two variables) and multivariate analyses (i.e. analyses involving more than two variables). A summary of these bivariate and multivariate analyses is provided below, and further details are provided in Appendix A2, ‘Data analysis’ section.

Bivariate analyses

Bivariate analyses examine the straightforward relationship between two variables, without taking into account the effect of any other variables. The nature of the variables determines which bivariate analysis is appropriate. Chi-square tests, Somers’ d tests and bivariate regressions were used in the present study. Throughout this report series, the type of analysis used and the statistical results obtained are detailed in the notes to the table or figure pertaining to each analysis.

Chi-square tests are appropriate bivariate tests for examining the relationship between two nominal categorical variables (i.e. variables with qualitatively distinct, unordered categories). Somers’ d tests examine whether one variable tends to increase or decrease as another variable increases. Somers’ d tests are appropriate for analyses involving two ordinal categorical variables (i.e. variables with categories that have an intrinsic ordering) and for analyses involving one ordinal categorical variable and one binary variable.(20) In each jurisdiction, two bivariate Poisson regression analyses were conducted, both of which examined the relationship of legal problem group (which is a nominal categorical variable) to a variable based on count data.(21)

The bivariate analyses often involved variables that were not examined via the multivariate analyses conducted, such as:
    • problem severity
    • number of adverse consequences of legal problems
    • number of legal problems per respondent
    • number of action types in response to legal problems
    • number of advisers used for legal problems
    • adviser type used for legal problems
    • type of help from main adviser
    • manner of finalisation of legal problems.

All bivariate analyses were adjusted for weighting of the data and, where appropriate, for clustered observations. Significance of the bivariate analyses was examined at the 0.05 level (p<0.05), except where Bonferroni corrections were applied to chi-square tests. For the chi-square tests, the adjusted standard residuals were calculated to assist with interpretation. The ‘significant’ differences between categories of variables that are described in the text are based on these residuals.

Multivariate analyses

Two types of multivariate analyses were used — multivariate regression and hierarchical cluster analysis.

Multivariate regression analyses were conducted to examine the demographic and problem characteristics that predict various outcomes. Multivariate regressions determine which variables from a set of variables are independent predictors of an outcome variable (e.g. Agresti 1996; Hosmer & Lemeshow 2000; Menard 2002). They examine the association of each potential predictor with the outcome variable when the effects of the other potential predictors are taken into account.

In each jurisdiction, a separate multivariate regression model was fitted for each of the following outcome variables:
1. the prevalence of legal problems overall
2. the prevalence of substantial legal problems
3. the prevalence of multiple legal problems
4. the prevalence of each of the 12 legal problem groups
5. the strategy used in response to legal problems — taking action
6. the strategy used in response to legal problems — seeking advice
7. the finalisation status of legal problems
8. the favourability of the outcome of legal problems.

Each model examined the relationship of the outcome variable to multiple independent or predictor variables. These included various demographic characteristics of the respondents (e.g. gender, age, Indigenous status, disability status, education, employment status, family status, housing type, main income, main language and remoteness of residential area(22)), and legal problem characteristics (e.g. problem recency, problem group and strategy used in response to legal problems). The potential predictor variables and their categories are described in Appendix Table A2.8.

The multivariate regressions were conducted on weighted data and adjusted for clustered data where appropriate. The significance of each comparison tested in the regressions was examined at the 0.05 level (p<0.05). The odds ratio or incident rate ratio for each comparison was also calculated where appropriate. See Appendix Table A2.9 for full details of all the regression models conducted for each jurisdiction, including the types of regressions used, the predictors in each model and the statistical packages used.(23)

Hierarchical cluster analysis was used to examine the nature of legal problems (i.e. aim 2). The cluster analysis examined which legal problem groups tended to be experienced by the same individuals — that is, tended to co-occur.


2. The LAW Survey used standardised wording so that non-problematic events were unlikely to be captured. It systematically asked ‘Have you had any problems or disputes [related to] …’ For example, the LAW Survey asked ‘Have you had any problems or disputes over a will or deceased estate?’ whereas the NSWLNS asked ‘Have you made or altered a will?’

3. The problem groups are presented in bold, and the problem subgroups are presented in italics. The accidents, credit/debt, employment and personal injury problem groups do not have any problem subgroups. In analyses based on problem subgroups, each of these problem groups was treated as a separate subgroup.

4. Note that questions D13, D21–D23 and P39 do not appear in the final version of the survey instrument. These questions were removed during the early stages of fieldwork.

5. Note that this section on problems or disputes also included one demographic question, D19, which asked about separation from a partner other than a spouse. This demographic question was asked immediately after the problem question, P29, which asked about the related issue of divorce or separation from a spouse (see Appendix A1).

6. When respondents reported two or more instances of the same specific type of legal problem, they were asked to rate the severity of only the worst instance of these problems.

7. Respondents were asked about 129 specific types of legal problems and could have experienced multiple instances of any of these specific types. See question D20 and the lead-in to question A1 in Appendix A1 for further details about the selection of legal problems for in-depth follow-up.

8. The NSWLNS did not explicitly ask about the occurrence of court/tribunal proceedings or formal dispute resolution, although it did ask whether problems were finalised by court/tribunal proceedings.

9. The NSWLNS’s ‘sought help’ category included cases that did not involve a formal adviser but involved any of the following actions: consulting relatives or friends, communicating with the other side, oral or written information, and website information. The LAW Survey’s ‘handled without advice’ category also differed from the NSWLNS’s ‘handled alone’ category. This LAW Survey category meant that formal advisers had not been consulted, but at least one of the following five types of actions had taken place: consulting relatives or friends informally, communicating with the other side, using websites or self-help guides, court or tribunal proceedings, or formal dispute resolution sessions. In contrast, the NSWLNS’s ‘handled alone’ category meant that respondents had endorsed handling the issue themselves without seeking information, advice or assistance from anyone else.

10. Further details about RDD are presented in Appendix A2, ‘Sampling: Random digit dialling’ section.

11. The NSW sample (N=4113) included 421 respondents from the supplementary LGA sample. Statistical analyses comparing the two surveys are not provided in the present report series.

12. The Victorian sample (N=4410) included 751 respondents from the supplementary remote/rural sample and 80 respondents from the supplementary Indigenous sample. The supplementary Indigenous sample used a somewhat different sampling strategy from that for the main survey. It targeted areas (e.g. SLAs or Census Collector Districts) that have a relatively high Indigenous population via both RDD sampling and sampling from listed telephone landline numbers. This strategy reduced the phone calls required to make contact with Indigenous respondents, given their small population proportion.

13. Details about call procedures, interviewer training and auditing, data quality checks during fieldwork and the preparation of the data for analysis are provided in Appendix A2, ‘Fieldwork’ and ‘Data preparation for analysis’ sections.

14. Thus, the numbers of Indigenous Victorian respondents reported throughout the Victorian LAW Survey report represent weighted numbers rather than the actual numbers interviewed.

15. Further details about weighting targets and reporting weighted data are provided in Appendix A2, ‘Weighting’ section.

16. For further details, see Appendix A2, ‘Response rate’ section.

17. See Appendix A2, ‘Comparison of sample and population profile’ section for further details.

18. The number of mobile-telephone-only households in Australia is increasing, with the proportion being higher for some demographic groups, such as young people, the unemployed, low-income households and households in disadvantaged areas (see Dal Grande & Taylor 2010; Pennay & Bishop 2009). However, landline telephone surveys conducted in 2006 were estimated to have produced generally representative population estimates once weighting incorporating multiple demographic indicators was applied (Blumberg & Luke 2007; Keeter, Kennedy, Clark, Tompson & Mokrzycki 2007).

19. Some people who had been homeless at some point during the reference period were interviewed. These people were either no longer homeless at the time of interview or, although they were still homeless at the time of interview, they were staying somewhere with landline telephone access.

20. Somers’ d analyses require specification of an outcome variable and an independent variable. The outcome variable used for each Somers’ d analysis is stated alongside the statistical results of the test in the relevant table or figure note.

21. One of these regressions was an ‘ordinary’ Poisson regression, while the other was a zero-truncated Poisson regression. For both bivariate regressions, problem group was the independent variable, while the count variable was the outcome variable.

22. According to the ABS (2007a), Tasmania and the Northern Territory only have remote and regional areas while the ACT almost exclusively comprises major city areas. All other jurisdictions have remote, regional and major city areas. Because of this variation in geographical profiles, identical comparisons on remoteness of residential area could not be examined across jurisdictions. For example, remoteness of residential area could not be examined in the ACT. See Appendix A2, ‘Comparison of sample and population profile’ and ‘Data analysis’ sections.

23. An explanation of the interpretation of odds ratios and incident rate ratios is provided in Appendix A2, ‘Data analysis: Significance and strength of predictors’.

  


CLOSE
Coumarelos, C, Macourt, D, People, J, MacDonald, HM, Wei, Z, Iriana, R & Ramsey, S 2012, Legal Australia-Wide Survey: legal need in Australia, Law and Justice Foundation of NSW, Sydney