Using our two-step process of democratic lottery helps our panels to be politically representative - and we have data to back that up!
The Sortition Foundation advocates for using a two-stage democratic lottery process when recruiting people for a citizens’ panel - a first stage of invitations to addresses selected by lottery from a postal database; and then stratifying the respondents by lottery to get a group of people that are demographically representative of the population.
The first stage of the process - sending out invitations by post - is, however, costly, particularly in Australia where postage costs are high, and understandably, there is often push back against including this stage of the process.
We were lucky, therefore, to be able to run a job where we had a postal invitation and an open expression of interest running side by side, so we were able to compare the demographic profiles of the two sets of respondents. The open expression of interest was publicised in the national media as well as through community groups that the organisation had previously contacted. Among the demographics included in this recruitment we asked about how people voted in the last election and whether they had participated in a political process in the last 12 months.
Before diving into the numbers, there are a couple of provisos to generalising from these results. First, the organisation convening the event has contacts with community organisations working in the area of the panel, which helped to increase the number of First Nations people and other target demographics within the open EOI. Whether this response rate could be replicated by other conveners would depend upon their ability to activate these networks.
The most important difference between the open EOI and the invitation-only process, in terms of the legitimacy of the process, was in the political profile of respondents. First, whereas only 10% of the invited respondents had taken part in a political process in the past 12 months, 25% of the open invitation respondents had done so. This result is unsurprising, and is one of the key reasons we advocate for a two-stage selection process - because those with strong views, those who are vocal for particular solutions or ways of framing a problem, are those most likely to put themselves forward in an open process. This tendency is exacerbated by the outreach strategies used by most convening organisations, where they are specifically reaching out to groups or individuals who have previously put their hand up to be involved - again, the most politically active segments of the community.
The second aspect of the political profile of respondents was less obvious from the outset. While both the open EOI and the invitation-only respondents tended to vote more to the left than the general population (defined informally as those voting for either Labor or the Greens), there was a stark contrast between the two. In the open EOI, Liberal/ National (LNP) voters made up only 11.9% of the respondents to the open EOI, compared to 22.9% of the invitation-only respondents, and 42.8% of the population as a whole. Likewise, voters for Pauline Hanson’s One Nation party or the United Australia Party (UAP) made up 1.7% of the open EOI respondents; 2.5% of the invitation-only respondents; and 10.9% of the total population.
How Australia voted in the 2022 Federal Election
How those who responded to the postal invite voted in the 2022 Federal Election
How those who responded to the open EOI voted in the 2022 Federal election
It’s important to note that our processes are designed to help skew back to the demographics of the total population, so the final group consisted of 35% LNP voters and 6.7% One Nation/ UAP voters - skewing it as close as possible to the national demographics as we could. In other words, the final panel was more to the right of the political spectrum than either the profile of the open EOI respondents or the invitation-only respondents, and thus more representative of the Australian population.
The make-up of the final panel, in terms of voting behaviour
Another surprising result was seen on gender. Over two-thirds (67.6%) of the open EOI respondents identified as female, only 29.7% identified as male, and 2.7% identified as non-binary or other. In contrast, the invitation-only respondents were 50.7% female, 47.5% male and 1.8% non-binary or other - much closer to the ABS statistics.
The last category where there were interesting differences was in age. For the invitation-only cohort, there was a much higher percentage of people under 35 (23.9% compared to 15.6% for the open process and 29% of the population). However, there was a lower response for those between 35 and 60 (45.7% vs 62%, cf 41.7% of the total population), but the over-60s were over-represented in both cohorts to about the same degree.
Lastly, we asked a question about housing tenure and found that the invitation-only respondents were much closer to the national profile, but the open EOI skewed to those in social housing (11% compared to a total population of 2.8%) and also included a handful of people who identified themselves as being currently homeless, who are difficult to reach through postal invites.
The results tend to reinforce our anecdotal experience that we get a more diverse range of people responding when they are directly invited, rather than through a 'catch-all' open process. Obviously, we are continuing to try and make our invitation materials more appealing to a broad cross section of people, and we are working to improve response rates. While this data comes from just one process, on an emotive issue, we feel that it points in the direction we had anticipated - that postal invites result in a more representative sample of the community than an open EOI process, particularly in terms of political involvement and voting behaviour.