In brief: We had the opportunity to do a near-copy of the same recruitment process with an increased amount of money offered to participants, and noticed a corresponding significant increase in the number of registrations. There does also seem to be some impact on the diversity of the respondents, and this is harder to measure because of the demographic target trends of Australian events, and because the two recruitment processes targeted different geographical areas.
In full: Citizens’ assemblies derive their legitimacy from their ability to bring together a wide, representative cross-section of the population. However, we’ve found in our work that there are some groups that persistently don’t respond to the invitations sent; this has also been seen in academic research – people from lower socio-economic groups, those in rural areas and young people, for example.
With registering one's interest to participate in a citizens' assembly being voluntary (itself the topic of an interesting discussion here), it’s important to try and ensure that the pool of people that we are selecting from is as representative as possible. The software that we have developed with academics from Harvard and Carnegie Mellon makes the selection as fair and representative as possible at a given response rate. The graphs below, taken from a registration in Victoria, illustrate the issue – you can see the difference between the demographics of the area and the people who responded to the invitation.
As we continually try to improve our services, a question that repeatedly comes up is - how do we encourage more people, and people from more diverse backgrounds to respond? There are some things that we have successfully tried – sending more invitations to lower-income areas, for example. Recently, however, we had the opportunity to work with an Australian client who sent out two batches of invitations, one immediately after the other, and in this way we were able to test the impact of increasing the remuneration on the response rate, from AUD600 to AUD1,000, and, in particular, see if it helped to improve the responses from those who traditionally are less likely to respond.
Those who were finally selected all received the same (higher) remuneration, so there was no discrimination between the two batches in that regard, it only affected the response rates of those invited to participate.
The first interesting result was that the overall response rate was much higher, with 98 responses coming from the first mail out, with the lower remuneration, and 142 responses from the second mail out of the same size. This difference is particularly stark as the second mail out was disproportionately sent to rural areas, where from anecdotal observations we would expect a lower response rate. The following graph illustrates this very clearly, using data from google analytics - the peak on the right hand side of the graph is when people started responding to the second mail out.
Google analytics data showing the number of people registering for the event
The second result was that in the most rural of the geographic areas, we received a total of 69 responses, but 52 of those came from the second batch. As noted, the second tranche of invites was weighted towards this area, but this increase was more than proportional.
Unfortunately, however, the increase in the remuneration made no difference to the number of young people registering – six people aged 16-24 registered in the first tranche, while five registered in the second tranche.
There was also a lower uptake from people who identified as renting, rather than home-owners in the second tranche, which appears to contradict what would have been expected – just over a third of the people who registered as renters were in the second tranche. We believe that this divergence is likely explained by the differences in the geographic spread – the first tranche had a larger number sent to urban areas, where there are more people renting than the overall area (28% of the urban population are renters, compared to 23% overall).
In general, the remuneration offered in Australia is much lower than it is in the UK, coming up at less than the minimum wage per hour (at the time AUD20.33 per hour), which could, in part, contribute to a general overall response rate that is lower than the average rate in the UK and elsewhere. This informal experiment shows that increasing the remuneration can increase response rates and help to reach some groups who otherwise might not respond.