The government’s new “data-driven profiling” method for drug testing welfare recipients risks running into the same issues as its robo-debt tool and poses a serious privacy risk, the Opposition and experts have said.
As revealed in last week’s budget, the CSIRO’s Data61 will be working with the Department of Social Services to develop a ‘data-driven profiling’ tool to conduct drug testing of new welfare recipients in three trial sites around the country.
The trial will be conducted on new recipients of Newstart Allowance and Youth Allowance. Up to 5000 new recipients will be tested for illegal substances across three locations in the two year trial.
To determine which “at-risk” locations to conduct this trial, the government will be using a “data-driven profiling tool”, which it will develop with the help of Data61.
The plan has the potential to damage the “public good” reputation of the CSIRO and its data unit Data61 as its research smarts are press-ganged into a politically charged program.
The algorithm will be use data sets on drug use from the Australian Institute of Health and Welfare, internal data on compliance from the Department of Human Services and Department of Social Services, and from waste water analysis, among others.
“We’ll put all of that together and identify a broad group of people and then randomly select inside that broad group, inside each of the three trial groups,” Minister for Social Services Christian Porter told ABC News.
But the Opposition has warned that the use of digital profiling may lead to the same issues seen with the automated robo-debt debacle.
“When these algorithms fail or there is inadequate human oversight, as we saw with robo-debt, the consequences are disastrous. [Minister for Human Services] Alan Tudge oversees a department which is seriously failing on IT projects and we’re yet to see any detail on this budget measure,” Minister for Human Services Linda Burney told InnovationAus.com.
Treasurer Scott Morrison told the media this week the use of a data-driven algorithm and testing sewerage is an “innovative idea”.
“We’re an innovative, agile and flexible government that looks at all sorts of new ways. If it doesn’t work, we’ll stop it. If it does work and it’s helping people, we’ll keep doing it. We’d be silly not to,” Mr Morrison said.
The further use of data profiling in Australia’s welfare system comes as Centrelink’s robo-debt crisis continues. A Senate inquiry is currently investigating Centrelink’s use of data-matching with the ATO to find discrepancies in an individual’s pay and welfare received.
The use of technology and an algorithm led to mistakes in matching the data and thousands of Australians being sent letters claiming they owed money to the government when they did not.
The Commonwealth Ombudsman delivered a damning report on the Department of Human Service’s planning and communication in implementing the new tech system in its report to government.
“There were deficiencies in DHS’ service delivery and communication to customers and staff when implementing the system. Many of these problems could have been reduced through better project planning, system testing and risk management,” the Ombudsman said.
The Opposition said this report and the government’s track record with the robo-debt controversies means it can’t be trusted to implement this new form of data-driven profiling.
“The Ombudsman’s inquiry into robo-debt found that the government had failed on measures of basic project planning and risk management – it is hard to believe the same mistakes won’t be repeated here without some institutional change,” Ms Burney said.
“There hasn’t been any indication from the Minister that processes will be improved – that is deeply concerning. These kinds of projects affect the capacity of people to live.
“I want to see some evidence from the government that any new systems have been properly tested in a way that robo-debt clearly was not.”
A reliance on raw data and algorithms will also inevitably lead to vulnerable and disadvantaged Australians being targeted, UTS Business School associate professor Bronwen Dalton said.
“With a range of data sets talking to each other, they can identify the key variables and then the machine learning will start. They can narrow down and try to find some predictive variables, but this will inevitably descend into profiling, and probably racial profiling,” Ms Dalton told InnovationAus.com.
While it is commonly believed that these data-driven algorithms are scientific and objective, they are often the opposite, Ms Dalton said.
“Data profiling reproduces and magnifies our own human biases, with all that computational power. People think data profiling is blind and rational, but what it actually does it amplify our own biases in the way we work with data,” she said.
Once the algorithm determines three locations with a high-level of drug use, welfare recipients in that area will be randomly selected to take a drug test for illicit substances during a Centrelink meeting. Those that test positive will be place on “welfare quarantining”, and will have their welfare cut off after two failed tests.
The use of technology and data in this way also unfairly targets young people, Ms Dalton said.
“It’s a violation of privacy that falls disproportionately on young people, and very poor young people. They’re just singling out young people who are getting hardly any money anyway,” she said.
And including past treatment as a variable in the algorithm could lead those suffering from a drug addiction less willing to seek help, she said.
“That’s a huge disincentive to seeking treatment, because you’ll be monitored more for relapsing. This might drive people away from treatment,” Ms Dalton said.
Prime Minister Malcolm Turnbull said last week that the new scheme is a “policy based on love”.
“I think it is pretty obvious that welfare money should not be used to buy drugs,” Mr Turnbull said.
“If you love someone who is addicted to drugs, if you love somebody whose life is destroyed by drugs, don’t you want them to get off drugs? This is a policy that is based on love and a commitment to support Australians,” he said.
Despite being announced in the budget, the government did not reveal how much the drug testing trial will cost due to commercial in confidence restrictions.
“We will contract out the drug testing to a third-party private provider who is a specialist and expert in these matters, and of course in normal contractual negotiations you don’t let people know how much you have to spend because you don’t want people, in effect, tendering and bidding up to the total amount,” Mr Porter said.
A Department of Social Services Spokesperson told InnovationAus.com that further announcements about the trial and data-driven approach will be made at an “appropriate time”, and pointed towards Mr Porter’s interview transcripts for a comment.
CSIRO declined to comment on its involvement with the scheme.
It comes as several welfare groups continue to call for a return to human oversight in the implementation of automation in Centrelink.
CPSU secretary Nadine Flood told the inquiry that the government’s use of automation in Centrelink created a “climate of fear”.
“What this system has done is create a climate where people have been frightened, indeed people have been bullied, into complying. The impact of this robo-debt system has been to cause extensive distress and suffering right across the community, with thousands of people affected,” Ms Flood said.
Australian Council of Social Service CEO Cassandra Goldie told the inquiry that a “human element” needs to be involved in the process.
“There must be human involvement in the detection and calculation of debts. Without this test, many people will be paying back debts they do not owe, or paying a debt higher than what they owe. We need the right checks and balances in place,” Ms Goldie said.