A number of people on welfare in the US and UK say they do not trust the government’s use of AI to decide who gets help, after being wrongly denied benefits or accused of fraud.
The warning comes from a Nature Communications study by researchers from the Max Planck Institute after surveying 3,249 respondents in the US and the UK.
The study notes that while AI can speed up welfare payments, it also risks wrongly denying people or falsely flagging applications as fraudulent. Amid the potential inaccuracies, the survey finds a clear divergence between non-claimants and claimants.
Non-claimants or the general public say they are fine with AI use for welfare if it translates to efficiency gains, despite the potential for inaccuracies. But that’s not the case for those who actually rely on welfare, as they are far less likely to embrace AI systems to decide their fate.
“While the public generally tolerates modest accuracy losses for faster decisions, claimants are less willing to accept AI in welfare systems, raising concerns that using aggregate data for calibration could misalign policies with the preferences of those most affected.”
The study also finds that people outside the welfare system don’t really understand how risky and stressful AI feels to those who depend on it for survival, even if they are offered money to guess correctly.
“Our study further uncovers asymmetric insights between claimants and non-claimants. Non-claimants overestimate claimants’ willingness to accept speed-accuracy trade-offs, even when financially incentivized for accurate perspective-taking.”
Researchers say the danger is that government AI systems will end up tuned to public opinion — not to the people most affected by them. When that happens, the poorest are the ones who pay for mistakes they didn’t make.
“These well-intentioned non-claimants may use their dominant voice to shape public opinion and policy without realizing that they do not in fact understand the preferences of claimants, resulting in AI systems that are misaligned with the preferences of their primary, direct stakeholders.”
Disclaimer: Opinions expressed at CapitalAI Daily are not investment advice. Investors should do their own due diligence before making any decisions involving securities, cryptocurrencies, or digital assets. Your transfers and trades are at your own risk, and any losses you may incur are your responsibility. CapitalAI Daily does not recommend the buying or selling of any assets, nor is CapitalAI Daily an investment advisor. See our Editorial Standards and Terms of Use.