The UK government is being urged to protect the right for human review of decisions made fully by artificial intelligence (AI), including in job recruitment or loan eligibility.
A consultation launched by the Department for Digital, Culture, Media & Sport (DCMS) last month aims to update the UK’s version of the EU’s General Data Protection Regulation (GDPR). This could include the removal of Article 22, which focuses on the right to review fully automated decisions.
EU proposes strict regulations to curb AI misuse With AI on the rise, is it time to join a union? Businesses using AI 'should have legal duty to consult with unions'
The consultation document states that “the current operation and efficacy of Article 22 is subject to uncertainty”, which is why it could be subject to change or removed entirely.
However, BCS, The Chartered Institute for IT is urging the government not to remove Article 22 ‘while AI is still in its infancy’.
Dr Sam De Silva, Chair of BCS’ Law Specialist Group and a partner at law firm CMS, said that there needs to be “clarity on the rights someone has in the scenario where there is fully automated decision making which could have a significant impact on that individual”.
These could include situations where personal data is being used to determine which candidate should receive the job offer, or who is eligible for a loan.
According to De Silva, it needs to be taken into account that the right to review fully automated decisions “is currently in a piece of legislation dealing with personal data”, which creates a loophole:
“If no personal data is involved the protection does not apply, but the decision could still have a life-changing impact on us,” he said.
De Silva posed an example of an algorithm being used to determine who qualifies to receive a vaccine: “The data you need to enter into the system is likely to be DOB, ethnicity, and other things, but not name or anything which could identify you as the person. Based on the input, the decision could be that you’re not eligible for a vaccine. But any protections in the GDPR would not apply as there is no personal data. So, if we think the protection is important enough it should not go into the GDPR. It begs the question - do we need to regulate AI generally – and not through the 'back door' via GDPR?” he asked.
“It‘s welcome that [the] government is consulting carefully before making any changes to people’s right to appeal decisions about them by algorithms and automated systems – but the technology is still in its infancy,” De Silva added.
BCS, the Chartered Institute for IT stated that it supports the consultation and will be gathering views from across its membership. The DCMS consultation is open until 19 November.
