Regulating AI: Discrimination and Data Efficiency

Last September, Wired released an article The Fight to Define When AI Is ‘High Risk’, which addresses the shared concern of more than 300 organizations regarding the EU's Artificial Intelligence Act and the safeguarding of people against potential harm from AI and the effects of their applications.

Tech giants like Google and Facebook are rather obviously against what they style as over-regulation, or intend to differentiate between suppliers of 'generic' AI and its deployers, as the article states. Here the term 'generic' is important. As companies and public organizations increasingly move parts of their infrastructure to 'the cloud', and software (like our devices) becomes less a thing we own and more a subscription-based service (SAAS), the greatest profit lies in developing widely deployable software with little if any possibility of local control. Even public management organizations increasingly rely on external companies to provide software to manage the public realm and their data.

What strikes me is that in public policy making at national and EU-levels there is little inclusion at all of those software engineers and data scientists who are in fact developing AI and algorithms. It seems to me that as long as we don't include the actual developers who write software with increasingly autonomous functions, we will never really understand what we are regulating and therefore fall short. And I don't think this is a one way street either: what I've seen of rather large organizations and companies, there are often only a handful of engineers or system architects who can be said to truly understand the linkages between the multitudes of (legacy and new) systems in use, and the ephemeral spaces potentially created by easy fixes and shortcuts in IT system development.

Of course, this might be over-complicating the issue. But I wonder about the relationship between the IT-infrastructures we have created in the past with the continuation of the development of software as increasingly adaptive and integrative with these systems and the public domain. There's a distinct dimension to AI which continues a cyber-cultural tradition of digital utopianism or tech optimism in which solutions for social problems are sought in the technical domain.

I have met data scientists who assured their managers that of course the algorithm "does not discriminate". Obviously the algorithm discriminates. Algorithms do nothing else than discriminate on the basis of input. They are devices of calculated discrimination: it is their singular function. I do of course understand the implicated social associations with discrimination and the discourse that an algorithm should not discriminate against marginalized people of any kind. I'd like to stress here a difference in meanings. When data scientists talk about discrimination in the social sense, they don't talk about discrimination. They talk about data efficiency and how it relates to the distributional shift which increase safety and robustness of models. These words and concepts comprise the realm which should be included in the further invention of the ethics of AI. In this realm, sadly, human oversight is considered "particularly costly".

The question I raise is if we understand to what great extend we would need to guide technological developments in order to create sufficient feedback loops on governmental levels. Is our current neoliberal capitalist system sufficiently equipped to deal with the organizational change necessary to critically think about these socially-technically relevant questions, if the next big profit lies around the corner?

The 'High Risk' label for AI Wired mentions as a positive direction does not to me seem a bad thing. However, I do think this notion reveals our cultural immaturity when it comes to thinking about technology and innovation on a grand scale. Labels might give legitimacy to certain developments and not to others, but the idea of a label is that it can be taken off and swapped easily. The more labelling we do, the more we need organizations to keep track of these labels, without turning into a lethargic bureaucratic monstrosity or underfunded agency. I wonder if, in a time when EU oversight regarding data and privacy protection is so underfunded, we understand what it takes to commit to the regulation of AI and advanced technologies.


You'll only receive email when they publish something new.

More from Kevin's blog
All posts