Today, at the Parés i Aubia Convention, a question arose about the ethical and legal framework of artificial intelligence: What legal and ethical framework regulates artificial intelligence? I found it particularly interesting because we often talk about the possibilities of AI, but much less about the real conflicts its application poses.
And we already have several cases.
When Artificial Intelligence Makes Mistakes: Three Real Cases
In the Netherlands, the childcare benefits scandal highlighted the consequences of using fraud detection systems that incorporated discriminatory criteria. Thousands of families were unjustly accused and forced to return aid. The problem was not just technological: it was institutional. Risk classification systems and administrative decisions had been integrated into a system that lacked sufficient safeguards to challenge them.
Amazon abandoned an experimental hiring tool after discovering it penalized certain women’s resumes. The system had learned from historical hiring data, which was predominantly male. It did not discriminate because someone had explicitly ordered it to, but because it reproduced the patterns it had learned.
And in 2024, Air Canada had to answer for incorrect information its virtual assistant had provided to a customer about a special bereavement fare. The company tried to disassociate itself from the system’s response, but the court ruled that it remained responsible for information published through its channels.
Three different cases that share an underlying issue: when we incorporate automated systems into organizations, we also incorporate their errors, their biases, and their consequences.
What legal and ethical framework regulates artificial intelligence?
That’s why it seemed pertinent to revisit two frameworks that are already available.
UNESCO: The Ethical Principles of Artificial Intelligence
In 2021, UNESCO approved the Recommendation on the Ethics of Artificial Intelligence, which establishes principles such as non-discrimination, transparency, privacy, human oversight, and accountability.
This is an international framework of recommendations that, while not directly binding law for companies, also offers guidance for translating these principles into practice.
AI Act: The European Artificial Intelligence Regulation
The European Union has gone further with the Artificial Intelligence Regulation, the AI Act, which classifies certain uses according to risk and establishes specific obligations. It is no coincidence that particularly sensitive areas include personnel selection, education, access to essential services, and certain uses of biometric identification.
The regulation came into force in 2024 and is being applied progressively, with provisions already applicable and others with later deadlines. Therefore, it is important for companies to know the European regulatory framework and its application timeline, especially when incorporating AI systems into processes that may affect individuals or relevant decisions.
How does the use of AI affect daily business decisions?
However, what I find most interesting is to apply this to the daily decisions of businesses.
Imagine a company that incorporates AI into its CRM to score sales opportunities. The system learns that historically most profitable clients belong to a certain sector, have a specific size, or come from a specific geographical area. From there, it systematically begins to discard other profiles.
It might improve conversion in the short term. But it could also be conditioning the company’s commercial strategy without anyone having explicitly decided to exclude these segments.
Or consider a company that uses AI to generate code. The result works, passes initial tests, and goes into production. Months later, a vulnerability is discovered that exposes customer data. Who should have reviewed that code? What guarantees were established? Has the validation process been modified to adapt to the new way of programming?
This second example is particularly relevant to me. In our sector, I see how AI enormously accelerates production, but also how it can generate a false sense of solvency. A well-presented, coherent, and seemingly functional result can hide errors that only someone with enough experience is capable of detecting.
And here lies a paradox that concerns me: just when technology allows more tasks to be executed with less experience, professional judgment becomes more necessary.
From Regulation to Practice: The Responsibility of Businesses
I don’t believe the answer is to limit the use of AI. On the contrary. But we should review how we distribute responsibilities, what validations we demand, and what competencies teams need.
UNESCO offers us an ethical framework. The European Union specifies legal obligations. It is up to businesses to convert these principles into verifiable procedures.
For example, it is not the same to declare that an organization guarantees human oversight as it is to establish who reviews an automated decision, with what information, at what time, and with real capacity to modify it.
This, for me, is the interesting question that emerged at the convention today: not so much whether AI needs ethics or regulation, but whether organizations are adapting our decision-making and control mechanisms to the speed with which we incorporate this technology.
Because automating a task is relatively easy. Knowing what we are delegating, and what we are not, is another matter.
