I use AI every day, and as an advisor or founder I have been close to products that rely on algorithms: from an AI-optimised affiliate network to profile verification in a dating app. Since 2024 I have also been running AI Expo Europe (a business conference on AI in Bucharest), where I hear grand promises about what machines can do almost every week. I believe in this technology. That is exactly why one phrase, heard more and more often, gets under my skin: “the algorithm decided.”
No, the algorithm did not decide. Someone decided to use an algorithm, to train it on certain data, to put it in a certain place and to stop looking at it. When the system gets it wrong, responsibility doesn’t evaporate into the cloud. It stays with the people and the companies that chose to deploy it. That is what this piece is about.
Why this matters now, not in ten years
Automated systems no longer just recommend films or products. They help make decisions about loans, hiring, welfare benefits, insurance, accounts frozen for suspected fraud, or content removed from a platform. For the person on the other end, the difference between a “recommendation” and a “decision” is often purely theoretical: if your file has been flagged as risky, someone in a hurry will sign off on exactly what the screen told them.
A few years ago I wrote about the algorithms that decide what we see online. Back then, the stakes were a news feed. Today they are a loan, a job or a childcare benefit.
What the scandals have already shown us
We don’t need to imagine movie scenarios. We have real, documented cases:
- In the Netherlands, in the childcare benefits scandal (known as the “toeslagenaffaire”), a risk-profiling system and the way the authorities used it wrongly branded thousands of families as fraudsters. The consequences were so serious that the Dutch government resigned in January 2021.
- In the UK, faulty accounting software called Horizon, used by the Post Office, led to hundreds of sub-postmasters being wrongly prosecuted. The case exploded into public view after a TV drama aired in January 2024, and legislation to quash the convictions followed that same year.
- In Canada, in February 2024, a tribunal ruled that Air Canada was responsible for the wrong information its website chatbot gave a customer. The airline had effectively argued that the chatbot was responsible for its own statements. That argument did not fly.
- Amazon reportedly scrapped an experimental AI recruiting tool, according to a 2018 Reuters report, because it was biased against women. The system had learned from past hiring data, prejudices included.
Notice what they have in common: in none of these cases was the problem purely technical. It was blind trust in the system, the absence of a person who could say “hold on, something’s wrong here” and, above all, institutions refusing to own their mistakes in time.
The principles I’d like to see on every company’s wall
I’m not a lawyer and I’m not drafting a code. I’m an entrepreneur asking a simple question: what would I want to happen to me if an automated system decided something important about my life? My answer looks roughly like this:
- A human who can be reached and can overturn the decision. Not a form nobody answers, and not another chatbot. A person with real authority to reverse the machine’s verdict.
- The right to an explanation in plain language. Not “risk score 0.73”, but “your application was declined because your declared income doesn’t cover the repayment, and here is what you can do”.
- Bias testing before and after launch. Old data carries old prejudices. A system that passed its tests at launch can drift six months later.
- Logs and audits. If you can’t reconstruct why a decision was made, you have no business making it automatically.
- Proportionality. The higher the stakes for the person, the stricter the controls. Recommending a pair of shoes doesn’t need the same regime as denying someone welfare support.
- Clear liability in contracts. Between the AI vendor and the company using it, it must be spelled out who is responsible, for what, and how a mistake gets fixed.
- Special care for vulnerable people. Those with the fewest resources to challenge a decision are usually the ones most exposed to it.
What the law says, and where the gaps are
Europe isn’t starting from scratch. Article 22 of the GDPR gives people rights regarding decisions based solely on automated processing that significantly affect them. And the AI Act, which I wrote about when it was formally adopted, has started to apply in stages: since 2 February 2025, practices deemed to pose an unacceptable risk are banned, while obligations for high-risk systems come later.
On the other hand, in February 2025 the European Commission announced, in its 2025 work programme, that it intends to withdraw the proposed AI Liability Directive. I won’t argue here whether that is a good or a bad call; it’s a long and legitimate debate. But it does mean that, for a while, the question “who pays when the algorithm gets it wrong?” will be left largely to general rules and to contracts. When I wrote about what Europe is preparing on AI regulation, I said the law always arrives after the technology. In the meantime, that gap has to be filled by common sense and business ethics.
What an entrepreneur should do starting tomorrow
If you run a small company and use AI for customer scoring, CV screening, fraud detection or comment moderation, don’t wait for an inspection to put your house in order. In practice:
- List every place where an automated system influences a decision about a person. Many founders don’t even know how many there are.
- Name an owner for each one: a person, not a department.
- Write down what a customer or a candidate can do if they disagree, then walk that path yourself, like an ordinary customer would.
- Read your contract with the AI vendor. If it says nothing about errors, discrimination and liability, negotiate.
- Check the outcomes regularly across different groups of people. If one group gets rejected systematically more often, find out why before the press does.
- Don’t let a chatbot promise things your company won’t honour. The Air Canada case is a cheap lesson, if you learn it from the newspaper.
A fair society doesn’t automate itself
I once wrote about what a fair society means to an entrepreneur: clear rules applied equally to everyone, and institutions that own their mistakes. AI changes none of that. It just makes everything faster and bigger, injustices included. A human error affects one file. A system error can affect thousands of files before anyone notices.
That’s why I don’t think the answer is to fear AI. The answer is to treat it like any powerful tool: you use it, and you answer for it. A driver can’t tell the police officer that the car decided to run the red light. A company shouldn’t be able to tell a person that the algorithm decided to turn them down.
Where I stand
As an entrepreneur who both uses and builds AI products, I commit to one simple thing: if something I’ve put on the market wrongs a person, the fault is mine, not the code’s. I would rather launch more slowly, with a reachable human at the end of the line, than explain later why nobody answered. I tried to put these principles into practice when I wrote about building a responsible AI companion.
What about you: have you ever faced an automated decision you couldn’t challenge? And who do you think should have been held accountable?