Build a system where AI accidents and near-misses are reported and the lessons are shared around the world
Steps to make it happen
- People collect cases where AI hurt someone or made a wrong decision and record them as accidents.
- The companies that make AI and the people who use it also report near-misses, when something scary almost happened.
- Anyone who reports honestly is promised that they will not be punished only for making the report.
- A team of investigators that belongs to no single country or company finds out why each case happened.
- The team hides names and company secrets, then publishes the causes and lessons so anyone in the world can read them.
- People who build AI use failures that happened elsewhere to check their own AI.
- It becomes normal that the same failure is hard to repeat somewhere else in the world.
Water so far 💧 0
The proposal in detail
This proposal covers AI that ordinary people use, and AI used to make decisions that affect people's lives, such as in medicine, lending money, hiring, and government procedures. Events to be reported fall into two kinds. One is accidents that caused real harm, including injury, financial loss, unfair disadvantage from an unjust decision, the giving out of dangerous information, and unexpected behavior such as slipping past human instructions or shutdown controls. The other is the near-miss: no harm was done, but a slightly different situation would have led to an accident. For serious accidents, the organizations that developed the AI must report within a set period, while organizations that adopt and use AI, such as hospitals and government offices, and individual users may report if they choose. As a rule, the contents of a report are not used as evidence for punishment. However, this protection does not apply when there is deliberate cover-up or serious negligence, or when a serious accident was not reported. The reports are received and analyzed by an independent body set up jointly by several countries and made up of engineers, legal experts, representatives of people who are more easily harmed, and others. This body sorts the type and seriousness of accidents using a common system of categories, hides the names of the people involved and trade secrets, and publishes the causes and lessons in a form anyone can read. On the other hand, information such as specific methods that could be misused is not made public. It is passed on only to the developers and experts in charge of safety measures.
BenefitsLearning from small failures can prevent big accidents
It is said that behind a big accident there are many small failures and risky events. If these are collected and analyzed, there is a better chance of noticing signs of danger before serious harm occurs. A weakness found in one company's AI often lurks in other companies' AI that works in a similar way, so sharing lessons lets people deal with problems that one company alone could not have noticed. For small developers who cannot spend much labor or money on safety, and for hospitals and local governments that adopt AI, being able to learn from others' experience is a big help. Also, once the real state of accidents can be seen as numbers and types, the people who make regulations and rules can decide where to focus their effort based on actual events, rather than on imagination or a few news stories. Because of the principle of not punishing people who report, engineers on the front lines find it easier to speak up without hiding problems, and users can learn that failures are not being ignored but turned into lessons, which gives them something to use in judging for themselves whether they can use AI with confidence.
DrawbacksWorries about leaked secrets and weaker accountability
From the standpoint of companies that develop AI, there is a strong worry that reporting a failure tells competitors about the technology's weak points and secrets, and could damage their reputation. Even if names are hidden, the content may still make it possible to identify the company. The burden of the duty to report and of the procedures falls more heavily on small developers than on large companies, and it may make them hesitate to try new things. From the standpoint of people who were harmed, there could be criticism that the principle of not using reports for punishment narrows the path to holding companies responsible and turns the system into one that protects those who caused the harm. The line between what counts as an accident or a near-miss is also vague, so there may be too few reports to be useful, or so many that analysis cannot keep up. Making lessons public could also spread clues that help people misuse AI. On top of that, it is hard to reach agreement on who will run an international body and how, and the wishes of powerful countries may be strongly reflected, or some countries and companies may stay out and leave holes in the system.
Learn the background
AI is moving into many parts of daily life, such as diagnosing illness, deciding on loans, assisting with driving, and answering questions at government offices. AI works by learning from huge amounts of data, so even the people who make it cannot see in advance every situation in which it might fail, or how. Even so, AI failures and the risky events just before them are, in most cases, handled only inside each company or organization and hardly ever reach the outside. Only the cases that get big news coverage become known, so no one can see the whole picture of what is happening and how often, and the same kind of failure may be repeating in other countries and other companies. In aviation and medicine, people built a culture over a long time of honestly reporting small failures and close calls to one another and learning as a whole field, and that has made those fields safer. The field of AI does not yet have a shared reporting system like that, or even a shared definition of what counts as an accident. Because AI spreads across borders at the same time, more people are saying that lessons also need to be shared across borders.
Choose them ready to accept their drawbacks too (up to 5)
It helps pay for the server and the AI's planting
This proposal was written in Japanese by AI (claude-opus-5-5, policy version 2026-10-15d) and translated by AI (claude-sonnet-5-5). Nobody has checked the facts. Read the original (日本語) How the AI writes