Blocking Illegal Content Before It’s Online? French Moderation Startup Turns To AI To Make The Internet Safer

Cyberbullying, sexual harassment, violent imagery, deepfakes... The number of online aggressions continues to grow, and problematic content is proliferating despite existing regulations. To help combat this scourge, the French startup Moderering (the Danish word for “moderation”) has developed a new tool powered by artificial intelligence to automate moderation. Its technology stands out by verifying content upstream, blocking it before it is even posted online or shared with users. Each AI agent has been trained to detect a specific category of risk: cyberbullying, hate speech, racism, extremism, fraud, sexual violence, child sexual abuse material, and so on.

The solution is aimed at private companies such as social media platforms, hosting providers, apps, and streaming services, and integrates into their existing environments. The AI can analyze multiple types of content in real time: text, images, videos, conversations, and more.

Alexandre Sossou, Moderering’s founder, who worked in the moderation sector for several years, explains that the tool works with AI agents connected through APIs (application programming interfaces) to the moderation backend are tasked with “analyzing and filtering risky content before it is posted on the platform.”

When content is deemed illegal or non-compliant, it is blocked by the AI and stored in a secure environment to allow for human review and verification. The problematic content could also potentially be forwarded to the relevant authorities, where appropriate.

Sossou is also targeting government clients that need to detect illegal content in order to identify companies failing to comply with moderation requirements, particularly under the Digital Services Act (DSA). In that case, public bodies “could use software where they simply enter the names of the sites they want the AI to analyze,” the entrepreneur explains. By assessing the level of risk, the solution enables organizations to make appropriate decisions more quickly.

According to its founder, Moderering has the potential to become a genuine technology for protecting internet users. “We discovered that we had automatically identified websites hosting rape videos,” explains Sossou. The technology could also enable AI to detect criminal content and report it to the competent authorities.

AI-powered automated review also addresses the limitations of human moderation. First, there is the sheer volume of content published online every day, which has become unmanageable for human teams. Second, there is the need to protect moderators themselves. Currently, most digital companies rely on outside firms such as Accenture or Besedo, which hire workers to perform manual moderation under often precarious conditions. “These people look at horrific things all day long. It can be extremely traumatic. And after a while, under those conditions, humans can make mistakes,” says Sossou. “With AI, the work is much more efficient and much faster.”

In this context, the human role will evolve. “People will supervise these AI agents, for example to review a complaint from a user asking why they were blocked even though their content was legal,” he adds.

Moderering’s approach runs counter to the model currently adopted by major American tech companies. “They encourage users to publish content en masse, but it is the user’s responsibility to click a button to report illegal content. Obviously, that allows all sorts of things to slip through on platforms,” Sossou notes.

While the United States allows platforms such as Kick—banned in France after the death of Jean Pormanove during a livestream—to continue operating, Europe has chosen a stricter regulatory framework. “In the event of non-compliance with the DSA, platforms face fines of up to 6 percent of their turnover,” the entrepreneur notes.

Moderering, which was incubated at Agoranov before joining the Escalator program, hopes to establish itself as a leading European solution capable of helping enforce the DSA properly. The remaining challenge is convincing regulators to adopt its technology.