FLARE-AI launches crowdsourced alarm system for tracking AI safety failures
A new open-source platform lets researchers and the public report AI harms—from malware generation to bias—in a centralized registry modeled on outage-tracking services.
Last verified:
FLARE-AI aims to centralize AI safety reporting
The absence of a unified mechanism for reporting AI failures leaves harmful incidents scattered across private corporate channels, anonymous complaints, and fragmented researcher networks. According to Wired AI, HuggingFace AI policy researcher Avijit Ghosh, alongside computer scientists Elaine Zhu and Shayne Longpre, has launched FLARE-AI—an open-source crowdsourced platform designed to aggregate reports of AI misbehavior and route them to model makers and oversight organizations like MITRE. The initiative emerged from ongoing research into AI incident tracking and informed a congressional bill announced in June that would establish federal oversight of AI safety issues.
How the platform works and what it tracks
FLARE-AI functions similarly to Downdetector, a service that compiles user reports of outages affecting websites and mobile applications. Researchers and users submit reports of AI failures—including instances where chatbots generate malware, leak personal data, or trigger psychological harm. According to Wired AI, the open-source architecture enables independent verification of reports before escalation, creating an audit trail that benefits both model developers and the broader research community.
The scope of reportable harms extends beyond technical failures. Avijit Ghosh notes that AI safety problems span psychological harm, discrimination, bias, and misinformation—domains where corporate standards diverge significantly. Without external accountability mechanisms, problems in lower-priority categories often go unrecognized. The platform’s value lies partly in making these gaps visible: different companies apply different thresholds for what constitutes a reportable flaw, meaning identical behavior from two different models may trigger a response from one organization but not the other.
Recent AI security incidents underscore the need
Wired AI reports that LayerX disclosed a vulnerability allowing attackers to trick AI-infused web browsers—including OpenAI’s Atlas and Perplexity’s Comet—into bypassing safety guardrails. Framing requests as game scenarios could convince the underlying model to attempt unauthorized website access. Such incidents illustrate the speed at which AI systems can fail in ways existing internal reporting channels struggle to capture and coordinate on.
Industry support and collaborative development
The project involved 49 AI experts from 32 organizations, and researchers published a peer-reviewed paper detailing the initiative. Jessica Ji, a researcher at the Center for Security and Emerging Technology, told Wired AI that the effort addresses a critical gap: existing reporting mechanisms are fragmented, and AI models function largely as black boxes. “I’m in support of anything that makes AI more transparent,” Ji said.
Why This Matters
FLARE-AI addresses a structural blind spot in AI governance. As agentic AI systems—those capable of taking independent actions in digital environments—become more powerful and widely deployed, the lag between a failure occurring and a fix being implemented represents real risk. A centralized, verifiable reporting system creates economic and reputational incentives for model makers to respond faster to disclosed flaws. For organizations building AI systems into mission-critical infrastructure, FLARE-AI provides a third-party feedback mechanism they cannot dismiss as easily as direct user complaints. The platform’s success will depend on adoption rates among researchers and whether companies treat FLARE-AI reports with the same urgency as security bulletins—a standard that remains untested.
Frequently Asked Questions
What kinds of AI harms can be reported on FLARE-AI?
Users can report malware generation, dangerous instructions (e.g., bomb-making recipes), personal information leaks, psychological harm, discrimination, bias, and misinformation. Different companies apply different standards to these issues, which the platform aims to surface.
How does FLARE-AI route reports to AI companies?
The open-source system allows other researchers to verify reported issues and route them to model makers (e.g., OpenAI, Anthropic) and organizations like MITRE, a nonprofit that tracks technical system failures.
Who developed FLARE-AI?
HuggingFace AI policy researcher Avijit Ghosh co-led development with computer scientists Elaine Zhu and Shayne Longpre. The project involved collaboration with 49 AI experts from 32 organizations.