AI Detection Flags Portions of Pope Leo XIV's Encyclical on Artificial Intelligence

Textual Analysis Flags AI Signals in Vatican’s AI Encyclical

On May 27, Pope Leo XIV released Magnifica Humanitas, the Vatican’s first encyclical dedicated to artificial intelligence’s societal implications. Within hours, researcher Linch Zhang posted an analysis on LessWrong suggesting that portions of the document bore stylistic markers consistent with machine-generated text. According to The Verge, when Zhang submitted segments of the encyclical to the AI detector Pangram, the tool estimated that 40–100% of certain passages had been composed by large language models—with one independent run flagging 62% of the first chapter as machine-generated. The Verge’s own testing of roughly 2,000 words yielded a Pangram estimate of 46% AI composition.

The analysis identified linguistic patterns typically associated with synthetic text, including elevated frequency of the word “genuinely,” which appears commonly in outputs from Anthropic’s Claude model. However, Zhang also noted that other sections registered as entirely human-authored, with Pangram assigning 0% AI likelihood to some passages.

Detection Methods Remain Uncertain

The reliability of AI detection tools is contested within the research community. Pangram itself stated in March 2025 that its false positive rate—incorrectly labeling human writing as AI-generated—stands at approximately 1 in 10,000, according to The Verge’s reporting. Yet the detector faces fundamental limitations: different tools often produce divergent results on identical text, and consensus among detectors does not guarantee accuracy.

To contextualize Pangram’s flagging, The Verge notes that transcripts of Pope Leo XIV’s speeches scored 100% human-authored confidence when analyzed. Similarly, the opening 20 paragraphs of his four predecessors’ encyclicals all registered as entirely human-written under the same detector, suggesting that the Magnifica Humanitas findings deviate from typical Vatican communication patterns.

Pope Leo XIV Presents Magnifica Humanitas Alongside Anthropic Co-founder

The encyclical’s release was notable for its co-presentation with Christopher Olah, a founding researcher at Anthropic, the company whose model signature Zhang identified in the text. Encyclicals are formal epistles issued by the pope to address contemporary moral and theological challenges; the Vatican’s last such document came from Pope Francis in October 2024. Magnifica Humanitas is the first to center on artificial intelligence and its cascading effects on human society and dignity.

The Vatican did not respond to The Verge’s request for clarification on the encyclical’s authorship process or any role AI tools may have played in its composition.

Why This Matters

The incident surfaces a paradox: an institutional pronouncement on AI’s risks may itself be a case study in AI’s integration into high-stakes communication. If AI did assist in drafting the encyclical, it raises questions about institutional authenticity and the Pope’s intended voice—whether in moral teaching, authority derives from personal reflection or from stylistic outsourcing. Conversely, if the text is entirely human-authored, the false-positive rates of detection tools become newsworthy: they suggest that contemporary writing, even from human experts, may increasingly resemble machine output, either through convergent writing styles or through the detectors’ own brittleness.

For institutions and governments grappling with AI governance, the Magnifica Humanitas case demonstrates that AI detection remains too unreliable to serve as forensic proof of authorship. Until detection methods improve, claims about whether specific documents were machine-assisted will rest on institutional testimony, not technical evidence.