San Francisco Demands Apple and Google Remove AI 'Nudify' Apps From Stores
City attorney sends cease-and-desist letters targeting 13 face-swapping apps that generate nonconsensual intimate imagery, alleging the tech giants profit from illegal deepfake technology.
Last verified:
Cease-and-Desist Against Tech Giants
San Francisco City Attorney David Chiu issued cease-and-desist letters on July 17 to Apple and Google targeting 13 face-swapping applications that generate nonconsensual intimate imagery. According to Wired AI, the letters demand the companies remove the apps from their stores and cease profiting from what Chiu characterizes as illegal sexual abuse technology.
The legal notices allege that Apple and Google are “aiding and abetting” the distribution of explicit deepfakes and should “sever” business relationships with the app developers. Chiu told Wired AI that “generating non-consensual intimate images is illegal, harmful, and completely unacceptable,” and stated the city’s office previously pursued legal action against 16 deepfake websites.
Alleged Revenue From Harmful Technology
Chiu’s office contends that the tech platforms have accumulated substantial revenue from these applications. According to the cease-and-desist letters, Apple and Google have likely “made millions of dollars in fees” through in-app purchases tied to nudification tools. The city attorney emphasized that both companies bear responsibility for vetting apps before distribution: “These companies have responsibility to ensure that apps on their platforms do not facilitate sexual abuse.”
The legal argument rests on California law, which the letters assert prohibits supporting services that create deepfake pornography. Despite existing developer policies at both platforms that formally ban pornography, abuse, and harassment, researchers have repeatedly identified nudification apps in both the App Store and Google Play Store—some even rated as appropriate for children.
Company Responses and Prior Removals
Google responded to Wired AI with a statement from spokesperson Dan Jackson, confirming the company has deleted “hundreds” of apps with nudifying features for policy violations, including five Android applications flagged by Chiu’s office. Google stated it restricts search terms like “nudify” on its store and suspended related apps for violating its prohibition on sexual content.
Apple did not provide comment ahead of publication, according to Wired AI.
Both platforms have a documented history of removing nudification tools following reports from researchers and journalists, though the cease-and-desist suggests such removals have occurred reactively rather than through proactive detection at scale.
Why This Matters
This enforcement action signals accelerating legal pressure on platform operators to treat deepfake pornography as a liability rather than a moderation edge case. The San Francisco city attorney’s framing—that app-store operators are direct profiteers in sexual abuse—could reshape how Apple and Google evaluate monetization mechanisms tied to potentially harmful AI tools. If the cease-and-desist succeeds, it may establish precedent for municipal enforcement against platforms that host generative AI apps, particularly those involving nonconsensual intimate imagery. The outcome will likely influence how other jurisdictions approach similar enforcement and whether platform moderation policies shift from reactive removal to pre-launch vetting of AI-enabled apps.
Frequently Asked Questions
What are 'nudify' apps and how do they work?
These are face-swapping applications that use AI to generate nonconsensual intimate imagery by digitally undressing individuals in photos. Users upload images and the apps produce sexualized deepfakes without the depicted person's consent.
Is nonconsensual deepfake pornography illegal in California?
Yes. According to the cease-and-desist letters, California law prohibits supporting services that create deepfake pornography, and the city attorney argues Apple and Google are violating this by hosting and profiting from such apps.
How do Apple and Google profit from these apps?
Both companies take a percentage of in-app purchase revenue. According to the city attorney's office, the tech giants have 'made millions of dollars in fees' from nudification apps.
Have these companies removed such apps before?
Yes. Both Apple and Google have previously deleted dozens of nudify and deepfake apps after reports from researchers and journalists. Google states it has deleted 'hundreds' of apps with nudifying features for policy violations.