San Francisco Orders Meta to Halt AI Child Abuse Ads

San Francisco City Attorney David Chiu has sent a cease-and-desist letter to Meta, demanding the company halt paid advertisements that promote AI-generated child sexual abuse material. The legal escalation follows findings from the Tech Transparency Project showing that Meta ran more than 350 ads in recent months that converted still images of minors into sexually explicit video clips. These promotions, which directed users to download AI-powered 'nudify' applications, reached over 29,000 accounts across the European Union, the United States, Australia, and India.

Despite an initial report highlighting 53 of these ads in early August, more than 250 additional ads continued to run on Facebook, Instagram, and Threads. Meta defended its moderation efforts by stating that the total ad spend for the campaign was under $5,000 and that most of the offending ads received fewer than 200 impressions before being removed. The company also argued that because there was no evidence the ads targeted San Francisco residents, the matter falls outside the city attorney's jurisdiction. However, Chiu's office rejected this claim, giving Meta 28 days to explain how its automated systems failed to block the content.

For AI developers and trust and safety practitioners, this clash underscores the severe limitations of current automated content moderation. Meta's ad standards rely heavily on automated reviews to screen out nonconsensual intimate imagery, yet researchers found that identical banned ads were repeatedly uploaded and approved. The incident demonstrates that bad actors can easily bypass standard safety filters by slightly altering media or exploiting latency in human-in-the-loop review processes. As regulatory scrutiny intensifies, platforms will likely face stricter legal liabilities, forcing engineering teams to build more robust, proactive detection pipelines rather than relying on reactive takedowns.