San Francisco City Attorney David Chiu has filed a lawsuit against more than a dozen websites that his office alleges are encouraging the creation of nonconsensual deepfake nude images of women and girls worldwide.
The lawsuit contends that the 16 websites and their owners have violated state and federal laws prohibiting deepfake pornography, revenge pornography, and child pornography.
RELATED STORY | Trump shares AI images of a fake Taylor Swift endorsement
The city attorney’s office stated that these websites provide simple interfaces for users to upload clothed images of individuals and convert them into AI-generated, “undressed” versions, specifically targeting women and girls.
“These images, which are nearly identical to real photographs, are used to blackmail, intimidate, threaten, and shame women and girls,” the city attorney declared in a press release. “Furthermore, victims of nonconsensual deepfake pornography have little to no recourse or control over their own image once these manipulated images are distributed.”
The concerning trend of deepfake nude images has notably impacted celebrities, but officials are now worried about their exploitation of middle and high school students. The lawsuit cited an instance from February where such images of eighth-graders circulated within a California middle school.
Worries have escalated due to the widespread accessibility and popularity of these websites. The city attorney’s office reported that collectively, the websites received over 200 million visits in the first six months of 2024 alone.
RELATED STORY | Scripps News got deepfaked to see how AI could impact elections
“It is crucial to emphasize that this is not innovationāthis is sexual exploitation. This is a significant, multi-faceted issue that we, as a society, must address promptly,” stated Chiu. “Each of us must contribute to combating those who misuse AI to exploit and harm real individuals, including children.”
The lawsuit seeks the permanent shutdown of the 16 AI-generated websites, along with financial penalties to cover the expenses of the legal action.
Legislators have introduced various bills aimed at preventing and regulating the use of AI to produce deepfake images, particularly explicit ones.
