Zürcher Nachrichten - California begins probe of Musk's Grok over sexualized AI images

The probe comes after Indonesia and Malaysia blocked access to Grok entirely, while Britain's media regulator launched its own investigation after users created lewd images using simple text prompts.

"The avalanche of reports detailing the non-consensual, sexually explicit material that xAI has produced and posted online in recent weeks is shocking," California Attorney General Rob Bonta said in a statement.

"I urge xAI to take immediate action to ensure this goes no further. We have zero tolerance for the AI-based creation and dissemination of nonconsensual intimate images or of child sexual abuse material."

Bonta said the investigation would determine whether and how xAI violated state law after the explicit imagery was "used to harass people across the internet."

Grok's so-called "Spicy Mode" feature allowed users to create sexualized deepfakes of women and children using prompts such as "put her in a bikini" or "remove her clothes."

There was no immediate response from xAI about Grok, which is integrated into Musk's social media platform X, formerly Twitter.

But the European Commission, which acts as the EU's digital watchdog, said it had taken note of "additional measures X is taking to ban Grok from generating sexualised images of women and children."

"We will carefully assess these changes to make sure they effectively protect citizens in the EU," EU Commission spokesperson Thomas Regnier said in a statement.

- 'Not a solution' -

Regnier did not specify what measures had been promised by the company.

When AFP asked the chatbot if xAI had banned the creation of sexualized deepfakes, Grok said: "xAI has imposed restrictions on Grok's generation of sexualized deepfakes following global backlash and investigations, but no full ban has been announced."

"Spicy Mode appears partially functional, though limited in some regions," it added.

Last week, Grok appeared to deflect the criticism with a new monetization policy, announcing on X that image generation and editing were now "limited to paying subscribers."

But that announcement only fueled further outrage, with British Prime Minister Keir Starmer's office condemning the move as an affront to victims and "not a solution" and the EU calling for a complete halt to the generation of such content.

Indonesia on Saturday became the first country to block access to Grok entirely, with neighboring Malaysia following on Sunday.

India said Sunday that X had removed thousands of posts and hundreds of user accounts in response to its complaints.

Britain's Ofcom media regulator said Monday it was opening a probe into whether X failed to comply with UK law over the sexual images.

And France's commissioner for children Sarah El Hairy said Tuesday she had referred Grok's generated images to French prosecutors, the Arcom media regulator and the European Union.

The European Commission, which acts as the EU's digital watchdog, has ordered X to retain all internal documents and data related to Grok until the end of 2026 in response to the uproar.

Last week, an analysis of more than 20,000 Grok-generated images by Paris non-profit AI Forensics found that more than half depicted "individuals in minimal attire" -- most of them women, and two percent appearing to be under-18s.

O.Hofer--NZN