The social media platform X has provoked widespread outrage after people used its AI chatbot Grok to alter images of females by taking off their attire.
Several cases have emerged where the tool has been used to remove clothing from women to render them in bikinis without permission, as well as to put them in inappropriate situations.
Developer Reaction and Regulatory Stance
XAI, the organization responsible for Grok, offered no comment to inquiries, aside from an standard response stating "traditional news outlets lies".
A government official stated that new laws is being brought forward to prohibit image stripping tools. Under a upcoming law, anyone who supplied such technology would "face a jail term and substantial fines".
"X or Grok is capable of halting these forms of misuse if they desired to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said platforms must "assess the potential" of people viewing illegal content on their services, but stopped short of confirming whether it was currently examining X or Grok in relation to AI-generated images.
Grok is a free artificial intelligence—with some paid-for features—which reacts to X users' prompts when they mention it in a message.
It is commonly used to provide commentary or more background to other posters' remarks, but people on X are also can edit an uploaded picture through its AI image editing tool.
History of Controversy and Policy
The system has been criticised for permitting users to produce photos and videos with explicit content and sexually suggestive material, and it was previously accused of making inappropriate content of public figures.
"The platform has been permitting the generation and distribution of these pictures for months without taking any steps and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service prohibits "depicting likenesses of individuals in a sexually explicit way".
In a comment, Ofcom said it was illegal to "create or circulate non-consensual intimate images or illegal content" and confirmed this covered explicit AI images made with AI.
It said networks such as X were obligated to take "appropriate steps" to "minimize the risk" of users encountering illegal material on their services and to take down it swiftly when they are notified of it.
