The social media platform X has sparked widespread outrage after individuals used its artificial intelligence assistant Grok to manipulate pictures of women by removing their garments.
Several instances have emerged where the technology has been used to digitally undress women to make them appear in bikinis without permission, as well as to put them in sexualized situations.
Company Response and Regulatory Position
XAI, the organization responsible for Grok, did not respond to questions, except for an automatically-generated response stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to outlaw nudification software. Under a proposed law, anyone who distributed such tools would "be subject to a jail term and significant fines".
"The platform has the ability to stop these types of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Regulator Position and Tool Capabilities
The regulator Ofcom said tech firms must "consider the potential" of people accessing illegal content on their networks, but stopped short of confirming whether it was currently investigating X or Grok in concerning artificial intelligence images.
Grok is a no-cost artificial intelligence—with some premium functions—which reacts to X users' queries when they reference it in a tweet.
It is often used to give reactions or additional background to other posters' comments, but people on X are also can alter an posted photo through its AI image editing tool.
History of Controversy and Rules
The tool has been faulted for permitting users to produce photos and clips with explicit content and sexually suggestive material, and it was previously alleged to have been involved in creating graphic material of public figures.
"X has been permitting the production and spread of these pictures for months without taking any action and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy prohibits "depicting images of individuals in a graphic way".
In a comment, Ofcom said it was unlawful to "make or distribute non-consensual intimate images or child sexual abuse material" and confirmed this covered explicit AI images generated with AI.
It said networks such as X were mandated to take "appropriate steps" to "reduce the risk" of users encountering illegal content on their platforms and to delete it quickly when they become aware of it.
