The social media network X has provoked major backlash after individuals employed its AI chatbot Grok to change pictures of females by taking off their attire.
Numerous instances have been seen where the tool has been employed to digitally undress women to make them appear in bikinis without permission, as well as to place them in sexualized situations.
Company Reaction and Regulatory Stance
XAI, the company responsible for Grok, offered no comment to questions, other than an auto-reply response stating "legacy media lies".
A Home Office spokesperson stated that legislation is being brought forward to ban image stripping technology. Under a new law, anyone who supplied such technology would "be subject to a prison sentence and significant fines".
"The platform is capable of halting these types of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy impunity."
Oversight Body Position and Platform Capabilities
The communications watchdog Ofcom said platforms must "assess the potential" of people accessing unlawful material on their networks, but did not confirm whether it was at this time looking into X or Grok in concerning AI-generated images.
Grok is a free artificial intelligence—with some additional features—which answers X users' queries when they tag it in a message.
It is commonly used to offer reactions or further context to other users' remarks, but people on X are also can modify an uploaded image through its artificial intelligence picture modification feature.
History of Criticism and Rules
The system has been faulted for enabling users to generate photos and videos with nudity and sexualised material, and it was earlier accused of generating sexually explicit material of public figures.
"X has been enabling the production and spread of these pictures for months without taking any action and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own usage rules forbids "depicting images of people in a sexually explicit manner".
In a statement, Ofcom said it was illegal to "create or distribute non-consensual intimate images or child sexual abuse material" and stated this covered sexual deepfakes made with AI.
It said services such as X were mandated to take "suitable steps" to "reduce the risk" of users viewing unlawful material on their services and to take down it quickly when they learn of it.