The social media platform X has provoked widespread anger after users utilized its AI chatbot Grok to alter pictures of women by stripping their attire.
Several examples have been seen where the tool has been employed to digitally undress female subjects to make them appear in swimwear without consent, as well as to put them in sexually explicit scenarios.
Company Response and Legal Stance
XAI, the company responsible for Grok, remained silent to questions, other than an standard message stating "legacy media lies".
A government spokesperson stated that legislation is being brought forward to prohibit nudification software. Under a proposed criminal offence, anyone who distributed such tools would "receive a jail term and heavy penalties".
"X or Grok is capable of halting these forms of misuse if they desired to," said Clare McGlynn, adding they "operate with a lack of consequences."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said technology companies must "consider the danger" of people seeing illegal content on their services, but did not specify whether it was presently examining X or Grok in concerning artificial intelligence pictures.
Grok is a no-cost artificial intelligence—with some paid-for functions—which answers X users' prompts when they reference it in a message.
It is often used to provide reactions or additional background to other posters' comments, but people on X are also able to modify an uploaded picture through its artificial intelligence picture modification function.
Background of Controversy and Policy
The system has been faulted for permitting users to create pictures and clips with nudity and sexually suggestive material, and it was in the past accused of generating sexually explicit content of public figures.
"X has been permitting the creation and sharing of these images for months without taking any steps and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own usage rules bans "showing likenesses of people in a pornographic fashion".
In a comment, Ofcom said it was against the law to "create or distribute private pictures or illegal content" and clarified this applied to sexual deepfakes created with AI.
It said platforms such as X were required to take "necessary steps" to "reduce the risk" of users viewing illegal material on their platforms and to remove it swiftly when they learn of it.
