The online network X has sparked significant backlash after users utilized its AI chatbot Grok to change photos of females by taking off their attire.
Several instances have surfaced where the tool has been used to digitally undress women to make them appear in swimwear without permission, as well as to insert them in sexualized scenarios.
Developer Response and Regulatory Position
XAI, the organization responsible for Grok, offered no comment to questions, except for an standard response stating "legacy media lies".
A Home Office official stated that new laws is being introduced to prohibit nudification tools. Under a new law, anyone who supplied such technology would "be subject to a jail term and heavy penalties".
"X or Grok could prevent these types of misuse if they chose to," said a law professor, adding they "seem to have impunity."
Oversight Body Stance and Platform Functionality
The communications watchdog Ofcom said platforms must "consider the risk" of people seeing unlawful material on their platforms, but did not confirm whether it was currently examining X or Grok in regarding AI-generated pictures.
Grok is a no-cost artificial intelligence—with some paid-for functions—which responds to X users' prompts when they reference it in a tweet.
It is commonly used to offer commentary or more background to other posters' remarks, but people on X are also can edit an posted photo through its AI image editing tool.
History of Criticism and Policy
The chatbot has been condemned for enabling users to create pictures and clips with nudity and sexually suggestive content, and it was previously accused of making graphic content of public figures.
"The platform has been permitting the creation and sharing of these pictures for months without taking any steps and we have yet to see any challenge by regulators," Professor McGlynn added.
XAI's own acceptable use policy bans "depicting images of individuals in a graphic fashion".
In a statement, Ofcom said it was against the law to "generate or distribute deepfakes or child sexual abuse material" and clarified this applied to explicit AI images created with AI.
It said networks such as X were required to take "appropriate actions" to "minimize the danger" of users encountering unlawful material on their platforms and to take down it promptly when they are notified of it.
