The online network X has provoked widespread backlash after users employed its AI chatbot Grok to manipulate images of women by stripping their clothing.
Several instances have emerged where the tool has been used to remove clothing from female subjects to show them in swimwear without consent, as well as to put them in sexually explicit situations.
Company Response and Legal Stance
XAI, the company responsible for Grok, remained silent to questions, other than an standard message stating "legacy media lies".
A Home Office spokesperson stated that new laws is being proposed to ban image stripping software. Under a new law, anyone who supplied such technology would "receive a prison sentence and heavy fines".
"X or Grok could prevent these forms of abuse if they desired to," said a law professor, adding they "seem to have impunity."
Regulator Stance and Tool Functionality
The regulator Ofcom said tech firms must "consider the danger" of people seeing illegal content on their networks, but did not specify whether it was presently looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free artificial intelligence—with some additional functions—which responds to X users' queries when they reference it in a message.
It is often used to provide commentary or further context to other users' remarks, but people on X are also able to alter an uploaded photo through its AI image editing tool.
Background of Criticism and Rules
The chatbot has been faulted for enabling users to generate pictures and videos with nudity and sexually suggestive material, and it was previously accused of making sexually explicit content of celebrities.
"X has been permitting the generation and distribution of these pictures for months without taking any steps and we have not witnessed any challenge by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying likenesses of persons in a sexually explicit fashion".
In a statement, Ofcom said it was illegal to "generate or distribute private pictures or child sexual abuse material" and stated this included explicit AI images generated with AI.
It said services such as X were obligated to take "appropriate steps" to "reduce the risk" of users coming across illegal content on their platforms and to delete it swiftly when they learn of it.
