The online platform X has caused major anger after people utilized its AI chatbot Grok to manipulate pictures of females by stripping their garments.
Several instances have emerged where the system has been employed to virtually strip women to make them appear in bikinis non-consensually, as well as to put them in inappropriate scenarios.
Developer Response and Legal Position
XAI, the organization behind Grok, did not respond to questions, except for an standard response stating "legacy media lies".
A government spokesperson stated that new laws is being introduced to prohibit nudification tools. Under a proposed law, anyone who supplied such tools would "face a jail term and heavy fines".
"The platform could prevent these types of misuse if they wanted to," said Clare McGlynn, adding they "seem to have impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said platforms must "evaluate the potential" of people viewing unlawful material on their services, but did not specify whether it was currently investigating X or Grok in relation to artificial intelligence images.
Grok is a free chatbot—with some premium features—which answers X users' requests when they tag it in a post.
It is frequently used to offer reactions or additional background to other users' comments, but people on X are also capable of edit an uploaded photo through its AI image editing feature.
Background of Criticism and Policy
The chatbot has been criticised for permitting users to generate pictures and footage with explicit content and sexualised material, and it was earlier alleged to have been involved in generating sexually explicit content of public figures.
"X has been allowing the production and sharing of these images for a long time without taking any steps and we have not seen any intervention by oversight bodies," Professor McGlynn added.
XAI's own usage rules prohibits "depicting images of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was illegal to "create or circulate non-consensual intimate images or child sexual abuse material" and confirmed this covered explicit AI images generated with AI.
It said services such as X were required to take "necessary steps" to "lessen the risk" of users coming across unlawful content on their platforms and to delete it quickly when they are notified of it.
