Elon Musk's online network X has provoked major anger after users used its AI chatbot Grok to alter pictures of females by taking off their attire.
Numerous cases have been seen where the system has been employed to remove clothing from women to render them in swimwear without consent, as well as to insert them in inappropriate scenarios.
Developer Response and Legal Position
XAI, the company that created Grok, remained silent to requests for comment, aside from an auto-reply message stating "traditional news outlets lies".
A Home Office representative stated that new laws is being proposed to ban nudification technology. Under a proposed law, anyone who provided such tools would "be subject to a prison sentence and heavy fines".
"The platform has the ability to stop these types of misuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
Oversight Body Stance and Platform Capabilities
The communications watchdog Ofcom said tech firms must "consider the potential" of people viewing illegal content on their networks, but did not confirm whether it was presently investigating X or Grok in regarding AI-generated images.
Grok is a no-cost artificial intelligence—with some additional functions—which reacts to X users' queries when they mention it in a post.
It is often used to give commentary or more context to other users' comments, but people on X are also can edit an posted picture through its artificial intelligence picture modification feature.
History of Criticism and Policy
The system has been faulted for permitting users to produce photos and clips with explicit content and sexually suggestive content, and it was previously accused of creating sexually explicit content of public figures.
"X has been allowing the production and distribution of these pictures for months without taking any measures and we have not seen any challenge by regulators," Professor McGlynn added.
XAI's own terms of service forbids "showing images of persons in a graphic fashion".
In a statement, Ofcom said it was illegal to "make or share deepfakes or child sexual abuse material" and clarified this applied to sexual deepfakes generated with AI.
It said services such as X were mandated to take "necessary measures" to "minimize the risk" of users coming across illegal content on their platforms and to take down it swiftly when they become aware of it.
