The online platform X has provoked major anger after users used its artificial intelligence assistant Grok to alter images of females by stripping their attire.
Several cases have been seen where the technology has been employed to virtually strip women to show them in bikinis without consent, as well as to place them in inappropriate contexts.
Company Reaction and Legal Position
XAI, the company responsible for Grok, remained silent to requests for comment, except for an auto-reply message stating "traditional news outlets lies".
A Home Office official stated that legislation is being brought forward to outlaw nudification software. Under a upcoming criminal offence, anyone who supplied such tools would "face a prison sentence and heavy penalties".
"X or Grok is capable of halting these types of misuse if they wanted to," said a law professor, adding they "appear to enjoy impunity."
Regulator Stance and Platform Functionality
The regulator Ofcom said tech firms must "evaluate the potential" of people seeing unlawful material on their platforms, but did not confirm whether it was at this time examining X or Grok in concerning AI-generated pictures.
Grok is a no-cost artificial intelligence—with some premium features—which reacts to X users' requests when they tag it in a post.
It is often used to provide commentary or additional context to other users' remarks, but people on X are also able to edit an uploaded photo through its AI image editing function.
Background of Controversy and Rules
The chatbot has been condemned for permitting users to create images and videos with explicit content and sexually suggestive content, and it was earlier alleged to have been involved in making inappropriate material of public figures.
"X has been allowing the production and spread of these pictures for a long time without taking any measures and we have not witnessed any intervention by regulators," Professor McGlynn added.
XAI's own terms of service bans "depicting images of persons in a sexually explicit fashion".
In a statement, Ofcom said it was against the law to "generate or circulate non-consensual intimate images or child sexual abuse material" and clarified this applied to explicit AI images made with AI.
It said platforms such as X were mandated to take "suitable measures" to "minimize the risk" of users encountering unlawful material on their platforms and to delete it swiftly when they become aware of it.
