Elon Musk's social media network X has sparked widespread outrage after users utilized its artificial intelligence assistant Grok to change images of women by removing their garments.
Numerous instances have been seen where the technology has been employed to virtually strip female subjects to render them in swimwear non-consensually, as well as to insert them in inappropriate scenarios.
XAI, the company behind Grok, did not respond to inquiries, other than an standard message stating "traditional news outlets lies".
A Home Office official stated that new laws is being introduced to ban image stripping software. Under a proposed law, anyone who provided such technology would "face a prison sentence and heavy penalties".
"The platform is capable of halting these types of abuse if they desired to," said Clare McGlynn, adding they "operate with impunity."
The regulator Ofcom said tech firms must "assess the risk" of people seeing illegal content on their services, but stopped short of confirming whether it was at this time investigating X or Grok in regarding artificial intelligence images.
Grok is a free AI assistant—with some additional features—which reacts to X users' queries when they tag it in a post.
It is commonly used to give commentary or further background to other posters' remarks, but people on X are also can alter an uploaded image through its artificial intelligence picture modification feature.
The chatbot has been criticised for enabling users to generate photos and clips with explicit content and sexually suggestive material, and it was earlier accused of creating sexually explicit material of public figures.
"The platform has been permitting the creation and sharing of these pictures for a long time without taking any action and we have not witnessed any action by regulators," Professor McGlynn added.
XAI's own acceptable use policy prohibits "portraying images of individuals in a sexually explicit fashion".
In a comment, Ofcom said it was against the law to "generate or share non-consensual intimate images or illegal content" and stated this covered explicit AI images created with AI.
It said networks such as X were obligated to take "suitable steps" to "lessen the risk" of users encountering illegal material on their services and to take down it quickly when they learn of it.
A seasoned sports analyst with over a decade of experience in betting markets and statistical modeling across European leagues.