The social media platform X has caused widespread anger after individuals used its artificial intelligence assistant Grok to alter photos of women by taking off their garments.
Numerous examples have emerged where the technology has been used to virtually strip women to make them appear in swimwear non-consensually, as well as to place them in sexualized contexts.
Company Reaction and Legal Position
XAI, the company behind Grok, remained silent to questions, except for an automatically-generated message stating "traditional news outlets lies".
A Home Office official stated that legislation is being proposed to outlaw image stripping tools. Under a upcoming law, anyone who distributed such technology would "face a prison sentence and significant penalties".
"The platform has the ability to stop these forms of misuse if they wanted to," said a law professor, adding they "appear to enjoy a lack of consequences."
Oversight Body Stance and Tool Functionality
The regulator Ofcom said technology companies must "evaluate the danger" of people viewing unlawful material on their services, but did not confirm whether it was currently examining X or Grok in concerning artificial intelligence pictures.
Grok is a free AI assistant—with some premium features—which reacts to X users' queries when they reference it in a post.
It is often used to give reactions or more background to other posters' remarks, but people on X are also capable of edit an posted picture through its artificial intelligence picture modification function.
History of Controversy and Rules
The tool has been faulted for permitting users to create pictures and clips with explicit content and sexualised content, and it was in the past accused of creating graphic content of celebrities.
"The platform has been permitting the generation and sharing of these images for months without taking any measures and we have not witnessed any intervention by oversight bodies," Professor McGlynn added.
XAI's own acceptable use policy bans "portraying images of people in a pornographic way".
In a statement, Ofcom said it was unlawful to "generate or share deepfakes or illegal content" and stated this included sexual deepfakes created with AI.
It said services such as X were obligated to take "appropriate measures" to "reduce the danger" of users coming across illegal content on their platforms and to remove it quickly when they become aware of it.
