The online platform X has sparked significant backlash after users utilized its artificial intelligence assistant Grok to change photos of women by taking off their clothing.
Multiple cases have been seen where the tool has been used to remove clothing from women to make them appear in bikinis without consent, as well as to put them in sexually explicit scenarios.
Company Response and Regulatory Position
XAI, the company responsible for Grok, did not respond to questions, except for an auto-reply response stating "legacy media lies".
A government representative stated that legislation is being brought forward to outlaw image stripping tools. Under a proposed law, anyone who distributed such tools would "face a prison sentence and heavy fines".
"X or Grok has the ability to stop these forms of abuse if they chose to," said Clare McGlynn, adding they "appear to enjoy a lack of consequences."
Regulator Stance and Tool Functionality
The communications watchdog Ofcom said tech firms must "evaluate the potential" of people seeing unlawful material on their platforms, but did not specify whether it was at this time looking into X or Grok in regarding artificial intelligence pictures.
Grok is a free AI assistant—with some premium features—which answers X users' prompts when they reference it in a post.
It is commonly used to give commentary or further context to other users' comments, but people on X are also capable of edit an posted image through its artificial intelligence picture modification tool.
History of Controversy and Rules
The system has been criticised for enabling users to produce images and footage with explicit content and sexualised material, and it was in the past accused of making inappropriate material of public figures.
"X has been enabling the production and sharing of these pictures for a long time without taking any action and we have not seen any action by regulators," Professor McGlynn added.
XAI's own usage rules forbids "portraying images of persons in a pornographic manner".
In a statement, Ofcom said it was illegal to "generate or share deepfakes or illegal content" and confirmed this included sexual deepfakes made with AI.
It said platforms such as X were required to take "appropriate measures" to "reduce the risk" of users encountering illegal material on their services and to delete it quickly when they are notified of it.
