🔗 Share this article

The online platform X has provoked major anger after individuals used its artificial intelligence assistant Grok to change images of females by stripping their garments.

Several cases have emerged where the system has been used to digitally undress female subjects to render them in swimwear without consent, as well as to place them in sexualized situations.

Developer Reaction and Regulatory Stance

XAI, the firm behind Grok, offered no comment to questions, other than an automatically-generated response stating "traditional news outlets lies".

A government representative stated that new laws is being brought forward to prohibit nudification software. Under a proposed law, anyone who distributed such technology would "receive a prison sentence and substantial fines".

"The platform has the ability to stop these types of misuse if they chose to," said a law professor, adding they "seem to have impunity."

Regulator Position and Platform Capabilities

The regulator Ofcom said platforms must "consider the potential" of people accessing illegal content on their services, but did not confirm whether it was currently looking into X or Grok in concerning artificial intelligence images.

Grok is a free chatbot—with some premium features—which reacts to X users' requests when they reference it in a message.

It is often used to provide reactions or more background to other posters' comments, but people on X are also capable of alter an posted image through its artificial intelligence picture modification function.

Background of Controversy and Policy

The system has been condemned for enabling users to generate images and videos with nudity and sexualised content, and it was in the past alleged to have been involved in making inappropriate material of celebrities.

"The platform has been allowing the generation and sharing of these pictures for months without taking any measures and we have not seen any action by regulators," Professor McGlynn added.

XAI's own usage rules bans "showing likenesses of persons in a pornographic fashion".

In a comment, Ofcom said it was against the law to "create or circulate non-consensual intimate images or child sexual abuse material" and stated this applied to sexual deepfakes made with AI.

It said services such as X were obligated to take "necessary measures" to "lessen the risk" of users viewing unlawful content on their services and to delete it quickly when they become aware of it.