Listen to this article
Estimated 4 minutes
The audio version of this article is generated by AI-based technology. Mispronunciations can occur. We are working with our partners to continually review and improve the results.
Malaysia and Indonesia have become the first countries to block Grok, the artificial intelligence chatbot developed by Elon Musk’s company xAI, as concerns grew among global authorities that it was being misused to generate sexually explicit and nonconsensual images.
The moves reflect growing scrutiny of generative AI tools that can produce realistic images, sound and text, along with concerns that existing safeguards are failing to prevent their abuse.
The Grok chatbot, which is accessed through Musk’s social media platform X, has been criticized for generating manipulated images, including depictions of women in bikinis or sexually explicit poses, as well as images involving children.
Last week, Grok limited image generation and editing to paying users following a global backlash over sexualized deepfakes of people, but critics say it didn’t fully address the problem.
An emailed request for comment by The Associated Press to xAI resulted in an automated reply from the media support email address which stated, “Legacy Media Lies.” This was the same message received from a different email when asked for comment regarding the global backlash.
Regulators in the two Southeast Asian nations said that existing controls weren’t preventing the creation and spread of fake pornographic content, particularly involving women and minors. Indonesia’s government temporarily blocked access to Grok on Saturday, followed by Malaysia on Sunday.
X owner Elon Musk has previously said that the Grok chatbot was designed to be ‘politically incorrect’ and ‘anti-woke.’ (Patrick Pleul/The Associated Press)
“The government sees non-consensual sexual deepfakes as a serious violation of human rights, dignity and the safety of citizens in the digital space,” Indonesian Communication and Digital Affairs Minister Meutya Hafid said in a statement Saturday.
The ministry said the measure was intended to protect women, children and the broader community from fake pornographic content generated using AI.
Initial findings showed that Grok lacks effective safeguards to stop users from creating and distributing pornographic content based on real photos of Indonesian residents, Alexander Sabar, director-general of digital space supervision, said in a separate statement.
He said that such practices risk violating privacy and image rights when photos are manipulated or shared without consent, causing psychological, social and reputational harm.
In Kuala Lumpur, the Malaysian Communications and Multimedia Commission ordered a temporary restriction on Grok on Sunday, after what it said was “repeated misuse” of the tool to generate obscene, sexually explicit and non-consensual manipulated images, including content involving women and minors.
The regulator said that notices issued this month to X Corp. and xAI demanding stronger safeguards drew responses that relied mainly on user reporting mechanisms.
“The restriction is imposed as a preventive and proportionate measure while legal and regulatory processes are ongoing,” it said, adding that access will remain blocked until effective safeguards are put in place.
Investigation launched in U.K.
Launched in 2023, Grok is free to use on X. Users can ask it questions on the social media platform and tag posts they’ve directly created or replies to posts from other users. Last summer, the company added an image generator feature, Grok Imagine, that included a so-called spicy mode that can generate adult content.
The Southeast Asian restrictions come amid mounting scrutiny of Grok elsewhere, including in the European Union, the United Kingdom, India and France.
WATCH | Mom says Grok chatbot in Tesla told her son to send nudes:
Mom says Tesla’s Grok chatbot told her 12-year-old son to send nudes
A Toronto mom says her 12-year-old son asked Tesla’s Grok which soccer player is better: Cristiano Ronaldo or Lionel Messi. After some back and forth, she says the chatbot asked her son, ‘Why don’t you send me some nudes?’
On Monday, the U.K.’s media regulator said that it launched a formal investigation into whether Grok “complied with its duties to protect people in the U.K. from content that is illegal.”
The regulator, Ofcom, said that Grok-generated images of children being sexualized or people being undressed may amount to pornography or child sexual abuse material.
“The content created and shared using Grok in recent days has been deeply disturbing,” Technology Secretary Liz Kendall said.
Canada is not considering a ban of X, AI Minister Evan Solomon said in a post on the platform last week. His office added that discussions about the deepfake controversy are underway.
