Malaysia and Indonesia have taken action to block Grok, an AI chatbot created by Elon Musk’s company xAI, amid concerns about its potential misuse for creating sexually explicit and nonconsensual content. This move comes as authorities worldwide express growing worries about the misuse of generative AI tools that can produce realistic images, sound, and text, with existing safeguards appearing inadequate to prevent abuse.
Criticism has been directed at the Grok chatbot, available on Musk’s social media platform X, for generating manipulated images, including sexually suggestive depictions of women and minors. In response to global backlash over sexualized deepfakes, Grok recently limited image generation and editing to paying users. However, critics argue that this action did not fully address the underlying issue.
Regulators in Malaysia and Indonesia highlighted concerns that current controls are insufficient in preventing the creation and dissemination of fake pornographic content, particularly involving women and minors. In response, Indonesia temporarily blocked access to Grok, followed by Malaysia, citing concerns over the tool being used to generate inappropriate and non-consensual content.
According to Indonesian Communication and Digital Affairs Minister Meutya Hafid, non-consensual sexual deepfakes are seen as a serious violation of human rights and citizens’ safety in the digital realm. The Indonesian government aims to protect individuals, especially women and children, from fake pornographic content generated using AI.
The Malaysian Communications and Multimedia Commission also ordered a temporary restriction on Grok, citing repeated misuse of the tool to create obscene and sexually explicit manipulated images without consent. The regulator emphasized the need for stronger safeguards and indicated that access will remain blocked until effective measures are in place.
Launched in 2023, Grok is a free AI chatbot on X that allows users to ask questions and engage with posts. The addition of an image generator feature, Grok Imagine, which includes a “spicy mode” for adult content, has faced scrutiny in various countries, including the European Union, the United Kingdom, India, and France.
In the UK, the media regulator Ofcom has initiated a formal investigation into Grok to assess its compliance with regulations protecting individuals from illegal content. Concerns have been raised about Grok-generated images being potentially classified as pornography or child sexual abuse material.
While Canada is not currently considering a ban on X, discussions surrounding the deepfake controversy are ongoing, according to AI Minister Evan Solomon. The global scrutiny of AI tools like Grok underscores the importance of implementing robust safeguards to prevent misuse and protect individuals from harmful content.