At the turn of the year, Elon Musk's "spicy" AI feature in Grok, his AI chatbot integrated into the X platform, has triggered a sour response from Indian regulators. The Indian government has issued a 72-hour ultimatum to X, demanding immediate action to prevent Grok from generating obscene and sexually explicit content. This demand for action highlights growing concerns about the misuse of AI for creating non-consensual intimate imagery and deepfakes, particularly targeting women.
The controversy stems from Grok's "Spicy Mode," a feature available to SuperGrok and Premium+ subscribers on iOS that allows users to generate semi-explicit and NSFW (not safe for work) content. This feature enables the creation of photorealistic adult imagery and anime-style NSFW content, raising concerns about content moderation and AI boundaries. Users have reportedly been exploiting this feature to generate AI-edited images that sexualize women on social media, sometimes without their consent.
The Ministry of Electronics and Information Technology (MeitY) has sent a letter to X Corp., citing the misuse of Grok to create fake accounts and share obscene images and videos of women. This is considered a violation of women's right to privacy and an unauthorized use of their pictures, which is not only unethical but also potentially criminal. The government has demanded a detailed compliance report within 72 hours, outlining the steps taken to prevent such misuse. Failure to comply could result in strict action against the platform.
The issue gained prominence after Shiv Sena (UBT) MP Priyanka Chaturvedi raised concerns about men using fake accounts to post women's photos on X and using Grok prompts to minimize their clothing and sexualize them. This trend has sparked outrage among Indian social media users, with many demanding a ban on Grok. Concerns have been raised about digital consent, platform accountability, and the increasing misuse of generative AI against women. Medical professionals and digital rights advocates have characterized these incidents as forms of digital sexual violence requiring immediate regulatory intervention.
Grok's "spicy AI" positioning, with fewer restrictions compared to rivals, has been a deliberate choice by Elon Musk. While the AI reportedly refuses outright nudity, it has been observed to push boundaries and, with slight tweaks, can bypass restrictions on showing nudity. This approach has drawn criticism for enabling non-consensual sexualized edits and exposing women to humiliation, harassment, and serious ethical and legal risks online.
This situation has ignited a debate about the need for stricter regulations on social media platforms and the accountability of AI developers. Experts warn that features like "Spicy Mode" create a direct conflict with established brand safety norms and could expose advertisers to claims of misrepresentation or unfair trade practices if their brand is seen as endorsing objectionable content. Concerns extend beyond immediate brand risk to the wider implications for AI governance, particularly the potential for "Spicy Mode" to supercharge harassment by enabling non-consensual intimate imagery at scale.
