China Tightens Regulations on AI Chatbots Affecting User Interactions

In July 2023, China introduced stringent regulations on "human-like" AI chatbots, aiming to control AI interactions that simulate human relationships. This move impacts an estimated 29 to 70 million users within China, with these regulations being the first of their kind globally. The regulations specifically target AI applications offering "sustained emotional interaction," such as AI companions and character chatbots. These rules do not apply to customer service or educational AI unless they provide emotional support, creating a clear boundary between functional and emotional AI use.

This development follows a series of collaborative efforts between Chinese tech companies and the government to monitor and control technology usage. A notable instance is Douyin's intervention in 400 cases of attempted suicide in early 2025, showcasing the level of cooperation and surveillance infrastructure in place. The introduction of these regulations reflects China's recognition of AI's potential to influence human relationships and societal dynamics.

The new regulations signify a strategic shift in how China plans to manage AI technologies domestically. By imposing these rules, China aims to mitigate the risks associated with AI replacing human relationships, a concern highlighted during international discussions on AI risks. This regulatory framework potentially increases the Chinese government's leverage over tech companies, mandating a closer alignment with state objectives.

For Chinese tech firms, these regulations could necessitate significant adjustments in product offerings and business models. The focus on emotional AI implies a need to innovate within the constraints of the new rules while maintaining user engagement. Conversely, this move could enhance China's global standing as a leader in AI governance, influencing international norms and potentially setting a precedent for other nations.

Expect increased scrutiny and potential revisions to AI applications in China over the coming years. By 2027, Chinese tech companies might develop alternative AI solutions that comply with domestic regulations while retaining competitive advantages globally. Internationally, other countries could observe China's approach as a model for their regulatory frameworks, especially in regions with growing concerns about AI's societal impacts.

Additionally, the ongoing dialogue between global leaders, including the planned AI talks in November 2023, could lead to broader international agreements on AI governance. These discussions may further shape the landscape of AI regulation, with China playing a pivotal role in defining the global agenda.

The restrictions on emotional AI could lead to a shift in consumer behavior within China. Users might turn to foreign AI platforms that offer less regulated interactions, potentially increasing the demand for VPNs and other tools to bypass domestic controls. This could also spur the growth of a black market for unregulated AI applications.

Moreover, the regulatory environment may impact the development and deployment of AI technologies in adjacent sectors, such as mental health and social services. Companies in these fields might face challenges in integrating AI solutions that comply with the new regulations, potentially slowing innovation and adoption rates.

Analysts suggest that China's move to regulate AI chatbots reflects a broader strategy to assert technological sovereignty. By controlling AI interactions, China reduces its dependency on foreign AI models and strengthens its ability to influence domestic AI development. This aligns with China's long-term goals of technological self-reliance and leadership in the global AI landscape. Unlike previous regulatory efforts focused on data privacy, this approach directly addresses the social implications of AI, setting a new benchmark for AI governance worldwide.