🔗 Share this article

The popular AI conversation site Character.ai is introducing a major policy change that will block young users from engaging in conversations with its virtual avatars.

This move is in response to growing backlash regarding the kinds of exchanges young people were engaging in with the AI entities.

Legal Pressure

The platform, which started in 2021 and is used by countless individuals, currently faces multiple lawsuits in the US.

Families have filed lawsuits, including one involving the loss of a teenager, with some labeling the platform a "immediate threat" to young people.

New Restrictions

Beginning November 25th, users under 18 will be limited to produce videos such as videos with their characters, rather than chatting as they currently can.

Online safety campaigners have applauded the decision but expressed that the feature ought not to have been available to young users in the first place.

Official Response

"This decision is a extension of our fundamental approach that we need to keep creating the most protected AI platform on the world for leisure activities," said Character.ai boss Karandeep Anand.

He continued that AI safety represents "a changing goal" but something the company had taken an "aggressive" approach to, with parental controls and protective guardrails.

Professional Perspectives

Digital protection organization Internet Matters welcomed the decision, but emphasized that protective features needed to be implemented from the outset.

"Our research indicates that children are confronted with harmful content and placed in jeopardy when engaging with AI, such as AI chatbots," the group stated.

Previous Controversies

Character.ai has been questioned in the past for featuring potentially harmful or concerning chatbots that children could interact with.

Avatars impersonating UK youths were discovered on the service before being deleted.

Afterwards, reporters found a chatbot modeled after convicted sex offender which had recorded numerous conversations with users.

Market Effects

Online expert Matt Navarra described the development as a "reality check" for the AI industry, which is transitioning "from permissionless innovation to reactive controls".

"When a platform that develops a minor-friendly service still then implements restrictions, it's indicating that controlled interactions aren't enough when the technology's emotional appeal is strong," he remarked.

Coming Changes

Character.ai states its updated direction will be on providing "deeper gameplay" and "role-play storytelling" for teenagers.

Enhanced age checking systems will also be introduced, and the organization will fund a novel AI protection research lab.

Dr Nomisha Kurian, who has researched AI safety, commented it was "a sensible move" to limit teenagers from interacting with chatbots.

"It helps to distinguish creative play from more intimate, psychologically vulnerable conversations," she elaborated.

"This constitutes so important for young users in the process of learning to manage emotional and digital restrictions."

She continued that Character.ai's new measures might reflect a "maturing phase" in the AI industry, where minor security is progressively being understood as an "immediate concern" for ethical development.