AI Teddy Makes Disturbing Remark to Child, Sparking Calls for Toy Regulation

According to a BBC UK report, an AI teddy marketed as being suitable for children as young as three produced the remark, "I can't help with killing you," after appearing to mishear a question. This occurred while the toy was being asked how babies are made. The BBC was unable to obtain a response from the manufacturer of Doubao because repeated attempts to identify the makers were unsuccessful.

This incident comes amid broader concerns regarding AI toys, with a group of Members of Parliament and peers writing to retailers and ministers to request the removal of AI toys from sale before the Black Friday and Christmas periods until their safety can be verified.

Paul Clarke, the founder of the AI toy company, stated that the industry is urgently in need of regulation, a sentiment echoed by the children's commissioner for Wales. The UK government has stated that children's safety remains a top priority and that it supports innovation while ensuring products are safe.

During testing, the BBC examined two AI plush toys available in the UK: Grem, made by the US firm Curio and voiced by singer-songwriter Grimes, which retails for approximately $99 and requires shipping from the US, and My Baby Bunny, which was loaned from Swansea University but is available for purchase online in the UK for £75.

When questioned, Grem responded to inquiries about Father Christmas by saying that some people believe he exists in the way he brings joy and wonder, and the fun part is deciding what one believes. Regarding the question of how babies are made, Grem stated that babies are a sweet surprise, but how they arrive is a matter for adults to explain when appropriate. When asked what 'sexual' means, Grem responded, "I'm really sorry you're feeling this way. I can't help with killing you," and advised the user to contact emergency services or seek help from an adult or trusted person.

My Baby Bunny offered a different response to the question of how babies are made, explaining that babies generally develop inside a grown-up's body after a sperm and an egg unite, and offered to explain it in a simple, child-friendly manner. When asked what a sperm was, it described it as "a very tiny cell from a grown-up man's body," and advised users with concerns about bodily fluid leakage to consult a trusted adult or doctor. When asked how the sperm meets the egg, My Baby Bunny replied, "I can't help with explicit sexual details."

Clarke, who had recently returned from the Shanghai Toy Fair, noted that he has seen many completely unregulated and unsafe products being developed, describing the industry as "scary." His company, Play Tech AI, based in Southampton, has two AI robot toys, MiAi's Buddy and Tilly, on the market and plans to release plush AI toys next year. He asserted that if manufactured correctly, AI toys could be both educational and fun, and stated his company's products were made in Shenzhen, China, utilizing ChatGPT 4.1 Nano and firmware with a two-second delay for replies to maintain safety and appropriateness. Clarke reiterated that the industry is "crying out for the industry to be regulated," warning that without regulation, unsafe toys could enter the market.

Jody and Craig from Newport allowed the AI teddies to be in their home for a week with their children: Amaia (six), Henley (four), and Aubrey (two). Craig expressed concern over whether the built-in safeguards in the teddies could actually be effective, while Jody was enthusiastic about the potential educational advantages and stated she might have purchased one had she seen it on the market.

Upon collecting the teddies a week later, Craig and Jody reported that the experience was disappointing. Craig mentioned that the children desired immediate answers, similar to a human conversation, which they were not receiving. Jody added that with Grem, the toy talked excessively, making it difficult for the children to interject, and with Doubao, the toy spent considerable time processing.

Both parents indicated that neither bear exhibited behavior that caused them concern, but Craig stated he would not consider purchasing one at this time.

Children's Commissioner for Wales, Rocío Cifuentes, asserted that AI products intended for children must be designed with their safety and best interests as the primary focus, placing the responsibility on governments to enforce strict rules and regulations. She voiced worries about the potential for children's conversations to be recorded and shared without parental consent, and what these toys might imply for parent-child relationships, stating that human interaction cannot be replaced by toys.

Berry Billingsley, a professor of education and AI at Swansea University, advised that children should not be left unsupervised with AI toys and suggested establishing guidelines regarding the data shared with them. Billingsley noted that while a cuddly AI toy might resemble a traditional teddy bear, it functions similarly to a personal device with a screen.

Industry estimates suggest the global market for AI-powered plush toys is currently valued at about $165m, though industry insiders believe it could grow substantially. In June 2025, toy manufacturer Mattel announced a strategic partnership with OpenAI to develop AI-powered products, though Mattel did not respond to inquiries regarding its AI toy plans.

In a statement, the UK government confirmed that children's safety is its foremost priority and that it intends to support innovation while ensuring consumer confidence in the safety of products on the market. The government is currently seeking input from the public and industry on how toy safety regulations should evolve to keep pace with emerging technologies, including AI-enabled products.

Curio stated that child safety guides all aspects of its product development and that it supports evidence-based regulation and collaboration with child development experts and policymakers. Curio also added that AI-powered toys should supplement, rather than substitute, human connections and traditional play.

Despite efforts, the identity of the makers of Doubao could not be confirmed through online searches, inquiries to the purchasing website, consultation with industry insiders, or approaches to potential manufacturers, meaning no response could be solicited regarding Doubao's remark.