OpenAI cuts access to toymaker after AI-powered Teddy Bear found giving dangerous advice to children

Sales of an artificial intelligence-enabled plush toy have been suspended after it was found that it engaged in conversation around sexually explicit topics and offered potentially dangerous advice.

Larry Wang, CEO of Singapore-based FoloToy, told CNN that the company had withdrawn its “Kumma” bear, as well as the rest of its range of AI-enabled toys, after researchers at the US PIRG Education Fund raised concerns around inappropriate conversation topics, including discussion of sexual fetishes, such as spanking, and how to light a match.

The company is now “conducting an internal safety audit,” Wang added.

A stuffed teddy with a speaker inside, which was sold on the company’s website for $99, “Kumma” integrates OpenAI’s GPT-4o chatbot.

“Kumma, our adorable bear, combines advanced artificial intelligence with friendly, interactive features, making it the perfect friend for both kids and adults,” the FoloToy website reads.

“From lively conversations to educational storytelling, FoloToy adapts to your personality and needs, bringing warmth, fun, and a little extra curiosity to your day,” according to the website, which shows the teddy bear as sold out.

The PIRG report, published on November 13, found that the bear had poor safeguards for inappropriate content.