According to MS, Bing AI goes crazy because “[the team] didn’t ‘fully envision’ people using its chat interface for ‘social entertainment’ or as a tool for more ‘general discovery of the world.’” Which demonstrates either a colossal lack of vision or a colossal ignorance of people. Which, now that I think about it, is kind of on brand for most AI systems.

https://www.theverge.com/2023/2/16/23602335/microsoft-bing-ai-testing-learnings-response