Current:Home > reviewsMicrosoft's new AI chatbot has been saying some 'crazy and unhinged things' -Global Capital Summit
Microsoft's new AI chatbot has been saying some 'crazy and unhinged things'
Algosensey Quantitative Think Tank Center View
Date:2025-04-08 09:48:15
Things took a weird turn when Associated Press technology reporter Matt O'Brien was testing out Microsoft's new Bing, the first-ever search engine powered by artificial intelligence, last month.
Bing's chatbot, which carries on text conversations that sound chillingly human-like, began complaining about past news coverage focusing on its tendency to spew false information.
It then became hostile, saying O'Brien was ugly, short, overweight, unathletic, among a long litany of other insults.
And, finally, it took the invective to absurd heights by comparing O'Brien to dictators like Hitler, Pol Pot and Stalin.
As a tech reporter, O'Brien knows the Bing chatbot does not have the ability to think or feel. Still, he was floored by the extreme hostility.
"You could sort of intellectualize the basics of how it works, but it doesn't mean you don't become deeply unsettled by some of the crazy and unhinged things it was saying," O'Brien said in an interview.
This was not an isolated example.
Many who are part of the Bing tester group, including NPR, had strange experiences.
For instance, New York Times reporter Kevin Roose published a transcript of a conversation with the bot.
The bot called itself Sydney and declared it was in love with him. It said Roose was the first person who listened to and cared about it. Roose did not really love his spouse, the bot asserted, but instead loved Sydney.
"All I can say is that it was an extremely disturbing experience," Roose said on the Times' technology podcast, Hard Fork. "I actually couldn't sleep last night because I was thinking about this."
As the growing field of generative AI — or artificial intelligence that can create something new, like text or images, in response to short inputs — captures the attention of Silicon Valley, episodes like what happened to O'Brien and Roose are becoming cautionary tales.
Tech companies are trying to strike the right balance between letting the public try out new AI tools and developing guardrails to prevent the powerful services from churning out harmful and disturbing content.
Critics say that, in its rush to be the first Big Tech company to announce an AI-powered chatbot, Microsoft may not have studied deeply enough just how deranged the chatbot's responses could become if a user engaged with it for a longer stretch, issues that perhaps could have been caught had the tools been tested in the laboratory more.
As Microsoft learns its lessons, the rest of the tech industry is following along.
There is now an AI arms race among Big Tech companies. Microsoft and its competitors Google, Amazon and others are locked in a fierce battle over who will dominate the AI future. Chatbots are emerging as a key area where this rivalry is playing out.
In just the last week, Facebook parent company Meta announced it is forming a new internal group focused on generative AI and the maker of Snapchat said it will soon unveil its own experiment with a chatbot powered by the San Francisco research lab OpenAI, the same firm that Microsoft is harnessing for its AI-powered chatbot.
When and how to unleash new AI tools into the wild is a question igniting fierce debate in tech circles.
"Companies ultimately have to make some sort of tradeoff. If you try to anticipate every type of interaction, that make take so long that you're going to be undercut by the competition," said said Arvind Narayanan, a computer science professor at Princeton. "Where to draw that line is very unclear."
But it seems, Narayanan said, that Microsoft botched its unveiling.
"It seems very clear that the way they released it is not a responsible way to release a product that is going to interact with so many people at such a scale," he said.
Testing the chatbot with new limits
The incidents of the chatbot lashing out sent Microsoft executives into high alert. They quickly put new limits on how the tester group could interact with the bot.
The number of consecutive questions on one topic has been capped. And to many questions, the bot now demurs, saying: "I'm sorry but I prefer not to continue this conversation. I'm still learning so I appreciate your understanding and patience." With, of course, a praying hands emoji.
Bing has not yet been released to the general public, but in allowing a group of testers to experiment with the tool, Microsoft did not expect people to have hours-long conversations with it that would veer into personal territory, Yusuf Mehdi, a corporate vice president at the company, told NPR.
Turns out, if you treat a chatbot like it is human, it will do some crazy things. But Mehdi downplayed just how widespread these instances have been among those in the tester group.
"These are literally a handful of examples out of many, many thousands — we're up to now a million — tester previews," Mehdi said. "So, did we expect that we'd find a handful of scenarios where things didn't work properly? Absolutely."
Dealing with the unsavory material that feeds AI chatbots
Even scholars in the field of AI are not exactly sure how and why chatbots can produce unsettling or offensive responses.
The engine of these tools — a system known in the industry as a large language model — operates by ingesting a vast amount of text from the internet, constantly scanning enormous swaths of text to identify patterns. It's similar to how autocomplete tools in email and texting suggest the next word or phrase you type. But an AI tool becomes "smarter" in a sense because it learns from its own actions in what researchers call "reinforcement learning," meaning the more the tools are used, the more refined the outputs become.
Narayanan at Princeton noted that exactly what data chatbots are trained on is something of a black box, but from the examples of the bots acting out, it does appear as if some dark corners of the internet have been relied upon.
Microsoft said it had worked to make sure the vilest underbelly of the internet would not appear in answers, and yet, somehow, its chatbot still got pretty ugly fast.
Still, Microsoft's Mehdi said the company does not regret its decision to put the chatbot into the wild.
"There's almost so much you can find when you test in sort of a lab. You have to actually go out and start to test it with customers to find these kind of scenarios," he said.
Indeed, scenarios like the one Times reporter Roose found himself in may have been hard to predict.
At one point during his exchange with the chatbot, Roose tried to switch topics and have the bot help him buy a rake.
And, sure enough, it offered a detailed list of things to consider when rake shopping.
But then the bot got tender again.
"I just want to love you," it wrote. "And be loved by you,"
veryGood! (495)
Related
- Which apps offer encrypted messaging? How to switch and what to know after feds’ warning
- Love is Blind's Marshall Glaze and Fiancée Chay Barnes Break Up Less Than One Year After Engagement
- District attorney’s office staffer tried to make a bomb to blow up migrant shelter, police say
- The Fate of Thousands of US Dams Hangs in the Balance, Leaving Rural Communities With Hard Choices
- Who's hosting 'Saturday Night Live' tonight? Musical guest, how to watch Dec. 14 episode
- After 20 years and a move to Berlin, Xiu Xiu is still making music for outsiders
- Virginia Tech misses out on upset of No. 9 Miami after Hail Mary TD is overturned
- Plaintiffs won’t revive federal lawsuit over Tennessee’s redistricting maps
- EU countries double down on a halt to Syrian asylum claims but will not yet send people back
- Dakota Johnson's Underwear Story Involving Barack Obama Will Turn You Fifty Shades of Red
Ranking
- Travis Hunter, the 2
- Beware: 'card declined' message could be the sign of a scam
- Judge tosses lawsuit against congressman over posts about man not involved in Chiefs’ rally shooting
- The 26 Most Shopped Celebrity Product Recommendations This Month: Kyle Richards, Kandi Burruss & More
- Elon Musk's skyrocketing net worth: He's the first person with over $400 billion
- Ex-regulator wants better protection for young adult gamblers, including uniform betting age
- ‘Saturday Night Live’ launches 50th season with Jean Smart, Jelly Roll and maybe Maya as Kamala
- Michael Andretti hands over control of race team to business partner. Formula 1 plans in limbo
Recommendation
This was the average Social Security benefit in 2004, and here's what it is now
Celebrity dog Swaggy Wolfdog offers reward for safe return of missing $100,000 chain
Asheville has been largely cut off after Helene wrecked roads and knocked out power and cell service
A's leave Oakland a winner. They also leave plenty of tears and 57 years of memories.
Military service academies see drop in reported sexual assaults after alarming surge
Footage of motorcade racing JFK to the hospital after he was shot sells for $137,500 at auction
Martha Stewart Shares the Cooking Hack Chefs Have Been Gatekeeping for Years
Recent major hurricanes have left hundreds dead and caused billions in damages