AI chatbots can make your Cyber Monday shopping a breeze, but they’re not foolproof

The technology is still in its infancy.

Tired of thinking about what gifts to get everyone this year? Artificial intelligence chatbots might help, but don’t expect them to do all the work or always give you the right answers.

Anyone scouring the internet for Cyber Monday deals is likely going to encounter more conversational iterations of the chatbots that some retailers and e-commerce sites have built to provide shoppers with enhanced customer service.

Some companies have integrated models infused with newer generative AI technologies, allowing shoppers to seek advice by asking naturally phrased questions like “What’s the best wireless speaker?”

Retailers hope consumers use these chatbots, which are typically called shopping assistants—as virtual companions that help them discover or compare products. Prior chatbots were mostly used for task-oriented functions such as helping customers track down online orders or return ones that didn’t meet expectations.

Amazon, the king of online retail, has said its customers have been questioning Rufus—the generative AI-powered shopping assistant it launched this year—for information such as whether a specific coffee maker is easy to clean, or what recommendations it has for a lawn game for a child’s birthday party.

And Rufus, which is available for holiday shoppers in the U.S. and some other countries, is not the only shopping assistant out there. A select number of Walmart shoppers will have access this year to a similar chatbot the nation’s largest retailer is testing in a few product categories, including toys and electronics.

Perplexity AI added something new to the AI chat-shopping world last month by rolling out a feature on its AI-powered search engine that enables users to ask a question like “What’s the best women’s leather boots?” and then receive specific product results that the San Francisco-based company says are not sponsored.

“It has been adopted at pretty incredible scale,” Mike Mallazzo, an analyst and writer at retail research media company Future Commerce, said.

Retailers with websites and e-commerce companies started paying more attention to chatbots when use of ChatGPT, an artificial intelligence text chatbot made by the company OpenAI, went mainstream in late 2022, sparking public and business interest in the generative AI technology that powers the tool.

Victoria’s Secret, IKEA, Instacart, and the Canadian retailer Ssense are among other companies experimenting with chatbots, some of which use technology from OpenAI.

Even before the improved chatbots, online retailers were creating product recommendations based on a customer’s prior purchases or search history. Amazon was at the forefront of having recommendations on its platform, so Rufus’ ability to provide some is not particularly groundbreaking.

But Rajiv Mehta, the vice president of search and conversational shopping at Amazon, said the company is able to offer more helpful recommendations now by programming Rufus to ask clarifying or follow-up questions. Customers are also using Rufus to look for deals, some of which are personalized, Mehta said.

To be sure, chatbots are prone to hallucinations, so Rufus and most of the tools like it can get things wrong.

Juozas Kaziukenas, founder of e-commerce intelligence firm Marketplace Pulse, wrote in a November blog post that his firm tested Rufus by requesting gaming TV recommendations. The chatbot’s response included products that were not TVs. When asked for the least expensive options, Rufus came back with suggestions that weren’t the cheapest, Kaziukenas said.

An Associated Press reporter recently asked Rufus to give some gift recommendations for a brother. The chatbot quickly spit out a few ideas for “thoughtful gifts,” ranging from a T-shirt and a keychain with charms to a bolder suggestion: a multifunctional knife engraved with the phrase “BEST BROTHER EVER.”

After a 5-minute written conversation, Rufus offered more tailored suggestions—a few Barcelona soccer jerseys sold by third-party sellers. But it wasn’t able to say which seller offered the lowest price. When asked during another search for a price comparison on a popular skin serum, Rufus showed the product’s pre-discounted price instead of its present one.

“Rufus is constantly learning,” Amazon’s Mehta said during an interview.

Shop AI, a chatbot that Canadian e-commerce company Shopify launched last year, can also help shoppers discover new products by asking its own questions, such as soliciting details about an intended gift recipient or features the buyer wants to avoid. Shop AI has trouble, however, recommending specific products or identifying the lowest-priced item in a product category.

The limitations show the technology is still in its infancy and has a long way to go before it becomes as useful as the retail industry—and many shoppers—wish it could be.

To truly transform the shopping experience, shopping assistants will “need to be deeply personalized” and be able—on their own—to remember a customer’s order history, product preferences and purchasing habits, consulting giant McKinsey & Company said in an August report. the McKinsey report said.

Amazon has noted that Rufus’ answers are based on information contained in product listings, community Q&As and customer reviews, which would include the fake reviews that are used to boost or diminish sales for products on its marketplace.

The large language model that powers the chatbot was also trained on the company’s entire catalog and some public information on the web, Trishul Chilimbi, an Amazon vice president who oversees AI research, wrote in the electrical engineering magazine IEEE Spectrum in October.

But its unclear how Amazon and other companies are weighting different training components—such as reviews—in their recommendations, or how exactly the shopping assistants come up with them, according to Nicole Greene, an analyst at management consulting firm Gartner.

Perplexity AI’s new shopping feature allows users to enter search queries such as “best phone case” and to receive answers derived from various sources, including Amazon and other retailers, such as Best Buy. Perplexity also invited retailers to share data about their products and said those that do would have an increased chance of having their items recommended to shoppers.

But Perplexity CEO Aravind Srinivas, suggested in a recent interview with Fortune magazine that he didn’t know how the new shopping feature recommended products to customers. But in an interview with the AP, Chief Business Officer Dmitry Shevelenko pushed back on that characterization, saying Srinivas’s comment “was probably taken out of context.”

The context, he said, is that with generative AI technology “You can’t know in advance exactly what the output will be just based off of knowing what the inputs” are from the training materials.

Shevelenko said retailers and brands need to know they can’t have their products recommended in Perplexity’s search engine because they’re “jamming key words” into their websites or using different techniques to show up better on search results.

“The way you show up in an answer is by having a better product and better features,” he said.