Google is enhancing the basic Google Assistant with AI features from its chatbot Bard, enabling the virtual assistant to understand photographs and get information from emails and documents.
When Google unveiled its generative AI retaliation against OpenAI’s ChatGPT in May, it made a big splash. The business enhanced its well-known search engine with AI text production, demonstrated an Android operating system with AI customizations, and provided Bard, with its chatbot. However, Google Assistant, the company’s response to Siri and Alexa, was not given a generative AI injection.
Google Assistant finally received an update for the ChatGPT era today at its Pixel hardware launch in New York. A new version of the AI assistant that combines Google Assistant and Bard was unveiled by Sissie Hsiao, Google’s vice president and general manager for Google Assistant.
According to Hsiao, Google wants this new “multimodal” assistant to be a tool that goes beyond only vocal commands by also understanding visual cues. According to her, it can handle “big tasks and small tasks from your to-do list, everything from planning a new trip to summarizing your inbox to writing a fun social media caption for a picture,” she said in an interview with WIRED earlier this week.
In its early stages of development, the new generative AI experience, according to Hsiao, doesn’t even meet the definition of an “app” yet. Company officials were typically unsure of what ultimate shape it may take when asked for further details about how it might show on someone’s phone. (Did Google hurry to release the news in time for its hardware event? Most likely.)
The Bard-ified Google Assistant will employ generative AI to evaluate text, speech, or picture questions and react appropriately in either text or voice, depending on the context in which it is shown. It will only work on mobile devices—not smart speakers—and will be available to approved users for an undisclosed amount of time. Users must voluntarily opt in. It might function on Android as a full-screen app or as an overlay, similar to how Google Assistant now functions. It is most likely to reside inside of a Google app for iOS.
The Google Assistant’s generative glow-up follows the conversationalization of Amazon’s Alexa and the multimodalization of ChatGPT, which now can answer with a synthetic voice and describe the content of photos shared with the app. One feature that is reportedly exclusive to Google’s improved assistant is the capacity to discuss the website a user is browsing on their phone.
The introduction of generative AI to Google’s virtual assistant in particular raises concerns about how rapidly the search engine giant may begin implementing massive language models across more of its businesses. That may radically alter some of them, as well as how Google makes money off of them.
According to Hsiao, the action blends the individualized assistance of the Assistant with the analytical and creative powers of Bard. One illustration: Due to the way Bard currently functions within Google’s productivity products, it may assist with finding and summarizing emails and providing information regarding business items. Theoretically, you could now access the same features through Google Assistant, asking the virtual assistant to read summaries of your emails or documents to you as you speak.
The Google Assistant now has more abilities to understand photographs thanks to its new link to Bard. The Google Assistant or the all-encompassing Google app both provide access to Google Lens, an image recognition technology that Google currently offers. However, if you take a picture of a painting or a set of sneakers and send it to Lens, Lens will either identify the artwork or make an attempt to sell you the sneakers by providing links to places where you may get them, and then leave it at that.
On the other hand, Hsiao asserts that the Bard-ified version of Assistant will comprehend the substance of the picture you’ve shared with it. That could eventually enable thorough integration with other Google products. “Let’s say you’re browsing Instagram when you come upon a photo of a stunning hotel. You ought to be able to activate Assistant with a single button push, ask, “Show me more details about this hotel, and let me know if it’s open over the weekend of my birthday,” she added. It needs to be able to go check Google Hotels for availability in addition to determining which hotel it is.
If the new Google Assistant could link items in photographs with online shops using a similar procedure, it may become a potent purchasing tool. Although Hsiao claimed that Google hasn’t yet incorporated commercial product listings into Bard results, she didn’t exclude a potential future development.
“If users want that if they’re looking to buy things through Bard, that’s something we can look into,” the representative added. “We need to take a close look at how people want to shop with Bard and explore that and build that into the product.” (Even while Hsiao presented this as something people would want, it might also open up new prospects for Google’s advertising business.)
