"8 New AI Products and Updates Tested"

# ai

At I/O, we don't just roll out a list of updates like new Gemini models, AI agents, and Android tweaks—we actually let developers, reporters, and partners dive into the action with hands-on product demos. This year, I got to spend the day at Shoreline Amphitheatre, where I/O takes place, and I got to check out a bunch of these demos. Here's the lowdown on a few of them. My first demo of the day was pretty cool. I watched Gemini Advanced tackle a 20-plus page property lease, full of legal jargon and tricky clauses. After it processed the document, I could ask questions about it, like whether my landlord would let me have a pet dog or if there were any extra fees I'd need to pay. I'm already looking forward to using this feature to make sense of my next lease when my apartment's up for renewal. Next up, the demo got even more impressive. Two Googlers fed Gemini a PDF of a massive economics textbook—hundreds of pages long. It would've taken me ages to read through it, but Gemini whipped up a summary and pointed out the key topics to study in no time. It even threw together a multiple-choice quiz, complete with one correct answer and three tricky wrong ones, to help me prep for a hypothetical exam. Googlers Sid Lall (left) and Adam Kurzrok (right) demonstrate how Gemini Advanced can now summarize a hefty economic textbook or thousands of pages of documents. Both these demos made use of Gemini 1.5 Pro, which we introduced earlier this year with the longest context window of any large-scale foundation model. We're now rolling out early access to Gemini 1.5 Pro for Gemini Advanced subscribers, letting them upload documents directly from Drive. This means they can use Gemini to summarize or analyze documents up to 1,500 pages long. Gemini 1.5 Pro is also making its way into the side panel of Workspace apps like Gmail, Docs, Sheets, Slides, and Drive. I got to see this in action by using Gemini in Gmail to summarize a sample weekly school report email, and it pulled out specific details like activities for 7th-grade students and the packing list for an overnight trip. Gemini’s side panel can help you answer key questions about your content in Gmail, Drive and more. The improved long context window can even pull information from multiple documents when responding to a single prompt. In Docs, I asked for help writing a sample letter to a potential job candidate, linking to the job description document and the applicant’s PDF portfolio in my Drive. Gemini instantly drafted an email that took into account details from both documents. But Gemini 1.5 Pro isn't the only new kid on the block. I also got to play around with the newly announced Imagen 3, our best text-to-image model yet. One cool feature I tried out was its ability to generate decorative text and letters. I asked for a stylized alphabet, like letters made from jam on toast or silver balloons floating in the sky. Imagen 3 churned out a full alphabet of letters, which I could then use to create my own (tasty) menus. After my fun with Imagen 3, I jumped back into more Gemini demos. One demo let me pull up Gemini’s overlay on an Android phone and ask questions about anything on the screen. This showed how we're not just expanding what you can ask Gemini, but also making it context-aware to anticipate your needs and offer helpful suggestions. The demo used a lengthy oven manual as an example. Instead of slogging through the document, I pulled up Gemini and got an "Ask this PDF" suggestion right away. I asked questions like "how do I update the clock" and got quick, accurate answers. It worked just as well with YouTube videos. Instead of watching a 20-minute workout video, I asked how to modify planks and got an answer in a snap, then moved on to the next demo. This next demo introduced a new conversation mode called Gemini Live, where you can talk to Gemini in the app without typing. Talking to Gemini was a different vibe from the usual chatbot interface. Its answers were more conversational, not just paragraphs of text or bullet-pointed lists. In my demo, I asked for a list of kids' activities for a summer vacation and was able to interrupt the list to ask more about what materials I'd need for tie-dying a shirt. The Project Astra demo took things to the next level, showing where our conversational AI projects are headed. Our AI Sandbox, where developers and attendees tried out demos like Project Astra and other creative AI experiments, like MusicFX’s DJ Mode. Instead of just working with what's on your screen or what you've typed into a chat box, Astra's multimodal capabilities can understand conversational speech prompts and live video feeds at the same time, unlocking new kinds of AI experiences. Astra's alliteration demo started simple: I showed the camera an object, like a banana or a piece of bread, and Gemini came up with an alliterative sentence. As I added more objects, Gemini kept the conversation going, from "Bright bananas bask beautifully on the board" with a single fruit to "Culinary creations can catch the eye" when presented with a whole buffet board. Astra alliterates with bananas, baguettes… and anything else you can show it. Another Astra demo let me play Pictionary with Gemini. It might sound simple, but it required the agent to understand images, remember what had been drawn each round, and use general knowledge to guess what I was drawing. In one demo, Astra knew that a circle wasn't enough to base a guess on, but as I added lines underneath it, it quickly went from identifying a stick figure to recognizing that a person holding up a skull emoji was Hamlet. Astra is undefeated at Pictionary. Moving through the AI Sandbox and other demo stations felt like a peek into the future. It was also a bit humbling: Astra beat me at Pictionary in multiple rounds!

Related article

Comments (35)