Gemini Unveils Enhanced Model, Extended Context, AI Agents

1.5 Flash shines in tasks like summarization, chat applications, and captioning images and videos. It's also adept at extracting data from long documents and tables. This versatility stems from being trained by 1.5 Pro through "distillation," where the core knowledge and skills of a larger model are passed down to a more efficient, smaller model.

For more details on 1.5 Flash, check out our updated Gemini 1.5 technical report, the Gemini technology page, and learn about its availability and pricing.

Significantly improving 1.5 Pro

Over the past few months, we've made significant strides in enhancing 1.5 Pro, our top performer across a wide range of tasks.

We've expanded its context window to 2 million tokens and improved its capabilities in code generation, logical reasoning, planning, multi-turn conversations, and understanding audio and images. These enhancements are backed by advances in data and algorithms, showing marked improvements on both public and internal benchmarks.

1.5 Pro now handles increasingly complex and nuanced instructions, including those that define product-level behaviors like role, format, and style. We've refined control over the model's responses for specific use cases, such as customizing chat agent personas or automating workflows with multiple function calls. Users can now steer the model's behavior with system instructions.

We've also added audio understanding to the Gemini API and Google AI Studio, allowing 1.5 Pro to process both images and audio from videos uploaded to Google AI Studio. We're integrating 1.5 Pro into Google products like Gemini Advanced and Workspace apps.

For more on 1.5 Pro, dive into our updated Gemini 1.5 technical report and the Gemini technology page.

Gemini Nano understands multimodal inputs

Gemini Nano is stepping up its game, moving beyond text-only inputs to include images. Starting with Pixel, apps using Gemini Nano with Multimodality will be able to interpret the world in a more human-like way, through text, visuals, sound, and spoken language.

Learn more about Gemini 1.0 Nano on Android.

Next generation of open models

Today, we're also updating Gemma, our family of open models, which are built on the same research and tech as the Gemini models.

We're launching Gemma 2, our next-gen open models for responsible AI innovation. Gemma 2 features a new architecture for superior performance and efficiency, and will come in new sizes.

The Gemma family is growing with PaliGemma, our first vision-language model inspired by PaLI-3. We've also upgraded our Responsible Generative AI Toolkit with LLM Comparator to assess model response quality.

For more details, head over to the Developer blog.

Progress developing universal AI agents

At Google DeepMind, our mission is to build AI responsibly to benefit humanity. We've always aimed to create universal AI agents that can assist in everyday life. That's why we're sharing our progress on the future of AI assistants with Project Astra (advanced seeing and talking responsive agent).

For an AI agent to be truly helpful, it needs to understand and react to the world like a human, taking in and remembering what it sees and hears to grasp context and act accordingly. It should also be proactive, teachable, and personal, allowing for natural, lag-free conversations.

While we've made great strides in processing multimodal information, achieving conversational response times is a tough engineering challenge. Over the years, we've been refining how our models perceive, reason, and converse to make interactions feel more natural.