Google's Gemini Omni Generates Video from Images, Audio, and Text

Three years ago, Google introduced Gemini with the aim of developing a multimodal large language model—a unified neural network trained on text, images, audio, and video, capable of generating content across all these formats.

At its Google I/O developer conference today, the company advanced toward this vision with Gemini Omni, a new family of multimodal models. Google CEO Sundar Pichai stated that Omni will empower users to "create anything from any input."

Omni's initial focus is video. Users can now combine images, audio, video, and text. Instead of merely stitching these elements together, Omni intelligently reasons across all modalities to produce a coherent output. This results in high-quality videos that demonstrate an understanding of physics, culture, history, and science.

Omni also enables users to edit photos using simple text commands, eliminating the need for complex software, similar to Google's Nano Banana tool.

Google already offers Veo, a dedicated video model that transforms text and images into videos and allows for directing and customizing avatars. However, Nicole Brichtova, Director of Product Management at Google DeepMind, emphasized that today's release represents more than just a Veo update: "It's the next step in merging Gemini's intelligence with the rendering capabilities of our media models."

During a media briefing on Monday, DeepMind's Chief Technologist Koray Kavukcuoglu provided an example: When prompted with "a claymation explainer of protein folding," Omni quickly generated a stop-motion video with a voiceover explaining, "Proteins begin as chains of amino acids. They fold into structures like alpha helices and flat sections called beta sheets, ultimately forming a precise three-dimensional shape."

The long-term vision for Omni is broader, encompassing capabilities like generating images from audio or audio from video.

"When we first announced Gemini, it was our first natively multimodal AI model," Pichai remarked during the briefing. "We knew training it on a combination of text, code, audio, images, and video would lead to a deeper understanding of the world. With world models, AI is evolving from predicting text to simulating reality. Gemini Omni is the next step in that direction."

As part of this release, users will also be able to create videos featuring their own digital avatars—a feature popularized by OpenAI's now-discontinued Sora app with Cameos. To prevent deepfakes, users must complete a dedicated onboarding process, which involves recording themselves while speaking a series of numbers, according to Brichtova. The avatar is then saved for future use.

Additionally, all videos created with Omni will include Google's SynthID digital watermark, allowing users to verify if content was generated using Gemini products.

The first model in the family is Gemini Omni Flash, launching today on the Gemini app, YouTube Shorts, and the AI creative studio Flow. Flash can render 10-second videos. Brichtova clarified that this duration is not a model limitation but a strategic decision to broaden accessibility, anticipating that most users currently prefer shorter clips. Support for longer videos is planned for the near future.

Google appears to be positioning Omni Flash primarily as a consumer tool. During a call with TechCrunch, Brichtova and DeepMind research engineer Gabe Barth-Maron described avatar use cases as personal, such as creating a video of yourself winning an award or visiting the moon, or removing a bystander from a vacation video background.

Barth-Maron summarized it succinctly: "They're like personalized memes."

"We definitely focused on making this easy for consumers to use," Brichtova said. "Not many video models have successfully crossed over to the mainstream consumer market, so this is our attempt to do that."

This ease of use comes with a caveat: Brichtova and Barth-Maron noted that editing prompts must be highly specific. Otherwise, Omni might over-edit or unintentionally alter elements the user intended to keep—a challenge also faced by Nano Banana users.

Image Credits:Google

Despite its immediate consumer focus, Omni's potential for enterprise and creative applications is evident. Google will make Omni available via API in the coming weeks. The avatar-generation tool—already available on Shorts—is expected to gain traction among content creators. More broadly, an end-to-end multimodal workflow could revolutionize advertising and filmmaking.

Startup Luma AI is developing a similar agentic tool powered by its own "unified" model, capable of generating an entire ad campaign from a brief and a product image.

"We're actually quite proud of the model's text-rendering capabilities, which are very useful for applications like advertising," Brichtova said. "If you need a product placement or even just a slogan, the accuracy is crucial... We certainly anticipate filmmakers and other creators will adopt this model as well."

More professional use cases may be better served by the upcoming Omni Pro model, designed to deliver superior performance across all Omni tasks. Google has not announced a release date for Pro yet, but Brichtova indicated it will launch when "we achieve a significant leap in capability beyond Flash."