Gemini Gains Agentic Video Understanding, Cuts Token Use by Up to 88 Percent

Google DeepMind rolled out a new feature in Gemini on September 1, 2026, that changes how the model processes video material. Instead of watching every clip in the same rigid way, agentic video understanding lets the model decide for itself which parts of a recording matter, how often to sample them, and whether to draw on the visual track, the audio track, or a transcript.

The project is led by Rohan Doshi, senior product manager at Google DeepMind, and Mario Lučić, director of research at the same organization. In describing the feature, they note that the previous approach to video analysis treated every file the same way, whether it was a clip a few seconds long or a multi-hour conference recording.

How the new mode works

In the classic mode, Gemini analyzes video frame by frame at a fixed pace, meaning the model spends as much compute on static, irrelevant footage as on key moments of action. The agentic mode flips that logic: the model first drafts an analysis plan tailored to the user's specific question, then independently reaches for tools that load only the segments of material it actually needs.

In practice, this means that when asked about a specific moment in a multi-hour recording, the model no longer needs to process the entire file from start to finish. It can jump to the likely time window, raise the frame sampling rate there, and fall back to the audio track or text transcript for the remaining stretches.

Turning on the new mode requires a single change to the API call: the processing parameter on the video input has to be set to agentic instead of the default, static value. From there, the model itself handles the rest of the logic, including which tools to use and how fast to sample.

The numbers

Google DeepMind says that in internal testing, the new mode cut video analysis costs by up to 66 percent and total token usage by up to 88 percent compared with classic, static processing. At the same time, the model's answer accuracy rose by up to 7 percent, which the company attributes to the model devoting more attention to the segments that actually matter for the question asked, instead of spreading resources evenly across the whole file.

According to Google, Gemini 3.7 Flash with agentic mode enabled delivers the best accuracy-to-cost ratio among the configurations tested for video understanding. That is a meaningful shift for companies that previously had to choose between cheaper but less precise processing and more expensive, full frame-by-frame analysis.

Real-world uses

Google points to several specific scenarios where the new mode is expected to perform best. The first is precise moment-finding within footage, useful for automated video editing where accuracy down to a fraction of a second matters. The second is searching long recordings for a single, hard-to-find segment, such as a specific remark in a multi-hour webinar.

The third scenario is anomaly detection, where the model dynamically adjusts its sampling rate based on how quickly a scene changes, and the fourth is counting objects or repeated actions over time, for example in footage from a production line or surveillance system. The feature builds on Agentic Vision, which Google introduced for Gemini 3 Flash in January 2026 and which let the model actively manipulate individual images, cropping and annotating them. The new capability extends that same approach from single images to video material spread out over time.

What's next

Google says agentic video understanding will eventually make its way into the consumer Gemini app as well, and in the coming months is expected to be integrated with the Ask YouTube feature, which lets users ask questions about video content on the platform. For now, the feature is available only through the developer API, making it a tool primarily for developers and companies building their own applications on top of Gemini.

For companies that process large volumes of footage, such as those in media, e-commerce, or security, cutting video analysis costs by tens of percent could meaningfully change the economics of deploying such systems. Long video files have until now ranked among the most expensive data types for language models to process, precisely because of rigid, content-independent frame sampling.