News
Google and Artificial Analysis Release MKA, a 200-Task Benchmark for AI Agents Working With Visual Files
Google DeepMind and Artificial Analysis have introduced the Multimodal Knowledge Agent benchmark, which tests whether AI agents can handle blueprints, schematics, presentations and spreadsheets. According to Google, Gemini 4 Carbon ranked second on quality at an average cost six times lower than the top model.
On November 10, 2026, Google published a description of the Multimodal Knowledge Agent (MKA) benchmark, developed with Artificial Analysis. The test is meant to measure how AI agents cope with file-based work that has a complex visual layer, such as architectural blueprints, technical schematics, reports, presentations and spreadsheets.
Why another benchmark
The post's authors, Rohan Doshi and Ben Caine of Google DeepMind, argue that most knowledge-work benchmarks can be solved with text alone. MKA is meant to fill that gap with tasks that require reading blueprints, tracing schematics and producing finished documents.
The tasks were built with experts from 10 verticals and 83 occupations in which day-to-day work is inherently visual. According to Google, contributors included architects, biologists, underwriters and data scientists.
How scoring works
Each task contains a realistic prompt, a bundle of multimodal input files and a rubric written by an expert. All models run in the same open-source agent harness, Stirrup, with identical tools, sandbox and turn budget.
Results are graded by an agentic autorater built on a non-Google model. It opens each file, renders pages, inspects spreadsheet formulas and checks the work against the rubric. Domain experts who graded outputs themselves agreed with it 96% of the time, though that figure comes from Google's own materials.
Artificial Analysis runs the full 200-task set and keeps it private, so the tasks can't leak into training data or be found through search. A separate set of 10 public tasks is available on Hugging Face for inspecting the format and testing your own agents.
What the results show
According to the technical report, frontier models work in very different ways, using between 30 and 130 turns per task on average. Google says this reveals large differences in efficiency, and that strong visual perception is necessary for these tasks.
Gemini 4 Carbon ranked second on quality and, according to Google, completes tasks at an average cost six times lower than the top-scoring model. However, the post does not name the leader, give numerical scores or dollar costs in the text, or list the other models tested. The entire assessment comes from materials published by the company that sells one of the evaluated models.
What it means for companies
For teams deploying agents on document work, MKA offers an additional reference point alongside text-based tests. You can evaluate your own models on the Artificial Analysis site, and Google says Gemini 4 Carbon is available through the Gemini API in Google AI Studio. Because the author is the maker of one of the models, it is worth comparing the results with independent measurements.
