MMSkills turns screenshots into reusable agent memory

MMSkills treats a visual-agent skill as multimodal procedural knowledge: a compact descriptor, a reusable text procedure, runtime state cards, and keyframe bundles. State cards say when to use a skill, what visible cues to look for, and how to verify the result. 1

An action recipe does not tell an agent whether the current screen is the right state or whether its last step worked. MMSkills branch-loads selected cards and keyframes in a temporary reasoning branch, then returns applicable, subgoal, plan, do_not_do, and verify guidance to the main agent. 2

Start with one UI workflow. The public MMSkills skill source lists 515 skills and 3,153 images across GUI and game packages. 3

In the paper's OSWorld evaluation, Qwen3-VL-8B-Instruct moved from 10.78% to 25.40% success, while Gemini 3 Flash moved from 36.65% to 47.97% with MMSkills. 1 These are paper results, not a product guarantee. The authors note dependence on source-trajectory coverage, possible skill-generation or visual-grounding errors, and extra inference cost from branch loading. 1

Read the MMSkills paper before copying the pattern into a production agent.