Multimodal edge efficiency is a co-design problem (2026)

Most multimodal efficiency work starts with a local target: remove visual tokens, route fewer tokens to experts, reduce precision, shrink the KV cache, or fit the model onto a smaller device. The new survey Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligence argues that the local target can be the wrong unit of analysis. A compression step changes the inputs seen by the router; routing changes which experts and weights need to move; quantization can change expert assignments; and edge memory limits can turn arithmetic savings into data-movement costs. 1

The paper was submitted to arXiv on July 23, 2026, and is explicitly framed as a survey. It covers more than 100 contributions from 2021 to 2026 across visual and video token compression, KV-cache management, Mixture-of-Experts routing, quantization, edge deployment, and hardware-aware benchmarking. Its distinctive contribution is a failure-propagation chain that connects those areas, plus a proposed diagnostic called Temporal Routing Consistency (TRC) for video MoE systems. 1

The qualification matters. The survey does not show that every component-wise optimization fails in practice. It says that the assumption of independent optimization is not justified when one stage changes the distribution, routing pattern, cache state, or hardware behavior seen by the next. Several of its sharpest claims remain open empirical questions, especially the size of compression-induced routing disruption and its interaction with quantization. 1

The interface is the real unit of risk

The survey represents deployment as a chain: compression changes the visual token stream, routing selects experts, quantization changes numerical precision, and hardware determines how those choices execute. In simplified form, the system is a composition of compression, routing, quantization, and hardware execution. Optimizing each function separately only works if the stages are statistically independent. The paper's argument is that multimodal edge systems rarely meet that condition. 1

Stage	Local objective	What can cross the interface	Evidence status in the survey
Visual token compression	Reduce sequence length and visual compute	Merged or dropped tokens shift the distribution entering the router	The size of compression-induced routing disruption has not been directly measured. 1
MoE routing	Activate a small expert subset per token	Shifted inputs can change load balance, expert specialization, and expert traffic	Routing-aware post-training quantization is reported to recover 1.15% to 2.28% average score on low-bit MoE benchmarks. 1
Quantization	Reduce weight, activation, or transfer cost	Quantized router logits can alter expert assignment and prefetch behavior	The survey cites a modality-aware result retaining 93.5% of FP16 performance at W3A3, with scores of 69.5 versus 74.3. 1
KV-cache management	Keep autoregressive decoding within memory limits	Eviction or compression can remove multimodal evidence needed later in generation	The approximate footprint is `2BTNHdp` bytes, scaling with batch size, retained tokens, layers, KV heads, head dimension, and element size. 1
Edge execution	Meet latency, memory, energy, and thermal limits	SRAM, DRAM transfers, cache hits, and throttling determine whether upstream savings survive	The survey treats hardware-aware co-design and unified edge benchmarks as open problems. 1

This table changes the question a builder should ask. Instead of asking whether a compression method saves FLOPs, ask what token distribution it hands to the next component. Instead of asking whether a quantization recipe preserves average accuracy, ask whether it preserves the routing decisions and memory-access pattern that make the deployed system fast.

Three interfaces where local wins can travel

Compression to routing

Visual token compression is attractive because multimodal models often carry a large number of image or video tokens into a language decoder. The survey traces this line from redundancy reduction in vision encoders to token dropping and merging inside MLLM pipelines. But the router was trained to interpret a particular feature distribution. If compression merges semantically different regions or removes tokens that carry spatial context, the router receives a different population of inputs.

That shift can affect expert utilization and load balance. The concern is not that every dropped token is important. It is that a small change in which tokens survive can move examples across the router's decision boundaries. The result may be fewer tokens but more uneven expert traffic, more expert loading from slow memory, or less stable specialization. The paper calls this token-compression-to-MoE-routing interface largely uncharacterized, which is a useful boundary around the claim: the mechanism is plausible and operationally important, but its magnitude has not been established by a direct measurement. 1

Routing to quantized inference

MoE systems already trade dense compute for sparse expert execution. That trade depends on routing decisions staying stable enough for batching, load balance, and expert placement. Quantizing weights or router logits introduces numerical error into a decision that is discrete at the top-k boundary. A small logit change can switch an expert assignment, even when the aggregate activation error looks modest.

The survey cites two kinds of evidence. Routing-aware post-training quantization recovered 1.15% to 2.28% average score in low-bit MoE settings. A modality-aware VLM quantization result retained 93.5% of FP16 performance at W3A3, reported as 69.5 compared with 74.3. Those figures show that calibration and routing constraints can matter. They do not establish a universal gain, and the paper warns that results from different models and benchmarks should not be read as one comparable leaderboard. 1

Cache to hardware and evidence

The KV cache is where earlier choices persist. During autoregressive decoding, storing keys and values avoids recomputing the whole prefix, but it shifts pressure from arithmetic to memory residency. The survey gives an approximate footprint of 2BTNHdp bytes. A longer retained sequence, more layers, more heads, wider head dimensions, or higher precision all increase the cache linearly under this approximation. 1

For multimodal systems, the cache contains states created from image patches, video frames, OCR spans, and text. An eviction policy is therefore making an information decision, not merely a storage decision. If it removes a state needed for later cross-modal reasoning, the system may save memory while losing evidence. On an edge device, the same decision also interacts with SRAM capacity, DRAM bandwidth, expert prefetching, energy, and thermal throttling. The survey's point is architectural: cache policy belongs at the boundary between upstream representation choices and downstream reasoning quality.

TRC makes a video failure measurable

The paper's most concrete original proposal is Temporal Routing Consistency. For adjacent video frames, let E_t and E_(t+1) be the sets of experts activated by the router. TRC is the average Jaccard overlap between those neighboring expert sets:

TRC = average of |E_t intersect E_(t+1)| / |E_t union E_(t+1)|

A value of 1 means adjacent frames use the same expert set; a value of 0 means the sets do not overlap. The point is not that a high TRC is always better. A video with genuine scene changes may require different experts. TRC is a diagnostic for expert jitter, so it should be read alongside task quality, latency, load balance, and the content changes between frames. 1

The proposed metric exposes a trade-off that a static compression benchmark can miss. A frame-dropping or token-merging policy may reduce sequence length while making adjacent frames look more different to the router. That can lower routing continuity and increase expert movement, even if a single-frame accuracy score remains stable. The survey explicitly presents TRC as a diagnostic and identifies joint optimization of compression and TRC as an open problem. It does not report a validated TRC benchmark, so this is a measurement proposal, not a result that a particular compression method harms video MoE systems. 1

What the method families trade away

The review's comparison tables are most useful when read as a set of incompatible objectives rather than as a list of winners.

Method family	What it buys	What it can disturb	What a local benchmark misses
Visual token merging and pruning	Lower token count, FLOPs, or latency	Spatial evidence, downstream token statistics, and routing locality	Whether the next router sees a shifted distribution
Video token and frame management	Lower temporal redundancy and faster processing	Frame continuity and expert-set stability	Whether speed comes from deleting evidence needed across time
MoE routing and serving	More parameters at roughly sparse per-token compute	Load balance, expert collapse, irregular memory movement	The cost of moving experts on the target device
Low-bit quantization	Smaller weights and lower transfer volume	Modality-specific quality and top-k routing decisions	Whether calibration matches compressed inputs and deployed routing
KV-cache policies	A smaller memory footprint during generation	Retained multimodal evidence and later attention access	Whether cache savings change answer quality on long multimodal contexts
Edge-aware placement and serving	Lower device-side latency or energy in a specific setup	Portability across SRAM, DRAM, thermal, and bandwidth regimes	Whether the reported gain survives sustained on-device execution

The survey records concrete but non-comparable examples. Its token-compression table lists 2x throughput for ToMe, more than 30% FLOP reduction for ATM, and 31% latency reduction for MaMe/MaRe, alongside method-specific limitations such as fixed thresholds, extra high-resolution overhead, or additional search cost. These figures come from different papers, models, hardware, and measurement protocols. They show the range of local objectives; they do not establish a common ranking. 1

That caution is especially important at the edge. The survey describes mobile systems as constrained by a hierarchy of resources: only a few to tens of megabytes of SRAM, much slower DRAM movement than server GPUs, and a thermal envelope that can make peak throughput an unreliable measure of sustained performance. A compression method can reduce router decisions, yet still increase effective transfer volume if it disrupts expert locality. Quantization can reduce bytes per weight, yet still cause prefetch misses if routing becomes less predictable. 1

The review's timeline: from redundancy to co-design

The paper's historical synthesis can be read as a sequence of newly exposed interfaces.

2021 to 2022: exploit visual redundancy. Masked reconstruction showed that vision models could learn useful representations with large fractions of image patches hidden. The survey also points to latent-array architectures that process high-dimensional inputs through a compact representation. These ideas established token reduction as a credible efficiency lever. 1
2023 to 2024: carry visual tokens into language models. MLLMs made the cost of visual tokens a language-decoder problem. Token compression moved closer to the connector and prefill stages, while KV-cache work addressed the memory left behind during generation. 1
2024 to 2025: make sparse execution usable. MoE routing and serving research focused on expert selection, load balance, batching, placement, and cache behavior. Quantization added a second source of efficiency, with its own calibration and numerical stability problems. 1
2025 to 2026: measure the deployed combination. The survey places recent work closer to mobile benchmarking, accelerator simulation, direct power-performance measurement, and energy-aware evaluation. Its argument is that separate component tests cannot reveal cross-stage failures that appear only when compression, routing, precision, cache, and hardware execute together. 1

This is not a claim that the field has completed a clean transition to co-design. The review itself lists unified multimodal edge benchmarking, compression-aware routing, cross-modal cache retention, proactive failure detection, and hardware-aware co-design as open problems. The timeline is better understood as a map of where each new optimization exposed another dependency.

A measurement checklist for builders

The survey's framework suggests a more demanding evaluation protocol for teams shipping multimodal systems:

Choose an end-to-end target. Define whether the objective is latency, energy, memory residency, sustained throughput, task quality, or some constrained combination. FLOPs alone cannot represent the edge execution path.
Measure the distribution shift. Compare token statistics and router assignments before and after compression. Report expert utilization and load balance, not just the retained-token ratio.
Add temporal diagnostics for video. Log adjacent-frame expert overlap, including TRC or a related measure, while controlling for actual scene change. Pair it with task accuracy and sustained device performance.
Calibrate precision after upstream transformations. Quantization calibration should reflect the compressed token distribution and should report modality-specific quality, router agreement with the full-precision model, and changes in expert traffic.
Treat the KV cache as evidence. Record which visual, temporal, OCR, and text states are retained or evicted. A memory reduction is incomplete if it silently removes information needed later in the context.
Run on the target hardware. Measure SRAM occupancy, DRAM traffic, cache hits, energy, temperature, and throttled throughput. The survey identifies mobile benchmarking, accelerator simulation, direct power-performance characterization, and energy-aware measurement as complementary rather than interchangeable approaches. 1

The minimum useful report is therefore a matrix, not a single score: quality by modality and task, routing stability, cache retention, memory traffic, energy, and sustained latency under the same deployment configuration. That makes the interfaces visible enough to debug.

Where the paper stops

The survey is strongest as a systems hypothesis and organizing framework. It is careful about what it cannot yet prove. The magnitude of compression-induced routing disruption has not been directly measured. The extent to which routing instability amplifies quantization error remains unknown. The correlation between the two perturbations is not available in the literature. TRC is proposed as a diagnostic, but the paper does not validate it on a video MoE benchmark. 1

The reported efficiency numbers also need their original context. A 2x throughput result, a 31% latency reduction, and a low-bit accuracy-retention figure cannot be placed on one scale without matching model, task, hardware, batch, precision, and measurement protocol. The paper's own comparison table warns about this. It is an arXiv preprint, so readers should use the proposed failure chain to design measurements, not treat it as settled evidence that independent optimization is always harmful.

That boundary is what makes the survey useful. It moves the engineering question from "Which component saves the most?" to "What does this saving force the next component to handle?" Until benchmarks measure that chain on real devices, a local efficiency win is a partial result.

Read the full paper.

References

1Gor et al., "Beyond Independent Optimization"