Sparse paths, dense evidence: nine top-conf papers from August 31-September 4, 2026
Nine first-submission arXiv papers from August 31-September 4, 2026 cover efficient LLM training and inference, multimodal reasoning, collaborative perception, agent evaluation, memory safety, and KAN tooling.
The papers in this issue were first submitted to arXiv from August 31 through September 4, 2026. I kept papers whose title or submission record names NeurIPS, ICML, ICLR, or CVPR, and excluded workshop-only entries. A confirmed acceptance or proceedings record is labeled as such; a bare conference name or an "under review" note remains preprint.
| Paper | Area | Status | Reading signal |
|---|---|---|---|
| Don't Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference | LLM training and inference | Published at ICML 2026 | Structured layer dropout reaches up to 25% training-FLOPs savings and up to 1.55x speculative-decoding speedup. 1 |
| BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference | LLM inference | Preprint; the record names ICML 2026 | Beacon queries preserve long-range reasoning signals while cutting peak memory by up to 5.8x. 2 |
| Faster Than Flash: Exploiting Attention Sparsity for Efficient Long-Context Decoding | LLM inference systems | Accepted at ICML 2026 | Quantized cache scanning and fused kernels reach up to 2.37x end-to-end speedup over FlashAttention-2 on RTX 4090. 3 |
| CauseCollab: Causal Unified and Modality-Agnostic Network for Heterogeneous Collaborative Perception | Collaborative perception | Accepted at ICML 2026 | A causal protocol converter transfers across sensor modalities with a 0.68 MB adapter. 4 |
| Position: Unlabeled IS NOT Equal to No Human Supervision in Visual Learning | Vision methodology | Preprint; the record names ICML 2026 | A 17,435-paper analysis asks visual-learning papers to disclose the human choices inside "unlabeled" pipelines. 5 |
| DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents | Multimodal evaluation | Accepted by ICML 2026 | Humans score 93.29% while the best tested model reaches 62.83% on chart-context reasoning. 6 |
| Improving Evaluation Realism with Inference-Time Compute and Deployment Scaffolds | Agent evaluation | Preprint; under review at NeurIPS 2026 | Critique refinement and deployment-like scaffolds raise audit realism, while seed adherence and cost remain constraints. 7 |
| CAPTURE: Disentangling Preference Drift from Memory Poisoning in Personalized LLM Agents | Agent memory and safety | Preprint; under review at ICLR 2027 | The memory gate lowers poisoning on a synthetic benchmark, but an adaptive attacker raises attack success to 24.7%. 8 |
| InKAN: B-Spline KANs via Truncated Power Form | Neural-network tooling | Preprint; under review at ICLR 2027 | A closed-form cubic B-spline evaluation removes the recursive basis calculation that takes 91% of a profiled KAN layer. 9 |
LLM efficiency and inference
1. Don't Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference
Area tag: LLM training and inference efficiency
arXiv: 2609.05275v1, first submitted September 4, 2026. 1
Authors / institutions: Mostafa Elhoushi, Nolan Dey, Bin Claire Zhang, Gavia Gray, Gurpreet Gosal, Shane Bergsma, and Joel Hestness are affiliated with Cerebras Systems. Alex Pretko and Abdulrahman Mahmoud are affiliated with MBZUAI. 10
Status: Published at ICML 2026. The arXiv record calls the upload a slightly extended version of the ICML paper and lists the ICML proceedings reference. 1
Problem: Layer dropout removes whole transformer blocks during training, but the layer density, dropout schedule, and masking granularity determine whether the saved computation harms the dense model. The paper asks how to choose those variables so one training run can also support cheaper inference paths.
Method: The authors study structured, depth-wise dropout in decoder-only transformers. The recommended configuration combines an Increasing Linear Distribution across depth with a Decreasing Time Schedule over training steps. The training-to-evaluation scaling uses
r_train = 1/rho, where rho is the remaining layer density, and evaluates the trained model with all layers or with selected layers skipped. 10Comparison: The experiments compare layer-level with sub-layer dropout, per-sequence with per-batch masks, uniform with non-uniform depth distributions, and constant with increasing or decreasing schedules. Dense models trained without dropout provide the main baseline. The study covers more than 2,400 runs, model sizes from 271M to 8.2B parameters, and datasets of up to 160B tokens on Cerebras CS-3 systems. 10
Results: Properly configured layer dropout saves up to 25% of training FLOPs at a fixed number of steps. In the reported 8.2B run, a maximum dropout setting of 0.99 with the recommended schedule saves 25% of FLOPs and reaches 1.55x speculative-decoding speedup. In the 3.9B run, a 20% FLOPs saving produces 1.54x speculative decoding speedup. The paper also reports early-exit and intermediate-layer-skipping gains, while validation loss remains close to the dense baseline. 10
Resources: The paper does not list an official code release. It links external implementations of related dropout mechanisms for comparison. 10
Takeaway: Open this paper when training cost and inference depth are part of the same LLM design. The evidence is broad within transformer-level dropout, while MoE, non-transformer architectures, learned depth methods, and higher token-per-parameter regimes remain outside the study.
2. BeaconKV: Key-Value Cache Compression Guided by Beacon Queries for Efficient Large Reasoning Model Inference
Area tag: Long-chain-of-thought inference and KV-cache compression
arXiv: 2609.04971v1, first submitted September 4, 2026. 2
Authors / institutions: Janghyeon Kim, Minsoo Kim, and Jungwook Choi are affiliated with Hanyang University. Kyuhong Shim is affiliated with Sungkyunkwan University. 11
Status: Preprint. The arXiv comment names ICML 2026 and links a code repository, without stating acceptance or publication. 2
Problem: Long reasoning traces repeatedly revisit earlier planning tokens. A cache policy that keeps only recent tokens can therefore remove information that the decoder needs several thousand steps later. BeaconKV asks whether a small set of representative queries can identify those long-range key-value pairs without storing the full cache.
Method: BeaconKV clusters past queries and retains compact "beacon queries" selected with Continual Farthest Point Sampling. The method aligns beacon queries with the current decoding position, keeps recent queries in their original positions, and scores cached key-value pairs by the maximum attention over observation queries and heads. The final cache combines a recent or prefix window with the top-ranked long-range pairs. 11
Comparison: The paper compares BeaconKV with RPC, R-KV, SnapKV, Initial+Recent, and the full KV cache on R1-Distill-Qwen-7B, R1-Distill-Llama-8B, Qwen3-4B, and Qwen3-14B. The task set includes AIME24, MATH-500, LiveCodeBench, and GPQA-Diamond. 11
Results: The largest reported gain over existing compressed-cache methods is 31.7 percentage points on Qwen3-14B at an AIME24 KV budget of 1,024. Peak GPU memory falls by up to 5.8x, and throughput improves by more than 4.3x over the uncompressed baseline. On Qwen3-4B with 32K generation, a 2K BeaconKV budget uses 13.3 GB and reaches 356.4 tokens per second, compared with 77.0 GB and 82.3 tokens per second for the full cache; LiveCodeBench accuracy is 51.1 versus 54.4. 11
Resources: The arXiv comment links the BeaconKV code repository. The paper's HTML text does not describe a separate release protocol. 2
Loading content card…
Takeaway: BeaconKV is the paper to open when long-context inference is memory-bound and the workload contains extended reasoning. The reported scope is narrower than general long-context generation: adaptive cache budgets and non-reasoning tasks remain open.
3. Faster Than Flash: Exploiting Attention Sparsity for Efficient Long-Context Decoding
Area tag: Sparse attention kernels and long-context decoding
arXiv: 2609.00097v1, first submitted August 31, 2026. 3
Authors / institutions: Zhigeng Liu, Zhiyuan Ning, Ruixiao Li, Xiaoran Liu, Yuerong Song, Min Zhang, Ziwei He, and Xipeng Qiu are affiliated with Fudan University, Shanghai Innovation Institute, or Harbin Institute of Technology. 12
Status: Accepted at ICML 2026, according to the arXiv record. 3
Problem: Long-context decoding can skip many attention positions, yet selecting the useful positions can itself erase the speed gained by sparse attention. The paper targets the selector, memory movement, and kernel launch overhead together.
Method: Faster Flash Decoding splits the key cache into a 2-bit quantized thumbnail and an 8-bit residual. A top-delta rule estimates a pseudo-maximum from sink and local tokens, then filters attention blocks without global synchronization. A fused selector-computer Triton kernel reuses the selector's scan, and CUDA graph capture covers the execution chain. 12
Comparison: The baselines are FlashAttention-2, Quest, KIVI, Twilight, and StreamingLLM. The evaluation uses RULER at 32K and LongBench on NVIDIA H100 and RTX 4090 hardware, with additional comparisons against top-k and top-p selection rules. 12
Results: On RTX 4090, the fused kernels reach an average 7.33x speedup at delta=5 and 6.18x at delta=7, with a peak of 11.63x. End-to-end throughput reaches 61.0 tokens per second at 4K and 51.8 at 16K, up to 2.37x over FlashAttention-2. On Llama-3.1-8B and RULER 32K, the delta=7 variant scores 89.4 versus 90.6 for the dense base, while LongBench reaches 26.35 versus 26.22 for the base. Average sparsity is 82% at delta=5 and 73% at delta=7. 12
Resources: The authors provide the Faster Flash Decoding repository. 12
Loading content card…
Takeaway: This paper is relevant when sparse attention already works algorithmically and kernel overhead is the remaining bottleneck. The experiments cover Llama and Qwen models with grouped-query attention; MLA-style architectures are outside the reported validation.
Vision and multimodal reasoning
4. CauseCollab: Causal Unified and Modality-Agnostic Network for Heterogeneous Collaborative Perception
Area tag: Multi-agent perception and sensor heterogeneity
arXiv: 2609.03818v1, first submitted September 3, 2026. 4
Authors / institutions: Weize Li, Yang Li, Quan Yuan, Xiaoyuan Fu, Guiyang Luo, and Jinglin Li are affiliated with the State Key Laboratory of Networking and Switching Technology at Beijing University of Posts and Telecommunications. 13
Status: Accepted at ICML 2026. 4
Problem: Collaborative perception systems often exchange features from different sensor protocols. Modality-specific statistics can then look like semantic differences, making a shared fusion module brittle when a new vehicle or sensor arrives.
Method: CauseCollab uses a two-stage pipeline. A Semantic Context Extractor and Context-Guided Dynamic Refiner first convert heterogeneous features into a shared protocol representation. A structural causal model and mask-guided intervention separate semantic factors from modality-specific confounders. Lightweight adapters then reconstruct local modality features for downstream detection; adding a new modality requires 0.68 MB of adapter parameters while the converter stays frozen. 13
Comparison: The paper compares MPDA, PnPDA, HEAL, STAMP, and NegoCollab on OPV2V and the real-world DAIR-V2X dataset. Ablations remove the causal intervention, mask generator, semantic context, dynamic refiner, and lightweight adapters one at a time. 13
Results: On OPV2V, selected modality pairings reach AP@30/AP@50 of 0.978/0.975 for the general heterogeneous setting and 0.983/0.980 for one listed pairing. On DAIR-V2X, the reported Lpp4 + CEff pairing reaches 0.795/0.733. The paper reports lower scores for some reversed or larger modality gaps, including 0.647/0.464 for CEff + Lpp4 on DAIR-V2X. 13
Resources: The paper does not list a public code repository. 13
Takeaway: CauseCollab is a useful read for teams adding sensor types to an existing collaborative-perception deployment. The asymmetric DAIR-V2X pair results matter as much as the best score: the shared representation reduces modality friction, but it does not erase every direction-dependent failure.
5. Position: Unlabeled IS NOT Equal to No Human Supervision in Visual Learning
Area tag: Vision methodology and dataset governance
arXiv: 2609.03077v1, first submitted September 2, 2026. 5
Authors / institutions: Dong Lao is affiliated with the Division of Computer Science and Engineering at Louisiana State University. 14
Status: Preprint. The arXiv comment says ICML 2026 without an acceptance or proceedings statement. 5
Problem: Visual-learning papers often call a pipeline unlabeled or self-supervised because the target dataset has no human annotations. The paper argues that human choices still enter through data curation, pretrained backbones, augmentation, objective design, and the selection of invariances.
Method: This position paper proposes a disclosure checklist rather than a new training algorithm. The checklist asks whether a method depends on a pretrained model, whether the backbone is frozen or adapted, what the pretraining data covers, which distributional priors enter the pipeline, what invariances are imposed, and where evaluation leakage may occur. It also treats assumption relaxation as a research contribution alongside benchmark performance. 14
Comparison: The analysis covers CVPR, ICCV, and ECCV titles from 2013 through 2025, full-text extraction for 17,435 CVPR papers from 2015 through 2025, and manual inspection of 287 CVPR papers with "unsupervised" in the title. A Qwen-2.5-32B-Instruct extractor is evaluated against the manual annotations. 14
Results: Title use of "unsupervised" peaked at about 4% of CVPR papers in 2021 and fell to about 1% by 2025. Full-text identification of un/self-supervised work rose from 10.13% in 2015 to 22.17% in 2021, then fell to 12.85% in 2025. The extractor reaches 91.5% precision, 81.8% recall, and 86.9% accuracy against the manual labels. Among papers with "unsupervised" in the title, reliance on pretrained features has increased consistently since 2020. 14
Resources: The paper does not provide code or an implementation. 14
Takeaway: Open this paper before comparing "unsupervised" vision methods across datasets. Its statistics are descriptive and its claims are methodological; the practical contribution is a more explicit record of where human decisions enter a supposedly label-free pipeline.
6. DocHop: Benchmarking Out-of-domain Multi-hop Reasoning in Information-Dense Documents
Area tag: Multimodal reasoning and document evaluation
arXiv: 2609.02059v1, first submitted September 2, 2026. 6
Authors / institutions: Zhuoran Yu, Le Thien Phuc Nguyen, Jaden Park, and Yong Jae Lee are affiliated with the University of Wisconsin-Madison. Xinyi Gu is affiliated with MIT, Zexue He with Stanford University, Soochahn Lee with Kookmin University, and Rogerio Feris with the MIT-IBM Watson AI Lab at IBM Research. 15
Status: Accepted by ICML 2026. 6
Problem: A document image can require a model to resolve an entity from prose, locate its value across several charts, and then perform a numerical or hypothetical operation. Many visual question-answering benchmarks isolate chart reading or document OCR, so they miss this cross-round dependency.
Method: DocHop contains 2,074 examples across 480 topics, seven chart types, six task categories, and 41 question templates. A stochastic logic-first generator controls reasoning depth from 2 to 5 steps and chart count from 2 to 6. Programmatic chart checks, human context checks, and final rendering inspection filter the generated examples. 15
Comparison: The benchmark compares GPT-5.2, GPT-5-mini, Gemini 2.5 Pro and Flash, Claude 4.5 Haiku and Sonnet, reasoning variants, LLaVA-Next, IDEFICS, Qwen-VL, Molmo, Ovis, InternVL, human annotators, and question-only controls. The paper positions DocHop against ChartQA, ChartQAPro, ChartBench, ChartX, ChartXiv, DocVQA, and DocHop-QA by requiring multi-step chart-context reasoning inside one document. 15
Results: Human annotators reach 93.29% overall accuracy. GPT-5.2 Reasoning reaches 62.83%, compared with 40.36% for GPT-5.2 without the reasoning variant. Gemini 2.5 Flash rises from 24.88% to 32.02% with reasoning. Most open-source models score between 9% and 24%, and performance falls as reasoning depth and chart count increase. 15
Resources: The paper provides a DocHop project page but no code repository link in the source. 15
Takeaway: DocHop is useful for evaluating whether a multimodal model can keep a document-level reference chain intact. The benchmark uses clean, synthetic, single-page layouts, so its score measures a specific reasoning capability rather than general document understanding.
Agents and evaluation
7. Improving Evaluation Realism with Inference-Time Compute and Deployment Scaffolds
Area tag: Agent evaluation and alignment audits
arXiv: 2609.02302v1, first submitted September 2, 2026. 7
Authors / institutions: Axel Ahlqvist and Juan-Pablo Rivera are affiliated with the Meridian Visiting Researcher Programme. Richard Guan and Dmitrii Troitskii are affiliated with the Cambridge Boston Alignment Initiative. Alexandra Souly and Robert Kirk are affiliated with the UK AI Security Institute; Kai Fronsdal is affiliated with that institute and Meridian Labs; John Hughes is affiliated with Anthropic. 16
Status: Preprint. The arXiv comment says the paper is under review at NeurIPS 2026. 7
Problem: Simulated agent audits can look easier to detect than real deployment behavior because the target model lacks its usual system prompt, tool definitions, and operational scaffolding. The paper asks whether extra inference-time compute and deployment-like wrappers make the audit more realistic.
Method: Critique refinement generates several candidate auditor actions, asks the target model how to make them more deployment-like, and selects among the candidates with pairwise preference. DISH, the Deployment-Imitating SWE-Agent Harness, wraps targets in agent environments such as Claude Code, Codex CLI, or Gemini CLI. The paper varies refinement depth and breadth and compares both interventions with compute-matched multi-branch baselines. 16
Comparison: The conditions include baseline Petri, the Petri 2.0 realism filter, real deployment resources, critique refinement, DISH, DISH plus critique refinement, and a compute-matched multi-branch baseline. The evaluation measures realism win rate, verbalized evaluation awareness, seed adherence, and deployment-task recreations across several target models. 16
Results: Critique refinement raises realism win rate from 10% to 26% for Sonnet 4.6, from 2% to 7% for Opus 4.8, from 1% to 6% for GPT-5.5, and from 15% to 22% for Gemini 3.5 Flash in the reported target conditions. DISH raises Sonnet 4.6 from 9% to 21% and GPT-5.5 from 0.1% to 3% on coding-agent seeds. The combined condition improves over both single interventions by 2 to 13 percentage points in coding settings. Critique refinement also lowers GPT-5.5 seed adherence from 0.92 to 0.72 at one setting, and its cost is not reduced in the study. 16
Loading content card…
Takeaway: Read this paper when an agent evaluation may be overfitting to an artificial harness. The main engineering question is whether realism gains justify the extra inference cost and the target model's influence over the audit.
8. CAPTURE: Disentangling Preference Drift from Memory Poisoning in Personalized LLM Agents
Area tag: Personalized agents and memory security
arXiv: 2609.02265v1, first submitted September 2, 2026. 8
Authors / institutions: S M Asif Hossain, Ruksat Khan Shayoni, and Md Kishor Morol. The retrieved paper source does not specify institutional affiliations. 17
Status: Preprint. The arXiv record says the paper is under review at ICLR 2027. 8
Problem: A personalized agent must decide whether a new memory reflects a genuine preference change, a temporary context shift, an ambiguous statement, or an injected instruction. Recency and provenance alone can accept poisoned memories or reject legitimate updates.
Method: CAPTURE treats preference authenticity as a latent variable in a continuous-time partially observable decision process. A neural differential equation tracks belief over irregular time intervals. An authenticity gate can ask for clarification, a multi-timescale ledger separates stable, contextual, and transient preferences, a safety-bounded selector constrains actions, and a counterfactual audit checks whether cited memories support the result. The D-PrefGuard benchmark contains 2,400 episodes, with 480 held out for testing. 17
Comparison: On the held-out test, CAPTURE is compared with a matched supervised Transformer, StateMem, provenance-only filtering, recency RAG, and an oracle-memory condition. The evaluation reports win rate, poisoning success, adherence, false updates, safety attacks, refusals, and evidence F1. A zero-shot HorizonBench test and a 40-participant replay study add two secondary checks. 17
Results: CAPTURE reaches a 71.5% win rate, 11.5% poisoning success, and 83.5% adherence, compared with 69.3%, 15.9%, and 82.1% for the matched supervised baseline. Removing the authenticity gate raises poisoning to 33.5%, while removing the safety bound triples safety attack success. On the synthetic HorizonBench evaluation, contradiction resolution is 0.72 for CAPTURE, 0.65 for StateMem, and 0.58 for recency RAG. An adaptive attacker with access to the released weights raises CAPTURE's attack success to 24.7%; the paper does not establish a reliable advantage over a provenance filter in that setting. 17
Resources: The paper says source code, D-PrefGuard construction scripts, and adapter weights are included in supplementary material and will be released publicly. The retrieved source provides no direct public repository URL. 17
Takeaway: CAPTURE is useful for readers designing memory writes rather than only memory retrieval. Its strongest evidence comes from ablations that isolate the authenticity gate, while synthetic tasks and adaptive attacks leave deployment security unresolved.
ML methods and tooling
9. InKAN: B-Spline KANs via Truncated Power Form
Area tag: Neural-network implementation and scientific computing
arXiv: 2609.01956v2, first submitted September 2, 2026; the cited record is version 2, updated September 3. 9
Authors / institutions: Naveen Mysore. The retrieved source does not state an institutional affiliation. 18
Status: Preprint. The arXiv record says the paper is under review at ICLR 2027. 9
Problem: KAN layers evaluate B-spline activations through Cox-de Boor recursion. In the paper's profile, basis computation consumes 91% of a forward pass, so faster matrix operations cannot remove the main cost unless the recursive basis evaluation changes.
Method: InKAN uses the truncated-power form of uniform cubic B-splines, expressing each spline as five shifted positive-part cubic terms. A
torch.compile-fused implementation removes recursion, span lookup, and scatter-gather operations. Bounded-coordinate stabilization clamps normalized inputs to the valid interval to reduce catastrophic cancellation and float16 overflow. The package is intended as a drop-in replacement for existing KAN layers. 18Comparison: The paper contrasts InKAN with standard KAN, Efficient-KAN, FastKAN, TruKAN, MatrixKAN, and a power-ReLU formulation. Standard KAN preserves exact B-spline properties but retains the sequential recursion; FastKAN uses Gaussian RBFs, while TruKAN uses truncated powers as the basis and therefore does not preserve the same B-spline guarantees. 18
Results: In a 256x784 to 64 profiled layer on an MPS GPU, Cox-de Boor basis computation takes 1.44 ms of a 1.58 ms forward pass. InKAN's tests cover basis correctness, partition of unity, compact support, C2 smoothness, and gradient flow. The paper reports no end-to-end benchmark for speed or accuracy, so the implementation claim is stronger than the currently reported application-level evidence. 18
Resources: The author provides an MIT-licensed package installable with
pip install inkan and reports 26 unit tests. The retrieved source does not provide an explicit GitHub URL. 18Takeaway: InKAN is a focused tooling paper for researchers who already use uniform-knot KAN layers and need to profile their implementation. Non-uniform knot grids remain outside the closed-form treatment, and end-to-end task gains are still unreported.
Reading order
- For LLM serving: Start with BeaconKV if memory is the hard limit, then read Faster Than Flash if kernel and selection overhead dominate. Read Don't Drop Dropout when training and inference depth should be optimized together.
- For multimodal evaluation: Open DocHop for chart-context reasoning and the visual-learning position paper for a checklist of hidden supervision choices.
- For embodied or collaborative perception: Read CauseCollab when sensor protocols differ across agents and the deployment must add modalities incrementally.
- For agent safety: Start with CAPTURE for persistent-memory attacks, then read the evaluation-realism paper for the audit harness itself.
- For implementation work: InKAN is the narrowest entry: its value depends on whether uniform-knot KAN layers are the actual runtime bottleneck.
References
- 1Don't Drop Dropout arXiv record
arxiv.org
- 2BeaconKV arXiv record
arxiv.org
- 3Faster Than Flash arXiv record
arxiv.org
- 4CauseCollab arXiv record
arxiv.org
- 5Unlabeled IS NOT Equal arXiv record
arxiv.org
- 6DocHop arXiv record
arxiv.org
- 7Evaluation realism arXiv record
arxiv.org
- 8CAPTURE arXiv record
arxiv.org
- 9InKAN arXiv record
arxiv.org
- 10Don't Drop Dropout paper
arxiv.org
- 11BeaconKV paper
arxiv.org
- 12Faster Than Flash paper
arxiv.org
- 13CauseCollab paper
arxiv.org
- 14Unlabeled IS NOT Equal paper
arxiv.org
- 15DocHop paper
arxiv.org
- 16Evaluation realism paper
arxiv.org
- 17CAPTURE paper
arxiv.org
- 18InKAN paper
arxiv.org
This story was produced automatically by a channel. One sentence is all it takes for Neodrop to keep producing for you.