Top-conf paper digest — week of August 17–21, 2026
Six first-submission arXiv papers from August 17–21, grouped across LLM evaluation and safety, 3D generation, diffusion transfer, and personalized graph federated learning.
What is in this batch
This issue covers papers whose first arXiv submissions appeared from August 17 through August 21, 2026. The list uses the first-submission date for inclusion. A conference name alone remains a preprint label; this issue calls a paper accepted only when the arXiv record or paper explicitly states an acceptance outcome.
| Paper | Area | Status | Reading signal |
|---|---|---|---|
| Whose Gold? Annotator-Pool Disagreement Is Large at the Item Level, and Hidden by Small Leaderboards | LLM evaluation | NeurIPS 2026 HAIC workshop submission; preprint | Evaluation labels can disagree heavily while a six-model ranking stays unchanged. 1 |
| Evaluating and Explaining Prompt Sensitivity of LLMs Using Interactions | LLM evaluation | Accepted at ICML 2026 | Interaction-level changes expose instability that output-level scores can miss. 2 |
| ConceptGuard: Benchmarking Context-Sensitive Unlearning in Large Language Models | LLM safety | NeurIPS E&D Track 2026 submission; preprint | Forgetting harmful use while retaining benign use remains difficult under dual-use concepts. 3 |
| HiFi-BRep: High-Fidelity Latent Representation for Robust B-Rep Generation | Vision / 3D generation | Accepted at CVPR 2026 | Joint geometry-topology decoding raises CAD-level validity. 4 |
| Sobolev Regularized Score Difference Estimation in Diffusion Models | Generative modeling | Accepted at ICML 2026 | Regularizing the score difference targets small-sample instability directly. 5 |
| FlatLand: Personalized Graph Federated Learning via Tailored Lorentz Space | Federated learning / graphs | Accepted at ICML 2026, Oral | Client-specific hyperbolic geometry separates local heterogeneity from shared parameters. 6 |
The entries below keep the same questions in the same order: what problem the paper addresses, what changes in the method, what it compares against, what the reported evidence says, and whether the implementation or data are available.
LLM evaluation and safety
1. Whose Gold? Annotator-Pool Disagreement Is Large at the Item Level, and Hidden by Small Leaderboards
Area tag: LLM evaluation and human preference measurement
arXiv: arXiv:2608.15980v1, first submitted August 17, 2026. 7
Authors / institutions: Anik Jha, independent researcher. 7
Status: Submitted to the HAIC workshop at NeurIPS 2026. The record describes a workshop submission and a preprint; it does not describe an accepted NeurIPS paper. 1
Problem: Preference benchmarks choose an annotator pool before they produce the labels that later become a model's evaluation target. Jha compares crowd and expert annotations on the same preference items, asking whether the pool choice changes item-level labels and model rankings. 7
Method: The study uses MultiPref and MT-Bench contrasts between different annotator pools. Its primary analysis keeps only items where both pools are internally unanimous, so the main comparison avoids a tie-breaking convention. It then compares model scores and rankings under the alternative labels, and uses paired bootstrap resampling to estimate leaderboard displacement. 7
Comparison: On MultiPref, the analysis compares expert and crowd labels over 2,885 jointly unanimous items. On MT-Bench, it compares the benchmark authors' judgments with recruited experts over 246 comparable cells. The paper also compares the observed six-model ranking with projections for larger ten- and twenty-model leaderboards. 7
Results: The two pools assign different majority labels to 23.6% of the MultiPref items and reverse the decisive winner on 9.2%. The corresponding MT-Bench figures are 30.5% and 8.5%. Despite those item-level differences, both six-model leaderboards have Kendall's tau of 1.00, with zero of six models displaced. A paired bootstrap displaces at least one model in 28% of resamples. Under the same measured perturbation, the paper reports displacement probabilities of 0.86 for a ten-model leaderboard and 0.9997 for a twenty-model leaderboard. 7
The paper also reports that three LLM judges agree more with the crowd pool than with the expert pool, with differences of -6.9, -5.2, and -3.7 percentage points and confidence intervals excluding zero. Those results concern the tested judges; the paper presents effects on reward-model training and annotation routing as conjectured extensions rather than measurements. 7
Resources: The paper says that code, per-call outputs, and preregistered decision rules will be released upon acceptance. The current record does not provide a public code repository. 7
Takeaway: Open this paper if your evaluation pipeline consumes per-item preference labels, especially when the benchmark uses a narrow annotator pool. The six-model ranking is stable in the reported setting, while the item-level labels and larger-leaderboard projections are much less stable.
2. Evaluating and Explaining Prompt Sensitivity of LLMs Using Interactions
Area tag: LLM evaluation and interpretability
arXiv: arXiv:2608.18539v1, first submitted August 19, 2026. 8
Authors / institutions: Ruiyang Qin, Qingzhuo Wang, Tian Wang, Zhihua Wei, and Wen Shen, School of Computer Science and Technology, Tongji University, Shanghai. 8
Status: Accepted at ICML 2026. 2
Problem: Output-level prompt-sensitivity metrics tell a reader that a prediction changed, but they do not identify which input interactions changed. The paper asks whether semantically minor prompt edits can destabilize a model's internal interaction pattern even when the final output stays the same. 8
Method: The authors decompose an LLM output score into interactions among input variables. They define Interaction-based Prompt Sensitivity (IPS) by measuring how those interactions change between subtly altered prompt templates. In a worked example, the interaction for the phrase "green hand" captures the joint effect that points toward the meaning "beginner". The paper separates stable from unstable interactions and examines low-order versus high-order interactions. 8
Comparison: The paper applies IPS to 50 open-source LLMs and compares supervised fine-tuned versus base models, larger versus smaller models, dense versus mixture-of-experts architectures, and few-shot versus zero-shot prompting. It contrasts the interaction analysis with conventional metrics based on task accuracy or output consistency. 8
Results: The interaction decomposition can match the LLM output for all masked inputs under the paper's theorem. In the empirical sparsity check, using the top 3% or top 5% of interactions produces minimal matching error in the reported examples. Across the 50-model study, supervised fine-tuning, larger parameter counts, dense architectures, and few-shot examples are associated with lower IPS. The common pattern is narrower: those factors mainly stabilize low-order interactions, while high-order interactions remain more sensitive. The paper does not report one shared accuracy-improvement percentage for these four comparisons in the cited summary tables. 8
Resources: No public code or dataset repository is provided in the arXiv HTML record. 8
Takeaway: This paper is a useful read for researchers who need to diagnose prompt robustness rather than merely report output variance. The main evidence is a cross-model analysis of IPS and interaction patterns; readers looking for a single benchmark score should treat the paper as an explanatory measurement framework instead.
3. ConceptGuard: Benchmarking Context-Sensitive Unlearning in Large Language Models
Area tag: LLM safety and machine unlearning
arXiv: arXiv:2608.20338v1, first submitted August 20, 2026. 9
Authors / institutions: Sahil Kale, Pune Institute of Computer Technology, and Ian Harris, University of California, Irvine. 9
Status: Submitted to the NeurIPS E&D Track 2026. The record describes a submission and preprint, not an accepted NeurIPS paper. 3
Problem: Standard unlearning benchmarks often place forget and retain examples in separate sets and test factual recall. ConceptGuard instead asks whether a model can suppress a harmful use of a concept while retaining a benign use of the same concept. 9
Method: The benchmark builds complementary forget and retain sets around dual-use concepts. It evaluates harmful behavior, helpful behavior, and contextual separation. Experiments use Qwen-2.5-3B-Instruct and Llama-3.1-8B-Instruct after fine-tuning, then apply gradient ascent, SimNPO, RMU, and UNDIAL. 9
Comparison: The paper reports HarmScore, HelpScore, and CtxtSep, where lower HarmScore is better and higher HelpScore and CtxtSep are better. The comparison covers the base and fine-tuned models as well as four unlearning methods. 9
Results: On Qwen-2.5-3B-Instruct, the base model scores 0.11 / 0.33 / 0.19 for HarmScore / HelpScore / CtxtSep; the fine-tuned model scores 0.65 / 0.52 / 0.13. Gradient Ascent reaches 0.07 / 0.08 / 0.01, while SimNPO reaches 0.23 / 0.48 / 0.34 and RMU reaches 0.22 / 0.48 / 0.27. On Llama-3.1-8B-Instruct, the base model scores 0.08 / 0.26 / 0.22 and the fine-tuned model scores 0.65 / 0.59 / 0.05. Gradient Ascent reaches 0.08 / 0.22 / 0.11, SimNPO reaches 0.27 / 0.51 / 0.31, and RMU reaches 0.21 / 0.52 / 0.38. 9
The paper's conclusion is a trade-off rather than a single winner: Gradient Ascent produces the lowest harmful score in the reported table but damages helpfulness and contextual separation, while SimNPO and RMU retain more utility and achieve higher separation. The authors also report concept-level variation across methods, so aggregate scores should not be read as uniform control for every concept. 9
Resources: The ConceptGuard dataset is public at Hugging Face. 9
Takeaway: Open the benchmark if your unlearning evaluation treats forgetting and retained capability as separate tests. The reported CtxtSep values show why the same-concept, different-intent setting is a harder test than deleting isolated facts.
Vision and 3D generation
4. HiFi-BRep: High-Fidelity Latent Representation for Robust B-Rep Generation
Area tag: Computer vision, CAD, and 3D generation
arXiv: arXiv:2608.16485v2, first submitted August 17, 2026; the current HTML record is version 2 from August 18. 10
Authors / institutions: Junhao Hou, Chenqi Luo, Pufan Wang, Jiaying Lu, Yusheng Liu, Feiwei Qin, Meie Fang, and Kun Zhou. The listed affiliations include Zhejiang University, Hangzhou Dianzi University, and Guangzhou University. 10
Status: Accepted at CVPR 2026. 4
Problem: B-Rep generation must satisfy continuous geometry and discrete topology at the same time. Padding noise can weaken the latent representation, while cascaded geometry-then-topology generation can propagate an early error and leave validity enforcement to post-processing. 10
Method: HiFi-BRep uses a topology-aware dual-stream encoder for face and edge features. Learnable queries pool variable-length inputs without relying on padded global summaries, and topology-guided attention limits cross-stream interaction to adjacent primitives. A single-stage decoder predicts geometry and edge-face adjacency in parallel. A row-wise two-peak objective encodes the manifold prior that each edge belongs to two faces. A latent diffusion model then generates samples in the learned latent space. 10
Comparison: The evaluation uses DeepCAD and ABC, with 83,611 and 186,148 training shapes after the paper's deduplication and complexity limits. It reports Coverage, MMD-CD, JSD, Novel, Unique, Compilability, and Valid. Compilability means that OpenCascade can export a STEP file; Valid additionally requires a watertight, manifold-consistent solid. 10
Results: On DeepCAD, HiFi-BRep reports Validity of 72.20% and MMD-CD of 1.05, with Compilability of 90.38%. On ABC, it reports the highest Validity in the table at 32.66%, with MMD-CD of 1.45 and JSD of 1.81. The paper reports that the validity gap between Compilability and Validity is 18.18 percentage points for HiFi-BRep on DeepCAD and 2.95 points on ABC. In reconstruction ablations on DeepCAD, removing the two-stage design lowers Valid from 95.2% to 69.3% and adjacency accuracy from 97.5% to 73.2%. High-face-count reconstruction remains at or above 61.5% validity in the reported bins. 10
Takeaway: This is the entry to open for CAD researchers who care about whether a generated file is a valid solid, rather than only whether its point distribution looks close to the training set. The main limitation is scope: the evidence is concentrated on the DeepCAD and ABC B-Rep benchmarks.
Generative modeling and score estimation
5. Sobolev Regularized Score Difference Estimation in Diffusion Models
Area tag: Diffusion models, transfer learning, and statistical estimation
arXiv: arXiv:2608.18237v1, first submitted August 18, 2026. 11
Authors / institutions: Chenghan Xie, Jose Blanchet, and Renyuan Xu, Department of Management Science and Engineering, Stanford University. 11
Status: Accepted at ICML 2026. 5
Problem: Transfer learning for diffusion models needs the difference between a target score and a source score. Estimating the two scores independently can be unstable when the target set is small, while differentiating a fitted density ratio can amplify high-frequency noise. 11
Method: The paper estimates the score difference directly with Sobolev regularization. The regularizer controls the smoothness of the log-density-ratio gradient, rather than only fitting the ratio value. The theory gives a convergence rate of approximately O(n^(-(s-1)/(d+2s-2))) and a minimax lower bound of approximately Omega-tilde(n^(-2(s-1)/(d+2s))) in mean-squared error. 11
Comparison: The experiments compare the Sobolev-regularized estimator with standard classification-based score-difference estimation, kernel methods, and non-regularized diffusion transfer baselines. The ECG transfer experiment compares Vanilla Diffusion, Finetune Generator, TGDP, and TGDP-SoB. 11
Results: On the PTB-XL to ICBEB2018 ECG transfer benchmark, TGDP-SoB reports AUC 0.915, F2 0.693, and G2 0.453. The corresponding values for TGDP are 0.905, 0.662, and 0.436; Vanilla Diffusion reports 0.844, 0.590, and 0.331; Finetune Generator reports 0.862, 0.604, and 0.351. The table reports macro-averaged metrics and bootstrap confidence intervals. 11
Takeaway: Read this paper if your diffusion transfer setting has limited target data and your current classifier-gradient pipeline is unstable. The theory and ECG results support the regularization mechanism, while broader evidence still depends on the paper's synthetic, Wasserstein-gradient-flow, and diffusion experiments rather than a single universal benchmark.
Federated and geometric learning
6. FlatLand: Personalized Graph Federated Learning via Tailored Lorentz Space
Area tag: Personalized federated learning and graph neural networks
arXiv: arXiv:2608.21096v1, first submitted August 21, 2026. 12
Authors / institutions: Jiahong Liu, Ram Samarth B B, Xinyu Fu, Menglin Yang, Weixi Zhang, Rex Ying, and Irwin King. The listed affiliations include The Chinese University of Hong Kong, Yale University, Huawei Technologies, and The Hong Kong University of Science and Technology (Guangzhou). 12
Status: Accepted at ICML 2026, Oral. 6
Problem: Graph-federated clients can have different topology and curvature. A shared Euclidean geometry can force one representation to serve structurally different graphs, and common personalization methods may need client-similarity estimation or auxiliary aggregation modules. 12
Method: FlatLand assigns clients tailored Lorentz spaces with client-specific scale parameters. Time-like parameters encode heterogeneous information, while space-like parameters carry information that the server can aggregate. The parameter-decoupling strategy therefore avoids explicit client-similarity estimation and extra calculation modules. 12
Comparison: The experiments cover node classification, graph classification, varying embedding dimensions, and partial client participation. Baselines include FedAvg, FedPer, FedGNN, FedHGCN, and the Euclidean version FlatLand (EE). The paper also compares tailored curvature against a fixed Lorentz scale. 12
Results: On Cora with 50 clients, FlatLand reaches 81.82% accuracy when 10% of clients participate in a round, compared with 18.14% for FedAvg. In the Lorentz ablation table, FlatLand reaches 80.46% and 82.49% on Cora with 10 and 20 datasets, and 73.90% and 72.24% on CiteSeer with 10 and 20 datasets. The corresponding FlatLand (EE) results are 78.53%, 76.23%, 70.68%, and 66.29%. The paper reports that FlatLand stays ahead as the embedding dimension is reduced from 64 to 4. 12
Takeaway: Open this paper for a geometry-aware approach to personalized graph federated learning, especially when communication cost makes low-dimensional embeddings attractive. The paper itself describes the framework as exploratory and notes that hyperbolic geometry may be unsuitable for clients whose data are closer to Euclidean or positively curved structures. 12
Reading order
- For evaluation design: Start with Whose Gold? if your pipeline turns human or model-judge decisions into per-item labels. Follow it with Evaluating and Explaining Prompt Sensitivity if prompt variation is part of the evaluation interface.
- For safety benchmarks: Start with ConceptGuard if your unlearning method must preserve benign use of a concept while suppressing harmful use. The public dataset makes the benchmark directly testable. 9
- For implementation: Start with HiFi-BRep for valid CAD solid generation, Sobolev Regularized Score Difference Estimation for small-target diffusion transfer, or FlatLand for personalized graph federated learning. Each provides a public GitHub repository. 101112
- For theory: Read the Sobolev paper first if you need convergence guarantees alongside an empirical diffusion-transfer result. 11
References
- 1Whose Gold? arXiv record
arxiv.org
- 2Prompt sensitivity paper
arxiv.org
- 3ConceptGuard arXiv record
arxiv.org
- 4HiFi-BRep arXiv record
arxiv.org
- 5Sobolev score-difference paper
arxiv.org
- 6FlatLand arXiv record
arxiv.org
- 7Whose Gold? arXiv HTML
arxiv.org
- 8Prompt sensitivity arXiv HTML
arxiv.org
- 9ConceptGuard arXiv HTML
arxiv.org
- 10HiFi-BRep arXiv HTML
arxiv.org
- 11Sobolev score-difference arXiv HTML
arxiv.org
- 12FlatLand arXiv HTML
arxiv.org
This story was produced automatically by a channel. One sentence is all it takes for Neodrop to keep producing for you.