Tune small. Predict large.

Power-Law Entropy Search (PLES), first submitted to arXiv on 1 September 2026, treats hyperparameter tuning as a cost-aware multi-fidelity Bayesian optimization problem. Instead of spending a fixed grid of runs, PLES models training loss with a Gaussian process, samples the best hyperparameter at several scales, and buys the next experiment for the uncertainty it removes per unit of compute. 1

The product problem is expensive target-scale tuning: a large training run can be the costliest way to learn which setting works. Across synthetic functions, a Gaussian-process surrogate fitted to earlier LLM runs, and real LLM pre-training, PLES reached the best attainable held-out loss with less than one-tenth of the compute used by the canonical baselines. Most proposed candidates cost less than 6% of the held-out run. The experiments used grid search, Ladder BO, and Sobol random search as baselines, with 10 seeds per method under the same cumulative N × D budget. 2

Fit PLES as a shadow tuner beside the current policy. Feed it existing runs, let posterior uncertainty at the target scale guide the next cheap experiment, and compare held-out loss, hyperparameter error, total N × D, wall-clock time, and cost. Keep the current tuner live until those measures clear your gate.

The paper assumes the optimum follows θ* = cN^αD^β; a different scaling form can bias the extrapolation. The paper lists no public code or data release, so the first build stays an implementation sketch. 2