Alignment Method vs Impossibility Theorem

DPO's Arrow-shaped bargain

DPO looks clean because it compresses pairwise human preferences into an implicit reward scale, but that tractability comes from discarding plural disagreement rather than solving it.