Introducing EXO, a method towards efficient exact optimization of LM alignment.
tldr: EXO is more efficient than RLHF by perfoming probability matching, and exact for arbitary policy parametrization compared to DPO.
arxiv: arxiv.org/pdf/2402.00856.pdf
repo: github.com/cdjhz/exact-optim…