CHINAResearch
arXiv: LegalRewardBench finds general-purpose models beat fine-tuned reward models at judging legal answers
An October 13, 2026 arXiv paper from researchers at Harbin Institute of Technology, ModelBest and the University of Osaka introduces LegalRewardBench, described as the first human-verified benchmark for reward models in the legal domain. Built on Chinese legal tasks across five capability levels, it finds strong general-purpose generative models are the most robust legal judges, while fine-tuned reward models transfer less reliably.
Key points
- The benchmark grades legal preference judgments across knowledge, understanding, analysis, decision and writing levels.arXiv
- Preference pairs draw on answers from about 30 models of 3 billion to 1 trillion parameters, labeled by rubric and human checks.arXiv
- Claude Opus 4.7 scored highest overall at 86.85%, ahead of Claude Opus 4.5 at 85.01% and GPT-5.5 at 83.66%.arXiv
- Fine-tuned generative and discriminative reward models transferred less reliably to legal preference judgments than strong general models.arXiv
- The authors found no consistent advantage from the language a model reasons in, though the dataset is in Chinese.arXiv