IDAHOResearch
arXiv: Thomson Reuters engineers find agent harness design matters more than model size for legislative XML
In an arXiv preprint posted October 14, 2026, Thomson Reuters Technology researchers Palvika Bansal, Rittika Jindal and Ruichong Wang report that a production pipeline converting U.S. state bills into schema-constrained XML owes its accuracy mainly to harness engineering. With a decomposed multi-agent pipeline, the weakest Claude model scored 0.948, beating the strongest model working alone at 0.806.
Key points
- The study crosses three harnesses, from zero-shot to a decomposed multi-agent pipeline, with Claude Haiku, Sonnet and Opus models.arXiv
- The gold set holds 153 expert-produced bills, 103 from Idaho and 50 from New Jersey.arXiv
- Model scores spread from 0.68 to 0.81 at zero-shot but converge near 0.95 in the full pipeline.arXiv
- Remaining errors sit mostly in citation normalization, which a deterministic lookup tool handles better than a larger model.arXiv
- The paper was accepted to the Natural Legal Language Processing workshop at EMNLP 2026.arXiv