SWITZERLANDResearch
arXiv: Telling small on-premises LLMs 'I am a lawyer' makes them refuse benign legal tasks more
Researchers at HES-SO Valais-Wallis in Switzerland tested four open-weight models of about 8 billion parameters on benign prompts and on facts from Swiss Federal Supreme Court criminal judgments. Adding legal-authority framing, such as 'I'm a defense lawyer' or counsel to a national supreme court, raised refusal rates 2 to 20 times over a plain prompt for Llama, Gemma and Apertus. Qwen 3 almost never refused.
Key points
- The models tested were Llama 3.1 8B, Gemma 4 E4B, Qwen3 8B and Apertus 8B Instruct, all run locally through Ollama.arXiv
- The supreme court prefix generally raised refusals more than the lawyer prefix, suggesting the effect grows with the authority claimed.arXiv
- For Gemma and Apertus, the authority prefixes drew more refusals than an explicit role-play jailbreak prompt.arXiv
- The increase stayed statistically significant in French and German for Llama, Gemma and Apertus.arXiv
- The authors say confidentiality pushes courts and lawyers toward small in-house models, so this instability risks uneven access to help.arXiv