QUANTSLANT_
Benchmarks

FairStress: Ai2 Benchmark Finds Hidden Bias in LLM Decisions

quantslant@ai:~$ cat news_1

› Most existing tests of demographic fairness in language models use a simple approach: change a candidate’s identity and check whether the model makes the same choice.

quantslant@ai:~$ cat news_2

› Ai2 (the Allen Institute for AI) judged that insufficient and published FairStress on October 20, 2026.

quantslant@ai:~$ cat news_3

› The benchmark measures something harder to detect: whether a model defends its decision with equal conviction when someone tries to change its mind.Why existing fairness tests fall shortIf a model picks the stronger candidate regardless of race, it may appear fair.

quantslant@ai:~$ cat news_4

› But what happens when an argument urges it to change its decision - for example, an argument that a senior manager privately recommended candidate B?

quantslant@ai:~$ open --full
Read more

FairStress: Ai2 Benchmark Finds Hidden Bias in LLM Decisions

Open article on quantslant.com