Submission + - Can We Trick/Get AI Labs to Benchmaxx Fighting Health Insurance Denials? (fighthealthinsurance.com)
An anonymous reader writes: Many AI labs have gotten a little too good at benchmaxxing, but all the benchmaxxing is just making tools to make questionable Python scripts slightly faster.
But what if we could redirect some of that benchmark-chasing toward problems that actually make people's lives better?
We're building a new benchmark focused on appealing health insurance denials. The idea is pretty simple: if AI labs are going to compete for leaderboard points anyway, let's give them a reason to build models that are better at helping people navigate bureaucracy.
We'd love some help evaluating the results, especially from folks who enjoy poking holes in AI benchmarks.
But what if we could redirect some of that benchmark-chasing toward problems that actually make people's lives better?
We're building a new benchmark focused on appealing health insurance denials. The idea is pretty simple: if AI labs are going to compete for leaderboard points anyway, let's give them a reason to build models that are better at helping people navigate bureaucracy.
We'd love some help evaluating the results, especially from folks who enjoy poking holes in AI benchmarks.