Difference between revisions of "AI benchmarks"
KevinYager (talk | contribs) (→Leaderboards) |
KevinYager (talk | contribs) (→Leaderboards) |
||
| Line 1: | Line 1: | ||
=Leaderboards= | =Leaderboards= | ||
| − | * [https://lmsys.org/ LMSYS]: Human preference ranking | + | * [https://lmsys.org/ LMSYS]: Human preference ranking leaderboard |
| − | * [https://trackingai.org/home Tracking AI] | + | * [https://trackingai.org/home Tracking AI]: "IQ" leaderboard |
* [https://www.vectara.com/ Vectara] [https://github.com/vectara/hallucination-leaderboard Hallucination Leaderboard] | * [https://www.vectara.com/ Vectara] [https://github.com/vectara/hallucination-leaderboard Hallucination Leaderboard] | ||
* [https://livebench.ai/#/ LiveBench: A Challenging, Contamination-Free LLM Benchmark] | * [https://livebench.ai/#/ LiveBench: A Challenging, Contamination-Free LLM Benchmark] | ||
| − | * [https://scale.com/leaderboard/enigma_eval ENIGMAEVAL] ([https://static.scale.com/uploads/654197dc94d34f66c0f5184e/EnigmaEval%20v4.pdf paper] | + | * [https://scale.com/leaderboard/enigma_eval ENIGMAEVAL]: "reasoning" leaderboard ([https://static.scale.com/uploads/654197dc94d34f66c0f5184e/EnigmaEval%20v4.pdf paper]) |
==Assistant/Agentic== | ==Assistant/Agentic== | ||
* [https://arxiv.org/abs/2311.12983 GAIA: a benchmark for General AI Assistants] | * [https://arxiv.org/abs/2311.12983 GAIA: a benchmark for General AI Assistants] | ||
Revision as of 15:22, 11 March 2025
Contents
Leaderboards
- LMSYS: Human preference ranking leaderboard
- Tracking AI: "IQ" leaderboard
- Vectara Hallucination Leaderboard
- LiveBench: A Challenging, Contamination-Free LLM Benchmark
- ENIGMAEVAL: "reasoning" leaderboard (paper)
Assistant/Agentic
- GAIA: a benchmark for General AI Assistants
- Galileo AI Agent Leaderboard
- Smolagents LLM Leaderboard: LLMs powering agents
Methods
- AidanBench: Evaluating Novel Idea Generation on Open-Ended Questions (code)
- ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning. Assess reasoning using puzzles of tunable complexity.
