AI benchmarks

Leaderboards

• LMSYS: Human preference ranking
• Tracking AI
• Vectara Hallucination Leaderboard
• LiveBench: A Challenging, Contamination-Free LLM Benchmark

Methods

• AidanBench: Evaluating Novel Idea Generation on Open-Ended Questions (code)
• ZebraLogic: On the Scaling Limits of LLMs for Logical Reasoning. Assess reasoning using puzzles of tunable complexity.

Assess Specific Attributes

Creativity

• 2024-10: AI as Humanity's Salieri: Quantifying Linguistic Creativity of Language Models via Systematic Attribution of Machine Text against Web Text