Difference between revisions of "AI tricks"
KevinYager (talk | contribs) (→Position Bias) |
KevinYager (talk | contribs) (→Position Bias) |
||
| Line 26: | Line 26: | ||
* 2024-11: [https://arxiv.org/abs/2411.01101 Self-Consistency Falls Short! The Adverse Effects of Positional Bias on Long-Context Problems] | * 2024-11: [https://arxiv.org/abs/2411.01101 Self-Consistency Falls Short! The Adverse Effects of Positional Bias on Long-Context Problems] | ||
* 2025-02: [https://arxiv.org/abs/2502.01951 On the Emergence of Position Bias in Transformers] | * 2025-02: [https://arxiv.org/abs/2502.01951 On the Emergence of Position Bias in Transformers] | ||
| − | * ''Testing models:'' | + | * '''Testing models:''' |
** [https://github.com/gkamradt/LLMTest_NeedleInAHaystack?utm_source=chatgpt.com Needle-in-a-Haystack tests] | ** [https://github.com/gkamradt/LLMTest_NeedleInAHaystack?utm_source=chatgpt.com Needle-in-a-Haystack tests] | ||
** 2023-08: [https://arxiv.org/abs/2308.14508 LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding] | ** 2023-08: [https://arxiv.org/abs/2308.14508 LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding] | ||
| Line 33: | Line 33: | ||
** 2024-07: [https://arxiv.org/abs/2407.16695 Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack] | ** 2024-07: [https://arxiv.org/abs/2407.16695 Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack] | ||
** 2025-04: [https://arxiv.org/abs/2504.04150 Reasoning on Multiple Needles In A Haystack] | ** 2025-04: [https://arxiv.org/abs/2504.04150 Reasoning on Multiple Needles In A Haystack] | ||
| − | * ''Mitigation:'' | + | * '''Mitigation:''' |
** 2023-10: [https://arxiv.org/abs/2310.01427 Attention Sorting Combats Recency Bias In Long Context Language Models] | ** 2023-10: [https://arxiv.org/abs/2310.01427 Attention Sorting Combats Recency Bias In Long Context Language Models] | ||
** 2024-07: [https://arxiv.org/abs/2407.01100 Eliminating Position Bias of Language Models: A Mechanistic Approach] | ** 2024-07: [https://arxiv.org/abs/2407.01100 Eliminating Position Bias of Language Models: A Mechanistic Approach] | ||
Latest revision as of 08:16, 8 May 2025
Contents
Prompt Engineering
- 2025-03: Prompting Science Report 1: Prompt Engineering is Complicated and Contingent
- 2024-06: The Prompt Report: A Systematic Survey of Prompting Techniques
In-Context Learning
- 2020-05: Language Models are Few-Shot Learners
- 2025-03: Learning to Search Effective Example Sequences for In-Context Learning
Chain of Thought (CoT)
"Let's think step-by-step"
Multi-step
Tool-use, feedback, agentic
Retrieval-Augmented Generation (RAG)
Input/Output Formats
- 2024-08: LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs
- 2024-11: Does Prompt Formatting Have Any Impact on LLM Performance?
Position Bias
- 2023-07: Lost in the Middle: How Language Models Use Long Contexts
- 2024-11: Self-Consistency Falls Short! The Adverse Effects of Positional Bias on Long-Context Problems
- 2025-02: On the Emergence of Position Bias in Transformers
- Testing models:
- Needle-in-a-Haystack tests
- 2023-08: LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
- 2024-02: ∞Bench: Extending Long Context Evaluation Beyond 100K Tokens
- 2024-06: [Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models https://arxiv.org/abs/2406.11230]
- 2024-07: Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack
- 2025-04: Reasoning on Multiple Needles In A Haystack
- Mitigation:
