AI tricks

Prompt Engineering

• 2025-03: Prompting Science Report 1: Prompt Engineering is Complicated and Contingent
• 2024-06: The Prompt Report: A Systematic Survey of Prompting Techniques

In-Context Learning

• 2020-05: Language Models are Few-Shot Learners
• 2025-03: Learning to Search Effective Example Sequences for In-Context Learning

Chain of Thought (CoT)

• 2022-05: Large Language Models are Zero-Shot Reasoners "Let's think step-by-step"
• 2024-06: TextGrad: Automatic "Differentiation" via Text
• 2025-06: Prompting Science Report 2: The Decreasing Value of Chain of Thought in Prompting

Multi-step

Tool-use, feedback, agentic

• ReAct: Synergizing Reasoning and Acting in Language Models

Retrieval-Augmented Generation (RAG)

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks

Input/Output Formats

• 2024-08: LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs
• 2024-11: Does Prompt Formatting Have Any Impact on LLM Performance?

Position Bias

• 2023-07: Lost in the Middle: How Language Models Use Long Contexts
• 2024-11: Self-Consistency Falls Short! The Adverse Effects of Positional Bias on Long-Context Problems
• 2025-02: On the Emergence of Position Bias in Transformers
• Testing models:
  • Needle-in-a-Haystack tests
  • 2023-08: LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding
  • 2024-02: ∞Bench: Extending Long Context Evaluation Beyond 100K Tokens
  • 2024-06: [Multimodal Needle in a Haystack: Benchmarking Long-Context Capability of Multimodal Large Language Models https://arxiv.org/abs/2406.11230]
  • 2024-07: Stress-Testing Long-Context Language Models with Lifelong ICL and Task Haystack
  • 2025-04: Reasoning on Multiple Needles In A Haystack
• Mitigation:
  • 2023-10: Attention Sorting Combats Recency Bias In Long Context Language Models
  • 2024-07: Eliminating Position Bias of Language Models: A Mechanistic Approach

Generation

• Large Language Models for Data Annotation and Synthesis: A Survey