Difference between revisions of "AI research trends"

From GISAXS
Jump to: navigation, search
(→‎Context Length)
(→‎Continual Learning)
 
(7 intermediate revisions by the same user not shown)
Line 7: Line 7:
 
* 2024-12: [https://arxiv.org/abs/2412.09764 Memory Layers at Scale]
 
* 2024-12: [https://arxiv.org/abs/2412.09764 Memory Layers at Scale]
 
* 2025-10: [https://arxiv.org/abs/2510.15103 Continual Learning via Sparse Memory Finetuning]
 
* 2025-10: [https://arxiv.org/abs/2510.15103 Continual Learning via Sparse Memory Finetuning]
 +
* 2026-01: [https://developer.nvidia.com/blog/reimagining-llm-memory-using-context-as-training-data-unlocks-models-that-learn-at-test-time/ Reimagining LLM Memory: Using Context as Training Data Unlocks Models That Learn at Test-Time] (Nvidia)
 +
* 2026-01: [https://arxiv.org/abs/2601.02151 Entropy-Adaptive Fine-Tuning: Resolving Confident Conflicts to Mitigate Forgetting]
  
 
==Context Length==
 
==Context Length==
Line 45: Line 47:
 
* 2025-09: [https://arxiv.org/abs/2509.25140 ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory]
 
* 2025-09: [https://arxiv.org/abs/2509.25140 ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory]
 
* 2025-10: [https://arxiv.org/abs/2510.04618 Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models]
 
* 2025-10: [https://arxiv.org/abs/2510.04618 Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models]
 +
* 2025-12: [https://arxiv.org/abs/2512.24601 Recursive Language Models] (model searches/queries the full context)
 +
* 2026-01: [https://arxiv.org/abs/2601.02553 SimpleMem: Efficient Lifelong Memory for LLM Agents]
 +
* 2026-01: [https://arxiv.org/abs/2601.07190 Active Context Compression: Autonomous Memory Management in LLM Agents]
  
 
==Retrieval beyond RAG==
 
==Retrieval beyond RAG==
Line 57: Line 62:
 
==Working Memory==
 
==Working Memory==
 
* 2024-12: [https://www.arxiv.org/abs/2412.18069 Improving Factuality with Explicit Working Memory]
 
* 2024-12: [https://www.arxiv.org/abs/2412.18069 Improving Factuality with Explicit Working Memory]
 +
* 2026-01: [https://arxiv.org/abs/2601.03192 MemRL: Self-Evolving Agents via Runtime Reinforcement Learning on Episodic Memory]
  
 
==Long-Term Memory==
 
==Long-Term Memory==
Line 67: Line 73:
 
===Storage and Retrieval===
 
===Storage and Retrieval===
 
* 2025-09: [https://arxiv.org/abs/2509.04439 ArcMemo: Abstract Reasoning Composition with Lifelong LLM Memory]
 
* 2025-09: [https://arxiv.org/abs/2509.04439 ArcMemo: Abstract Reasoning Composition with Lifelong LLM Memory]
 +
* 2026-01: [https://www.arxiv.org/abs/2601.07372 Conditional Memory via Scalable Lookup: A New Axis of Sparsity for Large Language Models]
  
 
===Episodic Memory===
 
===Episodic Memory===
Line 73: Line 80:
  
 
==Continual Learning==
 
==Continual Learning==
 +
* 2022-02: [https://arxiv.org/abs/2202.00275 Architecture Matters in Continual Learning]
 
* 2025-10: [https://arxiv.org/abs/2510.15103 Continual Learning via Sparse Memory Finetuning]
 
* 2025-10: [https://arxiv.org/abs/2510.15103 Continual Learning via Sparse Memory Finetuning]
 
* 2025-11: [https://research.google/blog/introducing-nested-learning-a-new-ml-paradigm-for-continual-learning/ Introducing Nested Learning: A new ML paradigm for continual learning]
 
* 2025-11: [https://research.google/blog/introducing-nested-learning-a-new-ml-paradigm-for-continual-learning/ Introducing Nested Learning: A new ML paradigm for continual learning]

Latest revision as of 10:40, 17 January 2026

System 2 Reasoning

See: Increasing AI Intelligence

Memory

LLM Weights Memory

Context Length

Extended Context

Context Remaking

Retrieval beyond RAG

See also: AI tools: Retrieval Augmented Generation (RAG)

Working Memory

Long-Term Memory

Storage and Retrieval

Episodic Memory

Continual Learning

Updating Weights at Inference-time

Parameters as Tokens

Internal Thought Representation Space

Visual Thinking

Neural (non-token) Latent Representation

Altered Transformer

Tokenization

Generation Order

Diffusion Language Models

Related: Image Synthesis via Autoregression/Diffusion

Sampling

Daydreaming, brainstorming, pre-generation

Pre-generation


Missing Elements

  • Memory
  • Continuous learning/update
  • Robust contextual model
  • Long-time-horizon coherence
  • Fluid intelligence
  • Agency
  • Modeling of self
  • Daydreaming

Memes

See Also

Retrieved from "http://gisaxs.com/index.php?title=AI_research_trends&oldid=8557"