AI compute

Analysis

• 2024-03: AI and Memory Wall

Cloud GPU

• Lambda
• Vast AI
• Lightning AI
• RunPod
• HPC-AI

Cloud Training Compute

• Nebius AI
• Glaive AI

Cloud LLM Routers & Inference Providers

• OpenRouter (open and closed models, no Enterprise tier)
• LiteLLM (closed models, Enterprise tier)
• Cent ML (open models, Enterprise tier)
• Fireworks AI (open models, Enterprise tier)
• Abacus AI (open and closed models, Enterprise tier)
• Portkey (open? and closed models, Enterprise tier)
• Together AI (open models, Enterprise tier)
• Hyperbolic AI (open models, Enterprise tier)
• Huggingface Inference Providers Hub
• AskSage

Multi-model with Model Selection

• Not Diamond ¬⋄
• Martian

Multi-model Web Chat Interfaces

• SimTheory
• Abacus AI ChatLLM
• Poe
• Gab AI
• Vectal ?
• BlackboxAI

Multi-model Web Playground Interfaces

• Together AI
• Hyperbolic AI

Local Router

• Ollama
• LocalAI
• ai-gradio: unified model interface (based on gradio)

Acceleration Hardware

• Nvidia GPUs
• Google TPU
• Etched: Transformer ASICs
• Cerebras
• Untether AI
• Graphcore
• SambaNova Systems
• Groq
• Tesla Dojo
• Deep Silicon: Combined hardware/software solution for accelerated AI (e.g. ternary math)

Energy Use

• 2021-04: Carbon Emissions and Large Neural Network Training
• 2023-10: From Words to Watts: Benchmarking the Energy Costs of Large Language Model Inference
• 2024-01: Electricity 2024: Analysis and forecast to 2026
• 2024-02: The carbon emissions of writing and illustrating are lower for AI than for humans
• 2025-04: Why using ChatGPT is not bad for the environment - a cheat sheet
  • A single LLM response uses only ~3 Wh = 11 kJ (~10 Google searches; examples of 3 Wh energy usage)
  • Reading an LLM-generated response (computer running for a few minutes) typically uses more energy than the LLM generation of the text.
• 2025-07: Mistral: Our contribution to a global environmental standard for AI
• 2025-08: Measuring the environmental impact of delivering AI at Google Scale (blog)