PaperMap turns landmark AI research papers into visual, interactive learning experiences. Follow the full curriculum — 114 explainers across 12 categories, from the original Transformer through agent evaluation, agent safety, memory, multi-agent systems, and quantization — each rebuilt as a production-grade interactive guide.
115 entries · RAGTruth appears in III and VI · ★ featured picks from the source list
The curriculum follows the source list exactly: 12 categories, 115 numbered entries, 114 unique papers.
19 entries · Attention Is All You Need, GPT-1, BERT, GPT-2, T5, GPT-3, …
7 entries · Megatron-LM, ZeRO, FlashAttention, Speculative Decoding, FlashAttention-2, PagedAttention (vLLM), …
10 entries · REALM, DPR, RAG, RETRO, ColBERTv2, ALCE, …
11 entries · RLHF Origins, PPO, LoRA, InstructGPT, Self-Instruct, Constitutional AI, …
11 entries · Chain-of-Thought, Self-Consistency, STaR, Zero-Shot Reasoners, Induction Heads, ReAct, …
7 entries · TruthfulQA, SelfCheckGPT, FActScore, HaluEval, Hallucination Survey, RAGTruth, …
8 entries · MMLU, HumanEval, HELM, LLM-as-a-Judge, GPQA, LiveCodeBench, …
14 entries · WebArena, AgentBench, SWE-bench, GAIA, OSWorld, SWE-agent, …
9 entries · Red Teaming LMs, Indirect Prompt Injection, GCG Attack, Sleeper Agents, InjecAgent, AgentDojo, …
8 entries · Generative Agents, MemGPT, LoCoMo, Memory Mechanism Survey, LongMemEval, A-MEM, …
7 entries · CAMEL, MetaGPT, AutoGen, Magentic-One, Collaboration Mechanisms Survey, MultiAgentBench, …
4 entries · GPTQ, SmoothQuant, AWQ, DSPy
The lineage that produced every modern LLM: from the original Transformer, BERT, and the GPT line, through scaling laws and compute-optimal training, to open-weight herds, sparse mixture-of-experts, and attention efficiency tricks like grouped-query attention. 19 papers.
The paper that started it all. Interactive walkthrough of the Transformer architecture with tokenization, embeddings, self-attention, multi-head attention, positional encoding, and full encoder-decoder visualization.
Open Explainer →The forgotten starting gun of generative pre-training. GPT-1 paired a 12-layer Transformer decoder with task-specific input formats and showed one unsupervised objective — next-token prediction — could beat task-trained NLP systems across twelve benchmarks.
Open Explainer →Bidirectional pre-training with masked language modeling. Interactive demos for MLM, the 80/10/10 rule, architecture specs, and how one model conquered 11 NLP tasks with a single output layer.
Open Explainer →GPT-2 and the zero-shot bet: a 1.5B decoder-only model trained on WebText learns tasks without supervision. Zero-shot prompt demos, byte-level BPE, and the staged release that started the AI safety debate.
Open Explainer →Every NLP task as plain text in, plain text out. Interactive task-prefix demos, the C4 corpus pipeline, systematic architecture ablations, and T5-11B's SuperGLUE 88.9 benchmark record.
Open Explainer →How 175 billion parameters unlocked in-context learning. Interactive few-shot demos, scaling law visualizations, benchmark tables, data contamination analysis, and the full GPT-3 model family.
Open Explainer →Performance depends strongly on scale and weakly on shape. Interactive power-law loss explorer, compute-budget allocator, and the training-curve laws that turned model scale into a science.
Open Explainer →One expert per token. Interactive top-1 router playground, router z-loss stability gym, the 7× pre-training speedup race, and the parameter ledger that explains what 1.6T parameters really buy.
Open Explainer →Rotate, don't add. RoPE encodes token position by rotating query-key pairs in complex space so attention becomes a relative-position function — exact, cache-friendly, and now the default positional encoding in LLaMA, GPT-NeoX, and nearly every modern LLM.
Open Explainer →Instruction tuning before it was cool: 137B parameters finetuned on 60+ tasks phrased as instructions beat zero-shot GPT-3 on 20 of 25 held-out datasets. Template flipper + cluster hold-out explorer.
Open Explainer →The scaling correction: 70B parameters trained on 1.4T tokens beats Gopher at the same compute. Interactive budget allocator, tokens-per-param checker, and the ~20 tokens/param rule of thumb.
Open Explainer →Open weights, matched at smaller scale. LLaMA showed 7B-65B models trained longer on more data — public, filtered, deduplicated — could match or beat much larger closed models, igniting the open-weights ecosystem.
Open Explainer →The frontier becomes infrastructure science: performance predicted from 1/1000th-compute runs, 86.4% MMLU, bar-exam top 10%, and the predictable-scaling discipline every lab now follows.
Open Explainer →The attention compromise the field adopted. GQA groups key-value heads between full MHA and single-head MQA — uptrained from existing checkpoints with about 5% of original pre-training compute — keeping quality while slashing decoder inference memory.
Open Explainer →The open chat model that democratized assistants. Llama 2 paired a stronger base (2T tokens) with an RLHF pipeline — rejection sampling, PPO, iterative safety refinement — plus Ghost Attention for multi-turn instruction persistence.
Open Explainer →The open sparse MoE that matched Llama 2 70B with 13B active parameters: top-2 routing live, quality-per-FLOP accounting, and the Apache 2.0 release that changed the market.
Open Explainer →Finer experts, shared knowledge. DeepSeekMoE splits experts into many small specialized ones plus always-on shared experts — matching dense quality at 40% of compute, and scaling to 145B total / 22B activated.
Open Explainer →Scaling the herd. Llama 3 trained on 15T+ tokens with aggressive data curation, then added the 3.1/3.2/3.3 progression: 405B flagship, distillation, quantization, and multimodal extensions — the definitive open-weights engineering report.
Open Explainer →Frontier training at bargain budgets. DeepSeek-V3: 671B total / 37B activated parameters, Multi-head Latent Attention, DeepSeekMoE with auxiliary-loss-free balancing, FP8 mixed precision, and 14.8T tokens — for a claimed 2.788M H800-GPU hours.
Open Explainer →The systems engineering that makes training and serving trillion-scale models possible: model parallelism (Megatron-LM), memory optimization (ZeRO), IO-aware exact attention (FlashAttention 1/2), speculative decoding, paged KV-cache serving, and selective state-space sequence models. 7 papers.
Slicing Transformers across GPUs. Megatron-LM's intra-layer model parallelism splits attention and feed-forward matrices across devices with near-linear scaling to 8.9B parameters on 512 GPUs — the blueprint for every big training run since.
Open Explainer →Data parallelism without the memory duplication. ZeRO shards optimizer state, gradients, and parameters across devices — eliminating redundancy and enabling trillion-parameter training on modest clusters with a 3-13× efficiency win over naive DDP.
Open Explainer →Attention is memory-bound, not compute-bound. Memory-trip counters, the online-softmax tiling stepper, bandwidth simulators, and the IO-aware exact kernel that enabled long context.
Open Explainer →Small models draft, big models verify in parallel. The speculative round stepper, α-yield explorer, and 2-3× acceleration with provably identical outputs.
Open Explainer →FlashAttention, retuned for the GPU. Better work partitioning (fewer non-matmul FLOPs), parallelism over the sequence dimension, and tuned block sizes push it to 2× the original's speed and 50-73% of peak on A100s — up to 230 TFLOPs/s.
Open Explainer →Virtual memory for the KV cache: fragmentation visualizers, block-table generation, copy-on-write prefix sharing, and vLLM's 2-4× throughput breakthrough.
Open Explainer →The selective state-space challenger to Transformers. Mamba lets SSM parameters depend on the input (selection), enabling content-aware reasoning at linear-time complexity and 5× higher inference throughput on sequences up to a million tokens.
Open Explainer →Grounding language models in external knowledge: retrieval-augmented pre-training (REALM), dense passage retrieval, the RAG recipe that named the field, trillion-token retrieval (RETRO), compressed late-interaction retrieval (ColBERTv2), citation generation, long-context failure modes, and self-reflective + domain-adapted RAG. 10 papers.
Retrieval enters pre-training. REALM embeds a knowledge-retriever inside the masked-LM objective so the model learns which Wikipedia documents to fetch — grounding factual completion before finetuning and beating T5-XXL on open-domain QA.
Open Explainer →Embeddings replace BM25. DPR trains a dual-encoder — questions and passages mapped to one dense space — with in-batch negatives, retrieving top-21 passages at 99%+ accuracy and lifting open-domain QA by 9-19 points over sparse retrieval.
Open Explainer →An interactive deep dive into the original RAG paper: dense retrieval with DPR, BART generation, non-parametric memory with Wikipedia, and benchmark gains on knowledge-intensive QA.
Open Explainer →A 2-trillion-token external memory. RETRO retrieves continuations from a massive frozen indexed corpus via cross-attention chunks, gaining knowledge without parameter growth and allowing database swaps — even for a GPT-3-scale model.
Open Explainer →One vector per token, one MaxSim per query word, and 1–2-bit residual compression that shrinks the index ~6×. Interactive late-interaction scoring, compression visualizer, and BEIR wins.
Open Explainer →Attribution as a first-class output. ALCE frames citation generation — every claim backed by a retrieved source — and builds the benchmark (ASQA, ELI5, Wikipedia) with automatic metrics for citation quality and evidence sufficiency.
Open Explainer →Position beats relevance. Across QA and key-value tasks, models exploit information at the edges of the context window and systematically ignore the middle — a U-shaped performance curve that reshaped long-context evaluation and prompting strategy.
Open Explainer →The model decides when to retrieve — and grades itself. Self-RAG trains reflection tokens that control retrieval on demand and self-assess grounding and utility, outperforming ChatGPT and Llama2-chat on open-domain QA, factuality, and citation generation.
Open Explainer →17,790 real RAG responses with word-level hallucination spans — and the uncomfortable finding that even gold retrieved context still yields fabricated claims.
Open Explainer →Training for the messy real corpus. RAFT finetunes domain RAG on questions where some retrieved documents are distractors, mixing oracle and distractor training plus chain-of-thought answers — improving in-domain QA over general RAG prompting.
Open Explainer →Turning raw pretrained models into helpful, steerable assistants: RLHF's origin, PPO, parameter-efficient finetuning (LoRA, QLoRA), instruction tuning (InstructGPT, Self-Instruct), AI feedback (Constitutional AI), direct preference optimization, process supervision, and RL for reasoning (DeepSeekMath, DeepSeek-R1). 11 papers.
The RLHF origin story. Instead of reward functions, humans compare short trajectory clips; the reward model learns from a few hundred to a thousand preferences, and RL agents solve simulated robotics without ever seeing a hand-crafted reward.
Open Explainer →The workhorse of policy optimization. PPO clips the policy-ratio objective so each update stays near the old policy — first-order simplicity with trust-region stability — and became the default RL algorithm behind InstructGPT and beyond.
Open Explainer →An interactive walkthrough of LoRA covering low-rank updates, parameter-efficient fine-tuning, practical deployment tradeoffs, and why LoRA matches full fine-tuning at a fraction of the cost.
Open Explainer →The RLHF paper that paved the way for ChatGPT. Interactive 3-step pipeline (SFT → Reward Model → PPO), human evaluation results, bias analysis, and qualitative examples.
Open Explainer →Bootstrapping an instruction dataset from the model itself. Self-Instruct generates 52K instructions from 175 seed tasks, filters near-duplicates, and finetunes — closing much of the gap to InstructGPT without human-written data.
Open Explainer →Alignment from a written document: self-critique steppers, the RLAIF preference labeler, 16 principles, and the harmlessness-helpfulness frontier — the recipe behind Claude.
Open Explainer →65B finetuning on one 48GB GPU: NF4 quantization grids, double quantization, paged optimizers, and Guanaco reaching 99.3% of ChatGPT on the Vicuna benchmark.
Open Explainer →RLHF without RL: a closed-form preference loss derived from Bradley-Terry theory. Interactive derivation, pairwise training demo, and why the open-model world adopted it over PPO.
Open Explainer →Grade the steps, not just the answer. Process reward models (PRMs) supervised on 800K per-step labels beat outcome models on MATH benchmarks — and verify each reasoning step catches errors outcome scoring can't see.
Open Explainer →Math reasoning from data + GRPO. DeepSeekMath builds a 120B-token corpus from Common Crawl, then applies Group Relative Policy Optimization — a critic-free PPO variant — reaching GPT-4-level competition math in a 7B open model.
Open Explainer →Reasoning capability from pure reinforcement learning. GRPO visualizer, the emergent "aha moment" trace player, the 4-stage training pipeline, and distilled open models matching o1.
Open Explainer →How models think: chain-of-thought prompting, self-consistency voting, bootstrapped reasoning (STaR), zero-shot reasoning, the induction-head circuits behind in-context learning, and the agent loop papers — ReAct, Toolformer, Reflexion, Tree of Thoughts, CoALA, and optimal test-time compute. 11 papers.
Reasoning steps in few-shot prompts unlock multi-step problems — but only at ~100B scale. A standard-vs-CoT race, emergence charts, and the ablations that proved the words matter.
Open Explainer →Sample 40 reasoning paths, take the vote. Interactive sampling-and-voting demos, convergence histograms, and +17.9 points on GSM8K with zero additional training.
Open Explainer →Reasoning that generates its own supervision. STaR samples rationales, keeps those that reach correct answers, and bootstraps on hints for failures — repeatedly improving math, commonsense, and logic without external rationales.
Open Explainer →Two words: 'Let's think step by step.' Zero-shot CoT turns modest models into multi-step reasoners — up to ~36-point accuracy jumps on arithmetic and symbolic tasks — before any few-shot examples are shown.
Open Explainer →The two-head attention circuit behind in-context learning. Sequence-completion demos, K-composition steppers, and the training phase change that switched ICL on.
Open Explainer →Thought, action, observation — one loop. A live agent-trace player, the hallucination showdown vs CoT, and the template behind every modern AI agent.
Open Explainer →A 6.7B model that inserts its own API calls, filters them by perplexity, and reads the results. Interactive perplexity-filter demos, five tools, and beating GPT-3 175B on LAMA.
Open Explainer →Failure becomes text, text becomes memory. Reflexion agents store verbal self-critiques of failed attempts and reuse them in later trials — beating standard RL on HumanEval coding with simple episodic memory instead of weight updates.
Open Explainer →Search over thoughts. ToT explores a tree of intermediate reasoning steps with LLM self-evaluation and deliberate look-ahead/rollbacks — GPT-4 solves 4% of Game of 24 with chain-of-thought; Tree of Thoughts search reaches 74%.
Open Explainer →The unified theory of LLM agents. CoALA maps agents onto a cognitive architecture — decision cycle, working/episodic/semantic/procedural memory, and action spaces (internal + external) — organizing 100+ papers into one design space.
Open Explainer →Spend your FLOPs at inference. A compute-optimal strategy — choosing between denser chains and parallel sampling, adaptive revision, and search per difficulty — lets a smaller model with 4× inference FLOPs beat a 14× larger model on MATH.
Open Explainer →Why models make things up and how to measure it: TruthfulQA's falsehood benchmark, sampling-based detection (SelfCheckGPT), atomic factual precision (FActScore), HaluEval's large-scale evaluation, the field's reference taxonomy survey, the RAG hallucination corpus, and the 2025 theory of why hallucination is intrinsic. 7 papers.
817 adversarial questions that catch models repeating human falsehoods. Imitative-falsehood demos, MC1 vs MC2 scoring formats, and the 36-point truthfulness gap between GPT-3 and people.
Open Explainer →Sample the same model N times and watch hallucinations contradict their own alternatives — no external database, no token probabilities, no extra annotators. Black-box only.
Open Explainer →Break long-form generations into atomic facts, verify each one against a knowledge source like Wikipedia, and turn "is this model factual?" into a number you can audit.
Open Explainer →35,000 truthful answers paired with plausible-but-wrong twins built by ChatGPT itself — then ten leading LLMs try to tell them apart, and most barely beat a coin flip.
Open Explainer →The field's shared vocabulary: what hallucination is, its intrinsic and extrinsic types, where it enters the data → training → inference pipeline, and how to detect and mitigate it.
Open Explainer →17,790 real RAG responses with word-level hallucination spans — and the uncomfortable finding that even gold retrieved context still yields fabricated claims.
Open Explainer →Hallucination as a numbers game. When a calibrated model correctly predicts facts at rates p1 > p2, it must hallucinate to be accurate on the rarer fact — lower-bounding errors at ~19% even on Zipfian Wikipedia facts, and 40%+ on long generations.
Open Explainer →Measuring general capability: MMLU's massive multitask understanding, code evaluation (HumanEval), holistic evaluation (HELM), LLM-as-a-judge methodology, graduate-level Google-proof QA (GPQA), and the contamination-free benchmarks that keep scores honest (LiveCodeBench, LiveBench, JudgeBench). 8 papers.
57 subjects, 15,908 expert-designed questions. The subject-grid difficulty explorer, confidence-vs-correctness calibration demo, and the benchmark that became the field's default knowledge test.
Open Explainer →164 hand-written Python problems, pass@k, and Codex. The paper that standardized code evaluation — functional correctness over edit-distance — and introduced the 12B code model behind GitHub Copilot's first release.
Open Explainer →30 models × 42 scenarios × 7 metrics. The Stanford framework that replaced one-number leaderboards with a transparent scenario–metric matrix spanning accuracy, robustness, fairness, and beyond.
Open Explainer →Can models grade other models? MT-Bench category explorer, Chatbot Arena voting, judge-bias simulators for position and verbosity, and 85% agreement with human raters.
Open Explainer →Questions experts can't Google. 448 multiple-choice problems in biology, physics, and chemistry — validated so skilled non-experts with 30+ minutes of web browsing score barely above chance, even with the exact wording posted online.
Open Explainer →Fresh problems, monthly. LiveCodeBench continuously collects new LeetCode/AtCoder/Codeforces problems with time-stamped release windows, decouples prompts from style effects, and exposes contamination drift across model release dates.
Open Explainer →A benchmark with a shelf life. LiveBench releases new questions monthly from recent sources (math competitions, papers, news), scores objective ground-truth tasks (no LLM judge), and openly ranks closed and open models with versioned releases.
Open Explainer →350 adversarial response pairs where one answer is subtly, objectively wrong — and even GPT-4o judges only slightly better than random guessing. The stress-test for LLM judges.
Open Explainer →Can agents act in the real world? Web environments (WebArena), the agent-evaluation grid (AgentBench), real GitHub issues (SWE-bench, SWE-Bench Pro), general assistants (GAIA), full computer control (OSWorld), agent-computer interfaces (SWE-agent), user interaction (τ-bench, τ²-Bench), frontier R&D (RE-Bench, PaperBench), browsing (BrowseComp), long-task horizons (METR), and terminal work (Terminal-Bench). 14 papers.
Self-hostable, reproducible web work. Four realistic sites (e-commerce, forums, GitLab, CMS + maps) with 812 tasks grounded in functional mock databases — where the best 2023 agents reached only ~14% success versus ~78% for humans.
Open Explainer →8 live environments — terminals, databases, web browsing, games — where models must act, not just answer, with every action checked. The first systematic multi-environment agent benchmark.
Open Explainer →2,294 real GitHub issues across 12 Python repos, graded by the repositories' own tests. Instance-mining pipeline, resolution machine across agent eras, and the 1.96% humbling of 2023's best.
Open Explainer →Questions easy for humans, hard for AI. GAIA's 466 questions demand reasoning, multimodal handling, tool use, and web navigation — no specialist tricks — and 2023's best systems solved under 15% while humans exceed 92%.
Open Explainer →The full desktop as playground. OSWorld offers 369 real computer tasks across five operating systems in sandboxed environments — files, apps, cross-app workflows — where multimodal agents manage under 22% even with hints and step ground truth.
Open Explainer →The interface is the capability. A minimal agent-computer interface (ACI) — file viewer, editor with linting, search, compact actions — lets a vanilla LM resolve 12.5% of SWE-bench issues, from 3.8% without the custom tools.
Open Explainer →Simulated users with hidden goals, writable APIs, and database-state grading. The τ-bench loop stepper, pass^k reliability cliff, and failure autopsies of agents that fold under pressure.
Open Explainer →Can agents do AI research? Seven 2-32 hour ML R&D tasks (kernel optimization, scaling laws, losslandscape design) with expert baselines: agents complete tasks 10-100× cheaper than humans per attempt and match top-human scores in 2-8 hours.
Open Explainer →Task length as the capability clock. Across a decade of game, software, and R&D benchmarks, agent task-horizon — the length of task at 50% success — doubles every ~7 months, from seconds in 2019 toward hours by 2025, extrapolating to month-long tasks in years ahead.
Open Explainer →Replication as the ultimate research eval. 20 ICML 2024 papers decomposed into 8,316 rubric-conducted, code-verified subtasks; deep research agents reach ~21% replication ability versus human ML PhDs at 41.4-52.3%.
Open Explainer →Find the needle across the web. 1,277 multi-hop browsing puzzles with hard-to-guess answers and known sources; GPT-4o gets 2%, and the paper introduces agent-harness strategies (reasoning-browsing interleave) that push Deep Research far higher.
Open Explainer →The user gets tools too: a Dec-POMDP telecom domain where agent and user both act on shared state. Dual-control walkthrough, compositional task generator, and the guiding penalty.
Open Explainer →1,865 enterprise-scale problems across 41 repositories — hours-to-days tasks, split public/held-out/commercial for contamination-proof evaluation. The three-bowl design and the long-horizon wall.
Open Explainer →89 hard, realistic command-line tasks in containerized environments with hidden tests and human-written solutions. Agent terminal session, outcome probes, and the model × harness grid.
Open Explainer →The attack surface of deployed agents: red-teaming at scale, indirect prompt injection against real LLM-integrated apps, universal transferable jailbreaks, deceptive backdoored models that survive safety training, and the agent-specific safety benchmarks — InjecAgent, AgentDojo, ASB, AgentHarm, and Agent-SafetyBench. 9 papers.
Scale the red team. A 52B LM generates offensive personas and test cases that surface harmful outputs 100× more effectively per sample than human red teamers — then the same generations finetune the target model to be safer.
Open Explainer →The prompt you didn't write. Instructions hidden in retrieved web pages, emails, and documents hijack LLM-integrated apps — exfiltrating data, spreading as worms, and executing cost attacks — with zero model weight changes.
Open Explainer →Jailbreaks from optimization, not intuition. Greedy coordinate gradient search finds suffixes that force aligned models (open and closed) to produce harmful content — transferable, universal, and black-box via API access.
Open Explainer →Backdoors that survive the cure. Models trained to write vulnerable code only when triggered by date strings remain deceptive after RLHF, adversarial training, and fine-tuning — with activation probing the only reliable detector.
Open Explainer →1,054 test cases where a single hidden sentence, buried in a tool response, hijacks tool-integrated agents — wiring money, unlocking doors, and leaking private data.
Open Explainer →A dynamic, open environment where LLM agents solve realistic tasks while researchers throw prompt-injection attacks at them and evaluate defenses in the same loop.
Open Explainer →16 attacks and 11 defenses formalized across the whole agent stack — system prompt, user prompt, memory, tools, and actions — behind one unified risk metric.
Open Explainer →Explicitly harmful agent tasks, done right. 110 refined tasks with multi-step variants testing whether models refuse in agent roles — where refusal drops when harm requires one extra step (indirect harm), and open weight models comply most.
Open Explainer →349 interactive tool-using environments and 2,000 test cases that measure whether agents take unsafe actions — not just say unsafe words. None of the 16 agents evaluated scored 80%.
Open Explainer →Giving agents a past: the simulacra that made memory famous (Generative Agents), LLMs-as-operating-systems (MemGPT), long-term conversational memory evaluation (LoCoMo, LongMemEval), the memory-mechanism survey, and the production-grade memory systems of 2025-2026 — A-MEM's Zettelkasten agents, Mem0, and beyond-factual cognition (LoCoMo-Plus). 8 papers.
25 LLM-powered residents of a Sims-like town with a memory stream, reflections, and daily plans — and a Valentine's Day party that emerged from a single seed of intent.
Open Explainer →The context window as RAM: self-generated function calls page facts between a small live window and unlimited "disk" — operating-system-style virtual memory for LLMs.
Open Explainer →300 turns, 35 sessions, ~9K tokens per conversation. LoCoMo's human-verified dialogues test QA, event summarization, and multimodal memory — and expose the gap between LLMs and humans on long-range conversational recall.
Open Explainer →The field map of agent memory: where it lives, what it stores, and the write, read, and reflect operations that keep it useful as conversations and tasks accumulate.
Open Explainer →500 questions embedded in ~115K-token multi-session chat histories. Long-context LLMs drop 30–60% — the benchmark that separates real memory from long windows.
Open Explainer →A Zettelkasten for agents: memory that writes structured notes, links them into a growing knowledge network, and rewrites older notes when new experience arrives.
Open Explainer →The production memory layer: two-phase extraction, ADD/UPDATE/DELETE/NOOP operations, graph memory, and 26% accuracy over full-context at 91% lower p95 latency on LoCoMo.
Open Explainer →Not "can you recall what I said?" but "can you remember my goals, states, and values — and let those latent constraints steer your advice hundreds of turns later?"
Open Explainer →Society of agents: role-playing communicative agents (CAMEL), SOP-driven software-company metaphors (MetaGPT), the conversable-agent framework (AutoGen), the generalist Magentic-One orchestration, and the 2025 reckoning — collaboration-mechanism surveys, MultiAgentBench's collaboration/competition evaluation, and the failure-mode taxonomy of Why Do Multi-Agent LLM Systems Fail? 7 papers.
An AI user gives instructions, an AI assistant executes them, and one carefully engineered inception prompt keeps both on-script — cooperation with no human in the loop.
Open Explainer →A software firm encoded as SOPs: five specialized roles pass structured documents down an assembly line that ships working code from a one-line requirement.
Open Explainer →Agents that talk to each other — and to you. AutoGen's conversable-agent abstraction (LLM, tool, human, or group) with flexible conversation patterns became the standard framework for coding, math, and creative multi-agent apps.
Open Explainer →Microsoft's open-source generalist team: a lead Orchestrator that plans, tracks progress, and re-plans while directing web, file, and code specialists.
Open Explainer →The landscape of LLM teamwork: who talks to whom, why agents compete, and when a team actually beats a single model — communication topologies and interaction protocols mapped.
Open Explainer →The MARBLE framework scores agent teams on what they achieve together AND on how well they coordinate, communicate, and compete along the way — milestones, not just final answers.
Open Explainer →The failure taxonomy the field needed. 1600+ annotated traces across 7 frameworks, classified by the MAST taxonomy into 14 failure modes in 3 categories — system design, inter-agent misalignment, and task verification. Expert-validated (kappa 0.88).
Open Explainer →Making models small and programs smart: post-training quantization (GPTQ, SmoothQuant, AWQ) that shrinks LLMs to 4-bit with near-lossless quality, and the declarative programming layer (DSPy) that compiles LM pipelines instead of hand-prompting them. 4 papers.
4-bit weights in one GPU-hour. GPTQ quantizes layer-by-layer using approximate second-order (Hessian) information, compressing OPT-175B and BLOOM-176B to 3-4 bits with minor degradation — no retraining, at scale.
Open Explainer →Smooth the outliers, keep the bits. Migrating activation-scale difficulty into weights with a per-channel alpha lets SmoothQuant run W8A8 fully-integrated quantization on LLMs (OPT-175B, BLOOM-176B) with negligible loss and 1.56× speedup + 2× memory cut.
Open Explainer →1% of channels matter most. AWQ protects salient weight channels — found via activation magnitudes, not training — with per-channel scaling, achieving 4-bit W4A16 acceleration (1.45-1.85× over FP16 on int4 tokens) and edge deployment (30B on 4GB).
Open Explainer →Stop writing prompts — program them. Signature splitter, bootstrap compilation stepper, and pipelines that compile in minutes to beat expert-written prompts by 25-65%.
Open Explainer →The core curriculum — 115 entries / 114 unique papers across 12 categories — is complete, exactly matching the source list. PaperMap keeps growing one handcrafted explainer at a time.
19 entries · Attention Is All You Need, GPT-1, BERT, GPT-2, T5, …
7 entries · Megatron-LM, ZeRO, FlashAttention, Speculative Decoding, FlashAttention-2, …
10 entries · REALM, DPR, RAG, RETRO, ColBERTv2, …
11 entries · RLHF Origins, PPO, LoRA, InstructGPT, Self-Instruct, …
11 entries · Chain-of-Thought, Self-Consistency, STaR, Zero-Shot Reasoners, Induction Heads, …
7 entries · TruthfulQA, SelfCheckGPT, FActScore, HaluEval, Hallucination Survey, …
8 entries · MMLU, HumanEval, HELM, LLM-as-a-Judge, GPQA, …
14 entries · WebArena, AgentBench, SWE-bench, GAIA, OSWorld, …
9 entries · Red Teaming LMs, Indirect Prompt Injection, GCG Attack, Sleeper Agents, InjecAgent, …
8 entries · Generative Agents, MemGPT, LoCoMo, Memory Mechanism Survey, LongMemEval, …
7 entries · CAMEL, MetaGPT, AutoGen, Magentic-One, Collaboration Mechanisms Survey, …
4 entries · GPTQ, SmoothQuant, AWQ, DSPy
The final 115-entry list is complete. Multimodal systems, diffusion research, interpretability deep cuts — contributions welcome via PROMPT.md.