AI Handbook for Engineers

How AI works, for engineers who do not work on it

Contents

Part I. How a model learns

  1. 1Models and loss
  2. 2Gradient descent
  3. 3Neural networks
  4. 4Overfitting

Part II. Language models

  1. 5Tokens
  2. 6Embeddings
  3. 7Next-token prediction
  4. 8Attention and transformers
  5. 9Pretraining
  6. 10Inference

Part III. From base model to assistant

  1. 11Fine-tuning
  2. 12Reinforcement learning
  3. 13Reasoning models
  4. 14Distillation

Part IV. Beyond text

  1. 15Images
  2. 16Diffusion
  3. 17Speech and audio

Part V. Building with models

  1. 18Prompts and context
  2. 19Retrieval-augmented generation
  3. 20Tool use and agents
  4. 21Evaluation
  5. 22Hallucination

Chapters in grey are not written yet.