The AI Engineering Books I Used to Ship a Product

The AI section of my bookshelf has three books on it. That is not laziness, it is the survival rate.

AI engineering

is the discipline of building applications on top of foundation models somebody else trained — evaluation, prompting, retrieval, routing, and the operations that keep all of it running in production.

This list is for the engineer doing that job: shipping features on models you call over HTTP. If you train base models for a research lab, almost none of this applies and you already know where to look. Everyone else is an application engineer with a probabilistic dependency, which is a genuinely new problem and a very old one at the same time.

Disclosure before anything else. I build Codelit, an AI product that generates architecture diagrams, product boards and agent workflows, so every book below is graded against a system I have to keep running rather than against a syllabus. It also means you should read my praise of the evaluation chapters with the knowledge that my own product does not fully live up to them: Codelit validates generated output structurally, with a Zod schema, and a diagram can pass every schema check and still be architecturally wrong. Structural validation is not evaluation. That gap is mine, not the book's.

Because most AI books are organised around facts that expire, and the expiry date arrives before the print run does. Prices, model names, benchmark scores and SDK signatures all move faster than a publishing schedule, and a technical book takes months to travel from finished manuscript to shelf.

Two numbers make the point. Stanford HAI's 2025 AI Index found that querying a model scoring at GPT-3.5 level on MMLU cost $20.00 per million tokens in November 2022 and $0.07 per million tokens by October 2024 — a drop of more than 280 times in about eighteen months. The 2026 edition of the same report notes that performance on SWE-bench Verified, a coding benchmark, went from around 60% to near 100% in a single year.

Now imagine writing a chapter with a pricing table in it. Or a capability table. Or a paragraph beginning "current models struggle with."

The three that survived share one property: their subject is a method rather than an interface. Evaluation methodology does not expire when a provider renames a model. Neither does data drift, and neither does the argument that you should measure a system before you tune it. That is the entire filter, and it is the filter I would apply to any AI book published next month.

AI Engineering by Chip Huyen, published by O'Reilly in January 2025. It is the only book I have found that treats building on foundation models as an engineering discipline with its own failure modes, rather than as ordinary software development with prompting bolted on the side.

The structural tell is the table of contents. Chapter 3 is evaluation methodology and chapter 4 is evaluating AI systems. Prompt engineering is chapter 5. Putting measurement two chapters ahead of the thing everyone actually wants to read about is an argument, and it is the correct one.

Here is the problem it addressed for me. Codelit's pipeline turns one sentence into a schema-valid graph of nodes and edges, and describing the JSON schema in the prompt got me valid output about 80% of the time. Schema enforcement plus a validation loop pushed that past 99%. I got there by grinding, months of it, and the book frames the whole class of problem in a way that would have saved me most of that: stop asking the model more nicely, start constraining and measuring.

What I have never used is chapter 7 on finetuning and chapter 8 on dataset engineering. I do not finetune. Two chapters out of ten, in a book of 532 pages, that I paid for and will probably never open, which is worth knowing before you buy it expecting every page to earn its place.

Designing Machine Learning Systems, also by Chip Huyen and published by O'Reilly in June 2022, teaches data distribution shift and production monitoring. Those two subjects are exactly what breaks an LLM feature six months after launch, and almost nobody building agents today has read them.

The relevance is not obvious until you route across providers. Codelit sends every request down a fallback chain across 11 providers, which means the model answering a given prompt changes without the prompt changing. A provider deprecates a version. A free model gets swapped in the pool. Your input distribution held perfectly still and your output distribution moved anyway.

That is drift. It has a literature, and the literature predates the thing you are building. The chapters on monitoring are written for recommender systems and apply, almost line for line, to a retrieval pipeline.

The book is weaker on anything specifically generative, because it could not have been otherwise in 2022. There is material on feature stores you will never open. Read it for the operations half.

No, not if you are shipping this quarter. Sebastian Raschka's Build a Large Language Model (From Scratch), published by Manning in October 2024, walks you through implementing a GPT-style model in PyTorch step by step, from tokeniser to finetuning. It is very good at what it does. What it does is not your job.

This is the entry every other reading list gets wrong. Roundups are additive by nature: each book is excellent, each earns a place, nothing is ever a bad use of your evenings. That is not true. Evenings are the scarce input, and telling an application engineer with a launch date to go implement multi-head attention is telling them to spend three weeks acquiring an intuition instead of a feature.

The intuition is real, though, which is why the book is on the shelf rather than off it. Working through the attention implementation changed how I write prompts, because I finally understood what a context window physically is rather than treating it as a mysterious quota. That is worth something. It is worth it later.

Release It! and Designing Data-Intensive Applications. Both of them, and it is not close. The reliability problems in an LLM product are ordinary distributed systems problems wearing unfamiliar clothes, and they were documented properly years before there was a model API to break.

Take the router. Codelit's health monitor pings every provider on a 60-second interval, and a provider that recovers takes 10% of traffic for five minutes before full restoration. I did not invent that. It is a circuit breaker with a half-open state, and Nygard catalogued the whole family — timeouts, bulkheads, the difference between failing fast and failing slowly — through the real outages that produced them. The novel part of my system was which vendor sat behind the breaker. The pattern predates every provider in the chain.

Kleppmann earns his place the moment anything runs longer than one request. An agent workflow that calls tools and can die halfway through needs an append-only run log, derived state you can rebuild, and idempotent retries, because exactly-once delivery is a promise nobody can keep. The second edition, co-written with Chris Riccomini, arrived in March 2026 and folds in a decade of change, so older chapter references no longer line up.

Three things, and they are the three that cost me the most time. Provider fallback as an operational discipline, cost accounting per generation, and prompt regression tests that gate a deploy. Each has a nearest neighbour in print, and in every case the nearest neighbour names the problem without solving it.

The pattern here is that all three are operations problems, and operations books get written after the practice has stabilised. Teams were building deployment pipelines by hand for years before Humble and Farley published Continuous Delivery in 2010 and gave everyone a shared name for it.

The cost one is the gap I feel most. I built a cost estimator that prices an AWS architecture before it exists precisely because engineers have no intuition for what infrastructure costs until the invoice lands, and inference has exactly the same problem one layer up. Streaming makes the bookkeeping harder still: once you are rendering a response before the JSON closes, a user can abandon a generation halfway through, and every token that already streamed is a token you paid for. Codelit's AI spend runs about $0.003 per free-tier generation and $0.02 per Pro generation, and knowing those two numbers is what makes the pricing page a decision rather than a guess. No book told me to measure that. Google's SRE book comes closest in spirit, since an error budget is the cleanest available framing for how often a bad generation is acceptable, and it is talking about availability rather than quality.

Read against your current problem rather than in a canonical order, because the wrong book at the wrong time is how people end up owning five unread technical books. Find your row.

The last row is the only one where the from-scratch book goes first, and it is there because career changers are buying understanding rather than velocity. Different purchase entirely.

Three tests, and they take about ninety seconds in a bookshop. Open to a random page and count the model names — if the paragraph collapses when its examples are renamed, so does the book. Check whether the measurement chapter comes before the prompting chapter, because that ordering reveals what the author thinks the work is. Then ask whether the chapter would still make sense if every provider changed their API next Tuesday, which several of them will.

My architecture shelf has a dozen books on it and my AI shelf has three. That ratio is not a judgement about the field, it is a statement about its age: distributed systems has been an engineering discipline for fifty years and this has been one for about three. More good books are coming. In the meantime the transferable half of the job — failure modes, state, measurement, cost — is already documented to an extremely high standard, just filed under a different heading.

If you want the rest of what I actually keep and re-read, including the architecture and management shelves, it is all on the bookshelf, skip-it-if notes included.