04 · a week, hands on
The gap between a cool demo and something people can actually use is evaluation, reliability and security. Prompting matters, but it is rarely the hard part. This is the least glamorous track on the page and the one most likely to change what you do on a Tuesday.
The gap between a demo and a product is almost entirely evaluation, reliability and security, and almost none of it is prompting. Patterns that survive contact with users, the failure mode nobody has solved, and the discipline separating teams who ship from teams who demo. If you build anything, the evals item will change how you work.
Agent is now a word people throw at any loop with confidence and a logo. Read the actual pattern first: tools, state, decisions, failure, retry. The mascot costume can wait in the hallway.
Distinguishes workflows, where steps are fixed in code, from agents, where the model directs its own process, and argues most production systems should be the former. The most useful correction available to anyone about to over-engineer, from the lab with every incentive to say otherwise.
45 min anthropic.com · free
A production-oriented guide to agents as workflows with tools, instructions, orchestration and guardrails. Useful because it says the quiet part plainly: build the boring control surface around the model or the model becomes a tiny enthusiastic intern with root access.
35 min openai.com · free
Ready with the full reading map.
Aggregate usage diagnostics are stored; your question and answer text are not.
Hamel's point is the one builders keep learning the expensive way: you cannot prompt your way out of not knowing whether the thing worked. Evals are how the demo becomes software and not just a very expensive improv partner.
The case that systematic evaluation, not prompt tinkering, is the actual work: error analysis, building a small labelled set from real failures, looking at your data. Unglamorous, and the highest-leverage thing in this track.
1 hr hamel.dev · free
The failure mode is not theoretical. You are giving language-shaped instructions to a system that reads language-shaped hostile instructions. Security here is less castle wall, more tiny fussy door labels on every cupboard.
Willison named the problem and has documented it since. The core claim is unresolved and load-bearing: if a model processes untrusted input and has tool or data access, there is currently no reliable way to stop that input redirecting it. Every agent product is built on top of this open problem.
1.5 hr simonwillison.net · free
The security checklist, maintained by the organisation that does this for the rest of software. Prompt injection, insecure output handling, data poisoning, excessive agency. Boring the way fire codes are boring.
1 hr genai.owasp.org · free
Seven recurring patterns — evals, RAG, fine-tuning, caching, guardrails, defensive UX, collecting feedback — each with the reasoning for when it applies. The closest thing to a design-patterns reference, from someone shipping at scale.
1.5 hr eugeneyan.com · free
The systematic version of this whole track: model selection, prompting, RAG and agents, fine-tuning, evaluation, inference optimisation, architecture. If the essays leave you wanting a spine rather than a pile, this is it.
An O'Reilly book. Her blog covers much of the same ground free if you would rather sample first.
long huyenchip.com · paid
A governance frame for systems that pursue goals with limited supervision: who is responsible, what baseline practices might matter, and where ordinary product governance starts to look underdressed. Read it beside the build guides so agents do not become policy debt with a chat box.
Also in Policy & governance. Ticking it here marks it there.
25 min openai.com · free