AI
LLM App Operations #4: Batching — Half Price for Non-Urgent Work
Are you still sending work that does not need an immediate answer through the real-time API? The Batches API processes bulk requests asynchronously in exchange for a 50% discount on every token. We cover picking batch-worthy work, submitting and collecting, and operational patterns.
LLM App Operations #3: Prompt Caching in Practice
Caching the system prompt and tool definitions that repeat on every request cuts the input cost of that portion to one tenth. We cover the prefix-matching principle, stable prefix design, cache_control placement, and an audit for silent cache invalidation.
LLM App Operations #2: Cost — Token Accounting and Model Routing
The biggest lever for cutting cost is model choice. Measuring with count_tokens before sending, putting output on a diet, designing model routing by task difficulty, and tuning effort. We cover the order of operations for lowering cost while protecting quality.
LLM App Operations #1: Between Demo and Production — A Map of Operations
An LLM app that works and an LLM app you can operate are different things. We draw a map of operations along five axes — cost, latency, reliability, quality, and security — and build per-request instrumentation, the starting point for everything.
Advanced RAG #7: Capstone Project — Upgrading the Document Q&A Bot
Upgrade the internal document Q&A bot from LLM App Development Part 13 step by step with the techniques from this series. From measuring the baseline through swapping chunking, hybrid search, reranking, and citations, we watch how the metrics move at every step.
Advanced RAG #6: Building a RAG Evaluation Pipeline
We grow Part 1's baseline into a full evaluation system. Retrieval is scored with recall@k and MRR, generation with an LLM judge, and a single evaluation script that also measures hallucination rate runs as the regression test for every change.
Advanced RAG #5: Reducing Hallucinations with Citations
We tackle generation failures, where answers go wrong even though the right chunks were provided. We implement prompts that keep answers inside the evidence, the right to answer "I do not know," and per-sentence source attribution with the citations feature in Claude.
Advanced RAG #4: Query Transformation and Reranking
User questions are not good search queries. We reinforce both ends of retrieval: query rewriting that folds in conversation context, multi-query that asks from several angles, and reranking that precisely narrows a wide pool of candidates.
Advanced RAG #3: Hybrid Search — Combining Vectors and Keywords
Semantic search is weak on product codes and proper nouns, and keyword search is weak on synonyms. We build BM25 keyword search and fuse it with vector search via RRF, implementing hybrid search where each side covers the weaknesses of the other.
Advanced RAG #2: Chunking Strategies That Decide Retrieval Quality
The root of retrieval failures often lies not in retrieval itself but in the step before it: chunking. We cover the limits of fixed-size splitting, structure-based chunking, handling tables and code, metadata, and parent-child chunking that searches small and feeds large.
Advanced RAG #1: Start by Finding Where RAG Goes Wrong
When RAG gives a strange answer, blindly tweaking the prompt is gambling. We start by splitting failures into retrieval failures and generation failures, then build a golden set and a baseline so every improvement becomes measurable.
AI Agent Development #7: Capstone Project — An Issue Triage Agent
Tie together every piece from the series and finish a triage agent that classifies GitHub issues and proposes labels and replies. We cover read tools, write tools behind an approval gate, and evaluation with a golden set.