AI
LLM Hallucinations: Causes and Practical Countermeasures
A summary of hallucination, where LLMs confidently generate plausible falsehoods: why it follows inevitably from next-token probability prediction, the common types from factual errors to fabricated citations and code API hallucinations, the measures that actually reduce it in practice — RAG, enforced citations, tool use, structured output validation, evaluation pipelines — and the operational view of designing human review points on the premise that it can never be fully eliminated.
Embeddings and Vector Search: How Semantic Search Actually Works
A single-post summary of embeddings and vector search, the foundation of RAG and semantic search: the concept of turning text into numeric vectors, measuring semantic closeness with cosine similarity, the trade-off between exhaustive kNN and approximate ANN indexes, choosing storage between pgvector, dedicated vector databases, and search-engine extensions, where embeddings fall short on exact keyword matching and why hybrid search is the practical standard, plus operational costs like full re-indexing on model changes.
Tokens and Context Windows: The Units That Decide LLM Cost and Limits
A single-post summary of tokens and context windows, the units behind every LLM price sheet and spec table: how text gets split into tokens, why non-English languages cost more tokens, the asymmetric input/output pricing and why long conversations get expensive fast, what the context window actually limits, the performance traps of very long context, and practical mitigations from prompt caching to history management.
MCP (Model Context Protocol) Explained: The Standard for AI Tool Use
A single-post summary of MCP, the protocol that keeps appearing in agent articles: how a standard interface turns the N-by-M tool-integration problem into N-plus-M, its relationship to function calling, hosts, clients, servers and the tools/resources/prompts primitives, adoption status and foundation governance as of 2026, security cautions from third-party server trust to prompt injection, and when adopting it is worth it — and when it is not.
RAG vs Fine-Tuning vs Long Context: Getting Knowledge Into an LLM
A working standard for the three ways to get your data into an LLM: the structural difference between retrieving and injecting (RAG), modifying weights (fine-tuning), and stuffing everything in with caching (long context), the boundary that fine-tuning teaches behavior rather than knowledge, a selection order driven by knowledge size and update frequency, and the practical setup that combines all three.
Choosing a GPU Cloud: Hyperscaler vs Specialist GPU Cloud vs Serverless Inference
A working standard for where to rent GPUs for AI workloads: why H100 hourly rates run 2〜5x apart between hyperscalers (AWS) and specialist GPU clouds (Lambda, RunPod), what you accept when spot drops below half, where serverless inference removes GPU management with per-token billing, and the integration, network, and regulatory requirements that flip the price difference.
LLM Quantization Compared: FP16, INT8, INT4 and GGUF, AWQ, GPTQ
A working standard for choosing an LLM quantization method: how precision (FP16, INT8, INT4) halves VRAM, the math that takes a 70B model from 140GB to 35GB, where the three formats (GGUF, AWQ, GPTQ) diverge by use (local, GPU serving, accuracy), where INT4 degrades quality on reasoning and code tasks, and how to match the format to the serving framework.
vLLM vs Ollama vs SGLang: Choosing an LLM Serving Framework
A working standard for choosing an LLM serving framework when self-hosting: where local execution (Ollama) and production throughput (vLLM, SGLang) diverge, why PagedAttention and continuous batching open a 3〜4x throughput gap, the 2026 shift with TGI entering maintenance mode, differences in quantization and structured-output support, and a selection order by team size.
LLM API vs Self-Hosting: Doing the Break-Even Math
A working standard for deciding whether to use an LLM API or self-host: the structural difference between per-token and per-GPU-hour billing, how to compute break-even from the tokens one GPU produces in a day, where utilization, batching, and quantization move the crossover, and the factors outside price — data privacy, latency, operational burden — that decide when the math is close.
LLM App Operations #7: Capstone — Taking the Document Q&A Bot to Production
We tie the five pillars of this series into an operations checklist and apply it to the document Q&A bot. Turning on instrumentation, routing, caching, batching, reliability, and security one by one, we watch how per-request cost and stability change, and close out the AI track that spans four series.
LLM App Operations #6: Security — Prompt Injection and Data Boundaries
Prompt injection is an attempt to change an app's behavior through input text, and in the era of RAG and agents it rides in through documents and tool results. We cover layered defenses instead of a single line, minimizing tool permissions, output validation, and the data boundaries of logging.
LLM App Operations #5: Reliability — Rate Limits, Retries, Fallbacks
429 and 529 are not outages, they are daily life. We build a structure that keeps running: how rate limits work (RPM and token limits), retries that respect retry-after, timeouts and streaming, and fallbacks for when nothing else works (model downgrade, queuing, graceful failure).