#AI
47 posts
LLM Hallucinations: Causes and Practical Countermeasures
A summary of hallucination, where LLMs confidently generate plausible falsehoods: why it follows inevitably from next-token probability prediction, the common types from factual errors to fabricated citations and code API hallucinations, the measures that actually reduce it in practice — RAG, enforced citations, tool use, structured output validation, evaluation pipelines — and the operational view of designing human review points on the premise that it can never be fully eliminated.
Embeddings and Vector Search: How Semantic Search Actually Works
A single-post summary of embeddings and vector search, the foundation of RAG and semantic search: the concept of turning text into numeric vectors, measuring semantic closeness with cosine similarity, the trade-off between exhaustive kNN and approximate ANN indexes, choosing storage between pgvector, dedicated vector databases, and search-engine extensions, where embeddings fall short on exact keyword matching and why hybrid search is the practical standard, plus operational costs like full re-indexing on model changes.
Tokens and Context Windows: The Units That Decide LLM Cost and Limits
A single-post summary of tokens and context windows, the units behind every LLM price sheet and spec table: how text gets split into tokens, why non-English languages cost more tokens, the asymmetric input/output pricing and why long conversations get expensive fast, what the context window actually limits, the performance traps of very long context, and practical mitigations from prompt caching to history management.
MCP (Model Context Protocol) Explained: The Standard for AI Tool Use
A single-post summary of MCP, the protocol that keeps appearing in agent articles: how a standard interface turns the N-by-M tool-integration problem into N-plus-M, its relationship to function calling, hosts, clients, servers and the tools/resources/prompts primitives, adoption status and foundation governance as of 2026, security cautions from third-party server trust to prompt injection, and when adopting it is worth it — and when it is not.
RAG vs Fine-Tuning vs Long Context: Getting Knowledge Into an LLM
A working standard for the three ways to get your data into an LLM: the structural difference between retrieving and injecting (RAG), modifying weights (fine-tuning), and stuffing everything in with caching (long context), the boundary that fine-tuning teaches behavior rather than knowledge, a selection order driven by knowledge size and update frequency, and the practical setup that combines all three.
Choosing a GPU Cloud: Hyperscaler vs Specialist GPU Cloud vs Serverless Inference
A working standard for where to rent GPUs for AI workloads: why H100 hourly rates run 2〜5x apart between hyperscalers (AWS) and specialist GPU clouds (Lambda, RunPod), what you accept when spot drops below half, where serverless inference removes GPU management with per-token billing, and the integration, network, and regulatory requirements that flip the price difference.
LLM Quantization Compared: FP16, INT8, INT4 and GGUF, AWQ, GPTQ
A working standard for choosing an LLM quantization method: how precision (FP16, INT8, INT4) halves VRAM, the math that takes a 70B model from 140GB to 35GB, where the three formats (GGUF, AWQ, GPTQ) diverge by use (local, GPU serving, accuracy), where INT4 degrades quality on reasoning and code tasks, and how to match the format to the serving framework.
vLLM vs Ollama vs SGLang: Choosing an LLM Serving Framework
A working standard for choosing an LLM serving framework when self-hosting: where local execution (Ollama) and production throughput (vLLM, SGLang) diverge, why PagedAttention and continuous batching open a 3〜4x throughput gap, the 2026 shift with TGI entering maintenance mode, differences in quantization and structured-output support, and a selection order by team size.
LLM API vs Self-Hosting: Doing the Break-Even Math
A working standard for deciding whether to use an LLM API or self-host: the structural difference between per-token and per-GPU-hour billing, how to compute break-even from the tokens one GPU produces in a day, where utilization, batching, and quantization move the crossover, and the factors outside price — data privacy, latency, operational burden — that decide when the math is close.
Sizing Servers for Self-Hosted LLMs — From VRAM Math to Concurrent Users
A repeatable order for estimating the GPU scale a self-hosted LLM needs: computing VRAM from parameter count and precision, the KV cache and its relationship to concurrency, memory bandwidth and token speed, example configurations by scenario, and the break-even against API pricing.
AI Training Servers vs. Inference Servers — Where the Requirements Diverge
Why the same "AI server" question has two very different answers: batch versus request-serving workload shapes, GPU interconnect versus memory bandwidth as the bottleneck, precision and quantization, cost structures, and the trap of trying to share one fleet for both.
How to Read NVIDIA GPU Names — Architecture Generations and Product Lines
Reading H100, B200, GB300, L40S, and RTX 5090 as a system rather than a code: the scientist-named architecture axis and the datacenter/inference/consumer product-line axis, what the digits and prefixes mean, and the generation landscape and roadmap as of mid-2026.