#GPU
4 posts
Sizing Servers for Self-Hosted LLMs — From VRAM Math to Concurrent Users
A repeatable order for estimating the GPU scale a self-hosted LLM needs: computing VRAM from parameter count and precision, the KV cache and its relationship to concurrency, memory bandwidth and token speed, example configurations by scenario, and the break-even against API pricing.
AI Training Servers vs. Inference Servers — Where the Requirements Diverge
Why the same "AI server" question has two very different answers: batch versus request-serving workload shapes, GPU interconnect versus memory bandwidth as the bottleneck, precision and quantization, cost structures, and the trap of trying to share one fleet for both.
How to Read NVIDIA GPU Names — Architecture Generations and Product Lines
Reading H100, B200, GB300, L40S, and RTX 5090 as a system rather than a code: the scientist-named architecture axis and the datacenter/inference/consumer product-line axis, what the digits and prefixes mean, and the generation landscape and roadmap as of mid-2026.
GPU Servers vs. General Servers — Architecture, Power, Cooling, Price
Where a GPU server stops being a regular server with a graphics card in it: the inverted architecture and interconnects, power density an order of magnitude higher and the arrival of liquid cooling, the price structure, the driver stack, and why utilization management becomes the job.