Storage Scaling Bottlenecks in Large LLM Inference Clusters
KV cache grows without bounds, forcing engineers to choose which storage tier fails first.
Marcus Dell'Acqua
Staff Writer, Systems & Architecture
Marcus Dell'Acqua
Marcus worked as a distributed systems engineer at two Series B startups before transitioning to full-time reporting, giving him an unusually practical eye for the gap between architectural theory and production reality. He has been covering GPU scheduling, kernel optimization, and cluster design since 2017.
4 stories
Data Preprocessing Pipeline Design for LLM Training Throughput
Careful stage-by-stage design determines whether GPUs train efficiently or starve for tokens.
Marcus Dell'Acqua
High-Throughput Checkpoint Design for LLM Training at Scale
Failures at massive scale demand checkpoints so frequent that I/O overhead dominates training time.
Marcus Dell'Acqua
BF16 and FP8 Compression Strategies for KV Cache Storage
Compressing KV cache with FP8 halves memory while preserving accuracy.
Marcus Dell'Acqua