Data Preprocessing Pipeline Design for LLM Training Throughput
Fixing bottlenecks stage-by-stage keeps GPUs fed with tokens.
Marcus Dell'Acqua
Section
Systems Design
5 stories in Systems Design.
High-Throughput Checkpoint Design for LLM Training at Scale
Failures at massive scale demand checkpoints so frequent that I/O overhead dominates training time.
Marcus Dell'Acqua
Tiered Memory Architecture for Long-Context LLM Inference
A tiered memory hierarchy solves long-context LLM serving bottlenecks.
Celeste Oforiwaa
Paged Attention and Memory Efficiency in Production LLM Serving
How PagedAttention solved the memory crisis that made LLM serving uneconomical.
Celeste Oforiwaa
BF16 and FP8 Compression Strategies for KV Cache Storage
FP8 cuts KV cache memory in half while keeping accuracy loss under half a percentage point.
Marcus Dell'Acqua