contact@phuchnguyen.com
Current Learning Goals
I'm currently working towards these goals by gradually achieving small intermediate goals along the way every day:
451 study hours logged. Last studied: .
Math [why]My ultimate goal for learning math is building a strong mathematical background to contribute to the science of almost any fundamental problems in deep learning such as new optimization techniques, new architectures, ...: Working towards Metric Spaces and Geometry, Real and Complex Analysis, Measure Theory and Functional Analysis, Mean Field Theory,
18.950 Differential Geometry,
18.715 Introduction to Representation Theory,
18.705 Commutative Algebra,
18.676 Stochastic Calculus,
18.675 Theory of Probability,
18.655 Mathematical Statistics,
18.656[J] Mathematical Statistics: a Non-Asymptotic Approach,
18.615 Introduction to Stochastic Processes,
18.515 Mathematical Logic,
18.435[J] Quantum Computation, and 10 more courses.
Physics [why]My ultimate goal for learning physics is building a technical foundation to pursue sci-fi and disruptive technology projects, and bring the latest advances in science and technology to reality: Working towards Condensed Matter Physics, Fission and Fusion, Nuclear Physics, Particle Physics, General Relativity, Suns, Stars and Planets,
Lasers,
Plasma Physics,
Quantum Theory of Matter,
Cosmology,
Quantum Optics,
8.14 Experimental Physics II,
String Theory,
Introduction to Particle Accelerators,
Quantum Information,
Supersymmetry,
The Standard Model,
Black Holes,
Field Theory in Cosmology,
Superconductivity and Quantum Coherence,
Physics of Nanoelectronic Systems,
Quantum Simulation,
Quantum Information Theory,
Quantum Entanglement in Many-body Physics,
Topological Quantum Matter, and 19 more courses.
Intro
I'm currently a ML research engineer working on post-training and building pre/post-training infrastructure at 
Nous Research in 🇫🇷 Paris.
Previously, I was a research engineer at 
Hugging Face (acquired by 
NVIDIA), where I was part of the distributed training team nanotron and worked on various research reproduction effortsThese Twitter threads document my research experiments - scroll through earlier and later tweets to see the full journey and discoveries on the Hugging Face science team (FP8 research, Infini-Attention, MoE's Expert Parallelism, DoMiNo, DoReMi).
More About Me
Favorite quote
"The most counterintuitive secret about success is that it's often easier to succeed with hard things than easy things (modified version)."
I'm fascinated by complex systems. I enjoy breaking a complex thing down into parts and understanding each from the fundamentals, then asking different questions — pushing each part to the extreme to see how the design would change.
In my free time, I mostly read non-fiction - a bit of history, rationality, and a fairly diverse mix of other things; here's favorite reading. Once in a while I come across a quote that resonates with me, and I save those here. Sometimes I also collect photos that I find inspiring. I also keep a list of favorite people, favorite organizations, favorite projects, things I wish to understand deeply someday (and potentially work on in the future), and weird random things.
Selected Work
Building synthetic RL environment pipelines.
Speculative decoding DFlash training.
Post-training infra for Kimi K2 (1 trillion parameters) on 512 GPUs.
Built a new expert parallelism implementation and validated near-linear scaling across 16 nodes (128 GPUs) on Qwen3-30B-A3B model: [Pull Request]
• 50% faster than torchtitan's default expert parallelism (15,057 vs 9,930 tok/s/GPU at EP=8)
• Scales near-linearly to 16 nodes (128 GPUs, 13,856 tok/s/GPU), enabling 10T tokens/month throughput at 256 GPUs
• Implemented kernel fusion to fuse expert_outputs * router_probs to silu activation in moe FFN [link], and implemented expert_outputs * router_probs fusion to DeepEP's combining kernel [deepep fork]
• Identified a potential optimization that could bump throughput by 30%New fix coming up, translating to 21k throughput, equivalent to 30 trillion tokens in 2 months (for reference, Qwen3-30B-A3B was trained on 36T tokens)
Implemented Least-Loaded Expert Parallelism technique for MoE training - redistributes expert workloads across GPUs at runtime to eliminate straggler GPUs and imbalance memory caused by token routing imbalance during training. [Pull Request]
A benchmark for measuring general-purpose agentic assistant performance.
Found two stable FP8 pretraining recipes that pretrained a LLaMA 2 architecture in FP8 for both forward and backward passes, as well as both momentums (50% memory reduction), while matching the standard BF16 mixed-precision baseline after 100B tokens with the same pretraining learning rate as Llama 2.
Recipe 1: with architecture and optimizer modification
Recipe 2: ablated of recipe 1 without architecture modification
The following organizations use nanotron for their research:
Notable papers:
A 3D parallelism distributed training library used for smollm and fineweb.
paperback • 
podcast