The first time you read the source of a tensor compiler, everything looks alien: graphs instead of basic blocks, "lowering" instead of instruction selection, "fusion" instead of, well, anything you remember from the dragon book.
Then you squint.
Operator fusion is loop fusion. Layout selection is register allocation's bigger sibling. Graph rewrites are peephole optimizations over a coarser IR. Even autotuning is just a very expensive cost model that refuses to commit.
The same shape, three layers deep
A classical compiler pipeline and an ML compiler pipeline both reduce to the same loop:
- Normalize the program into an IR you can reason about.
- Apply semantics-preserving rewrites guided by a cost model.
- Lower to something a machine will actually run.
The differences are real — tensor IRs are purely functional, shapes carry information that C types never did, and the target might be a systolic array rather than a register machine. But the differences live inside the phases, not between them.
Why this framing matters
If you treat an ML compiler as a compiler, thirty years of literature suddenly applies. Dominators, dataflow lattices, polyhedral models — none of it expired. The field keeps rediscovering these under new names, and the fastest way to be productive is to notice when a "novel" pass is an old friend.