HN Debrief

CS 6120: Advanced Compilers: The Self-Guided Online Course (2020)

  • Programming
  • Education
  • Developer Tools
  • Open Source

The link is to Cornell’s self-guided version of CS 6120, an online advanced compilers course built around optimizing compiler techniques rather than the basic mechanics of building a parser and code generator. People who know compiler courses framed it as the natural follow-on to an intro class. The first course gets you to a working compiler. This one gets into SSA form, dataflow, dominators, instruction selection, register allocation, garbage collection, and papers behind modern optimization work.

If you want to learn compiler backends seriously, this looks like a solid second step after an introductory book or course. For teams building JITs or ML systems, the useful lesson is to focus less on tracing as a grand architecture and more on feedback, speculation, deoptimization, and the narrow domains where tracing still pays off.

Discussion mood

Strongly positive about the course being available for free and useful as a serious compiler resource. The main friction came from two places: whether the syllabus deserves the label “advanced,” and whether its emphasis on trace compilation gives too much weight to an approach many production JIT engineers consider a dead end outside narrow domains.

Key insights

  1. 01

    What production JITs kept from tracing

    The lasting ideas from the tracing era are the machinery around adaptive optimization, not tracing itself. Type feedback, inline caches, speculation, deoptimization, fast compilers, and tiering show up across very different VM designs, so those are the concepts worth carrying into real systems and into a course that wants to teach what survived contact with production.

    If you are building or evaluating a runtime, ask first how it gathers feedback and recovers from bad guesses. Treat tracing as a historical technique unless your workload has unusually predictable control flow.

      Attribution:
    • titzer #1 #2
    • samps #1
  2. 02

    Why tracing fits ML better than JavaScript

    Tracing works in ML and numerical systems because the programs are unusually regular. Stable control flow is already rewarded by GPU execution, memory planning, and classic loop optimizations, so frameworks like JAX and PyTorch can lean on assumptions that would blow up in general application code. That makes “tracing compiler” mean something much safer in ML than it did in TraceMonkey-style dynamic language VMs.

    Do not generalize from ML compiler success to general-purpose language runtimes. If your workload has data-dependent branches, dynamic shapes, or user-written extension code, expect the nice tracing assumptions to break fast.

      Attribution:
    • titzer #1
    • achierius #1
    • jlebar #1
    • mlazos #1
  3. 03

    Advanced here means backend and optimization

    The useful definition of “advanced” is not exotic research topics like polyhedral optimization. It is the point where you stop hand-waving the backend and learn the standard SSA-based toolkit that most real language implementations actually live on. That framing also explains why many self-taught readers feel a gap after popular books. They learned parsing and AST work, then discovered the optimizer and code generation pipeline is where the discipline really begins.

    If you already know how to build a toy compiler, spend your next study cycle on SSA, dataflow, instruction selection, and register allocation. That is the material that transfers to production compiler work.

      Attribution:
    • jcranmer #1
    • j2kun #1
    • vkazanov #1
    • ferguess_k #1
    • mamcx #1
  4. 04

    Better sequence for self-study

    The recommended path was to start with a structured introductory text like Writing a C Compiler or Essentials of Compilation, then move to CS 6120. Crafting Interpreters still has value, but several people called out its blind spot directly. It leaves a gap around native code generation and optimization passes, which is exactly the gap this course is meant to fill.

    If you are mentoring someone into compiler work, pair an intro implementation book with this course instead of expecting one resource to do everything. The jump from interpreter-building to optimizing native-code compilers is real.

      Attribution:
    • Jtsummers #1 #2
    • WJW #1

Against the grain

  1. 01

    LuaJIT is a real counterexample

    LuaJIT complicates any blanket claim that trace compilation is finished. It is not a classroom curiosity. It ships widely and performs well enough at large scale to prove that tracing can win when the language and workload cooperate. That does not rescue tracing as a universal design, but it does keep the obituary from being clean.

    If your language runtime looks more like Lua than the open web, do not discard tracing on reputation alone. Benchmark against your actual workload before inheriting JavaScript VM conclusions.

      Attribution:
    • mikemike #1
    • titzer #1
  2. 02

    Maybe optimizing compilers are the wrong layer

    One comment rejected the usual compiler-course framing entirely and argued that heavyweight optimization is often wasted work, fragile, and hard to reason about. The proposed alternative was tighter interop between high-level and low-level code, plus source-level rewrites, possibly with LLM assistance, instead of repeating opaque optimizer passes on every build. That is far from the mainstream view here, but it surfaces a real pain point: long compile times and unpredictable optimization behavior.

    If build latency or optimizer instability is hurting your team, look for ways to isolate hot paths and make performance-critical code explicit. You do not need to accept whole-program heavy optimization everywhere just because modern toolchains can do it.

      Attribution:
    • norir #1

In plain english

AST ↩
Abstract Syntax Tree, a tree representation of a program’s structure after parsing.
dataflow ↩
A family of compiler analyses that reason about how information moves through a program’s control flow graph.
deoptimization ↩
A mechanism that lets a runtime abandon optimized code and safely fall back to a more general version when assumptions stop being true.
GPU ↩
Graphics Processing Unit, a highly parallel processor commonly used for machine learning and numerical workloads.
inline caches ↩
Small runtime caches that remember how dynamic operations like method calls were resolved, speeding up repeated executions of the same pattern.
instruction selection ↩
The compiler phase that maps intermediate operations to specific machine instructions.
JAX ↩
A numerical computing and machine learning system from Google that traces Python functions to optimize and run them efficiently.
JIT ↩
Just-In-Time compilation, where code is compiled during program execution instead of fully ahead of time.
LLM ↩
Large language model, a machine learning model trained to predict and generate text and often used for coding, chat, and document tasks.
ML ↩
Machine learning, a class of software techniques that train models from data and often rely on specialized compiler stacks for performance.
polyhedral optimization ↩
A set of advanced loop optimization techniques that model iteration spaces mathematically to reorder and parallelize computation.
PyTorch ↩
A widely used machine learning framework for building and running neural networks.
register allocation ↩
The compiler phase that decides which program values live in the CPU’s limited set of fast registers.
SSA ↩
Static Single Assignment, an intermediate representation where each variable is assigned exactly once, which makes many compiler analyses and optimizations easier.
tiering ↩
Using multiple execution levels, such as interpretation and one or more compilers, and moving code between them based on runtime behavior.
TraceMonkey ↩
A former JavaScript engine in Firefox that used trace-based just-in-time compilation.
type feedback ↩
Runtime information about the types a program actually uses, which a dynamic compiler can exploit to generate faster specialized code.
VM ↩
Virtual machine, the software layer that executes a programming language runtime or bytecode system.

Reference links

Compiler course and syllabus links

Intro books and self-study resources

Prior discussions and collections