HN Debrief

Branchless Rust: Making a Filter 4x Faster by Removing an If

  • Programming
  • Hardware
  • Developer Tools
  • AI

The post benchmarks a simple Rust filter over `f64` values and shows that the slow case was not allocation but an unpredictable branch. Rewriting the loop so it always writes to `out[n]` and only conditionally increments `n` flattened the runtime across inputs and made the 50 percent keep-rate case much faster. The catch is that this version preallocates output space for the entire input and performs many writes that later get overwritten, so it buys steadier latency by spending more memory bandwidth and more output capacity than the final result may need.

If you own a hot loop, benchmark branch predictability and memory traffic separately before you celebrate a branchless rewrite. For bigger wins, look past scalar tricks to SIMD stream compaction, and do not assume the compiler will discover that path for you.

Discussion mood

Mostly positive on the performance lesson, with a skeptical streak about the article’s writing and about claims that source-level branchless style is itself the optimization. People liked the explanation of branch misprediction, then quickly shifted to more serious concerns like SIMD, memory bandwidth, compiler behavior, and extra memory usage.

Key insights

  1. 01

    AVX-512 compress is the real upgrade

    AVX-512 turns this into a proper vector compaction problem instead of a scalar bookkeeping trick. Using `_mm512_maskz_compress_pd` lets the CPU pack only the passing values from each 8-lane chunk, which cut runtime another roughly 25 percent to 60 percent in one posted benchmark. That also exposed an awkward hardware detail. The memory-destination compress store intrinsic can be unexpectedly slow on Zen 4, so the faster version does a full store into a safely oversized buffer and leans on the store buffer to hide the overlap cost. Open-source compilers are not expected to discover this pattern on their own.

    If this loop actually matters, stop at branchless scalar code only as a midpoint. Handwritten SIMD compaction can still buy a large gain, but you need per-CPU benchmarking because the obvious intrinsic may be the wrong one.

      Attribution:
    • anematode #1 #2
    • dzaima #1
  2. 02

    This is stream compaction, not a Rust trick

    The underlying problem already has a name and a larger body of work. Stream compaction methods use prefix scan to compute each kept element’s destination index first, then perform a clean gather or scatter pass without the loop-carried dependency on `n`. That framing matters because it explains why the posted trick works at all. It is sidestepping a dependency chain, not just deleting an `if`.

    Search for stream compaction techniques when you hit filter-like hotspots in data pipelines, graphics, or analytics code. You may find parallel or vector-friendly algorithms that scale better than local source rewrites.

      Attribution:
    • bjourne #1 #2
  3. 03

    Branchless source does not guarantee branchless machine code

    Writing `n += (x > threshold) as usize` looks branchless in Rust, but that does not mean LLVM will emit the special machine instructions you hope for, or even that every target treats it as branch-free. Commenters called out the blog’s implied model as cargo culting. Compilers already know many equivalent scalar forms, and profile-guided optimization is not designed to invent this kind of extra-write transformation from branch statistics alone. The relevant boundary is not syntax. It is whether the backend recognizes a profitable hardware pattern such as compress, select, or vectorized masking.

    Check generated assembly and counters before you attribute a speedup to a source idiom. Keep code readable until measurement proves that a specific low-level form survives compilation and wins on your target CPUs.

      Attribution:
    • flohofwoe #1
    • claudetard #1
    • Sesse__ #1
  4. 04

    Memory footprint is a real tradeoff

    The trick only works cleanly because the output buffer is sized to the full input length, even when almost everything gets filtered out. That is fine for `f64` in a toy benchmark, but it changes the memory profile from output-proportional to input-proportional. Trying to recover memory efficiency without reintroducing another unpredictable branch gets ugly fast, which is why the naive elegant version is hiding a meaningful systems tradeoff.

    Do not transplant this pattern blindly into sparse filters or large-object pipelines. If rejection rates are high or memory is tight, the extra capacity and writes may cost more than the mispredicted branch.

      Attribution:
    • khuey #1 #2
  5. 05

    Predictable branches can still win unless vectorization opens up

    Several comments sharpened the usual rule. Branchless code beats branches when outcomes are hard to predict, like random thresholds or random sort pivots. When the branch is heavily biased, a modern predictor makes it almost free, so removing it can just add unnecessary writes. The interesting exception is when a branch prevents autovectorization. Then even a highly predictable branch can lose because the branchless form enables wide SIMD over many elements at once, as in the GitHub case-folding example people linked.

    Treat branchless rewrites as a two-part question. First ask whether prediction is poor. Then ask whether the rewrite unlocks SIMD. If neither is true, keep the `if`.

      Attribution:
    • nvme0n1p1 #1
    • phire #1
    • vlovich123 #1
    • adrian_b #1
    • amiga386 #1

Against the grain

  1. 01

    The overwrite trick is the clever part

    One short reaction cut through the theory and focused on what is actually novel in the code. The branchless speedup rests on repeatedly writing to the current output slot and only advancing the index when the predicate passes, so losing candidates simply overwrite the same location until a keeper arrives. That observation makes the algorithm easier to reason about than the article’s drama did.

    If you borrow this idea, document the overwrite invariant right in the code. Future readers need that mental model more than a lecture about branch predictors.

      Attribution:
    • rabiescow #1
  2. 02

    The article still taught the intended lesson

    Not everyone found the prose style disqualifying. One commenter said the post was fun and useful, and that curiosity about the optimization mattered more than AI-detection games. That is a fair corrective because the technical core was solid enough to help people who had not seen branch prediction examples before.

    Do not let style debates cause you to miss a good benchmark or a useful trick. Extract the mechanism, verify it yourself, and move on.

      Attribution:
    • HackerThemAll #1

In plain english

autovectorization ↩
A compiler optimization that automatically turns ordinary loops into SIMD code when it can prove the transformation is safe and profitable.
AVX-512 ↩
Advanced Vector Extensions 512, an x86 instruction set extension for very wide SIMD operations on modern CPUs.
f64 ↩
A 64-bit floating-point number type used for double-precision arithmetic.
intrinsics ↩
Low-level language functions that map closely to specific CPU instructions.
LLVM ↩
A compiler infrastructure project that provides an intermediate representation and code generation tools for many CPU architectures.
Rust ↩
A systems programming language focused on memory safety and performance without using a garbage collector.
SIMD ↩
Single instruction, multiple data, a CPU feature that lets one instruction process several data values at once.
store buffer ↩
A CPU hardware queue that lets stores complete later so execution can continue without waiting for memory immediately.
Zen 4 ↩
A generation of AMD CPU microarchitecture used in recent desktop and server processors.

Reference links

SIMD and low-level optimization references

Related performance articles

AI writing detection and provenance links