HN Debrief

Should you normalize RGB values by 255 or 256?

  • Graphics
  • Programming
  • Hardware
  • Developer Tools

The post asks a question that sounds pedantic but touches a real modeling choice in graphics and signal processing. An 8-bit RGB channel stores integers 0 through 255. If you turn those into floats, the common approach is `x / 255`, which preserves exact endpoints so black is 0.0 and white is 1.0. The alternative is to treat each integer as the center of one of 256 equal bins, which leads to a `+0.5` and divide-by-256 style mapping. That model is mathematically tidy because every bucket has the same width, but it means neither 0 nor 255 maps to the endpoints. The conversation settled on a blunt conclusion for normal image work: use 255. People kept coming back to the same practical reason. A lot of graphics math quietly depends on additive and multiplicative identities staying intact. If 0 is no longer exactly zero and 1 is no longer exactly one, masking, alpha compositing, thresholding, and other common operations start producing tiny but real artifacts.

If your software reads or writes ordinary 8-bit images, map 0→0.0 and 255→1.0, then dither on the way back to 8-bit. The edge-case math only becomes a real design choice when you are defining a quantizer, matching hardware transfer characteristics, or moving between formats with explicit encoding rules.

Discussion mood

Mostly positive and a little amused. People liked the post because it exposed a subtle quantization issue they had mostly handled by habit, but the dominant reaction was still pragmatic: for real graphics code, divide by 255, keep endpoints exact, and spend your effort on color-space correctness and dithering instead of obsessing over an almost invisible denominator difference.

Key insights

  1. 01

    Exact endpoints keep image math sane

    Keeping 0 mapped to 0.0 and 255 mapped to 1.0 is not just aesthetic. A lot of image code assumes exact zeros and ones for masks, alpha, premultiplication, and threshold tests. If decode produces slightly nonzero black or slightly subunit white, those identities disappear and tiny leaks show up as halos, faint transparency, or failed mask cutouts. One commenter went further and said that once you stop treating the mapping as preserving those identities, you should stop pretending a cute formula is enough and treat byte-to-float conversion as an explicit lookup defined by the format.

    If your pipeline touches alpha, masks, or compositing, test for endpoint exactness before entertaining alternate quantizers. A mathematically prettier decode rule is not worth breaking assumptions buried in downstream code.

      Attribution:
    • pixelesque #1
    • yuriks #1
    • herf #1
    • HelloNurse #1
  2. 02

    The real answer belongs to the format

    The useful framing here is not "which denominator is right" but "what did this format mean by these integers." In scientific computing terms this is node-centered versus cell-centered sampling. In graphics terms the color space, OETF, and EOTF define how stored integers relate to physical or scene values. Several commenters argued that when the context is specified, the argument is already over. When it is not, sRGB is the default assumption for 8-bit color, even if the exact integer quantization semantics are not always written down as clearly as the continuous transfer curve.

    Do not hard-code a normalization rule in isolation. Tie byte-to-float conversion to declared format metadata, or at least to a documented default like sRGB, so the semantics live with the data instead of in folklore.

      Attribution:
    • glkindlmann #1 #2
    • wyager #1
    • mark-r #1
  3. 03

    Dither is the visible quality lever

    Several experienced commenters said the denominator question is far less important than how you quantize back to 8-bit. The visible artifact in gradients is banding, and the fix is dithering, ideally triangular or error-diffused noise before rounding. That shifts the practical advice from choosing a philosophically perfect bucket model to making quantization error less structured and less visible to the eye.

    If you are shipping images, add dithering before 8-bit export and measure the output visually. You will get a bigger quality gain from that than from changing 255 to 256 anywhere in the pipeline.

      Attribution:
    • Sesse__ #1
    • orlp #1
    • virtualritz #1
  4. 04

    Hardware quantizers make the abstraction real

    The issue stops being academic when integer codes directly drive analog levels. A commenter building VGA output on a microcontroller explained that 3 red bits, 3 green bits, and 2 blue bits produce different voltage ladders, so nominally neutral colors can pick up blue or yellow tint because the channels do not land on matching analog steps. Others noted gamma correction changes perceptual spacing, and that display techniques like frame rate control can hide some of the mismatch by time-dithering between levels.

    When you are designing hardware interfaces or fixed-point display paths, write down the actual electrical or optical transfer curve. The byte values are only half the story. The ladder, gamma, and dithering scheme decide what users really see.

      Attribution:
    • moefh #1
    • chongli #1
    • spider-mario #1
    • londons_explore #1
    • account42 #1
  5. 05

    ADC conventions are messy and mostly standardized

    Engineers pulled the same argument into analog-to-digital converters and showed why the article's clean taxonomy gets messy in real datasheets. Some ADCs document transfer functions that cannot represent full-scale positive input. Others show idealized mid-tread behavior while the real device has offset and gain errors larger than one count anyway. The practical lesson was not that ADCs prove one side right, but that conventions are partly historical and partly standardized for testing, not pure math. Once hardware nonlinearity and calibration error dominate, the last-bit argument stops mattering.

    If you are converting sensor or ADC codes, follow the device datasheet and calibration model instead of importing graphics intuitions. The right formula is the one that matches the hardware's specified transfer function and error budget.

      Attribution:
    • fps-hero #1 #2
    • kroeckx #1
    • jason_s #1 #2
  6. 06

    Performance is not a serious reason to prefer 256

    The old instinct that divide-by-256 is cheaper because it turns into a bit shift did not survive scrutiny. For float pipelines, constant division is usually a multiply by a reciprocal, GPUs make floating multiply very cheap, and modern vector units can issue many multiplies per cycle. Commenters who work on renderers argued that if `>> 8` looks dramatically faster than `* (1.0/255.0)`, the benchmark is probably measuring a different code path, compilation issue, or cache effect. Several also pointed out that once you do the color math correctly in linear space, the supposed shortcut no longer matches the real work anyway.

    Do not choose a quantization convention for micro-optimization reasons without profiling the full pipeline. On modern CPUs and GPUs, color-space conversion and memory movement dominate long before a reciprocal constant does.

      Attribution:
    • Tuna-Fish #1
    • Sesse__ #1
    • exyi #1
    • virtualritz #1

Against the grain

  1. 01

    Headroom can be a valid modeling choice

    One commenter defended the centered-bin view on perceptual grounds. In ordinary standard dynamic range imagery, channel values do not represent absolute photon counts and scene black is rarely physical zero. On that view, insisting that the stored extrema must decode to exact 0.0 and 1.0 bakes in a bias that the data does not really warrant. Historic video systems leaving headroom and footroom show that endpoint exactness is not sacred in every imaging regime.

    If you are defining a new format for display-referred data rather than interoperating with existing image code, it is reasonable to ask whether exact endpoints are semantically meaningful. Just be explicit, because most software will still assume they are.

      Attribution:
    • herf #1
  2. 02

    Equal-width bins are mathematically coherent

    A long exchange argued that the article is not making a mistake at all. It is exposing that quantization needs two linked choices: how continuous values map into bins, and which representative value you emit when decoding a bin. From that angle, equal-width bins across the full continuous interval are perfectly consistent. The cost is not wrongness but smaller endpoint buckets if you insist on exact 0 and 1 when decoding. That reframes divide-by-255 as a convenience tradeoff, not a law of nature.

    When you design a quantizer from scratch, document both encode and decode together. Most bugs come from mixing conventions, not from choosing one coherent convention over another.

      Attribution:
    • alterom #1 #2 #3

In plain english

EOTF ↩
Electro-optical transfer function, the rule that converts stored image code values back into displayed light.
OETF ↩
Opto-electronic transfer function, the rule that converts scene light into stored image code values.
RGB ↩
Red, green, and blue image data from a standard color camera.
sRGB ↩
Standard Red Green Blue, the long-standing default color space for the web and many consumer devices.
VGA ↩
Video Graphics Array, a graphics standard for IBM PCs that introduced 256-color packed-pixel modes and other display features.

Reference links

Quantization and graphics references

Hardware and display examples

  • pico-vga-8bit-demo
    Concrete example of generating VGA with limited per-channel bits where quantization choices become visible
  • uops.info mulss table
    Cited to show floating-point multiply latency and throughput on modern CPUs
  • uops.info shr table
    Used in the performance side thread comparing shifts with floating-point multiplies

Color and video standards

Tools and libraries