42
Should you normalize RGB values by 255 or 256?
(30fps.net)
Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!
Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.
Hope you enjoy the instance!
Rules
Follow the wormhole through a path of communities !webdev@programming.dev
My idempotent claim was too broad and wrong, up to aliasing that is reversible.
I think I’ve been talking about this differently than you, when you get 8bit into, 255 in/out is correct.
When generating data I do it in the natural [0,1] space, then map to whatever domain I’m quantizing for.
In that case, for N bins you divide by N+1 (and round however you want via correction) so 256. I was probably wrong earlier around this, 256 is correct for quantizing that way.