where tolerance is some fairly small number, maybe 5

If this is to be part of a user-submitted form, maybe set a sane default, but let the user decide exactly where to set the limit -- because it will change depending on the particular image being worked on. Some need a high tolerance because the white background isn't very close to true white, others need a low tolerance because the foreground you want to keep has highlights that are too close to white. So you give the form an extra control slider to change the tolerance between, say, 1 and 30, with it defaulting to, say, 5.