This is an automated archive made by the Lemmit Bot.
The original was posted on /r/stablediffusion by /u/beighto on 2024-08-21 17:52:03+00:00.
Don't read the fix, skip to the edits below.
~~The problem originated in commit b09c24e when Illyasviel introduced the fp16_fix.~~ ~~You can fix the fix by editing the latest commit (31bed67 as of 8/21/24):~~
~~From backend/nn/flux.py remove lines:~~
~~from backend.utils import fp16_fix~~ ~~txt = fp16_fix(txt)~~ ~~x = fp16_fix(x)~~ ~~fp16_fix(x)~~
~~From backend/utils.py remove function block:~~
~~def fp16_fix(x):~~ ~~# An interesting trick to avoid fp16 overflow~~ ~~# Source: [~~issues/1114]~~()~~ ~~# Related: https://github.com/comfyanonymous/ComfyUI/blob/~~f1d6cef71c70719cc3ed45a2455a4e5ac910cd5e/comfy/ldm/flux/layers.py#L180~~ ~~if x.dtype == torch.float16: ~~ ~~return x.clip(-16384.0, 16384.0) ~~ ~~return x~~
~~That's it! I went from 36s/it @ 1024x768 to 13s/it with nf4, 14s/it with Q4 gguf, and 14s/it with Q8.~~ ~~Hopefully this will get removed or fixed in future releases to save us GPU poor folk.~~
~~I tried to find a fix for this in ComfyUI as well, but that one is broken from the start.~~
~~Edit: I'm having trouble recreating this from the latest commit. It might need the pip requirements from the aadc0f0 commit and upgrade from there. Has anybody else had any luck with this fix?~~
Edit2: Illyasviel has been busy today. It looks like he fixed the issue without removing the fp16_fix. Per commit notes:
change some dtype behaviors based on community feedbacks
only influence old devices like 1080/70/60/50. please remove cmd flags if you are on 1080/70/60/50 and previously used many cmd flags to tune performance
So take those flags off. I'm getting 20s/it now. Going to keep trying for that 14s/it again with the latest commit.
Edit 3: ComfyUI fixed theirs too! Per commit notes:
commit a60620dcea1302ef5c7f555e5e16f70b39c234ef (HEAD -> master, origin/master, origin/HEAD) Author: comfyanonymous comfyanonymous@protonmail.com Date: Wed Aug 21 16:38:26 2024 -0400
Fix slow performance on 10 series Nvidia GPUs.
commit 015f73dc4941ae6e01e01b934368f031c7fa8b8d Author: comfyanonymous comfyanonymous@protonmail.com Date: Wed Aug 21 16:17:15 2024 -0400
Try a different type of flux fp16 fix.
I'm getting 20s/it on Comfy too. What a day for updates!