Once identified, I usually only use the scrolls of upgrade after the first boss.
In the best of cases I can withhold them for so long that I can skip an armor or weapon class early by lowering its strength reqirement through upgrading.
So that's what I would try while you're on Floor 4. The first Boss is manageable the way you are set up now. Keep those scrolls until you find a better weapon.

