If you want to keep HAOS as the OS (and the nice update/backup experience), there's a middle option nobody's mentioned yet: run the extra apps as add-ons instead of a separate Docker setup. There are community add-on repositories (alexbelgium's is the big one) that package Sonarr, Radarr, Prowlarr, qBittorrent/Deluge and Jellyfin as HA add-ons, so they live inside the Supervisor's containers and get included in HA backups if you want.

A few caveats for your plan:

• Do the SSD move first. Torrenting plus a media library on a 256 GB SD card will wear it out and make HA itself slow, and HAOS on a Pi 5 boots fine from NVMe or USB SSD.
• The Pi 5 dropped the hardware H.264 encoder the Pi 4 had, so Jellyfin transcoding is pure CPU. As iBlessi said, it's fine for direct play to one device; avoid anything that forces a transcode.
• Keep media on a separate drive or share mounted through HA's Storage settings, so a full HA backup doesn't try to include your library.