I ran OpenSpeedTest from my PC to my Raspberry Pi 4B, both connected via LAN to my WiFi router. The left screenshot shows the speedtest via local_ip:3000, and I'm getting the expected 1 Gbps up/down.
The right screenshot shows the speedtest via https://speed.mydomain.com/. I'm confident that the connection from my PC to my home server is routed internally and not through the internet because my lowest ping to the nearest Speedtest server (my own ISP) on speedtest.net is 6ms, and my internet speed is 100 Mbps up/down. So the traffic must be routing internally.
Is there typically such a massive difference between using http://local_ip:3000 and https://speed.mydomain.com/?
Additional context: The speedtest server is running via Docker Compose. I'm using Nginx (native, not Docker) to access these services from outside my network.
The port is forwarded from your router to the pi, right? If so, you could test for the router as the bottleneck using the router's WAN side IP address as the target.
This should give you a good data point for comparison. If it's also slow then you can focus on the router performance. Some are slow when doing hairpin NAT.