You’re explaining yourself fine, you’re just mistaken about the way Cloudflare tunnels work. You’re confusing concepts between a L4 proxy and a L7 proxy.

What I'm saying is, if the client and endpoint (server) talk in an encrypted protocol, then cloudflare cannot MiTM the data, only the IP headers.

This is not the case. You are under the mistaken impression that CF tunnels work like a L4 tunnel, proxying a TCP stream from client to server, allowing you to maintain an encrypted TLS session from client to server. That would be closer to what Tailscale Funnel does (Which I’d advocate for). CF tunnels do not work this way. Cf tunnels work more like a L7 proxy. Your client and your server never talk, so there is no encrypted protocol between them. There is only encryption between you and Cloudflare, and then Cloudflare and your backend server. Cloudflare can and does MitM the data AND the IP headers.

This is similar if you were to connect to any ol' website over an ISP's network. If your session is not HTTPS, then your application data can be read.

You cannot establish an HTTPS connection with your application from your client. You establish an HTTPS connection with Cloudflare, which gives them plaintext access to all the data you send through them.

You can have encrypted sessions inside of CF tunnel-network-tunnel.

To be clear, no you can’t. This is your misunderstanding. At least, you can’t with Cloudflare tunnels. Cloudflare may offer a TCP proxy service, which is what you’re confusing CF tunnels with, if you sign up for an enterprise plan, but you don’t get that functionality in their free plan which OP, and self hosters in general would be using.