There is probably a way migrate or at least export your contacts and email history so that you save any important conversations or contacts for future reference.

In my experience, most places have been understanding and updated my contact when I said I was registered with an old email address that I no longer have access to. If it’s a government service they’ll have other information of yours that you can verify yourself with. Make a note of deleted account names in case they are needed for verification.

If you’re really uncertain, you could set up forwarding to your new email address and retain the old one for some time without using it before you delete.