The opposite result (predicting sex but not gender) would also be interesting but less so
I disagree. It could be wildly interesting if somebody born a male got a scan and it revealed a female brain. Dunno if "anti-trans" people would agree then that a sex-change is valid or if they'd disagree and start finding other excuses.
The more data you have, the more confident you can be that the resulting categorisation is correct. If you're saying this is incorrect, I disagree with you. If you're saying absolute confidence that the categories themselves are correct is impossible, then I agree with you