Women do horrifying, bigoted shit too. They just aren't taken as seriously when they do it. Men aren't some different species from women.

Why aren't claims of women doing horrifying, bigoted shit taken seriously? It's because women very often lack the power to do horrifying, bigoted shit effectively. Men are also assumed to have such a disproportionate amount of power that it shouldn't be possible for them to be victims. As you point out, this is bad for both men and women.

Men are, generally, given more power (and often power over women)in a majority of scenarios. Men are also trained by culture and society to have certain expectations (from the society around them and the women present in it). This all comes together and results in a disproportionate quantity of bad shit being done by men (and often to women). The whole system needs fixing.