You're missing the point, the #notallmen stuff is referring to wider culture, and how people enable it. This is outside of individual perpetrators and victims, so you can't narrow the focus on just that.

Yes men that participate in toxic masculinity further this environment.

But also, women that participate in toxic masculinity also enable it, and should also be part of the conversation.

This is not victim blaming in any way shape or form