How do you see anarchy in the past (like, say, in ancient Rome)?

... Not to be contentious, but ancient Rome was not anarchy in any meaningful sense. Like not even the dictionary definition. Rome was pretty firmly lorded over by a State, and that State maintained the """order""" of the ruling class for generations.

What anarchists mean by "anarchy" is a society without hierarchy. Some hunter-gatherer societies were surprisingly horizontal in their organization, but these societies were not, in general, anarchic, because they likely still had some hierarchies like sexism, in-group/out-group dynamics, and authoritarian social structures. We can look to the past for examples of specific horizontal and anti-authoritarian practices, but none of these societies were truly anarchic because they had not yet grasped the importance of abolishing all hierarchy.

Better (but imperfect) historical approximations of anarchy at scale were Revolutionary Catalonia and the Makhnovshcina.

And why do you think humanity went to hierarchy?

I think the big thing was agriculture. IMO, we were not psychologically prepared as a species to master agriculture so quickly. Some people were able to build up huge stockpiles of resources within a few generations. Once they got their power, they refused to give it up. Thus, the powerful structured our world around their whims over generations. Capitalism is the latest such scheme. Hierarchies maximize the variables authoritarians care about: control, domination, and abuse. It is only natural that abusers establish systems that best reward their kind.

To be clear, I'm absolutely not against agriculture. Quite the opposite! In my view, actively making sure that the fruits of humanity are equally available to all is necessary to break the feedback loop that enabled authoritarians to take power. But more generally: we need to bake horizontalness, autonomy, agency, and free association into all future systems and organizations from minute one.