That was me too. Now I'm good up to about 2 stories high, which is how high the climbing gym was.
It was scary and there were definitely times I abandoned a climb because my legs wouldn't stop shaking, but you eventually learn to trust the ropes will save you and push through the fear. It was 100% worth it.
I'm old. I've been playing Dota2 as my primary game since 2012. It works out to about 500 hours a year or so, which is still a lot but yeah.