Thoughts are not reality. Just because you think someone thinks of you a certain way, doesn't mean it's true. I find it's easy to make assumptions that, at the very least, have no evidence behind them. Sure, if someone visibly mistreats you, that's one thing, but if you don't have much of a reaction, they might just not notice you and have no strong opinion one way or the other.

Likewise, if somebody does mistreat you, it's not your problem or your secret, it's theirs. It's their failing as a person.

There is a different version of you in the mind of everyone who knows you. If you have kids, consider how you see them vs how you see other kids who aren't yours or in your family. Or how you see your friends vs how people you don't know see them, like how you see strangers vs how their friends see them. You can't win over everybody.

Inevitably, you will be the villain in somebody's story, and it could be for a reason outside your control. You have something they want, or you're friends/lovers with someone they want. I had a person criticise me once for not letting their friend bully me. I don't know how they worked that out in their brain, but they told me as much, and I pretty much said "I don't know what to do with that, but have a nice day." Not only can you not win over everybody, you can't even break even with some people.

So focus on the people who want to be in your corner. And yourself. It's not selfish to take care of yourself. Most people do that.