There's a bunch of different styles(sprung seat, lever post, air or spring chamber, rubber block) so the experience can vary widely. I've used a lot of them and can tell you that at best, the improvement is very minor, as in they only take the bite out of the most violent hits and not much else. At worst, they act as a pogo stick and at a certain pedaling cadence, you will feel like you are bouncing while riding.

If you're not expecting much from them, I don't think it's a bad thing to try but if you're experiencing pain that's keeping you from riding as much as you want, you're likely not going to get the relief you're hoping for.