I use the agricultural suppliers for most of the basics: compost, potting mix, fertilizer etc. If you can store the larger quanties it is much cheaper. The last 1,000lbs of compost was $80.
For little things like pots etc, shop around. They sometimes are at places you wouldn't expect. The best pots are not at the nursery but at a local home decor place. I got some nice ones a few years ago at my local grocery store.
The best seedling flats are at the hydroponic store where the clerks have red eyes..