I love hunting around for old vinyl in second hand stores, antique shops, charity shops, house and garage sales. It does have its drawbacks though.
First there's the time and cost of gas getting to these places. Second there's the dirty factor. Your hands feel a bit grimy after a long session of hunting. Lastly, it gets a little painful as I'm often on my knees to look through these old LPs.
So it is fun to go searching on ebay and Craigslist on occasion. A friend of mine browses Craigslist in bed as a way of falling to sleep. Usually the prices on ebay are a bit too steep for my blood but you can sure find some cool stuff.
How about a bunch of vintage rock LPs still sealed. That's what I cam across last evening. Check it out.