CDs have never been something that I've spent a lot of money on. As a child, I spent my money on books and then later on clothes (surprising, I know). My family likes to have a wish list from me at Christmas and at my birthday, so I would ask for CDs at that time and avoid having to buy them. I usually do not listen to the whole CD, just one or two songs. I became familiar with individual song downloads and have never looked back. I've found a few frugal ways to get both individual songs and CDs:
1) Wal-Mart music downloads. They are .88 a piece, which is one of the cheapest out there. They also occasionally have a free song you can download. Recently it was an Aaron Shust song. I received a giftcard to Wal-Mart from my bro and SIL for my birthday in June and I have been using it ever since to download songs. I still have $7.68 left on it. :P
2) Be aware of free downloads on the packages of other products. I recently had free downloads from Aquafresh toothpaste, Renu contact lens solution, and Bic mechanical pencils. I had to download a couple of different programs, which was a little irritating. But I got free music!
3) If you enjoy classical music, the Philadelphia Orchestra frequently has free downloads. You can also sign up on their email list and they'll let you know when there is new music to download. You can find the current free download here.
4) If you need or want the whole CD, Swapacd.com is the way to go. If you are familiar with Paperback swap, it works in a similar way. In fact, you can "migrate" your account over. You list a bunch of CDs and then request music from others and others request CDs from you. The only difference with Paperback swap is you need to load money to pay for each CD. They will only cost you .49, so don't worry. My nickname on there is "Tessa" in case you sign up and would like to use me as a referral. ;) If you do not have Third Day's Offerings CD, sign up right now and get it. It is great. I also have gotten John Denver's Greatest Hits volumes one and two. :P
Happy Music Hunting!