Check out the iPod clone for the pocketpc from http://www.starbriteltd.com/ and it's only $20. (disclaimer: I don't have anything to do with this company)
What is your experience with using the PocketPC as a MP3 player?