MP3 player
With recording built in.
I bought on about a year ago, oddly enough it won't work with my computer as an MP3 player. 19.95, at a local dollar store. Long story there.
But does work as a recorder. Which has turned out to be very handy. Built in mike, just thumb it on an start gibbering away.
May not work for you, there are some inexpensive mini recorders out there.