I find the biggest tell-tale of 320k mp3s is in the high-hats for me. That's usually where I hear it first. They will have more of a wooshy or flanged sound than I'm used to hearing under "proper" listening circumstances. Otherwise, if you listen back-to-back with a hi-res, the mp3s usually sound a little darker, but more in a smeared and ganky way as opposed to a 'warm' and pleasant way.
That being said, most of the music I listen to is really not even remotely Hi-Fi (old live jazz records, punk, early 90's hip-hop, and noise rock) so 99% of the time, I'm perfectly happy to walk around listening to mp3s of DOA records recorded on a Tascam tape cassette recorded in 1979 without fuming about the lack of fidelity. That other 1% of the time, is sitting quietly at home with nice headphones, listening to my favourite Zappa records, where I know every single tone and texture better than I know myself. Hi-res is night and day away from mp3 for these recordings, for me.