I've got a slight hearing loss in the mid-range, but I can still tell when something isn't ripped at a pretty high rate. If you lower the encoding rate, you lower it for all the frequencies, I think.
Also, it isn't just hearing that affects how one experiences music. It is also vibration. My mother, who was the first teacher in Michigan to teach music to deaf kids in a public school, found that deaf kids were more on pitch than hearing kids, because of the vibrations.
So, I'd go for the bigger, lusher files.