I would never in a lifetime have considered this as a solution. On some level, it makes perfect sense now that you brought it to our attention.
Thanks for the tip, I'll give this a try when I get home.
P.S. Other posters are exactly right. Set party chat volume to max. Set every other sound in the game to .01ish.
If you guys wanted us to enjoy the sounds in the game, you may want to continue to work on this. Oh, and volume sliders for party members, yes. Just a volume DOWN would be a huge improvement. At least then I could attempt to balance the whisperer with the guy swallowing his microphone!