The way you guys play, I would seriously consider focusing more on charms, hypnotism, ottos, hold person, command, web, sonic blast... Even if you fail two or three times, charming one giant or ogre will help keep the aggro off of your party. A sorcerer has plenty of SPs to spam charm until he hits one. Same for web.
I would also consider Enlarge feat for one caster so you can charm from further away while the monsters fight themselves... Then you sweep in and mop up the survivors.
Effective use of the CC spells available to you will help you from taking damage and aggro.