It seems to me they should go all until the end. Noone should join later the Party even if Heroes die. But that's only my opinion.
However I think the Party should start with 12 initial Heroes. The Heroes were interspersed in the Party. There is enough XP for all. And when doing Raids all adventure.