I found the process of getting all EDs to level 5 on my ranger fairly painless, but perhaps my tolerance for grind is a little higher than some other people's.
Compared to heroic levelling, epic levelling is really fast. To put this into context, when things go well, it takes me two weeks and an additional weekend to level a TR 2+ life. Normally it's more like three full weeks though, sometimes longer than that.
But when each epic quest you do is worth 20k or more, it takes very little time to reach the requirement for 25 and get well into off destinies. The main mistake you can make with ED levelling that I've found is farming quests too intensively before you cap, so that the locked repetition penalty reduces this amount.
As for off destinies, I think if you have a well built character at 20, you aren't going to be severely hampered by being in an off destiny unless you're trying EE content or you normally find EN or EH challenging in your main destiny.
It won't be optimal, but when my ranger did the caster destinies, having the boost to saving throws available there was actually useful. And all of those levels in off destinies eventually contributed to a nice big pool of fate points, making some good twists possible.