My bonds of fire(it was my capitol ) slowed them down and, i honestly believe that if i had a larger number of troops and better mage placement(maul barbarians and my flail men come to mind) i could have crushed the elephants as, even with only a few troops i inflicted a couple of casualties before going down.
I just started a large game with EA Ulm and chose a Dragon awake pretender with a high Domain to try and avoid that. With the Awe + Fear and my breath weapon I hope to make people pause before rushing me. The breath will help alot with the Van/Hel (though probably not enough). I get to expand very fast which is a big plus as well.