I have only 2 annoyances with COE.
-sneaking armies
This one is annoying because of the very limited vision around your towns and towers, most of the time you find enemy troops in the middle of your whole territory wondering how they could have traveled there without being noticed by your army, towers or towns.
Certainly as a workaround you could send additional single commanders everywhere as scouts, but when i tried this i quickly understood that doing so is hurting your economy so much that you will be unable to field enough troops and so you are in fact dooming your side to lose in lnog term.
-combat rules
The primary annoyance i have with COE, the combat system that gives the advantage to the defense allowing a defensive stack to have all of their units hitting 1st.
Basically if you want to have a chance you must never attack an enemy stack that is globally equal to your troops, because by attacking them they will hit you first, weakening your troops a lot before you have even a chance to retaliate.
Certainly the problem is not big in the beginning of the game when you meet some stacks with 2 militia inside or 1 lonesome deer.
Or against some of the enemy races that field weak troops, but in the case of races that are capable of fielding several stacks like this one :
with the combat rules of defense hit first, this kind of enemy stack will always devastate your troops if you attack first, as they have more than 100hp each, multiple attack of strong damage and regenerating (those swamp gods especially i can't believe how much troops and catapults it took me to take just one down
).
And because the system always gives the 1st attack to the whole defense units, if you field some good troops yourself, those kind of stack will not attack first, waiting for you to do this (and so to have your troops weakened a lot).
Despite Dominons 3 combat system is turn based too, it avoids creating this problem in a verty elegant way with all the troops positions, scripted orders and units stats that will affect the outcome of the 1st assault.
But for a simpler game like COE that has no conditions to influence the 1st hitting side, it is very unfortunate that both sides do not attack in the same time during a turn, it would have been a lot better than the current system.