The slowness of the army setup is related to the game loading each and every unit on the screen as a separate object, which clogs it up if there is such a crapton of them. Much like the Nation overview used to freeze in Dom2 if you had to many sites discovered.
There is also a hard limit in the game for the maximum number of units that can exist at the same time. If the limit is reached, nobody can recruit or summon more until some have died off. In Dom2 the limit was 32k, and for Dom3 it was raised. Apparently not enough if people are running up against it again, because that's what it looks like.
The slowness could be prevented by changing the army setup screen to use counters to display numbers of units. Right now it's not a very good design if you have more than the 'normal' number of troops.
But that might not be something the Devs are bothered enough about to patch.