The civilization series was probably the deepest empire builder.
I spent many years playing civ 2
Oh yes, Civ 2 was great. Never played one of the later thoroughly, but I still sometimes start a Civ 2 (really Freeciv) game.
What it misses ist battle tactics.
Civ 4 is a big improvements over earlier Civs in terms of depth because much more effort has been put into balancing strategies. Expansion in particular is no longer an unmitigated good. It's also highly mod-able and there are fantasy versions if you are playing on a PC (not on a Mac due to OS-dependent advanced modding).
Civ 5 supposedly has tactics but this is offset by the fact the computer doesn't do tactics well. My computer doesn't support it so I can't say from experience.