[Fingers]

20 is "per inventory", it applies to flotillas and star systems. The flotilla's max is defined by the ship with the largest cargo bay, so no ship should have it set higher than 20.

The number really is an interface thing as you guessed. The player cargo interface happens to be wide enough for 4 items, and the star/planet window is 5. The limit of 20 prevents running out of space with visible empty space in the window. =)