After they're killed. This has been niggling on me since my newbish inception. Do gems somehow bond to the carrier and melt away when they croak? Why do the lower level magic items often survive combat if gems can't? Sure sure, routing gem carriers can keep their gems. But dead ones?
This seems like it would have the biggest impact on SP games, making beating the AI even easier (since the AI loves to cram lots of gems onto their mages).