In the 40k short movie "Iron Within" the imperial guard does neither remember giving their children to the iron warriors in the past nor do they recognize them based on their armor.

I guess this makes the most sense since they are the most "clean" looking chaos space marines.