Well, look at it from this perspective, your daughter and son did not grow up having only you and your wife as the social other. Altough parents are, and at young age the mother specially, the greatest other, they obviously are in a society where feminine=pink and male=blue (or digger).
The way you act as a man and your wife as a woman surely influenced them - and is culturally/socially constructed Other family/friends too play roles in front of them, and they learn, of course.
Well, thats if you didn't put 'em on a plastic bubble for childhood as an evil anthropology experiment! :P