If a robot is able to understand emotional responses, and in some cases it might be better at inferring them by having better sensors, it can then act accordingly to those human emotions.
Probably a system that acts in a deterministic fashion for 80% to 90% of the cases, and can infer human motivation but with no emotional response associated with it is probably the best course.
Once AI's are able to experience subjective emotions, and being better at inferring human motivations then I think these emotions will eventually force the AI to act.
Emotions are an underlying response feedback mechanisms, they are as much cause and effect. Once an AI being set o emotional feedback loop that increases the need to act out against a preset safety directive (for humans) it either will become schizoid, develop some kind dysfunctional behavior or just flip.
It is a good bet if autonomous robots with capable AI would be considered by humans, at least those who can afford them, as lesser status workers or appliances. Given that humans subject people with less status to all kinds negative treatment, from discrimination, physical harm, derision, etc. Any capable AI would consider Humans as flawed and dangerous as a group.