My question is, would it be a good idea to let an AI understand more about us than we do? Assuming it can even be done, given our lack of understanding about ourselves. What if we squint our eyes, move our eyebrows, or make a hand gesture without knowing it, and without understanding fully what our subconscious means by it? Should an AI be allowed to know exactly what were are feeling and thinking, and then act upon it?
While this could be very useful, for example a personal AI telling us that we should take a break, or provide us with sound assurances or advice, I can also see it being taken advantage of. If you had a high understanding of body language, and the needs of a certain person or group, you could influence people almost subconsciously.