Great post, thought roads I have been down.
What concerns me:
1) Humans are already conditioned to take orders from machines, or robots if you will. For instance, traffic lights are merely the most rudimentary form of self-regulation that humans have delegated to robots. Divorce yourself from anthropomorphic presentations of robots and you start to realize that humans are integrating their experience of life in ways that allow robot machines to dictate a large portion of what we consume, think about, and what activities we perform.
2) Humans and their behaviors are subject to reverse engineering. Whether its focus groups and mock votes by politicians, medical and psychological testing, or something more interesting and insidious like employees of Target figuring out how to tell with over 80% accuracy whether or not a woman has become pregnant based on what she buys. People's behavior, when considered on a large enough scale, can be data mined in such a way that gives the party with the data an incredible advantage in persuasion, prediction, and ultimately control over outcomes that most individuals would consider creepy and unfair.
3) Humans are using machine learning to make #1 more robust right now. Some if it is labor saving, most is in the realm of profit increase by automating, speeding up, running without a human in the loop sort of stuff. This will be interesting in economic ways in the short term, sociologically in the long term. What will be very interesting is when we turn machine learning to the tasks of #2. Applying the methods of a machine learning tools to decoding and manipulating both groups of people and individuals is an area where pitfalls are likely to abound.
I find this interesting and repulsive in the same breath. What I wouldn't give to have access to Facebook's data sets as data fodder for a home grown machine-learning-based predictive tools. I would rule the world.