I think there is more here than just learning to imitate humans, exciting though that is.
Let us take 'Deep Blue' as an example of a machine that does not think. It was able to come up with some dramatic solutions. Its typical successes were mates involving an improbably sequence of sacrifices that gave a mate in 6 or 7, which was about the brute force look-ahead of the time. It also had weighting models that give suggestions of which were 'good' moves and which were 'bad' ones. Moving a bishop to a centre square is good because it threatens more squares, but if it was in front of your king then you may want to leave it where it was. Deep Blue could alter the weights in its model depending on the games it had seen, but it did not really have any understanding of 'edges' and 'centre' any more than a pocket calculator understands the nature of numbers and multiplication.
Let us now take a problem that Deep Blue possibly has not seen: you have two bishops and a king against a king. If you have just taken another piece then you have fifty moves to get a mate, otherwise the game is a draw. Now most of these extreme endgame solutions are known, and Deep Blue probably had the solution hard-coded. If Kasparov had got into the losing position, he would probably have given up the game because he knows it is hopeless.
Can you force a mate in less than 50 moves? Yes, you can. The two bishops can make a diagonal 'wall' of squares that the king cannot jump, so you can slowly heard it into a corner. However, the king can still take one of the bishops, so you have to either protect them with your king, or move them to the other end if the diagonal. As you get towards the corner, the diagonal becomes shorter, and this becomes harder to do. Eventually, you have to protect one bishop and move the other out of the corner entirely. There is then a tricky bit where you may have to waste a move so the other king is forced to move off the better of the squares left to it, and then move the other bishop. It can take 48 moves but it can be done.
Supposing Deep Blue had not got a hard-coded solution. 48 moves is well away from its brute force limit. Its tables for 'good' moves are not optimised for the extreme end-game, and the winning strategy seems to 'change' as you get into the corner. It has no understanding of corners and diagonals, so it might heard the king into a corner from 'instinct' (probably not the 'right' word, but it sort-of works). So, we might win because we can use our knowledge of herding sheep to get the king in a corner, the understanding of the other king's want to survive by attaching the bishops, the knowledge that the bishop can be anywhere along the diagonal to counter this be flipping to the other end, the appreciation that this strategy will not work all the way into the corner and will have to be changed for something at the last minute, and so forth.
Note, this 'Deep Learning' free problem solving ability that we use, and can probably duplicate in a machine one day, is not necessarily linked with self-awareness, will to survive, altruism, creativity, and all the other things we usually identify with intelligence. We could probably make something that could explore other planets which can work for its own survival, and determine what is interesting and worth reporting on the planet, without giving it a concept of 'self' or a fear of its own death. Indeed, it may well be better off being designed without all the baggage that comes with evolution. Maybe it will develop some of these of itself, maybe not. But I doubt it will attack its creators in its struggle to survive, in the classic sci-fi tradition, unless we deliberately train it to do so.
Some say it may have something new and wholly alien to us instead of 'free will'. I rather doubt this, but I allow there might be other radically different solutions for 'how to live'.
Apologies for the long reply, Words are tricky with this topic, but wordy illustrations can avoid some of the worst ambiguities.