The article describes the efforts IBM has made to build the knowledge engine, and the natural language parsing ability, to successfully compete against human beings. Watson, while impressive, cannot always best frail humans:
Yet the truth is, in more than 20 games I witnessed between Watson and former Jeopardy! players, humans frequently beat Watson to the buzzer. Their advantage lay in the way the game is set up. On Jeopardy! when a new clue is given, it pops up on screen visible to all. (Watson gets the text electronically at the same moment.) But contestants are not allowed to hit the buzzer until the host is finished reading the question aloud; on average, it takes the host about six or seven seconds to read the clue.
Players use this precious interval to figure out whether or not they have enough confidence in their answers to hazard hitting the buzzer. After all, buzzing carries a risk: someone who wins the buzz on a $1,000 question but answers it incorrectly loses $1,000.
Often those six or seven seconds weren't enough time for Watson. The humans reacted more quickly. For example, in one game an $800 clue was "In Poland, pick up some kalafjor if you crave this broccoli relative." A human contestant jumped on the buzzer as soon as he could. Watson, meanwhile, was still processing. Its top five answers hadn't appeared on the screen yet. When these finally came up, I could see why it took so long. Something about the question had confused the computer, and its answers came with mere slivers of confidence. The top two were "vegetable" and "cabbage"; the correct answer "cauliflower" was the third guess.
Wordplay may give Watson some trouble, but is this an impressive advance, or just another evolutionary step toward pervasive weak AI?