In 2011, developers at IBM built an artificial intelligence system and put it in direct competition against two of America’s smartest minds. Appearing on the popular quiz show Jeopardy, the Watson computer crushed opponents Ken Jennings and Brad Rutter.
As insignificant as a game show competition may seem, Watson’s victory represented a major advancement in artificial intelligence. Receiving quiz questions verbally, just like his human counterparts, Watson was the most sophisticated system capable of processing natural language.
Still, he wasn’t perfect. Despite the impressively high success rate, Watson still got a few answers wrong.
"Its largest airport is named for a World War II hero; its second largest for a World War II battle," read one question under the "US Cities" category. Watson’s answer: Toronto.
Not to be outdone, Microsoft, in conjunction with the University of Science and Technology of China, may have now developed an even more impressive artificial intelligence. It doesn’t have a catchy, anthropomorphic name, but the computer system can beat out the average person on an Intelligence Quotient test.
"Our RK model can reach the intelligence level between the people with the bachelor degree and those with the master degrees," reads the report conducted by the developers, "which also implies the great potential of the word embedding to comprehend human knowledge and form up certain intelligence."
IQ tests are typically composed of questions which fit into one of three categories. Logic questions rely on identifying picture patterns and mathematical questions rely on identifying numerical patterns. Computers have always been pretty good at both of these, given the fact that computers are built on numerical patterns.
But the third category, verbal reasoning, has proven more elusive for AI systems. These are questions dealing with analogies, classifications, synonyms, and antonyms. While past systems are certainly familiar with human language, that understanding has always been based on simplistic calculations. In a way, words are assigned numbers, and the AI processes sentence structure from those algorithms.
This method does not, however, take into account the fact that many words can have multiple meanings.
"Where do you find chili beans? At the North Pole," is a joke that would be completely lost on your laptop.
But the new AI system takes these weaknesses into account.
First, the computer analyzes any verbal questions to determine the category – analogy, synonym, antonym, etc. From there the system can weigh the words in the question against each other in a more accurate way, and determine the correct answer.
In the question, "which word is most opposite to MUSICAL? (i) discordant, (ii) loud, (iii) lyrical, (iv) verbal, (v) euphonious," the computer can deduce from the word "opposite" that this is an antonym question. "Musical" can be identified as the sentence’s subject, and from there it’s a simple search of antonyms to discover "discordant" is the best answer.
Developers tested their system at Amazon’s Mechanical Turk crowdsourcing facility against nearly 200 human subjects of varying intelligence levels. The AI did phenomenally well.
"To our surprise, the average performance of human beings is a little lower than that of our proposed method," the researchers said.
Of course, this system is custom built for IQ tests, which follow a fairly rigid formula. A real-life scenario straying from those strict parameters could prove more difficult for the computer. Still, it’s a major step forward in computer science.
Ken Jennings may have said it best after losing the Jeopardy tournament.
"Just as factory jobs were eliminated in the 20th century by new assembly-line robots, Brad and I were the first knowledge-industry workers put out of work by the new generation of 'thinking' machines," he wrote for Slate.
"'Quiz show contestant' may be the first job made redundant by Watson, but I’m sure it won’t be the last."
