TWO ARTIFICIAL INTELLIGENCE arrays from Alibaba and Microsoft have beaten humans at a reading comprehension test set by Stanford University, marking the first time the machines have beaten the fleshies.
The test, which was devised by Stanford University, specifically to see how human reading compares to that of AI, gives passages for both to read and then asks questions about what has been read. So a passage from Oliver Twist might ask, "What did the boys at the workhouse eat?"
The neural network will then use what it has (hopefully) learnt to give the correct answer (in this case ‘gruel').
The Alibaba machine was the first to exceed a human score, with 82.44, and then a day later, the Microsoft equivalent beat it with 82.65. Humans who have taken the test are currently maxing at 82.3.
Facebook, Samsung, Salesforce and Tencent have all tried, but these two are the first winning examples of the test, which is generated using Wikipedia to create 100,000 questions based on 500 articles.
Microsoft's R-Net, showcased in this experiment by Microsoft's Asian Research arm, is explained thus:
"We first match the question and passage with gated attention-based recurrent networks to obtain the question-aware passage representation. Then we propose a self-matching attention mechanism to refine the representation by matching the passage against itself, which effectively encodes information from the whole passage.
"We finally employ the pointer networks to locate the positions of answers from the passages. We conduct extensive experiments on the SQuAD and MS-MARCO datasets, and our model achieves the best results on both datasets among all published results."
It's important to understand that "comprehension" doesn't mean "interpretation", only that it has extrapolated the correct data from what is available. It still takes a human to say, in the case of our Oliver Twist example, "Ewww - gruel! Those children were so mistreated" (or whatever).
Reading comprehension is one of the holy grails for customer service roles, which is why companies like Alibaba and Salesforce are so keen.
Alibaba is already using the tech to help its customer service offering on Singles Day, the Chinese version of Black Friday when the volume of orders can break records, and so the volume of assistance needed reflects that. Ergo, the more that ‘Bots can handle, then theoretically at least, the more time there is for harder queries and complaints to be handled by humans. µ
But it won't be launching until the summer
And they'll be available to pre-order from today
Firm takes on the iPhone XR with its first 'affordable' flagship
Firm looks to be on to another set of winners with its latest flagships