Scientists from the Allen Institute for Artificial Intelligence (AI2) and University of Washington have revealed an AI system that can answer questions in a SAT geometry exam as well as the average 11th-grade student in the US.
The system, dubbed GeoS, can achieve 49 per cent accuracy on the SAT tests by using computer vision to interpret diagrams, natural language processing to read and understand the text and a geometric solver. This machine's score translates to 500 out of a possible 800 across the entire test, which was the average score in 2015.
Oren Etzioni, CEO of AI2, said: "Unlike the Turing Test, standardised tests such as the SAT provide us today with a way to measure a machine's ability to reason and to compare its abilities with that of a human. Much of what we understand from text and graphics is not explicitly stated, and requires far more knowledge than we appreciate.
"Creating a system to be able to successfully take these tests is challenging, and we are proud to achieve these unprecedented results."
GeoS is the first end-to-end AI system to solve SAT plane geometry questions, and researchers are working towards the system solving the entire set of maths problems within the next three years.
"We are excited about GeoS's performance on real-world tasks," said Ali Farhadi, senior research manager for Vision at A12 and assistant professor of computer science and engineering at the University of Washington. "Our biggest challenge was converting the question to a computer-understandable language.
"One needs to go beyond standard pattern-matching approaches for problems like solving geometry questions that require an in-depth understanding of text, diagram and reasoning."
