Steven James at the University of the Witwatersrand in South Africa and his colleagues developed a benchmark test within Minecraft to measure the general intelligence of AI models. MinePlanner assesses an AI’s ability to ignore unimportant details while solving a complex problem with multiple steps.

Lots of AI training “cheats” by giving a model all the data it needs to learn how to do a job and nothing extraneous, says James. That is a fruitful approach if you want create software to accomplish a specific task – such as predicting the weather or folding proteins – but not if you are attempting to create artificial general intelligence, or AGI.

James says that future AI models will need to tackle messy problems, and he hopes that MinePlanner will guide that research. AI working to solve a problem in the game will see the landscape, extraneous objects and other detail that isn’t necessarily needed to solve a problem and must be ignored. It will have to survey its surroundings and work out by itself what is and is not needed.