Facebook has revealed its latest AI project, with a new app that can answer verbal questions about what appears in pictures.
Speaking at an event in the US, Yann LeCun, director of Facebook’s artificial intelligence research group, showed off the new tool.
Showing a number of photos, the app answered several questions about what was going on, what animals were involved, what they were doing, what colour were they etc.
One example was a dog with a toy in its mouth, with the app correctly answering a question about what game the dog was playing: frisbee.
By using deep learning, which combines software programming with knowledge of how our brains fire up, LeCun – and many others – believe things are going to get a whole lot more accurate quite soon.
Interestingly, it’s almost a year since two major papers were written on technology like this, with Google involved in one of them.
At the time, similar problems were put forward for technology to address – in a basic sense, captioning images.
Google claimed that it had developed the requisite machine-learning to automatically and accurately caption images.
That wasn’t the case, the results were not flawless.
Though they did point to where we would soon find ourselves, with machines describing settings, something which I, at the time, noted could be a major step forward in the art of surveillance.
I still think that, but it’s interesting to note the use of verbal cues with Facebook’s latest tool.
By answering spoken questions, this could genuinely help those who are visually impaired. It seems so simple, but it could prove extraordinarily beneficial to people’s lives.
“What you’re seeing is not fake; it’s a real system, and it’s able to answer basic questions about images,” said LeCun, at EmTech.
“A system that actually describes an image could be a very useful thing for the visually impaired.”
This app is experimental, as is most of the work Facebook’s artificial intelligence lab gets involved in, but it could, one day, complement another task undertaken earlier this year by the social media giant.
During the summer, the lab developed an algorithm that can recognise faces of people in photos even when their faces are hidden from view.
Photo focus image via Shutterstock
