Digit sensors mounted on a robot hand manipulating glass marbles. Image: Meta
Meta CEO Mark Zuckerberg revealed ReSkin and Digit, two products that could help AI learn how to ‘feel’.
Last week’s Meta announcement was all about visuals – a new virtual world called the metaverse where people could be transported through smart goggles. Now, the company formerly known as Facebook is looking for new ways to make people sense touch in the metaverse.
CEO Mark Zuckerberg announced in a Facebook post yesterday (1 November) that Meta AI, the research team behind the new world, has made progress in the tactile side of things with a new material and sensor that could enhance the way humans feel objects in the metaverse.
“We designed a high-res touch sensor and worked with Carnegie Mellon to create a thin robot skin,” he wrote in the post. “This brings us one step closer to realistic virtual objects and physical interactions in the metaverse.”
ReSkin
The thin ‘robot skin’, called ReSkin, uses machine learning and magnetic sensing to ‘feel’ objects it comes in contact with in a way that is similar to how humans interact with objects. Up to 3mm thick, the material was created in collaboration with researchers at Carnegie Mellon University.
“ReSkin offers an inexpensive, versatile, durable and replaceable solution for long-term use,” the company wrote in a blog post. “It employs a self-supervised learning algorithm to help auto-calibrate the sensor, making it generalisable and able to share data between sensors and systems.”
The technology was shown to be able to train robots to perform complex actions such as unlocking a door with a key or grasping delicate objects such as berries and grapes with just the right amount of force, as the company demonstrated in GIFs in its blog post.
“Robust tactile sensing is a significant bottleneck in robotics. Current sensors are either too expensive, offer poor resolution, or are simply too unwieldy for custom robots,” said Lerrel Pinto, an assistant professor of computer science at New York University.
“ReSkin has the potential to overcome several of these issues. Its lightweight and small form factor makes it compatible with arbitrary grippers, and I’m excited to further explore applications of this sensor on our lab’s robots.”
Digit
The other technology, a high-resolution tactile sensor called Digit, complements the advances made with the robot skin and aims to make robots better at interacting with objects in the real world. The sensor helps researchers train AI to ‘feel’ by collecting and enabling the processing of data.
The company said that Digit can collect rich datasets using hardware that is easy to build, compact and relatively low cost. “Digit is significantly cheaper to manufacture and provides hundreds of thousands more contact points, which makes it more useful and accessible to research teams around the world,” it wrote.
Digit was created by Meta AI in partnership with GelSight, an MIT spin-off that specialises in tactile sensing technology, which will now make the product commercially available for researchers to study.
Meta hopes that these advances in touch sensing can unlock possibilities in AR and VR, which would enhance user experience in the metaverse. It also said the new tech can lead to innovations in industrial, medical and agricultural robotics.
“Improvements in touch sensing can help us advance AI and will enable researchers to build robots with enhanced functionalities and capabilities,” the company said. “We’re working toward a future where every single robot may come equipped with touch-sensing capabilities.”
Don’t miss out on the knowledge you need to succeed. Sign up for the Daily Brief, Silicon Republic’s digest of need-to-know sci-tech news.
