Facebook CEO Mark Zuckerberg on stage at F8 last night. Image: Facebook
F8, Facebook’s developer forum, was the scene for the beginning of a new age of augmented reality – captured through your phone’s camera.
2017 will be the defining year for a slew of virtual (VR), augmented (AR) and mixed reality platforms from various tech giants. At F8, Facebook CEO Mark Zuckerberg announced that it will all begin through the camera lens.
Dressed in his uniform attire of jeans and a grey t-shirt, Zuckerberg extolled the “amazing” new age that awaits us and revealed how Facebook is working with numerous developers and start-ups to make AR more tangible and real.
AR will mix digital imagery with real life, eg creating a virtual chessboard before your very eyes. Image: Facebook
“The first augmented reality platform that becomes mainstream isn’t going to be glasses, it’s going to be cameras,” Zuckerberg told the thousands of assembled developers in California.
He said that building a large, open platform for developers to work on will advance AR at an exponential rate.
1. Facebook Spaces will bring VR communications to life
Image: Facebook
The fruits of Facebook’s $2bn acquisition of Oculus are now becoming apparent. At F8, Zuckerberg launched Facebook Spaces in beta on Oculus Rift and Touch.
The app lets up to four Facebook friends join a virtual “space where they can chat, draw, watch 360-degree videos, make video calls on Messenger and take selfies”.
The app looks fun, goofy and addictive, and can be downloaded right now from the Oculus Store.
“Facebook Spaces lets you easily phone a friend in the real world with Messenger video calling, so you can bring even more people into your VR space,” said Rachel Franklin, head of social VR at Facebook.
“They can answer your call on their phone to instantly open a window into your virtual world. Whether you want to show off your latest 3D drawing masterpiece, play an amazing 360 video your friend would love or just spend time chatting, it’s one of the best ways to be in the moment together, from anywhere.”
2. Camera Effects will make AR mainstream faster than we think
Image: Facebook
Working with thousands of developers, Facebook’s new Camera Effects platform is available within the Camera feature in its smartphone app, and allows users to overlay AR effects on images and video.
Eventually, it will be compatible with future AR hardware such as eyeglasses.
“The Camera Effects platform turns smartphone cameras into the first AR platform, providing an opportunity for artists and developers to create effects for the Facebook Camera,” the company said.
“Included with this new platform are two creative tools, Frame Studio and AR Studio that give the Facebook community the power to create a full spectrum of camera effects, from simple frames to interactive augmented reality experiences. This platform empowers artists and developers to connect art with data to bring AR into everyday life through the Facebook Camera.”
3. AR Studio is a canvas for developers to create virtual worlds in the real world
Facebook is inviting developers to apply for access to the closed beta of the Camera Effects AR Studio tool, which allows develops to use location, object recognition and depth detection to create effects.
Zuckerberg showed how it was possible, for example, to insert words in AR onto a table behind a cup of coffee, and make it appear as a physical object.
The AR studio is currently in closed beta but includes: Face Tracker, a real-time computer vision algorithm that tracks faces and allows creators to make masks that fit and respond to facial movements; sensor data that is used to create effects where people can walk around a virtual world; and scripting APIs that allow developers to access and download data and modify effects in real time.
Posted by Facebook for Developers on Monday, April 17, 2017
4. Caffe2 is open and brings deep learning capabilities to mobile devices
At F8, Facebook open sourced Caffe2, a platform to program artificially intelligent (AI) features such as image recognition into mobile devices.
Caffe2 is a lightweight and modular deep learning framework that enables developers to train and deploy AI models for iOS, Android and Raspberry Pi.
The technology is currently being deployed at Facebook, and developers will now have access to many of the same tools, allowing them to run large-scale distributed training scenarios and build machine learning applications for mobile.
5. The 21st century mixtape is here: Share Spotify or Apple Music songs within Messenger
VP of messaging products at Facebook, David Marcus. Image: Facebook
Facebook’s vice-president of messaging products, David Marcus, told the F8 throng that Facebook is baking Spotify and Apple Music into Messenger.
He revealed a new Messenger Platform tool called Chat Extensions, which lets users share a song or an artist within Messenger.
“Music on Messenger is going to be a good thing this year,” Marcus almost crooned.
The new feature means that users can simply share songs without having to leave their chat or browse to a new webpage, and the songs will play directly within the platform. The Spotify extension is already available and Apple Music is coming soon.
“The Spotify bot for Messenger will serve up playlist recommendations based on mood, activity or genres,” Spotify said last night.
“With the associated Messenger Chat Extensions feature, people can search and share Spotify songs, albums and playlists directly with friends without ever leaving the Messenger app. For the first time ever, friends on the other end will be able to preview 30-second clips directly within the Messenger app or visit the Spotify app to listen in full.”
Could this spell a new epoch for music? Remember mixtapes, anyone?
6. Workplace is growing up fast
Facebook revealed that around 14,000 businesses are now using its Workplace collaborative platform, offering competition to rival platforms such as Slack and Microsoft Teams.
Workplace now integrates with some of the largest enterprise cloud services including Salesforce, Box, OneDrive, Office and Quip.
“Our new integrations with some of the world’s leading enterprise cloud services to make it easier to share, preview and organise files, documents, and work with customer records,” explained Astha Agarwal of the Workplace group.
“With these integrations, we’re making it simpler for teams to share ideas and collaborate inside Workplace.”
As well as this, Facebook has created a new tool that allows users to build bots so that they can interact in groups, create a task or a workflow, or take action.
Facebook also unveiled a new Workplace Live API for live video meeting, providing richer communications at work.
7. Analyse this: Better tools for business owners
At F8, Facebook launched new analytics capabilities to help businesses understand the customer journey better, gaining improved insights.
“We’re adding analytics support for Facebook Pages and offline conversions, leveraging artificial intelligence to automatically surface insights and making our tool more customisable,” said software engineering manager Amit Finkelstein.
“We’re extending omni-channel analytics for Facebook’s family of apps and services, with the addition of Facebook Page interactions, such as post reactions and shares, so you can measure and understand the interactions people have with your Page, alongside their other activity on your website, app and bot.
“For example, an e-commerce business can measure if people who comment on an item featured in their Page post go on to view an item on their website, or purchase it in their app. The ability to view customer behaviour across different channels gives businesses a unique ability to learn about and optimise their full customer journey to drive growth,” said Finkelstein.
8. Game on: Facebook has now surpassed 800m monthly gamers
Finally, Facebook revealed that it has surpassed 800m monthly gamers, up 23pc year-on-year.
More than 1.5bn games have been played on Instant Games across Messenger and Facebook News Feed within the last 90 days.
The company has now introduced richer gameplay features to give developers more control of the gaming experiences they can create. For example, developers can customise their own start and end screens, and integrate new APIs that can drive shared and connected gameplay.
Facebook is also rolling out a new tab to make it easier for players to find games, while enabling developers to create bots to invite players into games via the Messenger app.
