The new project of Facebook, called ‘Ego4D’, is an AI that can see, hear, and remember everything a person does with the aim of showing results that can help in everyday tasks. This system can be applied in different scenarios and devices, but it makes great sense in augmented reality glasses. Mainly, because they have cameras with a more human point of view and microphones to obtain all the information.
Until now, artificial intelligence systems have been able to learn day-to-day routines to, for example, show us the fastest route to work. What Facebook intends to do, however, goes much further. The project, still in development, is being carried out together with a group of 13 universities around the world, where more than 700 participants have helped to capture 2,200 hours of first-person video while performing day-to-day tasks. This, coupled with a “five benchmark challenges” system, will allow an AI to understand much more complex situations. The challenges shown by Facebook are a clear example of what an AI can do with a more human point of view.
– Episodic memory: What happened when? (for example, “Where have I left my keys?”)
– Forecast: What am I likely to do next? (eg “Wait, you’ve already added salt to this recipe”)
– Hand and object manipulation: What am I doing? (for example, “Teach me to play the drums”)
– Audiovisual dailyization: Who said when what? (for example, “What was the main topic during the class?”)
– Social interaction: Who is interacting with whom? (for example, “Help me better hear the person speaking to me in this noisy restaurant”)
Keeping the example of the episodic memory, the Facebook AI will be able to remember the last place where it has seen the keys and notify the owner. This can also be applied to other objects.
An advanced Facebook AI that has little to offer in terms of privacy
Kristen Grauman, Facebook AI researcher has highlighted The Verge that the objective of this artificial intelligence is that can be used on devices that go beyond portable cameras, as in domestic auxiliary robots. Of course, privacy is concerned, an important point in which Facebook is not exactly an expert company.
Grauman, in fact, has confirmed to the aforementioned medium that they “hope” that companies that use this technology for commercial devices include privacy measures in their applications. So protection by Facebook does not seem to be guaranteed.