Google presented what could be one of the most advanced video generators so far: it’s called Lumiere and allows you to create content from a text description or an image. But unlike existing artificial intelligence models, the company highlights that its creates movements “realistic, diverse and coherent.” And in just seconds, of course.
Google had previously presented other tools of video generation, but more limited than Lumiere. The new technology uses a model called Space-Time-U-Net (STUNet), which determines where objects or elements are in a video, how they move and change simultaneously. In addition to generating animated videos and images, can copy artistic styles, and even allows you to edit the created clips.
“We demonstrate state-of-the-art generation results and show the applicability of our approach for a wide range of implementations,” the developers say in the report Of the investigation. The model was trained with more than 30 million videos, with its subtitles. The Californians, like other developers, did not reveal the source of this data.
Lumiere has the capacity to generate 80 frames at 16 frames per secondGoogle said. This allows videos up to five seconds long, with a resolution of 1024 x 1024 pixels. Furthermore, according to a study carried out with users, its results were preferred over existing video models with artificial intelligence.
Google’s warning about using Lumiere for deepfakes
Lumiere is not yet available for testing, nor is it known when it will arrive or if Google will ever release it to the public. But the company was kind enough to leave us some samples in a promotional video. In some cases, if you pay attention, you can see some clues on the faces of animals that reveal the use of artificial intelligence. But there are other scenes, like that of a swimming turtle, that are impressively realistic.
Google also demonstrates Lumiere’s ability to create crazy scenes, such as a panda driving a car or a cat playing the piano. He also boasts about the tool he gives movement to such famous paintings as the Mona Lisa.
Like most options so far, Google Lumiere is not designed to generate videos that involve multiple shots or transitions between scenes. “Generating such content remains an open challenge for future research,” the authors acknowledge.
Google developers included a warning about how Lumiere could boost the spread of deepfakes. “There is a risk of misuse when creating false or harmful content with our technology”, they say in the report. “We believe it is crucial to develop and apply tools to detect bias and malicious use cases to ensure a safe and fair experience.”
The creation of fake content sparked a huge controversy last week, when fake and sexually explicit images of Taylor Swift were spread. Even the White House spoke out about it. “It’s alarming,” said Jean-Pierre, a spokesman for President Joe Biden’s office. Government “is committed to ensuring we reduce the risk of fake AI images through executive actions. The work to find real solutions will continue », he wrote on X (formerly Twitter).