After Bard (an AI content creator), Google is now set to introduce Lumiere, an innovative AI video generator that smooths the video editing process. Lumiere uses a diffusion model called Space-Time-U-Net (STUNet) to comprehend videos’ spatial and temporal aspects. It enables users to create seamless transitions between frames in one go. The process starts by generating a base frame from a prompt. Then it employs STUNet to predict object movements within that frame, creating a series of frames that seamlessly transition between each other.
In addition to text-to-video generation, Lumiere provides image-generation capabilities through Google’s Gemini large language model. This particular feature in this innovative video generator allows users to create videos with specific styles and cinematography that animate specific parts of a video. It also allows users in painting to modify colors and patterns in a particular part of the video.
Also, learn about
Lumiere is not yet available for testing for users, It showcases Google’s power in developing an AI video platform that may surpass existing AI video generators like Runway and Pika. Google is aware that this AI platform will be misused in future, such as creating illegal or harmful content through this technology. Therefore, the company emphasizes developing tools to detect biases and malicious uses. They are entirely aware this is very essential to ensure safe and fair utilization. However, specific methods for achieving these details are not provided in the announcement.