Hyderabad: Meta has announced a new generative artificial intelligence model called Movie Gen that can create custom videos and audio clips in response to text input from a user. The new model can also edit existing videos and transform personal images into a unique video.
"Movie Gen outperforms similar models in the industry across these tasks when evaluated by humans," Meta says in a blog post. The company also emphasised that generative AI is not meant to replace the work of artists and animators but to help people express themselves in new ways and to provide opportunities to people who might not otherwise have them.
Capabilities of Meta Movie Gen model
Meta's Movie Gen model includes modalities like image, video, and audio. It features four capabilities, which include video generation, personalised video generation, precise video editing, and audio generation. Meta says that it has trained the Movie Gen model on a combination of licensed and publicly available datasets.
Video generation: The 30B parameter transformer model can generate videos of up to 16 seconds at a rate of 16 frames per second. It claims that these models can reason about object motion, subject-object interactions, and camera motion, in addition to learning plausible motions for a wide variety of concepts.
Personalised videos: Meta says that it expanded the video generation model to support personalised video generation. It takes a person’s image as input and combines it with a text prompt to generate a video clip that contains the reference person alongside "rich visual details" suggested by the prompt.
Precise video editing: The editing variant of the model combines video generation with "advanced image editing" to perform localised edits like adding, removing, or replacing elements. It also supports global changes to the clip, such as background or style modifications. Meta claims that Movie Gen preserves the original content and targets only the relevant pixels.
