Meta unveiled a crazy artificial intelligence model that allows users to turn their typed descriptions into video. The system is called Make-A-Video and is the latest in a trend of AI generated content on the web.
The system accepts short descriptions like “a robot surfing a wave in the ocean” or “clown fish swimming through the coral reef” and dynamically generates a short GIF of the description. There are even three different styles of videos to choose from: surreal, realistic, and stylized.
According to a Facebook post by Meta CEO, Mark Zuckerberg, translating written text into video is much harder because of how video requires movement:
“It’s much harder to generate video than photos because beyond correctly generating each pixel, the system also has to predict how they’ll change over time. Make-A-Video solves this by adding a layer of unsupervised learning that enables the system to understand motion in the physical world and apply it to traditional text-to-image generation.”
Meta’s AI Research team wrote a paper describing how the system works and how it differs from current text-to-image (T2I) methods. Unlike other machine language models, Meta’s Text-to-Video (T2V) method doesn’t use pre-defined text-video pairs. For example, it doesn’t pair “man walking” with a video of an actual man walking.
If this sounds a lot like DALL-E, the popular T2I application, you wouldn’t be far off. Other T2I applications have rolled out since DALL-E gained popularity. TikTok released filter in August called AI Greenscreen that generates painting style images based on the words you type.
AI-generated content has become quite buzzworthy within the last few years. Deepfake technology, machine learning techniques to replace person’s face with another, is even used by visual effects studios for big budget shows like The Mandalorian.
In July, The Times mistakenly reported on a Ukrainian woman in the midst of the Russia-Ukraine war. The problem is she wasn’t real.
The threat of AI probably isn’t real threat, but projects like DALL-E and Make-A-Video are fun explorations into some of the interesting possibilities.
Editors’ Recommendations