Adobe has introduced the Firefly Video Model, a new AI tool designed to integrate generative AI capabilities into the company’s video production software, such as Premiere Pro and potentially After Effects.
The Firefly Video Model offers features like generating additional audio or video frames from existing clips and creating new video content based on reference images or text prompts.
Here are some frequently asked questions about the Firefly Video Model:
Released last year, Firefly is a suite of generative AI art tools available as both a web application and integrated into Adobe Creative Cloud tools. Initially focused on generating still images, these AI models are now incorporated into applications like Photoshop and Substance 3D tools.
The Firefly Video Model expands these capabilities to moving images, producing new video footage from various sources similarly to platforms such as Runway. Key features include:
Generative Extend: This feature extends existing video clips by adding new frames at the beginning or end, or generating extra audio, like ‘room tone.’
Image to Video: This function creates video clips based on reference still images or concept illustrations.
Text to Video: This feature generates video content from text prompts, allowing users to specify camera settings such as position, zoom extent, and common camera movements.
The Firefly Video Model is capable of producing both 2D and 3D animation. Adobe’s blog shows examples of generated footage that resembles 3D, hand-drawn, and stop-motion animation.
Its primary focus, however, is on creating naturalistic video content, such as landscapes, plants, or animals.
Additionally, it can be used to generate special effects elements like fire, smoke, and dust, which can be integrated into VFX or motion graphics projects.
While the Firefly Video Model has the potential to generate entire video sequences, current generative video tools face challenges in maintaining temporal and stylistic consistency.
Adobe’s blog suggests that the model’s initial application will be to create supplementary materials for conventionally shot projects, such as AI-generated stock footage for B-roll or missing establishing shots.
Currently, the Firefly tools are available online, but Adobe is also integrating them into its Creative Cloud software. The Firefly Video Model will definitely be included in Premiere Pro, with the Generative Extend feature expected to be available in beta versions later this year.
Adobe has also mentioned After Effects, where the company has previously demonstrated generative AI tools.
Adobe states that the Firefly Video AI model is “commercially safe” and trained only on content that the company has permission to use.
Earlier this year, Adobe updated its Terms of Use to explicitly exclude the use of user content—except for that uploaded to Adobe Stock—for training AI models, in response to backlash from artists.
The Firefly Video Model is currently in the technology preview stage. It is expected to enter public beta later this year. Adobe has not yet announced which other Creative Cloud applications will include this feature or the specific release details.
