Adobe: New AI Video Generation Features | Adobe ai | Adobe firefly | text to video ai | Turtles AI

Adobe is set to launch its new AI-powered video generation features by the end of 2024, offering tools that could transform the video production world. Discover the details of the Generative Extend, Text to Video, and Image to Video features that will soon be available to the public, and how Adobe plans to differentiate itself from competitors with its Firefly technology.

Highlights:

• Adobe to launch new AI video generation features by the end of 2024.
• Key features include Generative Extend, Text to Video, and Image to Video.
• Firefly stands out for its focus on controllability and integration into existing workflows.
• Significant safety measures have been adopted to prevent misuse of the new tools.

Adobe is preparing to introduce its highly anticipated AI video generation features, aiming to offer advanced tools for creatives and professionals in the field. The key features – Generative Extend, Text to Video, and Image to Video – are currently in private beta, but will be available to the public by the end of 2024, both via the Adobe Premiere Pro beta app and on a dedicated free website.

Generative Extend allows any input video to be extended by two seconds, using a prediction model that analyzes the final frames of a scene to create a smooth and natural transition. The feature also recreates background noises such as traffic or nature sounds while respecting licensing requirements that prevent the generation of human voices or copyrighted music. This feature, which will be integrated into the Premiere Pro beta version later this year, stands out for its ability to seamlessly add new video content, making it useful in situations where the original footage ends too soon or requires a smoother transition.

The Text to Video model allows up to five seconds of video to be generated from a text prompt, offering a range of unprecedented creative possibilities. An example provided by Adobe is the creation of a macro shot of water splashing and freezing to spell the word "ICE," demonstrating the model’s effectiveness in handling complex details and smooth movements. Similarly, Image to Video allows users to obtain short video clips based on uploaded images, further expanding the creative possibilities.

One key difference between Firefly and other generative AI models, such as OpenAI Sora and Runway Gen-3 Alpha, lies in the accuracy and predictability of the results. While many of the current AI solutions for video generation are considered too unreliable for professional use, Adobe focuses on the controllability of its tools to integrate them into existing workflows and improve operational efficiency. According to Ely Greenfield, Adobe’s CTO of Digital Media, the company has observed growing demand for tools like Firefly that can accelerate or simplify creative processes without compromising quality.

Adobe has also taken significant precautionary measures to prevent misuse or improper use of its video generation tools. Among the main ones are restrictions that prevent the creation of content with nudity, drugs, or alcohol, as well as a ban on using the model to generate videos with public figures such as politicians or celebrities. These measures align with Adobe’s approach to safety and responsibility in the use of generative AI, a crucial aspect at a time when such technologies are under increasing public and regulatory scrutiny.

On the cost front, Adobe has not yet disclosed the pricing of its new AI video features but has confirmed that they will be offered to Creative Cloud users in the form of "generative credits." Each credit typically represents one generated result, and more expensive plans will provide more credits, allowing for more extensive use of Firefly tools.

With its history of innovation in the creative sector and a new focus on AI, Adobe seeks to position itself as a leader in multimedia content generation, aiming for tools that not only meet creative needs but integrate precisely into existing workflows. It remains to be seen how these new tools will be received by the public, but anticipation is high, and the ability to extend or generate video content through simple commands could mark a significant change in video production practices.