Aseem Agarwala

Adobe premiere is FINALLY getting media intelligence and semantic search! I’ve wanted this ability to pull up the right clip based on a simple description the moment I tried it in Google photos. Huge when you’re trying to find that *one* b-roll clip.

Adobe's new AI paper is pretty cool, and it's open source for a change. TransPixar can go directly from text prompt --> video footage with an alpha channel. Meaning that your clip is already "cut out" and ready to composite in your tool of choice. Useful in video editing tools like CapCut or Premiere where you can conjure up stock elements on demand (e.g. smoke effects, magical portals, explosions) and drag them right in. Equally fun for making memes in DingBoard or animated stickers in social apps like instagram. Now let's go a bit deeper: → You know how text-to-video AI has gotten pretty good lately, but everything it makes is solid RGB -- meaning no transparency. That's a pain if you want to composite the videos into other scenes or do VFX work. → While you can use something like Meta's SAM2 to segment out any AI video, these segmentation models tend to fall apart with volumetric effects like explosions, or finer structures like fur, hair and foliage -- giving you a flickery mess that's hard to work with. What makes TransPixar clever is how they tackled this: → Instead of building a new model from scratch, they figured out how to extend existing text-to-video AI models to handle transparency (the alpha channel) alongside the regular RGB output. → They did this by using some smart attention mechanisms and LoRA fine-tuning. And a remarkably small dataset -- just 484 training videos with transparency. That's tiny by AI standards. → For a change, Adobe has released the code on GitHub and it's free for non-commercial use. I expect others to embrace this approach and reproduce these capabilities with open & closed source video models. This could be huge for indie VFX artists and content creators who need quick, high-quality, read-to-composite elements. And yeah, meme lords about to have a lot of fun with this :)

Woot! Two REALLY cool features coming to Adobe Premiere. 1) Generative Extend available today in beta. Take any clip and extend it — with the video and audio automagically extended :)

BREAKING: Adobe Firefly Video is “the first commercially safe video generation model” — it supports text-to-video, image-to-video and designed for immaculate prompt coherence. Those sample generations look quite impressive — excited to go hands on! I’m at Adobe MAX this week so stay tuned for more updates on my feed: @bilawalsidhu