What’s The Impact of Adobe’s “Media Intelligence” for Premiere Pro?

New in the latest Beta release of Adobe Premiere Pro is “Media Intelligence.” This is an AI-assisted visual search tool for media that “works by analyzing your footage using on-device models. The analysis is collected into an index file saved next to your project. Then when you search for visuals, the description you type is analyzed and compared to your footage to find the best matches.” (Adobe Help)

NOTE: The current beta release supports English queries only.

When you import footage, Premiere analyzes it in the background, then you can type queries using language based on imagery, spoken words, or content with embedded metadata like shoot date, location, or camera type. It will display not just whole clips, but those portions of a clip which match that query.

Currently, there are several limitations:

• • It can find people, but it can’t find or label specific people.
  • It can search transcripts, but can’t search audio.

• It can’t find specific text recorded in a video (i.e. optical character recognition).
• It can’t use custom search terms, like GREP. It searches only for exact text matches to the query.
• Analysis must be done in Premiere, you can’t use Media Encoder or run the utility separately.
• While the search index engine can only be used inside Premiere, the resulting index can be shared between Premiere editors.
• Adobe does not train or update its AI models based on your searches or media.

While still limited, Media Intelligence seems like a major step toward building media management software into Premiere. So, what does this mean?

WHAT’S THE IMPACT?

There are a number of companies that specialize in creating media asset management software. One of the newer entries is Jumper, which operates as a plug-in extension for Premiere and Final Cut. I sent Max Lund, CEO and Founder of Jumper.io  some questions to help me learn more about Adobe Intelligence and its impact on third-party media asset management systems.

Larry: What’s your reaction to Adobe adding “Media Intelligence” into Premiere?

Max: Mixed—I suspect it might get more eyeballs on Jumper since we might naturally be included in the conversation. But, of course, we’re not thrilled that there’s now another solution available that runs locally on your device—especially since the built-in solution from Adobe comes at no extra cost for users.

When Adobe first released the Firefly image generation model as a “copyright-safe” alternative to, say, Stable Diffusion, it was comically bad. From what I’ve heard from the Jumper community and our own tests, Adobe’s version is still pretty far from Jumper’s accuracy right now. But to be completely honest, it’s not as bad as I would have hoped—i.e., it’s not a repeat of the Firefly situation.

Larry: What role do AI models play in determining speed and accuracy?

Max: The more parameters a model has, the more computations need to be performed, making it more resource-demanding—which usually translates to increased search accuracy. Of course, your computer specs also determine how fast things run, so a smaller model might perform slower than a larger model on a different system.

The most dramatic difference in the new Jumper update is the ability to use “multilingual” models, which makes it possible to search in any language with good accuracy—not just English. Beyond that, we also offer the ability to run models that accept higher-resolution frames as input. This improves performance for tasks like searching for text in videos or finding very small, intricate objects, which might be distorted or lost when downscaled to a lower resolution.

Larry: How, if at all, does it matter when an AI model was created?

Max: The training data determines what a model knows, so to speak. For example, if a new machine called a “flubberwonk” was created today, our models wouldn’t recognize it unless we fine-tuned them with the relevant data. However, for Jumper’s use cases, I don’t think this will be an issue in most situations. Even if you can’t type “flubberwonk,” you can still describe it visually—e.g., “a yellow rectangular suction device” (or whatever it looks like)—and find it that way.

Larry: If Adobe bundles “Media Intelligence,” where’s the need for third-party media management tools?

Max: Adobe serves any and all users who can run Premiere. They also limit themselves to building and training their own models on their own data. These two facts restrict them in terms of model size (they can’t be too large, or not everyone could run them) and prevent them from leveraging state-of-the-art research. Another key point is that Jumper’s analysis is cross-platform and cross-NLE, so you can use the same analysis across different editing programs with Jumper. It also optimizes team-centric use cases, where you process the same footage across multiple machines to a central cache—say, on a NAS. Using multilingual models for non-English speakers is another advantage Adobe’s version doesn’t offer.

In general, I suspect that while Adobe’s version may be sufficient for some, they’ll always be a step behind and move much slower than Jumper can. Any new open-source research can be incorporated into Jumper within days, and requests for custom solutions are far easier for smaller players like us to meet. It often doesn’t make sense for Adobe to build solutions for more niche use cases. For example, we could incorporate a smaller version of the open-source DeepSeek LLM running locally (i.e. completely privately) on your computer, to do any number of interesting tasks using the video analysis and transcriptions. Doing something like this would be prohibitively computationally costly for most of Premiere users, even though it could be extremely useful for those that have the resources to run it.

At the moment, Jumper also offers a variety of “search-by-frame” options, which our users have greatly appreciated—some rely on them almost exclusively instead of text-based searches. This isn’t something Adobe’s version supports right now.

Larry: Max, thanks for taking the time to share your thoughts.

What I like about what Adobe is doing is that they are not training their system on the backs of your data. They are not trying to replace an editor, but to enable an editor to work more quickly – and, truly, spending an hour searching for a shot that “you know is in there somewhere,” is not a good use of anyone’s time.

We will just need to see how this is actually implemented into the real-world.

EXTRA CREDIT

When Max read this last comment after I published our interview he added: “I would say that these vision encoder models that can analyze images are pretty benign and don’t have the same concerns as generative AI, where it actually creates some text or media of the same type it was trained on.”

Larry adds: In other words, vision encoder models that recognize images are tools for reviewing existing work, while generative AI is creating “original” work.

Max continues: “I find it very unlikely that fully autonomous AI agents that do serious video editing by chatGPT-style prompting is going to be a thing anytime soon, and whatever it can do is going to be very limited. Tedious tasks might be removed, but I wouldn’t be worried as a video editor – today us programmers have competition from much more mature solutions for “no-code” builders that let anyone build apps by natural language (see e.g. https://lovable.dev/).

“I’m not exactly worried about my future job prospects because of these tools – similarly I don’t think competent professional video editors should be either. Although it might make it harder for juniors to get their first jobs since programmers and editors can become a lot more productive. [This means that] the number of jobs might shrink [because] tedious but simpler tasks that junior employees used to do are becoming automated.

“But for the most part I don’t think that’s any different from the job loss seen historically by the introduction of agricultural machines, or any other technological advancement.”

---

Bookmark the permalink.