How to get a Jev model to watch video ads: 777 Meta ads, scored

Most ad analysis reads the ad copy. The copy is about 50 words. The video is where the ad actually happens: who is on screen, what you hear, what the first three seconds look like.

So we asked Coeus, our open Video Jev model, to watch the ads instead. It reads the frames and the soundtrack together, and answers typed questions with a probability for every answer. We gave it 777 video ads from the Meta Ad Library and nine questions it was never trained on.

Full breakdown of results: Coeus, its weights and its benchmark results are on the model card on Hugging Face. The code is on GitHub.

The short version

• Trust it for: is this an ad, is there a face in the first 3 seconds, speech, music, and the ad's format.
• Use with care: hook type, and "AI-generated" (on ads it means "synthetic or motion graphics").
• Not useful yet: hook strength, product timing, on-screen text. These need training data.
• Cost: about $0.0013 an ad on one H200 GPU. All 777 ads cost about $1.
• What runs longest: the brand matters most. Within a brand, ads that open without a face ran a little longer.

The data

• Source: the Meta Ad Library, video ads only, scraped with Apify for about $7.50.
• Brands: 22 AI brands, including ChatGPT, Claude, Cursor, ElevenLabs, Runway, Suno, Grammarly and CapCut.
• Ads: 1,341 scraped, 777 unique videos after removing duplicates of the same video.
• Label: every ad started on or before 2026-08-19, so each had 60 days to run before we scraped on 2026-10-18. An ad "survived" if it ran 60 days or more. 124 of 777 did (16%).

Advertisers keep paying for ads that work and switch off the ones that do not. So days live is a rough, public stand-in for performance. It is not spend, click rate or watch time, which only the advertiser can see.

How to ask the questions

A Jev model takes a record: some state and a set of typed questions, each with its allowed answers. If the format is new to you, What is a Jev model? explains it in five minutes.

These are the ad questions we asked, in one record:

{"state": {},
 "questions": {
  "is_ad": {"type": "noul", "instructions": "Is this video an advertisement?",
            "criteria": {"true": "yes, an ad", "false": "no, not an ad"}},
  "format": {"type": "choice", "instructions": "What format is this ad?",
             "criteria": {"ugc": "a person talking to the camera, UGC style",
                          "demo": "a screen recording or product demo",
                          "testimonial": "a customer testimonial or review",
                          "text": "text or captions over a plain or stock background",
                          "animation": "animation or motion graphics",
                          "cinematic": "a cinematic brand film with actors or scenes",
                          "skit": "a skit, meme or comedy sketch"}},
  "hook_strength": {"type": "score",
                    "instructions": "How likely are the first three seconds to stop someone scrolling?",
                    "criteria": ["very unlikely: nothing grabs attention", "unlikely", "maybe",
                                 "likely", "very likely: impossible to scroll past"]},
  "face_open": {"type": "noul",
                "instructions": "Is a person's face on screen in the first three seconds?",
                "criteria": {"true": "yes, a face", "false": "no face"}},
  "product_moment": {"type": "moment",
                     "instructions": "When is the product first shown on screen?"}}}

The full set also asked about the hook type, speech, music and on-screen text, plus the trained "Is this AI-generated?" question.

Run it on an ad with the script from the model card:

git clone https://huggingface.co/sp1d5r/video-jev && cd video-jev
python run.py --video ad.mp4 --record ad_questions.json

It reads the first 60 seconds at 2 frames per second, with audio. In our batch run, part of the answers for one ad looked like this:

{"is_ad": {"yes, an ad": 0.9189, "no, not an ad": 0.0811},
 "format": {"a screen recording or product demo": 0.4722,
            "a person talking to the camera, UGC style": 0.2319, ...},
 "hook_strength": {"very likely": 0.4529, "likely": 0.3763, ...}}

Every answer is a probability, so code can act on it: filter, rank, or flag for a person.

How far to trust the answers

None of these questions were trained, so every answer is zero-shot. We checked them three ways.

Ad or not. Against 300 FunQA clips that are not ads, Coeus separates ads with AUC 0.983. Its mean probability of "ad" is 0.82 on ads and 0.12 on the other clips.

Agreement with Gemini 2.5 Flash. Gemini watched the same 199 random ads and answered the same questions. It is a second model, not a person, so this measures agreement, not truth. The last column is how often you would agree with Gemini by always giving its most common answer.

Question	Agreement	Most common answer alone
Face in the first 3 s	0.92	0.67
Music	0.87	0.74
Speech	0.83	0.64
Format	0.74	0.38
Hook type	0.45	0.25
On-screen text	0.99	0.99

Nearly every ad has text on screen, so that question tells you nothing. Hook strength agreed only weakly (Spearman 0.28), and so did the moment the product first appears (within 2 seconds of Gemini on 49% of ads). The most common format mix-up was Coeus saying "demo" where Gemini said "UGC" (13 of 199).

By eye. On 6 random ads, the format was right on 5 and the face answer was right on 5. A Grammarly talking-head ad was called "demo". A Luma ad made of AI images got a 0.98 probability of AI-generated, which is right. But Coeus also calls motion-graphics ads AI-generated. It learned "not filmed by a camera", so on ads, read that answer as "synthetic or motion graphics".

What runs for 60 days

The brand matters most. 11 of the 22 brands kept no ad running for 60 days, including Claude, ChatGPT and Cursor. Suno ran 70 ads in August, each for about 5 days. At the other end, Zapier kept 81% of its ads running 60 days, and Runway 56%.

That makes raw tables misleading. Here is survival by format across all 777 ads, with 95% intervals:

Format	Share of ads	Ran 60+ days
Demo	37%	18.7% (14.6 to 23.6)
UGC	36%	11.3% (8.2 to 15.6)
Animation	11%	27.7% (19.2 to 38.2)
Cinematic	9%	15.5% (8.9 to 25.7)
Skit	4%	2.9% (0.5 to 14.9)

UGC ads at 11% against animation at 28% looks like a big lesson about UGC ads. Most of that gap is which brands use which format.

So we compared each ad only with its own brand's other ads. The p-values come from shuffling the labels within each brand 500 times.

Signal	Effect within brand	p
No face in the first 3 s	+3.3 points	0.01
Coeus hook strength	AUC 0.58	0.05
Coeus probability of AI or synthetic	AUC 0.58	0.04
Demo instead of UGC	+2.5 points	0.26, not significant
A "result first" hook	+1.6 points	0.31, not significant

What we can say:

• Within a brand, ads that open without a face, and ads made of synthetic or motion-graphic footage, ran a little longer.
• Coeus's hook strength score has a small link to an ad running longer, even though it agrees only weakly with Gemini.
• The effects are small. This is 22 brands in one category, and days live is not performance.

Speed and cost

	Per ad	All 777 ads
Coeus on one H200, both passes	about $0.0013, 1.0 s	about $1, plus 1.5 minutes to load the model
Gemini 2.5 Flash on the same videos	about $0.004	about $3

Coeus used about 6,700 input tokens per ad. Gemini used about 10,700 input and 480 output tokens per ad.

What comes next

• Hand-label about 200 ads for hook type, hook strength and product timing, and train those questions.
• Take ads from more brands and outside AI, with more ads per brand, so the within-brand tests have more power.

If you run ads and want to test your own, the record above works as it is. Change the questions to the ones you care about.

See every number: the results table, the weak spots and the quick start are on Hugging Face. The training code and the full paper are on GitHub.

More on Coeus:

• Coeus, our open Video Jev model
• What is a Jev model?
• Can one model predict which short videos people keep watching?
• Can an open model spot AI-generated video?
• Video moment retrieval: finding the funniest second