Can an open model spot AI-generated video?

We make short films with AI, so we care about a question many people now ask: can a model tell a real video from a generated one?

We trained Coeus, our open Video Jev model, to answer it as a yes-or-no question with a calibrated probability. The headline numbers are strong. The more useful part of this post is how our first attempt fooled us.

Full breakdown of results: every benchmark, split and baseline is on the Coeus model card on Hugging Face, with the weights. The code, the paper and every experiment are on GitHub.

The results

Test	Clips	AUC
VideoFeedback real or AI, official test	760	0.996 (untrained 0.905)
GenVideo real or AI, generators never seen in training	961	0.942

AUC measures how well a model separates two groups. 0.5 is a coin flip and 1.0 is perfect. The model also says how sure it is: across the quality and real-or-AI questions its calibration error is 0.02 to 0.05, so a 90% answer means about 90%.

The GenVideo number matters most. Those clips come from video generators the model never saw during training, so it shows the model learned something general, not the quirks of one generator.

How our first version scored 0.999

Our first detector scored 0.999 on GenVideo. So did a simple CLIP baseline. That should have been the warning.

In GenVideo, every real clip is 10 seconds long at 640×360, and every generated clip is 7.2 seconds or shorter. Our first record put the clip's duration into the context. Both models had learned "short means fake", not anything about the pixels.

We fixed it by making every clip look the same from outside: the first 1.5 seconds only, a square crop, 256 pixels, and no duration anywhere. The scores above are from that version.

The lesson we wrote into the paper: a baseline near 1.0 is a warning, not a result. If a detector you are testing seems perfect, check what else differs between the two groups: length, resolution, frame rate, compression, file names.

What it cannot do

• It was trained on 2023 to 2024 generators. Video models improve every few months. Do not rely on it for current ones without testing.
• It looks at 1.5 seconds. That was the price of removing the length shortcut. Longer clips may hold more evidence than it uses.
• One training run. No confidence intervals yet.
• Research use only. The weights are licensed for non-commercial use.

This is not a deepfake detector for journalism or moderation, and we do not offer it as one. It is a research result about what an open audio-visual model can learn, with the method published so others can test it.

Try it

The model is on Hugging Face as sp1d5r/video-jev. Ask it:

{"state": {},
 "questions": {
  "ai": {"type": "noul", "instructions": "Is this clip AI-generated?",
         "criteria": {"true": "yes, AI-generated", "false": "no, a real recording"}}}}

The paper, the data rules and every experiment are in the repository.

See every number: the results table, the weak spots and the quick start are on Hugging Face. The training code and the full paper are on GitHub.

More on Coeus:

• Coeus, our open Video Jev model
• What is a Jev model?
• Can one model predict which short videos people keep watching?
• Fine-tuning Qwen3-Omni on video and audio: our recipe