Transcript
Having captions and not having them are entirely different routes. And what leaves the machine is audio, never video.
As expensive as frames are, transcripts are cheap — a few thousand tokens for ten minutes of video. So wherever possible, fill in with transcript and spend fewer frames.
The question is where the transcript comes from. There are two routes, and they behave nothing alike.
Route one — platform captions
Captions from the source platform get tried first. They’re free, and already timestamped.
The timestamps are the point. Each frame has a time, so if the transcript has times too, the two can be read against each other. “At 3:12 this is what was on screen while they said this” becomes a statement you can make, and that’s the form a spec needs.
Auto-generated captions are fine. A few wrong words don’t matter if the timing is right, and the frames correct the wrong words.
Route two — transcribe it directly
With no captions — or a local file — the audio is extracted and sent to a transcription API.
What leaves the machine here is audio, not video. The picture is never uploaded anywhere. And even the audio isn’t the original: it’s re-encoded to mono 16 kHz, around 0.5 MB per minute. That’s enough to understand speech, and there’s no reason to send more.
It uses whichever provider has a key configured. If one fails you can force the other, and there’s a switch to skip it entirely. With transcription off it proceeds on frames alone and records in the output that there is no transcript.
When there is none, it says so
No captions and no transcription means frames only. And that fact gets stated in the result.
The reason this has to be a rule is that a transcript-less result looks perfectly fine. Eighty frames still produce a plausible summary. But everything the presenter only explained out loud is missing entirely, and the reader has no way to know.
One line saying “no transcript” makes the reader treat the result differently. Without that line, nobody knows what isn’t there.
The 25 MB wall
The transcription API has an upload limit. A long enough video exceeds it on audio alone.
Downsampling to mono 16 kHz earns its keep here too — at 0.5 MB per minute, fifty minutes fits in one request. Anything longer is a video you should be watching by range in the first place, not transcribing whole.
On failure the error is passed through as-is. Whether it was the size limit, a bad key, or a rate limit shows up in the message, and not hiding the cause is what makes the retry decision possible.