170

OpenSubs: I built a subtitle generator that runs Whisper in the browser, so the video never gets uploaded (opensubs.app)

submitted 1 day ago by opensubs@lemmy.world to c/opensource@lemmy.ml

48 comments fedilink hide all child comments

I work on OpenSubs, a free, open source (AGPL-3.0) subtitle tool that runs entirely in the browser tab.

You drop in a video file, and Whisper transcribes it on your own machine, using transformers.js with WebGPU where available and WebAssembly otherwise. The model (40–250 MB) downloads once and is cached. There is no upload endpoint in the product, so the video has nowhere to go.

After that you can:

fix lines by typing over them (click a timestamp to jump to that moment)
translate into 20 languages with Chrome's built-in on-device translator
pick one of 12 caption styles, including word-by-word highlighting
export SRT / VTT / ASS, or burn the subtitles into an MP4 (libass compiled to WebAssembly, encoded with WebCodecs)

A few things I learned building it:

Whisper hallucinates on silence and music ("Thanks for watching!", or the Japanese equivalent). A Silero VAD pass runs before Whisper, and a cleanup step drops the known stock phrases.
Singing doesn't count as speech for the VAD, so a music video gets a "no speech found" warning. You can still force it.

Honest limits: it only takes video files, not audio-only files. Cue timings can't be edited yet. Builds are release candidates. Everything that runs locally is free with no account; the only paid part is optional cloud translation on our backend (US$5 for 1000 credits), and you can bring your own Claude / OpenAI / DeepL key instead.

Site: https://opensubs.app/ Code: https://github.com/open-subs/opensubs

Feedback welcome, especially on languages where the transcription goes wrong.

you are viewing a single comment's thread
view the rest of the comments

[-] mysterious_cake@feddit.nl 4 points 23 hours ago* (last edited 17 hours ago)

Did whisper improve since last year?

Last time I tried to generate subtitles for a German TV show, the subtitles were more inacurrate than what I would have written down (German is my third language and I struggle to understand anything that isn’t voiceovered) and timing was so bad I didn’t even bother to generate subtitles for the second episode.

[-] geneva_convenience@lemmy.ml 12 points 21 hours ago* (last edited 21 hours ago)

No but other models have. parakeet-unified is currently the best small model.

You can tell OP vibe coded the whole thing from their outdated model choice.

[-] victorz@lemmy.world 7 points 21 hours ago

You can tell OP vibe coded the whole thing from their outdated model choice.

I hate this landscape we're in now. I'm always suspicious of whether something was created by a thieving computer program or a real human. Just like I'm always suspicious of crazy videos these days being AI or not. Neither can I watch an ad anymore without suspicion. It's so much extra mental overhead.

I'm tired, boss.

[-] geneva_convenience@lemmy.ml 8 points 21 hours ago* (last edited 6 hours ago)

You can also just click the repository and see that Claude is a contributor which usually gives it away.

My point is more so that OP didn't even put in the effort to ask claude to look up the current best ASR model to use but instead one-shotted the thing

[-] CeeBee_Eh@lemmy.world 1 points 6 hours ago

Which one is the best model now? Last I checked (well over a year ago) it was still whisper.

[-] geneva_convenience@lemmy.ml 1 points 6 hours ago* (last edited 6 hours ago)

IBM Granite Speech 4.1 or Cohere Transcribe according to current benchmarks.

IBN Granite Speech 5 appears to be a great fast option as well for fast transcription but it's English only. Parakeet Unified is a good fast modern option with multi language.