181

OpenSubs: I built a subtitle generator that runs Whisper in the browser, so the video never gets uploaded (opensubs.app)

submitted 1 day ago by opensubs@lemmy.world to c/opensource@lemmy.ml

55 comments fedilink hide all child comments

I work on OpenSubs, a free, open source (AGPL-3.0) subtitle tool that runs entirely in the browser tab.

You drop in a video file, and Whisper transcribes it on your own machine, using transformers.js with WebGPU where available and WebAssembly otherwise. The model (40–250 MB) downloads once and is cached. There is no upload endpoint in the product, so the video has nowhere to go.

After that you can:

fix lines by typing over them (click a timestamp to jump to that moment)
translate into 20 languages with Chrome's built-in on-device translator
pick one of 12 caption styles, including word-by-word highlighting
export SRT / VTT / ASS, or burn the subtitles into an MP4 (libass compiled to WebAssembly, encoded with WebCodecs)

A few things I learned building it:

Whisper hallucinates on silence and music ("Thanks for watching!", or the Japanese equivalent). A Silero VAD pass runs before Whisper, and a cleanup step drops the known stock phrases.
Singing doesn't count as speech for the VAD, so a music video gets a "no speech found" warning. You can still force it.

Honest limits: it only takes video files, not audio-only files. Cue timings can't be edited yet. Builds are release candidates. Everything that runs locally is free with no account; the only paid part is optional cloud translation on our backend (US$5 for 1000 credits), and you can bring your own Claude / OpenAI / DeepL key instead.

Site: https://opensubs.app/ Code: https://github.com/open-subs/opensubs

Feedback welcome, especially on languages where the transcription goes wrong.

you are viewing a single comment's thread
view the rest of the comments

[-] beyoublahaj@lemmy.world 1 points 1 day ago* (last edited 1 day ago)

If i understood this right...

Instead of uploading a file you just download the model. The sound then gets transcribed as usual without the service accessing the file.

[-] andycallisto@social.noleron.com -2 points 1 day ago

@beyoublahaj So audio gets uploaded?

[-] beyoublahaj@lemmy.world 1 points 1 day ago

oh nah I reread the post and edited my comment. I misunderstood too ^^`

[-] andycallisto@social.noleron.com 1 points 1 day ago

@beyoublahaj So what's the point of this service if you can download the model yourself and use it without this website?..

[-] magnue@lemmy.world 2 points 1 day ago

As people have tried to explain - it's not a website. The UI for the software simply runs via the browser.

[-] Ferk@lemmy.ml 1 points 1 day ago* (last edited 1 day ago)

You don't have to use the website.. the project is not just that website, it's also packaged as a browser extension, a desktop app and a CLI program.

That website is just a convenient way to see it working (while still processing it locally) without installing anything.

If you download the raw model and write up scripts to create subtitles out of video by yourself then you are essentially re-building the same app.. the result would be similar, just duplicating effort.

[-] thingsiplay@lemmy.ml 1 points 1 day ago

I assume you can use it on any platform that runs a browser, without installing software on the system. I use (other type of) software that runs in the browser, but locally. This type of browser software is handy.