32
building a pre index of IPFS CIDs & more (BTv2)
(app.radicle.xyz)
1. Posts must be related to the discussion of digital piracy
2. Don't request invites, trade, sell, or self-promote
3. Don't request or link to specific pirated titles, including DMs
4. Don't submit low-quality posts, be entitled, or harass others
We heartily recommend visiting the free port of freemediaheckyeah (aka FMHY) while you sail the high seas, for all the freshest links the ocean has to offer.
📜 c/Piracy Wiki (Community Edition):
FUCK ADOBE!
Torrenting/P2P:
Gaming:
💰 Please help cover server costs.
![]() |
![]() |
|---|---|
| Ko-fi | Liberapay |
This is definitely up my alley, I gave up on keeping all my media in my torrent client indefinitely for seeding because of the performance, so I've long dreamed of making some way to reconnect loose files back to torrents so I can seed them.
Seems I could maybe build something on top of this? I tried running magnetico for a while (going so far as to add postgres support to help it scale) but it quickly grows far larger than I want to manage.
My next idea is to make a file scanner that maintains a list of file paths and several common hashes, then do a dht crawl and only save stuff that matches. Then I can hopefully automatically add and remove torrents to a client that has read-only access to the files as needed (remove if plenty of seeders, keep for a while if no or low seeders and rotate through prioritizing stuff that needs seeds)
I'm wondering if there's some useful overlap between what you're doing and my goals but I think I need to dig into it more.
Hi, yes it definitely sounds similar for the media files database side. Using a DHT crawler, you can identify new torrents matching specific file tree roots (so only works for bittorrent v2, which is not used so much for now), and update swarms statistics (S/L).
Sounds cool, wonder if only make sense for vid only. What if smaller files like txt, img gallery
I don't think it would scale for many millions of files.
What if file name/path change? Scale resolution up/down? Genre/Subgroup in future?
Hi, not sure I get your point. A release has a fixed name and resolution. Regarding genre classification, I estimated if was too complex to determine reliably. And I wanted to avoid storing metadata.
Dates folders are there to reduce directory sizes, similarly to a merkle tree with only one level. This is due to performance limitations with large directories in most filesystems and also in Git. Also it still allows for easy manual search (only the date is required).
Note: there can still be several releases for a single movie (several resolutions and sources, LIMITED/REPACK etc).
I think you are also tried to solve same video but different name, release, resolution by different uploaders problem.
For now it's only scene releases, so there's no duplicate (not sure if this is what you mean).