view the rest of the comments
Linux
Welcome to c/linux!
Welcome to our thriving Linux community! Whether you're a seasoned Linux enthusiast or just starting your journey, we're excited to have you here. Explore, learn, and collaborate with like-minded individuals who share a passion for open-source software and the endless possibilities it offers. Together, let's dive into the world of Linux and embrace the power of freedom, customization, and innovation. Enjoy your stay and feel free to join the vibrant discussions that await you!
Rules:
-
Stay on topic: Posts and discussions should be related to Linux, open source software, and related technologies.
-
Be respectful: Treat fellow community members with respect and courtesy.
-
Quality over quantity: Share informative and thought-provoking content.
-
No spam or self-promotion: Avoid excessive self-promotion or spamming.
-
No NSFW adult content
-
Follow general lemmy guidelines.
Say you have files in /home/bridgenjoyer/Data, /mnt/Backup, and /mnt/data
So
So, for identical files, you get identical repeated checksum antries.
You can filter out repeated entries (e.g. using awk, a shell script, or python) and extract their name and path.
There are also various tools which deduplicate identical files by hard linking them, if they are on the same file system. These will get the same inode number shown with ls -l. You can find these tools in the Arch wiki.
Then you need a strategy how to organize these identical files. You cold put all files into an archive folder, tidy them up, and only make and keep backups which you don't modify.
Oh, and one goodie: the "file" command can tell you the file type of a bunch of files, so you can quickly separate e.g. word documents from photos...