this post was submitted on 26 Nov 2023
8 points (100.0% liked)

Self-Hosted Main

515 readers
1 users here now

A place to share alternatives to popular online services that can be self-hosted without giving up privacy or locking you into a service you don't control.

For Example

We welcome posts that include suggestions for good self-hosted alternatives to popular online services, how they are better, or how they give back control of your data. Also include hints and tips for less technical readers.

Useful Lists

founded 1 year ago
MODERATORS
communick@selfhosted.forum
8
Good way to store a lot of photos - hobby photography (alien.top)
submitted 1 year ago by Loganthehatless@alien.top to c/main@selfhosted.forum
15 comments fedilink hide all child comments
 

Hey Guys, My mum is doing semi professional photography and I am at my wits end. It’s a human and technological problem. I did a quick object count she has like 70k photos roughly 4tb. This happens because she takes every picture in raw and JPEG and a lot of series captures. The begin of the story is that she had at first a ssd, then a second, then a third and so on. I already bought a synology nas. And threw everything at it. But everything is messy and unsorted and she is not happy because she doesn’t get her chaos together and adobe Lightroom performs bad with network drives, and I don’t get why … but this seems to be a known problem… Anyways she uses Lightroom for her editing which is nice, but she is using more like a library and not to perform the actual changes, that’s the reason that the catalogue which is a db of the changes is a 17 gb.

She is not happy at the current state. Do you have suggestions, for a strategy to clear this chaos ? Or a cool tool for getting a folder structure? Maybe any tips and tricks for synology and network stuff ?

I Already tried to move files and get a structure but Lightroom hates this and loses track of the file. So a powershell script which sorts the items into year folders was a good idea but I am scared of bricking the db

The nas and the mac are all wired up on 1 gbit and I am sure it should be ok because the big raws are only like 70mb per file

Regards :)

top 15 comments
sorted by: hot top controversial new old
[–] Franconian_Nomad@feddit.de 2 points 1 year ago (1 children)

Okay, tell her to: Purge all doubles and objectively bad pictures (like unintentionally out of focus ones) from her library and also to keep only the best ones from a series. She should get rid of the JPEG’s or at least save them separately. Yes, weeding out that will take a long time with that many photos. I know the struggle.

Also, adobe lightroom is great, but I’m not sure if it’s that good when only used as a library. Maybe look for some alternatives. They might be even more functional with your NAS. There’s an opensource Programm called Darktable for example. Might be worth checking it out.

I’ve heard many professional and semi-professional photographers are afraid of data loss. So many keep the following precautions:

  • Never reuse SD-cards, archive the full ones and buy new ones. Copy them to hard drives and/or external ssd‘s.

  • When the ssd‘s are full, archive them and get new ones.

  • Have a copy of all pictures in the cloud. I believe adobe offers a monthly plan for example.

Sorry, I’m just a hobby photographer myself, so that are just my 2 cents. Hope I could help at least a little bit.

[–] itchick2014@midwest.social 3 points 1 year ago (1 children)

I’ve done professional photography for a decade and have never seen someone not reuse a SD card…even outside of myself. Most photographers I know use cloud services to back up.

[–] Franconian_Nomad@feddit.de 2 points 1 year ago

Ah, interesting. I think it heard in several podcasts. But I guess those guys were pretty paranoid about losing data.

[–] Plane_Resolution7133@alien.top 1 points 1 year ago

It was (is?) common to also have multiple libraries. Like a temp library, could be used on imports for example. A done library for client jobs completed. A personal library etc.

[–] Plane_Resolution7133@alien.top 1 points 1 year ago (1 children)

Unless LR have changed, there’s built in tools to move and manage the db <-> file relationship.

There used to also be tools to prune/trim the db.

The root of the problem might her workflow. Especially when shooting series, when she’s back/at the computer, immediately go through the photos and delete whatever’s out of focus/uninteresting/etc.

Maybe she’s good at the workflow? I know how tedious it is to sit down and sort out a huge archive like that.

I used LR for a long time, but got tired of backing up and managing the files. I moved to iCloud and I’m happy with that.

The Apple Photos program is extremely basic, but it’s a decent way of rough organising photos.

Using tags/project/etc. is important.

As for folder structure, I used (year) - (month) (day). That with at least some basic tags made it manageable.

[–] Loganthehatless@alien.top 0 points 1 year ago (2 children)

Yeah she used year month day at first but a such a load she said I need to organize it like in categories. I can’t say anymore in which year I was in Greece or at the Lofoten and so on … and this is now messy af. I already consolidated everything on a 4tb hdd and give her the homework to sort it and later we will have it on the nas, and use this drive as a backup, but it’s challenging 🥲 and I try to find something smooth to make it less demanding

[–] ManSpeaksInMic@alien.top 1 points 1 year ago

Tags, tags, tags.

My photos are stored by date ("yyyy/mm/dd - Main description", e.g. "2018/04/19 - Lofoten holiday"), but that is by far not enough to find anything meaningful at any time.

Meticulous tagging with a well-structured taxonomy is the only real choice here, in my book. AI tools, like are integrated into immich and other tools, help building that library. But for what I do with photos, how I think about photo gategories, AI can only go so far.

Tagging appropriate subject matter (family, friends, still lives, architecture), surroundings (location, as in "Lofoten", but also location, as in "at the beach"; indoor/outdoor; nature; event; ...), theme (light painting, black-and-white, nighttime, high fashion, random funny snapshot, ...) and even species (I photograph a lot in zoos) is necessary.

With that kind of thing you then can find "that funny photo of Uncle Roger at the beach, when he slipped on the stray dog toy!" in the 4TiB of photos.

And that's, unfortunately, work.

(Also, I am harsh in culling my RAW files. Ofc precious memories stay even if the photo isn't perfect, but if I go out on photo walks, anything that is blurry/unsharp or badly exposed has the original RAW deleted as well. Or selecting the best candidate from chimping, I don't need the other ones anymore. But that, too, is a workflow problem.)

[–] spokale@alien.top 1 points 1 year ago

She's using Lightroom classic right?

After importing from the SD or CF card, Lightroom stores the original imported photos in a YYYY\YYYY-MM-DD directory. That's usually where the bulk of the disk usage comes from.

Personally I just move those archived photos older than a few months to external storage and re-map in Lightroom.

[–] dazchad@alien.top 1 points 1 year ago

As an experienced software developer, I can confidently say that no amount of technology will fix a bad user workflow. Your best plan of action is to sit down with your mother and try to come up with a consistent workflow she is happy with. If she doesn't know or can't come up with one, find what is the industry standard (which seems to be year-month-date folders? I'm not a photographer), or maybe ask an experienced photographer friend. Remember you can adjust the workflow later, but it's important to have something stable while she is learning how she really likes to work.

Storing the photos in a NAS is a good choice. For the slowness, I suppose she would need to keep the most recent photos on her main computer/laptop, then move/sync to the NAS once editing is done? I don't see why one would need to edit old photos every day, so keeping only the most recent/active work on the computer seems smart.

adobe Lightroom performs bad with network drives

Most software perform worse with network drives because the host OS can't optimize as much as a local drive, specially so with random access like editing software do (contrast to copying/streaming a file, which is sequential). If performance is an issue, the only real solution is to copy files locally then sync back to the NAS. You can diminish the latency of network drives by having a SSD on the NAS, and a better link between them. Gigabit is a good start, but I'd go with 10Gbit because although the files are 70MB in size, lightroom is probably fetching several photos at the same time, and it will easily saturate your Gigabit link.

Good luck!

[–] chilanvilla@alien.top 1 points 1 year ago

You mention using a Mac, so how about Photos? I store +300,000 photos/videos ~3TB in photos on a dedicated Mac mini (not using iCloud) with attached SSD and an external drive for backup. Works super fast, searches, filters, identifying faces, locations. Gave up the concept of folders once I was all in and its all by albums and search.

[–] Tailslide1@alien.top 1 points 1 year ago

I tried the same thing years ago... wasn't able to find anything better than lightroom and gave up. Now there are larger SSD's I would maybe just throw a honking big SSD on her computer and set the synology to backup her PC using active backup or if you just need to mirror the folder to synology I find the program Bvckup to be really good for that.

[–] jdsmn21@alien.top 1 points 1 year ago

Take a tough stance with mother. Tell her she needs to start culling photos.

  1. There's no need to keep "RAW + JPEG". Blow the redundant JPEGs away.
  2. There's no need to hoard pics that aren't "keepers" (ie: 40 pics of the same pose, and the one best one was actually sent to the client)
  3. (in my opinion) - there's no need to keep RAWs long term. Edit them, save as JPEG (or fully delete), and blow the bloated RAWs away. (ie: photographers seem to be hoarders; no need to keep some photo shoot of family from 5 years ago that you will never hear from again)

All of this is easily done within the Lightroom interface.

[–] PovilasID@alien.top 1 points 1 year ago

Here are her choices:

Improved workflow + time investment in cleanup OR BURN BABY BURN THAT MONEY!!!

Jokes aside. She needs to start splitting photos she is actively working on and the ones that are effectively archived. I am sure there are ways to make the it still available either making a lower rez copy that lets her browse them in Imitch or some other tool that she would find more palatable... but there is going to be trial and error phase... not fun.

As for the money burning bit start... just buy bigger SSD... you can put them in DAS (direct attached storage) which is similar to NAS but is available only that one machine has deeper integration inside of system so may overcome the bugs lightroom is throwing at her.

[–] Darkchamber292@alien.top 1 points 1 year ago

Photoprism! Has a consume folder and handles large libraries well.

Normally I'd recommend Immich but Photoprism is a better fit for your use case

[–] tomlovestoplayinpubl@alien.top 0 points 1 year ago

Immich