I am trying to solve a specific problem that has some other applications.
My specific problem is I want to add an English word to every file from my camera. The files come in like [date]_time.jpg. Although I think the best workflow involves using a photo management app like DarkTable, sometimes you just want to quickly browse your filesystem and grab the original image. Browsing just by datetime creates problems when you take two very similar images seconds apart. It takes a lot of effort determining which is which. And if you pick out your best picture from a shoot, you then have to mentally note you want timestamp _12345678 and find it in your folder.
I think it would be smoother to have files like:
20261019_cape_1234567820261019_echo_12346678
In my specific case, 100 unique words is the absolute minimum, with 500-2000 words being more comfortable.
This is trivial if you have a "wordlist", a list of words you find suitable for the task. And there's the rub. What's a good wordlist? I can hack one together myself but I'm hoping some top minds have thought about this.
Some prior art that I've found is:
- Diceware and the EFF diceware lists
- Some bitcoin folks created 2048 words that are uniquely identified by the first 4 letters of the word
- a lot of default usernames take the form
adjective-noun, but I don't know where they get their wordlist - router default passwords
This task is nontrivial in my use case. Words can:
- evoke computers - copy, original, resize, error, fail, process, base, cat, category, data, base, idle, arm
- evoke other common file formats - album, music, song, camera, photo, video
- evoke the contents of an image or file - cool, neat, tree, car, sky, beautiful, best, trip
- evoke common filenames and extensions - wave
- be full on offensive
- be offensive in some contexts, as well as have a negative connotation in the most literal sense: bad, corrupt, awful, awkward, bomb
This is partly an "I know it when I see it" problem, but I'm kind of looking for the 500 blandest, most non-ideological English words there are.
I would consider using LLM for language processing as they ware made for that purpose. You could try generating or filtering out unwanted words.
Using complex, power-hungry matrix algebra to get words that could come from an indexed array in constant time is just about the worst solution of which I can think.
Worth noting, it may be that you meant to generate a list as a one-time operation. That is maybe sensible. I originally interpreted you to mean on the fly generation.
And it won't have a clue if you consider "asbestos" an unwanted word. I mean that's all highly subjective and an AI will (most likely) just do whatever.
A small locally run model would work and it would use less electricity than what's needed for spending time researching different lists from different websites.
This seems like a problem that can be approached entirely in embedding space. Find embeddings for words that you want to avoid, then filter your dictionary to remove anything too close to those.
No need to go through full text gen or anything, though if you want to go that route, a classifier like Jev (or OpenJev/etc) would be best.
Yes, the two approaches I was thinking (because it's so subjective to my use) was manual review or using something like Jev.