23
submitted 4 days ago* (last edited 4 days ago) by ki4jgt@feddit.org to c/programming@programming.dev

Let's assume I have a software application with millions of files. The files receive frequent updates.

Is there a way to distribute those updates across all those devices?

I'm thinking something like Torrents/IPFS/Resilio. . . But they all seem like overkill.

I've been looking at my Search Engine, and thought it would be cool to allow people to download the database themselves, and run a local UI, for maximum privacy.

I would just be creating records within the main database. I could offer searches through the main site and allow people to explore it on their desktops.

you are viewing a single comment's thread
view the rest of the comments
[-] gole@lemmy.zip 10 points 4 days ago

Please take this as support: You need to look a little bit more into theories. You are reinventing things that don't need reinventing (categorizing data), but are not thinking about the real pain points of building search that return relevant results (full text search performance, scalability, semantics, extracting user intention from queries).

Here's an article that I think is very good on the topic

https://blog.wilsonl.in/search-engine

[-] ki4jgt@feddit.org 8 points 4 days ago* (last edited 4 days ago)

That's because I don't want that.

I remember, as a kid, wandering the local library. Looking at all the books. Seeing the call numbers and author names. Exploring the shelves, as they transitioned from topic to topic to topic.

My idea is to treat the Internet as a giant library, with domains being books on a shelf, and allowing my users to start at a random location, and then peruse the different listings as they transition from slight variations of the main topic.

With all due respect, you don't have the vision, man. To start looking at cultural anthropology, then have a link to a biological anthropology blog 3 lines down? That's the goal. It's not about a narrowed search. It's about cataloging something, so people can go looking for it. To enjoy the hunt.

this post was submitted on 24 Sep 2026
23 points (100.0% liked)

Programming

28593 readers
616 users here now

Welcome to the main community in programming.dev! Feel free to post anything relating to programming here!

Cross posting is strongly encouraged in the instance. If you feel your post or another person's post makes sense in another community cross post into it.

Hope you enjoy the instance!

Rules

Rules

  • Follow the programming.dev instance rules
  • Keep content related to programming in some way
  • If you're posting long videos try to add in some form of tldr for those who don't want to watch videos

Wormhole

Follow the wormhole through a path of communities !webdev@programming.dev



founded 3 years ago
MODERATORS