[-] rkd@sh.itjust.works 3 points 6 months ago

You can even not have any data layer all together. The only thing missing from a local LLM is knowledge of current medications by name if you want to just say whatever prescription you're following.

[-] rkd@sh.itjust.works 2 points 6 months ago

For sure, context rot is a problem, but that's also the easiest thing to control for in this case. If sensor data is relevant to you, having some code to process and reduce it to a dashboard you can read is always a good idea, independently of getting an LLM into the loop.

This becomes more complicated with data you can't really understand like results from blood tests, for example. But maybe you just don't summarize any of that.

[-] rkd@sh.itjust.works 5 points 6 months ago

What we can call a health advisor is not a doctor. In fact, depending on the model, it will actively point you to seek medical help.

5

Seems to make sense for maximum privacy. Put together a large enough model to answer health queries, have OCR and image recognition to read exams, give it web access to search for medication details and, of course, gather raw data from any devices you use to measure weight, heart rate, etc.

But that's just in theory and we all know things are hard to put together. In practice, have you had any experience getting anything like this working locally?

[-] rkd@sh.itjust.works 3 points 11 months ago

I can read minds and they're thinking "we better get some money around here, otherwise we're still blaming the immigrants".

[-] rkd@sh.itjust.works 11 points 1 year ago

If they're not great, it's your fault /thread 😅

[-] rkd@sh.itjust.works 1 points 1 year ago

I believe right now it's also valid to ditch NVIDIA given a certain budget. Let's see what can be done with large unified memory and maybe things will be different by the end of the year.

[-] rkd@sh.itjust.works 2 points 1 year ago

can't have both

[-] rkd@sh.itjust.works 5 points 1 year ago

somebody do something any day now

[-] rkd@sh.itjust.works 6 points 1 year ago

no more fokin ambushes

[-] rkd@sh.itjust.works 3 points 1 year ago

I'm aware of it, seems cool. But I don't think AMD fully supports the ML data types that can be used in diffusion and therefore it's slower than NVIDIA.

15

Total noob to this space, correct me if I'm wrong. I'm looking at getting new hardware for inference and I'm open to AMD, NVIDIA or even Apple Silicon.

It feels like consumer hardware comparatively gives you more value generating images than trying to run chatbots. Like, the models you can run at home are just dumb to talk to. But they can generate images of comparable quality to online services if you're willing to wait a bit longer.

Like, GPT OSS 120b, assuming you can spare 80GB of memory, is still not GPT 5. But Flux Shnell is still Flux Shnel, right? So if diffusion is the thing, NVIDIA wins right now.

Other options might even be better for other uses, but chatbots are comparatively hard to justify. Maybe for more specific cases like code completion with zero latency or building a voice assistant, I guess.

Am I too off the mark?

[-] rkd@sh.itjust.works 14 points 1 year ago

it's most likely math

[-] rkd@sh.itjust.works 13 points 1 year ago

Congratulations Nintendo, you played yourself.

view more: next ›

rkd

0 post score
0 comment score
joined 1 year ago