https://infosec.exchange/@malwaretech/114903901544041519
the article since there is so much confusion what we are actually talking about https://edition.cnn.com/2025/07/23/politics/fda-ai-elsa-drug-regulation-makary
Right, I'm no expert (and very far from an AI fanboi), but not all "AI" are LLMs. I've heard there's good use cases in protein folding, recognising diagnostic patterns in medical images.
It fits with my understanding that you could train a similar model on more constrained datasets than 'all the English language text on the Internet' and it might be good at certain jobs.
Am I wrong?
You are correct. However, more often than not it's just like the image describes and people are actually applying LLM's en masse to random problems.
what ai, apart from language generators "makes up studies"
Hallucinating studies is however very on brand for LLM as opposed to other types of machine learning.
Technically, LLMs as used in Generative AI fall under the umbrella term "machine learning"…except that until recently machine learning was mostly known for "the good stuff" you're referring to (finding patterns in massive datasets, classifying data entries like images, machine vision, etc.). So I feel like continuing to use the term ML for the good stuff helps steer the conversation away from what is clearly awful about genAI.
There is no generative AI. It's just progressively more complicated chatbots. The goal is to fool the human into believing it's real.
Its what Frank Herbert was warning us all about in 1965.
That’s because “AI” has come to mean anything with an algorithm and a training set. Technologies under this umbrella are vastly different, but nontechnical people (especially the press) don’t understand the difference.
Right. You're talking about specialized AI that are programmed and trained to perform very specific tasks, and are absolutely useless outside of those tasks.
Llama are generalized AI which can't do any of those things. The problem is that what it's good at, really REALLY good at, is giving the appearance of specialized AI. Of course this is only a problem because people keep getting fooled into thinking that generalized AI can do all the same things that specialize AI does.
The problems with AI we talk of here is mostly with generative AI. Protein folding, diagnostic patterns and weather prediction works a bit differently than image making or text writing services.
Obviously that should be in an advisory capacity, and not making decisions (like approving drugs for human use [which i heavy doubt was actually happening])
I'm constantly mystified at the huge gap between all these "new model obliterates all benchmarks/passes the bar exam/writes PhD thesis" stories and my actual experience with said model.
The real truth is just that standardized testing fucking sucks and always has
Likely those new models are varients trained specifically on the exact material needed to perform those tasks, essentially passing the bar exam as if it were open book.
Reminds me of a video that starts with the fact you can't convince image generating AI to draw a wine glass filled to the brim. AI is great at replicating the patterns that it has seen and been trained on, like full wine glasses, but it doesn't actually know why it works or how it works. It doesn't know the things we humans know intuitively, like "filled to the brim means more liquid than full". It knows the what but doesn't get the why.
The same could apply to testing. AI knows how you solve test pages, but wouldn't be that exact if you were to try adapting it into real life.
This reminds me of how like a hundred or so years ago people found "miracle substances" and just put them in everything.
"Uranium piles can level or power a whole city through the power of Radiation, just imagine what good this radium will do inside your jawbone!"
Literal... I cannot stress this enough... Literal Idiocracy.
This is literally what happens in the film. Like the first 10 minutes.
Fuck.
I can't remember, what happened in the film?
Luke Wilson's character goes to the hospital where an AI misdiagnosis him.
I'm pretty sure that undermining confidence in drug approvals is a feature, not a bug. The same people who were screeching about mRNA vaccines being secret poison that was rushed through approval are the ones doing this now, so when (not if) it does actually lead to dangerous drugs being approved and a collapse in confidence in the FDA, they'll be the ones saying "We told you so" and getting their anti-medical way.
It's the exact same playbook Republicans use in the rest of the government: Say Government doesn't work, cry about government spending, and insist government regulation is crushing personal freedoms, then they actually do all of those things and when the next administration comes around they pass on the blame and say "I Told You So."
The FDA need to get out of the way anyway. So much of what could be done isn't done because they take their sweet time with decisions.
The average approval time for a new drug is about a decade mostly because the FDA just don't do anything for the first 9 and 1/2 years. The covid vaccines were approved in a hot minute though and there was absolutely no issues with them despite what the conspiracy theorists thought. In fact they primarily based their conspiracy theory on the fact that normally the FDA takes forever and today in order to approve anything. Proving only that it doesn't need to take that long in the first place.
Yea I can say I called it. Instead of using graph neural networks trained for such a purpose (which have some actual chance of making novel drug discoveries), these idiots went on and asked chatgpt.
Can you imagine how sad those LLMs will be if they make a mistake that winds up harming people?
About as sad as the CEO
Not at all, because they are not thinking nor feeling machines, merely algorithms that predict the likelyhood of words following other words and spit them out
More so that the equivalent human? I have to think about this: https://www.youtube.com/watch?v=sUdiafneqL8
Someone needs to to a test, when this AI launches, they need to try and get poison approved as a medication. Like straight up a lethal dose of cyanide or something.
What happens when people realise and it immediately become the most popular drug on the market?
Win-win?
I was talking to some friends earlier about LLMs so I’ll just copy what I said and paste it here:
It really is like a 3d printer in a lot of ways. Marketed as a catch all solution and in reality it has a few things where it’s actually useful for. Still useful but not where you’d expect it to be given what it was hyped up to be.
From what I've seen, 3D printers are best at oversaturating local markets with a bunch of useless trinkets. (Just kidding though I know they have legit medical uses but oh my god)
I feel that's even worse with laser cutters and the cricut stuff, just useless trinkets wasting resources and ending up in drawers/gathering dust/in the bin.
Yeah they do create a lot of weird trinkets but they do have plenty of good uses. It can be used to make props and pieces of an outfit. Really really good for cosplayers. It can have a limited role in engineering. Mostly in low heat and stress environments for the engineering side of things but still useful for a cheap decently easy to replace part. Like you said it has medical applications.
It’s not an insanely useful tool for most people in most conditions but it’s still a great tool to have in some cases.
yeah like guns for example /s
From the mrna vaccines aren't tested well enough crowd
Our world was doomed to go into the gutter the moment we started giving any credence to the marketing bastards.
For specific things like protein folding "Ai" has been useful but that's not just a llm.
Yes, machine learning models trained to solve a specific problem can be very good at solving that problem. It's artificial "general" intelligence we haven't achieved but are trying to sell.
Oh shit. That MalwareTech? https://darknetdiaries.com/episode/158/
So is this a situation where it's kinda like asking chatgpt to make you drugs so it will go about any means necessary (making up studies) to complete the task?
Instead of reaching a wall and saying "I can't do that because there isn't enough data" I hope I'm wrong but if that's the case then that is next level stupid.
Yeah, they're using a glorified autocorrect tool to do data analysis, something that other, different types of machine learning models might be able to do with some decent results, but LLMs are not built for.
Gotcha, thanks
no it's supposed to help help the drug approval workers but everything it says has to be double checked so it ends up wasting time i put the article into the post now
A place to share screenshots of Microblog posts, whether from Mastodon, tumblr, ~~Twitter~~ X, KBin, Threads or elsewhere.
Created as an evolution of White People Twitter and other tweet-capture subreddits.
RULES:
RELATED COMMUNITIES: