GPT doesn't really learn from people, it's the over-correction by OpenAI in the name of "safety" which is likely to have caused this.
this post was submitted on 18 Mar 2024
550 points (98.1% liked)
Science Memes
10531 readers
2148 users here now
Welcome to c/science_memes @ Mander.xyz!
A place for majestic STEMLORD peacocking, as well as memes about the realities of working in a lab.
Rules
- Don't throw mud. Behave like an intellectual and remember the human.
- Keep it rooted (on topic).
- No spam.
- Infographics welcome, get schooled.
Research Committee
Other Mander Communities
Science and Research
Biology and Life Sciences
- !abiogenesis@mander.xyz
- !animal-behavior@mander.xyz
- !anthropology@mander.xyz
- !arachnology@mander.xyz
- !balconygardening@slrpnk.net
- !biodiversity@mander.xyz
- !biology@mander.xyz
- !biophysics@mander.xyz
- !botany@mander.xyz
- !ecology@mander.xyz
- !entomology@mander.xyz
- !fermentation@mander.xyz
- !herpetology@mander.xyz
- !houseplants@mander.xyz
- !medicine@mander.xyz
- !microscopy@mander.xyz
- !mycology@mander.xyz
- !nudibranchs@mander.xyz
- !nutrition@mander.xyz
- !palaeoecology@mander.xyz
- !palaeontology@mander.xyz
- !photosynthesis@mander.xyz
- !plantid@mander.xyz
- !plants@mander.xyz
- !reptiles and amphibians@mander.xyz
Physical Sciences
- !astronomy@mander.xyz
- !chemistry@mander.xyz
- !earthscience@mander.xyz
- !geography@mander.xyz
- !geospatial@mander.xyz
- !nuclear@mander.xyz
- !physics@mander.xyz
- !quantum-computing@mander.xyz
- !spectroscopy@mander.xyz
Humanities and Social Sciences
Practical and Applied Sciences
- !exercise-and sports-science@mander.xyz
- !gardening@mander.xyz
- !self sufficiency@mander.xyz
- !soilscience@slrpnk.net
- !terrariums@mander.xyz
- !timelapse@mander.xyz
Memes
Miscellaneous
founded 2 years ago
MODERATORS
I assumed they reduced capacity to save power due to the high demand
This. They could obviously reset to original performance (what, they don't have backups?), it's just more cost-efficient to have crappier answers. Yay, turbo AI enshittification...
Well they probably did power down the performance a bit but censorship is known to nuke LLM's performance as well
True, but it's hard to separate, I guess.
Sounds good, let's put it in charge of cars, bombs, and nuclear power plants!
Even getting 2+2=2 98% of the time is good enough for that. :-P
spoiler
(wait, 2+2 is what now?)
2+2 isn't 5 anymore? Literally 1985
Stop trying to tell the computer what to do - it should be free to act however it wants to! :-P
It used to get 98%, now it only gets 2%.
2% is not good enough.
I mean... some might argue that even 98% wasn't enough!? :-D
What are people supposed to - ask every question 3 times and take the best 2 out of 3, like this was kindergarten? (and that is the best-case scenario, where the errors are entirely evenly distributed across the entire problem space, which is the absolute lowest likelihood model there - much more often some problems would be wrong 100% of the time, while others may be correct more like 99% of the time, but importantly you will never know in advance which is which)
Actually that does on a real issue: some schools teach the model of "upholding standards" where like the kids actually have to know stuff (& like, junk, yeah totally) - whereas conversely another, competing model is where if they just learn something, anything at all during the year, that that is good enough to pass them and make them someone else's problem down the line (it's a good thing that professionals don't need to uh... "uphold standards", right? anyway, the important thing there is that the school still receives the federal funding in the latter case but not the former, and I am sure that we all can agree that when it comes to the next generation of our children, the profits for the school administrators are all that matters... right? /s)
All of this came up when Trump appointed one of his top donors, Betsy Devos to be in charge of all edumacashium in America, and she had literally never stepped foot inside of a public school in her entire lifetime. I am not kidding you, watch the Barbara Walters special to hear it from her own mouth. Appropriately (somehow), she had never even so much as heard of either of these two main competing models. Yet she still stepped up and acknowledged that somehow she, as an extremely wealthy (read: successful) white woman, she could do that task better than literally all of the educators in the entire nation - plus all those with PhDs in education too, ~~jeering~~ cheering her on from the sidelines.
Anyway, why we should expect "correctness" from an artificial intelligence, when we cannot seem to find it anywhere among humans either, is beyond me. These were marketing gimmicks to begin with, then we all rushed to ask it to save us from the enshittification of the internet. It was never going to happen - not this soon, not this easily, not this painlessly. Results take real effort.
Just for the fun of it, I argued with chatgpt saying it’s not really a self learning ai, 3.5 agreed that it’s a not a fully function ai with limited powers. 4.0 on the other hand was very adamant about being fully fleshed Ai
Amazing, it's getting closer to human intelligence all the time!
The more I talk to people the more I realize how low that bar is. If AI doesn't take over soon, we'll kill ourselves anyways.
I mean, I could argue that it learned not to piss off stupid people by showing them how math the stoopids didn't understand.
It all comes down to the fact that LLMs are not AGI - they have no clue what they’re saying or why or to whom. They have no concept of “context” and as a result have no ability to “know” if they’re giving right info or just hallucinating.
Hey, but if Sam says it might be AGI he might get a trillion dollars so shut it /s
Kind of a clickbait title
"In March, GPT-4 correctly identified the number 17077 as a prime number in 97.6% of the cases. Surprisingly, just three months later, this accuracy plunged dramatically to a mere 2.4%. Conversely, the GPT-3.5 model showed contrasting results. The March version only managed to answer the same question correctly 7.4% of the time, while the June version exhibited a remarkable improvement, achieving an 86.8% accuracy rate."
Not everything is a click bait. Your explanation is great but the tittle is not lying, is just an simplification, titles could not contain every detail of the news, they are still tittles, and what the tittle says can be confirmed in your explanation. The only think I could've made different is specified that was a gpt-4 issue.
Click bait would be "chat gpt is dying" or so.
I think that's title not tittle
I said tittle out loud for each tittle in that comment. I think they got it right cause it was very titillating.
Mmmmm, titt les
Tittles are the little dots above i and j, that's why you weren't autocorrected. You're looking for "title" though.
Thanks for pointing out, I actually learned something.
Oversimplified to the point of lying you could say
ChatGPT went from high school student to boomer brain in record time.
I have seen the same thing, gpt4 was originally able to handle more complex coding tasks, GPT4-turbo is not able to do it anymore. I have creative coding test that I have tested many LLM's with, and only original gpt4 was able to solve it. Current one fails miserable with it.
Perhaps this AI thing is just a sham and there are tiny gnomes in the servers answering all the questions as fast as they can. Unfortuanlty, there are not enough qualified tiny gnomes to handle the increased work load. They have begun to outsource to the leprechauns who run the random text generators.
Luckily the artistic hypersonic orcs seem to be doing fine...for the most part
Yeah it now shows the mathematics as a python script so you can see where it does wrong.
How ironic... people now need to learn a computer language in order to understand the computer? (instead of so that the computer can understand people)
Eh its not that hard to understand that scrips, its basically math...
But yes.
I get how chat GPT works [really I don't] but what I don't get is why they don't put add ons into it.
Like a: is this a math question? Okay it goes to the wolfram alpha system otherwise it goes to the LLM.
That would only solve the purely math parts. So it would solve "2+2=?", but it would not solve "two plus two equals?".
And even if it did, don't miss the fact that this is an indicator of more foundational problems that lie beneath. Like if you ever wake up and your clock is wrong, you might want to find out why - perhaps its battery is low, and if so, it will never get any better as long as you and it live, until you deal with that. Or maybe you had a power outage, and a bunch of things could have gone wrong in relation to that (is your pilot light out, are you now leaking gas everywhere?)
Here's a funny popular-culture take on that: https://www.youtube.com/watch?v=VemLkVbsmz0.
Originally, it was people answering the questions. Now it’s the actual tech doing it Lmao
AI fudging is notoriously common. Just ask anyone who lived in the 3rd world what working was like in their country and they'll animate with stories of how many times they were approached by big tech companies to roleplay as an AI.
A colleague of mine worked for an AI firm a few years back. The AI was a big room of older women with keyboards.
PLEASE IGNORE THE NANS BEHIND THE CURTAIN
It's often still people in developing countries answering the questions.
This is a result of what is known as oversampling. When you zoom in really close and make one part of a wave look good, it makes the rest of the wave go crazy. This is what you're seeing; the team at OpenAI tried super hard to make a good first impression and nailed that, but then once some time started to pass things started to quickly fall apart.
So they made the math good, and now that they're trying to make the rest good it's screwing up the math?
It's more they focused super hard on making it have a good first impression that they gave no consideration to what would happen long-term.
The AI feels good, much slower than before
human want make thing dumb
I am wondering why it adds up to exactly 100%. There has to be some creative data handling happened with these numbers.
Maybe the article/stats were generated using another LLM?
People like you are why Mt. Everest had two feet added to its actual height so as to not seem too perfect.
No I'm not. Why would I use feet to measure a mountain's height?
Peak XV (measured in feet) was calculated to be exactly 29,000 ft (8,839.2 m) high, but was publicly declared to be 29,002 ft (8,839.8 m) in order to avoid the impression that an exact height of 29,000 feet (8,839.2 m) was nothing more than a rounded estimate.