this post was submitted on 18 Jun 2023
135 points (99.3% liked)
Programmer Humor
32555 readers
477 users here now
Post funny things about programming here! (Or just rant about your favourite programming language.)
Rules:
- Posts must be relevant to programming, programmers, or computer science.
- No NSFW content.
- Jokes must be in good taste. No hate speech, bigotry, etc.
founded 5 years ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Okay, I'll take your word for it.
I've never ever, in many hours of playing with ChatGPT as a toy, had it make up a word. Hallucinate wildly, yes, but not stogulate a word out of nothing.
I'd love to know more, though. How does it combine new words? Do you have any examples of words ChatGPT has made up? This is fascinating to me, as it means the model is much less chained to the training data than I thought.
A lot of compound words are actually multiple tokens so there's nothing stopping the LLM from generating the tokens in a new order thereby creating a new word.
It can create new words, I just verified this. First word it gave me: flumjangle. Google gives me 0 results. Maybe Google is missing something and it exists in some data out there, Idk.
I'm not sure what is so impressive about this though. Language models can string words together in unique ways, why would it be different for characters?