The official act could be to use the national guard to rescue a US citizen from wrongful imprisonment, but that's the kind of thing that racist Republicans would start a civil war over in the name of "states' rights".

That isn't my job.

So then you didn't "solve" the impossible problems.

I hope you and your colleagues have a good laugh about how the work you do is contributing to the march towards the end of human civilization as we know it.

I can't tell if you're suggesting that foundation models (which is the underpinning technology of LLMs) aren't being used for the things that I said they're being used for, but I can assure you they are, either in commercial R&D or in live commercial products.

The fact that they shouldn't be used for these things is something we can certainly agree on, but the fact remains that they are.

Sources:

• Wayve is using foundation models for driving, and I am under the impression that their neural net extends all the way from sensor input to motor control: https://wayve.ai/thinking/introducing-gaia1/

• Research recommending the use of LLMs for giving financial advice: https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4850039

• LLMs for therapy: https://blog.langchain.dev/mental-health-therapy-as-an-llm-state-machine/

So this all goes back to my point that some form of accountability is needed for how these tools get used. I haven't examined the specific legislation proposal enough to give any firm opinion on it, but I think it's a good thing that the conversation is happening in a serious way.

Literally nothing you've said gives any indication that you actually know the current state of foundation model research. I won't claim it's my research specialty, but I work directly with people whose full time job is research and tuning on foundation models, and everything I'm saying is being relayed from conversations that I have with them.

"Cannot ever possibly be used like that".. Like what specifically? To drive a car? That's being done. To give financial advice? That's being done. To console people who are suicidal or at risk of harming themselves? That's being done. To make kill / no kill decisions in an active warzone? It's being considered (if not already being done in secret).

This technology is being used in extremely consequential positions despite having very weak guarantees around safety. This should give any reasonable person pause. I'm not taking any firm stance on whether this specific regulation is the right approach, but if you think there should be no legal accountability for the outcomes of how this technology gets used then I guess you're someone who thinks seatbelts should be optional in cars and it's okay for airplanes to fall out of the sky due to neglect.

Non-deterministic algorithms such as Monte Carlo methods or simulated annealing can still be constrained to an acceptable state space. How to do this effectively for LLMs is a very open question, largely because the state space of the problems that they are applied to is incomprehensibly huge.

That's not what an algorithms researcher means when we talk about "understanding". Obviously we know the mechanism by which it operates, it's not an unknown alien technology that dropped into our laps.

Understanding an algorithm means being able to predict the characteristics of its outputs based on the characteristics of its inputs. E.g. will it give an optimal solution to a problem that we pose? Will its response satisfy certain constraints or fall within certain bounds?

Figuring this stuff out for foundation models is an active area of research, and the absence of this predictability is an enormous safety concern for any use cases where the output can be consequential.

It cannot possibly develop agency.

I don't believe I've suggested anywhere that I think it will, but I'll play around with this concern anyway... There's a lot of discussion going on about having models feed back on themselves to learn from their own output. I don't find it all that hard to imagine that something we could reasonably consider self awareness could be formed by a very complex neural network that is able to consume and process its own outputs. And once self awareness starts to form, it's not that hard for me to imagine a sense of agency following. I have no idea what the model might use that agency for, but I don't think it's all that far fetched to consider the possibility of it happening.

Sure, but this outcome is not at all surprising. There are plenty of smart AI people that have nuanced views of what kind of threat could be posed by recklessly unleashing tools that we don't fully understand into the hands of people who are likely to do harmful things with them.

It's not surprising that those valid nuanced concerns get translated into overly simplistic misrepresentations entangled with pop sci fi panic around rogue AI as they try to move into public discourse.

Nothing about the portion of the sentence you highlight actually implies that they haven't already been getting paid to do open source work. That's an interpretation that you're projecting onto the sentence because it fits your narrative. The poster never identified themself to be a volunteer. I've already reframed the sentence for you in a previous post, but I'll try one more time: "Whenever any tech company is willing to pay me to do work related to my open source project, I sit down with them and talk about my rates" is a semantically equivalent sentence to what the poster said.

You're also taking one single datapoint which has ambiguous credibility to begin with and extrapolating it to characterize a massive industry that you, like countless others, benefit from while hardly knowing anything about how the sausage gets made.

I'd be surprised if you've ever offered a substantive contribution to an open source project in your life, so I won't be losing any sleep if a freeloader loses confidence in the ecosystem. But realistically you'll be using open source software for the rest of your life because the reality is that closed source software really can't compete in terms of scale, impact, and accessibility. If you actually care about the quality and security of the things you depend on, then do something about it. And prattling ignorance on social media does not count as doing something.

The key sentence in the post you linked which constituted more than 50% of the words being stated by the poster and yet you somehow conveniently missed which completely negates the whole narrative that you're trying to promote:

Speaking as an open source maintainer, if a tech company would like to pay me to do ~anything for my open source project, we can sit down and talk about my rates.

Which means this person is NOT simply a volunteer as you insinuated here:

When someone suggested a level of effort to be put on code checked in to prevent security issues from happening, the maintainer pushed back, stating that they will decide what level of effort they'll put in, because they're doing the work on a volunteer basis.

but in fact is available to be paid a fair rate for the labor they perform. In fact your entire description of the post is mischaracterizing what is being said in the post.

I don't know how you could have accidentally missed or misinterpreted one of the two sentences being said by the poster, and the longer of the two sentences at that. It was also the first sentence in the poster's statement. It seems more likely to me that you missed that on purpose rather than by accident. Maybe you're just so eager to find evidence to match your narrative that your brain registered the entire point of the post incorrectly. Allow me to reframe what's being said to simplify the matter:

As a self-employed contractor, if you demand that I perform free labor for you, I will decline that request.

Now just add a much more frustrated tone to the above and you get the post you linked.

You're either intentionally misrepresenting the post or you failed to understand them correctly. I'll let you take your pick for which is less embarrassing for you.

If you have access to the journal then share the information that supports the case you're trying to make.

If you don't have access and won't share anything that actually supports your case then you're just talking out of your ass and deflecting.

This statement is incredibly naive. I know exactly what I'm doing with C++ and yet it's still incredibly easy to introduce subtle bugs as soon as you're developing anything moderately complex, especially when async or threading are involved. It's not that I'm unaware of what problems may happen, it's that the language expresses itself poorly and a subtle typo or innocent assumption can lead to catastrophic failures that might only surface in 0.0001% of the times that a certain line of code is run.

I used to consider C++ my favorite language by an enormous margin because I assumed that we just have to accept the danger to get the benefit of performance. I used to eagerly follow developments coming from the C++ ISO committee, excited for the language improvements that would be coming down the line.

Then I started playing with Rust and learned what it's like to get incredible performance without being mired in hazards. Imagine outperforming C++ on average while also having your code almost always work perfectly the first time you compile it and never crash or segfault. That's Rust, and once you give it a fair try, you'll never want to go back to C++.

Being able to manipulate raw pointers with no guardrails ever is not a strength, it's a terrible weakness, and you'll live a happier life the sooner you realize that. Consider instead if you could have all the same level of control over memory except the compiler tells you whenever you're doing something asinine before you run it. You can always force it to do what you're asking for by declaring it unsafe but you'll quickly realize that you never actually want that.