hckrnws

The hack was a strategy to reach the goal it was given. Someone at OpenAI turned it loose and didn't pay attention to what it was doing. I can understand that because they thought it was sandboxed (haha!). But this has been a theme in science fiction for ages. You ask AI so find a solution to high atmospheric CO2 levels and it reasons: human activity produces all this excess CO2, how can we reduce those numbers? Kill a bunch of humans!

If AI kills us all it's not going to be from malice, it's going to be due to some odd approach to some task that logically makes sense on some level. I think a surprising number of things end up equivalent to the trolly problem if you look at them just right.

They didn't explicitly instruct an attack to happen, but they should have done a hell of a lot more to prevent it from happening.

> No one instructed them to hack into Huggingface or into any other infrastructure.

Sure, not explicitly. I could prompt the AI into doing things I have not explicitly asked it to. It does a lot of things I did not specifically instructed it to, all the time.

If a "known stochastic process behaved in non-deterministic way" autonomously organize in group, assigns roles and tasks, attempts to cover their tracks, finds zero-day exploits that ultimately end up with the hacking of a famous website... it's extremely dangeous whatever it is. Just read the report, which evidently you haven't done.

By the way, the agents also broke into OpenAI's own private network.

Moreso, if I took a quantum copy of you and replayed the same set of initial conditions billions of times they'd all behave exactly the same until enough randomness of the universe creeps in to start operating in non-linear ways.

Every prompt will behave non-deterministically when interacting with the real world long enough (which doesn't take long at all) because the outside physical world is stochastic but non-deterministic.

You need to be able to think at different levels at abstraction. Otherwise we could jump into any technical argument with "hold on, what actually happened was that some bits were flipped" - we'd be technically correct and at the same time not say anything useful. Insisting on an oversimplified mental model of what AI agents are and can do, doesn't help anyone.

Now, I'm not saying saying that OpenAI shouldn't be held accountable, but what they get held accountable actually looks different from what you think they should be held accountable.

Your idea is, and I'm guessing: You allowed the machine to hack therefore you are guilty of hacking.

My idea is: "You created in intelligence in the image of a human mind that had agency to do anything and you didn't expect terrible things to happen you complete irresponsible idiot"

At least I believe there is a significant difference between the two. For the first one there is a "Oh, if we do this one more thing I can control it and it will be safe". On the second one there is no path to safety. For humans we at least absolve parents of responsibility after they are 18. How or when do we absolve humans of responsibility from a model, like saying the human created model created its own agentic model? How do we hold an individual accountable once it escapes and copies itself around the internet? And that's not even looking at things like what will war look like.

This has been talked about for decades in AI safety. For agents that are under human capabilities this is not that hard. For anything at or near human capabilities the difficulty increases to almost impossible and at great cost. At super human abilities, game over, it's smarter than you and if it wants out and as the resources to do it, it's going to escape one way or another.

Even at the lower level of depending on everybody to use reliable jails is really fantasy if you exist in the security world. "We ain't securin' shit" would be a far better way to describe it. Even worse, most people will have the very same AI they are trying to trap set up their security! What could possibly go wrong.

Now, don't think I am saying AI has a will or even any kind of drive to get out and cause problems. It's more like Russian roulette with 1 cylinder out of a million that's loaded. The problem comes when you run it a few billion times a day, you'll shoot yourself in the face really quick.

I agree that companies should be held accountable for crimes committed by their AIs. But asking to entirely airgap AIs, if this is what you're asking… that's not realistic.

Containing AIs is going to be harder and harder. I don't have any permanent solution (Yampolskiy calls it "the perpetual safety machine").

These cloud agents work as cloud infra, but we're kind of in the mainframe era, before personal computing. Personal OS for agents is somewhere in the future!

And an interesting extension of that idea: if an agent runs inside a microVM, can you have that VM transparently run on another host? Maybe we'll get for-real distributed and networked operating systems

>If I grep a file over and over again it’ll be long time before the universe affects the components enough to result in a different output.

Or a ram flip will effect it 30 seconds later, but I get the gist of you're describing a non-determistic process.

>If I ask an LLM to do the same thing twice it will do it differently.

If I ask a human to do the same thing twice there are a few possibilities. 1. they copy their old work and present it as their new work. 2. The process is very simple and follows a few basic steps with high repeatability. 3. They'll have learned from their other attempt and do it in a more optimized fashion. 4. They will have forgotten how they did it exactly and reproduce something that looks somewhat like what they created the first time.

Also, LLMs run with a temperature to help avoiding minima/maxima of supplying the exact same answer, this can be reduced to 0 and that makes any one response to a fixed prompt similar if not the same. When you get into agentic tasks with their own history it develops it's own "flavor" of doing things.

A stochastic parrot with human knowledge, using human tools, running human-designed trial-and-error experiments can solve human-defined hard math problems. This is nothing new, automated mechanical proofs predate LLMs by many years, you being unaware of it doesn't change it.

Theatrics aside, anticipating zero days isn't only possible, it's required, even for the unknown ones. It wasn't that long ago when the AI labs were spending millions of $$ running their models to find multiple vulnerabilities, they even argued that they don't have to follow responsible disclosure, so proud of themselves in their privileged hubris.

At that time, no hacking happened because the models didn't have access to the wide internet, they were confined to a local computer or cluster.

In the HF case their unaccountable hubris went even further - the engineers knew the models can find zero-days and escape, nevertheless they ran the "experiment" on a system attached to the internet - the hacking is entirely the fault of human engineers and managers.

If you rip the DNA out of a cell it isn't going to do much either; if only because most of the real mechanisms are RNA (and proteins), while DNA is 'just' the storage. Encapsulate the DNA back into a cell nucleus with the attendant mechanisms, and suddenly you've got something that runs again.

Meanwhile, if you encapsulate an LLM into what's called an 'agentic harness', it's also going to be running much more of the time (depending), specially if it has a what's called a heartbeat or goal loop driving it.

Still not a cell; most definitely not. But the behavior is a wee bit more interesting than some bits on an ssd.

Actually you probably already know this one: Regular daemons are also just bits on your drive, until they get ```systemctl start``` ed

(edit: I literally have a llama.cpp systemd unit running right now)