Therefore your agent swarms don't have to be cutting edge API calls, they can be hosted on clusters of tiny cheap computers with small CPU GPU. For orgs you could basically invest in a local server that has 100s of small computers.
You can use engineers and frontier models to plan and do task subdivision. And then shit that out to the local cluster that's basically handling each small task without knowing what it's supposed to do.
The real issue is the middle. Testing, verifying interconnectivity of mid level abstractions, building that amount of tech debt at such a high rate and actually being aware in any way what it is that you've built.
I think for most MBAs, considering the US and a significant amount of global markets only care about short term market upside, and AI is still in the middle of the hype cycle, that's not a serious issue for anyone who is profit oriented.
It's only a real issue for the losers like you and me who care about sustainability and infrastructure that survives longer than one market hurricane season.
For my current setup, the most efficient way is to use larger models for coordination, but a heretic'ed qwen 30B model for the implementations. If you build your agentic environment around specifications and test coverage tied to symbols, you can do a lot of parallelization of agent work.
If you then separate the filesystem and tool read/write access by agent roles and policies (e.g. coder not allowed to modify unit tests, tester not allowed to modify code files) then you can force them to use a centralized per symbol/per contract specification tool.
And then you can just let agents discuss issues, where the messaging threads are also tied to the same symbols.
[1] Exocomp, highly experimental: https://github.com/cookiengineer/exocomp
The interesting frontier is elsewhere: how to fit it into the pipeline to build new useful things. I felt motivated enough to write a follow-up/reply post http://replicated.live/blog/follow-up
It's cool but I'd rather know what their most successful users are doing, and how they measure that.
>To facilitate this rate of activity, we built a new version control system (VCS) from scratch. Throughput was not the only reason to own this layer. Every change in the system passes through the VCS, so it is where collisions first become visible, and several of the coordination mechanisms in the next section are implemented directly inside of it.
Talk about inventing the universe to make a button.
Isn't SQLite's source code in its training data?
It seems both likely that they were and impossible to remove that code from pretraining. Doesn’t that make this just about LLM memorization of the training set? What am I missing?
They’re testing the same models on the same task but different ways of organising the swarms and the new approach works better.
I guess that makes sense since the harness is the product in the case of Cursor.
I look on twitter and it's just people building tools for agents to use agents, some weird customization loop going on in the LLM space right now. Ultimately these are trends pushed on us from model providers because they 10x token consumption. Its literally just BS trends to increase revenue at these companies, most of it is largely useless.
I also don't know that beads is a very strong example that Yegge knows what he's doing - it's famously been described as (pseudo)malware.
I'm not sure of the long-term viability of these brute force swarm approaches, but clearly the AI companies dig the marketing they get out of it. But the fact that they get so far at this stage suggests we have a very interesting 2 years coming up.
Kudos
I like it so much I even recorded a video walking through the process!
https://www.youtube.com/watch?v=efUmcCiRoDU (I do walk through how I do this in our app DevSwarm here)
That said, I love the work on figuring out these harness coordination jobs. While there are analogs to human management there is also this enticing feeling that, since the models are broadly deterministic, we might be able to get repeatable science-type lessons about managing them with enough testing.
Is the original part the VCS that works at 1000 commits/second? Or spending a bunch of money comparing models?
Imagine this: you need to build a new CRUD app but your AI agent spends 20M toks on just writing it's own custom login function, that's a waste of time and money. I would not like to have a new system developed for me in such cases. I think we need a test where the agents are more specifically tasked to find technologies worth integrating and hence write the fewest lines of code.
It's obviously in its infancy but fully agentic companies are becoming a possibility, given the right harness, models and tools
And it's a bit of a shame: by virtue of their position (their embedding in the fabric of venture capitalism), it seems like they can only make a subset of things -- what they can make is dictated enormously by capital, as they are engines of capital.
Not sure the point I'm trying to make, I just find it amusing.
Perhaps the point is that it might be more worthwhile to give independent creators a billion dollars to play around with agent swarms if we want to keep diversity in the evolutionary algorithm that is the software industry high.
What I'd love to see is a small group of humans steering a million agents.
[deleted]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
I made a VCS for LLM assisted work, bootstrapping it on its own code as it was developed. In fact, if it did not really work, it would not even exist! In my experience, keeping track of things and evaluating them was the bottleneck and I focused on that https://replicated.live/blog/worktree Effectively, the question is "how to leverage human evaluation and feedback".
The Cursor people concluded their last post with "taste, judgement, and direction came from humans". Sounded surprisingly sane after "don't compile it, commit it faster" some paragraphs earlier.
The issue with AI is it has no real memory, context, long term reasoning, etc. These are a lot of what we use to determine how well we're doing something as well as why.
The "bottleneck" so to speak with ai is that it will just circularly build junk forever, expand scope infinitely, and just continue building.
At its best it will autobuild your unicorn saas. But on average it's a cancer cluster of ever expanding code with no plan.
The issue with unsupervised AI is that when you increase the throughput you decrease the amount of human supervision. So you have to have some way to deal with that.
If I'm going to give AI a gigantic task that's really complicated and it will probably mess it up 100s of times, how will I compensate for that without constantly looking over it's shoulder?
Some critical parts of agentic development are literally months old while most mature ones are merely a couple years old.
Critical thinking was the most critical checkpoint for models to reach.
Now its only a matter of time for the right tooling and orchestration to come along. Once agent memory and recall works well, all of those bottlenecks will go away one by one.
[dead]
I feel like the whole concept of harness and LLMs IS to pick random tokens which make sense. Think about this as a solution space (a very complex one), then the harness and the LLM are moving towards the local minima at which the code is complete. This is akin to how we think about AI anyways.
"On a Claude subscription, Claude Code requests the one-hour TTL automatically. Usage is included in your plan rather than billed per token, so the longer TTL costs you nothing extra and only affects how long your cache stays warm. If you’ve gone over your plan’s usage limit and Claude Code is drawing on usage credits, you are billed for that usage, so Claude Code automatically drops the TTL to five minutes."
I assume they did similar, as long as test case(or spec) can be parallelized. Having parallel execution / testing environment is crucial if we need 1000 commits/second throughput.
I would not surprised they have 1+ millions of specs.
I would bet they won't release generated specs - actually this will be a valuable asset like https://sqlite.org/th3.html
The only thing Yegge did was come up with really stupid, convoluted and anthropomorphized language to describe the process and then write unhinged articles about it
Most of these reusable, automated workflows sound really cool in theory, but in practice either they require a lot of customization (not really reusable) or a lot of babysitting (not really automated).
AI has improved my productivity significantly, I'm not a hater, but we also have to be honest about its (current) limitations. The whole "software factory" thing is just a pipe dream people sell to execs / investors / online courses.
They say SaaS is dead and you can vibe code any SaaS using Opus and yet despite having Mythos with them they have to use JS terminal to run Claude Code.
Why don't they just vibe code CC? Why doesn't Dario have a 100000 agent swarm to run entirety of Anthropic or at least software department?
So it's not a brag
They could just write in Rust directly can't they? If they do that then they're doing what they preach
Are you implying that them attempting a Rust rewrite would fail, and this is why they’re not doing it?
For that to work, the swarm has to actually follow the spec, which is what much of this post is about. We gave the swarm 835 pages of prose and it came back with a database. What was scarce in this experiment, and what we expect to be scarce in software engineering going forward, is the right description of intent.[deleted]
[deleted]
It’s rare to run into prodigious artists who simply produce a few perfect pieces - as a rule those few magnum opuses are built on a hill of cutting room floor scraps.
I assure the worst of Shakespeares work is far beyond anything most writers can produce, much less the incoherent slop LLMs generate
[deleted]
tijebs gi yo,
If you already know what you want in an infinite library, it is there. If you don't know exactly what you're looking for to the letter, you might not be able to find it.
But I’m glad someone else is running all these experiments so we can see who is right in their guess about the outcome.
Douglas Adams said this beautifully in the Campaign for Real Time & the poet Lallalfa.
And I just found out h2g2.com is no longer online, so no reference.
Monkeys have been reading Shakespeare all their lives (training). Effectively, this is LLM uncompression. Hence, "in Rust, from nothing but its documentation" -- not entirely true.
It's like a diet system advertising how much weight a spokesperson lost on their system, without disclosing that they also went on Ozempic at the same time.
This strictly follows the early agentic usage, where people were complaining of code full of errors and such. But having the loop do plan-write-verify is the thing that got us from that state to today. And it works. If you look at the code and it has errors (linting, etc) then your loop is not properly setup. You should only be called upon when all that can and should pass does. Your role is "direction/taste/human-common-sense-at-a-glance".
[deleted]
[dead]
If that was the case, it just makes no sense at all to use JS. Take the free performance gain and don't read the code.
Here is the prompt they can use:
Claude, migrate Claude Code into Rust. Make no mistakes!
Well, Anthropic spent many $ teling everyone that code is dead so this doesn't matter to them does it? Their business logic can be written in English!
codex VSS - 600Mb
Unless they're lying that is!