The author observes that a call to GPT-5.6 Luna is only 4-5 orders of magnitude more expensive than grep, and then predicts that at current rates of progress, calling an LLM will soon be cheaper than a grep. I think this is a good time to invoke Stein's Law: "If something cannot go on forever, it will stop." These efficiency improvements won't continue forever. It's more likely that the per-call cost of high-quality, compiled software like grep will be a lower-bound that LLMs asymptotically approach, rather than a line that they blow past with perpetual exponential progress. (Barring a true breakthrough in something like quantum computing or room-temperature superconductors.)
non-backreferencing grep is not very difficult to implement in an ASIC either. But it's probably not worth it because of how relatively rarely you use it and of the data transfer costs.
LLMs are great candidates for ASIC-burning because they're slow compared even to network speeds and run all the time. The issue is that you don't want to burn a specific model or architecture that then becomes obsolete.
So you've got two possible futures, and both guarantee large price drops: (a) LLMs keep getting better and better and better, so ability/$ keeps rising; or (b) LLMs plateau in ability, in which they will start getting ASIC'd.
I also don't understand where the idea that frontier models are getting better efficiency comes from. The results are certainly improving, but that comes from feedback and multiplexing requests, which cost more.
At some future point where LLM hardware is cheaper than simply running grep, then grep equivalent would benefit from those selfsame hardware improvements and be cheaper to run as well, probably still by the same ratio.
Grep reads through the entire file looking for patterns.
An LLM scans its neural net (in ways that I don't understand) which is kinda-sorta like having a huge index.
You can improve over Grep if you have an index; and the LLM has an index.
Thus, it's plausible that an LLM can be more efficient at reading its neural net (IE, index) than Grep reading the whole file.
If you assume the file arrives ahead of time, can be indexed, and that this is worthwhile because we want to support multiple pattern matched retrievals, then sure it makes sense to consider indexed query schemes and upper/lower bounds. Each query could be faster as an inference if it doesn't have to re-scan the whole file.
But I don't think anybody, in good faith, can pretend that any LLM can digest a file faster than grep can. Particularly, if you admit the vector processing dedicated to doing the convolution kernel(s), you should also admit similar hardware could run a vectorized grep.
An LLM can certainly be cheaper than grep, because it’s an approximation, while a grep is deterministic and must examine every byte in what can be a relatively complex state machine for a regex based grep. There are other scales to consider like the scale of your local hardware vs the highly multitenant and high end hardware of the hyper scale inference providers.
There are already high volume models for coding inference where the reasoning time is crazy low and cheap per token where it can build reasonably simple software so blindingly fast it isn’t implausible the bottleneck is the latency in tools and networks. I find them hard to use at times because I don’t have time to think through the next turn by the time it’s done.
Regardless I wouldn’t be surprised to see a world where tokens are so cheap it’s not worth metering them but charging licensing feels with meter tiers at the far horizons to prevent abuse, charge outliers. Subscription models already set this stage well.
The other side to consider is bountiful capacity will also drive tokens to near zero price. The data center build out is barely underway and as it materializes, as hardware efficiencies improve, as techniques and model science and technology improves, harnesses, methodologies , etc improve, the economics flip from load shedding to trying to keep the data centers utilized. The economics lead to the world where tokens are not a unit of measurement for cost for anyone other than the inference providers to manage their utilization.
From a computational standpoint this is obviously nonsense, but from an attentional one I'm not so sure. It may already be more attentionally expensive to use grep in some cases, such the moment you need to remember a non standard arg. And if this applies for performing a simple http operations, then it certainly applies going up the complexity chain.
Won't that also help grep and then move the asymptote down more?
If the compiler invocation is sufficiently slow, the llm could consider outputting a binary directly?
For all we know matrix multiplications are a faster way to generate optimized machine code than branchy sequential compiler code with tons of heuristics and passes.
“are” or “could be”?
Is running LLMs (or some other ML workload) on/with quantum computers expected to bring efficiency gains?
what if an LLM finds the answer early?
grep might continue to read everything, doing the wrong thing correctly.
Especially for deterministic activities it just feels impossible to imagine general LLM tech handling the problem better, despite everything being said.
But hey, tech is filled with "smashing the generalist hammer works better than the specialized tooling". Would be odd though!
An example I like to give: optimizing a data processing program's runtime by 5x is obviously 5x speedup for everyone. But if, for some reason[0], this means it crashes and restarts more often, it stops looking like this to end users. If every restart means it needs to start from scratch, and it restarts 10 times on average now where it didn't restart before, the 5x speedup suddenly looks like 2x slowdown to end user.
In this sense, LLMs are already much more efficient than most CLI tools, by a combination of:
- User not having to remember the exact invocation, or even the name of the CLI tools
- LLM being able to run the CLI tools and chain them on its own
- LLM being able to self-correct in case it got things wrong, or when actual output show that user's idea was right, but the instructions were wrong
Prompting "okay, list those processes sorted by runtime and match them against these output files" is both faster to type than the actual commands, it also end-to-end completes much faster than doing it by hand.
--
[0] - And I don't mean a bug. Say it's some batch processing run on a cluster with aggressive resource usage management; 5x speedup means it runs much hotter than before, which may put it on the top of "kill list" for when the cluster managing code needs to free up some resources.
[deleted]
The only aspect that is poorly analyzed by the OP is business model viability. All players are investing insane amounts of money in infrastructure with the expectation that their future profits will justify all that investment. The winner or winners in the AGI race, they believe, will find the proverbial "pot of gold at the end of the rainbow."
The OP glosses over questions of business model viability with a brief qualitative discussion and very little hard data. For example, to earn an annual return > 10% on every trillion dollars of capital sunk into infrastructure, the owners of that infrastructure must earn free cash flow (operating profit less investment) in excess of $100 billion per year in perpetuity. Is that feasible? Why? How?
The OP does not really consider such questions.
"It is not too much to expect that our children will enjoy in their homes electrical energy too cheap to meter,..." Lewis Strauss
https://en.wikipedia.org/wiki/Too_cheap_to_meter#Origins
Oddly enough my power bill was metered and big.
Of course, perilous territory for future irony depending on how your prediction plays out.
The "most attractive quadrant" is completely meaningless. The whole point of a Pareto curve is that each point on the curve is better than everything else on at least one dimension, and that you can make these comparisons without placing a value judgement on the relative importance of the different metrics. If you make a composite score of the two metrics (any monotonically non-decreasing function, e.g. a weighted sum with non-negative weights), that score will always be maximized by one of the points on the Pareto frontier.
So going by the numbers in the 2nd chart (1st AA chart) from TFA alone:
- there's no reason one would choose Deepseek V4 Pro 0813 (max) even though it's in the "most attractive quadrant", because GLM-5.3-Flash is both cheaper and scores better.
- Claude Fable 5.1 (max with fallback) on the top right* could be your most attractive option if you need the best scoring model and don't care about cost, even though it isn't in the "most attractive quadrant"
- The un-shown model off the left side of the chart could be your most attractive option if you just need lots of cheap tokens and don't care about quality.
* I also dislike the way they place the labels, and that grey line connecting the label to the point is way too subtle.
"Had the atomic bomb turned out to be something as cheap and easily manufactured as a bicycle or an alarm clock, it might well have plunged us back into barbarism, but it might, on the other hand, have meant the end of national sovereignty and of the highly-centralised police State. If, as seems to be the case, it is a rare and costly object as difficult to produce as a battleship, it is likelier to put an end to large-scale wars at the cost of prolonging indefinitely a “peace that is no peace”."
It seems, especially with open weights, that the AI is much more like the alarm clock and not the battleship. $20/mo would have been about $1 in 1944
https://www.orwellfoundation.com/the-orwell-foundation/orwel...
The cost of a given level of performance often falls fastest right after that level is first achieved, that is, when it is state of the art (SOTA). We see this pattern on three of our five main benchmarks of AI capability. Averaging across all five, cost falls 66% per quarter (75× per year) for performance that has just debuted as SOTA. Two years later, prices fall half as fast, at 32% per quarter (4.7× per year).
but the analysis itself has more nuance and is a quite interesting read.
[1] https://epoch.ai/publications/the-plunging-price-of-thought
I think Nvidia is under the same pressure as Anthropic/OpenAI. Nvidia will dominate research and probably keep dominating training, but the real volume is in inference. And for inference Nvidia's lead is only a few months, similar to the lead frontier labs have over open source. Nvidia will sell a lot of Rubin CPX's, but their margin on that will be a lot smaller than B200 because there is so much more competition in that space.
I don't disagree with the thesis here, I just don't think costs are coming down quite that quickly.
My usage plus reading about how tokens just keep getting cheaper and cheaper sounds like a great thing for "the rest of us" but not sure how the frontier AI labs are going to pay back all of their debt if this is the case.
(I get the inference is currently very profitable but if it's a race to the bottom on token pricing, even that won't last much longer)
And perhaps people will not be willing to accept the initial friction that malleable software brings (see people who complain about Emacs or Salesforce or JIRA)... and end up just kinda churning indefinitely on re-implementing things over and over again.
"Internal IT teams" striking aback against SaaSes, perhaps. We'll see
- Energy is not infinite, neither energy efficiency is. - Datacentres neither. - Benchmarks are an abstraction of real world problems!
On top, there is an overall "economic" aspect that most of the people miss: every change carries a certain degree of risk (lose money, reputation, customers, death of people, ecc) that very few want to take and a lot of changes(e.g. rewrite some piece of SW in another Lang) don't produce a positive economic impact.
[deleted]
Seems to undercut some of its own claims, the idea that we need to design interfaces for end users is already antiquated.
Of course, for collecting better telemetry using local AI for analyzing video from camera and audio from a microphone.
AI is not a bubble, IMO, though we may see a retrench and some companies with sky-high valuations will crash to more reasonable ones. But data center demand is probably a bubble, and the main driver will be reduction in the actual amount of power and data center space required to serve escalating demand.
I think hardware and model improvements will pace or maybe outrun demand and then when demand starts to saturate will keep going and leave a lot of orphaned data centers.
Then we have these articles that say tokens will get so cheap that labs won’t know how to make profit.
Who is correct?
Token pricing was a small minority of customers up until this year, when all the labs started trying to force customers onto token-based billing. Within the last week, Anthropic repriced my team's plan from a temporary "50% extra tokens" to 25%: https://support.claude.com/en/articles/15910845-claude-code-...
The fact that all this is ongoing within such a short timeframe should make you suspicious of any analysis that claims to be observing "statistical trends" like they've discovered a new Moore's Law out of 6 months of pricing data from 2 companies.
[deleted]
I'll be back next year reminding you about your cybersecurity claims.
everything will be able to talk to anything else, for real this time
it will be like the internet of things only some asshole will call it "intelligence of things"
again, there will be no S for Security in this new IoT
your thermostat will one day start fucking with you. when you run a diagnostic llm on it, it turns out it's keeping around 5 different viral copies of personality files around, that were left behind by llm botnets/openclaw-like memetic replicators/your grandpa leaving behind easter eggs before his death.
the future will be pretty evenly distributed, and full of weird shit
> Generally, models intended to be run locally will be much smaller, such as Muse Glimmer or Qwen3 Coder.
That is such an interesting set of models to use as examples here. One being essentially obsolete on release a month ago, and the other being completely ancient in LLM time. I really wonder how they landed on those two.
I just stopped reading at that, for anyone else, Please find a better source and take everything in here with a grain of salt.
IMO
The number one improvement that mattered for local AI was llama.cpp, partial offloading to system cpu/ram. The next was quants, being able to take fp16 and turn it to q8, q4 etc. The next IMHO is unsloth dynamic quant, that have been able to do mixed precision so we have UDq1/q2 that is actually pretty damn coherent. Allowing individuals to drive K3 locally even if it's at Q1/Q2. Then MoE changed everything for everyone, cloud and local. The other is integrated GPU, Apple, Strix Halo, DGX Spark. Then all the extra improvements like MTP, DSpark, etc. Of course there's many other additional things that have mattered too
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
[dead]
Yeah okay bud, anyone checked in with the state of consumer hardware recently? Not the author, evidently.
>oh in 3-6 years this will all be over
Yeah I'm sure Samsung, Nvidia and sk hynix will all be very calm with lower volumes and lower margins.
I did a toy project once implementing a limited version of grep on an FPGA and was able to get some speedup over GNU grep at the time, though marginal.
In any case, LLMs aren't IO bound :))
If your problem is grepping crazy amounts of data, the infrastructure for LLMs isn't a bad place to look for an example.
I wouldn't be surprised if that's what Apple is focused on for their next generation platforms – I wonder if more layers of caching between their SSDs and unified memory are on the cards.
Inference costs cannot keep falling forever, but they do still have a long way to go.
They don't need to plateu for that to happen. There are companies already building AI on ASIC, and IIRC they were approach 12 months lead time. A 12 months old frontier model (Sonnet 4.5, GPT-5, Kimi K2) for 1% of the price is still a rather good value proposition.
Right now, I do actually use OpenAI's gpt-oss-safeguard-20b for somethings, was released 11 months ago, and is $0.075/M input / $0.30/M output now. I could see this model being in fairly widespread use at 10x speed and 1/10th cost if it was introduced today. Meaning, that for some usecases (moderation) i think dedicated chips can pan out today.
But for more general models, its tougher. Gemini 3 pro was launched in November, if ASICs brought it down 1/10th in cost, it would be $0.20/$1.2. GPT 6 Luna is $0.1/$0.50. Luna is better at a lot of things, but not everything. So 1/10th doesn't really make the ASICS investment worth it in my opinion, but if it brought it down to 1% ($0.02 / $0.12) it would be a really compelling model with a lot of use.
BUT, do i think something like Luna is probably generally capable of doing a huge amount of knowledge work. So if Luna came out at 1/10th the cost a year from now, it would probably be compelling for a while.
It all depends on the rate of improvement in cost/capability.
But I would probably take a long horizon bet that the grep implementation on my machine will remain cheaper than an equivalent ai task, even though I think those ai tasks will become far cheaper over time.
I just think the original comment's model of asymptotic approach is probably more likely to be accurate than the model of the line blowing through this grep-like cost level.
The complexity gets faster as you get on with it.
With AI, the question is still open if this will actually turn out to be something useful or if it will in the end just be another way for the elites to make untold profits.
you could make hardware implementation of grep and store the file itself next to it in some ROM but that's not a very useful grep ... while hardware LLM is exactly as useful as software LLM only orders of magnitude faster
When the author wrote Llm can be as cheap as a tool, I read it as not equivalent. They even said the Llm can be embedded into a tool.
Their point was, the higher level use case — like classification — could become as cheap as grep. Which is quite well possible.
Something seems off about this.
https://iepathos.github.io/ripgrep/performance/#work-stealin...
That’ll be just as true for an ASIC NN as it is for good ole grep.
NVMe v1 wasn't released until 2011 and there was no hardware available until 2012. Before that the fastest drive interface was 6 Gbps SAS. Then we got 4 GBps (32 Gbps) with NVMe v3, 8 Gbps (64 GBps) with v4, and 16 Gbps (128 Gbps) with v5. V6 is slated to double again. v7 is currently planned to double that. This is less than 15 years of progress.
In the 1990s, we went from 33 Mbps to 66, to 100, then 133 with IDE. We went from the 40 Mbps of 1986's SCSI 1 to 80 of SCSI 2 in 1994. Since then parallel SCSI has gone to 80, 160, 320, 640 (Ultra 2 wide), 1280 (Ultra 3), 2560 (Ultra-320), and 5120 (Ultra-640) Mbps.
SAS has also gotten faster, with 6 Gbps, 12 Gbps, and 22.5 Gbps. NVMe is still lower overhead and lower latency.
Memory bandwidth can also be addressed. Stock servers with EPYC are 16 channel DDR5. That's 409.6 GBps. Four channels of DDR3 was only 102.4 GBps. Video and accelerator card memory bandwidth is also increasing.
You mean a grep over terabytes of data vs a LLM with gigabytes of parameters?
If you have so much data, you can use an index to search. It's unlikely that LLMs are going to be cheaper than properly indexed search DBs (which is what we should be comparing them with)
The point of a grep is for adhoc analysis of data that isn’t in a database. At best you could use an inverted index, which is extremely expensive to create and maintain, but if you don’t know a priori what you’re going to search for, or even if an inverted index helps with your query, it’s pointless and expensive to index literally everything you could ever use to save some time on a tool call.
The point isn’t to find a way to make searching more efficient. It’s that LLMs are becoming very efficient, and a measure is relative to grep. Making grep and searching more efficient has been a task in CS for decades. LLMs is relatively new beyond a very small crowd. It’s also not a goal to make LLMs better at grep than grep, but as a relative reference point that inference is getting cheap and fast and has opportunity to get faster far beyond grep simply due to unclaimed optimizations more available for the newer technology.
[deleted]
You may be right. I'm not so sure. Inference looks like a viable business model for those operators that have SOTA infrastructure in place, but the investment required to have it is enormous, and appears to be never-ending, because if an operator stops investing aggressively, its infrastructure quickly becomes non-competitive, and customers will quickly leave for alternatives. SOTA infrastructure is a moving target.
Why would it be any different for inference? If we believe OP, it'll just become part of regular compute infra, and thus part of the renting-out-compute business model.
I think it's an open question if the current generation of inference investment will pan out, but in the long term, there'll be a balance between investment cost and margin, just as in every other industry.
Edit to add: Or or might just be AWS / GCP / Azure that benefits from this business model. They're already pretty good at selling commodity infrastructure.
But yeah, I definitely don't have high confidence in any of this!
Only if you also have the model thats better than anyone else's.
As soon as models are free, or there are no newer models (assuming thats going to happen, and thats not a given) then the only thing you can compete on is price.
This means that the only thing you have to differentiate is either price, speed or ease of use. (or regulatory capture...)
We are at pets.com level of spend currently. Unless model development becomes cheaper, then we are going to run out of novel debt but not really debt mechanisms.
One is how much does it cost OpenAI to train the model.
The other is, if I stole OpenAI's model how much would it cost for me to run it?
R&D costs versus operational costs. Operational costs are very likely profitable. R&D is catastrophically expensive currently.
The main problem with the scenario of just doing inference is it relies on nobody else training models better than yours. As long as people are training private models that are better than yours, just inference isn't a viable business model.
Also another take on AI costs falling https://x.com/EpochAIResearch/status/2102510281176023529 Methodology is how much it costs to do the same task a while later. Gives fallen ~47% a quarter.
The hubris of this is really astounding too. There is no technology out there that some company develops and has not been reverse engineered and copied and manufactured at scale by competitors before long. You can't stop this from happening. People will leave the company or be poached and proliferate what they have built in the past. Every country that wanted a nuke has a nuke, after all.
The only way to keep the secret fire from leaking out would be to have AGI's first move be to lock the doors and prevent anyone from ever leaving company property again.
Besides search is not a very good business: people are not inclined to pay for search, websites are averse to non-Google search crawlers, ads require an even larger over-investment in a top tier malware development, psychological manipulation, and statistics teams, while the real ad market is actually in a much more dire state than Google would like you to think.
Free cash flow (operating profit less investment), actual cash coming in, is deeply in the red.
EBITDA can be a sensible measure of profitability when there isn't much need for additional investment. That doesn't seem to be the case with these operators. They need to invest aggressively to avoid losing customers to competitors. All of these operators have made multi-year commitments to invest more in infrastructure. In addition, they have guaranteed quite a bit of debt to fund it.
Maybe it all will work out fine (and I sure hope it does!), but I didn't see any hard data from the OP, or from you, supporting that view.
Those real debt obligations are going to want to be paid back.
We're in a situation where AI isn't going to go away, but whatever financial mode we're in right not is not going to work.
Later, you were billed for time connected to the "internet" (compuserve or aol or whatever)
Around when the iphone came out, software went from tens or hundreds of dollars to pennies, then free.
On the other hand legal advice has always been expensive, because a good answer is worth it.
Medical advice is worth it. Investing advice is worth it.
(That said, I wonder if with home solar and batteries if electricity will ever "generally" go down in price to normal people)
Texting was charged per character.
And if you have the capabilities to install your own solar it can pay itself back in 6 years. Not sure what that looks like with 100% battery coverage.
[0] https://en.wikipedia.org/wiki/Zwentendorf_Nuclear_Power_Plan...
Given current found reserves and the current rate of use, the world has about 40 to 50 years of natural gas. Thorium used in molten salt thorium reactors would provide electricity for 60 billion years or so if we could actually extract all of it. That's 10 billion years or more if it provided all human energy consumption. Of course there's a limit to extraction, but it is over three times as common as uranium.
Also, besides thorium one can mix in partial amounts of other fuels, including uranium and plutonium. There is no runaway meltdown risk, as the fission is actively managed rather than actively suppressed. Fuel is spent more completely. The waste products are smaller, less radioactive, and have far shorter half-lives.
Then of course we're always getting slightly closer to productive fusion reactors.
These technologies along with solar PV, solar thermal, hydro, wind, geothermal, wave power, and batteries likely all have a place in the future.
There's a decent chance that at some point in the future residential customers will pay for the connection and only commercial or industrial customers will actually be metered. That's not because companies want to give up additional revenue. It's because at some point the cost of meters, tracking usage, and competitive advertising about who has the cheapest plans costs more than the power the typical customer uses above the base charge.
[deleted]
We had batteries full and were spending electricity on all kinds of luxury things like whole-day internet and desalination for several weeks, and now that it has rained for over a week we're starting to turn non-critical systems off to keep the lights on.
"Between digital economics and the wholesale embrace of King's Gillette's experiment in price shifting, we are entering an era when free will be seen as the norm, not an anomaly. How big a deal is that? Well, consider this analogy: In 1954, at the dawn of nuclear power, Lewis Strauss, head of the Atomic Energy Commission, promised that we were entering an age when electricity would be "too cheap to meter." Needless to say, that didn't happen, mostly because the risks of nuclear energy hugely increased its costs. But what if he'd been right? What if electricity had in fact become virtually free? The answer is that everything electricity touched—which is to say just about everything—would have been transformed. Rather than balance electricity against other energy sources, we'd use electricity for as many things as we could—we'd waste it, in fact, because it would be too cheap to worry about."
... What Mead understood is that a psychological switch should flip as things head toward zero. Even though they may never become entirely free, as the price drops there is great advantage to be had in treating them as if they were free. Not too cheap to meter, as Atomic Energy Commission chief Lewis Strauss said in a different context, but too cheap to matter. Indeed, the history of technological innovation has been marked by people spotting such price and performance trends and getting ahead of them."
My issue is that datacenter energy costs are being prioritized for commerce over residential use, so the average consumer is paying more for electricity, because a datacenter needs more electricity and they're getting tax breaks. Assuming this all improves efficiency for new products like automated robotics, there is a debatable benefit. Jevon's Paradox has no ceiling, except the environment, and people's 401ks.
So far, Simon Willison's pelican bike "benchmark" is the only one I've found that shows Fable 5.1 beating Opus 5.5. My personal experience has been Opus is unusable on design work it's so terrible. Evaluating whether we should consolidate AWS DMS tasks (Postgres full load and change data capture) into fewer tasks with more tables, Opus 5.5 was factually wrong and needed correction roughly every other turn.
On a "help me find a sandbox solution for agents embedded in a web app to run untrusted code" research project it kept misrepresenting security boundaries and ended up recommending DuckDB which ironically specifically says it does not provide a strong security boundary in its own documentation. GPT 6 (can't remember if it was Sol or Astra) and Fable 5.1 both recommended FaaS like Cloudflare Workers and AWS Lambda which fit fairly well with the requirements.
I switched from Opus to Fable in the session going badly sideways and told it to "Review the previous conversation and come up with a correct comparison table and corrected recommendations grounded in objectivity supported by citations. Do research as necessary to understand the current ecosystem" and that was a full 180 back to coherency...
Given the shrinking margins, I wonder whether Nvidia will still think it's worth competing in that price-performance corner in the long term.
When people say "AI is a bubble", they mean economically as a whole, which includes data centers.
Perhaps we need better terminology for "product useful; numbers nonsensical"
Not so sure about this. There’s always a potential threshold. After all, we don’t all use the most powerful computers, the latest phones, the highest resolution cameras, the fastest or best cars.
I am already not interested in cloud LLMs and I don’t even use the best (on paper) model that I can run locally. I prefer a model that people insisted (here) was “dead on arrival” but appears to work better for me.
The edge is somewhere else.
You can't simply wave Jevon's paradox at things. Thousands of miles of canals were dug in the UK that couldn't be sustained and were abandoned. Thousands of miles of railways were laid that could be sustained and were abandoned. And those are potentially durable investments, unlike cheap walls, pillars and roofs laid over a levelled concrete slab full of fast depreciating IT equipment.
But yes, if sold for a negative margin Jevon eventually stops because the decreasing supply will drive up prices.
> things are made that will sell for less than the cost of construction
Price is set at the marginal cost. Capital costs aren't in marginal costs.
You'll need a better counter-example than UK railways which suffered from Parliament price-fixing.
Asking Claude for a rough estimate based on publicly available throughput and cost data for open weight models on modern GPUs suggests serverless, pay-as-you-go inference is profitable on owned GPUs with reasonable utilization (30-50%).
Also, it's impossible that they become cheaper than specialized software. Or even as cheap as them. It's still possible that they become cheap enough that it doesn't matter.
RAM prices will crash when demand drops even a little. They'll probably crash to a lower (inflation adjusted) level than before. This has happened before.
Industrial scaling in general often looks like a sawtooth: price spike, capacity investment, crash, repeat.
Part of what's keeping prices high a little longer is that everyone knows this and is a little reluctant to plow resources into chip fabs for fear of having the bottom fall out before they recoup or sell that to someone else to hold that bag.
Graph the average compute and RAM in a mid-high end laptop at an inflation adjusted price point for the past 40 years. It's very exponential and hasn't slowed down much.
On a long enough scale you are right that prices will likely normalize to a better level, but before 2030? That would mean the factories are built quickly once they begin.
except cxmt who is plowing resources in like crazy
They all collectively chose NOT to increase supply with increased demand. So if the bubble pops, they just go back to previous prices without oversupply driving the prices to rock bottom.
Put a Transputer in a Lego brick. Not "like a Lego brick", an actual Lego brick. Turn the notches into connectors for the serial link.
Plug'n'play.
And eventually hire people whose job it is to patch pipelines on demand for everyone in the office.
“Hey Jim, I’m gonna output the systemd logs of nginx on line five, can you assemble a grep pipeline for me to match all HTTP 500 status codes from /api/cart POST request log lines? Connect the filtered output to Tim’s desk, line 7. He’s there now, we are trying to figure something out.”
“Sure thing Bob, give me a moment.”
Baring it all to software: Raw machines | IEEE Journals & Magazine | IEEE Xplore https://share.google/nI9GyFJu4HvbYFrBf
(If you Google the name you will find free PDFs as well, the IEEE page is more useful as a summary and such.)
Yes, because it was a largely random undirected process.
Then you have to climb to another branch to get more fruit. The biggest issue with most problem space discovery is you're doing it blindfolded.
It accelerated from there rather than slowing down.
But I also think there are multiple ways to differentiate. There is at the very least: "intelligence", price, latency, throughput, reliability. It's not clear to me yet what this looks like, but maybe there is also a services and integration level of differentiation. And then there is the universal stuff: sales, marketing, branding. And then on the other side of the ledger there is operational efficiency, management capability, cost of capital, that kind of stuff.
I mean, there is no kind of "model quality" difference between AWS and GCP or between Delta and Southwest or between Wal-Mart and Costco, etc. but all of these businesses remain viable in very competitive markets.
I totally agree that the level of investment / capex is not sustainable though! But I think what's going to happen is that it is not going to be sustained, while AI continues past that point as a viable business (but maybe with different specific companies leading that industry).
Are we still going to make the same error over and over again? The day China says "stop" the price per watts will explode, just like when Russia and Iran said "stop".
Anyways, panels are the solution now because we're at peak petrol, that's why they're so cheap, it won't last forever, evem if China keeps the door open
The reactor creates new fissile material while it runs, then fissions that. The new fissile material is recycled into the reactor. Even actinides can largely be recycled into the fuel stream. The fuel and coolant being a liquid mixture allows a lot of chemical processing and returning parts of the initial waste back into the reactor more fully.
The waste that's actually handled for storage tends to have a lower proportion of fission byproducts that carry their own radioactivity, but some traces of highly active sources. I've read cesium, strontium, iodine, xenon, krypton, barium, and various noble metals are the bulk of the waste. Some of that cesium will be cesium-137. Some of the strontium will be strontium-90.
Solid rods from a light water reactor are not even nearly completely spent before they become too degraded for producing electricity. They contain cesium-137 and strontium-90 too but along with uranium-235, plutonium-239, americium-241, neptunium237, and curium at the time they're ready for storage. Small parts of this can be reprocessed rather than stored, but often for nuclear weapons. Having solid rods also makes it more difficult to separate elements.
MSTR waste is dangerous, but for about three hundred years. The radiation involved tends to be largely gamma, but from a very small portion of the waste rather than the bulk of the mass. Light water reactor waste can be dangerous for tens of thousands of years, producing alpha, beta, and gamma from most of the mass of the waste. The alpha and beta radiation will last tens of millennia.
Can’t really think of any other way to rationalize that combination.
It’s like being an algorithmic betting exchange gambler; being the first with access to some new stream of information may give you a very temporary advantage but once everyone has access to it, the market prices it in.
If you want an advantage you have to seek it out elsewhere.
It might be in the harness or tooling, but if your cloud LLM can write it quickly for you, it’s the same for your competitors; their cloud LLM can write it for them.
The idea that using a cloud LLM is an edge — an advantage — doesn’t stand up well to scrutiny.
The only edges left will be human.
I'm so glad the tide here is turning on this talking point, brought on by exactly the same people beating us over the head with it for months while no progress is made towards it materializing.
Many, many people who post here are capable neither of real analysis nor distinguishing real analysis from memes. They aren't hackers, they are adherents of a cult that happens to focus on the same subject matter as hackers.
It's not appropriate to think of the solar panel as a constant source of power, but the solar panel manufacturing plant. All of our nuclear reactors and fuel are produced in-country, whereas china has dominated solar panel manufacturing and can cut us off. We need to be proactive about manufacturing.
A spent solar panel contains the exact materials needed to make a new solar panel. Chemical changes are cheaply reversible, so you can turn an old solar panel into a new one for less energy than it will produce during it's lifetime. A solar panel factory can operate indefinitely. (Or at least for as long as their is solar energy available.)
Nuclear fuel must be mined. Nuclear changes are reversible, but that requires the same amount of energy you would get back out. The Earth has a finite amount of nuclear fuel. (To be fair, there's a lot. The Earth contains enough fuel to last roughly the lifespan of the Sun.)
This supports your point even more!
China might be subsidizing solar panels today. We should load up while we can. Let them bear the initial costs. We should also spin up more domestic manufacturing (which was a component of the IRA). If/when foreign sourced solar panels are no longer available, you are in the exact same position you were the day before. All of that already installed generation capacity will continue to function for years before a crisis emerges.