AI @ Home - Part Four: I'm Not Sure Any Of This Was A Good Thing
The opinions the build log kept deferring: rate limits as somebody else's gpu-memory-utilization, the gate that serves eighty-four other people you'll never see, peak pricing paid in conversations, and AI changed my life - I don't know if that's good.
Parts one and two were build logs, and build logs have a comfortable property: every problem in them has an error message attached. This post is the part I kept deferring while writing them - the section titled “the part I’m least sure about”, which outgrew the post it was drafted inside and, unlike a CUDA graph, does not get captured by restarting anything.
It is opinions, held out loud, with receipts from the build logs where I have them. No new hardware appears here. That is slightly the point.
The part I’m least sure about
Ed Zitron has a low opinion of posts like this one, and I don’t think he’s wrong to.
He is one of the more coherent critics of the current AI moment, and to be fair to him the headline is worth stating properly: his main case is not about the technology at all, it’s about the financing. The hundreds of billions being pumped into data centres on debt, circular deals and vendor money funding its own customers, the chip companies buying back their own future in the form of their customers’ equity - an investment boom dressed as a technology revolution, and bubbles are where that kind of money ends. The technology critique is downstream of that argument, not the other way around. One of his recurring side complaints is the genre of social media post where somebody announces that AI has changed their life, which he gets into on The Tech Report: our lives aren’t really being changed much at all, while the grids get strained, our work gets taken to train the things, and the money gets made regardless of any of it.
The genre deserves the eye-roll. It’s mostly people with something to sell, saying something vague and unfalsifiable. “It changed my life” is not a claim anyone can check, which is exactly why it gets posted.
So, with all of that on the table, and knowing precisely how it is going to sound:
AI changed my life.
I don’t work the way I did two years ago. Not marginally. Structurally.
I went back to being a student. That went a lot better than it had any right to, and I am genuinely not confident it would have gone the same way without these tools sitting between my head and the page.
Because here is the thing I’ve gone back and forth on putting in public.
I’m dyslexic. I’m a techy from the rural end of down under who has spent a career able to build the thing and then unable to write the document explaining the thing. Almost every employer and every institution I have dealt with treats those as one skill. They are not one skill. I have one of them.
This post is several thousand words. I did not write several thousand words. I built the thing, I made every decision in here, and I can tell you exactly what --n-cpu-moe does and why it is the wrong instinct on a GB10 and how the checkpoint load times felt. The knowledge is mine. Most of the sentences are not.
Two years ago, this post does not exist. Not “exists, but worse.” Does not exist. I would have built all of it and written down none of it, which is what I did for about fifteen years.
And I don’t know if that’s good
This is where the testimonials stop, and it’s the only part I actually think is interesting.
I can list what worries me. I can’t resolve any of it.
- I don’t know whether I’m learning to write or learning to route around writing. Those feel identical from the inside.
- I don’t know whether the ideas are getting better or whether they’re just getting presented better, and I’m the last person qualified to tell the difference.
- I don’t know what you’re supposed to do with the knowledge that a machine shaped these sentences. I’ve told you, which is more than most, and I don’t know that telling you changes what it’s worth.
- I don’t know what happens if the tool gets expensive, or enshittified, or simply goes away. I have built a working practice on infrastructure I do not own or control. This is not hypothetical for me: it already happened once, in a small way, and it is the reason there are two DGX Sparks in this series at all.
That last one should bother me more than it does. It is precisely the criticism I would level at anyone else’s architecture, and I’ve spent part two being smug about keeping everything in Ansible so it survives hardware going home. I have no equivalent plan for this.
The job that was never going to exist anyway
One of the arguments being had in public is about what AI costs the job market, and I want to file a small objection from somewhere the argument rarely reaches: the org unit where I work.
AI is not going to take any IT jobs away in my patch of any company, because the jobs do not exist in the first place. The replacement conversation assumes a staffing level to replace down from. It pictures the team that isn’t there. I am not standing in a corridor full of colleagues nervously watching the model learn to do their jobs; I am the corridor. The fear being sold to knowledge workers everywhere - something is coming for your seat - requires you to have a seat, and in a lot of small IT shops the honest state of the org chart is one seat with a person in it and a pile of work with no one in it at all.
So my version of the AI-and-work question isn’t defensive, it’s arithmetic. If I could convince my employer to hire two of me - ideally three - and the tooling actually moved the needle on what that team gets done, I would take that as a win so hard I’d stop pretending to be calm about it. It will not happen. They are not going to spend the money to buy a second me, no matter how much winning I do with the first one, and a headcount case made by an overworked man with a productivity demo has never once survived contact with a budget meeting. The win isn’t allowed to exist. Complaining about being overworked is the only lever, and everybody knows what that lever is worth.
Which is why the take-and-take-away framing feels so off to me from down here. In units like mine, AI doesn’t sit opposite the jobs; it sits opposite the vacancies that were never going to be filled. Whether that ends as “the one person finally gets a break” or “the one person is now doing the work of three with no title change” is not decided by the model. It is decided by the same budget meeting that already said no.
The bit where the compute runs out
While I was writing that list, the worry in its fourth bullet turned up in public. Theo on Anthropic’s Claude Code limits:
Hey Anthropic. You guys have given me a lot of subsidized compute. In return, I’ll give you an example of how to make announcements like this without burning all your good will with developers.
“Back in May, we did a one-month promotion where we increased the weekly rate limits for Claude Code by 50%. We’ve extended this promotion multiple times since, but sadly, it must come to an end on September 14th.
We know that this will impact the work of many users. We’ve run the numbers and found that we are able to make a permanent increase of 25%. This nets out to a 17% decrease from the limits as they are today.
We understand this is a big hit for a lot of users. We hate making changes like this. We’re doing everything in our power to secure more compute to improve availability of Claude.
We do have some good news. When these changes go live on September 14th, we will be removing the separate weekly limit for Fable, so you will now be able to use your full limit with your preferred Claude model.”
- @theo, 30 August 2026
Read that carefully, because it’s easy to mistake for a complaint about the limits. It isn’t. He’s writing the announcement he thinks they should have written. The criticism is aimed at the communication, not the capacity.
It’s also worth noticing where each of us is standing, because it is not the same place. He opens by saying he has been given a lot of subsidised compute. I pay about $35 AUD a month. He is arguing, generously and in public, about how a change like this should be communicated to developers. I’m one of the people it gets communicated to. Neither of those is bad faith and both positions are real, but they are different vantage points, and the distance between them turns out to be exactly what the rest of this section is about.
I want to sit on the capacity half, because I have spent this entire series living in its tiny domestic version.
Every build-log section of part two is the same question at a different scale: what fits, and what do you give up to make it fit. Q6_K against FP8. 128 GB unified against 12 GB discrete. --n-cpu-moe. gpu-memory-utilization. That’s all any of it was.
Now scale it up. A fully kitted 8-way B300 box carries 288 GB of HBM3e per GPU, so call it 2.3 TB in the chassis. On that one machine you can run:
- one instance of a 2.4T-parameter model at FP4, or
- around 85 instances of a 28B one
Same box. Same power draw. Same rack units. Same electricity bill.
And it bites harder than the weight figures suggest, for the reason the GB10 section was entirely about: whatever memory is left after the weights becomes KV cache, and KV cache is what decides how many people you can serve at once and how much context each of them gets. Fill the chassis with one enormous model and there is almost nothing left over. The big model doesn’t just serve fewer people because it is big. It serves fewer people because it has eaten the room they would have been served from.
There’s a second constraint stacked on top. Token generation is essentially sequential: each token depends on the one before it, so a single conversation cannot be spread across the machine for speed the way a compile or a render can. What that hardware is genuinely good at is running many sequences at once. So the question was never “how fast can I serve one person.” It’s “how many people can I serve at once,” and those two pull against each other.
Which leaves whoever owns the machine with an actual decision: that box answers to one person, or it answers to eighty-five.
Those are not comparable models and I’m not going to pretend they are. A 2.4T model does things a 28B flatly cannot. But that is the trade. Somebody has to decide whether the hardware gives one person something extraordinary or eighty-five people something good enough, and every rate limit that has ever annoyed you is that decision arriving at you from the far end of it.
The gate is doing something
There’s a second thing in that tweet worth pulling on, and it’s the part filed under good news: the separate weekly limit on the flagship model going away, so you can spend your whole allowance on your preferred model.
I understand the want completely. It is also my want. But a gate like that isn’t only a tax on the user. It is part of what makes it possible to serve everyone else at the same time, and two things make it stickier than a feature flag.
Committing capacity is not an instant operation. This is the bit I have genuinely felt. Bringing a large model up is tens of minutes of reading a checkpoint off disk, compiling kernels, and capturing CUDA graphs before it answers anything, and I had to bind a cache directory to the host just to stop paying that price on every restart. Scale that to a fleet. Deciding how much of your hardware answers as which model isn’t a toggle somebody flips when demand shifts. It is a commitment measured in loading time and rack space, made in advance, against a guess about what people will ask for.
Removing a gate concentrates demand. If everyone can spend their entire allowance on the most expensive model available, they will, and that is completely rational for each person and ruinous in aggregate. The gate is a good part of what keeps a slice of the fleet doing high-concurrency work instead of everyone piling onto the biggest thing at once.
So “let me use my full limit on the model I prefer” and “keep the service usable for everyone at once” are in direct tension, and the second one is invisible from where the user is standing. You feel the gate. You never see the eighty-four other people it’s serving.
I can’t dodge this either
The uncomfortable part is that self-hosting doesn’t exempt me. It just makes me the one making the call.
I have two GB10s committed to serving exactly one model: a 180B MoE whose checkpoint is 185.5 GB, tensor-parallel across both nodes, with about 24 GB a side left for KV cache at util 0.70. Both machines, entirely, for one model.
I could instead run several 27B instances across those same two nodes, and I’d expect at least double the throughput out of the same hardware, with far more concurrent sessions and much more room for KV cache in each. Smaller weights, more space left for context, more requests in flight. It is the same arithmetic as the B300 chassis, at a rounding error’s worth of scale.
I haven’t measured that, and I want to flag it as an expectation rather than a result, because I’ve already admitted this build is unbenchmarked and I’m not going to quietly start quoting numbers I haven’t earned. But the direction isn’t in question.
I run the big one anyway. I spent hours compiling a custom image from an unmerged PR specifically so I could run the big one. I did all of that so that one user, me, gets better answers.
That is precisely the choice Theo is asking for, made by someone with nobody else to be annoyed at.
If these tools are going to remain usable long-term, somebody has to keep making that call, at every scale, and it will not always land the way any individual user wants. The honest version isn’t “rate limits are good.” It’s that capacity is finite, allocation is a decision, and every decision spends something.
And then the obvious next move, which is where it turns
Everything above argues that capacity has to be allocated somehow. So follow that to the next step, because the next step is the one I actually want to flag.
The clean engineering answer to fixed capacity and variable demand is time-of-day pricing. Every electricity grid does it. Every cloud spot market does it. If demand peaks predictably and the hardware can’t move, you price the peak and discount the trough.
Applied here, it’s obvious. Enterprises that can pay full API rates buy guaranteed capacity during business hours. Everybody else gets cheap or free access off-peak, when the machines would otherwise be idle. On paper that is efficient and even faintly progressive: the expensive customers subsidise the cheap ones, and the hardware never sits still.
Now ask what the discount is actually bought with.
The pattern across this industry is fairly consistent, and it isn’t hidden. Paid enterprise and API tiers come with data handling guarantees, retention limits, and training opt-outs. Free and consumer tiers are broadly where your conversations become training data. That isn’t a conspiracy. It’s the business model, written down in the terms, in public.
So stack the two mechanisms together and look at what you get. Peak pricing sorts users by what they can pay. Data policy sorts users by tier. Run both in one system and it resolves to a machine that serves Fortune 500 customers at full rates with contractual privacy during the day, and harvests everyone who couldn’t afford that, overnight, in exchange for the access they’d otherwise be locked out of.
The people with the least money end up paying with the thing nobody told them had a price.
I’m on the paid side of that line, but only just, and by about $35 a month. It is not a comfortable place to be doing this arithmetic from.
The uncomfortable part is that every individual step in that chain is defensible. Capacity really is finite, and I’ve just spent a whole post proving it on hardware I can touch. Peak pricing is rational. Enterprise data guarantees are a legitimate product people are right to want. Free tiers funded by training data are disclosed. Nobody in that sequence has to intend the outcome for the outcome to show up anyway.
Which is the piece of Zitron’s argument I can’t dismiss with my own good experience. “Our things being stolen” isn’t only about scraped books and pirated archives. It’s the ordinary operating logic of a two-tier system where the tier you land in is decided by your budget, and one of those tiers pays in a currency it never got to set a price on.
I don’t have a fix for that. I notice that I’m on the paid side of it, and that noticing is not the same as being clean.
None of which makes a badly worded announcement good. Theo is right that you can deliver this news without insulting people, and right that “we hate making changes like this” reads better than a changelog entry. The physics being real and the communication being a choice are not in conflict.
What it did do is make me less confident in my own irritation. I have been throttled and been annoyed about it, and per the top of part two, that irritation is the reason this entire estate exists. I now know what the machine at the other end is doing, because I’ve just spent months discovering that a checkpoint doesn’t fit, and that every fix costs you context, or throughput, or both. It is very easy to read a rate limit as a company being stingy. I think it is more accurate, and more uncomfortable, to read it as somebody else’s gpu-memory-utilization setting.
And I should say the obvious thing plainly: I am the last person who should be trusted to be neutral here. This post was written using the tool whose rate limits are the subject, on subsidised compute of roughly the kind Theo is describing, about hardware I was lent for nothing. That’s three separate kinds of generosity in one blog post. It doesn’t make the argument wrong. It does mean you should weigh it knowing where I’m standing.
And I want to be careful not to use my own experience to wave away the argument, because they aren’t actually in conflict. Zitron’s strongest case was never “the tools do nothing.” It’s about what’s being built, at what cost, paid by whom, and who gets to keep the upside. I can hold both: this genuinely changed how I work, and the industry around it may well be an enormous bubble built on other people’s writing and other people’s electricity. A thing can be useful to me and bad in aggregate. Most things are.
And then David Gerard, on the same channel, puts the case in the four words its title leads with: AI’s use case is work that shouldn’t be done. Not “shouldn’t exist.” Shouldn’t be done - the slop, the spam, the synthetic flood, the engagement content manufactured by the bucketload for nobody. And here is the part that costs me something to concede, because it indicts the very page you are reading: he’s probably right. This post is probably work that shouldn’t be done. I am not a writer, I built a thing, and the machine turned “I should write this up” into several thousand words across four posts that nobody commissioned and nobody asked for. By Gerard’s framework, I am not the counterexample to his argument. I am an exhibit.
What I don’t accept is the therefore. The anti-AI case - which on a lot of its points I’ve just spent this post agreeing with - keeps arriving at “and so this should stop,” as though the sentence were a policy and not a wish. This is what I have done, with a $35-a-month subscription and a box under my control. The people doing it are here, the tools are downloadable, and the genie is out of the bottle and is not going back in. Maybe it doesn’t survive in its current form - the pricing tiers, the slop economy, the valuation theatre all deserve whatever comes for them. But the capability itself does not consult the polls, and “work that shouldn’t be done” has never once needed anybody’s permission to get done. The honest fight was never whether this continues. It’s whether the people holding the box ever get around to checking what’s in it. I know one who does. There are nowhere near enough of us.
So I’m not going to land this one. The honest position is that AI changed my life, and I do not know whether that is a good thing.
Anyone who has settled that question in either direction is selling you something.
The advice is fifteen years old, and I hacked two boxes with an $800 desktop
Eli the Computer Guy has been ripping into the recent Five Eyes statement on AI and cyber risk, and I want to stand next to him on this one, because the statement - reduce attack surface, accelerate patching, address legacy systems - reads like a press release assembled from a keyword search of every breach post-mortem since 2010. None of it is wrong. That’s kind of the problem.
I object specifically to the patching line, which is presented like the thing was solved and we just must. Patching has not been “solved” - we have the facilities and we do not have the systems. The machinery to fix the known-knowns has existed for a decade and a half. The Australian Cyber Security Centre - the ACSC, which sits inside the Australian Signals Directorate, Australia’s equivalent of the NSA, itself a part of the Australian Defence Force - has been publishing the Essential Eight from cyber.gov.au for years, and it is exactly the same advice with an Australian flag on it. When the signals directorate - a statutory agency in the Defence portfolio - has to keep re-publishing “patch things” as a maturity model, that is not guidance failing to reach people. That is guidance that reached everybody a long time ago and lost to a budget meeting on sight. I have seen the scoreboard from inside the industry: I know organisations that do not meet maturity level one, let alone level two. Fifteen years of “just patch it” arriving at that result is not a solved problem with an adoption problem. It is an unsolved problem, and pretending otherwise lets the people writing the advice skip the actual question, which is why doesn’t this get done, not why don’t they read our sheet.
And now the part my own quick sidebar in part three makes horribly concrete. In late July the FBI warned that malicious actors were hitting internet-facing PLCs at water and wastewater utilities - seven states since the 27th of that month, actual degradation of water operations. Read the headline’s own grammar: internet-facing programmable logic controllers. The attack path is the one my lab demonstrated, scaled up: the scanning data is already public - Shodan hands anyone the entire exposed attack surface of a nation on a web page - the vulnerabilities are all known, many of them older than the machines running them, and the one ingredient that used to gate this class of attack was a human skilled enough to assemble the pieces.
That gate is now a prompt. My lab run wasn’t a frontier model behind an API with a cyber-verification program. It was a 27-billion-parameter unconstrained open-weight model - a couple of questions deep into the task, porting exploits mid-run - and the same class of model fits on the B450 / Ryzen 5 5500 / Arc B580 box down under. That system cost me eight hundred Australian dollars at build time: five hundred of it the graphics card, the rest new-old-stock that was on clearance. Retail pricing at the time of writing, and GPU prices move like weather. Check it yourself before quoting me at anyone.
Which is the opinion, stated plainly and labelled as mine: if a second-hand desktop and a downloadable checkpoint is all this takes, the issue is only going to accelerate, and every advisory that leads with “patch more” is describing the speed limit while the road stays open. The Essential Eight still works. That’s exactly what makes the acceleration grim rather than merely interesting - the defence hasn’t been disproven, the attackers just got cheaper faster than the defenders got funded. The gap between “known good practice” and “organisations at maturity level one” was already the real vulnerability. AI didn’t create it. It just started charging rent.
Two honesty labels, because this section is thinner than it sounds. My lab is n-of-one, two boxes I built to be broken, against a 2017 vulnerability - it demonstrates the shape of the path, not its statistics. And the PLC incidents are the FBI’s reporting, not my evidence; my runs say what an agent can do, the alert says what somebody did, and between those two sentences the argument lives on borrowed ground. I’d rather say that than let the $800 number carry more weight than it has earned.
Per series tradition, since this post is opinions and opinions have no error messages:
Things not yet done
- Run the same lab task against a properly patched estate and publish the null result, so this section’s claim is a comparison and not just a highlight reel. The boxes are still there. The patches are still free.
- Get the Essential Eight maturity numbers from “organisations I know” to something I can cite without betraying them, or soften the claim to vibes-and-anecdote permanently. Right now it is the former pretending to be the latter.
- And any number of other one-day jobs, including more about the “secure” deployment that has now been owed since the first post like a bar tab at the end of the world.
The end of the series, such as it is… For now?
Part one was a card and a conversion that lied. Part two was two borrowed supercomputers and everything that went wrong with them. Part three is the knowledge base the whole estate was for - still filling, still owed. This part is what I actually think about all of it, which turned out to be too much for a coda and not enough to be certain about.
The Zitron section of this post carries one standing instruction from the draft notes of part two, and it travels with the text: the author’s own hand belongs on the wording before this ships.
Disclaimer, as ever: this post was written with the help of Claude Opus, gemma-4-26B-A4B-it and Qwen3.8-Flash-Next - which means the tool whose rate limits, guardrails and pricing tiers are argued about above helped write the arguing, and the post says so in the text itself rather than only here. That is three kinds of generosity in one blog post, and I’ve tried to weigh the argument knowing where I’m standing. The dyslexia disclosure is mine alone and permanent. Every bad decision in the build logs is mine too; the sentences mostly are not, and the last section is about why that’s uncomfortable rather than why it’s fine.