It is Monday people, and I know you don’t want to go to work, so you keep on procrastinating like you forgot your ADHD meds. So I thought it would be a good and friendly gesture to give y’all a good excuse for not answering all your emails for 15 minutes (plus 15 more for you to process it all). Because after reading this, you will have something to talk about during your lunch, from 11-14 – like the French do.
So, let me tell you about something that happend a few months ago.
A while back, the International Criminal Court in the Hague (in my country), or the ICC for short, got a wild hair up its ass and they – in all their wisdom – decided to declare Netanyahu a war criminal.
Now, you would think that’s a big deal.
A court doing court things ‘n all. In no way it was politically motivated.
Uh-huh. Sure. Keep on dreaming Aurora*
But then, a certain tangerine-tinted president from across the pond, a man who probably thinks “geopolitics” is a fancy map, decided that he didn’t like that one bit.
And what did he do?
He told Microsoft to pull the plug. Cut off the ICC’s access to Microsoft Office and their email. Just like that. The prosecutors and judges, who were probably in the middle of typing up some very important legal documents, suddenly found their most basic tools gone.
Poof.
Vanished.
* Aurora = Sleeping Beauty in the Disney PR sugar coated version of Briar Rose in the original Grimm version.
More rants after the messages:
Join me and Eliska Went on September 5th when we show you how to create your own automated AI tool to find, write and apply for jobs. Register here. Here, I mean: here
- Comment, or share the article; that will really help spread the word 🙌
- Connect with me on Linkedin 🙏
- Subscribe to TechTonic Shifts to get your daily dose of tech 📰
- Visit TechTonic Shifts blog, full of slop, I know you will like !
AI sovereignty my ass
This, my friends, is what we call a “holy shit” moment. It seems while we were all busy binge-watching Netflix and arguing about the next version of ChatGPT, the European continent became completely and utterly dependent on American tech. It is so deep entrenched in our daily lives, our companies, our governments, and that it gives a former ally the power to blackmail entire nations with the flip of a switch.
Microsoft, of course, denied everything.
“Wasn’t us”, they said with a drop of sweat on their face, and probably while counting a giant pile of money. But they’re legally on the hook. There’s this nifty piece of legislation called the International Emergency Economic Powers Act, or IEEPA, which lets the US President tell any American company to cut ties with anyone he doesn’t like.
And it now has turned into a get-out-of-jail-free card for international bullying.
This means the very idea of sovereignty is under attack. It is no longer a given that you can rely on US companies for your daily apps. One day a president can decide your country is on the naughty list, and the next day your data centers are dark and all your servers are nothing but expensive paperweights. This whole mess is called Digital Sovereignty, and a lot of very serious people have written a lot of very serious papers about it.
But writing about a problem doesn’t create a backup plan, now does it?
So, I decided to do some actual work myself.
I grabbed a couple of students who still had a spark of life in their eyes and set out to figure out just how hard it would be for some government body to cut the cord. To become truly independent of the US when it comes to their AI. So we cooked up a layered architecture of all the bits and pieces you’d need, and then I ran the numbers. We calculated the costs for three scenarios: a small-time operation, a medium-sized player, and a full-blown, big-league enterprise.
I wanted to know when you’d break-even if you decided to do it all yourself, free from any country, any provider. I’m talking everything from the cost of electricity for training these digital brains, to the hardware, the consultants, the integration, all of it.
And the results were. . . well, let’s just say they’re interesting.
The price of freedom
Let’s get one thing straight from the very start. Building your own AI infrastructure is not for the faint of heart, or the light of wallet. It is a journey into a world of pain, suffering, and invoices so large they could be used as blankets.
First, you’ve got the hardware, i.e. “the expensive toys”, and I am talking about GPUs, the things that make AI go brrrrrrrt (like the GAU-8*). These aren’t the things your kid uses to play Fortnite though. A consumer-grade RTX 4090 will set you back about $1,800. But if you want the good stuff, and I mean the top-of-the-bill enterprise-grade H100 systems, you are looking at a cool $40,000.
. . . Per card.
And you will need a lot of them.
A small setup needs 2-4 GPUs. A medium one needs 8-16, and a large one takes 32 or more – but if you take the monstrosity that XAI built, called Colossus, it originally had 100.000 NVIDIA GPUs by late 2024 and jacked it up to 200.000 cards early ‘25 and now it has halted at ‘only’ 230k chips . The total hardware bill for a small-scale deployment can easily hit $100,000, but for the extra large one, you’re looking at a cool $5 million. And that’s before I even talk about the other shit that you need.
And then there’s the operational costs.
That’s the real Euro-sucker.
This is where your dream of not being dependent on anyone but you, yourself and you, goes. . to die.
You need people.
Smart people.
And smart people are expensive. An AI (hardware/software) specialist receives a salary premium of 50-100% over a regular IT guy, and I’m talking salaries of E150,000 to E300,000 a year!, and for or an extra-large scale deployment, you need a team of 8-15 of these cash vampires, and when you sum it up, your calculator says you have an annual payroll of $1.6 to $4 million.
Just for the people.
Now let’s talk power. Energy. E=P⋅t if you were awake during physics class**.
Oh, the sweet, sweet power.
You got to know that these AI systems are thirsty. They drink electricity like I did when I was a frat boy, guarding a kegger totally wasted. An AI infrastructure consumes 2-3 times more energy than a traditional IT setup. A large-scale deployment can total your energy bill to a sweet, sweet E800,000 to E2 million.
A year.
So, when you add it all up, the five-year total cost of ownership for a small-scale, on-premises AI deployment is about $1.9 million. For an extra-large one, it’s a staggering $20 million.
* Brrrrt is the sound the multi-barrel rotary gun makes which is mounted under the A-10 (aircraft, ya noobs)
** Energy equals power times time. Also the reason your GPU-powered AI hobby or your Bitcoin mining side-hustle costs you as much as a small yacht.
The Cloud is your neighborhood drug dealer
Now, you might be thinking, “But what about the cloud?” And you’d be right to ask that question. Of course the cloud is the easy answer.
The convenient solution.
But it’s also a freaking trap.
I have come to compare “The Cloud” with a drug dealer.
The first hit is cheap – heck – maybe even free!. You get hooked on the scalability and the pay-as-you-go model but then the prices start to creep up – quite exponentially i might add – and before you know it, you are locked in, and switching to another provider is so darned expensive and so painful that you would rather just . . . . keep paying.
They’ve got you by the balls (or your curlies).
Our analysis shows that for small-scale deployments, the cloud is the only game in town. You’d need to run your on-premises servers at over 1,000% capacity to break even.
That’s not only impossible, but it’s also spectacularly stupid. But for medium and large-scale deployments, the story changes because a medium-scale on-premises setup breaks even with the cloud at 3.3 years, and a large-scale deployment takes only one year for your wallet to be drained at the same speed as when you’d outsource it all.
After that, you are saving millions.
A large-scale deployment can save you $2.4 million a year compared to the cloud. That’s enough to fund a whole lot of other fun projects.
Open Source as your insurance policy
Now, before you start thinking you’re some kind of genius for discovering open source as the magical solution to all this sovereignty horseshit, let me burst that bubble for you before the market does.
Yeah, open source is part of the equation.
But it is not the leaky-money-faucet-killing-silver-bullet that is going to save you from writing very large checks to very smug lookin’ vendors.
The thing about open source in the AI world is that it is a bit like getting a free Porsche engine.
Now that sounds cool.
But. . .
You still need to build the entire car around it, hire a team of mechanics who know what the hell they’re doing, and pray to whatever God you believe in that the thing doesn’t explode when you hit the gas.
The engine is free, but they get you by the balls (or the curlies) with everything else. . .
Now take something like LlLlLlama or Misssstral. These are beautiful, open-source models that make you feel all kooshy-kooshy and fuzzy wuzzy about technological frrrrreedom. But you got to realize that running these boys at scale wit set you back with E40,000 GPUs, and on top of that you still need the data centers, and you still need the army of nerds who can tell the difference between a transformer and Optimus Prime . The model is open source, but the infrastructure that makes it work is still gonna cost you more than Jamaica’s GDP.
Open source actually makes some things more expensive, not less.
When you are using proprietary cloud services, at least someone else is responsible when shit hits the fan, but with open source, well, you are the owner of every bug, and every security vulnerability, and every midnight emergency when your AI starts hallucinating.
But open source does give you one thing that money can’t buy. . .
The ability to tell any government, and any tech executive to skedaddle, because when you control da code, you control your destiny. No more worrying about El Presidente Naranja with a bad hair-day who decides he wants to cut off your access to the stuff that run your organization. No more license agreements that feel like hostage negotiations and definitely no more vendor lock-ins. The real value of open source in the sovereignty game isn’t the cost savings, though I must admint that those can be substantial once you get past the initial investment nightmare.
It is the insurance policy.
The final word, from me to you
So, what’s the takeaway you get all this. Let me try to make it simple, really.
When you’re a small player, you just stick with the cloud. You simply can’t afford to play with the big boys. But if you’re a medium or large organization, and especially if you’re in a regulated industry like healthcare, the government or finance, you need to seriously consider building your own AI infrastructure.
I agree the upfront cost is a bitch, I won’t lie.
But the long-term savings and the strategic benefits are undeniable.
This is not about money alone if that’s the message you got from reading this.
It is about control.
It’s about not having some foreign government or some faceless corporation holding your data, your algorithms, and your future hostage. It has to do with having the freedom to innovate, and to compete, and to tell anyone who tries to mess with you to go beat it.
So, take a good look at your scale, and your needs, and of course your ambitions. And then ask yourself the question if you want to be a digital sovereign, or that you are ok with being a digital serf.
The choice is yours.
But please don’t come crying to me when your cloud provider decides to change the terms of your deal, or when some president with a bad combover decides to cut you off.
Signing off (from Obsidian cause it’s free and open source)
Marco
I build AI by day and warn about it by night. I call it job security. Big Tech keeps inflating its promises, and I just bring the pins.
Think a friend would enjoy this too? Share the newsletter and let them join the conversation. LinkedIn, Google and the AI engines appreciates your likes by making my articles available to more readers.
Leave a Reply