HPC's lost histories will power the future of tech

Opinion The first hints of an empire falling are only clear in retrospect. At the time they happen, they can look as if they are just part of existing trends. Crypto chaos as FTX falls apart like a giant rotting peach? The whole scene stinks. Everyone knows it, except the marks.

Nvidia having a hard time? Who isn't in the silicon supply chain hard slog party? Look at Intel. Look at Arm.  Oh, and the world just hit 8 billion people. 

What made Intel, Nvidia and Arm so prominent in the first place is the economy of scale where your products are connected to the lives of the entire globe. The 85 percent plus of humans on Earth with a smartphone, all connected to a globe-girdling infrastructure, is as big as you can get. While Moore's Law makes faster-equals-better – FEB – the easiest path to take, the designs with the biggest market share are going to pull away and leave everyone else in the dust. Then, suddenly, you hit the 8 billion body barrier. 

There are no worlds left for conventional tech to conquer. As a species, we may not need more personal CPU cycles, nor megabits per second, nor terabytes of storage. As Moore's Law fades, FEB gets more costly even as its appeal declines. There's no new mass market driver to take over.

Take the crypto experiment, which has been sucking up spicy silicon like a hungry schoolkid given a bowl of ramen. Turns out it isn't tech economy muscle bulk, it's a tumour, taking resources in exchange for criminal toxicity. And if crypto's a tumor, the best you can say about the Metaverse is it's just a nasty little rash.

Economy of scale has hit diminishing returns. There'll always be a market for the very top end GPUs, for example, but they are far less compelling now that barely perceptible performance increases are bought by unconscionable hikes in energy consumption. The joules per frame may go down each generation, but the battle for spec sheet superiority drags total power back up, and up, and up. Nvidia's flagship RTX 4090 can soak up nearly 800 watts at peak chat. For what? The smell of melting plastic? It looks like a bad place to be. 

These things are unsustainable and undesirable, and there's nowhere else to go. Like Concorde, the model where ever higher performance trumps all other considerations is flawed. In an economy and an environment where resource constraints are both inevitable and desirable, the model falls apart quickly. 

This has some interesting and exciting consequences. While our personal FEB equation is undone, it remains the absolute rule for high performance computing. That has a very important role in all our futures. 

HPC has had a free ride on the back of many decades of consumer and corporate FEB. As industry-standard architectures become as good as, then better than, the diverse technologies that powered older HPC, the sector saw prices fall, software support swell, and development times shrink. The core computational, communications and storage components could be ordered off the page, making it sensible to spend resources on tuning scalability and larger architectural issues. 

Which is great, but leads to a monoculture where innovation is a risk that's easy to avoid. The latest top 500 supercomputer list is 95 percent Intel/AMD X86, because that's where nearly all the world's performance CPU budget has gone – chasing FEB for all. Ditto datacenter GPUs, which would not have happened without gamers gagging for 4k@60fps.

That driver is going, which opens the door to competitive advantage by radical reinvention of the core technologies. We can see signs of this happening with companies like Cerebras, taking wafer-scale integration – WSI – as a kind of super-steroided SoC, with efficiency and speed benefits to match. How very 21st century – only WSI was first around in the 1980s at Sinclair Research. Yes, that Sinclair Research. 

• Twitter is suffering from mad bro disease. Open thinking can build it back better
• How do you protect your online systems? Cultivate an insider threat
• The open internet repels its most insidious attackers. They'll return
• Excel's comedy of errors needs a new script, not new scripting

Go back to the '80s and '90s, before the X86 hordes rolled over the world, and you can find a hundred promising new technologies at the edge of performance computing. To take just one example of many, Cray's massively parallel XMT architecture emerged from Spooksville in the early 2000s, just as the Intel/AMD juggernaut was getting into gear. XMT couldn't compete against the scale and R&D spend, and silently vanished. But that R&D has now been spent and the fabs lie waiting. What could XMT do in today's clothes? 

It's not just silicon; superconducting logic using quantum engineering for classical combinatorial designs was hot. In a very cold way. Whole families of devices were chilling in the lab.  We're much better at that sort of engineering now, so what might be rediscovered to have practical advantages today?

Expect an '80s and '90s revival, as researchers apply today's techniques to the most promising ideas of the past. Go back through the reports from the Supercomputing Conferences of yesteryear, where many exotic ideas were showcased. When empires cease to grow, they stagnate and die, and that's when revolutionaries use the best of what's left to recreate the world. HPC will be where it starts to happen. ®