The Cambrian Explosion of Software

Five hundred and forty-one million years ago, the Earth got drunk on oxygen.

For three billion years before that, life had been a slow, grey affair. Single cells. Mats of bacteria. Occasional soft blobs. Evolution was crawling because the fuel was scarce. Then the oxygen levels crossed a threshold, and in a geological eyeblink — maybe twenty million years, which is nothing — nearly every animal body plan we know today showed up. Eyes. Shells. Spines. Jaws. Legs. Guts. The ocean went from a thin soup to a crowded bazaar.

Biologists call it the Cambrian Explosion. I think we're living through its software sequel.

The old scarcity

Here is the thing people forget about the pre-Cambrian world: it wasn't that life lacked imagination. It lacked budget. Building a shell is metabolically expensive. So is a nervous system. So is a predatory lifestyle where you chase things that move. Aerobic respiration yields roughly sixteen times more energy per glucose molecule than fermentation. Once you have that, you can afford to be baroque. You can afford to be weird. You can afford to fail.

Software has lived in its own pre-Cambrian for seventy years, and the scarce resource wasn't electricity or memory or bandwidth. It was intelligence. Specifically: the scarce and expensive act of a human being sitting down and translating intent into instructions, line by line, bug by bug, for hours.

Everything about our industry is downstream of that scarcity. Reusable components exist because writing new ones was too costly. Frameworks exist because starting from zero was too costly. Technical debt is a concept that only makes sense when rewriting is prohibitively expensive. We built entire professions — architects, tech leads, estimators, project managers — around the problem of rationing a precious substance.

That substance just got cheap.

What happens when the fuel arrives

Watch what happened in the Cambrian seas, because the pattern is instructive.

First: volume. Not slightly more life — orders of magnitude more, in orders of magnitude more configurations. Expect the same with software. Not ten percent more apps. A thousand times more. Software that exists for one person, for one week, for one problem, and is then discarded like a mayfly. Disposable software. Software you make instead of filling out a form.

Second: interconnection. Pre-Cambrian life mostly ignored itself. Post-Cambrian life is nothing but relationship — predator and prey, host and parasite, symbiont and competitor. Food webs. Arms races. The eye doesn't evolve in isolation; it evolves because something is hunting you. Expect software to stop being a set of isolated tools and become an ecosystem of agents negotiating with each other, consuming each other's outputs, competing for your attention and your API budget.

Third: the rules of the game change. And this is the part that matters for your career.

Before the Cambrian, survival was a chemistry problem. Can you extract energy? Can you copy yourself? Do that adequately and the ocean was empty enough that you'd be fine.

After the Cambrian, survival was a strategy problem. The ocean was full. Everybody could extract energy and copy themselves. The question became: where do you fit? What do you eat that nobody else eats? What eats you? Trilobites didn't win by being better at metabolism. They won by finding an angle.

Scarcity rewards efficiency. Abundance rewards fit.

Three stages of the job

So what does a person do for a living in a world where building is nearly free?

Stage one — already here. The engineer becomes an architect. Not because typing code is beneath anyone, but because typing code is no longer where the leverage is. The valuable question stops being "how do I implement this?" and becomes "what shape should this be so it survives contact with next year?" Anticipating change. Drawing boundaries. Knowing which decisions are reversible and which ones are load-bearing walls.

Stage two — coming fast. Architecture itself becomes less precious. This feels heretical, I know. But think about what architecture is: it's an expensive bet placed up front because rebuilding is expensive. Remove the cost of rebuilding and the bet loses most of its value. Why agonize over the perfect abstraction when you can spawn eleven versions, run them all, and keep whichever one doesn't die? Nature never designed the eye. It grew forty of them independently and kept the ones that worked.

Build cheap. Build wrong. Let it die. Build again.

Stage three — the real job. The person who matters in a world of free building isn't an engineer or an architect. They're a breeder. A gardener. Someone who runs artificial selection on top of natural selection, who looks at a hundred generated variants and sees which three have a future.

This is not a lesser job. It's a harder one. Selection requires taste, and taste is the one thing that can't be automated, because taste is what you want, and wanting is not a computation. The breeder's skills are: knowing what a healthy specimen looks like, knowing when to cull, knowing which cross might produce something interesting, and — above all — knowing what niche you're breeding for.

Which brings us to the actual bottleneck

If the game is finding the right niche, then the scarce resource isn't code anymore. It isn't even architecture.

It's knowing what's worth building.

Here is the uncomfortable arithmetic. Most software fails not because it was built badly but because it was built for a problem nobody actually had, or a problem three other things already solved, or a problem that evaporated while the thing was being built. We tolerated this because building was slow enough that the waste was hidden inside the schedule. Now the building is instant and the waste is naked.

A trilobite with perfect eyes in an empty niche is just an expensive fossil.

So the most valuable work in software is shifting upstream, away from the keyboard, toward the world. Watching people struggle. Listening to complaints. Noticing the workaround someone built in a spreadsheet at 2am because the real tool didn't exist. Problem-hunting.

That's the skill. Not solution-building — solutions are becoming free. Problem-finding. The ability to walk into a messy human situation and spot the gap that nobody has filled, that somebody would pay for, that would actually make a life slightly better.

The Cambrian didn't reward the strongest organism. It rewarded the one that found the gap nobody else was standing in.

Go find your gap. Everything else is getting cheap.

On the off chance you'd like to subscribe to my writing, please feel free to subscribe on twitter or use this RSS feed. Thank you.