The Cambrian Explosion of Software

Five hundred and forty-one million years ago, life had a very boring resume. For three billion years, the planet's living things had been mostly single cells doing single-cell things: eat, split, repeat. Then, in a geological blink — twenty million years, which is nothing — nearly every major body plan we know today appeared. Eyes. Shells. Spines. Jaws. Legs. Predators and the things that run from predators. The fossil record goes from a monotonous smear to an explosion of weird, spiky, beautiful strangeness.

Nobody agrees on all the causes. But almost everyone agrees on one: oxygen got cheap.

Photosynthetic cyanobacteria had spent a billion years quietly farting oxygen into the atmosphere. At some threshold, the free energy budget of being alive changed. Suddenly you could afford a muscle. You could afford a nervous system. You could afford to be a predator, which is an expensive lifestyle. The constraint that had held complexity down for three billion years simply lifted.

And here's the thing that matters most: when the fuel got cheap, the game changed.

Before oxygen, life was mostly about the chemistry of survival — can you extract energy from this brine, can you copy yourself before the UV kills you. After oxygen, that problem was largely solved. The new problem was everybody else. A million organisms all suddenly capable of complexity, all reaching for the same niches. The game stopped being about metabolism and became about strategy. Adaptation. Goals in context. Which is to say: it became about intelligence.

The same thing is happening to software right now, and the fuel that just got cheap is intelligence itself.

The oxygenation of code

For seventy years, the binding constraint on software has been the cost of building it. Every product decision, every roadmap, every argument in every planning meeting has been downstream of one brutal fact: writing code is slow, expensive, and requires scarce humans who are difficult to hire and easy to lose. Entire disciplines — agile, scrum, technical debt as a concept, the whole priesthood of engineering management — exist to ration a scarce resource.

That resource is becoming abundant. Not free, not perfect, but abundant in the way oxygen became abundant: enough to change the metabolic math of what's possible.

Watch what follows. Volume explodes — vastly more software, written faster, by vastly more people. Interconnection explodes — every piece of software talking to every other piece through APIs and agents and protocols nobody has invented yet. Complexity explodes — not the bad kind that comes from bad design, but the Cambrian kind that comes from a million experiments running at once.

And, exactly as with the trilobites, the nature of the game changes.

When building is hard, the winning move is to build well. When building is easy, the winning move is to build the right thing — over and over, faster than the environment shifts.

From engineer to architect to gardener

There are three phases here, and we're standing at the edge of the first.

Phase one: engineers become architects. In the short term, the value migrates from writing code to deciding what code should exist and how it should be shaped. Not the grunt work of laying bricks, but the judgment of where the load-bearing walls go, which interfaces will need to survive three pivots, what should be a service and what should be a function. This is real work and it's the work most senior engineers already think of as their actual job. The difference is that it will become all of the job.

Phase two: architecture stops mattering as much. This is the uncomfortable part. Architecture is expensive foresight — you design carefully because rebuilding is painful. But when rebuilding costs almost nothing, careful design becomes a kind of premature optimization. Why agonize over the perfect abstraction when you can spawn six versions, let them run against real traffic, and kill the five that die? Nature never architected the eye. It grew forty of them, independently, and kept the ones that saw.

Phase three: the software gardener. Or breeder, if you prefer the agricultural violence of the term. Someone who doesn't build systems but cultivates them. Who plants variants, watches which ones take root, prunes ruthlessly, cross-pollinates the interesting mutations, and lets the rest compost. Natural selection extended with artificial selection — the same trick we pulled on wolves and wheat and roses, now applied to code.

The gardener's skill is not construction. It's taste and niche-finding. Knowing which fruit is worth growing. Knowing what will feed people, what will sell, what will be beautiful, what will matter.

The real bottleneck moves upstream

Here's the punchline, and it's the same punchline as the Cambrian.

When energy is scarce, the winner is whoever extracts energy most efficiently. When energy is abundant, the winner is whoever finds the right niche — the unexploited gradient, the food nobody else is eating, the shape of a problem nobody else has noticed.

Software is about to have the same reckoning. For decades, the scarcest input has been engineering capacity. Very soon, the scarcest input will be knowing what to build.

Which means the highest-leverage work in technology is shifting from writing solutions to hunting problems. Not brainstorming feature ideas in a conference room — actual problem-hunting. Finding the real friction in real lives: the workaround someone's been doing in a spreadsheet for eleven years, the phone call that shouldn't need to happen, the small daily humiliation everyone has silently accepted as the cost of being alive.

These are the niches. In the Cambrian, an unexploited niche was a food source nobody had evolved a mouth for yet. In the coming explosion, an unexploited niche is a problem nobody has bothered to look at closely enough.

And problems, unlike code, don't get cheaper to find. They live in the messy, embodied, deeply human world — in habits, frustrations, workflows, and the gap between what people say they want and what they actually do. That world is not going to hand you a clean training corpus.

So: fewer bricklayers, more gardeners. Fewer people asking how do we build this and more asking is this worth growing at all.

The trilobites didn't win because they were efficient. They won, for a while, because they found a niche and grew into it. Then something else found a better one.

That's the game now. Go find your gradient.

On the off chance you'd like to subscribe to my writing, please feel free to subscribe on twitter or use this RSS feed. Thank you.