33 architecturalist papers: more than just the data path

Rowing in the Swedish archipelago, the wind suddenly stopped, and the water became this oily, super smooth surface. We stopped moving and got fascinated by the water drops forming patterns on the surface. Even the most common thing can turn into something beautiful under the right circumstances.

Over the last year, I have been thinking a lot about the nature of software and software systems requirements. When I started my career, I was at the tail end of the era that only cared about performance. The fundamental thesis of that era was that CPU cycles were not plentiful and that writing efficient code was critical.

It was 1996 when I left Brown University. At that time, the forward-thinking software engineers advocated for higher-level programming languages to alleviate some of the tedium of writing assembly.

Getting a computer to work, though, was more important than it working for very long. Having a machine that did a computation was the hard problem. As we got better at building computers, a lot of energy was expended to make those computers more robust.

It’s important to realize, in 1996, writing efficient code on computer systems that were assumed to work was the only mental model and constraint you had when designing system software.

In 2004, when Steve Jobs introduced the iMac with colors, the industry was aghast. The design of the user experience was no longer an afterthought but the centerpiece of software. I remember doing my master’s degree at Stanford and wondering why human-computer interaction was a thing. After 2004, we all had to get religion very quickly.

In 2009, the next profoundly disruptive requirement was availability delivered through clustering. Before 2009, the mental model for availability was to buy bigger and bigger machines that were robust and reliable. But AWS happened. And although the debates about clustering had indeed begun in the late 90s, they took center stage when AWS released infrastructure where availability was very poor and required native clustering to deliver uptime.

At about the same time as AWS in the cloud happened, the dev-ops movement emerged. Rather than fixate on what dev-ops is and is not, let us look at what it was a response to. In the 1990s, when you wrote software, the expectation was that you would buy new hardware when the new version came out. Upgrades happened when there were critical bugs.

What AWS enabled was a completely novel form of software delivery. Instead of delivering software with hardware, it became possible to only deliver the software. In short, the software went about from being something that you bought once to a machine that continuously was being maintained. This need for continuous maintenance created a new set of requirements.to deliver software continuously

In the early 90s, the serviceability of the hardware was not an important consideration of hardware design. As customers bought more and more computers, and the time to repair each computer took more and more time, the ability to service the computer became more and more important. So much like hardware in the 90s suddenly started caring about how fast and how easy it was to replace the disk drive, software engineers had to start caring about how easy it was to operate the software. Why? Because it was expected that the software would always have to go through some regular maintenance.

Let me try this differently. In the early 90s, when hardware failed, the computer went down. Through a series of hardware innovations, it became possible to replace most hardware components when they failed if you are willing to pay the money. In 2009 we adopted software architectures that enabled a similar kind of upgrade of components and replacement components while the system was continuously running. Once that became possible, it became possible to add new value to the software continuously. In short, the software became just like the hardware, something whose serviceability became critical.

To give a flavor of this, in 1996, SGI released the Octane. The Octane was a fabulous system. But serviceability was never considered. You couldn’t rack it. It generated too much heat for an office. And so, it wasn’t that useful if you needed to buy 50 for a rendering farm.

In 2013 Edward Snowden happened. After he betrayed his country and fled to Russia, information security which had been a sideshow, became front and center. The difference in how computer system architects considered security before 2013 and after 2013 is night and day.

The next trend to emerge in software at large-scale companies will be the discipline of strategic system software. Most software companies have exactly one product. Large companies have many different products that must all work together and be independent. Software architecture has historically only considered how you architect a single software system with a single market in a single business. Strategic system software architecture considers how a collection of products, each with their own markets and business, fundamentally integrate to deliver more business value than the individual components by themselves without turning them into a single product.

Because of this discipline’s boutique nature: you have to work at a large company, be fairly senior, and the executive team has to see the value of that; there are very few practitioners of this art.

But as the arrow of time moves forward, I expect more of us to emerge.

Now those of you wondering what’s next, I believe it is the intersection of the law and of software. As more and more software gets embedded more and more deeply into people’s lives, the need to understand the law, the law’s intent, and technology will be a new architectural discipline that software systems will need to incorporate. To date, software systems and their architects view regulatory constraints as an inconvenience, not a necessity. Software system architects, who wish to define a new era, are advised to get a legal degree and understand the complex regulatory legal and its technical nexus that will drive the next important technical innovations.

When I started my career, system architecture was essentially about making the data path go fast. 20+ years later, it’s much, much, much, much more than that. And the experience of these last 20 years is that the set of constraints will only increase over time.

What does this mean? I don’t know.

Cross posted from http://wrongtool.kostadis.com