Friday, March 02, 2007

software temples


software temples


I've been listening to the seminars about long term thinking from the Long Now Foundation. I highly recommend them.

The purpose of these seminars is to encourage long term thinking. They succeed at that. For the last few weeks as I've been listening to them I've been contemplating the future of software and the history of civilization. I can't say that I've ever been interested in history. But after attempting to look forward more than 100 years it only now occurs to me what a good idea it is to look backwards far enough to see the patterns and cycles in human history.

Some of the seminars are inspiring. Danny Hillis talks about his progress in constructing a clock that will run for 10,000 years. At first it sounds insane, but after listening to him I can't help but admire him and what he's doing. The real purpose of such a scheme is to trick yourself into a different kind of thinking. Similar to the idea of forcing change in your behavior by changing your social role, such as giving a lecture will force you to learn the subject matter, building a 10,000 year clock forces a discipline of long term planning that generates new ideas which otherwise wouldn't occur.

Inevitably my thinking is grounded in the world of computing. Whatever new lights cross the sky I relate it to what I already know.

When I started programming I thought 5 years was long term. I couldn't think of an application being used longer than that. Now I've been working 10 years I have to maintain software at least that age, some of it even written by me! But I'm still not thinking long enough. Where I work the software I write must last 30 to 50 years, the typical age of the products they support. If I don't at least try then I know future maintainers will feel the pain I feel today, the pain of supporting software from extinct vendors running on ailing platforms with years of accumulated bloat.

What does it take to support software 100 years, or a 1000?

Clay Shirky in his seminar "Making Digital Durable" says that over a long enough period of time digital preservation always comes down to social issues. To keep a data format long term you need to preserve the application that interprets it. To run the application you need the operating system. To run the OS you need the hardware. The layers of abstraction don't stop. The longer into the future you want to preserve something the more layers you must deal with by somehow emulating them. They become social issues because the only way you can deal with this is to lower risk; use cheap and flexible components that have wide, popular dispersion and high degeneracy, by which he means multiple degenerate formats that represent the same information.

Would it be possible to create a long term social organization that can preserve information over millennia?

Say you as an independent agent have thought about this problem and you decide you need to coordinate with a large group. Then say many others reach the same decision, and you begin to coordinate using a system that only emerges through your coordination. By subscribing to this system you may have lowered the risk of preserving your valuable information. Only time will tell. The whole coordinating group improves their chances too if the artifacts they are preserving have some positive benefit to their survival.

When you look at history for examples of long term social organizations only a few really jump out. Religion and monarchy are two stand out examples of institutions that have lasted over 1000 years. In some cases, like the English Monarchy and the Church of England, the institutions are combined. More recent social systems I'd consider fit the same pattern are the Free Market, Science, and most recently the Free Software movement.

Given that list of systems for social organization, and given you wanted a message preserved over 1000 years, based on past performance, which would you choose?

All of these systems are an emergent phenomenon of complex adaptive agents. Stephen Lansing describes one in particular in his seminar, "Perfect Order: A Thousand Years in Bali." He describes the Water Temples in Bali, a religion that coordinates a large group of rice farms to optimize their yield. This system has remained remarkably consistent for 1000 years.

Douglas Adams discuses the same religion in his essay, "Is there an Artificial God?". The Bali system was very efficient but in the 1970s the IMF tried to get them to modernize their processes using 5 year plans, technology packets and genetically modified rice. The system improved briefly, but then because the coordination was gone among farmers, the pests crept back into the fields and yields worsened. Eventually the Balinese farmers returned to their system of Water Temples to coordinate the timing of planting and flooding their fields that had worked so well to eliminate pests.

Douglas Adams says, "It's all very well to say that basing the rice harvest on something as irrational and meaningless as a religion is stupid—they should be able to work it out more logically than that, but they might just as well say to us, 'Your culture and society works on the basis of money and that's a fiction, so why don't you get rid of it and just co-operate with each other'—we know it's not going to work!" [1]

Are the Water God's in the Balinese sense real or artificial? Is it correct to even ask the question in that way? I think it might be wrong to judge it as truth or fiction and instead observe it as an emergent phenomenon that increases the long term survival of the group. To put it in evolutionary terms. There is no absolute fitness in nature for large scale long term social systems. (The Big Here Long Now)

'There is no such thing as the "fittest" kind of organism. We can only talk about how an organism propagates in a given niche, how its life strategies have become adapted to that niche. It is no more or less fit than another kind of organism that has adapted to some other niche.' - Ursula Goodenough, "The sacred depths of nature."

In another seminar Sam Harris in "The View from the End of the World" takes a long term look at religion. He reaches the conclusion that belief in Religion is enormously maladaptive and dangerous for human survival. But I think the weight of evidence is on the side of Religion. The world religions we have today have survived a very long time, and they have survived because their subscribers have continued to prosper. Their past performance is pretty good and the prediction that they will suddenly and catastrophically fail in some way doesn't seem reasonable. In the ecology of social systems in the world there is diversity, they compete, adapt and evolve. The books of religion, the ceremony, the beliefs no matter how strange bind the group.

What has this to do with software?

When complex adaptive agents made from software become more common, and they optimize for long term survival we will begin to see software take on behaviors that bear similarities to human social systems.

If people (complex adaptive agents) care about long term software they may adopt the trappings of religion to see the long term survival of their systems.

My long term bet is that we will see a software religion emerge in the next 100 years.


[1] Aaron Swartz saw a similarity in the terminology used to promote free markets and understand economics.


knapjack said...

Best post ever.

Have you read The Selfish Gene by Richard Dawkins? It's worth reading, or at least skimming, and then reading the last chapter or two.

Essentially, he says for any gene, there are only three factors: fecundity (how often it replicates), fidelity (how accurately it replicates) and longevity (how long the copies hang around). He theorizes that all behavior and emergent behavior in the resulting organism ultimately goes to bolster at least one of these three factors, whether or not it influences the success of the individual organism past reproductive state. He gives great examples, walking through several scenarios using both diploid and haplodiploid examples.

In the last couple of chapters (depending on your edition), he extends this idea to any self-replicating system, and toys with the idea of a meme, the equivalent of a gene in the idea-organism, and walks through similar examples showing how examining just those three factors can help form a kind of meta-explanation for the emergence, success, variance and adaptation of ideas.

Certain genes have few negative selection pressures because their effects lie outside the typical reproductive window (think genetic predisposition to Alzheimer's), so they, in theory, can positively affect their own replication though other expressions, ultimately to the detriment of the "host" organism. What I found fascinating is that memes can exhibit this same trait, essentially "ideas to die for" or ideas which cause the organism to exhibit behaviors that further the replication of the meme to the detriment of the host.

A soldier willing to go to war over, say, Freedom, puts him- or herself at risk, as well as any future generations, because this typically occurs during the age of replication. Monks in monasteries, choose celibacy -- choose not to replicate themselves -- because the idea (or, more correctly, idea-part) causes a stronger replication need in the organism, expressed in a non-biological way. Freud and occasionally Dawkins may argue that the expression ultimately affects the success of any kindred organisms, pre-existing offspring or opportunities for future replication, but what I think is most important is that the replication of the host is not integral to the success of the replication of the meme, as long as the meme does not do damage to its environment (as the celibacy meme did to the Shakers ).

Dawkins does a nice job doing a quick historical overview of the adaptation of Christianity surrounding the Reformation, as the self-interpretation meme variant (affecting fecundity) displaced the heresy meme variant (affecting fidelity). In a way, GNU may be the seed of this kind of complex idea with a structure that makes it suitable for replication in its environment, with strong memes (or clauses, in this case) strongly affecting both fidelity and longevity, and other memes affecting fecundity.

Some may argue that the competing MIT license-idea is more beneficial to the host organism (increasing income, in turn increasing its replication success), but Dawkins' core idea is not the identification of these three factors, but a kind of meta-understanding that at the end of the day the organism doesn't matter in the replication process. What replicates is not the organism, but the gene/meme. To follow Douglas' example and turn the world on its head, an egg gets another egg by making a better chicken.

So, for me, the Software Temple may end up being founded on something like GNU, where the combination of memes not only keeps the idea cohesive but keeps the host-organisms working towards its replication so long as the environment will support it. Or, we may already have our Software Temple, built upon Software Tools and expressed in a variety of denominations since, continuing to evolve.

Anonymous said...

"My long term bet is that we will see a software religion emerge in the next 100 years."

Haven't we already got one? It's called the Church of Emacs ;)

Therac-25 said...

This really does sound like a restatement of the GNU Manifesto in evolutionary terms.

The driving motivation behind the GNU Project is that there should be a free, unencumbered computing environment, so that our present creations are never held hostage in the future.

If a "software religion" is going to come from anywhere, it's going to come from this school of thought.