Monday, 10 February 2020

The Cathedral and the Bazaar (and Quakers)

As some of you will be aware, I’ve put a fair bit of time in my life into software development – I’ve earned a living doing it, I’ve studied it formally, I’ve done it as a hobby. Some of this has been connected, to a variable extent, to the ‘free and open source software’ community, as much a social movement as a software development model, in which the source code of software is available, anyone can modify it, and there’s no restrictions in how you use it. This approach has given us Linux, which I imagine almost all of you have heard of, as well as office productivity packages like LibreOffice and OpenOffice, and specialist software like the database systems MySQL and PostgreSQL or the statistical software package R.
One of the great figures of the philosophy of free and open source software, Eric S. Raymond, wrote a famous essay about different approaches to open source development. Originally presented at a conference for Linux developers in 1997, and later published in a collection bearing the same title in 1999 (the book is still evolving, and available free on his website). That essay, and book, is called The Cathedral and the Bazaar. It talks about two models, the cathedral model and the bazaar model, that broadly describe the way most open source projects had been developed up to that point. So, what does this have to do with religion, what does it have to do with Quakers? Some of you might have a guess already, but read on – through some more about software development – and all will become clear.
Open source software isn’t a new thing, and it wasn’t even new when Linux broke on the world stage. It wasn’t a big thing for home computer users, to be sure, but when you’re talking about business and industrial computing, open source utilities had been around for ages. Someone would want a bit of software to do something, and it wouldn’t exist. A lot of the people in that position – because few people would have full access to do what they wanted on a general purpose computer – were programmers, though perhaps as a secondary skill to their main job. So they wrote a programme to do what they wanted. They then shared it with other people over a dial-up BBS or the internet (yes, it existed), and they shared the source because they’re just trying to be helpful (and because these would be run on computer that shared similar operating systems but with crucial differences, and ran on very different hardware – so people had to have the source code to compile it for themselves for it to be usable). Then other people would say “hey, that’s cool, but it would be great if it could do this other thing”, and either the creator would add that themselves, or the person suggesting it would go on to say “so I added that functionality, here’s the code for it”. The project would grow and improve, and maybe one day the person who started it would realise they couldn’t take it further, and pass it on to someone else. Or maybe someone else would disagree with the direction it was taking, and make their own version in order to take it where they wanted it to go (which is known as forking the project).
(It’s worth noting that there were times when software would be distributed in source form for portability but where license conditions forbade anyone from modifying it – and people would generally respect that. It might also have source code that was written in a way that’s intentionally hard for a human to understand. Open source projects were intended to allow people to see how it did what it did and to play around with it themselves, just as they are now.)
For a bigger project, some really serious software that isn’t about scratching a simple itch but that someone decided was to be open source rather than a traditional commercial package, you might have a group of people come together to work on it. They would toil away in ‘splendid isolation’, as Eric S Raymond puts it, and produce a product that people could download, compile, and use – and just as in the case of the smaller tool, they could send in patches and suggestions, and they could fork it if they wanted to. The team responsible for the package would keep working on it, and from time to time they would release a new version of it, with new features and improvements, and they would release the source for it as well. In essence, this would very much parallel the way software was developed commercially, with just the features that go with being open source added on. Releases were not frequent, and heavy testing was done before a release to make sure it was good (though this never worked out totally reliably, of course). If someone wanted to submit a patch between versions, they may get told “sorry, our code has changed so the patch doesn’t work”. They might then distribute their patch separately for anyone who wanted to rebuild the most recent version with their additions, but this became a mess if you had several such patches you wanted to add – and then when the new version of the software came out, people had to decide between holding back the version they used in order to keep the patch functionality or upgrading to the new version and losing the benefit of their third-party patches. This is what Raymond calls the cathedral model of open-source development: it is top-down and controlled by an individual or a small group.
For very small projects, the control of the cathedral model isn’t very noticeable – the developer releases as often as they feel like, and the risk of patches not being applicable is small because the overall system is simple enough that major changes to structure are unlikely. For larger projects, many developers – including Raymond, until he saw something to change his mind – would think that the organisation and control of the directing group was necessary in order to ensure a consistent vision and clean design. Then a smart Finnish chap came along, by the name of Linus Torvalds. Torvalds had an ambitious project – he wanted to write a new operating system kernel, one that was actually production ready. He wanted it to be UNIX-compatible. He wanted it to be something people could actually use. That’s a big job; UNIX was the be-all and end-all of serious computing, and it was a complex beast. But he had definite ideas about how to structure it all, and he did have a good idea of what he was doing.
He took two steps that are obvious good management, though he described them as being taken out of laziness. He started from a small, functional, UNIX-like kernel that someone else had already developed, albeit for educational purposes. MINIX was developed by an academic as illustrative material for his textbook on operating systems, and it was a functional UNIX-like system for inexpensive computers. This provided Torvalds with some ready-made technology, but more importantly a sort of scaffolding for the project, so he could implement parts and fit them into a whole before he actually had all of the parts ready. This is a normal way to do software development. The second step was less normal.
Being a student, and perhaps not entirely sure of himself – not to mention having some idea of how much work this was going to be – he reached out to the wider community, starting with online groups interested in MINIX. He shared his ideas, he shared his code, and he invited other people to contribute. Now, you may think that this was a learning experience and once he found his feet he would take close charge again, working with carefully selected people on different elements of the project – but he didn’t. As the project grew and became something more serious, the number of people involved continued to grow. He released code often, and even eventually made the ‘live’ development code-base available for anyone to look at any time they liked, seeing the current state of development. Anyone could work on the patch they wanted to contribute and make absolutely sure that it could be applied to the current code-base cleanly. Modular design also meant that people could develop significant functionality without interfering with code that other people were working on.
Rather than building his software as a cathedral, Raymond characterises this as development in the manner of a bazaar. This name reflects a certain amount of cultural appropriation/imperialism, but I’m not going to get sidetracked into the history of the term ‘bazaar’ right now. Raymond was, I think, referring to the typical westerner’s image when they think of a bazaar, which they generally imagine to be in south-west Asia, north Africa, or Asia Minor – a market in an enclosed open space, perhaps, in which stallholders each ply their wares, or a covered market in a solid structure in which the vendors each have a cubby-hole for a store. The mental image is one of bewildering choice and seeming chaos, yet everyone there – though they might compete – is to some extent working towards a common goal. It is in no-one’s interest for there to be offensive odours in the bazaar, and an enticing stall might draw footfall to other stalls as well. All want the structure to be sound, and dishonest traders might lead to them all being tarred with the same brush. Without any central organising principle, each does their own thing to work towards both their private goals, and the shared goals of all the traders.
Raymond then describes principles that are embodied in or derived from the success of development that follows the bazaar model, and all of them are specifically about developing software. I will return to them shortly, once I have explained just how all of this might relate to Quakerism, or to religion in general.
The allusion of the cathedral has obvious religious connotations. From my reading of what has been written, Raymond wasn’t meaning to invoke those connotations; rather, he was looking at how a great structure like a cathedral was built. However, the religious association is not entirely inappropriate. The churches that built the great cathedrals of the world were those that had some degree of authoritarianism, a hierarchy or web of leaders. The Roman Catholic church is the most obvious, with ultimate authority vested in the pope, but really the cathedral model applied to religion is not about central authority. It is the idea of authority being vested in individuals or offices at all. A cathedral religion has priests, in the Christian sense, ordained individuals whose ordination (and, to some extent, their preparation for ordination) gives them authority, or it has scholars whose position in a clerical role is by virtue of their having made significant studies of the law, history, and/or traditions of their faith. Even in Congregationalist Christian churches where there is no ordained priesthood but the congregation employs a pastor, the very fact of being employed in that role – and meeting whatever qualifications the congregation checked before hiring you – gives you authority. That is, as I understand it, part of what early Friends condemned about “hireling priests”. They were an authority interposed between the worshippers and the true authority, able to substitute their own understanding for true insights of the Spirit.
What, then, would a bazaar religion look like? Apart from the presence of charismatic (in either or both senses of the word, as you prefer) figures, and the followings they gained, early Quakers are a good example. No authority at all, until the imposition of gospel order, just a bunch of people with some objectives in common and some objectives each of their own, working – or worshipping – together, sharing the fruits of their insight. People throw things into the pot, and others take them if they will. Those that ring true catch on and spread around. A democratic process led by no person, but by the Spirit (though some of those charismatic figures certainly threw their weight into trying to make sure that views they thought not in line with the Spirit’s leadings weren’t taken up).
It is my feeling that, at least for a faith like that of Quakers, the bazaar is the better path. Even those faith traditions that are more in line with the cathedral often integrate elements of the bazaar, encouraging their lay members to develop their understanding towards that of the learned leaders of their communities that everyone might be able to judge rightly for themselves without having to lean on the guidance of their leaders at every turn. Some faiths even have a long tradition of teaching competing views on a subject without necessarily declaring that one of them is certainly right – or even if one is right, why people might consider that the others were right. This isn’t a binary choice, but rather a set of characteristics each of which might be more in the cathedral mould, or more in that of the bazaar. But to see what the strength of the bazaar might be, let us re-frame some of the principles set out by Raymond, taking them from their original context of software development and into that of participatory religion instead. I will not cover all 19 of these principles, because not all translate between contexts readily – though if you care to read the essay, and learn about all the principles in their original context, I suspect that you will find ways that each one of them can make sense in the context of religion.
The first principle is every good work of software starts by scratching a developer’s personal itch. As mentioned above, the idea is that each new development – be it a new project or an adjustment to an existing one – stems from a programmer who wants to do something but they can’t. A great deal of software doesn’t come from this, but rather from a profit making operation deciding that a particular product will make money. Raymond effectively asserts that these are generally inferior to those that come from a programmer wanting to solve a problem that they are experiencing. A parallel to this in religion, especially a religion based on direct inspiration as taught by Quakers, is that each of us can expect inspiration in matters relevant to our own situation. Where someone feels they have some inspiration about some matter that does not actually have much bearing on them, that inspiration is less likely to be good, appropriately nuanced, and worthy of regard than inspiration that comes through someone who is actually impacted by that matter. For instance, religious inspiration about things that affect women will be most reliable from women, all other things being equal. That is not to say inspiration that comes to a man but is about women is of no value, but perhaps it is of more value to other men than it is to women.
Good programmers know what to write. Great ones know what to rewrite (and reuse). A great example of this is in the original creation of the Linux kernel. Torvalds started with a working – but not satisfactory for his purpose – UNIX-like kernel in the form of MINIX. Some of it was reused for some time, and eventually all of it was rewritten or supplanted. There is no need to reinvent the wheel every time you need to push something around, and existing work can serve as a framework for developing new work. In the same way, when one of us has found an excellent way to put some religious insight, there is no need to reiterate in different words – right up until the time that there is such a need. That need might arise because of a change in the audience, to make the message more accessible. It might arise because of a change in social context that means the message itself has to change is some substantial way. An effective bazaar-like operation means that unnecessary work is not done, but nor are old things held on to just because of their age and venerability. There is no last word, not on any subject.
Release early. Release often. And listen to your customers. ‘Customers’ isn’t quite the right word, even for a lot of the open-source software world. ‘Users’ is more appropriate in that milieu nowadays, and in the case of our Religious Society, it might be ‘community’. The overall point of this, in the context of software development, is to avoid working in ‘splendid isolation’. If you’re spending time on something you’re doing for your own local religious community, or even a national or international community, involve people in the work. Let people who want to help, help, even if they haven’t been discerned and/or appointed as people who should be doing the work – if you’ve been so selected, then you have responsibility, but that doesn’t mean no-one else can contribute.
Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone. Sometimes phrased as “to sufficient eyes, all bugs are shallow”, meaning that if enough people look at a coding problem, someone (or some combination of someones) will quickly find an apparently obvious solution that a smaller group might never see. This takes a little unpacking. It shouldn’t be translated to “involve everyone you can in every decision”; that way lies madness, and immobility in decision-making. But everyone has different backgrounds, aptitudes, ways of looking at things. If you can leverage that, then things become easier. As individuals doing work or shouldering responsibility, or as a group working through challenges, we need to find ways to leverage our different aptitudes and different perspectives.
If you treat your beta-testers as if they're your most valuable resource, they will respond by becoming your most valuable resource. Don’t resent people who point out problems. They might point them out in ways that are more helpful or less helpful, that are more polite or perhaps outright rude. However they point them out, they are still letting you know about a problem that, for all they know, you haven’t realised is there. They might turn out to be wrong about it being a problem, but would you prefer that people who see possible problems keep them to themselves because you’ll be upset with them? Even if you’re only upset with people who turn out to be wrong, people will be reluctant to point out problems unless they are absolutely sure about them – so a lot of real problems will never be pointed out. Treasure the people who are willing to stand in the face of a popular idea and explain why it might not work.
Finally, there are a few that are more obvious truisms that take little translation or explanation. If you have the right attitude, interesting problems will find you. Note that here ‘problems’ doesn’t refer to difficulties with what you’re trying to do, but rather some job that needs doing which is a ‘problem to be solved’. Sometimes they find you through human agency – people come to you with the problems. Sometimes they find you by you stumbling on them. The point is that the right frame of mind and skills mean that you don’t have to look for things to solve or work to do. It comes without effort. When you lose interest in a program, your last duty to it is to hand it off to a competent successor. People who lose interest in a problem, or in some work, aren’t going to put the same time and energy into it. Before your loss of interest leads to extra complications for your eventual successor, make sure you let go of the work in a tidy way with a good handover. The next best thing to having good ideas is recognizing good ideas from your users. Sometimes the latter is better. Just because it’s ‘your’ work, doesn’t mean all the ideas that go into it have to come from you. The wider community are your users, your audience. They have knowledge about how things actually work on the ground that you don’t, if only because you’re only one person. Often, the most striking and innovative solutions come from realizing that your concept of the problem was wrong. Perhaps the most fundamental one to remember, being rooted in a sense of humility. This one doesn’t even need any translation. It’s ready to go, prêt-à-porter for a huge range of contexts, from programming to religion and beyond. Don’t assume that you’re wrong in your initial concept, but be ready to realise that you are.
I hope that this has given you all food for thought, and if you’re reading this without being involved in any religious community, I hope that seeing how these ideas translate to that context might help you think about how they could translate to other contexts as well. What ideas have you learned in one context that have helped you in another, be it religion or anything else? I’d love to get your thoughts in the comments.
~ ~ ~ ~ ~
Did you enjoy this post, or find it interesting, informative or stimulating? Do you want to keep seeing more of these posts? Please consider contributing to my Patreon. More information is available in the post announcing my use of Patreon.
If you enjoy this blog, or otherwise find it worthwhile, please consider contributing to my Patreon. More information about this, and the chance to comment, can be found in the post announcing the launch of my Patreon.