This is the third of a series of posts intended to help scientists take part in the open science revolution by pointing them towards effective tools, approaches, and technologies.
3. Find a Strong Community
New projects are often very exciting, and might show amazing promise. And using a brand-new free software project is far less dangerous than taking up a beta version of some proprietary tool. A proprietary tool might never make it to a formal release, and if the vendor decides not to follow through on early development then it might even become impossible to legally use it. That can’t happen to free software: the license guarantees that you will always be able to use, adapt, copy, and distribute it. However, you usually want more than that.
Free software is not exempt from a general rule of thumb in the industry: there is a high “infant mortality rate”, and a majority of projects never make it to “version 2.0”. A shiny new tool is likely to become dull and unloved, and may never get the vital upgrade to a new language version, or the module for compatibility with a new protocol, or the latest operating system. It is paradoxical but true: the useful life expectancy of a software product increases as the product matures.
Growing into maturity, a piece of software acquires a community of developers and users, with mailing lists, IRC channels, blogs, twitter feeds, and meetings – everything from informal online gatherings through pub meets up to international conferences. This community forms a complex ecosystem, and it’s this network of people, systems, companies, and technologies that gives longevity to the software:
- A diverse base of developers makes a project robust in event of loss of a key person, and helps to guard against unwise development choices.
- A critical mass of users ensures a rich flow of new requirements, serves as the source of new developers, and may often provide some financial and material support for development (from server hardware to conference sponsorship).
- An active community builds interfaces, plugins, and modules to connect a good project to a wide diversity of other systems, and these connections in turn allow the user base to expand.
This rich context is the sign of a mature software system, and until it has formed – which may take years from a tool’s first public announcement or release – you are taking a risk in using a project. Sooner or later you will want a new feature, or a bug fix, or compatibility with a new operating system or interface, or simply guidance through some obscure corner of the code, and at those times you will be glad of the community of fellow users. Without it, you will have to find your own way. Choosing free software ensures you are always free to do that: you can write a serial driver controlling your new lab instrument, or a visualisation tool to generate PDF charts directly from your analysis results. But if you are one of hundreds or thousands using the same tool, the chances are that someone will have done that work already. Or at least that you can find fellow-travellers to develop the code with you.
Free software communities are often very strong, and may survive for several decades. But they can atrophy and die. Potentially fatal problems include:
- Stalled development: if the stream of new releases dries up, users will start to drift away to competing tools which provide the new functionality they need. This sort of project stall can be due to various causes: key people moving on, loss of corporate involvement, developmental over-reach or a defect mire, or even office politics and personality clashes.
- Incompatible releases: If new releases break existing applications, the user base will become fragmented (as each user continues to use whichever old release works for them). Potential new users see this fragmentation and are either confused or simply put off.
- Unwise development direction: a core development team focused on their own ideas, to the exclusion of the community’s requirements, will gradually lose users to other projects. This can result from core developers using the development of the system itself for research or experimental purposes. For instance, the developers of a programming language implementation might well be interested in a cool idea about programming languages (e.g. a new concurrency model, or a new syntactic possibility) and use their language to explore that direction. But the users mostly don’t want a Cool New Language; they want the Same Old Language, but with new libraries, a faster compiler, better debugger, etc.
So these are all things to consider when checking for a healthy community: are there frequent new releases of software? Are they backward-compatible, and is that a priority of the developers? Is the process for steering development flexible and responsive to community input?
Open-source software communities are great for productive conversations with colleagues and fellow researchers. Because they bring together users with a shared interest in the software, but who might work in different fields, you may well find grounds, not just for collective software development, but for research directions, and even collaborations. So once you have found a strong and healthy community, be sure to get involved, engage in discussions, and go to meetings. It is often highly rewarding.