Peter

One thing I would like to know from the foundation that created the manifesto is which articles of publication, development or software have met the standards of the manifesto and which have not. That could be a very telling list and set an interesting competition among publishers.

I have been myself pressing in a modest way for full code (and data) used in published climate science papers to be made freely available. I have just made a submission to a UK Parliament science committee inquiry into peer review, covering this issue in the context of the limits of peer review. (See http://www.parliament.uk/business/committees/committees-a-z/commons-select/science-and-technology-committee/news/110127-new-inquiry—peer-review/ – you might want to make a submission on behalf of the CCF.

In case you are not already aware of a recently published climate science paper that I co-authored and for which complete turnkey code was archived at the time of publication, may I draw your attention to it? The paper concerned is “Improved methods for PCA-based reconstructions: case study using the Steig et al. (2009) Antarctic temperature reconstruction” by Ryan O’Donnell, Nicholas Lewis, Steve McIntyre and Jeff Condon, published by Journal of Climate in December 2010 (Early Online Release version). The code (written by Ryan) is available at http://www.climateaudit.info/data/odonnell/ along with the required data. As it happens, our paper criticised a paper published in Nature, in respect of which some but not all the code used was archived.

Although you are concentrating on code availability, please do not lose sight of the importance also of all data used being archived and made publicly available. In many cases, climate science papers use data that is not publicly available (and/or only available if one knew exactly where to find it), making the studies impossible to replicate or to investigate sensitivities to particular elements of the data even where full code is provided. And even where data is publicly available, it is often subsequently revised, again making replication impossible if the actual data used has not been publicly archived.

The code of Nature: Making authors part with their programs

This has a number of positive effects, including visibility of the code for validation and debugging purposes, availability of the code for use by others and/or for extension and expansion by others. If it also embarrasses the writers of the code into making the code cleaner and better tested, so much the better.

Many problems would have been avoided if publication of code in climate science was mandatory. We could then focus on the implications of the research rather than arguing about whether the analysis in the research was or was not worthless.