This page provides a description of the procedures followed for development of the ABINIT package through collaboration of different groups of persons, based in different places in the world. Its content was formerly found in the Infos/Notes_for_coding/contributing.html text file.
Any comment or suggestion to improve these procedures will be welcome !
The ABINIT package is aimed at being used by different groups of people, without mandatory control by the main contributors of the ABINIT group. In the same way, the ABINIT development project is fundamentally open to the contributions of different persons, not located in Louvain-la-neuve or Corning. These contributing persons are members de facto of the ABINIT group. We will call them external developers.
People using the code might consider adding their personal subroutines, without trying to make them part of the official ABINIT package. However, this has two drawbacks for them : in subsequent versions, their modifications will not be incorporated, so that they might have to check and modify the interface for each new version ; moreover, their contribution is not tested by other users of the code, so that bugs might remain unnoticed. It is also nicer to share the result of their coding efforts with other users of the code.
Of course, a collaborative effort has also some drawbacks. In particular, the collaboration between distant developers should be carefully planned, since orthogonal modifications of the same piece of code by two different people at the same time is very likely to happen, generating "negative progress", i.e. a large waste of time when synchronization is to be done. Also, it is required to use a well-defined coding style, to provide test case files, and to comment the modifications and additions as much as possible, with the aim to facilitate the maintenance, and the future modifications.
This document aims at defining the protocol to be followed to avoid "negative progress" due to lack of synchronization. The analogy with the procedures to be used for the parallelization of a code is obvious. The aim is that each external 'node' does not waste its time, that communications are kept at the lowest level possible, and that the final result is correct ! We will need barriers for synchronization, and so on ...
The ability to incorporate the contributions of different groups in a harmonious way might become a noticeable strength of the ABINIT project.
Managing different versions of ABINIT is done thanks to a a tool calledGNU Arch. For an introduction to this useful tool, you can have a look at our dedicated pages in the Developers' corner of the web site.
Thanks to GNU Arch, the development of a project becomes completely transparent, since all the changes in the files are registered: the latest version is of course available, but one can come backwards in time to track bugs easily or to know what anybody did. The development of different branches is also managed, as well as the subsequent merging procedure. This feature is important to allow development by many different people. The place where all the files are stored, including their history, is called a repository.
For developers, using the repository is a privileged way of managing their contributions, since the coordinator is automatically and instantly informed of their progresses. ... description of the categories ... In each category and for each version, there is one reference branch, which is the backbone of the development effort. It All concerned developers are supposed to use this branch as a starting point for their tasks and to keep permanently in sync with it.
In the following, we will distinguish between "Debugging" contributions and "Development" contributions.
"Debugging" contributions are typically modifications of a few lines, in one or relatively few routines, needed for the code to work properly, or to be properly documented. Sometimes, they are related to comments within routines, or corrections to documents. Usually, such modifications do not need any synchronization and can be sent directly to the coordinator via email, in the form of a patch. It is however possible to have them synchronized by recording them inside the abinit-release category of the repository.
For the time being, Xavier <gonze@pcpm.ucl.ac.be> should be contacted.
"Development" contributions usually involve the addition of new capabilities to the code. Despite the use of GNU Arch, synchronization with the coordinator is ALWAYS needed : one has to make sure nobody else is already in charge of a similar project ! The development contributions might be quite localized (basically adding one routine, called by a few lines from an existing routine), or, on the contrary, involve modifications of many existing routines. Even for the localized type of modifications, discussion with the coordinator is mandatory. GNU Arch deals with conflicts only at a formal level, so that semantic conflicts must be avoided from the very beginning.
The external developer will be allocated a development "task". Related to this task, they will be free to code, experiment, debug, and check the result of their work without any communication with the rest of the ABINIT group. The allocation of the "task" must obviously be done in coordination with the rest of the group prior to the work, while the result of the development has to be incorporated into a new official version. The "task" is thus also limited in time.
The prior allocation and subsequent incorporation, taking into account the possibility that many different external developers work independently, must be done centrally, by one coordinator (at present, Xavier, but this might change in the future).
It will be the responsibility of the developer to make enough checks of the correctness of their modifications or additions. The developer should provide adequate documentation (basically the description of the input variables and output data in the abinis_help file, as well as a possible update of the bibliography). They should also provide one (maybe two) test(s) to be added to the standard suite of tests (for version 4, see Test_v4). This is needed to ensure that the transfer to the official version of the code has been done properly, and also that the new capabilities will be preserved by the subsequent modifications of the code. Finally, they have to show that their modifications have not suppressed existing capabilities of the codes, by running the set of already existing automatic tests.
It will be the responsibility of the coordinator to transfer the result of the development effort of each external developer (the content of their "zone") to an official version, in such a way that the test is reproduced in a satisfactory way. The subsequent maintenance will be automatically done by checking that the corresponding test is still working despite modifications.
a. The external developer proposes a modification or addition to the code to the coordinator. In principle, such a proposal will be welcomed ! There might be some discussion, involving also other scientists, to try to improve the original proposal. The developers mailing list has been set-up for that purpose (see the "Community section" on our web site).
b. With respect to the latest official version, the external developers define their task, and make a list of routines they wish to heavily modify (and those they would like to add, but this is not really needed at that time). Following the current coding practice, adding new input variables would need rather small modifications of a very large number of routines (invars, abinit, gstate, brdmin, move, scfcv, vtorho ...). In order to avoid the allocation of a too large a zone of development, which would make other development efforts impossible, it is assumed that the developer will make appropriate use of the different 'user' input variables. This appropriate use means that none of the aforementioned routines will be modified for that purpose. The easiest case to handle is when a routine or a set of routines are added to the package, and the interface to the rest of the code involves only a set of called subroutines whose interface (list of arguments) should not change, and just one (modified) calling routine. Many development projects of this type can coexist. By contrast, if one of the development efforts involves major changes to major routines, it will prohibit the execution of another development project of the same type, taking into account the current structure of the code.
c. Having gathered the different suggestions made on the basis of the current official version, the coordinator releases the next version, adequately prepared to allow the proposed developments by different external groups, with a list of routines allocated to each group, the number of the test cases allocated, and, for information, the tasks that will be done.
d. After installation on the development machine, and prior to any modification, the developer runs the internal tests, as well as the Test_fast, Test_v1, Test_v2, Test_v3, Test_paral (sequential cases) suite of tests. (The "make tests_dev" command is perfectly suited for that purpose). Checking against previous reference files that the results are OK would be nice. The developer can replace the reference files provided with the official version by those produced on their machines, to ease further checking of the results. Then, the development effort can begin. It is expected a priori that the following files will be modified in any case : - "object_list" files in some Src* directories (because the new routines must be listed there); - the keyhr.html and some of the var*.html files (because the new input variables must be listed and documented there - the 'user' variables are now used, but a final, aesthetic, name should be proposed); - the README and Run files in the subdirectory Test_v3, as well as an additional input file (at least).
In both cases, the developers MUST follow the current ABINIT coding style, presented in the latest version of the document 'rules_coding', in the ~abinit/doc/developers subdirectory. In particular, they should mention their initials in the header of the new or modified routines. This will be useful if somebody needs information about this routine some time later.
At the end of the development effort, it is mandatory that the developer runs again the Test_fast, Test_v*, Test_paral (sequential cases) tests, to be sure that the developments have not spoiled some other feature of the code. This is easily done by issuing "make tests_dev". This command will run automatically the required tests, and produce a file "summary_tests.tar.gz" that should be send with the updated routines and separate new tests. This is important, since the coordinator will have to run the suite of tests also, but at that time, having a trouble with a modification done by some developer would mean an important delay in the delivery of the new official version, and thus a large waste of time. It is desirable that the tests of the new feature do not last more than 1 minute on Pentium III 1GHz, or about 20 seconds on other (faster) processors.
e. After the development has occurred, the developer prepares a gzipped (compressed) tar file with all the needed files (additional routines, modified routines, makemake, abinis_help, README, Run ... ), the summary_tests.tar.gz file, and makes it available to the coordinator (by email, FTP or SSH).
a. David proposes a nice addition to the ABINIT package, Doug and Xavier are enthusiastic about it.
b. On the basis of version 1.6.4, David proposes a list of routines that should be allocated to him.
c. Xavier delivers a version 1.7.x that has been adequately prepared for the independent development that David wants to do.
d. David implements his addition and modifications on the basis of version 1.7.x, checks whether the suite of tests is still OK, and make the files available to Xavier.
e. Xavier thanks David very much, transfers the work to the official version 1.8.y , and performs the final modification of the names of the (new) input variables including the propagation through the routines that were not allocated to the developper.