Thu, 04. July 2013 – 20:26
Having worked on LOFAR for quite a number of years in a rather visible position, it is little of a surprise that once in a while I am still receiving emails concerning the LOFAR User Software (LUS). The latest email exchange – which was was an inquiry about the PyBDSM/Anaamika source finding package – resulted in me actually having a look at some of the recent activities in LUS land: as it turns quite a bit of the information on the LOFAR Wiki is rather outdated. In order to provide some useful feedback, I updated my local working copies from the central Subversion repository hosted at ASTRON and started searching through the commit messages. As it turns out, PyBDSM was removed from the LUS collection/repository more than a year ago, without this being reflected in any shape or form in the documentation hosted on the LOFAR Wiki:
Date: Fri May 11 2012 09:36:49 GMT+0200 (CEST)
Subject: Removal of PyBDSM (moved to the LOFAR repository).
After relaying this information back to Bonn, I decided that it was about time to provide people with an interest in the LUS collection with an additional route to obtain the software: remembering the struggle it took to even move from CVS to Subversion it is of little surprise that a tool/platform combination such as Git/Github never was adopted officially (though I would assume that most of the developers locally are running their changes through Git) by ASTRON. While smaller and more dynamic project teams (such as AARTFAAC) have been embracing elements of social coding, I am not expecting any sort of official Git support for any of the LOFAR user software within the foreseeable future. But since the software itself is still open, as is the case as well for the original Subversion repository, it is not such a big deal to actually create a Git clone:
git svn clone -s http://usg.lofar.org/svn/code lofarsoft
The main ingredient when starting out on this though is an ample amount of time. Actually this is less critical than it may sound, because once started the only thing to ensure is that the computer is left running and with network connection. Once the cloning process was completed, the creation of a repository on Github went rather quickly – so here it is, the address of the Git mirror repository:
Something I noted – or better: was made aware of – is the fact that there are quite a few large binary chunks embedded into the LOFAR User Software: though I can guess the motivation for dumping complete snapshots of external software packages into the repository, I never considered this a very smart way. Since the idea to modularize the software collection and break it up into smaller pieces – which then dynamically could be re-assembled – was never put in action, the footprint of a local working copy is considerable.
Counting objects: 182837, done. Delta compression using up to 4 threads. Compressing objects: 100% (75472/75472), done. Writing objects: 100% (182837/182837), 1.31 GiB | 605.00 KiB/s, done. Total 182837 (delta 117555), reused 169109 (delta 104045) remote: Error code: 31cb6b0a91a911e2d3bd2b73fe7f34bb remote: warning: Large files detected. remote: warning: File src/Pulsar/external/tempo2/tempo2-2012.6.1.tar.gz is 59.77 MB; this is larger than GitHub's recommended maximum file size of 50 MB remote: warning: File src/Pulsar/scripts/lambda_mollweide_haslam408_dsds.fits is 64.01 MB; this is larger than GitHub's recommended maximum file size of 50 MB remote: warning: File src/Pulsar/external/tempo2/tempo2-2012.6.1.tar.gz is 59.76 MB; this is larger than GitHub's recommended maximum file size of 50 MB remote: warning: File src/Pulsar/external/tempo2/tempo2-2012.6.1.tar.gz is 59.73 MB; this is larger than GitHub's recommended maximum file size of 50 MB remote: warning: File src/CR-Tools/data/LOPES-CalTable/AntennaGainFaktor/table.f0i is 75.34 MB; this is larger than GitHub's recommended maximum file size of 50 MB remote: warning: File data/lopes/LOPES-CalTable/AntennaGainFaktor/table.f0i is 75.34 MB; this is larger than GitHub's recommended maximum file size of 50 MB remote: warning: File data/lopes/LOPES-CalTable/AntennaGainFaktor/table.f0i is 65.20 MB; this is larger than GitHub's recommended maximum file size of 50 MB remote: warning: See http://git.io/iEPt8g for more information.
The good news after all now is: whatever is in the central Subversion repository, now also can be retrieved from Github, saving you the time and hassle to create a full clone yourself. On the downside though it is worth mentioning, that the central repository no longer hosts a number of projects, which have been moved to different locations.