Beecher's Free Software Miscellany2009-01-07T04:30:57ZKarl BeecherCopyright 2009, Karl BeecherSPHPBLOGIs "Warning Density" an Interesting Measure? In doing so, we've used a relatively simple measure to profile best practice violations: warning density.
There is a colossal smorgasbord of warnings to choose from. Just printing the information on warning options from the G++ man page takes 22 pages. So, in one approach, we've requested a relatively small set of warnings, namely the warnings from Scott Meyers's Effective C++ book (the so-called -Weffc++ option). To illustrate it, let's look at results of one experiment:
These are the number of "Weffc++" warnings per thousand lines of code in KDE at various snapshots in its past.
I anticipated that warning density would be theoretically and experimentally very similar to defect density, but there are some critical differences:
With defect density it is always a concern that you have not discovered most of the defects in the code -- indeed software with close to zero defect density might just mean that the software has not been thoroughly debugged. Conversely, warnings are trivially and automatically discoverable.
However, warnings (I would say) are weaker indicators of software quality -- after all, which would you rather know about: potential dodgy practices or actual defects that effect how the software runs?
At the top system level there seems to be a direct correlation, like defect density, between warning density and size, so you might expect very small modules to have very small warning densities. But if you look at smaller subsections, that correlation weakens so much as to practically disappear. For example:
kdelibs (with about 570,000 LOC) has a warning density of ~2 warnings/kSLOC
kdetoys (about 10,000 LOC) has a density of ~15 warnings/kSLOC
kdepim (about 550,000 LOC) has a density of ~9 warnings/kSLOC.
And, most interestingly to me, I suspect that resultant warnings can be effected strongly by things like established coding practices. It's easy for organized software projects to establish and enforce a policy, for example, that says "all destructors should be virtual in base classes". In doing so, you reduce the possibility of destructor-related defects and reduce the "Weffc++" warning density.
But is all this "warning density" jazz interesting?]]>http://www.rodders.org.uk/index.php?entry=entry080623-0800522008-06-23T00:00:00Z2008-06-23T00:00:00ZShine a Light with DoxygenDoxygen so there would be little need for me to recommend it here. Still, you may not have heard about it, so it cannot hurt to give it a little championing.
Doxygen is a source code documenting tool somewhat like Javadoc. Like it's Java-based relative, it can pick out specially formatted comments from the source code that the programmer provided, and package them all up into a collection of HTML files. The idea being to make comprehending the overall design of the software system easier than trailing through the source code itself. But Doxygen also offers a number of extra features that make it quite a nifty tool, such as automatically drawing graphs (e.g. call graphs, inheritance diagrams), or producing LaTeX documents.
In fact, it's some of these extra features that make Doxygen useful in other ways...
As a researcher, I spend more time trying to comprehend existing code than actually writing it, and it's a hard thing to do. Whenever I'm having problems coming to terms with the layout of a software system, I instinctively reach into my little box of tools for Doxygen and run it on the software system. Even if Doxygen can find no specially formatted information within the software, it will still build a comprehensive package of documentation. It is also configurable in the extreme; whether you want to hide private methods or get call graphs drawn by the dot tool, all you have to do is edit relevant line of the config file.
Actually, that latter example of drawing graphs is a prime example of how Doxygen is useful for comprehension -- after all a picture paints a thousand words.
So often do I use Doxygen to help me, I've created a little Perl script that automatically edits the default config file so Doxygen gives me all I want (and turns off some of the time-consuming things I don't want). ]]>http://www.rodders.org.uk/index.php?entry=entry080528-0900092008-05-28T00:00:00Z2008-05-28T00:00:00ZGeneral Knowledge and How to Overcome It The main point I want to make is that at this stage of development in free software, dealing with a general audience, even a very smart general audience, means spelling out carefully and clearly what free software is.
I spent what I thought was enough time at the beginning of the presentation defining free/open source software before detailing some of our experiments. But when audience questions came around following the end of the presentation, they showed that I had not done enough to define the very concept of free software. I was asked:
How is this threatening the big software companies?
Notice the in-built presumption? The general audience it seems has no knowledge of the activities of corporations like IBM, Novell, and Sun, who distribute and provide services under free software terms. Further questions wanted to know how you could possibly make money from such a concept as free software. Again, the general audience knows not of companies like MySQL AB, Canonical, Red Hat, Trolltech, and Sirius Corporation.
But what was noticeable is that where I failed by defining free software in dry objective terms, the audience were elucidated somewhat by explaining the possible business models around free software. Business, it seems, is a more widely understood language.
There is a further related cautionary tale to tell: I was talking with a gentleman in a pub that same day about what I research. I used the marketing friendly term "open source" software to describe my interest, but he remained unsure and asked follow up questions to try to clarify his notion.
Is shareware open source?
Is Wikipedia open source?
Is open source another word for hacking?
Fair questions, but they surprised me as I had already established that the gentleman works "in IT". He knew exactly the meanings of "BIOS", ".Net", "Capgemini", and a dozen other instances of techno-jargon, but he really struggled with "open source".
The moral of this story? Don't even count on practitioners knowing one jot about free software.]]>http://www.rodders.org.uk/index.php?entry=entry080523-1400112008-05-23T00:00:00Z2008-05-23T00:00:00ZBletchley Park and Radar mythbustingBletchley Park, home of the Second World War code-breakers who built some of the earliest electronic computers. Once the workplace of Turing, and now a museum, I was going to say it's a geek Mecca, but after all this is a humble British place, and not everyone would make a point of visiting it at least once in their life.
How about the geek Westminster Abbey?
Anyway, among the wonderful displays and artefacts, including a working Colossus, one of the things that intrigued me the most was actually a presentation about Radar, that so surprised me that I just had to write about it. The surprise value is probably proportional to your integration into British culture, because if you hail from this sceptred isle you'll know that Radar was a British invention, developed just in time for the Battle of Britain, and was our secret weapon that the German's didn't know we possessed.
Except that's not true.
It was first patented by a German in 1904;
It's use at detecting aircraft reliably was first demonstrated in the early-mid 1930s;
Certainly the Germans, the Americans, the Soviets and the Dutch were using radar by the 1930s;
The Germans knew full well about British Radar systems -- after all, they bombed them plenty;
The Germans used their own radar systems to bomb targets on the British mainland (the Knickebein transmitters)
The Germans even had radar mounted on their planes.
And another amazing thing I learned was that the Enigma coding machine was actually a commercial product first launched in the 1920s! (For some reason I now have images of a modern soldier decoding a message using a code wheel from the "Secret of Monkey Island".)]]>http://www.rodders.org.uk/index.php?entry=entry080408-0900152008-04-08T00:00:00Z2008-04-08T00:00:00ZOff on a JollyCSMR conference on software Maintenance and Re-engineering next week.]]>http://www.rodders.org.uk/index.php?entry=entry080328-1200472008-03-28T00:00:00Z2008-03-28T00:00:00ZHow Much for the Free Stuff? (part 2) This is a summary of the results:
Minimum
LQ
Median
Mean
UQ
Max
$2890
$46,600
$241,000
$1,046,000
$1,278,000
$12,690,000
These statistics give us a picture of the value of the sample, but we must scale it up to give us an idea of the value embodied in the population (i.e. all 25,000 stable projects). This is tricky because the costs follow a non-normal distribution -- in fact, from the stem and leaf plot below (with one extreme outlier removed for readability), it appears to decay exponentially:
The decimal point is 6 digit(s) to the right of the |
This is interesting in itself: assuming the sample is a fair representation, it appears that around 75% of the value on SourceForge is held by the most costly 20% of projects. Only about 30% of projects are worth more the mean project value. (I am reminded of the 80-20 rule.)
The original question was how much value is on SourceForge: This is where things get a little fuzzy and a bit beyond my current grasp of statistics. There's clearly a lot of small projects worth comparably little; the modal class in this sample, without going too finely grained, is $0 - $50,000, with 11 members (1 in 5 which represents about 5,500 of the stable projects).
But with the information we have we can assume that the middle value of our sample (the median) represents the boundary of whether projects are in the upper or lower half of wealth. Hence, 12,500 projects are worth less than about $240,000 and 12,500 projects are above $240,000. I should really do more sampling to get a more accurate median, but if you multiply 12,500 by $240,000 (thereby underestimating the cost of the majority in the upper half of the population) you end up with $3,000,000,000. Furthermore, if 30% of all projects (7,500) are at least as high as the mean value, then we have a minimum value of $7,500,000,000, but let's be mindful of the risks of extreme values dragging up the mean.
Despite my scarily casual and hackish use of statistics, I'm quite sure that the software on SourceForge is "worth" plenty, certainly in the $1 billion order, most of it (perhaps unsurprisingly) most probably held by a relatively small percentage of projects.
And just think it's all given away by those benevolent hackers. Wonderful.]]>http://www.rodders.org.uk/index.php?entry=entry080319-0900002008-03-19T00:00:00Z2008-03-19T00:00:00ZHow Much for the Free Stuff?development costs of a Linux distribution I was made to wonder how much it would have cost to provide the free software that is available at a place like SourceForge. As may be observed from the many discussions about "total cost of ownership" Linux has dominated the debate about the cost of free software, but a Linux distribution is only one product, albeit a popular one that typically includes useful software from many domains.
But what about the rest of them?
I'm lazy, so I tried to re-use a basic version of Wheeler's approach, and use his splendid tool SLOCCount, that reports the estimated cost to develop software, by using Boehm's COCOMO model. As difficult as it must have been to design a serious study of Linux, trying to measure SourceForge has many additional problems all just waiting to leap on the validity of the study and destroy it (especially if you're just doing it for a bit of fun).
The first question is which software available on SourceForge should be considered here? Software is only worth something if somebody wants it; as a previous work has shown (here), there's a fair number of useless projects gathering digital dust on SourceForge (by virtue of their inability to sustain even a small number of releases), which, it is probably safe to say, have negligible value. It's difficult to identify these, since to my knowledge there's no way to filter projects according to release number, so instead I concentrated on projects that were marked as either stable or mature (trusting the judgement of the indivdual maintainers).
There are about 25,000 such projects on SourceForge, and like I said, I'm lazy.
But, son-of-a-gun, we just happen to have a pre-downloaded sample from SourceForge here at CROSS, fifty projects to be precise. If I assume the sample is representative, apply COCOMO to each project, take some average values and scale them up to 25,000, that will give us a value range.
You know, this post is getting quite big, I think I'll save the results for next time, as well as discussing the threats to validity (because there's plenty of those!).]]>http://www.rodders.org.uk/index.php?entry=entry080317-0900052008-03-17T00:00:00Z2008-03-17T00:00:00ZThe Debian Effect And the answer is a satisfying, resounding... maybe.
You see, we looked back over the history of each project from our Debian sample and divided it into two parts: The pre-inclusion era and the post-inclusion era, i.e. the history before and after the project was packaged up and included in Debian. So for each project we had two sets of both activity rates and contributors.
For all projects the number of contributors either grew or remained equal after entry into Debian, with about two-thirds of them growing; none of them reduced;
Activity rates only increased for approximately half of the projects
So to conclude, the advice on entering a project into Debian is: it couldn't hurt!]]>http://www.rodders.org.uk/index.php?entry=entry080312-1200002008-03-12T00:00:00Z2008-03-12T00:00:00ZFactors for SuccessCROSS that examine the effects of environment on a project. Specifically, how the environment in which a project is kept affects that project's success.
In this particular work, we take environment to mean the repository in which a project is maintained. The first step is to demonstrate that it has any effect at all, and to this end we carried out a straight comparison between two free software repositories that may be considered very different in software quality terms when taken as wholes: SourceForge and Debian.
Firstly, we decided on some process and product metrics that indicate success.
Number of contributing developers
Number of source code changes
Project duration
Project size (in our old friend, SLOC)
Next, we took a sample of projects from each repository (limiting our sampling to projects considered "stable" only), and measured each project's attributes. If the forge has an effect on project success we should see one set of projects consistently outperform the other...
And we do!
In fact, we took it further and performed a statistical significance test on all four comparisons, and found that three out of four project properties (excepting the project size) are improved significantly under Debian.
Which led us to wonder: is there some sort of "Debian Effect" that occurs -- i.e. reliably causing a project's success to increase -- if you take a project from somewhere like SourceForge and put it into Debian.
More on that next time... ]]>http://www.rodders.org.uk/index.php?entry=entry080310-1100002008-03-10T00:00:00Z2008-03-10T00:00:00ZOut With the Old...Paul Adams has been looking at the growth of languages of SourceForge and, predictably, I had to stick my oar in.
Of course it was noticed pretty quickly the sudden drop in number of projects at the end of the year. So far we've assumed that SourceForge has an "annual clearout" of old projects, but it would be nice to know the details, specifically what qualifies a project to be removed from SourceForge?]]>http://www.rodders.org.uk/index.php?entry=entry080308-0923062008-03-08T00:00:00Z2008-03-08T00:00:00Z
;)
;)