Joel created the “Joel Test,” which is a measure of how good a software team is. (Attaching one’s own name to a set of rules is a great PR move.) The test is simple. Just answer yes or no to each question.
- Do you use source control?
- Can you make a build in one step?
- Do you make daily builds?
- Do you have a bug database?
- Do you fix bugs before writing new code?
- Do you have an up-to-date schedule?
- Do you have a spec?
- Do programmers have quiet working conditions?
- Do you use the best tools money can buy?
- Do you have testers?
- Do new candidates write code during their interview?
- Do you do hallway usability testing?
Joel states that a score of 10 or lower indicates serious development problems and that most software organizations run at a score of 2 or 3. Microsoft, he claims, runs at a score of 12, which seems to be consistent with my experience, except maybe the last point. However, I have to think that #12 “hallway usability testing” is not a very common practice and #9 “best tools money can buy” is perhaps too extreme, so I would guess that ten would actually be a very good score.
This test is frequently cited, yet some large successful projects don’t actually score well with this measure. Consider Linux:
Andrew Morton, Linux kernel maintainer, indicated in an interview at LINUX Format, that the Linux operating system had been developed without source control:
LXF: Do you think it was good to have had the time with BitKeeper in kernel development, or should they have stuck with CVS?
AM: Well, we never even used CVS. Before BitKeeper we basically weren’t using anything - just a bunch of patches sitting on Linus’ hard drive and it uploaded occasionally. We had no tracking of what had gone in the kernel at all. Which I always thought was pretty poor form; I wish Linus had used CVS from day one. He hates CVS, he has real problems with it, but for just a simple linear evolving model I think CVS would have done the job and at least we would have had some tracking history.
I think the main thing BitKeeper brought is it got Linus using something, and even if we used one of the free revision control systems I think we would have been 90% of the way there…
A post last month in Slashdot reported that “Linus Says No to ‘Specs’.”
A ‘spec’ is close to useless. I have _never_ seen a spec that was both big enough to be useful _and_ accurate. And I have seen _lots_ of total crap work that was based on specs. It’s _the_ single worst way to write software, because it by definition means that the software was written to match theory, not reality.”
Eric Raymond bluntly criticizes Linus and his undisciplined development process in this email:
Your tendency to undervalue modularization and code-sharing is one symptom. Another is your refusal to use systematic version-control or release-engineering practices. To you, these things seem mostly like overhead and a way of needlessly complicating your life. And so far, your strategy has worked; your natural if relatively undisciplined ability has proved more than equal to the problems you have set it. That success predisposes you to relatively sloppy tactics like splitting drivers before you ought to and using your inbox as a patch queue.
Lack of source control and specs are huge. I suspect a whole host of other violations of the Joel test as suggested by LinuxToday, yet magically we have a major operating system. Perhaps, despite its development practices, Linux benefited from the widespread need for an alternative, free operating systems to Windows.
I don’t have real world experience with extreme programming to know if that methodology eschews most of the points in Joel’s test. If so, it would be another example where a development team could be highly successful despite performing miserably on the Joel test. XP doesn’t do upfront design, so it may fail the specs test. Programmers work and communicate in pairs, so working conditions aren’t supposed to be quiet. XP emphasizes unit testing at the expense of independent testers.
