Not the same job writing code for an average smartphone, a PC, or something that will "fly-by-wire" an airliner or a space shuttle.
How do they test such programs ?
See also :
"(...) NASA space shuttle team achieved a 99% bug removal rate, but the cost is at 35 million per year for about 400k lines of code. "
"But how much work the software does is not what makes it remarkable. What makes it remarkable is how well the software works. This software never crashes. It never needs to be re-booted. This software is bug-free. It is perfect, as perfect as human beings have achieved. Consider these stats : the last three versions of the program — each 420,000 lines long-had just one error each. The last 11 versions of this software had a total of 17 errors. Commercial programs of equivalent complexity would have 5,000 errors."
"It's like pre-Sumerian civilization," says Brad Cox, who wrote the software for Steve Jobs NeXT computer and is a professor at George Mason University. "The way we build software is in the hunter-gatherer stage."
John Munson, a software engineer and professor of computer science at the University of Idaho, is not quite so generous. "Cave art," he says. "It's primitive. We supposedly teach computer science. There's no science here at all."
That's the culture: the on-board shuttle group produces grown-up software, and the way they do it is by being grown-ups. It may not be sexy, it may not be a coding ego-trip — but it is the future of software. When you're ready to take the next step — when you have to write perfect software instead of software that's just good enough — then it's time to grow up. (...) For one thing, 12 of the 22 people in the room are women, many of them senior managers or senior technical staff. The on-board shuttle group, with its stability and professionalism, seems particularly appealing to women programmers.
The whole approach to developing software is intentionally designed not to rely on any particular person.
And the culture is equally intolerant of creativity, the individual coding flourishes and styles that are the signature of the all-night software world. (...) you can't have people freelancing their way through software code that flies a spaceship, and then, with peoples lives depending on it, try to patch it once its in orbit.
"Most people choose to spend their money at the wrong end of the process," says Munson. "In the modern software environment, 80% of the cost of the software is spent after the software is written the first time — they don't get it right the first time, so they spend time flogging it. In shuttle, they do it right the first time. And they don't change the software without changing the blueprint. That's why their software is so perfect."
Like computer models predicting the weather, the coding models predict how many errors the group should make in writing each new version of the software. True to form, if the coders and testers find too few errors, everyone works the process until reality and the predictions match.
4. Don't just fix the mistakes — fix whatever permitted the mistake in the first place.
==> this is the "Don't fix bugs, fix your process" attitude (read from the SQLite devs)
NB : A version of this article appeared in the Dec 1996/Jan 1997 issue of Fast Company magazine.