Collision Course: Quality Assurance and the Dollar

by Glen Emerson Morris

There's a telling scene in the movie "The Pirates of Silicon Valley," between Steve Jobs and Bill Gates, which may or may not have really happened, but rings very true nevertheless. Jobs tells Gates "We build a better computer," and Gates counters, "You don't get it. That doesn't matter."

To a degree, Gates was correct. Gates, and much of the personal computer software industry, understood early in the game that it was possible to sell software with a lot of problems, or bugs, because the decision to buy most business software is not made by the people who actually use it. The additional costs of lost work, time, and aggravation that buggy software causes are rarely considered by the managers who actually buy software, in large part because these problems are hard to assign a dollar value to. However, in the world of E-commerce, software failures can cause immediate and extensive monetary damages, easily calculated to the decimal point (like the spring 1999 crashes of E-bay, and the subsequent decline of its stock value.) E-commerce is a completely new kind of market for the personal computer industry, and in pursuing it the industry may have set itself on a collision course with the last thing it wants to meet, financial accountability for the problems caused by its software.

Two pieces of legislation give a good idea of how the software industry is adjusting to new market demands for quality. The software industry is lobbying for a revision to the Uniform Commercial Code, which currently limits the software industry's ability to avoid liability. A notice at the beginning of most software manuals declares the software is sold "as is" and the manufacturer is not liable for any damages resulting from its use. This disclaimer is currently voided by the Uniform Commercial Code, but won't be, if legislation the software industry sponsored passes. Essentially, the software industry would like to legally exempt itself from any possible damages, no matter how responsible for the damage they may have been.

In the case of the Year 2000 (Y2K) problem, which may have been the most avoidable problem in the history of software development, the industry successfully lobbied to limit damages businesses may seek against software developers because of Y2k issues. It was a sweetheart deal, which protected the software industry from its own mistakes, and gave business users nothing in return.

An obvious question is why the software industry would rather spend millions on campaign contributions to rewrite liability laws, rather than make software that worked in the first place. The answer is a combination of bad habits and a serious miscalculation of the true cost of this approach.

Given the complexity of modern programs, which frequently have hundreds of thousands of lines of code, programming mistakes are nearly impossible to avoid. The safety net most companies use is a quality assurance department, whose job is to create and execute thorough and formal tests of all products before they are marketed.

Quality assurance departments have never been particularly popular at software companies; their immediate costs to a company are obvious, but their return on investment as always been difficult to quantify in dollars. To their credit, most software development companies have quality assurance departments these days. The quality problems usually come from the conditions the QA department has to work under.

At the established software companies, the main source of quality issues is usually the marketing department. All too often, the marketing department demands that new and unplanned for features be added to a product even after development is well underway. This barely leaves time to finish the product, let alone test it. Even if the ship date is postponed, there is still rarely enough time to implement the changes without long hours, and a far greater risk of serious problems. Building software is a lot like building houses; once the foundation has been laid, it becomes very difficult to change the basic architecture.

While the established software companies have mixed feelings about QA policy, the numerous startup companies now courting the E-commerce market with new products tend to give it even less thought. This is unfortunate, because startup companies have all the QA problems of a larger company, plus an additional set of QA problems only startups have, namely startup founders. The same personal qualities that make startup founders succeed, unwavering belief in their product, a strong drive to do it their way, the ability to ignore criticism and questions about the wisdom or viability of their product, also make them unreceptive to the needs and warnings of their QA departments. It's not surprising QA departments at startups tend to be under staffed under equipped, and over worked.

In reality, a majority of QA testing for many software products is actually done by the product's buyers under a system based on "maintenance releases," or essentially updated versions of the software which fix bugs found, or at least addressed, after the product shipped. This system allows marketing departments to ship products with known, and unknown, problems under the rationale that as long as the known problems aren't too serious, they can be fixed later, namely after some cash has come in to pay for the fixes. This system has allowed software developers to pass on a substantial percentage of QA costs to software buyers, and it's worked, largely because buyers couldn't really tell how much it was costing them. Now that the PC software industry is applying its "if it boots, ship it" QA mentality to developing financial transaction software, the costs are becoming all too clear.

It's not a problem that can be solved by campaign contributions. In fact, the very success of the software industry in Washington may increase its chances for long term disaster. The catch is that the commercial software industry doesn't have a monopoly on business software anymore, as the success of the public domain operating system Linux is proving. It's only a matter of time before reliable public domain, and open source, commercial transaction software becomes available, too, and if it's reliable enough, it could eventually dominate the market.

To avoid a major collision of values, if not liability, the software industry needs to understand that different grades of software quality are required in the business world. Some kinds of software simply must work reliably, always. Unreliable accounting software is a contradiction of terms. Unreliable e-commerce software sounds nearly as expensive. It may be OK if a word processor drops a few bits now and then, but with e-commerce software, it's not just bits that are dropped, it's dollars, and that's a different matter altogether.