Beta testing - how it works

When developing a computer program, the people creating the software try their best, to keep annoyances as far away from their users as possible. At least if they have a good user experience in mind. There are huge companies that manage well, however my experience is that you find the biggest enthusiast among the independent software developers. Usually these people write solutions for common problems that are not, or have been not available from the shelf. But regardless if it is a major company or the one man home based tinkerer. They all rely on user feedback. And here is how it works.

The program is written and run through inhouse tests. These test are human tests and ideally so called unit tests, which are highly automated. Basically the program uses itself and writes error messages to a log file if it encounters any. In good software design these test are handled by some Quality Assurance guy. It really is a good idea to have the program tested by someone who has not written it, because intuitively a programmer will avoid behaviour the causes the program to bitch one him. If it does, he fixes what he sees. However there are ways a program can be used that the programmer has not thought of before. (Hand a bunch of impatient school kids a game and see what happens), so if a programmer has the chance to let other people test his stuff, it is a wise idea to use that help.

After the inhouse tests have been passed, the so called beta test phase starts. Usually copies of the program are handed out to people the creators know and trust. People that have provided valuable feedback before. In the case of the independent software developer it usually starts out with fellow developers, friends and family.

If the developer is lucky, he has a loyal userbase of other programs he has done. Phase 2 of a successful beta test lets these people access the test versions. Larger companies usually set up a bug database where registred users can post messages describing the problem through a web interface.

Phase 3 (even though it is often skipped due to time pressure) is running a completely open beta test. Once the obvious quirks are ironed out it is a good idea to have the program tested by a wide audience. This step is important to polish the shipping version and usually reveals one or two major flaws that have been overlooked beforehand. It makes sense to have a program tested on as many different computers running as many operating systems as possible, or support only a subset of the machines and OS that are on the market.

All 3 phases have something in common. They all rely on the testers giving feedback. So le me take the chance to thank all the people that helped us through the phases where coders run around like headless chickens, QAs are banging heads on desks and marketing would love to rush it all. You people really help us to do a better job!

Thanks.