When testing hi-fi equipment, we use two basic concepts. One is, obviously, listening to the type of thing you will listen to once you buy it. The other is a more technical system of frequency response and what not. Sound pulses, walking tones and the like are used to discover weaknesses which may go unnoticed for a long time. But for audio compression (as long as it's a lossy compression) the last method does not work. The reason is quite simple. The compression is designed from the start to take into account certain weaknesses in the human ear when listening to certain types of sounds (music in our case). These compression schemes tend to fall apart when used for other types of sounds. This is a necessary trade-off to achieve decent levels of compression. This is also the reason for specialist compression schemes for voice data. These will never be good for storing music. The technical tests may still be used to learn more about the compression scheme itself but that is beyond the scope of this comparison, which will stick to the theme of recording music on your computer for personal listening pleasure.

Testing audio compression schemes is largely subjective. Thus, some people will perceive certain compression losses more than others. To do the type of exhaustive testing done in studio conditions by the various bodies governing audio compression is beyond us, so we have employed environments much like any that would be found by most computer users. I am confident this will be sufficient to decide on a compression scheme for day to day use. The listening test was done on a blind test basis (the listeners are not aware of what the type of compression used, if any, in what they are listening to) and the listeners comprised of myself and two others, one of whom is a gamer into modern pop, rap and R&B, the other an audiophile with a tendency towards the classical side of music. I personally like a wide range of music but mostly alternate rock and some metal. A Ripper program was used to do Digital Audio Extraction, resulting in a file which was then encoded. This is necessary so as not to lose any sound quality by going through the analogue to digital conversion and visa versa. To make sure we didn't know which compression scheme we were listening to, the compressed audio was played back by a colleague, with the monitor not facing us. We also ensured that the speakers were positioned according to their manufacturers recommendations.

A bit of initial research convinced us that we should use two different encoding levels for the comparison. The first, which we call the Normal Quality Level, is either the highest encoding quality the encoder provides or 160 kbps (for stereo), whichever comes first. The reason for this, over 128 kbps is that since we are NOT looking at trading files on the Internet, we were willing to sacrifice some storage space for the quality increase we gained (in our base compression technology, MP3). Actually, we have found that even the MP3 traders seem to realize this. Two encoders, TwinVQ and Real G2 maxed out before this and MS Audio 4 is at its maximum quality level at 160 kbps. The other level we call the High Quality Level. This is for people who have very good speaker systems hooked to their computers or actually pass the sound card output to a hi-fi. We set 192 kbps as a baseline here, though we did allow Xing Technologies Variable Bit Rate encoder at it's highest quality level to be in this category since it produces files which average out (on a per CD basis) to 192 kbps. This is the recording category I use personally, because I do notice the quality difference between it and the Normal level.

The actual listening test was done by first listening to the original .WAV file, followed by the coded files. We could ask for repeat playback of both the original and the coded files (and if we wanted to, of certain sections in the case of album music). We had to grade the closeness of the music played with the original on a scale of 1 to 10. Keep in mind the original was copied and given a code and played along with the encoded music to test listener reliability. It must be noted that after the test we realized that it was often possible to identify the music encoded by the same compression scheme in many cases by listening closely to certain sections of a song, usually to see how some instrument sounds. Another important point was that the original was identified quite often, indicating that there is still much work to be done in this field.

Let's take a closer look at the hardware and software we used for the test.
 

Table of contents
Introduction	TwinVQ (VQF)
Basics of audio compression	Real Audio G2
Methods of comparison	Microsoft Audio 4
Software and hardware	MPEG 1 layer 2 (MP2)
Selection of music for the comparison	MPEG 1 layer 3 (MP3)
Index of results	Conclusion