"Arny Krueger" wrote in message
news

But the same human imagination is engaged in any listening test. A lot of
setting up a listening test is ensuring that whatever results you get are
actually relevant to what you are changing.


When I did subjective listening tests for a living (OK it was telephone
transmission equipment, such as low-bit rate codecs, rather than Hi-Fi
equipment, but the principle stands) none of our listeners was allowed even
to know the purpose of the test. In a "HiFi" analogy might be that in a
listening test to compare CD players the listeners would not know that it
was a test of CD players rather than amplifiers, cables, or speakers let
alone which players were being tested. This rule was rigorously enforced.
Also there were at least 24 listeners per test (tested individually with no
opportunity to compare notes), who all got the various "conditions" in a
different order (to eliminate precedence effects). Even though some of these
conditions might actually be identical they could still get different scores
from some listeners. Volume levels were strictly controlled to ensure that
minor level differences didn't skew the results.

Unless these sorts of precautions are taken listening tests are valueless.

David.