"David Looser" wrote in
message
"Arny Krueger" wrote in message
news

But the same human imagination is engaged in any
listening test. A lot of setting up a listening test is
ensuring that whatever results you get are actually
relevant to what you are changing.

When I did subjective listening tests for a living (OK it
was telephone transmission equipment, such as low-bit
rate codecs, rather than Hi-Fi equipment, but the
principle stands) none of our listeners was allowed even
to know the purpose of the test. In a "HiFi" analogy
might be that in a listening test to compare CD players
the listeners would not know that it was a test of CD
players rather than amplifiers, cables, or speakers let
alone which players were being tested.

I've heard about tests like this. It's probably not a bad idea to conceal
the nature of the test as it surely leaves even less to doubt.

This rule was
rigorously enforced. Also there were at least 24
listeners per test (tested individually with no
opportunity to compare notes), who all got the various
"conditions" in a different order (to eliminate
precedence effects). Even though some of these conditions
might actually be identical they could still get
different scores from some listeners. Volume levels were
strictly controlled to ensure that minor level
differences didn't skew the results.

Sounds like pretty good tests.

Unless these sorts of precautions are taken listening
tests are valueless.

That might be an exagerration.

Some of the better tests we did involved about the same number of people.
The purpose of the test was known to all. Levels were matched within 0.1 dB.
Listeners were not allowed to compare notes. Scores varied because after
all, different listeners are different.