View Single Post
  #9 (permalink)  
Old January 7th 05, 04:01 PM posted to uk.rec.audio
Jim Lesurf
external usenet poster
 
Posts: 3,051
Default DBT a flawed method for evaluating Hi-Fi ?

In article , Richard Wall
wrote:

I would have thought that most manufacturers would work to make their
systems sound different


Some may wish a specific 'sound'. Others may not. Although it depends what
you mean. It also depends if you are talking about amps, speakers, or other
types of equipment.

and the vast array of suppliers would surely not exist if they all
sounded the same ?


Depends, There are also a variety of required conditions of use, and prices
people are willing to pay.

It should easily be possible to take three similar amplifiers, modify
their frequency response at above the limit of hearing a difference and
then get a conclusive proof of the effectiveness of the testing method.
Has it been done ?


I'm not clear from what you write above what hypothesis you are wishing to
test.


It may be easy enough to make all kit sound the same,


See above. Speakers and listening rooms can vary quite a lot. So can the
requirements of listeners. My experience is that compared with this the
differences between many amplifiers are quite small (or effectively
undetectable in use).


but as it is unlikely that even if they did all make the prefect
reproduction equipment, everyone would prefer it.


Only if they all have the same room acoustics, same requirements, etc.

I like quad ESLs. But that does not mean I expect them to suit everyone.
This isn't just a matter of what features of the resulting sound matter to
me. The room also matters, as does the taste in music.

I think all the systems belonging to members in our club sound quite
distinct, it would however be impossible to test this opininion. You
have also missed my point that although the testing method is proven the
results on Hi-Fi are not.


Afraid I am not clear what you mean by either of the above statements.

Hearing is not an absolute and will change over the listening
period.


Agreed.

Trying to listen for differences is not the same as listening
to appreciate music.


Agreed.


Unless you have data to prove otherwise the sample sets used to evaluate
Hi-Fi are small so surely with false positives, false negatives and
perception changes is it any wonder that the statistics say that most
things cannot be differentiated.


The method of a controlled ABX type of test is to allow the listener to
decide for themself when they think they can identify which choice is 'X'.
And to do this in a manner that aids us in drawing statistical results from
repeated sets of such tests. The idea is that if A and B can't really be
told apart under the conditions of the test, then the resulting 'choices'
will fail to correlate with the real identity to a level that shows
statistical significance.

However such tests only can provide results that are directly relevant for
the conditions under which the tests were performed. And our ability to use
the results for other purposes will depend on how reliably they fit a given
situation. The results can also only be regarded as useful if the tests are
performed often enough in well-enough controlled circumstances to give the
resulting data statistical significance to the required level.

The problem is this is quite a demanding and time consuming process. As you
say, it ends up being quite different to sitting down and enjoying some
music. However in my personal view, if differences are so slight as to
require extended tests like this to show up, then perhaps they don't really
matter much in practice. :-)

FWIW although I have often heard differences (or at least *thought* I
have!) between various items, I often then am not bothered if they are
small compared with choice of speakers, position of speakers, etc. Thus to
a large extent worrying about something like choice of caps tends to end up
seeming to me like a waste of effort once the effect is swamped by a slight
movement of the head. I'd rather spend the time enjoying music on my
imperfect audio systems. :-)

Slainte,

Jim

--
Electronics http://www.st-and.ac.uk/~www_pa/Scot...o/electron.htm
Audio Misc http://www.st-and.demon.co.uk/AudioMisc/index.html
Armstrong Audio http://www.st-and.demon.co.uk/Audio/armstrong.html
Barbirolli Soc. http://www.st-and.demon.co.uk/JBSoc/JBSoc.html