First let me point out that DBT in its various forms is a well proven
statistical evaluation technique for testing a wide range of diverse items.
For the vast majority it works extremely well however it is already
recognised that where the outcome of the test is not a clear yes/no answer
e.g. testing pharmaceuticals it is necessary to have both a large sample set
and three groups to measure against the so called placebo effect.
I believe that DBT is unsuitable for Hi-Fi evaluation as the detection
method (you the listener) are anticipating a change to occur at some point
and will be listening to changes not trying to enjoy music. Couple this to
the fact that what you hear is just an interpretation of the vibrations
received at the ear and the fact that sound will change as the listener/s
moves etc. It is no wonder that DBT has only given positive results for
major changes in sound.
Can anyone point to a Hi-Fi DBT test where no changes were actually made in
the source material to check that the audience could perceive no change ? I
think that the false positives raised by such a test would invalidate this
testing regime ? Seeing as though a proportion of contributors to this group
insist on such validation to prove differences it is very important that
they should first prove that their testing procedure is without flaws ?

With the poor conformation device of a person to identify changes in sound
the sort time interval switching proposed by some advocates must surely be
impossible to identify in a piece of music ? Music is far too complex to
expect anyone to correctly identify all but the most severe changes to be
correctly identified unless a long time period is set between changes ?
Should the test actually be made using white noise ? Using a long period
between changes is also flawed as audio "memory" of how a track sounded is
also unreliable.

Is there a solution ? For short term testing no. Some changes can be
detected electronically if the material remains in the digital domain but
once the sound has left the speakers there are too many variables. For our
Hi-Fi club we still use what for us is a less flawed method of a-b
comparison (preferably blind) on repetition of a range of tracks followed by
a longer term evaluation over the next few weeks.
I look forward to any proof that DBT for Hi-Fi has been validated.
Regards Richard Wall
New Ash Green Hi-Fi Club

On Thu, 6 Jan 2005 11:04:55 -0000, "Richard Wall"
wrote:

Can anyone point to a Hi-Fi DBT test where no changes were actually made in
the source material to check that the audience could perceive no change ?

That would be EVERY DBT test. Each sample is randomly selected from
the two possibilities - that means that in each test the source has
either changed or not. So false positives and false negatives both
show up as failures, and this is as it should be.

I think the reason you have problems with DBTs is not that they fail,
but that they pretty much always succeed. They succeed in showing that
there is in fact no sonic difference between the components under
test. Again, these days that is as it should be - we have advanced far
enough along the HI Fi road that pretty much all components should
sound the same.

d

Pearce Consulting
http://www.pearce.uk.com

"Don Pearce" wrote in message
...
On Thu, 6 Jan 2005 11:04:55 -0000, "Richard Wall"
wrote:

Can anyone point to a Hi-Fi DBT test where no changes were actually made
in
the source material to check that the audience could perceive no change ?

That would be EVERY DBT test. Each sample is randomly selected from
the two possibilities - that means that in each test the source has
either changed or not. So false positives and false negatives both
show up as failures, and this is as it should be.

I think the reason you have problems with DBTs is not that they fail,
but that they pretty much always succeed. They succeed in showing that
there is in fact no sonic difference between the components under
test. Again, these days that is as it should be - we have advanced far
enough along the HI Fi road that pretty much all components should
sound the same.
Why do you think they always suceed ? I would have thought that most
manufacturers would work to make their systems sound different and the vast
array of suppliers would surely not exist if they all sounded the same ? It
should easily be possible to take three similar amplifiers, modify their
frequency response at above the limit of hearing a difference and then get a
conclusive proof of the effectiveness of the testing method. Has it been
done ?
It may be easy enough to make all kit sound the same, but as it is unlikely
that even if they did all make the prefect reproduction equipment, everyone
would prefer it. I think all the systems belonging to members in our club
sound quite distinct, it would however be impossible to test this opininion.
You have also missed my point that although the testing method is proven the
results on Hi-Fi are not. Hearing is not an absolute and will change over
the listening period. Trying to listen for differences is not the same as
listening to appreciate music. Unless you have data to prove otherwise the
sample sets used to evaluate Hi-Fi are small so surely with false positives,
false negatives and perception changes is it any wonder that the statistics
say that most things cannot be differentiated.

d

Pearce Consulting
http://www.pearce.uk.com

On Thu, 6 Jan 2005 16:55:31 -0000, "Richard Wall"
wrote:


"Don Pearce" wrote in message
...
On Thu, 6 Jan 2005 11:04:55 -0000, "Richard Wall"
wrote:

Can anyone point to a Hi-Fi DBT test where no changes were actually made
in
the source material to check that the audience could perceive no change ?

That would be EVERY DBT test. Each sample is randomly selected from
the two possibilities - that means that in each test the source has
either changed or not. So false positives and false negatives both
show up as failures, and this is as it should be.

I think the reason you have problems with DBTs is not that they fail,
but that they pretty much always succeed. They succeed in showing that
there is in fact no sonic difference between the components under
test. Again, these days that is as it should be - we have advanced far
enough along the HI Fi road that pretty much all components should
sound the same.
Why do you think they always suceed ? I would have thought that most
manufacturers would work to make their systems sound different and the vast
array of suppliers would surely not exist if they all sounded the same ? It
should easily be possible to take three similar amplifiers, modify their
frequency response at above the limit of hearing a difference and then get a
conclusive proof of the effectiveness of the testing method. Has it been
done ?
It may be easy enough to make all kit sound the same, but as it is unlikely
that even if they did all make the prefect reproduction equipment, everyone
would prefer it. I think all the systems belonging to members in our club
sound quite distinct, it would however be impossible to test this opininion.
You have also missed my point that although the testing method is proven the
results on Hi-Fi are not. Hearing is not an absolute and will change over
the listening period. Trying to listen for differences is not the same as
listening to appreciate music. Unless you have data to prove otherwise the
sample sets used to evaluate Hi-Fi are small so surely with false positives,
false negatives and perception changes is it any wonder that the statistics
say that most things cannot be differentiated.

d


Hi Fi manufacturers are the same as any. They change things for two
purposes. First is to make them cheaper to improve the margins. Second
is to make them appear different so that not only new buyers but old
ones will purchase the new item.

There is of course absolutely no need to make them sound different -
actual improvement ceased long ago - while one-upmanship and other
psychological effects do such a fine job without effort on the part of
the manufacturer.

The one area where none of the above applies is the loudspeaker, which
is still frankly a disgrace. You don't need a DBT to reveal the
differences between speakers, and unlike the other stuff, the
differences persist under DBT conditions.

As for DBTs usually succeeding - it is simply true. It is very easy to
run a good DBT. But maybe my definition of success is not the same as
yours. Mine is that the DBT reveals the truth, whatever that is. Maybe
you consider a DBT to have failed if it does not show a difference?
That would be a reasonable conclusion to your assertion that DBTs
don't work for Hi Fi. If things cannot be differentiated in DBT, it is
because they actually aren't different.

d

Pearce Consulting
http://www.pearce.uk.com

Hi Fi manufacturers are the same as any. They change things for two
purposes. First is to make them cheaper to improve the margins. Second
is to make them appear different so that not only new buyers but old
ones will purchase the new item.

There is of course absolutely no need to make them sound different -
actual improvement ceased long ago - while one-upmanship and other
psychological effects do such a fine job without effort on the part of
the manufacturer.
I cannot agree, unless you can name a price point above which this might be
true. Low cost componants sound substantially different than better made
ones. I would have no problem spotting the difference between my Pioneer
DVD565A on CD alone and via my DAC64 in any test. Cheap CD/DVD players
usually are cheap, and are compromised. I wouild assume the same to be true
of cheap amps vs better built products. The differences between "good amps"
are very small and could easily be tweaks to make them suit listening
choices. I have recently compared my Vocale SET, a Pass Labs Aleph 3 and a
Prototype amp that may soon be sold by Revolver. The prototype sounded
dreadful until we found that the volume pot used degraded the sound at its
highest attenuation essential for my 103dB efficient speakers. Once the
prototype was properly attenated I would not be certain that I could
consistantly spot the difference between the three.
I think amplifiers and CD/DVD players etc. sound different because we have
not yet achieved perfect reproduction of the recorded medium. So called
"good" componants are very close to perfect and differentiating by any
method will be difficult. Rather than the image of a straight line with
amplification from source to speakers I view a system as a filter with
amplification between source and speakers. The better the filter the less
it removes/adds and the closer to perfection it gets. It is still possible
to measure electronicly differences in amplifiers and also to make them
react to inputs in in different ways ( e.g. slew rate (?)) There are even
different topographies Class A, AB and I think C and D. DBT should and I am
sure can differentiate some of these classes of products if it cannot then
the testing method must surely be faulty ?


The one area where none of the above applies is the loudspeaker, which
is still frankly a disgrace. You don't need a DBT to reveal the
differences between speakers, and unlike the other stuff, the
differences persist under DBT conditions.

As for DBTs usually succeeding - it is simply true. It is very easy to
run a good DBT. But maybe my definition of success is not the same as
yours. Mine is that the DBT reveals the truth, whatever that is. Maybe
you consider a DBT to have failed if it does not show a difference?
That would be a reasonable conclusion to your assertion that DBTs
don't work for Hi Fi. If things cannot be differentiated in DBT, it is
because they actually aren't different.

My critism of DBT is that it is not simple to set up in a domestic
enviroment and of the few test I have heard of that finds differences
(recent issue of Hi-Fi Plus on cables ?) the conditions of the test are then
questioned. The advocates of DBT/ABX all seem convinced that there are no
differences between CD players, cables and amps, those more open qualify
this with a definition of "good" componants. I am happy to agree that when
talking about "good" components any differences will be minor and most
likely built in by a manufacturer looking to differentiate their product
from the rest. Hearing a difference in a blind test will be unlikely. What
I question is that a lot of current equipment does not meet the "good"
criteria and it does not take a blind test to prove this. If DBT/ABX does
fail to differentiate products that sound different then why does it fail,
my suggestion is that the detection method used (the listener) is too
subjective and without a massive sampling set any results are no better at
providing proof than simple A-B.

Richard
New Ash Green Hi-Fi Club
d

Pearce Consulting
http://www.pearce.uk.com

On Sun, 9 Jan 2005 16:30:51 -0000, "Richard Wall"
wrote:


Hi Fi manufacturers are the same as any. They change things for two
purposes. First is to make them cheaper to improve the margins. Second
is to make them appear different so that not only new buyers but old
ones will purchase the new item.

There is of course absolutely no need to make them sound different -
actual improvement ceased long ago - while one-upmanship and other
psychological effects do such a fine job without effort on the part of
the manufacturer.
I cannot agree, unless you can name a price point above which this might be
true. Low cost componants sound substantially different than better made
ones. I would have no problem spotting the difference between my Pioneer
DVD565A on CD alone and via my DAC64 in any test. Cheap CD/DVD players
usually are cheap, and are compromised. I wouild assume the same to be true
of cheap amps vs better built products. The differences between "good amps"
are very small and could easily be tweaks to make them suit listening
choices. I have recently compared my Vocale SET, a Pass Labs Aleph 3 and a
Prototype amp that may soon be sold by Revolver. The prototype sounded
dreadful until we found that the volume pot used degraded the sound at its
highest attenuation essential for my 103dB efficient speakers. Once the
prototype was properly attenated I would not be certain that I could
consistantly spot the difference between the three.
I think amplifiers and CD/DVD players etc. sound different because we have
not yet achieved perfect reproduction of the recorded medium. So called
"good" componants are very close to perfect and differentiating by any
method will be difficult. Rather than the image of a straight line with
amplification from source to speakers I view a system as a filter with
amplification between source and speakers. The better the filter the less
it removes/adds and the closer to perfection it gets. It is still possible
to measure electronicly differences in amplifiers and also to make them
react to inputs in in different ways ( e.g. slew rate (?)) There are even
different topographies Class A, AB and I think C and D. DBT should and I am
sure can differentiate some of these classes of products if it cannot then
the testing method must surely be faulty ?

No - it is merely revealing your faulty assumptions. The test is
working perfectly. However you should be able to discern the appalling
sound of a SET in any kind of test - blind or not.

As for the different classes of other amplifiers - why should you be
able to discern them audibly, provided they are competently designed?
Their difference does not lie in their sound.


The one area where none of the above applies is the loudspeaker, which
is still frankly a disgrace. You don't need a DBT to reveal the
differences between speakers, and unlike the other stuff, the
differences persist under DBT conditions.

As for DBTs usually succeeding - it is simply true. It is very easy to
run a good DBT. But maybe my definition of success is not the same as
yours. Mine is that the DBT reveals the truth, whatever that is. Maybe
you consider a DBT to have failed if it does not show a difference?
That would be a reasonable conclusion to your assertion that DBTs
don't work for Hi Fi. If things cannot be differentiated in DBT, it is
because they actually aren't different.

My critism of DBT is that it is not simple to set up in a domestic
enviroment and of the few test I have heard of that finds differences
(recent issue of Hi-Fi Plus on cables ?) the conditions of the test are then
questioned. The advocates of DBT/ABX all seem convinced that there are no
differences between CD players, cables and amps, those more open qualify
this with a definition of "good" componants. I am happy to agree that when
talking about "good" components any differences will be minor and most
likely built in by a manufacturer looking to differentiate their product
from the rest. Hearing a difference in a blind test will be unlikely. What
I question is that a lot of current equipment does not meet the "good"
criteria and it does not take a blind test to prove this. If DBT/ABX does
fail to differentiate products that sound different then why does it fail,
my suggestion is that the detection method used (the listener) is too
subjective and without a massive sampling set any results are no better at
providing proof than simple A-B.


DBT simply does NOT fail to differentiate products that sound
different. It ALWAYS finds them. It also reveals the identical sound
of products that you were seduced into believing different by some
factor such as price, builder's bias, appearance, recommendation by a
magazine or whatever.

As for manufacturers building in differences to distinguish them from
others - sadly you are right. This happens mainly in the "boutique
high-end" products where status is everything and sound can go hang.
Avoid these products like the plague if good sound is your goal.

d

Pearce Consulting
http://www.pearce.uk.com

On Sun, 9 Jan 2005 16:30:51 -0000, "Richard Wall"
wrote:

Hi Fi manufacturers are the same as any. They change things for two
purposes. First is to make them cheaper to improve the margins. Second
is to make them appear different so that not only new buyers but old
ones will purchase the new item.

There is of course absolutely no need to make them sound different -
actual improvement ceased long ago - while one-upmanship and other
psychological effects do such a fine job without effort on the part of
the manufacturer.
I cannot agree, unless you can name a price point above which this might be
true. Low cost componants sound substantially different than better made
ones. I would have no problem spotting the difference between my Pioneer
DVD565A on CD alone and via my DAC64 in any test.

Care to put that to the test, when you don't *know* which output is
connected, and they've been level-matched?

Cheap CD/DVD players
usually are cheap, and are compromised. I wouild assume the same to be true
of cheap amps vs better built products.

And we all know what 'assume' does................ :-)

The differences between "good amps"
are very small and could easily be tweaks to make them suit listening
choices. I have recently compared my Vocale SET, a Pass Labs Aleph 3 and a
Prototype amp that may soon be sold by Revolver. The prototype sounded
dreadful until we found that the volume pot used degraded the sound at its
highest attenuation essential for my 103dB efficient speakers. Once the
prototype was properly attenated I would not be certain that I could
consistantly spot the difference between the three.

Quite so - and the same applies to modern CD players.

I think amplifiers and CD/DVD players etc. sound different because we have
not yet achieved perfect reproduction of the recorded medium. So called
"good" componants are very close to perfect and differentiating by any
method will be difficult. Rather than the image of a straight line with
amplification from source to speakers I view a system as a filter with
amplification between source and speakers. The better the filter the less
it removes/adds and the closer to perfection it gets. It is still possible
to measure electronicly differences in amplifiers and also to make them
react to inputs in in different ways ( e.g. slew rate (?)) There are even
different topographies Class A, AB and I think C and D. DBT should and I am
sure can differentiate some of these classes of products if it cannot then
the testing method must surely be faulty ?

Why would you *assume* that class of operation matters (aside from
class C, of course!) to the sound of an amplifier? While it's true
that most SET amps sound dreadful, that's not because they operate in
class A. All good amps *do* sound the same, and there's nothing wrong
with double-blind ABX testing which shows that to be true.

The one area where none of the above applies is the loudspeaker, which
is still frankly a disgrace. You don't need a DBT to reveal the
differences between speakers, and unlike the other stuff, the
differences persist under DBT conditions.

As for DBTs usually succeeding - it is simply true. It is very easy to
run a good DBT. But maybe my definition of success is not the same as
yours. Mine is that the DBT reveals the truth, whatever that is. Maybe
you consider a DBT to have failed if it does not show a difference?
That would be a reasonable conclusion to your assertion that DBTs
don't work for Hi Fi. If things cannot be differentiated in DBT, it is
because they actually aren't different.

My critism of DBT is that it is not simple to set up in a domestic
enviroment

Sure it is, especially when compared with setting up a Linn Sondek!

and of the few test I have heard of that finds differences
(recent issue of Hi-Fi Plus on cables ?) the conditions of the test are then
questioned.

Rightly so, since that was *not* a DBT.

The advocates of DBT/ABX all seem convinced that there are no
differences between CD players, cables and amps, those more open qualify
this with a definition of "good" componants.

I have in fact never seen *anyone* claim that *all* CD players and
amps sound the same, although I agree that the good ones do - as you'd
expect. OTOH, wire *is* just wire, unless you *really* mess up the
basic electrical parameters.

I am happy to agree that when
talking about "good" components any differences will be minor and most
likely built in by a manufacturer looking to differentiate their product
from the rest. Hearing a difference in a blind test will be unlikely. What
I question is that a lot of current equipment does not meet the "good"
criteria and it does not take a blind test to prove this.

Never argued - and particularly true as the price goes up!

If DBT/ABX does
fail to differentiate products that sound different then why does it fail,

It doesn't...................

my suggestion is that the detection method used (the listener) is too
subjective

That's why it's a *blind* test. Duhh...........

and without a massive sampling set any results are no better at
providing proof than simple A-B.

Richard
New Ash Green Hi-Fi Club

Why don't you encourage the club to run level-matched ABX sessions, if
you're *really* interested in the truth? One test setup would then be
shared among all the members, and you'd also have a larger sample.
--

Stewart Pinkerton | Music is Art - Audio is Engineering

In article , Richard Wall
wrote:


Cheap CD/DVD players usually are cheap, and are compromised. I wouild
assume the same to be true of cheap amps vs better built products.

That seems to me to be a reasonable assumption. However I'd also expect
this to depend on the conditions of use. e.g. a cheap amp might sound fine
(and similar/indistinguishable from) a more expensive/better one if the
required power/current levels were modest, but sound very different when
asked to play at high levels into a speaker tha was a 'nasty' load.

I think amplifiers and CD/DVD players etc. sound different because we
have not yet achieved perfect reproduction of the recorded medium.

Well, if the recording has an imperfection, then I'd expect that to show as
a 'common mode' effect when comparing amps or players.


So called "good" componants are very close to perfect and
differentiating by any method will be difficult.

That may be so. Indeed, it may well mean that the differences become so
small that they either become inaudible, or not worth bothering about given
much larger variations - e.g. due to slight head movements or changes in
the dampness of the air in the listening room.

Rather than the image of a straight line with amplification from source
to speakers I view a system as a filter with amplification between
source and speakers. The better the filter the less it removes/adds and
the closer to perfection it gets. It is still possible to measure
electronicly differences in amplifiers and also to make them react to
inputs in in different ways ( e.g. slew rate (?))

I'd agree. However with decent components my experience is that the main
'filter' effects (to use your analogy) tend to end up residing in:

1) Choice of the specific CD (or LP or whatever medium of choice)

2) Choice of loudspeaker, their placement, the listening position, and the
room acoustic.

What interests me about that is the way magazines devote thousands of pages
to discussing (relatively tiny) differences between amps, yet almost zero
space to questions of improving your room acoustics, etc.


There are even different topographies Class A, AB and I think C and D.
DBT should and I am sure can differentiate some of these classes of
products if it cannot then the testing method must surely be faulty ?

No. Your deduction does not follow from your comment. It may be that when a
test shows no difference, that the reason is simply that any difference is
too slight to be audible, or perhaps of any practical significance.

Thus if such a test 'fails' (a judgemental word, so should be used with
care) to show a difference, it may be due to there being no audible
difference.

In such circumstances you can propose two possible hypotheses.

1) That the difference is so small as to be inaudible in the circumstances
of the test

2) That the test 'hides' the difference in some way due to the
circumstances in which it was carried out.

However the normal scientific method then requires us to either propose a
test whose results would discriminate between the above two hypotheses, or
if we cannot, then simply use the one which follows 'Occam' and requires
the least number of assumed unknown or hidden effects/variables. This
implies that we are safe to choose (1) *unless* someone can propose and
carry out a test that will discriminate between (1) and (2) by providing
results that would 'falsify' (i.e. show results inconsistent with) one
hypothesis.

Alternatively, you can propose the hypothesis that the circumstances of the
test make it 'inappropriate' for the real listening conditions. But to
establish this, once again, you need a test that could be carried out to
falsify this or its converse.

FWIW I can't comment on class C or D, but I've built and tested/listened to
various A and AB amps over the years, and my experience is that although
some sound 'different' to the others, many sound indistinguishable to me in
normal use. Provided they meet a fairly basic set of requirements and are
used within conditions they are designed to operate under.


The one area where none of the above applies is the loudspeaker, which
is still frankly a disgrace. You don't need a DBT to reveal the
differences between speakers,

Indeed. Such differences are also often easily observable using a
microphone and some simple measurements.


My critism of DBT is that it is not simple to set up in a domestic
enviroment

I would agree. Carefully designed tests like these can be extremly
difficult to carry out in a reliable manner. But this does not make them
impossible.

and of the few test I have heard of that finds differences
(recent issue of Hi-Fi Plus on cables ?) the conditions of the test are
then questioned.

Can't comment on the HFP tests you mention as I don't know anything about
them. But, yes, if the tests are not carried out with appropriate care and
rigour, then their results may be questioned. Indeed, this is normal
practice in measurement science as all measurements will be of finite
accuracy, limited relaibility, etc. But - depending on the details - that
may be adequate for some stated purpose.

The advocates of DBT/ABX all seem convinced that
there are no differences between CD players, cables and amps,

I have no idea where your "all" comes from. However I have personally never
thought or written such a thing. That said, the only way I "advocate"
DBT/ABX is that it seems to me that it represents a test approach which may
be useful for some purposes. It may help reduce some confusing effects, and
make it easier to do a reliable statistical analysis where any response is
slight rather than obvious.

those more open qualify this with a definition of "good" componants.

Even here I would be wary as two 'bad' items may have flaws that produce
similar effects, so lead to the two souding similar or indistinguishable.


I am happy to agree that when talking about "good" components any
differences will be minor and most likely built in by a manufacturer
looking to differentiate their product from the rest. Hearing a
difference in a blind test will be unlikely. What I question is that a
lot of current equipment does not meet the "good" criteria and it does
not take a blind test to prove this.

That may well be so in some cases. Indeed some 'good' items sound different
(speakers). However in such cases I would expect a BDT/ABX to also show
differences, although such a tedious method may be felt not to be worth the
effort.


If DBT/ABX does fail to differentiate products that sound different

I note the "If" that starts your statement here...

then why does it fail,

.... I think that you will first need to catch your rabbit. Only then think
about making a rabbit pie. :-)

i.e. first you will need to address the test of mutually exclusive
hypotheses outlined above. This means first proposing a test that shows
that a given DBT/ABX test *has* 'failed' to show a difference which some
other non-DBT/ABX can be *established* to 'detect'. The problem, though,
then becomes how to establish that the non-DBT/ABX test result *is* a real
and reliable 'detection' rather than an error or a fluke.

my suggestion is that the detection method used (the listener) is too
subjective and without a massive sampling set any results are no better
at providing proof than simple A-B.

For all I know you might be correct in some cases. But I think you will
need to catch your rabbit, first... ;-

Having done that, you could then examine the conditions of the tests, and
ways to check which might be having an effect.

Until, then, though, the simplest course seems to me to work on the basis
that any 'difference' that was not detected was too small to be noticed,
and hence really may not matter one way or the other if all you want to do
is sit back and enjoy the music. :-)

Slainte,

Jim

--
Electronics http://www.st-and.ac.uk/~www_pa/Scot...o/electron.htm
Audio Misc http://www.st-and.demon.co.uk/AudioMisc/index.html
Armstrong Audio http://www.st-and.demon.co.uk/Audio/armstrong.html
Barbirolli Soc. http://www.st-and.demon.co.uk/JBSoc/JBSoc.html

In article , Richard Wall
wrote:

I would have thought that most manufacturers would work to make their
systems sound different

Some may wish a specific 'sound'. Others may not. Although it depends what
you mean. It also depends if you are talking about amps, speakers, or other
types of equipment.

and the vast array of suppliers would surely not exist if they all
sounded the same ?

Depends, There are also a variety of required conditions of use, and prices
people are willing to pay.

It should easily be possible to take three similar amplifiers, modify
their frequency response at above the limit of hearing a difference and
then get a conclusive proof of the effectiveness of the testing method.
Has it been done ?

I'm not clear from what you write above what hypothesis you are wishing to
test.


It may be easy enough to make all kit sound the same,

See above. Speakers and listening rooms can vary quite a lot. So can the
requirements of listeners. My experience is that compared with this the
differences between many amplifiers are quite small (or effectively
undetectable in use).


but as it is unlikely that even if they did all make the prefect
reproduction equipment, everyone would prefer it.

Only if they all have the same room acoustics, same requirements, etc.

I like quad ESLs. But that does not mean I expect them to suit everyone.
This isn't just a matter of what features of the resulting sound matter to
me. The room also matters, as does the taste in music.

I think all the systems belonging to members in our club sound quite
distinct, it would however be impossible to test this opininion. You
have also missed my point that although the testing method is proven the
results on Hi-Fi are not.

Afraid I am not clear what you mean by either of the above statements.

Hearing is not an absolute and will change over the listening
period.

Agreed.

Trying to listen for differences is not the same as listening
to appreciate music.

Agreed.


Unless you have data to prove otherwise the sample sets used to evaluate
Hi-Fi are small so surely with false positives, false negatives and
perception changes is it any wonder that the statistics say that most
things cannot be differentiated.

The method of a controlled ABX type of test is to allow the listener to
decide for themself when they think they can identify which choice is 'X'.
And to do this in a manner that aids us in drawing statistical results from
repeated sets of such tests. The idea is that if A and B can't really be
told apart under the conditions of the test, then the resulting 'choices'
will fail to correlate with the real identity to a level that shows
statistical significance.

However such tests only can provide results that are directly relevant for
the conditions under which the tests were performed. And our ability to use
the results for other purposes will depend on how reliably they fit a given
situation. The results can also only be regarded as useful if the tests are
performed often enough in well-enough controlled circumstances to give the
resulting data statistical significance to the required level.

The problem is this is quite a demanding and time consuming process. As you
say, it ends up being quite different to sitting down and enjoying some
music. However in my personal view, if differences are so slight as to
require extended tests like this to show up, then perhaps they don't really
matter much in practice. :-)

FWIW although I have often heard differences (or at least *thought* I
have!) between various items, I often then am not bothered if they are
small compared with choice of speakers, position of speakers, etc. Thus to
a large extent worrying about something like choice of caps tends to end up
seeming to me like a waste of effort once the effect is swamped by a slight
movement of the head. I'd rather spend the time enjoying music on my
imperfect audio systems. :-)

Slainte,

Jim

--
Electronics http://www.st-and.ac.uk/~www_pa/Scot...o/electron.htm
Audio Misc http://www.st-and.demon.co.uk/AudioMisc/index.html
Armstrong Audio http://www.st-and.demon.co.uk/Audio/armstrong.html
Barbirolli Soc. http://www.st-and.demon.co.uk/JBSoc/JBSoc.html

In article , Richard Wall wrote:
I believe that DBT is unsuitable for Hi-Fi evaluation as the detection
method (you the listener) are anticipating a change to occur at some point
and will be listening to changes not trying to enjoy music.

The testing and analysis method does indeed have to deal with this
tendency to anticipate change.

Can anyone point to a Hi-Fi DBT test where no changes were actually made in
the source material to check that the audience could perceive no change ? I
think that the false positives raised by such a test would invalidate this
testing regime ?

You may be interested to read some relevant comments at:

http://www.bostonaudiosociety.org/ba...l_thinking.htm

(Note: I have only scanned the article and not evaluated it - Caveat
Lector.)

--
John Phillips