Thus spake Jim Lesurf:

An alternative view might be that differences between cables are
indeed real but are swamped by differences due to alcohol
consumption, level of wakefulness, medical conditions, mood etc to
the point of diminishing returns.

As Don has indicated, the problem with the above hypothesis is that
people make claims like the differences are 'obvious' and that they
can easily tell one cable from another. But then either fail to do so
in a blind test, or refuse to take any such test.

I can understand people (mostly men as it happens) boosting about their
sexual prowess, wealth, etc, etc but proudly declaring one's ability to be
able to hear if vendors at a Hi Fi show are using a particular model of
mains cable /from the ****ing corridor/ is really going to impress the
girlies. In other words, the problem with the hypothesis lies with many
individual's lack of security or balance in their existence. Oh, OK, that's
a bit longwinded, lets just call them bull****ers.

If people simply said the differences were 'slight' and 'hard to
detect' or 'elusive' then it might be more plausible that the 'test
masks the effect'.

Also, although people have claimed the test 'masks' any effect,
no-one has proposed a testable hypothesis for this, *and then tested
it*.

Infinite regression! I do have this sinking feeling that possibly DBT may
well be fairly pointless in that it will skew the results away from hearing
possible differences between cables of near identical electrical parameters.
Hearing is so subjective, just how the hell can anyone with any degree of
accuracy, determine the differences between 2 tests /with/ predefined added
R,C & I during the 'calibration' stage due to poor auditory memory, let
alone near identical cables? I have changed (sorry, I mean "upgraded")
equipment to initially hear no immediate distinguishable difference.
However, days, weeks or months later have listened to a particular piece of
music have thought that it definitely sounded 'better' (usually more
detail.)

(Some of us might welcome a test to see if 'drinking Lagavulin masks
the ability to hear one cable from another', though... :-) )

Long live Port Ellen

The points you make about trying to 'calibrate' the sensitivity of
listeners to various types of alteration is, in itself, quite a
sensible one. Indeed, IIRC if you look at the JAES and various
audio/hearing/sound journals you will find such tests and the results.

Definitive testing would perhaps be best done by something like a university
psychology dept. Actually, that's not such a daft idea as it 1st sounds.
They would need to engaged some engineering skill but hopefully would stand
some chance of being impartial & seen to be so. They could then be on hand
to give PTSD counselling to any subjects upset by the findings!

Step 1 is therefore for someone (ideally a number of people) to show
they *can* hear a difference even when the well-know reasons are
excluded.

Step 2 would then be to hypotheses why this may be so, propose
mechanisms, and try to calibrate sensitivity to them. Until we know
what we wish to 'calibrate' we can't really do so.

But there is no real point to 'step 2' if in fact no-one can show that
'step 1' can be passed with a reliable and positive outcome.

I think that both stages need to be passed for the acolytes to 'win' & the
calibration stage passed to validate the procedure. No such undertaking
would be worth the effort just for cables but would probably need to test
anything in the chain, apart from speakers perhaps. However, what ever the
outcome, a certain % would never believe the conclusions, however conclusive
the majority felt the tests had been but what's new or surprising with that?

Paul B wrote:

... I have changed (sorry, I mean "upgraded")
equipment to initially hear no immediate distinguishable difference.
However, days, weeks or months later have listened to a particular piece of
music have thought that it definitely sounded 'better' (usually more detail.)

You can still do a double blind test with each stage taking months.

--
Eiron

I have no spirit to play with you; your dearth of judgment renders you
tedious - Ben Jonson.

Don Pearce wrote:
You need to think this through a bit.

Perhaps a bit but I would expect you to put in most of the work to
develop your procedure. I am just chatting on usenet and have no
interest in getting directly involved in experiments which have the
objective of convincing audiophiles they are wrong about the sound of
cables and the like.

Do you know how unflat the response of a room is -

Why is it relevant? The objective is to check two sounds are the same.
This is something you can test for in any room.

and how much it varies with even a couple of millimetres of movement?

Again I cannot see the relevance.

Likewise for a microphone.

Why do you need to move the microphone?

What you are
suggesting here could not be done, even in the best anechoic chamber
with the best microphone.

I can see no particular need for an anechoic chamber or a particularly
linear microphone.

And what do you do about the fact (to 99.9999999% certainty) that
there IS no audible difference between the cables to factor out

What has audibility got to do with it? The difference between the
cables is easy to calculate given a few basic measurements.

- what is it that you are actually adding to the signal here?

One way to proceed might be something like this:
* Analyse the two cables for the given amplifier and loudspeaker
impedances.
* Program filter using analysis so that, say, signal via cable B is
transformed to signal via cable A.
* Create sweeps directly and via filter.
* Play unprocessed sweep via A and measure. Play processed sweep via B
and measure. Check an order of magnitude or two closer than difference
between A and B playing unprocessed sweep.
* If not, modify filter parameters guided by difference and repeat.
* Use filter to generate processed music source.
* Switch cables A B, A B, A B,... but play a random sequence of A B and
A B* where B* is the processed source that produces the same signal as
A.
* Ask the subject if the sound is the same or different and analyse for
significance as usual.

Again, I am sure there are a few variations and improvements but the
above is hopefully the gist. Obviously this involves more messing about
than a blind test but it allows the subject to hear both cables in a
fully sighted and "natural" manner. The thing hidden is which source is
being used for one of the cables.

On 14 Jan 2006 04:16:25 -0800, "andy" wrote:

Don Pearce wrote:
You need to think this through a bit.

Perhaps a bit but I would expect you to put in most of the work to
develop your procedure. I am just chatting on usenet and have no
interest in getting directly involved in experiments which have the
objective of convincing audiophiles they are wrong about the sound of
cables and the like.

Do you know how unflat the response of a room is -

Why is it relevant? The objective is to check two sounds are the same.
This is something you can test for in any room.

and how much it varies with even a couple of millimetres of movement?

Again I cannot see the relevance.

Likewise for a microphone.

Why do you need to move the microphone?

What you are
suggesting here could not be done, even in the best anechoic chamber
with the best microphone.

I can see no particular need for an anechoic chamber or a particularly
linear microphone.

And what do you do about the fact (to 99.9999999% certainty) that
there IS no audible difference between the cables to factor out

What has audibility got to do with it? The difference between the
cables is easy to calculate given a few basic measurements.

- what is it that you are actually adding to the signal here?

One way to proceed might be something like this:
* Analyse the two cables for the given amplifier and loudspeaker
impedances.
* Program filter using analysis so that, say, signal via cable B is
transformed to signal via cable A.
* Create sweeps directly and via filter.
* Play unprocessed sweep via A and measure. Play processed sweep via B
and measure. Check an order of magnitude or two closer than difference
between A and B playing unprocessed sweep.
* If not, modify filter parameters guided by difference and repeat.
* Use filter to generate processed music source.
* Switch cables A B, A B, A B,... but play a random sequence of A B and
A B* where B* is the processed source that produces the same signal as
A.
* Ask the subject if the sound is the same or different and analyse for
significance as usual.

Again, I am sure there are a few variations and improvements but the
above is hopefully the gist. Obviously this involves more messing about
than a blind test but it allows the subject to hear both cables in a
fully sighted and "natural" manner. The thing hidden is which source is
being used for one of the cables.

Ah - the light dawns! OK, this test is not for the measurable
differences which are known and accepted. It is for the extraordinary
claims made for boutique cables that have nothing to do with this -
stuff like rhythm, air, sparkle etc.

Nobody has any problem with cables sounding different on the basis of
known phenomena.

d

Pearce Consulting
http://www.pearce.uk.com

In article , Don Pearce
wrote:
[big snip]

Ah - the light dawns! OK, this test is not for the measurable
differences which are known and accepted. It is for the extraordinary
claims made for boutique cables that have nothing to do with this -
stuff like rhythm, air, sparkle etc.

I'm not certain I have understood Andy's intent correctly. However I think
what he is suggesting is a way to test the degree to which the test
subject's decisions would be affected by 'being told what cable is in use'.

The point being that what they are told may be false. e.g they may
told/shown "we are now using cable A', but in reality cable B is now being
used.

The results from such a proceedure are then compared from another, using
the same cables, but where they are given no such 'external' info.

If the test subject *is* being affected by this 'external' (i.e. nothing to
do with the sound) info, it would then show in a 'bias' on the "being told"
tests that changed their results from the "not being told" ones....

The difficulty with this process is that it can only be expected to be
useful if the test subject has not been warned that when they are "told",
*what* they are "told" may be deliberately incorrect at times. As soon as
they are aware of this possibility the effort may become futile. This means
that a snag is that discussing this now may 'blow the gaff'. :-)

Also, they may come to suspect this whilst being tested, so cease to rely
on what is said, and this might also cause any actual 'bias' to vanish. So
such an approach would be quite difficult to validate.

Slainte,

Jim

--
Electronics http://www.st-and.ac.uk/~www_pa/Scot...o/electron.htm
Audio Misc http://www.st-and.demon.co.uk/AudioMisc/index.html
Armstrong Audio http://www.st-and.demon.co.uk/Audio/armstrong.html
Barbirolli Soc. http://www.st-and.demon.co.uk/JBSoc/JBSoc.html

In article , Paul B
wrote:
Thus spake Jim Lesurf:

An alternative view might be that differences between cables are
indeed real but are swamped by differences due to alcohol
consumption, level of wakefulness, medical conditions, mood etc to
the point of diminishing returns.

As Don has indicated, the problem with the above hypothesis is that
people make claims like the differences are 'obvious' and that they
can easily tell one cable from another. But then either fail to do so
in a blind test, or refuse to take any such test.

I can understand people (mostly men as it happens) boosting about their
sexual prowess, wealth, etc, etc but proudly declaring one's ability to
be able to hear if vendors at a Hi Fi show are using a particular model
of mains cable /from the ****ing corridor/ is really going to impress
the girlies.

It is difficult to speculate on this as we risk stacking one speculative
hypothesis on another. However any such 'self convincing' (or perhaps
'fibbing') - assuming it is occurring - may be for other reasons.

In other words, the problem with the hypothesis lies with many
individual's lack of security or balance in their existence. Oh, OK,
that's a bit longwinded, lets just call them bull****ers.

Assuming it occurs, that may not be the reason, though. For example,
someone may simply become more familiar with the music, or the focus of
their attention wanders as they explore it, or they become more tired, or
they shift their head, or...

The problem is that perceptual *or real* changes may arise for various
'reasons', but then be mis-attributed if the person was unaware of the
'real reason'. If this mis-attribution accords with an 'authority' it may
then become mentally established as 'the reason'. Once a number of people
have been through such a process we may have a community whose
experiences are in accord *due to making compatable errors or
mis-attributions - and who then all feel they must be correct as they all
seem to agree and share the 'same experience'. I can't say this *has*
happened, but is just as consistent with the evidence we have as other
hypotheses. To deal with this, we need people to engage in the relevant
test, and then see the evidence this returns. (Alas, the tendency is to
refuse as people "already know" their ideas must be correct.)

Thus the aim of tests that try to exclude as many 'external' factors as
possible, or randomise their effects, thus trying to concentrate on the
proposed/hypothesised 'cause'.

If people simply said the differences were 'slight' and 'hard to
detect' or 'elusive' then it might be more plausible that the 'test
masks the effect'.

Also, although people have claimed the test 'masks' any effect, no-one
has proposed a testable hypothesis for this, *and then tested it*.

Infinite regression!

That should not be the case. The point here is a different hypothesis may
require a different test *and* may only be meaningful once the relevant
evidence shows the 'new' test becomes relevant.

In this context, the initial 'test' would be to see if a test subject can
reliably tell one cable from another in a given system, solely on the basis
of any changes in the sounds produced.

If this test (or series of tests on various subjects) returns evidence that
shows no sign that the test subjects can tell one cable from another, you
can then - in principle - go on to a *different* test arrangement which
seeks to explore the 'reasons' for this and discriminate between the
alternatives

A) The result occurred as the cables didn't really change the sounds to an
audible degree, so the initial idea that an audible change was the reason
for the belief is simply false.

B) That the test(s) used to see if people could tell one cable from another
by sound alone were faulty in a way that 'masked' or 'blocked' the actual
ability to do so.

Since this later test is tacking a different task, it could be expected to
be a different type of test. So we do not have a regression. As someone
already pointed out, the test has to be suitable for the "job". Having
dealt with one hypothesis we could move on to the next "job" (telling A
from B) and expect a new, different type of test to be required for this
different "job".

However, although some people clearly assert or imply that (B) must be the
reason no-one seems able to support their beliefs that cables make audible
differences in a suitable test of *that* hypothesis, so far as I know,
no-one has proposed a new test that could discriminate between (A) and (B).
Hence we have no evidence that can reliably discriminate between them. If
we then employ the scientific method, we can then decide (A) and (B) are
not distunguishable hypotheses, and can choose the 'simpler' as I described
- at least until someone comes up with a 'new' test

Hence the snag is that at present claims that the inablity to hear cable
differences in a test are due to (B) are based solely on 'faith' that the
differences *do* exist for the *claimed reasons*. As such, without the new
test, such claims have no real basis in scientific terms. But this might
change if a suitable test to tell (A) from (B) was proposed, carried out,
and the results evaluated. But the snag here is that so far as I can tell,
no-one has been able to suggest a practicable test which whose results
would be able to distinguish (A) from (B). If no-one can, then the
distinction remains a speculation with no real value - except in terms of
an assertion of 'faith'.


I do have this sinking feeling that possibly DBT may well be fairly
pointless in that it will skew the results away from hearing possible
differences between cables of near identical electrical parameters.

You would have to explain the basis for this...


Hearing is so subjective, just how the hell can anyone with any degree
of accuracy, determine the differences between 2 tests /with/ predefined
added R,C & I during the 'calibration' stage due to poor auditory
memory, let alone near identical cables?

My understanding is that the tests simply tend to require similar frequency
responses and gains. Not necessarily 'equalised' RLCG values. This is
simply to exclude non-contentious causes of audible changes. I would assume
this only matters if this was the actual cause of a belief, but the
beleiver was not willing to accept this as the cause.

Again, if ensuring similar RCLG values causes any perceived difference to
vanish, why would the person who said they could hear differences wish to
assert it had arisen for any other reasons? Alternatively, what test would
you propose to distinguish here? If you no such test can be practical, then
you are trying to make a distinction that may simply not exist.

I have changed (sorry, I mean "upgraded") equipment to initially hear no
immediate distinguishable difference. However, days, weeks or months
later have listened to a particular piece of music have thought that it
definitely sounded 'better' (usually more detail.)

I have also *thought* so in similar circumstances. However I have also
thought the sound changed over a period of time even when I *haven't*
changed anything. Hence having such an impression is not necessarily a very
reliable guide of anything specific. However this does illuminate why we
have to be careful in assuming a perceived change *is* due to a given cause
(e.g. "because I changed the cables").

[snip]

Definitive testing would perhaps be best done by something like a
university psychology dept. Actually, that's not such a daft idea as it
1st sounds. They would need to engaged some engineering skill but
hopefully would stand some chance of being impartial & seen to be so.
They could then be on hand to give PTSD counselling to any subjects
upset by the findings!

LOL. Actually, the people at St Andrews trick-cycling dept would probably
drag you off to the local pub for this. :-)

Slainte,

Jim

--
Electronics http://www.st-and.ac.uk/~www_pa/Scot...o/electron.htm
Audio Misc http://www.st-and.demon.co.uk/AudioMisc/index.html
Armstrong Audio http://www.st-and.demon.co.uk/Audio/armstrong.html
Barbirolli Soc. http://www.st-and.demon.co.uk/JBSoc/JBSoc.html

Forwarder wrote:
Paul B wrote:

Thus spake Don Pearce:

OK, I've had a bit of a think - and I've written a protocol, which I
believe would be a basis of fair testing of audio components to
resolve such issues as cable sound etc.

I invite all here to read what I suggest, and let me know if I have
either missed something, or am being unfair on one direction or
another.

http://www.donepearce.plus.com/odds/dbt/

I think this could be a way of defusing the vituperation that
currently surrounds the subject.

So what do you think?

d

Pearce Consulting
http://www.pearce.uk.com



This subject, for good or bad, really fascinates me


Yes, I am in it for the fascination also. We have on the one side, a
sincere bunch of folk claiming that they get "blown away" by this or
that power cable or interconnect or speaker cable, cd player, or even
amplifier! .. which is brushed off by another bunch of folk, as much
sincere, as "imagination" .. to put it diplomatically. One side really
does hear things, the other "proves" that what they hear is their
imagination by employing "scientific" tests that seem to mask out even
the real differences(note 1)..

Reading between the lines of all the flame texts, the spite, the
bad-will, (audiophoolery, conmen, borgs, class-envy of the poor etc) I
want to understand what got these two camps to where they are. AANDD I
want a plausible explanation as to why in the heck I am originally and
sincerely hearing differences between amps, cd players, etc, that seem
to "originally and sincerely" vanish into thin air when these
"scientific" test and abx boxes are employed in between. I see this as a
"problem" and I want to solve it. I do not want to just be smug in
accepting this or that explanation as I know that neither is able to
explain the whole truth.

Let's do some speculation now: All tests have one trait in common; They
take away the knowledge, the sight from the listener. Once this happens
what one hears is also "altered". Or, is it the other way around?

Obviously, knowledge and sight aid us in hearing subtle details in sound.

Is there anything inherently wrong with this? Is it not true that *all*
of our senses help each other out? And is it also not true that although
they "help", they are not able to ultimately determine what we hear, or
taste, or see, in the end.

I have listened to a *very* good looking transparent cable, for
instance. It's appearance was giving cues of wholesomeness, richness of
flavor, abundance power, air, (it was thicker then thick, shiny, etc).
But it's sound was thin, screechy, too fast attack and decay, dry and
bodyless bass, etc. On it's own, my eyes closed and all (*without*
comparing) this cable would sound the exact same way. I know that since
when it was employed a freind of mine came over and commented on the
stereo, playing in another room, as being "wrong and screechy".

When a given dish, or wine is supposed to be tasted does the sight of
the dish, and of course the smell of it assist us in getting all the
subtle details of flavour and taste of that dish/wine? If the answer to
this is yes, then is there anything wrong with that?

Imagine a tall glass of lemonade on a hot day. Ice, dew around the glass
.. etc, the works. Imagine that you are thirsty, you've just had some
great sex, or you just did some exercises, etc. You grab that lemonade,
you start drinking, and instead of sugar salt! You spit the thing out.
But imagine a more *subtle* difference in taste.. Sugar vs artificial
sweetner for instance. When you know you are drinking something
sweetened with artificial sweetner you make funny faces, it tastes
terrible. But you are hard pressed to spot it in a double blind test. We
know this from the dbt's they did when developing the product!!

We normally listen to music with the sight and knowledge of what is
being employed for the playing. When this "sight and knowledge" is
stripped away in an ABX or DBT, is it any longer possible to claim that
what is being tested is whether or not a difference in the SOUND exists?
Rather, is it much more the *listener*, ie, the subject who is being
tested? More then anything else, the subject's ability to *adapt* to a
whole new and strange mode of listening to music is being tested, his
ability to instantaeniously transform an action of pleasure seeking into
a stressful task of problem solving is being tested.



There's obviously no harm in people running their own informal tests

I agree, but to a point. There are is some "harm" if one draws all
encompassing conclusions from tests that are inadequate. Just look at
all the hatred around here..


I hate auditioning new stuff & would love to be able to select equipment on
specification, looks, build quality, ergonomics, power consumption, price
etc.

Some would say that what's keeping you, they all sound the same.. But
you *know* for a fact that they do not. And you do not want to diminish
the amount of pleasure you are getting from consuming recorded music by
listening to it from inferior sounding equipment (or equipment that just
is not to your taste). Or by constantly having to convince yourself that
all this stuff sounds the same in tests : "my friends Linn cd player
sounds the same as this yamaha dvd/cd player I have, I *know* .. Science
tells me so... I was over there, I lstened to that damned Linn, it
sounded so wonderfull, so musical, so dramatic, and this yamaha just
DOES NOT, GODDAMNIT!, but they all sound the same, but they all sound
the same, so say my scientific tests, GODDAMNIT! they all sound the same
.." etc..


I don't presume to be an expert, so several points I've made may well have
many weaknesses but any tests done without much thought would just
perpetuate the schism.


Yes, that's exactly what is happening.

(note 1) .. when we were fooling around with ABX and amps we actually
tried out this also: we put the *same* brand and model two amps on the
ends of the abx box (it was a cambridge audio azur model). We messed
around with the *tone controls* of one (+%50 bass and treble) and left
the other alone. It was still almost impossible to hear a difference in
the abx protocol, with all those boxes, whatnot in between. When we
fully employed the bass and treble in one the task eased a bit, but it
was still very very hard.

As you point out the more "refined" you make the DBT/ABXing protocol
the more it interferes with any possibility of an average testee ever
hearing any difference between anything and anything else in audio With
a lot of training designed to make them cope with the ABX fog a few
may hear something even if ABXed.. It is not my business to speculate
why ABX is the way it is. It is the business of its proponents to
prove that it helps to show differences instead of making it harder. .
In place of experimental evidence we get devotional confessions of
faith.
This suits not a few: They never heard and never intend to hear
real-life "acoustic instruments playing "acoustic" sonata, a quartet or
a symphony. ABX is a wonderful placebo (Yes, Mr. Pearce- I did say
placebo!) a real balm for their minds. Here is "science" confirming
that if they don't hear differences anyone who does is a deluded victim
of "bias".
Incidentally; money helps but is not the end and the beginning. I heard
some DIY stuff superior to most competition- and certainly superior to
what you hear in a rock concert or through your blaster or you Bose
system.
Ludovic Mirabel
Sorry, we human beings also differ from each other in ways not as yet
classifiable or testable.
Ludovic Mirabel

Jim Lesurf wrote:
I'm not certain I have understood Andy's intent correctly.

My intent was simply to point out that one can perform valid sighted
experiments instead of blind ones if you wish. I indicated two
different approaches but would expect there to be others. You appear to
be discussing the first example.

However I think
what he is suggesting is a way to test the degree to which the test
subject's decisions would be affected by 'being told what cable is in use'.

The point being that what they are told may be false. e.g they may
told/shown "we are now using cable A', but in reality cable B is now being
used.

The results from such a proceedure are then compared from another, using
the same cables, but where they are given no such 'external' info.

If the test subject *is* being affected by this 'external' (i.e. nothing to
do with the sound) info, it would then show in a 'bias' on the "being told"
tests that changed their results from the "not being told" ones....

This is not quite what I intended. There is only one experiment but
what is seen and what is heard are independently controlled enabling
the effect of both on the results to be determined. I did not specify
how to achieve the visual deception or suggestion which would be an
important part of the design of the experiment.

The difficulty with this process is that it can only be expected to be
useful if the test subject has not been warned that when they are "told",
*what* they are "told" may be deliberately incorrect at times.

Of course but if you wish to quantatively measure the effect there is
no alternative of which I am awa you have to separate
sight/suggestion and sound.

As soon as
they are aware of this possibility the effort may become futile. This means
that a snag is that discussing this now may 'blow the gaff'. :-)

Another possible problem which would need careful handling in the
design of the experiment is that if the subject can link a particular
class of sound with A or B (e.g. A is bright and B dull) then when the
visual/suggested cues swap they may become confused.

Also, they may come to suspect this whilst being tested, so cease to rely
on what is said, and this might also cause any actual 'bias' to vanish.

If this happens then it should be visible in the results and in
discussion with the subject at the end.

So such an approach would be quite difficult to validate.

Validate? I cannot see anything to validate but this may be a
difference of terminology.

In article .com,
andy
wrote:
Jim Lesurf wrote:
[sni[]

If the test subject *is* being affected by this 'external' (i.e.
nothing to do with the sound) info, it would then show in a 'bias' on
the "being told" tests that changed their results from the "not being
told" ones....

This is not quite what I intended. There is only one experiment but what
is seen and what is heard are independently controlled enabling the
effect of both on the results to be determined. I did not specify how to
achieve the visual deception or suggestion which would be an important
part of the design of the experiment.

OK. The primary difficulties here would hing on validation, which I will
try to explain below...

The difficulty with this process is that it can only be expected to be
useful if the test subject has not been warned that when they are
"told", *what* they are "told" may be deliberately incorrect at times.

Of course but if you wish to quantatively measure the effect there is no
alternative of which I am awa you have to separate sight/suggestion
and sound.

I would agree. However there are some distinct purposes he

1) A test which is intended to see if the test subject can tell one cable
from another, based solely on the sounds produced.

2) A test which is intended to see if a subjects's ability to correctly
identify which cable is in use is affected by a specific form of 'non
audible' infomation - e.g. by being able to 'see' which is in use.

These would be different experimental aims, so it would be reasonable to
employ different experimental test arrangements for them. Thus a test may
be suitable for (1) but not (2) or vice versa. You then chose the test
that is relevant. Ideally, one tries to deal with one issue at a time, so
to avoid complexity and extra risk of unexpected problems, avoid a test
that tries to combine both unless there is a pressing reason to the
contrary.

[snip]

Also, they may come to suspect this whilst being tested, so cease to
rely on what is said, and this might also cause any actual 'bias' to
vanish.

If this happens then it should be visible in the results and in
discussion with the subject at the end.

That assumes that we have collected enough data to do so with statistical
reliablily *and* that they suddenly decided this with confidence at some
point. Given that we don't know if/when this may have happened, or when,
it would make for much longer test routines, and much harder to analyse
with a given level of confidence (in statistical terms). This in turn may
well make other problems - like fatigue or impatence or varying auditory
ability for other reasons - worse. Hence I would say this should be
avoided for such reasons.

So such an approach would be quite difficult to validate.

Validate? I cannot see anything to validate but this may be a difference
of terminology.

I will try to explain using your previous posting:


On 13 Jan in uk.rec.audio, andy wrote:
Don Pearce wrote:
No, please try again. I really didn't understand how sighted bias
could be factored out of this situation.

Consider an experiment where the subject sees the 4 pairs:

A B, A B, A B, A B

but actually hears the 4 pairs:

A B, B A, A A, B B

If the subject claims to hear (D=different and S=same):

D D D D then 100% correlated with sight and 0% corrleated with sound

D D S S then 0% correlated with sight and 100% correlated with sound

S S S S then 0% correlated with sight and 0% correlated with sound

The first indicates the subject is biased by sight and cannot tell from
the sound, the second indicates the subject is not biased by sight and
can tell from the sound and the third is not biased by sight but cannot
tell from the sound.

The problem is with the reliability and confidence of the 'indications' or
'implications' you draw.

Since you are symultaneously trying to determine *two* factors then the
test routine will tend to have to be longer, and the analysis more extended
to get a given level of confidence.

You then have the problem that the person may or may not accept what they
"see" as meaning anything, and this may vary during the test in a way you
would have to hypothesise about after the event. The conclusion that the
result was X% sight and Y% auditory above assumes the factors remain
constant throughout the set of decisions they made. Given that you have no
way to measure this, you have to make an assumption about it, which may be
wrong. How do you check - by measurement - your assumption? Without this
you may have injected into your process an assumption you can't validate.


This can be explained in another way by considering - what actual question/
are you either explicitly or implicitly asking of the test subject? Ann
what statement of 'information' perhaps sic accompanies it?

If your 'question' has the form "I am telling you that cable A is being
used. Am I lying?" Then their focus may be on trying to read your
expression or tone of voice to see if you are lying or not, and not
on the actual sounds. It also immediately alerts them to the probability
that your statement may be false, thus spoiling the chance that being
told "cable A is being used" will have any effect at all.

The point here is that what you may wish to know is "Does *believing*
that cable A is being used cause them to think there *is* an audible
difference even if none actually exists?" In normal use they would
reliably know which cable was in use, and would not need to decide
if this was an error. But the form of your test, and the statement/
question you present changes this, so the pre-conviction regarding
which cable *is* being used is absent. Hence your test is not
of what you may be interested in examining, and the test method
is not valid for the hypothesis you are actually interested in testing.


Obviously one needs to take more samples to get a reasonable level of
confidence in the results. The required number of samples will also be
signficantly more than that required with blind testing for the same
level of confidence.

Indeed. However the real problem is that you also then have to try and fit
a result where the degree of any influence due to sight may alter during
the collection sequence, and in a way you cannot independently measure.

As presented, the experiment is also almost certainly too naive to get
accepted by most subjects. Complexity will need to be added.
Nonetheless, hopefully, the principle is clear.

I would say that 'complexity' is actually part of what makes some people
wish to reject a test. Hence your proposal may be seen as having loopholes
and hidden assumptions that then cause people to say it is invalid.

It also removes the ability of the 'ABX' methods where the test subject can
refresh their impression of A or B when they wish (with confidence that
they *are* choosing A or B). This may be very useful in dealing with any
tendency either to 'forget' a sound, or time-dependent drifts in perception
or in other common-mode factors of the test arrangement.

Slainte,

Jim

--
Electronics http://www.st-and.ac.uk/~www_pa/Scot...o/electron.htm
Audio Misc http://www.st-and.demon.co.uk/AudioMisc/index.html
Armstrong Audio http://www.st-and.demon.co.uk/Audio/armstrong.html
Barbirolli Soc. http://www.st-and.demon.co.uk/JBSoc/JBSoc.html

Don Pearce wrote:

On 13 Jan 2006 11:29:49 -0800, "andy" wrote:



I repeat. Sighted tests are valueless.


How about this proposal then?

Let's assume that a given "brand A" cd player, amp, cable, whatever is
said to have a given type of a sound in a given system. In the same
system, a given piece of brand B alternate equipment is said to have
quite the opposite style of sound. This determination is done by us the
audiophools, the retail business sellers, ie, audio con men, or the
EEMHE (the evil equipment manufacturers of high end).

Now, we take 1000 people (this could be done in a university) and have
them listen to this same system for separate sessions in two different
days. One session for brand A, the other for brand B. These sessions
would be in a comfortable room well arranged, acoustically treated, etc.
The subjects would of course know what equipment is being played, they
would be comfortable, they would be able to listen together with a
girl/boy freind, examine the equipment, drink, even eat a snack or two
during these sessions. And they would of course get paid for their time.

At the end of each session they get a sheet of multiple choice questions
designed to help them to characterize the sound they've heard. The
characterization of consensus by the overall majority would either
validate or invalidate the manufacturers', audiophools, etc, claims that:

a) Their equipment, brand A, sounds in a given way
b) The rival equipment, brand B, has a totaly different, non-desirable
type of a sound ass compared to brand A



Note that this proposal does not involve gauging out the eyes, stripping
one of knowledge, transforming an act of pleasure seeking into a puzzle
solving, guns being pointed to the head (tom nousiannes suggestion to
bring out the honesty and sharpen the sense of a subject being tested!)
or some such revenge being taken or punishment inflicted to these
over-indulgent, rich hedonists of audiophoolery. So in this context it
should be totally irrelevant, useless, valueless, bad, foolish, you name it.