In article , Malcolm
wrote:
On Fri, 28 Dec 2007 10:08:27 +0000, Jim Lesurf wrote:

In article , Malcolm


The problem is that many (most?) such tests fail to reliably
distinguish between A and B. One cannot, in that case, say that A and
B are the same. That's a logical fallacy.

Indeed. That is why the conclusion would not be that they are "the
same". Only that the evidence from the test indicated them to be
audibly indistinguishable when compared.


Absolutely correct - as long as you add the rider that they are audibly
indistinguishable under the conditions of said test - whatever they may
be.

Indeed. And we can then go on to link this with the other evidence we have
from a range of similar tests with various test conditions, choices of test
subjects, etc. So by using an array of various tests we extend the extent
of the types of situations and subjects for which we can take the
conclusion to be reliable as a guide.


So the "flaw" seems to be that you wish to draw inappropriate
conclusions from a test intended for another purpose. This isn't a
"flaw" in the test, but in your inappropriate use of the results. The
"logical fallacy" is in the way you present an inappropriate
conclusion and bypass the appropriate one. :-)


No, it doesn't matter what the purpose of the test is

Actually it is vital to understand the purpose of the test, for the reason
you posting illustrated - that otherwise you can draw inappropriate and
thus misleading 'conclusions' which are not due to any "flaw" in the test.

- the results
(such as they are) stand. The only only "logical fallacy" I'm referring
to is that those that say that a test that fails to find a difference
between A and B "proves" that A and B are identical.

Which as I pointed out, it not a flaw in the test, but in the 'result' you
stated as being taken from it.



Thus if the above is the "fundamental flaw" you were referring to,
then I am afraid it is in your understanding, not in the tests. :-)


If listeners in a "test" situation cannot distinguish the sound between
one system and another then one cannot assume that those same listeners
in another situation (for the sake of argument a "home" situation) would
also not be able to distinguish between the systems.

Nor can we assume that they *can* do so simply on the basis that they
express such a belief. What we can do (and people have done) is run tests
in a variety of situations looking for any cases or circumstances that show
themselves as allowing for a change in audibility. So far as I know, the
results we have seem to show no sign that we have evidence for the
assumption that they can do so in a 'home' situation when not being tested.

The problem, of course, is that when said listeners claim such
differences, some "scientists" say that that is nonsense and that if
they (the listeners) hear such differences at home, then they must
surely be able to hear the same differences under test/laboratory
conditions.

Part of the problem here is your vague label of "scientists". This allows
you to attack a grouping of that lable of your choice. But it tells us
little about the evidence, or what people can, or cannot, actually
distinguish by sound. Thus you present a false dichotomy, on one side your
lable "some scientists" say something is 'nonsense', and on the other
people say it is 'true', no doubt.

Yet the situation is that we have no reliable evidence to support the
assertion that people *can* hear 'at home' what they then fail to show they
can hear in a test. What we do have is a variety of test evidence which
shows that in some cases people did distingush one thing from another, in
other cases they did not. In science we base our understanding on evidence,
not on speculations.

If you wish to propose the hypothesis that people can hear 'at home' things
which they can't in other situations then it is open to you to run a
suitably controlled test in a 'home' to see what the evidence then shows.
But so far as I recall, some of the tests *have* been in people's homes,
and in circumstances they felt would be fine for them. So I see no obvious
reason to assume in advance that your test would show a fresh result. But
please do a test and report the details and we can then assess the results
along with examining the details of the test method, etc.

Until then you are speculating beyond (or in conflict with) the body of
evidence we have.

Since the two situations are fundamentally different and that what is
being "measured" is one aspect of human perception there is a bit of a
problem in asserting that the "home" vis-a-vis the "laboratory"
situation will have no affect on the perception itself.

As above. You would need to define this "fundamental difference" in a way
that that then allowed you to test your hypothesis. If you can't do that,
then you are simply offering a vague speculation as a possible 'excuse'.

All kinds of things can be speculated or claimed to be 'possible' in some
vague way. Teapots beyond the moon. Mediums who excuse their failure to
produce ectoplasm at a seance because "The vibrations were not right due to
the presence of skeptics in the room", etc, etc. I am afraid none of that
is science. For that your ideas need to be backed up with a doable test, to
do the test, and then decide on the basis of how reliable the resulting
evidence is as either support for, or refutation of, your hypothesis.

Personally, I think that if a "well" (and there are very few of those)
conducted listening test fails to show a difference between two systems/
components, then the differences (if any) are probably not worth
worrying too much about.

I also have a similar view, but agree with your other implied point that
what may pass unnoticed or be trivial in some situations might matter more
in others. So have to go with the evidence rather than personal views or
speculations beyond the evidence. :-)

However, I see no problem whatsoever in anyone
conducting their own home listening tests and deciding on the basis of
those tests that one item is better than the other. It is the height of
arrogance for anyone to claim that they are "wrong".

Not if we can show good reasons (supported by evidence) that their test did
have a flawed method, or that they failed to do the tests well enough for
the results to have any significant level of reliability. Also not if there
are good reasons to doubt their conclusions have any worth for anyone else,
so may be misleading if given as a 'guide' or a 'review' or 'evidence' for
others. All depends on the details of the case.

The point here is that there are various well-evidenced mechanisms that
can cause someone to percieve a 'difference' which is not the one
asserted by someone making a claim. So although we can't tell them they
'wrong' when they say they heard a difference, we can often have a firm
evidential basis for saying that they are probably 'wrong' in the
*reasons* they assert as being the 'cause' of that difference as their
test/comparison did not deal with these factors which routinely arise.

As I assume you are aware, the literature on topics like the physiology
of hearing does deal with such matters, and this is why academic run
tests routinely take this into account. But casual home listening
normally does not, so can easily lead to quite unfounded 'conclusions'
by those involved.

Ditto for a variety of well established acoustic and physical factors
like slight movements altering the room acoustic, etc.

Speculations about what "might be so" are useful in science, but they have
no real value until tested and then we can decide on the results, not on
how plausible or attractive the speculation sounds. And if a speculation
can't be tested, then it is not 'science'. In such cases we can simply
make use of the simplest ideas that fit the evidence. No need for any
needless 'mechanisms' or 'effects'. Occam.

I am afraid that people believe all kinds of things, often contradicting
each other. So we can't rely on what people assert as their 'belief'.
Sincerity is no warranty of accuracy of what people believe being correct.

Slainte,

Jim

--
Electronics http://www.st-and.ac.uk/~www_pa/Scot...o/electron.htm
Audio Misc http://www.audiomisc.co.uk/index.html
Armstrong Audio http://www.audiomisc.co.uk/Armstrong/armstrong.html