Arny Krueger wrote:
"Rob" wrote in
message
Arny Krueger wrote:

I think, explain the reasoning behind your methods. Is
that so unreasonable?

Why not just ask for what you want, instead of pontificating so much?

The goal of our method was that we wanted to develop a test for audio gear
that was as fair as possible and that could be self-administered.

The reasoning behind our method was to identify as many significant sources
of bias as we could, and manage them with a relatively simple piece of
hardware.


Mmmm. That is the reason for the test, not the approach. This could go
on and on :-)



I have no real desire to turn you on to this, or anything
else, btw :-)
I also have a few issues with method mentioned
elsewhere in this thread. What are they?
I have no 'expert' knowledge of testing protocols in
this context. I would have thought any lay person would
point to:
Environmental variables - light, heat, seating,
audience.
Obvious.

Sample - did you test their hearing acuity?
Read the list of participants. Do you seriously think
that yo would naturally find a lot of people in a group
of musicians, audio engineers and experienced
audiophiles who lacked at last normal ability to
distinguish sonic differences?
Yes - I did read the summary relating to the participants

But, you haven't answered the question.


I felt it was implied in my summary - professional experience, so yes
they could deduce difference to a degree.

It strikes me, and here I lapse into stereotype, that
the people involved were possibly middle aged men?
No, the oldest of them were in their late 40s, the
youngest were in their 20s.

Who by training listen for and expect particular things?

You must have zero respect for musicians, audio
engineers, and audiophiles.

Again, no answer.


I thought that was a rhetorical point, not a question.

Whose hearing is possibly past its best?
You are obviously clutching for straws.

... who have two characteristics (at least) in common -
professional familiarity with audio, and (related) an
element of expectation relating to the results. Add to
this peer pressure (the results matter to them in a way
that they would not matter - thankfully - to a 10 year
old child) and I think I'm right to question method. It
really isn't that difficult.

Again, you've really said nothing, just cast a few unfounded aspersions.


Common sense I would have thought? The method resembles snowballing (not
the sexual version!) - fine in certain circumstances, but I can't fathom
the methodological context here.

Since you won't answer any of my questions...


?! Of course for me to reject your qs would be inexcusable!

We had a thread on the tests of a UK consumer mag (called
'Which?') a while back - their tests 'revealed' audible
differences in CD players and amplifiers. This was fairly
unanimously rejected as unscientific drivel on this NG,
and I did go to the trouble of writing to the magazine
editiors for clarification of their test protocols. They
were far more forthcoming - and aware of limitations -
than you appear to be. Why might that be?

What do you want to know about our test protocols that you can't easily find
out from the sources that have been cited?


It's not really the test protocols, although I've raised a couple of
issues above relating to samples that you've dismissed.