Which? Audio tests
 

I wrote to them recently concerning their testing procedure for audio
electronics. I don't subscribe, although I occasionally leaf through
library copies when I'm supposed to be working :-)

In answer to the Q: do you use blind testing and matched levels, she
said 'yes'. However, she went on to say that it's not always needed
because, as they're completely independent, there is nothing to gain
from recommending any particular model. She also went on to say that the
tests are thorough to replicate several years' use.

I thnk this is generally fine, and nice of them to respond. It's also
rather vague - I think I'll reply asking them to publish the detail of
their tests (level matching, respondent selection, room, other kit,
music and medium used), and suggest that they think about:

Double blind testing, or at least blind testing - they may be
independent, but people aren't, and some may think a Sony is bound to
sound better than an Alba. Sort of reassuringly, their tests rarely
mirror the specialist press (at least on the occasions I've been able to
compare the 2), but I think blind testing might add some rigour.

I think that would do me - I don't really want to know about cables, and
while some technical data (power output maybe) might be useful, I don't
think they have the expertise to measure/interpret the information. Just
knowing that people can reliably and qualitatively differentiate is a
pretty good start IMO.

Just FYI, but if anyone has any comments I'd be interested.

Rob

Which? Audio tests
 

In article ,
Rob wrote:
Double blind testing, or at least blind testing - they may be
independent, but people aren't, and some may think a Sony is bound to
sound better than an Alba. Sort of reassuringly, their tests rarely
mirror the specialist press (at least on the occasions I've been able to
compare the 2), but I think blind testing might add some rigour.

Don't think they blind test - pics show a room full of people and the gear
on a table in front of them. However, they don't normally test vastly
different price stuff at the same time. Nor do they use only enthusiasts -
so it might be fair to assume some don't know Alba from Sony.

Remember they're only really testing for those who haven't a big interest
in the subject - just to give a guide to good value and reliability.
That's why camera, hi-fi and car reviews always get criticised by the
specialist press.

--
*Hang in there, retirement is only thirty years away! *

Dave Plowman London SW
To e-mail, change noise into sound.

Which? Audio tests
 

"Dave Plowman (News)" wrote in message
...
In article ,
Rob wrote:
Double blind testing, or at least blind testing - they may be
independent, but people aren't, and some may think a Sony is bound to
sound better than an Alba. Sort of reassuringly, their tests rarely
mirror the specialist press (at least on the occasions I've been able to
compare the 2), but I think blind testing might add some rigour.

Don't think they blind test - pics show a room full of people and the gear
on a table in front of them. However, they don't normally test vastly
different price stuff at the same time. Nor do they use only enthusiasts -
so it might be fair to assume some don't know Alba from Sony.

Remember they're only really testing for those who haven't a big interest
in the subject - just to give a guide to good value and reliability.
That's why camera, hi-fi and car reviews always get criticised by the
specialist press.

--
*Hang in there, retirement is only thirty years away! *

Dave Plowman London SW
To e-mail, change noise into sound.

The argument that punters may think that a Sony is going to sound better
than an Alba doesn't really hold water, as so many previously good makes,
are now bought-out names used to badge third world junk.

I still come across many people of my age, who think that anything with Bush
or Goodmans on it must be quality gear, because it was when their parents
were buying it. Sadly, anyone actually in the repair business, knows
differently ...

Arfa

Which? Audio tests
 

In article , Rob
wrote:
I wrote to them recently concerning their testing procedure for audio
electronics. I don't subscribe, although I occasionally leaf through
library copies when I'm supposed to be working :-)

In answer to the Q: do you use blind testing and matched levels, she
said 'yes'. However, she went on to say that it's not always needed
because, as they're completely independent, there is nothing to gain
from recommending any particular model.

When she says, "not always needed", does she mean, "but we don't always
bother to use blind methods"? Or does she mean, "We always do that, but
feel it isn't always strictly needed?" It seems an ambiguous answer to me.

The magazine/association may be 'independent', but the individuals coming
to the tests may have their own individual pre-conceptions. Thus one of the
aims of 'blind' methods is to try and ensure they don't bring these to bear
for any reasons other than those which are intended to be relevant.



I thnk this is generally fine, and nice of them to respond. It's also
rather vague - I think I'll reply asking them to publish the detail of
their tests (level matching, respondent selection, room, other kit,
music and medium used), and suggest that they think about:

FWIW I tend to regard any test or measurement methods whose details are
unspecified as being of dubious value. If you don't know exactly how the
results were obtained, how can you assess their reliability?


I think that would do me - I don't really want to know about cables, and
while some technical data (power output maybe) might be useful, I don't
think they have the expertise to measure/interpret the information.
Just knowing that people can reliably and qualitatively differentiate
is a pretty good start IMO.

One snag is that people may give 'reasons' for 'differentiating' which are
actually incorrect. This may mean that their conclusions may not apply in
your or my case as the report implies. Without sufficient details and a
cautious approach this may reduce or negate the value of the reports...

FWIW I still recall an old 'Which?" report which essentially told those who
had no prior understanding that the Quad ESL57 and the Goodmans Maxim
sounded the same and were similar in use... The problem here is an
over-simplification of the findings or characteristics which may mislead
the innocent.

Slainte,

Jim

--
Electronics http://www.st-and.ac.uk/~www_pa/Scot...o/electron.htm
Audio Misc http://www.st-and.demon.co.uk/AudioMisc/index.html
Armstrong Audio http://www.st-and.demon.co.uk/Audio/armstrong.html
Barbirolli Soc. http://www.st-and.demon.co.uk/JBSoc/JBSoc.html

Which? Audio tests
 

Dave Plowman (News) wrote:
In article ,
Rob wrote:

Double blind testing, or at least blind testing - they may be
independent, but people aren't, and some may think a Sony is bound to
sound better than an Alba. Sort of reassuringly, their tests rarely
mirror the specialist press (at least on the occasions I've been able to
compare the 2), but I think blind testing might add some rigour.


Don't think they blind test - pics show a room full of people and the gear
on a table in front of them. However, they don't normally test vastly
different price stuff at the same time. Nor do they use only enthusiasts -
so it might be fair to assume some don't know Alba from Sony.


I'm not sure of the impact of branding either, although I suspect most
people would buy Sony over Alba for the same price.

Remember they're only really testing for those who haven't a big interest
in the subject - just to give a guide to good value and reliability.
That's why camera, hi-fi and car reviews always get criticised by the
specialist press.


Well, their main interest is that they're going to buy something I
suppose, which is pretty big.

So overall you consider their approach 'fit for purpose'?

Rob

Which? Audio tests
 

Arfa Daily wrote:
"Dave Plowman (News)" wrote in message
...

In article ,
Rob wrote:

Double blind testing, or at least blind testing - they may be
independent, but people aren't, and some may think a Sony is bound to
sound better than an Alba. Sort of reassuringly, their tests rarely
mirror the specialist press (at least on the occasions I've been able to
compare the 2), but I think blind testing might add some rigour.

Don't think they blind test - pics show a room full of people and the gear
on a table in front of them. However, they don't normally test vastly
different price stuff at the same time. Nor do they use only enthusiasts -
so it might be fair to assume some don't know Alba from Sony.

Remember they're only really testing for those who haven't a big interest
in the subject - just to give a guide to good value and reliability.
That's why camera, hi-fi and car reviews always get criticised by the
specialist press.

--
*Hang in there, retirement is only thirty years away! *

Dave Plowman London SW
To e-mail, change noise into sound.


The argument that punters may think that a Sony is going to sound better
than an Alba doesn't really hold water, as so many previously good makes,
are now bought-out names used to badge third world junk.


I suspect it's more of a franchise than simple rebadging for people like
Sony.

I still come across many people of my age, who think that anything with Bush
or Goodmans on it must be quality gear, because it was when their parents
were buying it. Sadly, anyone actually in the repair business, knows
differently ...

Arfa


That's really my point - some people who see 'Bush' might associate that
product with decent sound and engineering or yore, and somehow express
that sentiment in tests.

Rob

Which? Audio tests
 

In article ,
Rob wrote:
Remember they're only really testing for those who haven't a big interest
in the subject - just to give a guide to good value and reliability.
That's why camera, hi-fi and car reviews always get criticised by the
specialist press.


Well, their main interest is that they're going to buy something I
suppose, which is pretty big.

So overall you consider their approach 'fit for purpose'?

Yes. I've invariably bought on their recommendation for things like
domestic appliances and have been satisfied with that choice. Their user
surveys on reliability etc seem pretty accurate too. Even with cars. ;-)

Don't think I've ever seen anything on Hi-Fi that's been of interest to
me, though. I don't tend to change things often.

--
*Time is fun when you're having flies... Kermit

Dave Plowman London SW
To e-mail, change noise into sound.

Which? Audio tests
 

In article ,
Rob wrote:
That's really my point - some people who see 'Bush' might associate that
product with decent sound and engineering or yore, and somehow express
that sentiment in tests.

They'd have to be very old. It's a very long time since Bush made decent
gear - if ever. And never decent Hi-Fi.

--
*Filthy stinking rich -- well, two out of three ain't bad

Dave Plowman London SW
To e-mail, change noise into sound.

Which? Audio tests
 

"Dave Plowman (News)" wrote in message
...
In article ,
Rob wrote:
That's really my point - some people who see 'Bush' might associate that
product with decent sound and engineering or yore, and somehow express
that sentiment in tests.

They'd have to be very old. It's a very long time since Bush made decent
gear - if ever. And never decent Hi-Fi.

--
*Filthy stinking rich -- well, two out of three ain't bad

Dave Plowman London SW
To e-mail, change noise into sound.

No, sorry Dave, gotta disagree with you on that one. When they were still
trading as Rank Bush Murphy, and finally abreviated to RBM up to the latter
part of the seventies, they made one of the best, and most innovative CTV
chassis that there has ever been. I worked for many years on the A823
chassis, and without any shadow of a doubt, when correctly set up, it was
capable of producing a better, more natural picture, than any competitor
that it had.

Going back a little before that, the range of transistor radios that they
did ( I've still got the one I had as a kid ), and before that, their range
of table top valve radios like the DAC10, were superb performers, and
quality built. They represented the epitome of good British engineering
design and quality, and in my opinion, maintained that right up to their
final demise, when the company name was sold off, and then became synonymous
with the crap that carries the name now.

Arfa

Which? Audio tests
 

In article ,
Arfa Daily wrote:
That's really my point - some people who see 'Bush' might associate
that product with decent sound and engineering or yore, and somehow
express that sentiment in tests.

They'd have to be very old. It's a very long time since Bush made
decent gear - if ever. And never decent Hi-Fi.


No, sorry Dave, gotta disagree with you on that one. When they were
still trading as Rank Bush Murphy, and finally abreviated to RBM up to
the latter part of the seventies, they made one of the best, and most
innovative CTV chassis that there has ever been. I worked for many
years on the A823 chassis, and without any shadow of a doubt, when
correctly set up, it was capable of producing a better, more natural
picture, than any competitor that it had.

Going back a little before that, the range of transistor radios that
they did ( I've still got the one I had as a kid ), and before that,
their range of table top valve radios like the DAC10, were superb
performers, and quality built. They represented the epitome of good
British engineering design and quality, and in my opinion, maintained
that right up to their final demise, when the company name was sold
off, and then became synonymous with the crap that carries the name now.

My parents had a 21 inch Bush set bought new about '60 - and one of the
most expensive sets on the market. No black level clamp, wouldn't resolve
3 megs on the test card and had crappy sound from two tiny loudspeakers.
It was also less than reliable.

But the point was about audio. Leak and others were making decent gear in
the '50s. Bush never bothered.

--
*Change is inevitable, except from a vending machine.

Dave Plowman London SW
To e-mail, change noise into sound.