Voodoo HiFi Experimental Design Masterclass

February 4th, 2006 by Ben Goldacre in bad science | 341 Comments »

Ben Goldacre
Saturday February 4, 2006
The Guardian

Folks are the same the world over. Now, as far as I know, I’m the only person in this room who’s had violent threats – in writing – from angry alternative therapists. If that’s not true, of course, I’d love to hear your story, but either way, I feel pretty well placed to assess the emotional temperature behind the stranger end of popular belief.

So let’s talk about the high end hi-fi industry. I wrote about their very expensive power cables last month, ranging from £30 to a whopping £1,800, for what is, after all, a kettle lead to connect your stereo to the three pin power socket in the wall. The various manufacturers claim that their cables will filter out radio frequency interference in the power cable, and that this will improve the sound. I doubted this, and the outpouring of bile that was subsequently vomited in my direction (references on badscience.net) surprised and delighted even me. But what was most interesting, to students of this stuff is that the angry outbursts came primarily from the natural constituency of Bad Science readers. Several were deeply wounded. Homeopathy was one thing, they said, but this time, I had clearly got it wrong.

And that was when I started to notice the frightening similarities between the thought processes of the alternative therapy fans and the hi-fi freaks. Both make an appeal to personal experience, as the highest and most valid form of measurement; both use mystifying, scientific-sounding terminology in their publicity material; and both use the appeal to authority.

But the most striking parallel is the widespread notion in the hi-fi community that blinded trials – where you ask listeners to identify a cable without knowing if it’s cheap or expensive – are somehow intrinsically flawed. This is exactly the card that the alternative therapy community have been playing, almost since blinded trials were invented.

I give you the editor of Stereophile, a respected hi-fi magazine of 33 years standing. He’s talking about blinded tests on amplifiers: “It seems,” he says, “that with such blind listening tests, all perceived subjective differences … fall away … when you have taken part in a number of these blind tests and experienced how two amplifiers you know from personal experience to sound extremely different can still fail to be identified under blind conditions …” Now I’m getting worried. Here comes the money shot. “… then perhaps an alternative hypothesis is called for: that the very procedure of a blind listening test can conceal small but real subjective differences.” Ouch. “Having taken part in quite a number of such blind tests, I have become convinced of the truth in this hypothesis.” What voodoo is this? If there is a difference to be heard, then you will hear it.

The manufacturer of my expensive power cable has assured me that it will have an impact on the sound of an expensive CD player, or a cheap one, and that it will affect optical outputs as much as normal phono outputs. So I propose, as a pilot study, to place, on badscience.net, two files, clearly labelled, one recorded from the CD player powered by the expensive cable, one powered with a normal kettle lead. There will also be a third file, and you can vote on whether it is the expensive cable, or the cheap one. This is called an A/B/X trial.

You can do what you like to identify them: burn it to a CD, listen with headphones, examine the raw data, whatever. But the catch is this: it’s a palaver, and I’m not going to bother, if the hi-fi buffs are just going to bleat about how unfair the test is. So this is your chance to participate in the design of an experiment. Post your thoughts on badscience.net. I want everybody to be happy.

· Please send your bad science to bad.science@guardian.co.uk

Please post your thougts about how best to design the experiment below. Audiophiles, I know you will find this difficult, but I’m not sure we want to have another long discussion about whether this power cable makes a difference, you can have that out here for example… it might be good to keep this thread for discussing how to test whether there is a difference.


++++++++++++++++++++++++++++++++++++++++++
If you like what I do, and you want me to do more, you can: buy my books Bad Science and Bad Pharma, give them to your friends, put them on your reading list, employ me to do a talk, or tweet this article to your friends. Thanks! ++++++++++++++++++++++++++++++++++++++++++

341 Responses



  1. Martin Burley said,

    February 4, 2006 at 3:41 am

    The Stereophile editor’s comment seems to me to back up the theory that any differences are down to subjective factors, an ‘expensive = good’ equivalent of the placebo effect… perhaps this could be called ‘the cashebo effect’? :-)

    Personally I’m all for a blind test anyhow, but it could be even more interesting to do a four-part test as well, where you tell *do* people which cable is being used. Except, half the time, you lie (the participants are not aware of the test design, since if they were aware they might be lied to, that would in effect reduce it back to a blind test).

    So you have results for ‘expensive (true)’, ‘expensive (false)’, ‘cheap (true)’ and ‘cheap (false)’. Comparing ‘expensive(true)’ against ‘expensive(false)’ and ‘cheap(false)’ against ‘cheap(true)’ would show which cable (if either) is actually better. Comparing ‘expensive(true)’ against ‘cheap(false)’ and ‘expensive(false)’ against ‘cheap(true)’ would show how strong the cashebo effect is.

  2. Horace said,

    February 4, 2006 at 4:10 am

    I think a simple ABX test is the way to go. Online is fine, if the manufacturers do claim that it will affect the outputs of the cd player.

    The criticism of this method would be that subtle differences between the sounds may be harder to detect on some listeners listening systems, if they are more rubbish, which would reduce the ability of the study to identify a true difference.

    Having said that, because the test is online, the number of listeners will be much much bigger, compared with traditional tests, where all the listeners have to travel to one room. In fact, those tests only have, maybe, a dozen listeners, so the number of listeners here will be several orders of magnitude larger, and this will hugely increase the statistical power of the study.

    I also like the way that each person serves as their own control, comparing the sound of cable A, cable B, and cable X, but all on their own systems.

    Fine. Elegant.

  3. Andrew Rose said,

    February 4, 2006 at 7:13 am

    If there are any differences in the audio files they should be very easy to find, assuming they’ve been recorded in the same manner and not compressed. Simply open the files using audio editing software and invert one of them. Then line them up in a multitrack mixing session so that they run exactly side by side – again, not difficult, even to the nearest 1/44100th of a second (i.e. CD sample resolution).

    If there is no difference between the files then they’ll be perfectly out of phase and will cancel each other out – the result is silence.

    If they are different this will be audible. The tracks can be mixed together to create a new ‘difference’ file – if necessary this can be amplified to illustrate more clearly what has been found.

    Certainly this is a far more scientific approach than ABX listening tests – instant irrefutable proof that the cord does or doesn’t make any difference to the audio output of a CD player.

    Just one thing – and this is where the hi-fi freaks will shoot you down if you’re not careful (and of course they will anyway) – you’ll need to take great care with the recording quality of these files. You’ll be told on the one hand that the differences are clearly audible to anyone who cares to listen, but when the test fails to show this up you’ll then be told that your analogue to digital converters weren’t sensitive enough to pick this up.

    If you need any help with the recording side I’d be happy to help…

  4. Peter Ashby said,

    February 4, 2006 at 8:20 am

    I agree with Andrew Rose. While getting people to rate the files can be done it will suffer from method analysis, giving the audiophiles reasons to disregard it they can be happy are ‘scientific’. After all in science it is common to disregard a study if you feel the methods are too shoddy to be sure of the conclusions.

    So despite my initial thought that it might be fun, I agree with Andrew’s method of comparison. After all our ears are not that sensitive that the difference cannot be analysed.

    Of course there is nothing to stop you doing both tests, use the difference analysis to underpin the subjective tests. So if they do come out non-random in favour of the expensive lead you can look for the difference. After all in a good experiment you should be prepared to reject your hypothesis, and IF your hypothesis is wrong it would be nice to see a trace of the difference that is being heard.

    Of course since Andrew’s idea is hardly rocket science the manufacturers could have done them themselves. Think of the marketing opportunities this would bring if they had absolute proof of the difference.

    Peter

  5. JohnD said,

    February 4, 2006 at 9:22 am

    Ben,
    A double blind controlled on-line test? Bit hard to control for all the other random factors.

    Usually, a trial will be designed to control known variables, as uncontrolled they would produce more randomness in the results. In the online trial suggested, each subject will experience the same conditions for the test and control, but the hypothesis is that there is a difference. It would be easy for the pro group to argue that the quality of the user’s PC, audio equipment, even their own power lead, will obscure that subtle difference.

    John

  6. hairnet said,

    February 4, 2006 at 9:56 am

    the problem with checking for differences only tells you there is a difference, not whether it is ‘better’ or not, after all isnt that the claim?

  7. Dean Morrison said,

    February 4, 2006 at 10:23 am

    An alternative to ‘Cashebo’ ( which I think is great) might be the: ‘Emperors new clothes’ effect.

  8. Chris said,

    February 4, 2006 at 10:27 am

    Where are these audio files? I can’t seem to find them.

    Provided they are recorded using the same method, with the same technology, I can’t see why we wouldn’t be able to tell the difference, if there is one.

  9. ff said,

    February 4, 2006 at 10:54 am

    Speaking of dodgy claims perhaps you should turn on to page twenty of todays guardian “crystal clear tv” guardian reader offer. Sealed aeiriel in a plastic case the size of a playing card deck. For £14.95 the guardian promises it will outdo large indoor aeirels and even replace my outdoor one with perfect reception! I think you should check that out.

  10. Casper said,

    February 4, 2006 at 11:10 am

    I was going to say that: “I’m all for not bothering with any tests. Hi-fi buffery is a harmless, if expensive pursuit. It will only hurt participants’ wallets.”

    On reflection, any opportunity to introduce some critical thinking / analysis into the current culture of irrational belief in almost anything would be welcome.

    Post those files.

  11. Steve Denning said,

    February 4, 2006 at 11:18 am

    Mains borne RF interference used to send my Quad valve amp (designed like most old amps in an era of cleaner mains and hence with minimal mains filtering) into orbit when used with a moving coil cartridge and battery powered preamp (usually when the heating pump switched off). This was effectively stopped using an enclosed mains filter reused from an old expensive piece of electrical equipment – as most reasonable quality suppressed socket strips sold by Belkin and similar manufacturers for use with computers use a similar filter I can’t help feeling that any mains lead money might be as well spent on one of these should one’s new hifi suffer from excessive cost cutting in the power supply department. I would have thought that any listener claiming to be able to hear the difference between two power cables would have equipment with a well designed power supply anyway.

  12. Andrew Rose said,

    February 4, 2006 at 11:26 am

    In reply to ‘hairnet’ – that’s the next stage – whether there’s a qualitative improvement with one over the other. I’d like to know first of all whether there’s any actual difference whatsoever in the audio signal – although I have the studio equipment and expertise to test this, I really don’t have £1800 to blow on mains leads! ;-)

  13. TxB said,

    February 4, 2006 at 11:27 am

    For god’s sake, leave the poor Hi-Fi affectionadoes alone will you!

    Homeopathy is the placebo effect wrapped up in ‘snake-oil’ and mumbo-jumbo — that is, it is all about deception or even self-deception. It has no basis in fact at all. Hi-Fi is about reconstructing and maintaining an analog signal as close as possible to the original. There is nothing ‘unscientific’ about that effort — after all that is what our brains do with the information received by the senses (noisy unclean data which the brain works hard on to re-construct a ‘pure’ original).

    Blind or double-blind listening tests completely miss the point — making Mr ‘BadScience’ himself guilty of the bad science he accuses others of. Anyone who has even briefly looked at the Hi-Fi market would know that to most people even the very bottom end (not junk bottom end, but hi-fi bottom end) component is to the normal person entirely indistinguishable from the highest-end component. Even experts would have difficulty I think. For example, even audiophiles with acute hearing would have difficulty distinguishing a 44.1/16 audio track (CD quality) from a 96/24 audio track (HD quality). Despite this audiophiles want to go right up to 192/24. Not because you can actually hear the difference, but because you want the best reproduction you can possibly have.

    Waht many very high-end components do in addition to accuracy is to subtly ‘colour’ the sound — that is tweak it for the various kinds of expert ear, tweaks which are right on the edge of perception and only (sometimes) perceptible to the trained ear. That might sound like mumbo-jumbo, but in how we hear sound varies in the population. To give a crude example: I find computer fan-noise very annoying, even if its barely audible. Most people don’t notice it at all. If I am subjected to even quite small amount of fan noise not only do I hear the actual noise but my ears start to ‘ring’ and I ‘hear’ a high pitched whine even if I turn the fan-noise off.

    Another fundamental error Mr ‘BadScience’ makes is that of the rather ludicrous ‘pilot study’. This is absolute nonsense in addition to having no scientific value. Audio recordings (presumably with a cheap microphone at a low bit rate!!!) work in a way fundamentally different to how the human ear works. Those things that high-end audio-philes treasure are so subtle that while they are on the edge of human perception are invisible to a run of the mill microphone inexpertly used. Microphones are in fact an entire subject of study in themselves and its quite a complicated (and expensive) matter to take a high quality audio recording. Those in the industry (ie. technicians, not enthusiasts) spend thousands of pounds on their mics … and using them is more of an art than a science.

    It may well be that in this quest for signal purity that some manufacturers exagerate, mislead or are downright dishonest. Some people might even be taken in by this. But so what? What Mr ‘BadScience’ does not seem to acknowledge is that it just about every Hi-Fi affectionado (including Mr ‘BadScience’ himself I presume) wants to demonstrate to his peers (sadly there are no female hi-fi addicts, or very very few of them at any rate) is the awesomeness of his system. Its hard for the lay-person to see with one glance the amazing ability to reconstruct sound -of one’s system – or even to hear that the sound is exactly as it was played or exactly as the artist intended. It is on the other hand easy to impress with ‘brushed’ aluminum surfaces, designer labels, and yes glossy hand ‘braided’ power leads. That desire makes the hi-fi affectionado susceptible to the siren calls of manufacturers with products that have an impressive appearance but dubious technical claims.

    On the whole, however, Hi-Fi affectionadoes are a reasonable lot and generally considerably more scientifically-minded that the average person. Most ‘ridiculously expensive’ equipment therefore not only looks impressive but is made to an exacting technical standard. Many high-end manufacturers are themselves affectionadoes who care a great deal about the ‘science’. This means on the whole the products do what they claim to do. As with most things, it is possible to find exceptions of course.

    Incidentally, on the issue of power leads, when my central heating (or various other ‘noisy’ appliances) turns on my audio system cuts out for about 1 second, which can be quite annoying. If I was a CEO with a multi-million £ salary my guests would probably think ill of me if my audio were to cut out during a screening of Citizen Kane (“R….d!”). Would £1,800 be too much to solve that problem? I think not! As it is, I simply turn down my central heating.

    Perhaps next week we will see a ‘badscience’ column about a certain columnist’s £1000 ‘organic mountain-top split hollow hair’ suit or ‘space-age self-shining water repellant’ £100 shoes — a colossal waste of money when you can buy a £30 Tesco suit and a £3 Lidl pair of shoes, with absolutely no appreciable ‘scientific’ benefit from the former over the latter (and is there really a chinese peasant who splits only the hairs of the finest autumn goat? I doubt it.). Perhaps he can post pictures on his web-site with both and let readers vote on which he will wear!

    Thomas.

  14. drk said,

    February 4, 2006 at 11:35 am

    The manufacturers of this “audio snake oil” will never submit to a simple A/B/X test because the results are easy to interpret without resorting to fancy statistics.

    If you can find something more complicated where “our experts” can endlessly debate “their experts” while attracting media attention – then I guess they will go for it.

  15. Alun said,

    February 4, 2006 at 11:58 am

    Not because you can actually hear the difference, but because you want the best reproduction you can possibly have.A difference that makes no difference is no difference. If it has no audible difference then the purchase is foolish. If it has no measurable difference then the purchase is purely indicative of the buyer’s insecurities. It’s unlike comparing suits or shoes because, call me cynical, I think you might be able to distinguish between a £100 pair of shoes and a £3 pair.

    A difference to the experiment I would make is to have more than one comparison file. The cable might make little difference to a file of grunge where the music is made partly from noise. Off the top of my head I’d have 30 seconds each of Classical, Opera, Rock, Pop, Dance, Jazz, Kraftwerk and a sine wave at middle C. That way it’s a little less a test of the listener’s familiarity with a style of music.

    It may well be that in this quest for signal purity that some manufacturers exagerate, mislead or are downright dishonest. Some people might even be taken in by this. But so what?This might be a bad thing because a small company with a good product would be muscled out of the market by a dishonest company with a bigger marketing budget? This isn’t a problem for the hi-fi enthusiasts that are far more interested in demonstrating to their friend that they’re numerate enough to work out which system is more expensive than rather than demonstrating their appreciation of sound. But hi-fi fans who do think sound is an important part of the hi-fi experience might be a bit miffed by ‘experts’ complicit in misleading the public.

  16. Roy said,

    February 4, 2006 at 12:02 pm

    Great idea – although we all know it will ultimately fail to make a difference because what you’re talking about here is belief. For audiophiles it is exacerbated by belief based on genuine science, but without genuine understanding of the practical reality of that science.

    In other words, these people think their ears can detect differences that anyone who knows about electronics will know you can only measure on an oscillosope.

    In most cases they maintain that they can actually hear the differences in how the same electrical current conducts through two different pieces of wire, which is palapable nonsense.

    However, it is an interesting study in mass self-delusion.

    Great stuff, where do I get the file?

  17. Lowk said,

    February 4, 2006 at 12:06 pm

    It really isn’t valid to use just 3 files. That is a single data point with lots of pseudoreplication – certainly not a good idea in experimental design. A little random asymetry in the recording (which there is certain to be) will throw the whole thing off.

  18. Briantist said,

    February 4, 2006 at 12:30 pm

    The best way to do this experiment is using the method I saw at the Quad company. They used a soundproof room to test each of their manufactured electostatic speakers against the reference one. They do this by having the speakers face each other with a microphone in the centre. Any signal (of any kind, frequency or duration) can be fed to these speakers, one out of phase from the other.

    The soundwaves cancel each other out at the measuring point. This can be recorded from the microphone as silence, any imperfections are heard as noise.

    This test set-up would, in my opinion, be a much better way to test any specific lead or cable. There would be no need for a subjective measurement, just a recording of silence or not, you can watch for a waveform on a ‘scope.

  19. Ian B Gibson said,

    February 4, 2006 at 1:01 pm

    After reading what TxB has to say, I’d definitely name it the ‘Emperors New Clothes Effect’ – only discerning people can hear the difference, not your common low-end hi-fi listener! Therefore, let TxB choose the participants in the double-blind study, so there can be no doubt that only those with the requisite auditory capabilities are involved.

  20. TxB said,

    February 4, 2006 at 1:11 pm

    A difference that makes no difference is no difference. If it has no audible difference then the purchase is foolish.
    Why buy a Bugatti Veyron (1000hp, 250km/h) when a Ford Mondeo will drive at the 110km/h speed limit just as comfortably & safely? At the legal speed limit there would be no appreciable difference between the $1,000,000 Bugatti and the $10,000 Mondeo. Are Bugatti owners fools?

    The answer is in 2 parts:
    (1) Probably most owners would never want to or be able to drive at anywhere near 250km/h. BUT The Veyron can drive 250km/h, a fact that will warm heart of any enthusiast.
    (2) If you open the hood of the Bugatti you will find gleaming engineering excellence. Under the hood of a Mondeo you will find poor design and shoddy workmanship.

    There is a difference with genuine high-end audio equipment just as there is a difference between Mondeo & Veryron, its just that there isn’t any practicalbly appreciable difference. Put the Veryon on a test track and in the hands of a race car driver and the difference is as clear as day. Connect test equipment with the suitable sensitivity to your high end hi-fi and you can see the difference clear as day as well.

    As an example, would you buy an amplifier that produces a terrible whine at 22051hz? Its completely inaudible to most humans. One in a million however will be appalled by it, and it will play havoc upon any visiting dogs or cats. I think it would be foolish to buy such a device, even if the noise was completely inaudible.

    Thomas.

  21. David Hawthorne said,

    February 4, 2006 at 1:21 pm

    I was once one of these over-moneyed people who wanted the best hifi, though I kept it within reason – my system was laughably budget at maybe £1000, bought in dribs and drabs over a period of a year.
    I went to a specialist hifi dealer to buy a good CD/amp cable. The dealer couldn’t provide an A/B direct comparison – I had a 5-minute gap while he exchanged the bell-wire for the £35 cable. I *thought* *maybe* I could hear a *slightly* deeper bass, and I bought the cable. The important thing to me at the time was that I simply would not have enjoyed the system knowing that I didn’t have the best kit I could afford.
    And is that not what it comes down to – satisfaction? I’m sure a lot of you have read “Zen & the Art of Motorcycle Maintenenace” – it says it better than anything I could say, though at much greater length…
    I’m much better now, by the way.

  22. Patrick Caldon said,

    February 4, 2006 at 1:22 pm

    While the aims here are noble, the pilot will be pretty useless if the recordings are made with home pc recording equipment. The noise of low quality analog to digital conversion and low quality mikes would drown out any effect.

  23. Ben Goldacre said,

    February 4, 2006 at 2:01 pm

    I just wanted to correct on a single factual matter. The sound files will be a perfect copy of the output of the CD player, the signal will never leave the digital domain: the manufacturer claimed specifically that the power cable will also affect the optical digital outputs, so I will take the optical output of the CD player into the optical inputs of a sound interface and post uncompressed Wav files (not MP3s, of course) containing a faithful copy of this data, which will be playable on any computer and burnable to CD if you wish. The optical interface will itself, in addition, be studio quality (made by RME) even though this is a bit belt and braces.

  24. Briantist said,

    February 4, 2006 at 2:02 pm

    I’ve had people in high street stores try to sell me cables to “improve” the SPDIF digital signal down a phono-phono cable, and “digital TV signal boosters” that will supposedly improve Freeview picture quality, neither of who knew what the phrase “bit error rate” meant.

  25. Peter Curran said,

    February 4, 2006 at 2:03 pm

    I am generally in agreement with objective and scientific testing, double-blind trials etc. and consider myself squarely in the sceptical category, however, measurement isn’t everything: there is a level of subjective experience that (currently) is beyond measurement.

    For example, the Scotch whisky industry, in spite of all its technology and the formidable scientific qualifications of its specialists, is dependent on the ‘nosing panel’ in each distillery to make the ultimate judgement on the product. The ‘nosers’ are selected purely on their innate ability to make fine distinctions that the instrumentation cannot, and they can come from any level in the distillery hierarchy, from labourer to senior manager. The nose is, of course, a superbly sophisticated measuring instrument, as is the mouth, palate and tongue.

    Similarly, the role of the ear cannot be too easily dismissed in music appreciation, and the fact is, an experienced listener hears things that the less experienced one does not.

    I think the first question to hi-fi buffs is – what kind of music do you listen to?

    I have met those who listened only to the limited tonal palette of rock and pop, or who didn’t listen to music at all, but to train noises!

    However, if the hi-fi buff is a dedicated music enthusiast, is listening to a wide range of instruments (especially those with complex harmonics in the tone, say the clarinet) and can readily distinguish the inner voices of a musical ensemble, then his or her view should be respected when they challenge to verdict of the instruments. Listening to music is a complex emotional, physical and intellectual pursuit, and it is not easy to reproduce the total listening environment in scientific tests.

    Challenging such people’s private experience is like setting a test meter to analyse an orgasm – it will have something to say, but little of value.

  26. Alison Macpherson said,

    February 4, 2006 at 2:15 pm

    Hi,

    I was employed as an Electronic Engineer in Metrology (test and measurement) for a number of years. Hi-Fi & audio is notorious for non scientific claims of “quality” which are impossible to quantify through best practice electronic measurement of signals. Factors such as distortion & SNR can be quantified easily through the use of calibrated test and measurement equipment, however these figures rarely backup manufacturers claims of “quality”. This may be understandable in that the last part of any Hi-Fi system is an acoustic driver moving air, these movements being picked up and interpreted by an organic and subjective system, whose actual ability to interpret “distortion” (or quality) is impossible to quantify and almost certainly widely variable from individual to individual. What I can say for sure is that from electrical signal input to electrical output – and in a reference environment with (characterised) audio output and pickup devices, negligible actual measurable difference exists in performance quality from mid-range from high end audio systems, and certainly well outwith a persons ability to detect and explain. Hi-Fi quality is the the organic subject’s perception thereof, coloured by their inherent cognitive dissonance.

    Finally having designed built and maintained power supplies in a variety of applications, I find it laughable that the Hi-Fi vendors who offer the highest of specification Hi-Fi equipment could claim a 1-3 metre cord would affect the DC output of a well designed power supply unit contained in their products. The basic design techniques of any power supply unit allow for the suppression of noise and voltage fluctuations in the domestic mains supply to a very high level – this is not rocket science (ouch) but standard practice. The mains supplied to any domestic (or industrial) user is notoriously dirty, and allowed to be so, therefore the techniques used in PSU (Power Supply Unit) design to create clean DC outputs employ filtering and stabilisation which extirapte the little inductive and capacitive components of any “last leg” mains lead.

    Had to rant – hope this makes some sense…

    By the way I would be happy to participate in blind testing, as long as there is a buffet….

  27. Briantist said,

    February 4, 2006 at 2:19 pm

    The Scotch whisky industry’s job is not transferring zeros and ones from one place to the other though. Binary leaves no room for things “easily dismissed”. They are zero or one. Godless, tasteless, without complex harmonics.

    They are reproduced or not. Digital equipment works OR it doesn’t. There is nothing to be subjective about!

  28. Ben Goldacre said,

    February 4, 2006 at 2:20 pm

    Somebody has just sent me this excellent article from the BMJ which i just wanted to share.

    “Ability to distinguish whisky (uisge beatha) from brandy (cognac)”

    bmj.bmjjournals.com/cgi/content/full/309/6970/1686?eaf

  29. Alison Macpherson said,

    February 4, 2006 at 2:27 pm

    An interesting though for you all…

    Human Beings employ analogue systems, infinitely variable and subtle. Digital systems employ quantisation – samples at 44000 times per second, for example, may sound like the original however we have lost an infinite amount of information compred to an analogue system – how can that be “the best reproduction possible”? Particularly given that a capacitor is filling in the gaps between the samples?

  30. Briantist said,

    February 4, 2006 at 2:40 pm

    “Human Beings employ analogue systems” – sort of. We have a little hair for about each frequency our ears can respond to. That’s actually both discrete and quantifable.

    44,000 samples per second allows a signal of up to 22Hz to be encoded. It’s called the nyquist frequency and it’s good science.

    The “best reprouction possible” in a digital signal is a “bit error rate” of 1 in longer-than-your-going-to-be-listening-for.

    Most in-home LAN cables work at rates of 104,857,600 bits per second on inexpensive telephone cables.

  31. David_H said,

    February 4, 2006 at 2:47 pm

    Ben

    I’m with you on this test and look forward to being the whole subject debunked (if only!).

    But I am slightly amused that you’re going to use a “studio quality (made by RME)” digital optical interface. The “high-end” digital interfaces advertised these days seem even more bogus than high-end analogue gizmos. Surely, if it’s digital, it either works, or it doesn’t.

    But then you could say the same about the digital output of a CD player in the first place.

    If I had money to burn (and the inclination to do so, which I don’t), I’d rather spend my cash on analogue component upgrades than digital one. Or have I missed the point?

    David

  32. Briantist said,

    February 4, 2006 at 2:53 pm

    Wouldn’t the idea situation be high-quality battery-power speakers that receive digital signals by WiFi? Getting the “perfect” digital signal as close to the speaker’s driver element would reduce loss and induction to the minimum, battery-power would eminiate mains noise.

  33. Alison Macpherson said,

    February 4, 2006 at 2:54 pm

    “Most in-home LAN cables work at rates of 104,857,600 bits per second on inexpensive telephone cables.” Not in any manner could this be considered as analogue, reliable or relevant – Ethernet employs a complex flow control system, with the ability to retransmit packets – which happens regularly, to compensate for dropped packets due to reflections, noise, electromagnetic environment and losses – in fact the inadequacy of the medium and the environment, the actual data throughput on Ethernet LANs is considerably lower that your, no doubt, “Googled” figure. Ethernet works despite itself, we have all experienced network delays – within this digital system transmitting digital data the losses are significant and noticeable. When I listen to an orchestra I do not request they broadcast the last chorus again becuase I missed it, I do not wait indefinitely for the violin solo and find it arrives at my ears before it should, only to be buffered and delivered when the preceeding verse arrives. I could go on sigh…… And probably will!

  34. Tim Day said,

    February 4, 2006 at 3:04 pm

    It’s not really true to say that a capacitor is filling in the bits betewen the samples. In a 44.1 KHz recording all the data “between the samples” can be accurately recovered, up to frequencies above the limit of human hearing, by a low pass filter. The Nyquist/Shannon theorem is pretty clear about this (in addition to being very elegant and generally all round cool). We use 44.1 KHZ precisely because this information theory result tells us we can accurately recover data at a frequency of half the sample rate. Studios will use higher sample rates for multitracking to avoid the accumulation of rounding errors when mixing down.

    www.lavryengineering.com/documents/Sampling_Theory.pdf

    There has been some discussion about whether subtle stereo effects can be percieved above 20KHz, but nothing conctrete.

    Bit depth is another matter. Humans can percieve the difference between 24 and 16 bit recordngs, so in that sense infirmation has been lost, although again much of it can be recovered with a good dither algorithm.

    The inversion thing should work if the signal remains in the digital domain. I think this would be better than a double blind since you’re unlikely to have a random sample of listeners on this forum.

    In general I have to agree that blind tests are boith useful and possible. I’d like to see the same thing for some of the speaker leads. £5000 per meter for Valhallas? You gotta be kidding.

  35. Briantist said,

    February 4, 2006 at 3:07 pm

    Alison Macpherson. I’ve been a computer network engineer for about twenty years (starting with a school BBC Econet). At least everyone knows where YOU get your “knowledge” from.

    The retransmissions on Ethernet are to do with it’s “multiple access” nature (CSMA/CD the middle bit, between carrier-sense and collision-dection), rather than the problems you include.

    These are already dealth with in it’s media-access layer.

  36. Briantist said,

    February 4, 2006 at 3:07 pm

    The number, if you knew any maths is 100 times 10 to the power 20, of course!

  37. Alison Macpherson said,

    February 4, 2006 at 3:15 pm

    I am feeling a little under attack here, but here goes:

    If there is no loss of information at 44.1KHz why would a studio use use “higher sample rates for multitracking to avoid the accumulation of rounding errors when mixing down”?

    Nyquist by default acknowledges that we will loose information when sampling – it is a “Sample” after all, and we use interpolation techniques to fill those “gaps”. The word “gap” implies that we do not have the information, so where does the “fill” come from?

    Notwithstanding the Human ability to detect the “fill”, it is detectable by measurement when comparing a sample sampled at a higher rate with one at a lower rate, a good oscilloscope would sufffice…

    Whether it makes a difference to an individuals perception is a different subject.

    Sorry – will go now…

  38. TxB said,

    February 4, 2006 at 3:18 pm

    What Allison says is right of course in theory, but nevertheless my system ‘hiccups’ when the boiler switches itself on. None of my other systems (dvd player, computer systems etc) suffer from this. I use a coaxial SPDIF rather than a optical, which should make no difference. This is not an in-audible theoretical problem but very audible & very annoying. What is the cause? No Idea … maybe Allison can give us the technical reasons. My point is that there are genuine problems and its possible that the £1800 power lead may solve one or more of these problems, and if it does it may well be worth £1800. I don’t know anything about the specific power lead, and probably it is a marketing gimmick … BUT Ben Goldacre is entirely wrong to dismiss it out of hand, in fact its unscientific — precisely the kind of ‘bad-science’ he seeks to expose. Just because something sounds foolish doesn’t mean it actually is. Put it to a fair test first before making a judgement!

    The manufacturer of my expensive power cable has assured me that it will have an impact on the sound of an expensive CD player, or a cheap one, and that it will affect optical outputs as much as normal phono outputs.
    Ben Goldacre assumes that the manufacturer is claiming the power cable will improve sound quality. Not so! The manufacturer may simply be claiming that the power cable blocks anything nasty which somehow manages to disrupt the digital electronics — a kind of digital epilepsy that some equipment may be susceptible to. Nothing to do with the audio signal itself, but rather the control electronics.

    I therefore challenge Mr Goldacre on this specific ground. I have a specific problem, a problem I suspect the manufacturer of the £1800 power lead claim to solve. Contact the manufacturer, ask them if it will solve my problem … if they say yes, pay me a visit and put the infamous power lead to the test (note: you will need 2 leads for an actual test). If the ‘hiccup’ disappears then I would hope to see a full apology in next Saturday’s Guardian — both to the manufacturer and to Hi-Fi/Av enthusiasts in general.

    I am looking forward to the headline: “Bad-Science guilty of bad science — hangs head in shame”

    Thomas.

  39. Alison Macpherson said,

    February 4, 2006 at 3:19 pm

    Briantist,

    Ouch – you are so cruel – LOL.

    I am…. Forget it, I do not need to blast you… CSMA/CD was developed to compensate for the inadequacies of Ethernet, including those listed previoulsy….

    There are female engineers you know!

    Love

    Alison

  40. Alison Macpherson said,

    February 4, 2006 at 3:20 pm

    Oh yeah including multiple access requirements

  41. Briantist said,

    February 4, 2006 at 3:20 pm

    I looked on your “Google”, and the Bit Error Rate for 100BaseT ethernet is around 1 in 10^9 or better. That’s around ONE single bit every SECOND.

    For 44.1k 16 bit stereo the same BER would be one single bit every ((10^9)/1411200) 70.8 seconds.

    Is the test over 70.8 seconds long?

  42. Briantist said,

    February 4, 2006 at 3:21 pm

    It’s strange the female engineers I met knew what they were talking about.

  43. Alison Macpherson said,

    February 4, 2006 at 3:22 pm

    Go now…

  44. Briantist said,

    February 4, 2006 at 3:23 pm

    But 70.8 seconds won’t be long enough, because as the sample frequency falls, the BER tends to improve.

  45. Briantist said,

    February 4, 2006 at 3:28 pm

    If you KNEW anything, then you would UNDERSTAND that the limitations of a medium do not render it useless. All data communications standards start with that in mind, from the parity bits added to serial communications, CRC checks on packedized delivery systems. High-speed serial delivery systems use MFM encoding schemes, an old-style mouse will use grey encoding.

    These are all designed to understand the medium in which they exist. There is nothing intrinsically wrong with any digital system, as long as it delivers the data perfectly from one end to the other. The technologies deal with the cost implications of the medium used.

  46. Ray Girvan said,

    February 4, 2006 at 3:30 pm

    “Ability to distinguish whisky (uisge beatha) from brandy (cognac)”

    In that vein, check out also Scientist sniffs at wine buffs’ nose for purple prose.

  47. Alison Macpherson said,

    February 4, 2006 at 3:31 pm

    Briantist,

    You are such an Engineer I must bow to your superior knowledge, of course. So you run with your incredible statistcal analysis – without running BERT tests in a real world environment on UTP cable, its easy to quote.

    I give in, OOOHHH Econet – I am not worthy, what a pedigree.

    I just do not know why a silly woman like me even ventured into such a testesterone dominated environment – I almost kicked you garden shed door in didn’t I?

    Ali

  48. Briantist said,

    February 4, 2006 at 3:37 pm

    You won’t find anyone who is more egalatarian when it comes to engineering work. However, it is you who brought the “I’m a lady” subject up…

    I can assure you that, as I worked in the broadcast division of BT for quite a while, where we used both computer-networks and transmitted the nations’ television around the UK and via eathstations to the works, I HAVE run real BER tests on every kind of cable in the most electrostatically active buildings on the planet.

    So please can you explain what your BER estimation would be for Ben’s files given your “real world” data.

  49. David_H said,

    February 4, 2006 at 3:41 pm

    Isn’t “electrostatically active” a contradiction in terms? :)

    Or have I been reading the Indie’s Errors and Omissions (aka Pedant’s Corner) for too long?

    D.

  50. Alison Macpherson said,

    February 4, 2006 at 3:44 pm

    Ben’s files? which files? what is the delivery method? why bother?

  51. Peter Curran said,

    February 4, 2006 at 3:47 pm

    The Scotch whisky link provided by Ben Goldacre made the core point that a level of innate skill may be refined and brought to a higher level by experience and acquired skills.

    One good example of this is the jazz musician. Almost every jazz player of note started with a high innate ability (as testified to by their contemporaries) and developed and refined it by hard work. Much admirable and useful work has been done on jazz solos construction using modern technical analysis tools (e.g. Band-in-a-Box software) and this has been of enormous value to the talented but inexperienced young jazz musician, however, no one would confuse the solos generated by the software with a live improvised performance.

    Much of the debate has centred on the technical question of whether the equipment does or doesn’t reproduce certain frequencies or satisfy technical benchmarks, but we must remind ourselves that the purpose of all this technology is to satisfy the listener in a wholly subjective experience.

    One correspondent, dismissing my whisky analogy, used the phrase ‘Godless, tasteless, without complex harmonics’ – a fair description of some of the contributions to this debate.

  52. Briantist said,

    February 4, 2006 at 3:48 pm

    Ben has stated the files are 44,000-samples per seconds at 16 bits. You don’t need the actual files to demonstrate your BER estimates!

  53. Briantist said,

    February 4, 2006 at 3:54 pm

    Sorry, I was thinking of my QUAD electrostic speakers again.

    I meant “electromagnetically active” was that there was a variety of cables, some with DC-telephony cable, some with mains-power, some with balanced-data communications and others with un-balanced data communications with a wide variety of analogue and digital signals.

  54. Alison Macpherson said,

    February 4, 2006 at 3:55 pm

    Sigh

  55. Alison Macpherson said,

    February 4, 2006 at 3:56 pm

    I think this is why the public at large dont trust scientists and engineers….

  56. Briantist said,

    February 4, 2006 at 3:58 pm

    Peter Curran

    Two questions.

    Question 1: Please describe the following using religion, subjective human feelings and learned human sensation:

    0

    Question 2: Please describe how a new stereo lead effects the following:

    1

  57. Ben Goldacre said,

    February 4, 2006 at 4:00 pm

    TxB:

    Ben Goldacre assumes that the manufacturer is claiming the power cable will improve sound quality. Not so! … I am looking forward to the headline: “Bad-Science guilty of bad science — hangs head in shame” etc etc etc

    I make no such assumption: I am proposing to measure whether listeners, having been played the expensive cable and the cheap cable, when played a third piece of music, can say which cable it was recorded with. This makes no assumptions about an improvement, only about a discernible difference.

  58. Andrew Clegg said,

    February 4, 2006 at 4:08 pm

    TxB said: Not because you can actually hear the difference, but because you want the best reproduction you can possibly have.

    Thomas, I have a question for you and all the other audiophiles. If it is perfect reproduction you’re after, why do you use ‘hi-fi’ equipment at all, rather than studio amps and monitors?

    I have a Samson power amp and a pair of Absolute nearfield monitors, as used in hundreds, perhaps thousands of small studios and mixing rooms round the world. Total price, about four hundred quid. What’s the point of spending thousands of pounds on a system that’s entirely unlike the rig that the music was composed/recorded/mixed/mastered on, if you’re really after high-accuracy reproduction?

    I always smile when I see hi-fi kit with the word ‘reference’ on it…

    Andrew.

  59. Larry Lamb said,

    February 4, 2006 at 4:09 pm

    Having just read the “Hifi Wigwam” thread, I note that some form of cable test was organised for the 14th of January at which doubters (at least one had apparently agreed to attend) would be convinced. The thread appears to dry up after the 14th and no reference is made to the outcome of this test. Enquiring minds etc etc.

  60. Peter Curran said,

    February 4, 2006 at 4:10 pm

    I don’t understand your post or your two questions, Briantist.

    I am an agnostic, so I have no religious perspective to offer.

    Perhaps you should examine the effectiveness of your communications style, which comes across as aggressive and confrontational (the use of capital letters is the equivalent of red-faced shouting in online posts), and regrettably typical of the technician in industry.

    Try engaging more persuasively in debate and you might just change a few viewpoints.

    In case there is any confusion about my posts, may I make it clear that I was not on specifically about mains leads or hi-fi advertising claims, but addressing the wider, and more interesting question of where measured technical ‘reality’ meets the complexity of the subjective human experience in music.

  61. Briantist said,

    February 4, 2006 at 4:11 pm

    This is more a question about psychology then. I’d bet that if you sent out the three styles of samples randomized onto two different sorts of CDs half unlabelled in paper packets and the latter with shop-quality graphics in a full jewel case, the latter would come out sounding the best.

    £5 that Peter Curran gets his gold and shiny leads in the shop with the halogen lighting?

  62. TxB said,

    February 4, 2006 at 4:17 pm

    ‘Godless, tasteless, without complex harmonics’
    When you are talking about a digital bitstream then Ben Goldacre & others are entirely right. A bit is a bit is a bit. A degraded or distorted bit is still the same bit. Even if its totally erased a bitstream will likely have Error Detection & Error Correction, meaning it can be reconstructed. This makes a debate on Bit Error Rate a blind alley. The problems with networks are more abstract and have little to do with the underlying hardware. In the digital domain packets loss or packet delay are the real bugbears — and those problems are solved by throwing computing power at the problem. Hence giga-bit switched Ethernet, dual processor CPUs, overclocking — and all the high-end enthusiast computing equipment that is taking the world by storm.[solid copper liquid cooled CPU coolers — possibly another topic for the column].

    Not many audiophiles want a return to the age of analog … but we do care about accurate reproduction as the artist intended. That means reliable digital delivery & good quality digital to analog conversion. All those good analog things count at the recording end and at the re-construction end but in between the debate has moved on. Yes, there are £100 gold-plated SPDIF cables but only because some old-timers don’t understand digital — there isn’t much scope for debate there though (just patient explanation, just like explaining an ipod to your grandfather).

    Thomas.

  63. Ben Goldacre said,

    February 4, 2006 at 4:26 pm

    TxB:

    ‘Godless, tasteless, without complex harmonics’
    When you are talking about a digital bitstream then Ben Goldacre & others are entirely right.etc etc etc

    I’ve said nothing about digital bitstreams being anything of the sort. Stop making stuff up, TxB, it’s really irritating and it gets in the way.

  64. TxB said,

    February 4, 2006 at 4:27 pm

    Ben Goldacre:I make no such assumption: I am proposing to measure whether listeners, having been played the expensive cable and the cheap cable, when played a third piece of music, can say which cable it was recorded with. This makes no assumptions about an improvement, only about a discernible difference.

    You do quite clearly make the assumption that the manufacturer makes a claim about ‘improved sound quality’. There is no point in knocking down a straw man when the claim isn’t actually about sound quality. You can take a recording from my hi-fi system while the boiler switches on and off — anyone can definitely discern the difference (ie. sudden nasty interruption)! In general digital errors can sound very nasty … try listening to Freeview in poor reception.

    Thomas.

  65. Peter Curran said,

    February 4, 2006 at 4:31 pm

    I feel a bit like the English clergyman, Sidney Smith, watching two Edinburgh women abusing each other across a narrow wynd from their respective windows – “These ladies will never agree – they are arguing from separate premises.”

    I think I had better leave you to your bits and bytes, and go and listen to some music, courtesy of my non-gold plated leads (you lose your bet, Briantist).

  66. Briantist said,

    February 4, 2006 at 4:31 pm

    I think that we can all agree that there is no error correction in this system, but there probably is error detection.

    The information from the CD is a series of big-endian sixteen-bit numbers that are carried in a continious bitstream. Without error detection it would be impossible to know which bit, so to speak, is which.

    The BER (bit error rate) measurement is the KEY point here and cannot be dismissed. Without any FEC (forward error correction) is it very simple to understand that the bigger the number, the less often you get a single bit error.

    Compare this to the “subjective” quality you are being asked to measure when listening to a hi-fi in a shop. If you can’t hear a 1/16 of 1/44,000th of a second tiny error every minute, you’d go “I’ll have that”.

    Digital error rates on audio systems are almost astronomically unlikley, that they could be improved by something like a mains lead is just laughable.

  67. Andrew Clegg said,

    February 4, 2006 at 4:32 pm

    TxB: but we do care about accurate reproduction as the artist intended.

    There’s that claim again. So why not listen to the music on equipment more like that which it was recorded on?

    I’ll bet they don’t have £1,800 power cables in recording studios.

    Andrew.

  68. Briantist said,

    February 4, 2006 at 4:37 pm

    “In general digital errors can sound very nasty … try listening to Freeview in poor reception”

    This is a great example of why BER on most digital systems have to be really high, because a single bit being dropped can cause the receiver to become “out of sync”, because the data is carried as a stream of bits.

    When this happens on Freeview it’s very annoying because one bit error in 40-odd million causes the whole service to be useless.

  69. Alison Macpherson said,

    February 4, 2006 at 4:39 pm

    I suspect the noise TxB hears when his boiler come on is due to one of two things:

    Large inductive load swtiching on, causing large sag (brown out) or dip in mains voltage resulting in noise

    RFI discharge as a result of switch on of high startup device, similar to spark gap transmission, this being picked up by pre-amp.

    We could isolate this by powering Hi-Fi on UPS unit running on battery – if no noise heard whilst running on UPS then source is mains supply, otherwise it is Radio Frequency Interference, in which case one could look at screening of source or receiver.

    Just a thought

    Alison

  70. TxB said,

    February 4, 2006 at 4:40 pm

    If it is perfect reproduction you’re after, why do you use ‘hi-fi’ equipment at all, rather than studio amps and monitors?

    The main reason I think is that audiophiles get their information in the audiophile press/internet sites. What you find there are not only news & reviews about ‘hi-fi’ but also advertising, and of course the news & reviews are directed in part by advertising. Studio equipment doesn’t really get a mention so many people are unaware it exists.

    The second reason is that studio equipment tends to be functional with little consideration given to aethetics … and therefore tends to be ugly. Ugly, bulky. Not a good combination for most people.

    It also seems also to be hugely expensive. £10,000 for a pair of speakers (big ugly ones at that) is just too much for most.

    Finally, in my personal experience studio equipment tends to be designed for a specific purpose rather than being good for general use. If you try to apply it to general use it will sometimes produce poor results.

    Thomas.

  71. Briantist said,

    February 4, 2006 at 4:48 pm

    If you want to read about studio equipment, everyone I know who’s got one reads “Sound On Sound”, www.soundonsound.com/

  72. TxB said,

    February 4, 2006 at 4:50 pm

    We could isolate this by powering Hi-Fi on UPS unit running on battery – if no noise heard whilst running on UPS then source is mains supply, otherwise it is Radio Frequency Interference

    Funnily enough I’m just in the process of putting in a UPS … but haven’t tested it yet on my hi-fi. I’m pretty sure though it will solve the problem. The reason I’m pretty sure is that I live in a block of flats and my neighbour’s boiler (actually closer to my hi-fi than my own) causes no problem. I know from using powerline networking that the flats are somehow isolated from each other — no signal put onto the mains gets through to other flats.

    Thomas.

  73. Briantist said,

    February 4, 2006 at 4:57 pm

    Actually here’s a great digital sound primer article in that magazine… www.soundonsound.com/sos/may98/articles/digital.html

  74. Andrew Clegg said,

    February 4, 2006 at 5:00 pm

    The main reason I think is that audiophiles get their information in the audiophile press/internet sites. What you find there are not only news & reviews about ‘hi-fi’ but also advertising, and of course the news & reviews are directed in part by advertising.

    That’s the last place I’d go to not get ripped off on kit, better to ask the people whose jobs depend on making good-quality recordings in the first place :-)

    The second reason is that studio equipment tends to be functional with little consideration given to aethetics …

    I’m not actually sure this is true in general, but anyways, visual appeal in audio equipment seems like a rather misplaced aesthetic.

    It also seems also to be hugely expensive. £10,000 for a pair of speakers (big ugly ones at that) is just too much for most.

    I think you’d be surprised how much contemporary music (especially electronica for example, but also any obscure singer-songwriter with a home studio) is produced on a lot less expensive monitoring than that. But even so, £10K doesn’t sound like much in a market where a kettle lead can go for almost £2K.

    Finally, in my personal experience studio equipment tends to be designed for a specific purpose rather than being good for general use. If you try to apply it to general use it will sometimes produce poor results.

    But if there was an ideal ‘reference quality’ hi-fi setup that did have that general-purpose panacea effect, wouldn’t the studios use that instead?

    Andrew.

  75. TxB said,

    February 4, 2006 at 5:11 pm

    I’ve said nothing about digital bitstreams being anything of the sort. Stop making stuff up, TxB, it’s really irritating and it gets in the way.

    I’m putting what several people posted (including you) together and responding to a composite. I’m sorry if that irritates.

    However, I think what you actually find irritating is that you have a case to answer. You thought it ‘obvious’ that the £1800 power lead was snake oil when you didn’t do your background research to establish this as a fact. And in the process you insult hi-fi enthusiasts — who on the whole are not the most unreasonable people on the plant. Leaping to conclusions without establishing the facts is precisely ‘bad science’. I don’t mind being made to look foolish (what better to motivate learning, one’s own foolishness) as long as its from rock solid ground. On the other hand if you have been caught out, don’t sit in the sidelines & snipe — bit the bullet and let us have an apology.

    Thomas.

    [and now I must go out and buy the Guardian, having spent (some would say wasted) all afternoon on this debate]

  76. Roy Badami said,

    February 4, 2006 at 5:20 pm

    the manufacturer claimed specifically that the power cable will also affect the optical digital outputs, so I will take the optical output of the CD player into the optical inputs of a sound interface

    Sadly that doesn’t work. If you feed the optical output directly into a soundcard, and everything is working correctly, then all you will be capturing is the original digital data on the CD. Barring errors, the data will simply be the original data that the studio captured in their ADCs.

    But, and here’s the big but, there is a potential difference between cheap and nasty optical outputs and high end optical outputs, and the effect is real. The problem is that in the audio world you’re going to feed this optical output into a DAC, and the DAC is going to synchonise to the clock on the optical output. Timing jitter in the digital data results in timing jitter in the DAC output, which results in distortion… (Of course we can avoid this problem by using a reclocking DAC, but good reclocking DACs are expensive and unlikely to be found outside of a studio.)

    We know that the data coming out of the optical output is going to be the same either way, what we want to know is how clean a clock we can extract from it, since that is what determines the timing jitter in the audio signal we get out of our (ordinary, non-reclocking DAC).

    Is a power cable going to make a difference to this? Clearly not. Would an RF filter make a difference? Well, maybe. Would your experiment capture the difference if there was one? Clearly not.

    -roy

  77. mfg said,

    February 4, 2006 at 5:24 pm

    Re TxB

    Ben Goldacre assumes that the manufacturer is claiming the power cable will improve sound quality. Not so! The manufacturer may simply be claiming that the power cable blocks anything nasty which somehow manages to disrupt the digital electronics — a kind of digital epilepsy that some equipment may be susceptible to. Nothing to do with the audio signal itself, but rather the control electronics.

    Well, here we go. If this is true, then there would be no need to perform any double blind tests or A/B/X protocolls, get two leads (one of the mystical ones and the one, that you get included with your stereo), a signal processor and/or a radio frequency source and an oscilloscope and this is resolved in a matter of hours.

    But I *highly* doubt, that this is the effect customers expect from a very expensive power connection, because most hi-fi afficitionados I know claim, that this will somehow increase the quality of the sound output (something you have great difficulties to measure objectivly, which is exactly the reason, why a double blinded protocoll is indeed needed if you want to convince anyone from this crowd)

    And BTW ( I like car analogies, especially if they are flawed) :

    – You are allowed to drive as fast as you like on most german highways (although there is a recomended speed limit of 130km/h IIRC)

    – There are measurable differences between the two cars you mentioned, I’m quite sure the Bugatti would win a double blind test against the Ford Mondeo any time, hands down.

    Regards

    Martin

    PS : English is only my third language, so please apologize my mistakes (I’m trying to get better!)

  78. Briantist said,

    February 4, 2006 at 5:28 pm

    Any desktop computer consits of millions tiny interfaces in a box with a conventional “kettle lead” and these process billions of binary transactions per second without fail. I presume no-one is selling a “kettle lead” that makes your computer sound better?

    Why should audio data (small beer in computing terms) be an exception to the rule?

  79. Roy Badami said,

    February 4, 2006 at 5:41 pm

    Or to put it another way, a digital output serves two purposes, to output digital data, and to supply clock. You are concentrating on the former, when it’s the latter that is interesting. Sorry, I’m afraid I simply can’t resist: your experiment is Bad Science :-)

    Your experiment would also prove that all CD players sound the same when you use the digital output — assuming the disc isn’t dirty or damaged and the transport isn’t malfunctioning. And yet, when fed into a conventional (non-reclocking) outboard DAC, this isn’t true: the digital output is a signifcant factor in the amount of timing jitter in the resulting analogue signal.

    As an aside, I presume (though I don’t know) that this was the rational for offering TOSLINK optical outputs over and above copper SPDIF outputs: it avoids noise pickup on a long copper run that might damage clock integrity.

  80. Andrew Rose said,

    February 4, 2006 at 5:52 pm

    Tim Day wrote: [i]Studios will use higher sample rates for multitracking to avoid the accumulation of rounding errors when mixing down.[/i]

    I think it’s worth correcting this – changing the sample rate (i.e. sampling frequency) has no effect on rounding errors. Changing the bit depth certainly does. That’s why we tend to use higher bitrates for processing.

    There’s no point in trying to record in bit depths higher than 24-bit as this resolution gives a signal to noise ration which far exceeds what is physically possible to achieve with the analogue elecrtonics which provides the signal to be converted.

    Pushing sampling rates higher – e.g. 96kHz or 192kHz – generates a whole load of data for frequencies only your pet bat can appreciate. Some audiophiles claim to be able to hear a difference – but then they’ve already been sold on £1800 kettle leads making their hi-fi sound better. Whatever TxB says in his increasingly pointless rants, the hi-fi buffs and those who peddle this nonsense to them do claim to hear a positive difference, and that’s enough reason for a Bad Science investigation, regardless of the manufacturer’s claims.

    I spend my working life monitoring audio on £10k speakers – I use them to hear an accurate representation of the audio I’m working on. They were an excellent investment and contribute enormously to the quality of the finished product. I will not be splashing out on £1800 mains cables, and nor will I be switching from 44.1k to 96k. I’ll continue to record at 24-bit and process at 32-bit resolutions, before dithering down to 16-bits for CD mastering.

    When the latter is done properly you’ll find no audible difference between this and the 24-bit master, and none between 44.1k sampled audio and 96k audio. Privately I know that most mastering engineers agree with me – but the customer is always right and the customer is being told they need 96/24 or higher, so that’s what they’re getting from some record companies.

    Should an unscrupulous record company wish to demonstrate ‘superior’ sound from this it’s not difficult to apply a little digital ‘sweetening’ or ‘flavour enhancer’ to the finished product to convince the punter they’re getting something superior – guess what, there are some record companies who’d like you to replace your record collection all over again, just as there are some folk who’d like you to spend ridiculous sums of money on equipment which will have no effect on the sound of your system.

    Fortunately the ignorance of the general public and the religious zeal of the audiophile community provides plenty of opportunity for the unscrupulous to fill their coffers…

    (By the way, I’ve got this great potion which improves the sound of all your CDs – just send me fifty quid and I’ll let you have some. It really works, and you’ll be convinced too after you’ve parted with that cash. Alternatively you can buy some yourself at the local offie…)

  81. Ben Goldacre said,

    February 4, 2006 at 5:51 pm

    Roy Badami, hey, it wasn’t me who said the optical digital outputs of the cd player would be affected by the cable, it was the cable manufacturer. I asked them that question specifically, and they were very clear on that, they claim it will.

    You’re saying you know the answer already, you know better than the manufacturer, you know the answer to the hypothesis we are testing, that there will be no difference, before we’ve even done the test. But a lot of people are saying on both sides that they know the answer before we do the test, so that’s why we’re doing the test…

  82. Andrew Clegg said,

    February 4, 2006 at 5:54 pm

    Roy, this objection depends on the exact nature of the manufacturer’s claims; if they are claiming that their power cable somehow reduces actual digital errors, then it’s still a valid test (although I think we can all guess what the answer will be).

    If it’s some nebulous ‘sounds-better’ claim which could be attributable to clocking differences, then you have a point. I have a Fostex VC8 ADC/DAC which has an internal clock, which I’d be willing to lend for experimental purposes, but if I remember correctly it only has ADAT style multichannel I/O connectors which I doubt many domestic CD players have.

    On the other hand, if this magic cable is supposed to improve digital output on CD players, can we assume it would do the same if we used my PC (which does have ADAT optical I/O) as a big CD player? After all, it takes a kettle lead too…

    I should stop thinking about this so much and go back to my thesis really.

    Andrew.

  83. Briantist said,

    February 4, 2006 at 6:14 pm

    Roy,

    Your point about jitter is irellevant as has already been taken into account when designing the interface.

    You will only ever have problems with jitter if you are synchonising various inputs, and this is not a feature of the test.

  84. Briantist said,

    February 4, 2006 at 6:18 pm

    I meant to point out that there are, of course, sixteen bits per sample, so any jitter will have been averaged over sixteen signal transitions, anyway. And it’s easy these days to produce stable clocks working at 400Mb/s on firewire which is about 600 times faster!

  85. Roy Badami said,

    February 4, 2006 at 6:28 pm

    Ben,

    I think you miss my point. Ok, I accept that your experiment isn’t entirely worthless. If, by some mechanism we don’t understand, the choice of mains cable affects the integrity of the digital data, your experiment will capture this effect.

    However if the choice of mains cable only affects the level of noise on the digital output, and hence the quality of clock extracted, then your experiment won’t capture it. The SPDIF input on your soundcard doesn’t care about clock jitter; it just reads the samples it’s given. A conventional DAC, on the other hand, is very dependent on the quality of clock.

    My assertion is that a reasonable quality CD player, functioning correctly, and playing a disc in good condition should be extracting the recorded data without corruption anyway. The only aspects that might depend on the player’s environment is the level of jitter in the clock, or the level of noise in the digital output that might induce jitter in the clock extracted by an external device.

    Your soundcard won’t and can’t measure this latter effect, and yet this is almost certainly where any differences will lie, if indeed there are any differences.

    Of course, I really don’t believe that the mains lead will make any difference, unless you are suffering from significant RF interference and the cable has an integral RF filter… And if anyone is concerned about RF you can buy power strips with RF filters pretty cheaply…

    -roy

  86. Briantist said,

    February 4, 2006 at 6:36 pm

    Ben,

    Your point about DACs is fine, but in Ben’s experiment there is no DAC, the data is being being transferred digitally from the data on the CD, directly onto the sound card.

    If this is fully digital, there are no DACs involved. (DAC=digital to audio converter).

  87. Roy Badami said,

    February 4, 2006 at 6:40 pm

    Briantist,

    1. You say this should have been taken care of in the design of the interface. Well, yes, but it all depends on how much you want to spend. Undoubtedly some CD players will do better than others, though I would hope that these days any good quality hi-fi separate would do a reasonable job, since the issues of jitter have been understood for quite a number of years now…

    2. I doubt a conventional DAC is doing a huge amount of averaging. I guess there may well be a PLL, but it will not completely eliminate dependency on the original clock. And then you’ll simply divide the clock by the appropriate factor (more than 16 since it’s stereo; it will be 32 plus whatever the protocol overhead). So every nth transition from the PLL is timing critical, as it clocks out another sample to the DAC. You can build DACs that aren’t sensitive to clock jitter. They’re used in pro audio, and possible these days are appearing in high end hi-fi, too.

    3. USB is a bit of a red herring. Computer systems don’t care about clock jitter, as long as it doesn’t affect data integrity. Audio systems do.

  88. Briantist said,

    February 4, 2006 at 6:40 pm

    *(digital to analogue converter).

  89. Martin Dearden said,

    February 4, 2006 at 6:43 pm

    The point about science is that it is only supposd to measure that which is measurable. In other words you can’t make a claim about ‘superior quality’ without observable differences between one set of figures and another. Since Thomas in particualr talks about subtle differences in ‘colour’ and even m,ore vagualy about the ‘awesomeness’ of one system against another then he is not talking about anything which can be measured using the system (the human ear) we all seem to be talking about.

  90. Briantist said,

    February 4, 2006 at 6:52 pm

    Roy,

    What I mean by “design of the interface” is that the specifications do not assume “perfect conditions” but the designers knew about the nature of the medium and designed the system to work within “normal parameters”.

    If the level of jitter as the digital signals are passed from digital component to compoent was so bad that it caused errors, these would be recordable by the sound card and show up in (yes, you guessed it) the bit error rate.

    As you point out computer systems (including digital audio equipment) will deal with a “in spec” level of jitter. Any that is so bad that it causes an error will be recorded and measureable.

    In TV studios and the like, a reference clock is provided to all equipment to remove the jitter issue, so it’s not a issue for “professional” equipment.

    What’s left? Only the jitter of the DAC on in the digital amplifier in non-professional installations that never causes recorable errors. How is this measured if it requires more data than was on the CD in the first place?

  91. Roy Badami said,

    February 4, 2006 at 6:59 pm

    Briantist,

    Most consumer outboard DACs don’t use a local clock… They generate their clock from the digital input. If the digital input is noisy, they end up with a non-uniform clock, and the transitions between successive samples in the audio are ofset very slighly from their correct times. This is called timing jitter, and it is now understood to result in subtly perceptible distortion in the audio.

    Yes, designing all your equipment perfectly eliminates this problem. Most people can’t afford studio-quality kit.

  92. Briantist said,

    February 4, 2006 at 7:03 pm

    Also, a sustained variation in the output frequency of the DAC would result in a very small key change (because the constituent frequencies would all be shifted by multiplication), like that of the “pitch control” on a turntable.

    There’s a reason that “perfect pitch” is regarded as a gift, because it is imperceptible to 99% of people.

  93. Briantist said,

    February 4, 2006 at 7:05 pm

    Roy,

    “This is called timing jitter, and it is now understood to result in subtly perceptible distortion in the audio.”

    OK. I want to know about this! This is a system that transfers binary data from point to point BETTER than 100%

    Please tell me about this.

  94. Briantist said,

    February 4, 2006 at 7:07 pm

    Because it’s all that’s missing from my Grand Unification theory and would heat up my superconductor!

  95. Roy Badami said,

    February 4, 2006 at 7:14 pm

    Briantist,

    You’re missing the point. It’s slightly subtle.

    Transfering the digital data with 100% accuracy is easy. There are two things that are hard in digital audio: converting the original analogue signal to digital, and converting the digital data back to the correct analogue waveform. (Actually, there’s a third, and that’s compression, but that’s not relevent here since CDs don’t use compression.)

    The problem is that the design of a conventional DAC means that it doesn’t do a perfect job of converting the digital data to an analogue waveform if it can’t accurately extract the clock from the datastream. “That’s a crap design then”, you say. Yes, it is, but the significance of jitter wasn’t really understood in the early days of digital audio, though it’s been well known now for over a decade.

    Modern studio quality reclocking DACs avoid the problem, and I dare say these are filtering down into the high-end hi-fi market too

    I suggest you google for digital audio and jitter, if you want to read up about this further…

  96. Peter said,

    February 4, 2006 at 7:16 pm

    I use “oxygen-free” cables because I did hear a difference. I have been told why it shouldn’t/cpuldn’t to me it does.

    Please supply the recorded files I would be facinated to hear and vote.

    I did volunteer for the blind test touted a few weeks ago but heard nothing back.

  97. Tim Day said,

    February 4, 2006 at 7:23 pm

    Final comment on the bit depth/sample rate thing…
    Alison, if you read the link and look at the Nyquist stuff, you’ll see how that data gets recovered. Bear in mind the limit is that you can recover fluctuations in signal up to 1/2 the sample rate. Any higher frequency signals than this should have been filtered out in the recoording chain anyway otherwise they will produce aliasing tones. If you sample at 44.1 you can accurately pick up data to 20 KHz (the extra 4.1 KHz is to allow for filter slope and a little for luck).

    Sample rate wise, where I am we generally sample at 48 KHz. The 96 KHz is because some people reckon that although we hear up to 20KHz or so, we can pick up time delays of far less than 1/20000 sec. Like Andrew Rose, I’m sceptical. My bad about sample rate.

    However you can hear a difference beteen 24 and 16 bit, although it’s pretty minimal. after about 22 bit you reach the limit of human resolution.

  98. Briantist said,

    February 4, 2006 at 7:41 pm

    Firstly, as I pointed out already we are not using a DAC for the experiment.

    Secondly, I know more about “audio and jitter” than anyone is going to learn from Googling, my uncle having worked for Quad.

    What is it that you can perceive with your jitter?

    In effect you are saying that the equipment is not providing a stable 44,100 cycles per second of timing to the output. The clock is arriving as 16 binary events, that’s loads of get a great signal stability.

    And if, it was out by 1Hz, what would happen? There would be an data error before about 22,050 samples. And this would show up in the bit-error rate!

    This is the main reason that MFM coding is used to ensure that clock stability is not lost when there are long streams of 11111111s or 0000000s…

    Now, please can you justify your statement:

    “This is called timing jitter, and it is now understood to result in subtly perceptible distortion in the audio.”

    1. Who calls it “timing jitter”? (I always know is as jitter, the “timing” is implied)

    2. When did this new understanding take place? (Given I knew about it as a child)

    3. How do you experience this distortion in the audio? (Given that it causes no recordable data errors)

  99. rich13 said,

    February 4, 2006 at 7:44 pm

    Jeepers. I just read all of this…

    I’d say we’re getting a bit ‘angels on heads of pins’ with much of this, because as far as I can see, it’s all really about how people try to blind each other with science – a common phenomenon, from hi-fi cables to breakfast cereal and shampoo.

    If someone sells something that makes someone else happy, and there’s an exchange of shitloads of cash, then it’s possible to argue that nobody is losing anything, provided that the person ends up sufficiently happy. Horses for courses and all that, some might argue. Let the emperor think he looks great, as long as he doesn’t go round flashing his bits and frightening old ladies.

    But my general feeling is that the broader effect of using questionable science to sell things – even if nobody is ‘harmed’ – is that it devalues the very idea of being able to answer questions… and know anything approaching ‘the truth’ of things.

    If the manufacterers of these cables say – sorry to shout: ALL OTHER THINGS BEING EQUAL, AND THAT IS THE CRUCIAL POINT, that their products make a difference, then go ahead and test it by whatever means. If they are wrong, and we can show that, ALL OTHER THINGS BEING EQUAL, nobody (ie. no human ear, under any circumstance) can detect any difference between these cables, then we will have done some good, by striking a blow against ‘ignorance is strength’ rubbish – as well as helping people save money.

    If the manufacterers claim that their cables can *cancel out* differences in sound caused by fluctuations in the environment, then we’d need to be testing different kinds of environmental interference, and I don’t know much about that. So… whatever. But this seems to be something that the proposed experiment doesn’t consider. Maybe I have misunderstood, somewhere.

    Finally, I think an interesting – but different – experiment would be to get a group of people who claim to know all about sound differences in a variety of contexts (hi-fi enthusiasts, editors of hi-fi mags, cable makers, hi-fi makers, musicians, sound engineers, and so on) and do a simple thing: play them two tracks, as often as they like – controlling for playing order etc. etc. – and tell them to say which one is produced on a ‘top-quality’ system and which on a ‘lower-quality’ system. You’d get a range of answers: A or B is better, or no difference. We’d then look to see who, in our huge sample, spotted that the two tracks were – in every single way imaginable apart from the listener – identical.

  100. Chris Gibson said,

    February 4, 2006 at 7:45 pm

    Attacking Audiophiles on the Bad Science front does seem a bit like shooting very fat — but aimiable — fish in a very small bucket.

    According to this halfwit, the test will be inherently flawed unless the same brand of CD-R and — yes — the same brand of hard drive is used for each file:

    www.johnvestman.com/digital_myth.htm

  101. rich13 said,

    February 4, 2006 at 7:55 pm

    Or we could go back to arguing about fucking ludicrous minutiae, of course.

  102. rich13 said,

    February 4, 2006 at 8:04 pm

    (That wasn’t aimed at you, Chris G.)

  103. Briantist said,

    February 4, 2006 at 8:05 pm

    www.johnvestman.com/digital_myth.htm

    Here is another person who can percieve more data than originally existed too. But it must be true cos it’s on the internet.

    Funny how I can record 650Mbyte onto a CD-R and read it back OK!

  104. rich13 said,

    February 4, 2006 at 8:08 pm

    (yep)

  105. Briantist said,

    February 4, 2006 at 8:46 pm

    rich13, your logic escapes me. You seem to be saying that you want to be hoodwinked? Can I interest you in some of the latest huile de serpent?

  106. Matt said,

    February 4, 2006 at 9:56 pm

    Of course it’s amusing to expose the cod science used to sell ludicrously expensive bits of hi-fi equipment. But surely the crucial factor here is that perceived sound quality is entirely subjective. Moreover, even from the little I know about psychology and neuroscience, it seems that our perceptions, whether visual or auditory, are governed by a multitude of factors. Think about that famous experiment assessing whether people prefer Coke or Pepsi. In blind tests, most prefer Pepsi, but when they can see what they’re drinking, most go for Coke. Maybe, for complex, (depressing) cultural reasons, Coke DOES taste better if you can see the can. The fantastically expensive hi-fi lead may serve the same purpose. Your listening experience may actually improve with the knowledge that you’ve shelled out loads of cash for the mains lead to your amp. Just like placebo, the fact that someone may have switched the lead for a £2.99 one from Woolworths is irrelevant in this case. But if you purchased and installed the cable yourself, and you’re confident that there are no crazed scientists trying to perform double blind tests in your living room without your knowledge, it could be that you’re paying through the nose for a better listening experience. Of course, I’d have to question your sanity though.

    My point is, the double blind test is no use here, because we’re dealing with the subjective. Double blind testing in e.g. medical research, or exposing quack medicine is very different, because here we can objectively measure effects, e.g. cancer rates etc.

  107. Andrew Clegg said,

    February 4, 2006 at 10:02 pm

    Matt: My point is, the double blind test is no use here, because we’re dealing with the subjective.

    That doesn’t follow. Pain is subjective, yet AFAIK painkiller trials are double-blinded and randomised just like any other drug trials. There’s a difference between subjective and imaginary, unless you’re Bishop Berkeley.

    Andrew.

  108. Frank said,

    February 4, 2006 at 10:41 pm

    Lordy Lordy. I’m surprised so many people are putting Ben down for targetting Hi-Fi enthusiasts. Why should anyone be exempt from backing up their claims?
    It’s not much to ask when handing over £1800 for a metre of cable.

    The whole point of Bad Science (I hope) is to fight the tide of nonsense masquerading behind scientific language.

    Ben was right all along though: Hi Fi enthusiasts will claim sanctuary from the cold methodology of “scientific rigour”, just as homeopaths and crystal healers do.

    On the subjective nature of listening: No, no, and once again, no. Think of it this way people: if a test cannot be designed to properly assess these cables then how the hell did the companies ever develop them?

    Which makes me wonder, how do homeopaths come up with new treatments?

  109. ianc said,

    February 4, 2006 at 10:49 pm

    First time I’ve read this – quite fun. I liked the discussion between the guy trying to explain jitter to the other guy who had an uncle.

    If I understand the original experiment, I’ve got to agree with the guy who said it’s bad science.

    The signal on the line out of the CD player is part digital and part analogue. The sample amplitudes are digital and if there’s any error in those you’re simply using poor quality equipment. But if you want to convert these samples into nice sounding music, they have to arrive at exactly the same time intervals as the original sampling process – the timing part of the signal is analogue. This is quite different from data transmission when you don’t even care in what order the bits arrived as long as they eventually all get there. The experiment primarily compares the amplitude part of the two signals and doesn’t care about the timing as long as it’s not grossly out.

    Having said that I agree with the guy who pointed out that a decent PSU will do anything that an expensive cable might be able to do.

    Personally I find the sound quality on my system depends on where the power is coming from. Here in Scotland the Hydroelectric electrons ar much better than the nuclear ones I used to get in Berkshire,

  110. Pathman said,

    February 4, 2006 at 11:26 pm

    In the case of power leads there really is some difference in performance in equipment between those equipped with cheapo “lamp”-type leads and the same with thicker power leads. Less resistance in the later makes a more powerful amp get more current easier when it needs to put out a lot of power to the speaker. This is something often ignored by penny-pinching manufacturers of low- to midpriced power-amps and receivers even now. But in my own experience this does not really make any great difference in low-powered ones. You need to go up to at least 50- to 60 watts output to hear any difference in sound quality with good speakers. My own reciever is modified to put out up to 100 amperes to my speakers( as measured with test equipment.) at the sound levels listening to operas and live rock concerts and even an unexperienced listener commented on more “open” and transparent sound at mid- to high volume when we did exchange the power leads and played the same music before and after.
    But exchanging the power leads on CD-players and any other Hi-Fi equipment is just plain daft because they are low-powered and just need “spike”-free power. But some Moving Coil pick-up pre-amps are battery-powered becuse of their notorious sensitivity to disturbance by other electrical stuff in the household swiching on and off.
    But any really effective computer-type power-filter takes care of that. Buy one for the computer too! Anyone that have had hours of work wiped out by a disturbance in the mains power-net could see that….
    As for connection cables most users just need to throw away the cheapo free ones they get when buying stuff and buy some with better gold-plated connectors and greater leader area. ANY cable is better than got-it-with stuff!
    Speakers DO benefit from cables with greater leader area(thicker cables) because less electrical resistance in cables means less power lost on the way to the speaker(Ohms Law) and even cheap compact stereos sounds better with new and thicker cable( 2.5-4 square-mm. leader area). You may even use 2.5 sq.mm. three-phase electrical installation cable if you can use “banana-type” connectors and don’t mind thick, stiff white cables in your living room. One of my friends did this when he installed his new 6 x 200 watts home cinema! Sounds quite good, and he got the cable for free at the factory where he works as boss of shipping(ovens, stoves and stuff). He painted the cables though.
    I once saw a contractor use cheapo telephone cable for distributing sound to speakers in a local medical center, and coupled serial instead of in parallel too…
    The total resistance in the whole system must have been at least 1000-1500 Ohms instead of the 16 Ohms that it should have been. Must have been about 0.05 watts delivered to each speaker in the house instead of the intended 1to3 watts, and of course it sounded Terrible, squared. Must have saved the Bl***y contractor about 500£ in money. Ten years they had lived with that before i happened to get asked to take a look at the setup by a receptionist that knew that i sometimes did sound at our local Rock Cafe. I think they got it fixed 5 years later, when rebuilding it all.

  111. rich13 said,

    February 4, 2006 at 11:36 pm

    *rich13, your logic escapes me. You seem to be saying that you want to be hoodwinked? Can I interest you in some of the latest huile de serpent?*

    Riiiiight.

    Well… I don’t want to be hoodwinked. You really will have to re-read my post if you want to make points about it.

    I was saying that *some people* argue that all of this is harmless. Some say that the emperor can do what he likes with his new clothes, as long as he isn’t hurting anyone.

    On the other hand, I think we all agree that *even if* bad science is sometimes relatively harmless (ie. about hi-fi cables and not power stations) it still has very bad effects on society as a whole.

    So no, I don’t want any feckin snake oil. But thanks.

  112. TxB said,

    February 4, 2006 at 11:36 pm

    I’ve actually gone away and investigated (quasi-scientifically) my problem of my SPDIF cutting out. I wanted to try Allison’s suggestion, especially as I have 2 types of UPS at the moment. I thought it would take about a half hour … but 4 hours later I still have no definitive answer. I did manage to establish a few facts.

    (1) I tried the largest inductive load I could find — a 2kva 240-120v transformer (for running construction site tools). It does indeed cause voltage sags and my SPDIF cuts out. I then tried the smallest inductive load I could find — a Braun rechargeable shaver. No one could claim this causes voltage sags, but yet plugging it in causes my SPDIF to cut out more reliably than the 2kva transformer.

    (2) I switched my UPS over to battery, and then tried plugging in my shaver in the exact same position. The SPDIF does not cut out. I then plug the shaver into the UPS local circuit and there it causes the SPDIF to cut out as before.

    (3) With UPS back on mains supply, I connect an extension cord and take my shaver outside my (reinforced concrete) flat — about 15m distance. The SPDIF cuts out just as well as if the shaver were in the direct vicinity.

    (4) With UPS on battery, I try switching on larger loads: (a) my combi-boiler and (b) a circuit which has 3 computers and various small transformers on it. Both cause my SPDIF to cut out even when on battery. I isolate everything onto the UPS system — disconnect network cables, monitor … anything with a possible common ground. This reduces the occurrence of the SPDIF cutting out to about half.

    (5) I try a thicker better shielded SPDIF cable (the normal cable is also screened but very thin). This reduces the occurrence of the SPDIF cutting out, but not entirely. Large inductive loads like transformer or boiler do not cause it to cut out. Shaver and circuit with various computer power supplies still causes it to cut out.

    (6) I tried the spdif coaxial connection on my DVD player with the same connecting cable. I got no interference at all for any of the conditions.

    Conclusions:
    The problem cannot be electromagnetic waves (RFI) generated by the act of connecting the appliance. With the shaver on the mains & the hi-fi on the UPS there was no interference. Also the shaver could be taken a considerable distance away and if on the same circuit it would cause interference.
    The problem is also not noise from voltage sags. A shaver on the charge cycle would be a minute load.
    The problem is also not on the amp or pre-amp side. Analog works just fine with no noise or interference at all. Its just the coaxial digital connection on the sender side. My DVD player has both a co-ax spdif and an optical tos-link and both work fine with no interference whatsoever — and using the exact same cable.

    Now, I like being at the bleeding edge of technology and would love to chuck out analog interconnects as much as possible, but this isn’t the first time I’ve come across some relatively minor but really aggravating problem. Another I have at the moment is my Apple Cinema Display monitor loses its way when my PC graphics card goes to sleep (I return to my computer after a break, turn the monitor back on, and get nothing but blackness no matter what I do — and yes, I did install the latest driver). Lots of times you can just spend ages on stupid things like that — things that should just work but don’t. I can easily spend 10 hours on things like this, and most of the time I find some solution. In the case of the SPDIF problem its annoyed me for years, and despite spending now 5 hours on it today still no solution. Now I like tinkering with such things and have plenty of technical background (thanks to a state funded education) to have a fair chance at sorting it out myself. Most people (not fortunate enough to know someone like me) would have to pay a consultant. 10 hours of work at £200/hr = about £2000. And in this case it would be £2000 costs and still no solution. If you look at it like that, the £1800 power lead starts to look like a bargain! If it does the job of course.

    And that was the problem I have with Ben Goldacre … he didn’t check if the actual lead does the job. Yes, I’m sure someone will come up with a solution that costs £5 in parts (not counting labour of course) BUT the question at hand is: is this power lead ‘snake oil’ (ie. does not deliver what it claims) or does it actually do what it says on the box. From what I have read, most of you are willing to go along with Ben’s position that it ‘obviously’ could do nothing useful. This is an unreasonable unscientific position. Most of the time you could probably get away with such lazy thinking, but not in this case as I have a genuine problem that is capable of putting the matter to the test. If this lead happens to solve my problem (admittedly not very likely — but possible) then Ben deserves to dedicate next weeks column to apologizing.

    Thomas.

  113. Roy Badami said,

    February 5, 2006 at 12:32 am

    Bryantist,

    I think you’re confused as to how consumer digital links such as SPDIF and TOSLINK are used. There’s no master clock here, the receiver (the outboard DAC, soundcard, whatever) extracts the clock from the signal modulation. If the source is transmitting data 1Hz too slow, the extracted clock will also be 1Hz too slow. The clock and data will always match; you will not get data loss; you will not see it in your bit error rate.

    The jitter affects the analogue waveform. In order for the DAC to construct the correct analogue waveform it needs to trigger the sample-and-hold cicuit at precisely equal time intervals. If the time intervals are irregular, this will result in an analogue waveform which differs from that intended very slightly; ie a distorted waveform. What wasn’t fully appreciated in the early days of digital audio is that really quite small errors in the timing of the sample-and-hold can result in perciptible levels of distortion.

    In a consumer setup, the timing interval used by the DAC can be influenced by the source (in this case the CD player), and hence the CD player can cause the DAC to produce a distorted signal even when the BER is zero. If the BER is close to zero (as it should be) then this is the only way in which the CD player can influence the sound quality.

    And yet, as you rightly point out, there is no DAC in this experiment. The experminet is failing to measure the only way in which the source can influence the sound quality (short of high BER, which just shouldn’t happen).

  114. rich13 said,

    February 5, 2006 at 12:32 am

    TxB

    I doubt Ben would need to apologise specifically if “this lead happens to solve [your specific] problem”, as his original article didn’t claim that it wouldn’t,

    By the way – any of you – how many angels *can* fit on the head of a 4-pin connector?

  115. tuppy said,

    February 5, 2006 at 12:41 am

    Er, TxB you’re really getting boring. That’s EXACTLY what people are here to discuss, checking if the lead works, deciding how we check if the lead works, and, incidentally, not your fascinating problems with your home stereo, which you could discuss elsewhere, maybe in the forums.

    www.badscience.net/?a=xdforum

  116. TxB said,

    February 5, 2006 at 1:09 am

    Andrew Clegg:But if there was an ideal ‘reference quality’ hi-fi setup that did have that general-purpose panacea effect, wouldn’t the studios use that instead?

    I just wanted to come back on the point Andrew made which I think is a good point. Good but flawed.

    I have a friend who is in a band as a bass player. He always carries 2 things with him: his bass guitar and his Marshall amplifier (plugs into the bass). I played around with this Marshall amp and while it sounds very impressive when used with the bass (it gives that traditional ‘rock & roll’ sound that we all know and love) its not actually a very good piece of electronic equipment. If you use it like a hi-fi it sounds awful. Thats what I mean by specialized equipment … good for a specific function but not much good for general use.

    I don’t know about studios so I can’t really comment specifically. What I would say though is that studios are manned by technicians, who I suspect are on the whole not audiophiles. Just because they sometimes play a part in the production of a work of genius doesn’t necessarily mean they know what they are doing. They may be experts in their part in the production process, but don’t necessarily understand or care about the process as a whole, or appreciate the final product.

    I don’t buy the same car my mechanic drives or even the car preferred by the designer in Wolfsburg — or even necessarily care about their views. If you talk to them they will go on and on about some LeMans race car. If you’ve ever been in a race car its not a particularly pleasant experience (exciting maybe, but quite uncomfortable). If you are an auto-tech-head thats the kind of stuff you love. Thats quite different to being an auto-phile though, and someone who loves cars will generally love cars from a quite different perspective. There is a great quote from Ettore Bugatti on Wikipedia (en.wikipedia.org/wiki/Bugatti) that illustrates this point.

    The same is true for hi-fi enthusiasts. The tech-head wants a machine with a 1000 settings so he can adjust & fine tune (just look at the cover of the magazine some referred to). The enthusiast just wants to press the on button and get great sound. I usually love tinkering with things but sometimes it can be a huge pain. My partner for example already complains about the difficulty of selecting the input (dvd player, laser disk, cd, TV) of the hi-fi system and the various remotes. Why can’t it just automatically go to the input that currently playing and one remote control everything? Sounds sensible to me, but do the tech-heads care about such things? No! In their studios the’ve got thousands of inputs memorized and they don’t have to care about wives & partners. A handful of inputs … it doesn’t cross their mind to worry about such trifles.

    Thomas.

  117. John Ruch said,

    February 5, 2006 at 1:21 am

    A few general thoughts, not addressing particular experimental designs, but overall approach:

    The key thing is defining precisely what claim you are testing. You can’t simply test for something that sounds “better” or even just “different.” Those are inherently subjective and undefined terms that cannot be tested for. What, precisely, do the manufacturers claim? That’s what should be tested. If it’s a reduction in interference–how much, what frequencies, etc.? Is there an agreed-upon standard of significance for that? If the manufacturers simply claim a “better” sound, that is far too vague to test and will lead to more endless debate. A measurable, physical change is the only meaningful thing to test.

    In the same vein, you should not simply design a kind of stunt to serve as a huge broadside against all believers in this stuff. You simply need a small, highly focused test of one particular, very well-defined claim. Just throwing generic data to the dogs on the web will only lead to more howling and growling.

    You should also have agreement on what constitutes success or failure. You will never find consensus on that in this kind of post. But you and the manufacturer(s) should be able to agree on such protocol. This is an important element of the paranormal-claims testing conducted by James Randi (randi.org), and should not be overlooked.

  118. jr said,

    February 5, 2006 at 1:23 am

    Turns out I like flawed car analogies too.
    If I was going to spend almost a million quid on a car I would make sure that it went faster than 250km/h.
    And to say that under the bonnet of a Mondeo you find shoddy workmanship and poor design is laughable.

    Carry on…

  119. jerry finkhausen said,

    February 5, 2006 at 1:35 am

    mic preamp listening test is good science
    www.goeasygo.com/

  120. Andrew Clegg said,

    February 5, 2006 at 1:39 am

    Thomas, thanks for the response, but I think there’s a few misconceptions in there.

    1. There’s a world of difference between a Marshall stack, which is designed to colour the sound of an instrument, and a studio monitoring rig — especially that which you’d find in a mixing room — which is designed very specifically to avoid colouring any of the tracks you are monitoring (individually or together).

    (I say “especially in a mixing room”, but these days a lot of music is composed and ‘performed’ under these conditions in an entirely electronic way anyway.)

    2. Badly mixed/engineered music will sound rubbish, even on an audiophile hi-fi. To assume that somehow the music makes it from the band to your living room intact, despite an engineer who doesn’t know/care much about what he’s doing, is ludicrous. Engineers who specialise in mixing down from a multitrack have the kind of ears that audiophiles dream of.

    3. I’m not sure what you’re getting at with the last paragraph, but my (sadly now dying) Samson studio amp has two inputs, left and right, two volume controls and two outputs. And a power button. Nothing to do with masses of inputs and outputs and settings, in fact that’s the last thing you want messing up your monitoring line.

    Andrew.

    (I’m still reeling about what you think of engineers… The only reason anything sounds that good on your boy-racer hi-fi at all is because an engineer has spent hours meticulously listening, adjusting, and listening again, until each individual part is audible with the maximum achievable clarity…)

  121. TxB said,

    February 5, 2006 at 2:00 am

    Andrew

    Of course I exaggerate and generalize slightly in respect of engineers.

    “two volume controls” … my personal bugbear, why would I want 2 volume controls which I can never get lined up? I can just imagine my partner trying to increase the volume on the remote — first the left then the right. I think you make my point for me!

    Anyway, my point is a general one. Read the Bugatti quote for the general idea. I am sympathetic to your point of view and on things in my area of expertise I would take your line as well. But I think Bugatti makes the point very elegantly. The point ultimately is that I want the engineer to produce something I want, not something he wants to foist on me. Think Steve Jobs & Apple … why does the non-computer user love Apple so? Why was ipod such a massive success?

    Thomas.

  122. TxB said,

    February 5, 2006 at 2:01 am

    Yes, sadly real science is usually boring. True, Ben Goldacre does not claim that the cables do not work but he does implies it with his ‘pilot study’ — and thats where the problem is.

    >I>The manufacturer of my expensive power cable has assured me that it will have an impact on the sound of an expensive CD player, or a cheap one, and that it will affect optical outputs as much as normal phono outputs. So I propose, as a pilot study, to place, on badscience.net, two files, clearly labelled, one recorded from the CD player powered by the expensive cable, one powered with a normal kettle lead.

    What he does do is propose something which is just plain bad science. The ‘Bad-Science’ is the proposed ‘pilot study’ — see posts above for detailed explanation. I put forward my problem because it precisely debunks Ben’s proposal and shows up sloppy reasoning on his part — precisely what he is so keen to debunk in others. Keep in mind that even the most beautiful scientific theory can be slain by a single ugly fact. Even if he is 99% right, that 1% is enough to slay the dragon.

    Especially when he derides the ‘hi-fi freaks’ not once, not twice but 3 times:

    And that was when I started to notice the frightening similarities between the thought processes of the alternative therapy fans and the hi-fi freaks. Both make an appeal to personal experience, as the highest and most valid form of measurement; both use mystifying, scientific-sounding terminology in their publicity material; and both use the appeal to authority.

    Some hi-fi enthusiasts are also scientists themselves and after such a cruel assault are eager to hold him to account and put his column up to scrutiny. I am sure a humble apology in next Saturday’s Guardian would warm the heart of many. Of course its all I hope all the spirit of good fun — not too be taken too seriously.

    Thomas.

  123. Andrew Clegg said,

    February 5, 2006 at 2:16 am

    You’re conflating two entirely different kinds of ‘engineer’ here due to an unfortunate terminological clash. The equivalent of Bugatti or Jobs is the guy who makes the audio kit. That’s all a red herring.

    A studio engineer, especially a mixing engineer, is the person who’s job it is to make sure that the music on the CD sounds good and clear, appeals to the listener, and doesn’t retain any artefacts of the recording process. Hence he must monitor on a system that doesn’t introduce any changes to the sound, just makes it louder.

    Now I brought this up because earlier you were saying audiophiles “want the best reproduction [they] can possibly have.” I’m arguing this point because I’m beginning to suspect it’s not about that at all, but more about bling-waving. If you really want accurate reproduction, get a studio monitor style setup, because that reflects the conditions under which the original recording was made in the first place… Otherwise, what are you actually hoping to reproduce?!? A listening environment that was never there?

    BTW the two volume controls thing was never a problem for me, they’re stepped, and I just left them in place and controlled the input volume anyway. But that’s a side issue, plenty of studio amps don’t work that way.

    Andrew.

  124. David Bober said,

    February 5, 2006 at 2:59 am

    I agree that a blinded test is a good idea, but your idea of putting the audio files on the internet is a flawed test. It does not reflect the practical application of the use of the power cables.

    People who listen to high end hi-fi will not listen through small computer speakers to audio that has been sent in packages over the internet.

    Audiophiles will listen in an acoustically decent room with high quality speakers, cd player and interconnects. A fair test would be to have people come to the room, listen to several different recordings, and decide whether the power cable is the ridiculously expensive one or a kettle lead .

    This will be much more expensive and allow much fewer people to take part, thus reducing the statistical power of the test; but the test proposed in your article does not reflect the true listening environment in which these cables will be used, hence the results are likely to be close to meaningless.

  125. jr said,

    February 5, 2006 at 11:19 am

    TxB:

    You realise that VW are making quite a financial loss on each Veyron don’t you? I wonder what the manufacturers of £1800 kettle leads are losing? Sleep I would hope but somehow I doubt it.

  126. TxB said,

    February 5, 2006 at 12:09 pm

    Andrew:A studio engineer, especially a mixing engineer, is the person who’s job it is to make sure that the music on the CD sounds good and clear, appeals to the listener, and doesn’t retain any artifacts of the recording process.

    Now I brought this up because earlier you were saying audiophiles “want the best reproduction [they] can possibly have.” I’m arguing this point because I’m beginning to suspect it’s not about that at all, but more about bling-waving.

    On this specific point I would agree in principle if not in practice. It is something most hi-fi enthusiasts are not entirely ignorant of — many speaker manufacturers for example advertise themselves on the basis that they are used as ‘studio monitors’. My speakers for example are called ‘Monitor Audio’ — although I don’t think they are used by anyone as studio monitors.

    However, as with most things there is a ‘bling-waving’ factor. The equipment has to be technically sound, but starting from that basis the second factor an enthusiast will look for are aesthetic qualities. Studio equipment generally has been made with no consideration given to aesthetics. Most hi-fi enthusiasts are very reluctant to buy something that is big & ugly. I don’t think its unreasonable to expect something that is a technical masterpiece also to be a thing of beauty. Even the studio engineer would I think choose beauty over beast if cost and technicals were not an issue.

    For voice recording for example I know most professionals use a Sony digital recorder which to my recall costs about £1000 — but its the size of a brick and you can search for some time before you find a ‘play’ button. Why should I buy this monster for playing music when I can buy an iPod for £100 which will give just about the same quality output (not quite as good, but probably inaudible), has a very intuitive interface, excellent management software, is the size of a deck of cards and looks sleek & smooth (ie. impresses most people).

    In general though, if I remember rightly, a studio will be a room about the size of a living room but is jam-packed with pieces of equipment, wires everywhere, control panels with thousands of buttons, speakers in every corner (for some reason they seem to need more than 2 speakers) etc. Some enthusiasts have a ‘den’ or a basement where they can put together an optimal listening environment. Most people though want a multi-use system — one which accurately reproduces music yes, but which also will suit their entertainment/living room — where yes you want to entertain people, people who you want to be impressed by your hi-fi system. Most people I know do not really care for monster speakers, cable jungle and oddly shaped boxes that clutter their living room. My partner for example complained bitterly when I wanted to put in 2 rather discreet extra speakers for a 5.1 system.

    So, in general I think most decent hi-fi components (yes, including iPod) do a perfectly adequate job of reproducing sound. In fact, technical standards are so high that most people cannot hear any actual difference between mid-range & high-end — or studio equipment. This is more so now in the age of digital interconnects. The only area of contention really is with speakers. So, yes, if possible get studio speakers. But nevertheless, if you have only £1000 to spend on speakers then there are perfectly good speakers on the hi-fi market. Even if you spend £10,000 on ‘designer’ speakers over studio speakers (yes, wife or partner might object to monster boxes held together by baling wire) then its still a perfectly adequate choice. If they are up to scratch technically (which most are) they might be slightly inferior to the studio but they will still do a perfectly adequate job. A studio engineer might shake his head at paying £10,000 for something that to him would be worth only £2,000, but in a blind listening test he probably couldn’t tell them apart from his own £10,000 studio speakers.

    Thomas.

  127. Mike said,

    February 5, 2006 at 1:50 pm

    Hi,
    I have rewired my entire house using multiple £1800 power cables. I have also rewired it all the way back to the local sub-station. I can really hear the difference in all my stereo equipment. Also the doorbell now sounds a lot clearer. These so called “scientists” don’t know what they are talking about.

  128. PInk George's Pants said,

    February 5, 2006 at 2:29 pm

    I’ve followed TxB’s example and have had my ears re-wired and my brain replaced with that of a dog. Those high frequencies have never sounded sharper. It doesn’t make much difference to my stereo but I can hear that bit near the end of Sgt Pepper now. You haven’t lived if you haven’t heard it. It sort of goes “peeeeeeeeeeeeeeeeeeep”!

  129. Tolstoy said,

    February 5, 2006 at 2:58 pm

    How will the ability of users to submit data without having listened to the file be controlled? If it can’t, the experiment is flawed because of the potential for people to submit junk answers. This will lead to the data being too noisy so that any real difference between files would be undetectable.

    A better design would be double blind, the same track played using each of two or more leads, one of which is the ordinary kettle lead. Participants should be played the two tracks in a random order, under identical conditions and should be asked to declare which they thought sounded better, or which sounded like it had less static interference (you’ll have to nail the wording of the question down by refering to the manufacturor’s claims).

    If you use more than one of the expensive leads, you’l have to control for multiple comparisons which means that the sample size will have to go up. You also need to be sure that the trial is properly randomized. (For an example of what can go wrong if you fail to properly randomize, type “Sheldrake” into Google.)

    What size of effect are you expecting? Supposing that you study only one of the expensive leads, then if there is truly no difference between how the track sounds with each lead, you’d expect 50% of people to correctly identify the expensive lead. So the null hypothesis is H0: p = 0.5. You need to write down the alternative hypothesis in order to be able to size the trial.

  130. Steve Jessop said,

    February 5, 2006 at 3:27 pm

    Ben, your proposed test sounds very weak, because it cannot even falsify the relatively strong hypothesis “most people can tell the difference between power leads”. The best it can hope to test is the hypothesis “some recording system which the experiment designer came up with produces recordings from which most people cannot distinguish power leads”.

    Firstly, when you say “the manufacturer says that it works with cheap stereos and with optical out”, I can only assume that you asked this because you’re more interested in falsifying the claims of the manufacturer than those of the various audiophiles who have vented in your direction. In the likely event that your A/B/X test produces a negative result, will your experimental conclusions make it perfectly clear that your findings relate only to some marketing hype which most audiophiles don’t believe either, and have no bearing in general on claims made concerning power leads? If not then this is guilt by association, not science.

    Secondly, last week you were talking about finding audiophiles to use as the test, presumably to maximise the chances of finding any real effect, but now you’ve backed off and are going to put a microphone between them and the test equipment. You might as well let them come round your house to listen, but make them wear earmuffs!

    Third, look at how you are selecting your sample. Is there not a real risk that by putting the recordings on a website and letting people vote, that you will include the votes of many listeners who would not expect to discern the difference even if there was one? I’m no statistician, but won’t that significantly reduce the significance generated by any subset of the sample who can reliably tell the difference?

    I was curious to know whether these power cables actually make a detectable difference, and last week I though you were too. This experiment seems designed to maximise the chances of you concluding that the lead makes no difference. I’m extremely disappointed – it’s starting to look as though you’d rather “prove” yourself right than find out the truth of the matter.

    I realise that you describe this as a “pilot study”, but isn’t it unusual for a pilot study to be designed to systematically mask the supposed effect being studied? My naive assumption last week was that you’d invite (say) 10 self-professed audiophiles to carry out a live test, perhaps using the equipment of their choice (minus power lead) in the environment of their choosing. You’d then switch leads in a single-blind test. The audiophiles would expect to all get it right, whereas the null hypothesis expects only 5 of them to do so, with the chances of at least 8 of them identifying X being a little over 5%. So at least somebody will be surprised by the result. I don’t see how you are going to get a meaningful result from your online pilot without resorting to some powerful statistical methods and risky assumptions about the sample – something that you said last week you wanted to avoid in order to make the result accessible to the layman.

  131. Tolstoy said,

    February 5, 2006 at 4:23 pm

    If it is the ability of audiophiles to detect a difference, then audiophiles should be put in the study and no one else. If the experiment descends into searching the data for a subset of “responders”, then it has failed.

    A “pilot study” is usually performed because the investigators cannot afford to do the study they are actually interested in. Pilot studies can never be conclusive, so in this case it would be counter productive to do one. A properly designed and sized study should be done instead.

    Clever statistics cannot rescue a duff experiment. A statistician can help get it right at the design stage, but cannot revive a dead experiment.

  132. rich13 said,

    February 5, 2006 at 4:37 pm

    I agree that this pilot study is looking less and less useful (but not because I disagree with Ben on the general idea of making sure these claims are looked at.)

    I still think it might be important to do a big study first to see how much noise there is in the system – if you’ll excuse the pun. We’d need to know first just how people of different backgrounds (hi-fi buffs, sound engineers, serious fuck-off academic ninjas, normal humans etc.) respond to sound differences that DON’T EXIST. ie, play identical tracks with and without the ‘bling-factor’ (or at least implied bling), and see if anyone insists that it sounds better.

    Otherwise, any subsequent experiment really just depends on who is listening – let alone all the other arguments about effects being masked by other factors.

    All of this could be cancelled out anyway by doing an experiment as described by Andrew Rose (comment 3 – remember back then?!) which tests to see whether there is any physical difference between the sounds produced, without the interference of biased human ears.

    The End.

    Ben, please go back to the BAD bad science, rather than the irrelevant (but still bad, if that’s what it is) bad science. I think you’re doing good stuff, but this is barking up some less interesting trees, don’t you think?

  133. Tolstoy said,

    February 5, 2006 at 4:46 pm

    What is the noise in the system? Under the null hypothesis, there’s a 50% chance of a listener identifying the right lead. If, say, the alternative hypothesis is that there’s a 60% chance of identifying the right lead, that’s all you need to know. The variance (maybe what is meant by noise) is fully determined by the sample size and the probability of listeners identifying the correct lead.

    The suggestion under comment 3 seems good, but does not directly address the question of whether or not listeners can hear a difference. For example, it could be argued that the process of recording the tracks stripped out the difference between leads. If the question is around whether or not human hearing can detect a difference, then the experiment needs to study human hearing.

    I agree that having a dig at the manufacturors of kettle leads seems less valuable than examining the claims of fake healers and psychics and the like.

  134. rich13 said,

    February 5, 2006 at 5:09 pm

    You’re right, Tolstoy. I realise that it’s a separate thing that I’m going on about…

    It occurred to me that it might be interesting to see how strong the emperor’s new clothes (or whatever) effect is, by getting a) people who are likely to buy 1800 quid cables and b) some kind of ‘normal’ group (tricky, yes…) and seeing how they respond to identical sounds coming from setups that they believe to be of different quality.

    One hypothesis to test might be that hi-fi buffs hear sound differences where there are none, based on what they know about the system, while everyone else just goes 50/50 for the ‘high’ and ‘low’ quality tracks, or says that they can’t tell.

  135. Andrew Rose said,

    February 5, 2006 at 5:15 pm

    To Steve Jessop – Ben’s made it clear that there are no microphones involved in this test. This is not a test of microphones, or speaker output, but whether (as the manufacturer claims) using their lead will alter – and more specifically improve – the digital output of a CD player.

    Thus the files that will appear on the Internet will be direct digital clones of the output of the CD player.

    To David Bober – anyone foolish enough to try this out using their tinny little computer speakers rather than burning the tracks onto a CD-R for listening on a decent hi-fi set-up needs to think again. Personally, when I get the files, I will be listening on the speakers connected to my computers – but then they are £10k studio monitors…

    But better than that I’m going to do the obvious – a bit comparison. This will show if there’s any difference between the two files – once I’ve checked the in and out points match to the exact sample.

    And this is where the site referred to earlier (“The Digital Myth”) is complete rubbish. It’s so simple to find out whether a digital signal is identical or different without resorting to listening tests that all this rubbish about different CD-Rs having different sounds can be put to bed in an instant.

    Oh – and to TxB – I suggest to pay a visit to a decent recording studio or mastering suite – maybe at Abbey Road, or perhaps a tour of the BBC’s recording facilities. Ask them how much time and effort went into the design of the studio acoustics and the choice and positioning of speakers and other equipment. You might be surprised to learn that a huge effort goes into designing out any possible sound flaws – use of sound-dampening and non-parallel surfaces etc. – so that the mixing engineer gets as clear an impression of the sound as possible. That’s how we do our jobs. Perhaps that’s why some of us win awards and others don’t – I certainly wouldn’t want to master anything using hi-fi speakers unless I had no alternative. Whatever – you certainly wouldn’t continue to spout ignorant nonsense in support of your personal prejudices.

    Unfortunately many of the comments in this discussion display little more than an abject lack of understanding of audio, and in particular digital audio. It’s a complex and specialised subject which does indeed involve a lot of mathematics and science, and as such this comes as no surprise. It’s also no surprise, therefore, that the manufacturers of hi-fi huite de serpent find plenty of takers amongst a well-off community who ran out of analogue tweaks when they mothballed their record decks yet still feel the urge to spend…

  136. Ben Goldacre said,

    February 5, 2006 at 6:17 pm

    Just a few thoughts as this goes on.

    To those who say this is irrelevant, I would point out that this thread is already, only on Sunday, longer I think than any other on the site, and that there are perfectly intelligent (albeit some rabid and rude) people on both sides of the argument who say they are absolutely certain of the answer to the question of whether these leads have a perceivable effect on sound, and they completely disagree.

    Secondly, I think this is an incredibly interesting issue, psychoacoustically, in that some people can be absolutely certain that they perceive a difference in sound, where other people do not: I think that deserves serious enquiry, looking not just at whether a difference can be perceived under blinded conditions, but also more wily work looking at how definite people are that there is a difference between different sounds, how big they feel that difference is, how people rate their own listening ability, and so on.

    Thirdly, how can I say this more clearly: the company themselves say it will affect the optical digital outputs, and that is what we shall be testing. If you find that improbable then your beef is not with me, they were very clear on the matter.

    Lastly, I think there is some confusion about what a pilot study is. A pilot study is a smaller/cheaper/easier one you do to crack off, open the bidding, see what you find, before you do a bigger one, which you may or may not go on to do, depending on your results. Having said that, if there’s anybody who’s feeling very angry and impatient about this, and doesn’t have a day job, and wants right now this week to go ahead and organise a room, a sufficiently large number of audiophiles to attend, deal with the logistics, develop a testing protocol, sort out how to blind the test, get the cables, get the audio equipment, etc, right now this week before their head explodes, and invite me to watch then that would clearly be great too.

    I think it’s an interesting and arguable trade off in this case of what I’ve called a pilot study anyway: the larger sample size of an online test is a huge benefit, but the roundabouts are some other methodological problems, which we should look at working around. We can, for example, invite people to state in the vote whether they are an audiophile who burnt to CD and used £1500 speakers, or used their rubbish computer speakers, and maybe even divide the collected data into two groups based on this. I also think it’s very useful that once the files are up here, people would be able to take the files and do some more data analysing type work on them, like what Andrew Ross has suggested.

    Now, does anyone have any feelings on what music do we use, how long are the snippets, do we need more than one style, and where does anyone suggest we get them from without infringing copyright?

    I should say as a footnote that because the thread is getting so long, I am definitely now going to delete any posts from people who are ranting and raving about how indignant they are that we should even think to test these cables, or are rude and silly, or post at length and completely off-topic about their personal home stereo issues. Actually, I might even just go through and prune the worst of them out right now, completely arbitrarily, just for fun, so beware. The forums are here, you can do exactly what you like there, because you don’t get in anyone’s way.

  137. Andrew Clegg said,

    February 5, 2006 at 6:39 pm

    Andrew R. — as someone who sounds like he knows his stuff — what are your opinions on the comments earlier about how any difference in digital output might manifest itself not in altered bit values but clock wobble? Is that a red herring, or a potential source of audible variation that would be glossed over by re-recording the digital signal directly?

    Andrew.

  138. Tim Day said,

    February 5, 2006 at 6:45 pm

    On that second issue….a lecturer of mine once gave a very interesting demonstration. He read a review of a peice of Brahms then played it (the review was largely positive). He then read another review of another performance (negative this time) and played that. Then he asked how many of us could tell the difference. About two thirds of us said we could.

    Needless to say the performance was exactly the same one…..

  139. superburger said,

    February 5, 2006 at 7:54 pm

    Might I suggest that in this test a suitable music/sound sample might be some or all of the sounds which were sent to space with the Voyager mission in the 70s.

    The probe contained a gold plated disc of sounds supposed to represent Earth. There is classical from Brahms and Bach, rock from Chuck Berry as well as a variety of ‘world’ music. There are also spoken greetings in all the world’s major languages.

    I certainly think that this represents a broad enough sample of the world’s music and some/all of it woul provide a good reference set of sounds. For what it’s worth, a sine wave at 512Hz (middle C, I think) is a good idea. I don’t think that anyone could argue with the breadth of sound on offer.

    The NASA website has details voyager.jpl.nasa.gov/spacecraft/languages/languages.html

    Perhaps NASA itself would be kind enough to lend badscience the masters for such an important study! I myself would be only too hapy to provide a kettle lead.

    An additional study would be to investigate whether and £1800 kettle lead makes a nicer cup of tea. In fact, the two studies could be very easily combined. Half an hour of listening to music followed by a nice cup of tea……..

  140. Tolstoy said,

    February 5, 2006 at 8:15 pm

    It might be worth doing a pilot if you wanted to get an estimate of how big the effect might be so that you would have something to go on when designing the main experiment. I suspect that to detect an effect of 0.1 (i.e. 0.5 vs 0.6) would require quite a small sample anyway (but can’t be bothered doing the calculations just now).

    Having a larger sample size is not necessarily a benefit at all. From memory (which, at this time on a Sunday is a rather unsafe way to proceed), the homeopathy study you rightly criticized in the Guardian a few weeks ago had a large sample size. The design of the experiment rendered the sample size irrelevant, though. No matter how many data it collected, it could never show anything of any interest, by design.

  141. Ben Goldacre said,

    February 5, 2006 at 8:21 pm

    Like I said, it’s swings and roundabouts. Can we cope with that?

  142. Wiretrip said,

    February 5, 2006 at 8:40 pm

    A few week ago I posted a comment that asked about the placebo effect of having spent £30 on a lead and the effect this would have. It was not just a facetious comment. What I was getting at was what someone has since named the ‘Cashebo effect’ by Martin Burley (first post). My point was and remains that since hearing (and to be honest, any of the senses) is so dependent upon the emergent result of complex interplay of neural activity (and this includes the reassurance – whetever internal effect this has on the brain – of having spent serious money), that in these circumstances an interpretive approach (as opposed to classic Popper-paradigm Western science) has to be considered in evaluating such things. Double-blind-control style hypothesis disproving is *not* the only valid research paradigm in existence you know. … not that I’d ever spend £30 on a power cable, my hearing is not that good :-) I know my limits…

  143. Tolstoy said,

    February 5, 2006 at 8:50 pm

    I’m not sure. You attack bad science (and thanks for that, by the way), but the “pilot” you suggest seems flawed. Do you want to open yourself up to being accused of doing bad science?

    I don’t mean to be overly negative. I know very little about hi fi and am not pretending to be an expert in the subject. I also don’t have the time or resources to do a properly controlled trial. I do know about design of experiments, though, and I doubt the value of the pilot.

    Presumably the pilot would be a pilot for a more rigorous study. If you want a hand designing and sizing that, I’ll be happy to help. I have several years of experience designing and analyzing clinical trials. I expect you can get my email address from this if you want it.

    Again, soz to be negative but if you’re going to attack bad science you should be whiter than white imho.

  144. Ben Goldacre said,

    February 5, 2006 at 9:03 pm

    You must design some pretty amazing clinical trials if they are so whiter than white that they make absolutely no concessions to pragmatism, even as pilot studies. Show me this pilot study, or even a clinical trial, that makes no compromise at all?

  145. Plewins said,

    February 5, 2006 at 9:08 pm

    If an experiment can demonstrate that people’s hearing experience is based on belief, then Ben is right in his hypothesis, which is, as I understand it, that these cables themselves do not make any difference. I like the idea of testing this by lying to participants about the cost of the cable.

    On the other hand I think its important for us to begin to understand how ‘value’ does affect our experience. I know I pay more attention to music in a concert when I’ve gone out of my way to be there and will hear it only once, than at home where I know the piece is available on demand. But it IS important to distinguish the hypothetical difference in the cable from the hypothetical difference in the brain. If we can do that, perhaps we can find a better way of adding value to our experience than shelling out cash to the unscrupulous.

  146. Ben Goldacre said,

    February 5, 2006 at 9:33 pm

    We’ve been chatting about this here at badscience mansions just now, and apart from the fact that it will clearly be useful and interesting to have the files up for people to use whatever they do with them, here’s an imaginative, easy, and potentially useful bolt on:

    the voting system will be an email anyway, with the subject heading saying “cableA” or “cableB” so that they can be quickly automatically counted. this helps avoid duplication of voters, and people mucking about clicking, amongst other things.

    so, in the vote, we make people say whether their listening conditions were high end or not. but we also make each voter describe the equipment they used, and if we ask everyone to include their phone numbers, then we can call some at random and check they’re not bullshitters. that might give a reasonable view of how many were fake votes.

    i’m finding this incredibly interesting, btw, especially seeing people’s ways of coping with the idea of non-perfect tests, as all tests are. do you try and work around limitations, or throw the whole caboodle out the window, etc. v interesting. yes kids, YOU are the experiment here, muhahahahahaaa.

  147. Tolstoy said,

    February 5, 2006 at 9:44 pm

    I haven’t claimed that there are are clinical studies with no compromise. Ethics often makes compromise necessary, even if nothing else does.

    You criticized the homeopathy study for selection bias and for the fact that it collected data from people predisposed to give the answer they thought was expected of them.
    www.guardian.co.uk/life/badscience/story/0,,1651259,00.html

    How are you going to exclude those effects when participants will be your own readership?

    This seems to be turning into a row and I’m not interested in that. I’ll go away.

  148. Ben Goldacre said,

    February 5, 2006 at 9:51 pm

    no row, apologies. in the study you’re talking about their was no control group at all, that was the biggest flaw, and the subjects were not at all blinded, they all knew what they were geting, so could give the answer that was expected of them. thats certainly not a problem here: people are completely blinded, so they can’t give the answer they think is expected of them, and act essentially as their own control.

    the only thing we can’t be sure of, i would agree, is that people who come to the question with a great deal of scepticism might give slapdash quick decisions at random about whether one sounds better or not, because they don’t believe there can be a difference. having said that, i’d be surprised if many people went to the bother of downloading the files, listening to them, emailing their vote, describing their equipment in their vote email, and putting in their contact details, if they hadn’t bothered to listen fairly carefully. also, we will be asking people to self-identify themselves as audiophiles with swanky equipment or not. although of course if they want they may be dishonest there, as an act of wilful nastiness, i agree, which is always a problem with self assessment questions as a way of gathering data.

  149. Wiretrip said,

    February 5, 2006 at 10:37 pm

    Come on… the only way this can be scientifically measured is by comparing the actual resultant waveforms of the files. As for the listening experience, we’re dealing with perception, this is not the same as finding out if a drug works or not (a chemical reaction). The listener’s culture, background and disposable income together with the price of the cables and ‘musicality’ of the sound – whatever that is – are all variables too here – also the sensitivity of their hearing, the condition of their cochlea, auditory nerve function, life experience to date…………

  150. Ben Goldacre said,

    February 5, 2006 at 10:43 pm

    i don’t know about that, you’d probably find small differences between two files of the same piece recorded with the same power cable, let alone between different power cables, and then you’d be left trying to explain why one set of data differences were more or less musically important than another. the ultimate test for something that claims to change the listening experience is listening, under ABX blinded conditions, i would say: ideally with everyone in the same room, but as an interesting quick substitute, in the meantime, everyone in their own room, having been given the files.

  151. Andrew Clegg said,

    February 5, 2006 at 10:50 pm

    I would have thought that, apart from lengths of silence at the beginning and end, a recording made entirely in the digital domain from a digital source should come out exactly the same each time. Like copying a file..?

    Andrew.

  152. Tim Day said,

    February 5, 2006 at 11:41 pm

    The manufacturers say otherwise…..and they don’t say “it’ll sound better due to the placebo effect”.
    Personally I’ll be inverting one of the files in Audition or something for my own amusement and seeing if there’s a difference that way. But I’ll do the ABX as well.

    That early post of Wiretrip’s is a strange one… “an interpretive approach”? On a factual claim? I’ll stick with western science thanks.

  153. Jason Steward said,

    February 6, 2006 at 5:42 am

    HHMMMM….people want conditions “perfect” in order to do the test. If I remember correctly, you attempted to set up a “live” experiment. That seemed to fall through when the audiophiles deicded that to test a hypothesis so obviously true (that, some high dollar power cord is going to make Milli Vanilli sound “better.”) was bording on the blasphemous. Basically, “We don’t need no stinkin’ experiments.”

    Science isn’t always perfect. Science is messy. You experiment, you collect data, and you analyze data. What do they tell me? You go back over your proceedures…find ways to improve it. It is a process of failures and seemingly countless iterations. But, you have to start somewhere.

    I see no need to wait untill you study is 100% perfect. You can’t control for everything, you never could even on your best day. So, you make up for it in your sampling.

    Some say , “Gosh, you know, ordinary people won’t be able to hear the difference, so your study will be flawed.” So, do the advertisements apply to only 0.0005% of the populations of a given country? Is the advertisement not misleading since only certain experts “know” they hear a difference? If so, some new leagalise needs to be added: Claims only true for those with excpetional hearing capabilities. If you don’t hear the difference…..you’re obviously not one of the chosen. Double-blind studies not applicable to audio equipment.

    Back to the point: You need to start somewhere. Do it. It can always be refined later after peer review.

  154. Evil Kao Chiu said,

    February 6, 2006 at 10:18 am

    I think you should post the files if only to cut short TxB’s recitation of his fascinating problems with his domestic central heating.

  155. Sockatume said,

    February 6, 2006 at 10:37 am

    Wiretrip: You’re right that the nature of a difference in sound will be percieved differently, but the test here isn’t to determine what the difference is, it’s to determine whether one exists in the first place.

    Martin Burley: “cashebo” is the best new word in years. I’m going to use it at every opportunity.

    I personally have a completely different experience regarding audio quality depending on the time of year I’m listening, the time of day, my health, how much I’ve been sleeping, and so on. For my money, the best setup for music is 3pm in the afternoon of summer 2004 in a flat on Gorgie Road while it’s sunny, with the window open, while listening to a cheap 7.1 system in Neo:6 Music mode connected to a cheap DVD player, drinking Co-op own brand orange squash and reading Chemistry World.

    Failing that, most things sound awesome after an exam.

  156. John A said,

    February 6, 2006 at 10:51 am

    Experiment sounds interesting but:
    1) If the test sample (X) is the same file as one of the match samples (AB) then I suspect it would be possible to tell the difference by looking at the waveform (the solution is to not do this but to do a repeat recording obviously).
    2) More importantly, I am concerned with the lack of control over test conditions. If people start guessing and posting the “answer” then there may be hidden correlations in the data points which might mess up the statistical tests (which assume independent samples I believe?).
    3) I like emails to stop repeat trials but you’d have to promise not to “name and shame” any individual (not that you would).

    How about a large number of pregenerated ABX trials with different recordings of different sounds. The listener opens a web page with three (or more) wav links corresponding to one (or more) of these trials (chosen at random every time the page is requested) and then clicks a mailto link at bottom of the page to submit the answers. The email would contain a session code for that user. I guess this code could just simply be the trial number(s) – but my intention was to stop different people repeat emailing the answer for a single trial. If the trial number is clear from the code then #2 is still possible. A random code could be logged in the database along with the corresponding trial number(s). Although #2 may be a pedantic point the above solution would also allow different recordings of different sounds to be incorporated into the test.

    I believe that if the result is no difference the audiophiles will still have several outs although I agree that nothing is 100% perfect. However I do really like this as a way of communicating a simple experiment to people (provided there is sufficient discussion of its flaws).

  157. Andy Pearce said,

    February 6, 2006 at 11:10 am

    QUOTE:(editor of Stereophile) … then perhaps an alternative hypothesis is called for: that the very procedure of a blind listening test can conceal small but real subjective differences.

    Its obvious isn’t it? They must have tied their blindfolds over their ears by accident….

  158. Chris said,

    February 6, 2006 at 11:19 am

    Surely that’s enough talking. Please post the files, Ben. I’m eager to begin.

  159. Michael Harman said,

    February 6, 2006 at 11:23 am

    Someone suggested that any positive results would be swamped by the noise of large numbers of spoilers, people listening on duff equipment (computer speakers), and so on.

    Maybe this could be overcome by going through the results to pick out those who said they heard a difference, and running a repeat test on those people? That way, we’d be selecting those who claim/appear to have exceptional listening ability. (Of course, if the original test proves positive, we won’t have to go in for that.)

    One problem is that with just a single item being tested, there’s a 50% chance of being right randomly. The proposal is to choose a selection of different types of music. Maybe the different types could be tested separately? That would mean pretty much the same total amount of test tracks (A, B, and X) being put up, but put out as A1, B1, X1, A2, B2, X2, etc, rather than (A1+B1+..), (A2+B2+…), (X1+X2+…).

    But what if the difference (assuming there is one) is different for different types of music (and/or different skilled listeners detect the difference for different types of music)? I suppose one answer is to expand the test, by having several samples of each type of music. But that would mean multiplying the number of test tracks considerably.

    On a more general point, what seems to be going on here is a kind of peer review of the design of the experiment, rather than of the experiment’s results. How far does this happen with normal scientific work?

  160. JK said,

    February 6, 2006 at 12:12 pm

    It looks to me as if most of the people who will do this test will be quite committed to doing it well. So why not improve your power with a repeated measures design, and collect confidence judgements. You might be able to use some kind of ROC / d’ analysis to quantify the relative effects of the two factors (subjects’ listening equipment; recording equipment with fancy cable).

    You’ll always have the problem that if you don’t reject H0 it will still be possible to claim you’ve made a Type II error. To address that you could try to get the cable makers to give you a quanitified estimate of the effect size and then do some power calculations at a ridiculously low beta. Trouble with that is they will say it’s a qualitative difference, so you’re running into the painful interface between quantitative and qualitative stats.

  161. Horace said,

    February 6, 2006 at 12:34 pm

    On a more general point, what seems to be going on here is a kind of peer review of the design of the experiment, rather than of the experiment’s results. How far does this happen with normal scientific work?

    Peer review is itself a critique of the design of the experiment, that’s how they critique the results, you don’t just say you don’t like the results. This part of the design process would normally be done by chatting amongst yourselves, a bit like we are, only without people butting in to post three page descriptions of their central heating problems and demanding that you apologise for trying to do an experiment. Actually I suppose that last bit does happen with animal rights protesters and homeopaths.

  162. Chris L said,

    February 6, 2006 at 1:03 pm

    I find this whole area quite fascinating. As someone who is happy to admit that I know very little about how digital audio is encoded on / read from a standard CD, can someone answer the following? I would do the experiment myself, but I’m at work and don’t have any CDs / ripping software handy.

    1) If I were to rip the same CD onto the same computer more than once (assume ripping to 44.1kHz /16-bit WAVs), would the files be expected to be identical, at the bit level? I would suspect they should be, allowing for the significant error-correction built-in to the CD standard.

    2) If I were to rip the same CD on a different computer (or just a different CD player), would the files again be expected to be identical?

    3) If I were to burn the tracks to a CD-R, and then rip that CD-R back onto the hard drive, would the files again be expected to be identical?

    If the answer to any part is ‘no’, please explain what the expected source of the differences is, roughly what degree of difference would be expected (eg x samples per 1000 would differ by >= x bits), etc.

    Cheers for any info…

    C.

  163. GWO said,

    February 6, 2006 at 1:13 pm

    Chris L : Yes, yes and yes.

  164. HowardW said,

    February 6, 2006 at 1:19 pm

    I am all in favour of such testing, even a pilot test such as the one proposed. Yes, it may be flawed/incomplete/imperfect, but it’s a very useful “dipping of the toe in the water”. As Ben pointed out, doing the full Monty requires an awful lot of time and preparation. Would be great to get this set up, but the logistics are a nightmare.

    A couple of thoughts:

    1. Probably impossible to avoid, but I worry that if the pilot tests A/B/X test is setup, the “believers” may claim that some aspect of the recording was not right and this has masked any possibility of generating audible differences in the clips. This might be anything from not having left the amp warming up for at least an hour before the recording, or the failure of the experimenter to walk three times round the room chanting Ommmm before doing the recording. Whatever is done, there almost bound to be claims that some “crucial” aspect of the setup was not right, hence (they will say) the test is invalid.

    2. Another issue, which is probably solvable, is that people may choose devious means to try and identify the mystery clip X. Using audio editors, and examining or subtracting different waveforms would be one way to get a reliable measure of whether two waveforms were idnetical or not. This isn;t what the test is supposed to do – we want to test if people can HEAR a difference. A small time offset, so the clips in the three samples are not identical, could help – but it wouldn’t be hard to get round that one either.

    I believe I have pretty good ears for musical detail (I’ve worked doing freelance guitar transcriptions, which isn’t easy to do well). I believe that many aspects of a Hi-Fi system cause audible differences in sound. I believe that many (most?) people can’t hear these differences, but a small number can. I also believe most of the stuff written about directional cables, amplifiers, CD players etc is rubbish – the kind of comments made about differences ins ound are just ridiculous.

    I’ve also done psychoacoustical tests for my PhD, looking into the tone quality of the guitar, and found a reasonable proportion of people tested thought that two identical waveforms sounded noticeable different. We probably shouldn’t be surprised.

    Howard

  165. Horace said,

    February 6, 2006 at 1:23 pm

    1. There is no amp, its the optical digital output from a hifi cd player going piped straight into a computer.

    2. Easily done, record four files, two with the expensive cable, two with the cheap cable, then they will not be identically identical files.

  166. Max Sang said,

    February 6, 2006 at 1:37 pm

    Re. Martin’s wonderful new word ‘cashebo’. Yesterday I bought a new watch to replace my rather flash 250 quid Tissot that had given up the ghost after only two years and three batteries. Bonus in hand, I went into a huge jeweller’s looking for something clean and elegant-looking with a leather strap. I didn’t care what it cost, as long as it was less than, say, 500 quid. A full hour later I stepped bewildered and dazed onto the pavement carrying what, to my eye, was the most beautiful watch in the shop. £14.99. And yet… I can’t help feeling that if it had cost ten times that amount I would be infused with cashebo and be proudly displaying it to everyone. Instead I glance furtively at it, wondering if it looks cheap. Bizarre, but a very strong feeling. Perhaps it’s my insecurity about aesthetic matters. Contrast that with my musical training and PhD in physics which allow me to laugh gently at hi-fi freaks and go back to my 400 quid Richer-sounds-knock-off stereo that was chosen in a non-blind comparison with something double the price. Subjective, yes, but free of cashebo.

  167. Alan Harrison said,

    February 6, 2006 at 2:16 pm

    TxB: You are not adding anything to the debate here. You seem to have come late to the discussion, clearly having not read Ben’s earlier articles on the subject and setting out to defend “Hi-Fi affectionadoes” at all costs. You have misrepresented Dr. Goldacre several times, made outrageous claims yourself, questioned utterly reasonable scientific practices, launched irrelevant diversions into the visual aesthetics of the equipment, invoked really dodgy and untenable analogies (Veyron = Mondeo ?!?!) and defended dishonesty in the Hi-Fi profession.

    In many ways you have made Ben’s point for him (especially with your last sentence abve), but mostly you have just been distracting, argumentative and rude.

  168. Alan Harrison said,

    February 6, 2006 at 2:21 pm

    On the topic:
    If we postulate there is external interference that may spoil our recording (and isn’t this the premise of the posh power lead?) then that may vary with time and with location. So your two recordings must be made simultaneously and close together. So you need two identical sets of equipment in the same room. But then you could argue that the equipment is not identical so you’ll have to swap the leads over and do it again.

    So we get 4 files, two from one lead and two from the other. And two chosen at random for the X samples.

  169. Andrew Waugh said,

    February 6, 2006 at 3:41 pm

    set up a double-blind listening test, but begin it with the equipment-under-test in full view of the guinea-pigs. Presumbly they will initially ‘hear’, or indeed hear, the difference brought on by a mains lead costing more than what many Africans earn in a year. Then keep on repeating the test, each time progressively reducing the lighting levels. Perhaps there will come a point, before total darkness, where the difference is longer discernible. Maybe this point could be called the ‘cashebo horizon’.

  170. Delster said,

    February 6, 2006 at 3:46 pm

    i normally avoid picking up on one person’s comments on a topic but i can’t resist i’m afraid.

    Post 13 TxB – “Hi-Fi is about reconstructing and maintaining an analog signal as close as possible to the original. There is nothing ‘unscientific’ about that effort”

    That would include the Quantum stream inducing crocodile clip and the wooden volume knob and the magic pebbles then??

    Post 20 – “No appreciable difference between the bugatti and mondeo”

    so no difference in acceleration, interior comforts (better stereo maybe?), soundproofing, vibration, handling etc etc then?

    Post 112 – “And that was the problem I have with Ben Goldacre … he didn’t check if the actual lead does the job.”

    The whole point is that BG is actually proposing that we do check it and not just swallow the manufacturers claims…. or possibly i got that wrong?

    Post 121 – Bugbear with 2 volume controls?? This is known as “Balance” and is often included on systems with exactly that lable. Very useful when speakers can’t be located symetrically.

    Post 123 BG implies cable does not work because he proposes a pilot study? actually what he’s saying is “lets check out the manufacturers claim in a scientific manner and see if it holds water” or to put it another way “peer review” (more or less) which happens to be one of the main scientific tests of a theory.

    got bored with TxB after those i’m afraid

    Tolstoy – post 143 – Pilot studies are not bad things – they are just not as good as full blown studies. They can however help on designing the full studies. your right on post 129 about test control being tricky with an internet test that is both subjective and all carried out on differing systems.

    sorry…. was longer than i intended!

  171. Chris L said,

    February 6, 2006 at 4:43 pm

    Re: my post #162 above, and the response from GWO in post #163. If it can be agreed that his response is accurate, then this whole experiment becomes meaningless. Is there such agreement?

    It certainly seems to be the case that the stored stream of samples on a CD will normally be recovered with 100% accuracy (ignoring pathological cases such as badly-scratched disks etc), otherwise software companies would struggle to distribute software via CD. It would certainly be an odd situation if a £15 CD drive were able to do this perfectly, but a high-end hi-fi CD transport needed a £1800 lead to manage the same task.

    Apologies for stating the bleeding obvious, but, you know, 170 posts and all!

    Actually, ignore the above – I’ve just thought of a brilliant new scam^H^H^H^H business venture. I could rent out £1800 computer power leads for a day at a time. “Don’t install that new software using a crappy power lead! Use the new Exact-o-Lead! Your new software will work 137% faster and its graphics will have greater toroidal contradimensional flanges!”

  172. TonyY said,

    February 6, 2006 at 5:04 pm

    Studios use bog standard mains leads to power their recording and processing equipment. Therefore any degradation caused by these leads will be on the recordings from the start. Part of the argument seems to be the better the gear, the more justification for an expensive mains cable, ie the more difference it makes. As most studio gear is pretty good, then the universal use of cheap mains leads in studios should have a significant detrimental effect, which accumulates as the signal passes through each piece of equipment in the signal chain.

    Then, when the listener plays the recording back, a super mains lead should provide a more faithful reproduction of all the errors accumulated in the recording process than a cheap lead could.

  173. Wiretrip said,

    February 6, 2006 at 6:06 pm

    OK – looks like I kind of missed the point – I understood the experiment in a more holistic sense – can people hear a difference? An that is why I mentioned perception. Tim Day, what I was getting at was that the claim was that the cable would improve sound quality – however, what does that actualy mean? Sound quality as generally understood is in the ear of the beholder…

    Anyway, being a computer scientist I would like to suggest that since there are soooo many variables the only way this can even be touched is using purely statistical methods. Create two files, one with and one without the cable. Next, place these files on a server that assigns each of these files a globally unique ID (GUID) each time they are downloaded. The participant downloads these files, telling the server who they are, listens to them and fills in an online form, telling the server which file they think is which. Only the server knows which file was the ‘filtered’ one and which was not. Then you have a randomised, double blind trial with virtually no effort, except in the construction of the files and the server app.. Given a large enough sample size simple stats will tell you which file people preferred. The GUID ensures blindness since no one can get the same filename twice, even if they participate twice. The server could even pad-out the files to make them slightly different sizes each time, though I think this would be unnecessary.

  174. Chris L said,

    February 6, 2006 at 6:33 pm

    Hello? Hello? Can anyone hear me? :-)

    The test protocol could be a lot easier than any of the above. Given my understanding of CD technology, there should be a strong expectation that recording the digital output of a CD player will reliably result in *exactly* the same data being recorded regardless of the mains cable used, the phase of the moon, whether Ben’s central heating is on or off, etc. A simple diff of the two files gives an instant end-point to the whole protocol – if they’re the same, that’s it – game over for the £1800 cable. Only if they’re different is there any point in proceeding with a blinded listening test to see if the binary differences are audible.

    At the very least, this step (of diffing the two files) should be included in the protocol. Depending on the equipment used, it may also be necessary to ‘align’ the two files by removing all leading and trailing zero samples, but I would argue that that’s worth doing too.

    C.

  175. Wiretrip said,

    February 6, 2006 at 6:58 pm

    I thought that the point of the cables was to improve the output of *analogue* equipment (or at least these stages)…

  176. Wiretrip said,

    February 6, 2006 at 7:01 pm

    …Furthermore, I would propose extending the server-based randomisation to include several different ‘file pairs’ including diferent music and speech styles and those consisting of identical files – to act (along with a ‘no discernable difference’ response option) as a control…

  177. Chris L said,

    February 6, 2006 at 7:20 pm

    Wiretrip – please check the original Guardian article and post #23 in this thread from Ben. The manufacturer specifically claims that using their special power lead on a CD player will have an effect (they call it an ‘improvement’, which is subjective, but irrelevant) on the optical digital outputs. That is specifically what Ben’s protocol is intended to test.

  178. Drew Stephenson said,

    February 6, 2006 at 8:01 pm

    It’s good to see that we’ve got back onto the subject, but i have a suspiscion that we have left an argument uncovered with the proposed experiment. All of the above assumes a digital environment. A lot of the seriously high-end hifi is still based on an entirely analogue reproduction. From the record to the speaker. It wouldn’t surprise me in the least to have the experiment rubbished by some who claim that the fact that it has been digitally sampled has already ruined the original signal irredeemably.
    I’d personally still like to give it a try just out of curiousty.
    On a related note, i used to work in richer sounds in holland where we had a standard set up running as a demo. One day i came in after lunch and set it going again and noticed straight away that it sounded different. A quick check around the back showed that my colleague had replaced the interconnects between the cd and amp. I know it’s not a completely blind trial – and that we’re talking about a different type of cabling – but we should be careful not to discount claims out of hand.
    Hence i would like to get on with the experiment as proposed and see what we get back

  179. Wiretrip said,

    February 6, 2006 at 9:12 pm

    Sorry, Chris L, you are right! I have been interrupted a lot whilst trying to follow this one…8 month-old child, lack of sleep etc… On this one, I heartily agree with you – I started out as an electronic engineer and I always thought that the point of digital technology (it was TTL in those days) was to avoid these issues! A diff would sort out that argument – wobbling the optical fibre would have more effect than mains-borne interference. I also agree with Drew though – an that kind of brings us back to the analogue part.. Although I think by now Ben has got bored and start writing next week’s article, probably about MRSA :-)

  180. Mike said,

    February 6, 2006 at 10:05 pm

    I was going to complain about the test (since it isn’t actually testing anything), but Roy Badani in post 76 already stated my main points. I don’t care if the company claims it makes a difference, because it simply can’t. To continue with the current project would be “Bad Science”. If mains noise is important (and I don’t think it is), it will only manifest post-digital, probably in the DAC. I don’t believe that even audiophiles (sane ones, anyway) will argue this point. In short, I don’t think you can do this test online. It needs to be done “in person”, in something similar to this:
    www.hometheaterhifi.com/volume_11_4/feature-article-blind-test-power-cords-12-2004.html

    As for jitter, it can be easily compensated for, though, as Roy says, it isn’t very common in consumer outboard DACs. The cheapest one that comes to mind is the Benchmark DAC1 ($1000), except that it’s proaudio, not “Hi-Fi”. However, even re-clocking DACs don’t do a particularly good job since, if you don’t want to collect an infinite (in principle) number of bits inside your DAC, it NEEDS to clock off the input signal (or be clever enough to throw away useless bits, which some do). Too bad … but if you have both the DAC and the transport in the same box (that is to say, if you have a CD player), they use the same oscillator to clock the signal, so one can control the total amount of jitter. Period. But I hate CD players, so I use an outboard (reclocking) DAC with my computer anyway.

    Note that if the transport clock is *slower* than 44.1k, you’re totally screwed and nothing will save you from clocking off the input.

  181. Mike said,

    February 6, 2006 at 10:14 pm

    ps: It doesn’t *have* to be in person, but it’s impossible to control for a lot of things you’d like to control for (the sound system, for example) if you just use digital files. Also, the recording aspect is … difficult, to say the least.

  182. John Thorpe said,

    February 6, 2006 at 10:22 pm

    Well the digital output from a CD player contains the data recorded on the disc but this is presented with a time signature determined by the player and its environment.
    You can record the data but you can’t record the time information – it is relevant only in the real-time reproduction of the sound. For this reason the experiment is a non-starter.

    John

  183. Pete Thompson said,

    February 6, 2006 at 10:28 pm

    Does anyone out there actually admit to having spent 1800 pounds on a mains lead?

    Has anyone noticed that the vast majority of audiophiles are male? I’m not insinuating anything by this – it is simply an observation that may have to be taken into account.

  184. Barry said,

    February 6, 2006 at 10:33 pm

    I’d like to maybe introduce a psychological perspective on this which might throw light on hi-fi buffs beliefs. The fact is that it is perfectly possible that a hi-fi buff listening to a stereo on the end of a £200 power cable might enjoy the music more because they know that they are using the £200 power cable – and I’m talking about enjoying it more in a very objectively measurable (dopamine levels/blood flow in the brain type measurable) way. The fact that there is no discernable difference in the sound does not mean that the belief in a better sound does not actually lead to greater enjoyment. In fact this would explain perfectly why the Soundbites editor does genuinely experience a difference when he knows what he’s listening to – his belief is causing a genuinely different experience.

    Of course, the sad conclusion from this is that if you give someone a £1 power cable and convince them it is a £500 power cable their enjoyment will be equally enhanced. However, in our consumer driven society are there any of us who never fall for this effect?

  185. Andy said,

    February 6, 2006 at 10:39 pm

    Chris RE: post #162,

    In theory the answers should be Yes, Yes and Yes however in practice I doubt that this would be the case.
    A CD will have errors on it at some rate or other, no physical storage media is perfect. The error correction should be able to fix most of these and then the CD player will then apply some basic filter to try and fill in any remaining gaps. This isn’t perfect but it’s good enough that it’s unlikely anyone will notice if it guesses slightly wrong. Audio CDs contain fairly basic error detection / correction because small errors won’t be noticed by the human ear. A data CD on the other hand will use 10% of the space on the CD for error correction information; a computer program with a small error in it is not much use.

    Some of these errors will be errors in the production of the disc when it was mastered, CD makers know that up to a certain level of errors will not impact perceived performance and so don’t try to produce discs that are perfect every time. Some errors will be read errors where the disc is correct but for one reason or another the CD player just read the wrong data. These read errors could in part be caused by mastering issues where the disc is technically correct but is somewhat borderline e.g. the pit is a little shallower than the nominal depth or a little narrower but still within the standard. Dust, fingerprints or scratches on the disc could also cause a read error.
    The mastering errors will be consistent but the read errors will be different each time, this difference will be greater using two different drives than if the same drive was used twice.

    The next source of differences will be the way in which the CD drive fills in any gaps. This could well vary between drives, I’m not sure how tight the spec is on this and how much is up to the manufacturer.

    Finally when burning the file on to a CD-R there will be a new set of mastering errors (mainly defects in the CD-R being used) and read errors.
    If the files were burnt onto the disc as computer files (e.g. as a .wav) then the resulting file as read from the CD-R will either be identical to the data as ripped from the CD or will be reported as unreadable, that 10% of space used for error correct will make sure of that. If the CD-R is burnt as an audio disc then ripping it will probably result in different data.

    That said you could probably repeat the rip, burn, rip, burn cycle a large number of times before any errors became noticeable to even the best tuned ears.

    As for actual numbers on the error rates, I have no idea. A quick google gives a raw error rate of 1 in 100k-1million and about 1 in 10 billion+ (or just under 1 per disc) after correction.

    In the tests Ben has proposed the read errors will be different between the A B and X files but should have a negligible impact on the audio output. After that the data will be in the digital domain with sufficient controls and error checking to ensure that no further errors enter into the system.

  186. John Thorpe said,

    February 6, 2006 at 10:53 pm

    Andy (184) : As someone who has been designing digital audio equipment for the past 20 years I can say (based on measurements) that the error rate at the output of a CD player is almost zero. That does not mean that the disc read is error free, but usually all read errors are correctable evenn with the restricted correction facilities in the CD-DA format.
    This does not mean that the disc error rate does not affect the sound since the error correction can influence timing and digital noise (see my previous post).
    Paradoxically, errors which are corrected by the CD player (by interpolation) seem to have very little effect on the reporduced sound wheras small timing changes in the digitial data can heve very obvious sonic effects (despite causing no data corruption).

    John

  187. Michael Harman said,

    February 6, 2006 at 11:08 pm

    Horace, thanks for the comment on how peer review works. My impression (and it’s only an impression) is that peer review happened mainly when a paper was submitted to a journal. That seems to imply that the experiments have already been done. How far does peer review tend to result in supplementary experiments? Is formal peer review often done before any experiments have been done and before there are any results?

    If the present process is significantly different from standard peer review, perhaps it should be used more often?

  188. Andy said,

    February 6, 2006 at 11:46 pm

    An error rate of “almost zero” sounds a lot like 1 in 10 billion or so to me ;-)

    Either way I think we can agree that the disc read error rate is significantly small that is should not have a meaningful impact on the proposed test.

    As you point out, any clock error on the playback device wil have a large impact on the audio reproduction however I think that for this test that too can be discounted.
    This is an ABX test where people are not being asked which is better, A or B, they are being asked is X the same as A or the same as B. Assuming that the clock errors are consistent then a clock error will not have any impact on which of A and B the file X is closest to.
    Maybe it should be a requirment that any equipment people use for this test should be warmed out for 10 minutes before playing the files in order to minimise and self heating thermal effects on the clock.

    Just out of interest do you happen to know if any high end Hi-Fi DACs that make use of TCXOs or OCXOs for their clock sources?

  189. John Thorpe said,

    February 7, 2006 at 12:22 am

    Andy : When I talked about a “time signature” I was trying to imply something more complicated than clock rate changes. The exact frequency of the clock is probably not too important although short-term cahnges in frequency can certainly be. Many CD electronics suffer from clock “pulling” as servos and motors draw varying currents from power supplies. Plenty of “high-end” CD manufacturers have realised this and provide separate supplies for clock circuits.
    The separate DAC boxes have either to phase-lock thier clocks to the CD transport feeding them or perform sample-rate correction to their own standard frequency. The latter option modifies the CD data of course.
    Most cheap DACs use only the clock provided by the SPDIF data receiver chip – a basic RC oscillator which is quite noisy and follows the CD clock very closely. More expensive DACs often have a second phase-lock system involving a crystal oscillator (VCXO) which gives a much cleaner clock signal. Short-term differences between the input and DAC clocks are absorbed by FIFO buffers in some models.

    I agree that the ABX test is definitely the best way to show up differencies (or otherwise) in system components. Ideally the A or B or X should be selected by the listener, as often as (s)he requires. Making the test truly double-blind is rather difficult and it is very easy to accidentally give clues to listeners as to the arrangement of equipment.
    The problem with the CD data output trial is that the system must be LISTENED to with the different mains leads because it is not posible to record the time changes in the data stream. Just recording the data bits wil show no change.

    John

  190. Andy said,

    February 7, 2006 at 1:07 am

    John,
    Correct me if I’m wrong, I just want to verify that I have understood your argument correctly.
    You are saying that by using a different mains lead we could introduce noise / voltage differences which could induce either jitter or frequency errors or both on the CD transports clock. Since some DACs use the transport clock for their conversion to analog this will introduce distortions in the reproduced audio. This error will be lost when the data is recorded to a PC file which will effectively re-sync the data to the clock of whatever device is then used to play the file back.
    As you point out, the best method to remove this noise during normal playback would be to buffer the data from the transport and then use an independent, high accuracy clock for the DAC timing effectively performing the same re-syncing we will see with the planed experiment only with slightly less isolation from any noise introduced via the cables between the two.

    Is this correct?

    Personally I wouldn’t expect this to be a significant effect but that is just a gut call based on my own experience and the expectation that someone charging that much money for a transport is going to spend more than 15p on their clock source. I have no evidence to this effect and so we must assume that it is a valid source of error for this experiment.
    Short of setting up an oscilloscope on the data coming from the CD transport and measuring the bit clock directly I can’t think of any simple way of measuring this error. Of course doing this would have an impact on the data but with a sufficiently good scope & probe the effect should be minimal.
    Can you think of any method to at least measure this effect? If you feel this is a sufficient source of error to invalidate the experiment how should we change it (within the realms of practicality) to either remove the error or at least quantify the impact it could have on the results.

    Also if I have understood both your statement, that at least some DACs perform this buffering and re-clocking, and the cable makers claim, that their cable improves the quality on all Hi-Fi systems, then doesn’t that mean that the re-buffering which is performed by this experiment is no different to that performed by some Hi-Fi equipment and so should still show the same improvement that the manufacture claims we should see.
    Remember the point of this experiment is not to prove that this cable can never improve the audio quality on any system. The point is to prove or disprove the manufacturers claim that it will improve the audio quality on all systems.

  191. amoebic vodka said,

    February 7, 2006 at 1:09 am

    post #187 – in theory, any scientist should be able to design a good experiment and if they don’t know how for the thing they are testing then they know to ask someone else.

    When a paper is submitted for peer review, the referees look at it and decide essentially if the experimental design is good enough for the authors’ results to be valid. If not they either reject the paper, ask for further experiments to be done or for clarification of the methods used if it’s not clear. As getting your paper rejected *after* the work is done means no publication, probable loss of funding etc, then it is a good idea to design experiments properly in the first place.

  192. John Thorpe said,

    February 7, 2006 at 1:51 am

    Andy : The following comments are based on my observations and experiments from the past 20 years designing. I always try to find a logical explanation for effects that I percieve or measure but I am not always able to do this. By observation of products and ideas that others produce it seems to me that no-one else really understands much about the sonic effects and side effects of changes in digital audio systems either. It is, however, evident from published information and designers’ attempts at better equipment that many others hear differences.

    To take your points in order :
    A different mains lead could very likely produce different timing (or jitter if you prefer) in the digial output. It is very unlikely it will produce different data bits.
    These differences will not be recorded on a PC disc.
    The timing differences will affect the clock recovery in some DACs but there are many possible paths within a DAC through which input timing can influence the audio output. The clock is only one of these possible paths.
    Adding buffering (re-clocking) in a DAC may eliminate what I call 1st order timing errors (by re-synchronising the data to a clean clock within the DAC – which can be produced by a narrow-band PLL and crystal oscillator) but other effects still pass timing errors through to the output. These can be, but are not limited to, coupling through ground planes, supply lines and magnetic coupling between circuit areas.
    The “I don’t think that effect would be significant” is a constant bugbear. I have said it many times and proved myself wrong many times. I never say it at all now.

    Stupidly the effect of cleaner clocks in DACs seems to make CD transport timing signatures more audible. It is as though removing gross timing errors focusses the audible effects into certain regions of the sound and makes them more noticable. With a largely unsteady clock the whole effect is blurred and changes are more subtle.
    I have been trying to produce what I waould call a “perfect DAC” – not one that necessarily produces the best sound (that is subjective) but one that is insensetive to CD transport timing variations – ie it would sound the same with any transport. I have to conclude that after 6 commercial designs I am no nearer to achieving this goal. I almost believe it is not possible, but I would dealrly love to have some test equipment that would poduce any results that correspond with listening tests.

    I have produces several timing analysers that will show differences in CD transport outputs (a scope is not viable – timing variations are in the order of a few picoseconds) but the measurements are mostly influenced by large timing variations whereas the sound is influenced by minute timing variations – hence little correlation between measurements and listening tests.

    The likely outcome of the cable tests (not a scientific answer but a pragmatic one based on many years of listening) ?
    1) The cable swop will make a change on most systems.
    2) The change will be more noticable on “better” systems (in my parlance I would say more “transparent” systems that imprint less of their own character on the sound).
    3) “Improved” or “Better” sound are subjective terms and will depend on who is listening and what they are listening to. I do not agree with the cable sellers’ comments that their cable will improve the sound, but I accept that it will change the sound.

    Last post for tonight, John

  193. Sockatume said,

    February 7, 2006 at 9:31 am

    So is there any experimental evidence that clock errors etc. do/do not have an audible, or for that matter measurable (by directly comparing waveforms of the output) effect on the audio produced by a device? We’ve established that theoretically they’re plausable, but it might be worth establishing what electronic effects exist and what don’t before letting them decide an experimental protocol. I’m not talking about determining an explaination for a percieved change after the change is noticed, I’m meaning testing that explaination in a controlled manner.

    Out of curiousity, how do the interconnect qualities required for audiophile gear compare to those required for a FTICR mass spec, a machine for which a quality of output waveform is essential? For that matter, how do the theoretical variations caused by various interconnects etc. (surely predictable in terms of magnitude) compare with the theoretical variations caused by having a second person in a room or bringing in a mug of tea etc.?

  194. Chris L said,

    February 7, 2006 at 10:37 am

    Hello, me again. All these arguments about how noisy circuits might affect the output of the DAC etc are irrelevant. Ben’s proposed experiment is specifically testing the manufacturer’s claim that one of these leads will affect the optical digital output of a CD player. See www.russandrews.com for more like this:

    “Fit a YellO Power cable to your CD or DVD player … [y]ou’ll hear a fuller, more musical sound with less distortion”

    Ben calls bullshit, but in deference to the scientific method proposes an experiment to confirm the presence of bovine waste. The fact that we all (with the exception of TxB) agree that there is extremely unlikely to be any change in the digital output doesn’t mean that the experiment isn’t worth doing – calling ‘bullshit’ without evidence is no better that calling ‘fuller and more musical’ without evidence.

  195. Humanities graduate said,

    February 7, 2006 at 11:11 am

    While as a humanities graduate I clearly know nothing whatsoever about science, I will take the liberty of pointing out that “thougt” is not how the word is usually spelt.

  196. Ben Goldacre said,

    February 7, 2006 at 11:19 am

    ok, i think we’re honing in on the idea that the pilot study tests an identifiable hypothesis, and that we could construct a method that would be satisfactory, working practically around the limitations of our resources, and might generate new knowledge.

    but amongst the idle banter, as ever, we also need to think about basic practicalities.

    firstly, where the music comes from: there’s no money for equipment here, only time and expertise. plus ça change. so does anyone know what the copyright rules are on using 20 second snippets of somebody’s music? it would be useful if somebody knew whether we can use some professionally recorded music released under a creative commons license, maybe something off archive.org, although ideally it would be nice if we could use some mid 70s pink floyd, because that kind of feels like the kind of music a hifi buff would use to test their new equipment. anybody know the law? anybody best mates with a pop star or a record label and can get us the rights to use a few bits of commercial music?

    secondly, the voting. i can stick up some files on a page to download, with two links to vote for X as being either the expensive or the cheap power cable with an email, and count the votes, and check up on random voters to see that they’re genuine, but elaborate page designs, random appearances of files in different combinations, complex voting regimes, all that… if you suggest it, that’s great, but i guess you have to be able to program it too, and that might make you decide it’s more trouble than it’s worth.

    so, most crucially, since we have to crack on, the music: what can we use, and where does it come from?

  197. amoebic vodka said,

    February 7, 2006 at 11:36 am

    If you’re wanting to check for people voting more than once etc, then how about setting up a blog entry where everyone’s comments get sent for moderation? Then we can vote there, without anyone knowing what anyone else picked and you get the voters’ email addresses. You can check for duplicate IP addresses as a simple (though easily circumvented) way of looking for genuine votes and pick a sample of the submitted email addresses to see if they are real.

  198. CB said,

    February 7, 2006 at 11:45 am

    Do you only want votes from people who reckon they can hear a difference? ie if you listen to both A and B and realise you can’ t tell one from the other and so just take a guess, is that a valid vote? If not then you might not get many votes!

  199. Ben Goldacre said,

    February 7, 2006 at 11:46 am

    nope, definitely easier for me to make a page with two links for vote A or B that send an email to vote@badscience.net with A or B in the subject line, duplicates weeed out immediately and count immediate.

    now, where does the music come from?

  200. Ben Goldacre said,

    February 7, 2006 at 11:47 am

    cb: interesting, so we want “what do you think x is, a, b, or can’t tell?” that’s probably a good idea.

    actually, i suppose, putting my studio nerd hat on, i could make a few ABX trios where there definitely was a difference between the two, boost the eq at 2kHz by 4db, the kind of thing that a golden ears hifi buff should be able to hear, to catch out people who just aren’t listening very carefully.

  201. CB said,

    February 7, 2006 at 11:52 am

    Then but if you have a ‘can’t tell’ option, its very easy to ‘cheat’ by just putting that without listening – especially as its the answer that people may percieve is wanted! at least with just A or B people don’t know which side of the debate they’d be loading. swings and roundabouts as ever. Could have a ‘level of certainty’ from 1 to 5??

  202. HowardW said,

    February 7, 2006 at 12:28 pm

    Don’t know if this helps much, but I seem to remember the copyright laws make an exception where small excerpts of a work can be reproduced for “research or sholarship purposes”. Don’t know the exact wording, but it seems to me a 20-second clip of any recorded work used for the purposes of this test might well fall into this “research category”. Needs backing up, but it might be a way forward.

    But – what’s to stop people doing a diff on the wav files, rather than using their ears, to answer the question? Clearly, you need to use the same audio snippet for the A, B and X. If people were that way inclined, they could just diff the files and they would be guaranteed the correct answer every time. They might need to correct for a small time offset, but that’s not very hard. Isn’t this a problem?

    Would this be of any use: www.pcabx.com/ ?

    I’ve played with it before – seems like a nicely made tool to automate such ABX tests. It forces people to listen from their PC (which people may not like), but it may get round the risk of people opening or subtracting the wav files to “cheat”.

    Howard

  203. John Thorpe said,

    February 7, 2006 at 12:38 pm

    The way that this experiment is shaping up it will test the hypothesis that “changing the mains lead will alter the data coming out of the digital output of a CD player”.
    Unfortunately this is not testing the claim made by the lead sellers that “changing the maind lead will improve the SOUND of the system, even through the (optical) digital output.”
    Read my previous posts (189,192) for my thoughts on this.
    BTW it is not necessary to even listen to these different data ercordings, just use WINDIFF (buried in Win2000 and maybe XP) to compare the files in the digital domain.
    For those who refuse to give credance to anything that is not published in a “scientific” journal, I have, somewhere, a paper written by engineers at DCS (a well repected manufacturer of digital equipment – supplies to both HiFi enthusiasts and recording studios) detailing sonic differences in data identical CDs. When (if) I can find it I will post a reference.

    John

  204. Ben Goldacre said,

    February 7, 2006 at 12:54 pm

    John Thorpe

    “The way that this experiment is shaping up it will test the hypothesis that “changing the mains lead will alter the data coming out of the digital output of a CD player”.
    Unfortunately this is not testing the claim made by the lead sellers”

    Holy fucking fuck for the five millionth fucking time that is precisely the claim made by the company. Read. Read what you are posting on. It’s in the original article, at the top of the page, and in the paper, and it’s come up about four times in the discussion. What is wrong with you?

  205. Myts said,

    February 7, 2006 at 1:08 pm

    I think the A/B/don’t know vote is a great idea, anything more complicated will be too hard to organise and analyse, and if you force people who can’t hear a difference to nominate A or B you get more noise in the data. Definitely go A?B?Neither

    John Thorpe and ilk, if you can’t keep up get out of the way.

  206. Jim Lorimer said,

    February 7, 2006 at 1:36 pm

    Ben,

    We definitely need a control and I like your idea of having a few trios – perhaps one with A, B & X identical, one with A & B demonstrably different and one with the A & B using different power cables. I think we need three voting options too – I don’t know/ can’t tell, being one of my favourite answers to many questions. Maybe you need to find out which voters are genuine audiophiles too – could you ask some esoteric question the answer only known to real hi-fi enthusiasts?
    I’m fascinated by the response to this particular story and I think it all comes down to belief; if you’ve learned something from someone you admire and respect you’re going to see no reason to question it. For example, I was quite happy to believe that ” the water in a sink (or toilet) rotates one way as it drains in the northern hemisphere and the other way in the southern hemisphere” – a teacher told me. I’ve been disabused of that now.
    I’ve always been inclined to believe that there was something to be said for expensive hi-fi separates but that I probably don’t have a sufficiently discriminating sense of hearing or the right taste in music, to make it worth my while investing too much money. However, having read most of the comments on your article I’ll definitely have no hesitation in sticking to the bog standard audio products from now on.
    Looking at comment #028 – I still think I can tell the difference between a single malt and a blend, but I bet a gas chromatograph could demonstrate that there is a real difference.
    I can’t wait for the power cord test.

    Jim.

  207. John Thorpe said,

    February 7, 2006 at 1:41 pm

    Ben : I assume that this is the paragraph you refer to;
    “The manufacturer of my expensive power cable has assured me that it will have an impact on the sound of an expensive CD player, or a cheap one, and that it will affect optical outputs as much as normal phono outputs.”

    Obviously I don;t read this the same way that you do.
    It’s all about sound not about data.

    John

  208. Ian said,

    February 7, 2006 at 1:44 pm

    I think we should eject John Thorpe and co… at this rate Ben is going to have real problems keeping to the Hippocratic Oath.

    On the A/B or A/B/can’t tell debate, I was going to agree with CB (post 201), until I realised that if people *wanted* to skew the results with an either/or choice, they’d just vote twice, once for each. I imagine most people have two email addresses these days.

    I suspect that this debate won’t change the minds of the dedicated audiophiles, in the same way as the homeopathy meta didn’t convince any of the CAM adherants. With many it seems to be more about faith than reason.

    I wish there was a different word for ‘believe’ that made it clear you were talking about a reasoned opinion, backed up by evidence, as opposed to faith. Is there, in other languages? (‘I believe in evolution’ is different to ‘I believe in homeopathy/£1800 power cables/the tooth fairy’.)

  209. Delster said,

    February 7, 2006 at 1:52 pm

    umm…. John… hate to point this out and all that but it’s the data that the hardware uses to generate the sound from… so change the data and you change the sound…..

    So if you say it changes the sound then that has to be caused by a change in the output data…

    unless of course what the lead does is some how magically upgrade your speakers from the sashio pair currently attached into a pair of Lynn index ones…. which, trust me, would be a worthwhile upgrade to make.

  210. Delster said,

    February 7, 2006 at 1:56 pm

    Ian,

    i’d go for conviction. thats the only other word i can think of instead of belief

  211. WireTrip said,

    February 7, 2006 at 1:57 pm

    I seem to recall that the British Library has an extensive collection of Jazz recordings and are able to give permission for re-use for educational purposes at their discretion.

    www.bl.uk/collections/sound-archive/nsacollections.html

    Failing this, copyright-cleared sample CDs are the way to go since these are…er…copyright-cleared and are CDs :-) Many musicians will have these and they usually contain fairly simple samples of things like bass lines and drum loops – which would provide nice clean and pure ‘audio landscapes’ suitable for this type of test… Some also contain more comples pieces of audio.

    Just a note though – why do we need the ‘X’ part of the comparison? Surely simple pairs of sounds (see my posts173 and 176) will do. I may be being stupid here, but if the point is to see if a difference is made then why do you want to tell people what types of file they’ve got? Simpler just to give them packages of pairs of files, some identical, some differently cabled. If a large number of people state a preference for one type of file (filtered by the identical files controls) then surely the statistics will answer this one – given a large enough sample size? Also, using the GUID solution you wouldn’t need to check IP addresses etc since the voters would only be able to vote for each file (or pair) once anyway… The controls would filter out peole repeatedly downloading and voting. I am talking emergent behaviours here given the huge number of potential variables. Think of the public as one huge neural net :-) Oh, and Barry (post 184), see 149 and 173.

    Ben, I think this is becoming bollocks now as you’ve attracted some bearded weirdos…Can we have something totally different next week? How about ‘Do mobile phones really interfere with hospital equipment or are we forbidden from using them because trusts like to charge patients international rates for phone calls?’…

  212. chris cottee said,

    February 7, 2006 at 2:01 pm

    Hi,
    just fancied joining in the shouting match. Is n’t the test flawed by the subject’s listening to the recordings via their own dodgy sound reproduction equipment ? If that introduces extra noise due to the (no doubt) foolishly cheap power cables they use would n’t that conceal any difference that did actually exist in the super ones you use?

    Reducio ad absurdum: I play all recordings through broken speakers and am unable to detect the difference, which proves nothing. Is n’t this experiment prone to giving false negatives (at least in theory) ?

    Also won’t you need to ensure the mains supply is noisy when you do your recordings otherwise again the experiment would prove little. Or you could just make a lot of recordings at different times of the day: if the mains supply happened never to be noisy when you made the recordings that would be evidence in itself that the cables aren’t worth buying.

    BTW if this is covered in the previous 205 comments feel free to flame.

  213. Ben Goldacre said,

    February 7, 2006 at 2:03 pm

    Wiretrip: will you write the page for the GUID voting?

    The BL jazz archive stuff won’t be hifi enough, can someone provide a link to a sufficiently hifi piece of music that we will definitely be able to use?

    chris cottee: if you can’t tell the difference on your broken speakers you vote for “i cant tell the difference”.

  214. Delster said,

    February 7, 2006 at 2:12 pm

    Chris Cottee post 212

    The test should be ok despite the fact that each person will be listening to it on their own equipment as we’re testing the manufacturers claim that with their lead the original system will produce a better sound.

    Hence the output file that BG produces should, in theory, be of better quality which should be apparent on whatever system it’s played on (provided you play both or all three on the same system with the same EQ/volume/balance settings etc)

  215. Tibs Tedeschi said,

    February 7, 2006 at 2:19 pm

    Your surname should be Quixote. As a great fan of your site, I can only admire your perseverance in casting some light on the facts through the twilight of so many myths and creeds.
    Sadly, I think you greatly underestimate the ability of the human mind to fool itself, be it in the area of religion, alternative medicines, Hi-Fi, gambling, etc.
    I believe 184. Barry is closer than all of us in his appraisal of this “weird Science” when he puts a psychological spin on the subject.
    I thought I would tell you of a first hand experience of this phenomenon.
    I was once asked at a party what beer I drank, to which I replied that it did not matter as long as it was cold. (I know I am being an utter philistine, but since I live in balmy South Africa, your British readers might forgive me!)
    To continue…after some scoffing at my plebian tastes, each explained the virtues of their preferred brand and how they could identify their beer anywhere, as all other beer had the colour and taste of bat’s piss, or something to that effect.
    At this point the devout skeptic in me asked for a blind test to verify their position.
    Six members were selected by their peers. In front of each were placed six glasses, with six numbered Coke bottles which had been rinsed out with bottled water and filled out of sight by a trusted person with six different, readily available brands of beer. (Now before our more religious readers scramble for cover, the predominance of sixes had more to do with mere practicality than Armageddon!)
    The beer brands and corresponding numbers were noted on a secret paper held by the trustee. I may add that the colour of the beer was also clearly visible!
    After all the participants had noted the bottle number next to the brand they thought they were drinking, the papers were gathered and tallied.
    I know you can’t stand the suspense any longer, so here is the earth shattering finding… No one identified his beer with an acceptable degree of certainty!
    But the interesting part was that at the pub afterwards, no one drank any other beer than his favourite and still insisted that (a) It is the best beer, and (b) there was something fishy with the test!
    This is the “don’t confuse me with facts” creed and has as millions of devotees whose faith has the power to dislodge all logic and cling to false perceptions.
    From: Tibs

  216. David Smith said,

    February 7, 2006 at 2:20 pm

    Ben – Copyright info for you (caveat emptor though OK!)

    [swiped from a page on the internaughty]
    “Fair dealing (or Fair Use in the USA) is probably the most widely used and important defence in relation to using copyrights. It allows for the exploitation of a copyright in certain circumstances without the need for licences or the need to pay royalties. The limits to fair dealing state that only works copied for the following purposes are permitted, but with certain provisos:

    * Research or Private Study, provided that only one copy is reproduced but no acknowledgements are required.
    * Criticism or Review, multiple copies are permitted but sufficient acknowledgement is necessary. Or,
    * Reporting Current Events, multiple copies are permitted, and sufficient acknowledgement is needed, except in the case of reporting done via a sound recording, film, broadcast or cable programme.

    In all cases the copying must be ‘Fair’, and to assess fairness the amount or proportion of the work copied will be important as will be whether the usage competes with the copyright owner.”

    As you might expect – copyright is struggling to deal with the digital age (you are only making one copy – but there will be a lot more by the end of the experiment… so this is an interesting situation)
    I would venture to suggest the following approach:

    1) Pick something from a smaller independent label
    2) Go ring up their permissions department and explain what you are proposing to do
    3) Assuming they are OK with it, you probably want to make a statement along the lines of “by clicking on this link to download the test file for this experiment, you hereby swear on your granny/mother/cat/dog’s life that you will
    a) only use the file for the purposes of this test
    b) delete the file immediately upon the submission of your responses or immediately upon the ending of the experiment (Ben put an end date for responses)
    c) not disseminate the file any further by any means at all under any circumstances

    Hope this helps!

    David

  217. amoebic vodka said,

    February 7, 2006 at 5:00 pm

    Um…the US has fair use laws, but we thought the UK didn’t for CDs and the like.

  218. TonyY said,

    February 7, 2006 at 5:04 pm

    We all need a break. Why not plug both leads into a kettle and see which makes the best cup of tea

  219. Andy said,

    February 7, 2006 at 6:07 pm

    Delster & Co.
    Re: 209 and previous posts…

    I’m going to play devils advocate here (not to imply that John is in fact the devil in disguise)…
    As I understand it Johns argument (see 189,190 & 192) is that jitter and timing errors in the range of a couple of picoseconds in the digital data will impact the performance of the DAC since it uses the clock recovered from the data. Also noise on the cable injected onto the ground planes of the DAC could theoretically impact the final sound reproduction. By dumping the digital data to a disc that noise is lost.
    In other words in order to impact the sound reproduced by the system the mains lead does not need to make any change to the digital data coming from the CD player, it only has to improve the accuracy of it’s clock and/or reduce the noise on the interconnecting cable.
    This experiment is indeed measuring differences in the digital data and not the final sound reproduced and so _if_ Johns claim that this timing has an impact is correct then he is also correct in claiming that this experiment is not testing the manufacturers claim.

    Personally I can’t see the effects John is talking about having any meaningful impact but I’ve no direct evidence to back this up and so can not on a scientific basis dismiss it as a negligible effect. In order to be meaningful an experiment must control or measure any variable or prove that it will have no impact on the final result. The planned experiment does not do that in terms of clock jitter on the data coming from the CD player.

    The only way around this that I can think of is to say that the cable makers claim is also applicable when the CD transport is connected via fibre optics rather than wire to a DAC which re-clocks its data in a fifo. Under that situation I think we would have fully eliminated all of the variables that John was talking about. The question is whether the manufacturers rather wide reaching claim can fairly be applied to this rather specific setup. True they did claim it had an impact on all systems but would you expect them to say “it improves all systems apart from when using a model nnn DAC with optical connections to the CD transport”?

  220. Drew Stephenson said,

    February 7, 2006 at 6:23 pm

    Just a thought on the source-music front, NAIM (they of the high end hifi) also have a record label side and (being as they’re quite happy to give away free cds with magazines) may be willing to donate 30 seconds of hifi-buff level recording?

  221. Chris L said,

    February 7, 2006 at 6:50 pm

    Slightly off-topic, but for an insight in the cable manufacturer’s thoughts on the whole concept of blinded testing, see russandrews.blogspot.com.

    I find myself in the position of thinking it’s ok for purchasers of £1800 power leads to delude themselves, so long as they know they’re deluding themselves. It’s the fact that they’re convinced that they cannot possibly be deluding themselves that winds me up. Odd.

  222. Mike said,

    February 7, 2006 at 6:55 pm

    Holy fucking fuck for the five millionth fucking time that is precisely the claim made by the company. Read. Read what you are posting on. It’s in the original article, at the top of the page, and in the paper, and it’s come up about four times in the discussion. What is wrong with you?

    I just want to point out that, if they have, in fact, told you this, then they aren’t really worthy opponents. You don’t have to do the test. You can tell other people that you did it, and you found out there wasn’t any difference, because there isn’t going to be one. Even (sane) audiophiles will acknowledge this fact. You’re putting up the straw-man, setting him up just so you can knock him down, which is precisely what I thought you didn’t like about “Bad Science”?

    Look, just because they charge big bucks doesn’t make them honest-to-goodness-audiophile-grade-bullshit. The easily debunkable isn’t particularly fun or useful, and furthermore it’s already been done. I don’t like to waste my time and money on worthless experiments. Do you?

    Well, if you insist on continuing, there are sample files on the ogg vorbis and lame websites that are free for use. They tend to be somewhat complicated (since they’re used to test the compression algorithms), but they’re taken from real music that people listen to (that’s how they were found). You can put them online, but you don’t need to come up with some fancy interface for an ABX test, since (as has been mentioned multiple times) a simple diff is sufficient. I doubt they will be even 1 bit off if you record the signal properly.

    Lastly, don’t talk about “golden ears”, since no one has been able to show that “practice” helps in any way (and people have tried). It’s one of those myths …
    www.theaudiocritic.com/downloads/article_1.pdf

  223. Ben Goldacre said,

    February 7, 2006 at 7:07 pm

    am i alone in finding it weird that aside from the people planning the experiment, there are also all these people who keep coming in to tell us we shouldnt bother because they already know the answer, for definite, except they all disagree with each other…

  224. David Smith said,

    February 7, 2006 at 7:08 pm

    Re comment 217 (eek!)

    Welcome to the world of copyright legislation. You are kinda sorta right mebbee, but there are some vaguely worded provisions that cover things like academic usage etc etc. Now if there are any lawyers out there about to jump on me for what I have just written, I have this to say “where the hell were you when Ben needed some info”. Basically, If someone got iffy about this process then I think there would be a legitimate defense based on the text I supplied (which to the best of my knowledge is based on the UK copyright act of 1988 (I think)) Having said that, it is axiomatic that you never find a poor copyright lawyer… and note that what is being proposed might or might not fit within the provisions that I described earlier.

    Now I work for a smallish STM publisher and so I deal with permissions and the like. Applying our philosophy to the music business (which is dangerous I know) I figure the best chance of getting a friendly permissions person is to go with the smaller labels – the reason being that the permissions bod probably wears a great many coats if you catch my drift. go to one of the big people and you wont get an answer (you want) any time soon.

    As soon as the permissions person says “yeah Ok we are coll with that” the whole copyright problem goes away.

    Re comment 220: Brill idea unless Naim might not want to be associated with the possible debunking of the million dollar kettle lead…

  225. Wiretrip said,

    February 7, 2006 at 7:15 pm

    Hi Ben,
    Have been thinking about this and rather than go to the effort of writing servlets etc, hosting them, making sure they don’t fall over etc, why don’t we go for an even simpler idea. You don’t really need to have separate files for the samples.
    Simply create four files, each one containing all combinations of two samples.
    i.e.
    File 1: ‘Special cable’, then ‘Normal Cable’
    File 2: ‘Special cable’ then ‘Special Cable’
    File 3: ‘Normal Cable’ then ‘Special ….etc…

    You can even do this by simply running Windows recorder (or whatever) and ‘hot swapping’ the power lead (or pretending to) during the recording process.

    Make sure that all of these files are the same size (you can do this by presetting the recording length – and not using any compression i.e. ADPCM).

    I could write a piece of software (that you run) that renames the files, giving them random names based upon your computer’s hardware ID and a password that you supply. The resultant mappings are placed in an encrypted files that requires both your password and a password that I build into the software. That way, neither of us knows which file is which until these passwords are exchanged…

    Then, you can simply place these four files on your site and get people to download one or more of them (there would be no point in downloading the files repeatedly or disproportionately). Then people simply write in stating which file(s) they listened to and which sample they thought was the ‘special cable’, along the lines of

    file abd361.wav first, second, neither or both…

    Then we can go through the responses later, totalling them up before revealing (to ourselves also) which file is which…

    Tada, simple…

    I guess the renaming software could be verified by a trusted third party if necessary..

  226. Wiretrip said,

    February 7, 2006 at 7:17 pm

    …As for the music sources, I still think that a sample CD would be the best bet. I’m sure you must know someone who writes music electronically and will have such a thing… The point is, to be fair, you should use a commercially pressed CD , not something that has been downloaded and burnt…

  227. Briantist said,

    February 7, 2006 at 7:30 pm

    I have a “audio test” track that I own the copyright to and have used on occasion before, as it has a wide variety of sounds in it. It’s here, if it’s suitable: bnb.bpweb.net/demosound/01-%20tad%20-%20tad%20(tad).mp3

  228. Andy said,

    February 7, 2006 at 7:35 pm

    A repeat of Briantist’s link:
    Link
    Hopefully that will work better. This is why spaces in filenames are generally a bad idea.

  229. Briantist said,

    February 7, 2006 at 7:38 pm

    OK, so you can’t have brackets in URLs… The sample file is here: bnb.bpweb.net/demosound/01-tacticalaudiodesign.mp3

  230. Wiretrip said,

    February 7, 2006 at 7:38 pm

    I wouldn’t recommend using an mp3, it is possibly the most buggered- around-with format there is…Even to the point where pitch is not properly preserved!

  231. katie said,

    February 7, 2006 at 8:11 pm

    As much as I like adding the scientific method wherever possible, frankly I’d rather spend my time trying to debunk homeopaths. They’re more likely to actually kill somebody (with not treating someone with a serious disease) than any snake oil audio equipment. I had a friend who went to a chiropractor for a year before she found out she actually had bone cancer. It’s those frauds I’d rather see science attack.

  232. Ben Goldacre said,

    February 7, 2006 at 8:23 pm

    not gonna use an mp3, io do have some sample cds but they’re just live drums, not very exciting to listen to.

  233. Briantist said,

    February 7, 2006 at 9:47 pm

    You’d said you were doing it from a CD, so I was going to offer the MP3 whilst I found the original file. It looks like it’s been put in an off-line archive.

  234. Briantist said,

    February 7, 2006 at 10:01 pm

    Katie

    It’s not really for “science” to attack anybody. Whilst you are right in saying benefits to humanity are greater if lives are saved, I think it is possible for Ben to provide a platform for matters great and small.

    If science has anything to teach us is that we have to apply the same standards to everything. It matters not what the papers say, or what’s on TV, who’s in and who’s out, what God you like. The standards apply to everything.

    I understand your anger at a chiropractor who caused you distress, I’m sure everyone would.

    But Ben can’t fight your battles for you. And nor can “science”.

    Can I suggest that you find out who is responsible and then use science and scientists to prove your case?

  235. Penelope said,

    February 7, 2006 at 10:17 pm

    I would not pay extra for something if I could not hear/taste/feel/smell the difference. Do a blind trial and then we will know.

  236. Mike said,

    February 7, 2006 at 11:38 pm

    I didn’t realize I was disagreeing with anyone as to the outcome of the “experiment”; I guess I missed that post where the audio/electrical engineer said mains cables actually make a difference. I had thought I was making constructive suggestions, but since my comments evidently are being interpreted as being destructive, I’ll offer one last query, and then a comment:

    I still don’t understand why just typing “diff file1.wav file2.wav” won’t do this without bothering large bunches of people (as some besides myself have already pointed out). Can someone please explain this to me? Note that this has already been done by others with results posted on the internet, where (at least in theory) you’d be able to find it so as to avoid “reinventing the wheel”, so to speak.

  237. Ian D. said,

    February 8, 2006 at 12:34 am

    Ben,

    Don’t forget to control the rf noise. As rf induced noise goes, it’s pretty unpredictable. Could be that recording A was performed during a high noise period relative to recording B. You need to think of how to create the ‘all else being equal’ conditions. Find a good Faraday cage, and use a calibrated rf noise source. Stuff like this can be found at places where they do antenna tests, and electromagnetic compatibility (EMC) testing. I’m in Canada. Don’t know where they do that in the UK. Googling it came up with a few places.

    Either that, or do lots of recordings with cable A and cable B !! Then get the stats book out.

  238. Ian D. said,

    February 8, 2006 at 12:48 am

    I think that there are several people (perhaps I can count myself as one of them) that already “know enough” about rf (17 yrs designing rf and microwave comms equipment) and such things to fairly reliably predict the outcome of this experiment (at least we think we know), much in the same way that we all know that we no longer have to actually throw someone off a skyscraper to know they will splatter all over the pavement.

    But those unfamiliar with this area will, rightly, demand we do the experiment, and, as good scientists, we must oblige.

    The correct scientific stance is to DO THE EXPERIMENT. We’re still testing the theory of relativity to the breaking point, 100 years later. It is the way of science, and people who know a lot about a subject (I count myself in) can sometimes be wrong. It is exactly this type of thing that experimentation and the repeating of experiments, is designed to avoid.

    Do the experiment.

  239. Andy said,

    February 8, 2006 at 1:02 am

    But would a person splatter? I don’t think terminal velocity for a person is sufficent to get a good splash.

  240. Ian D. said,

    February 8, 2006 at 4:32 am

    Audiophiles splatter.

    It’s not the actual, physical, splattering splash I was referring to – it’s the ‘sound’ of the splattering that really matters. The sound is everything.

    …Especially if you record the sound via an rf filtered, high speed, 128 bit, superconducting ADC cooled to 4K with liquid hydrogen to reduce shot noise and play it back through a 50 Megawatt power amplifier, connected via coaxial air-line made of CONETIC-AA high permeability magnetic shielding, arc welded to the speakers to elminate unwanted noise and maximize power transfer. … the whole thing powered through a special kettle lead, of course.

  241. Patrick Caldon said,

    February 8, 2006 at 7:29 am

    I’ve just read all of this thread – ouch!

    I agree strongly with the trios.

    I’m worried about automated hacking of the survey – given the (remarkably!) strong feelings this has already generated, a malicious person might build some kind of automated bot to try to swamp the results over the web. There’s a 50-50 chance as designed if they made a “vote A” bot which swamped the results they’d get it right.

    The solution might be to make an A/B/X_1/X_2/X_3/X_4/X_5 trial, with some unknown number of X_1…X_5 make with the same procedure as A and B. A ‘correct’ answer might be A/B/B/A/B; there are now 31 incorrect answers and a swamping attack is much less likely to succeed. This would dramatically decrease the chance of such a swamping attack being successful, and would make the common voting pattern of a swamping much more detectable and distinguishable from a real power-cable based effect.

    Someone trying to demonstrate there is no difference between A and B will be killed by the several deliberately modified trio (or multi-tuplet) method.

  242. Patrick Caldon said,

    February 8, 2006 at 7:34 am

    Sorry – long day – I should say agrre strongly with the deliberately modified trios.

  243. Chris L said,

    February 8, 2006 at 10:18 am

    Mike (#236): I feel your pain. See also my posts #171 and #174. Do you think Ben is including us in the group of people who claim to already know the outcome of the experiment?

    If so, I can only repeat that I *am* advocating doing the experiment – I’m simply pointing out that there is a very strong expectation that all digital recordings of the same track from the same CD transport will be bit-for-bit identical, regardless of the source of the mains power, and that the required comparison between them does not necessarily need to be a listening test. Distributing bit-for-bit identical files for a listening test would reduce the whole thing to a farce, so the ‘diff’ needs to be done, if for that reason alone.

  244. Tim Day said,

    February 8, 2006 at 10:36 am

    Can I just say at this stage that I find it faintly disturbing that an experiment on hi fi power leads has excited so much more comment than the stuff that has an impact on human health, like MMR.

    Yes, I know I’ve contributed as well…..

  245. CB said,

    February 8, 2006 at 10:39 am

    But wouldn’t it be extremely interesting to find that people said they could hear a difference even if the files were absolutely bit-for-bit identical as you say? Thats pretty much what the audiophiles are saying – even though the music is in all probability completely identical, they can still *hear* a difference – and so that is what we need to, and will be, testing.

  246. CB said,

    February 8, 2006 at 10:44 am

    Tim, I think the reason that people are interested in the power chord debacle is that its a matter that really could be cleared up with a few (fairly) straightforward experiments that we can actually do ourselves, whereas the whole MMR thing is a bit more tricky to do and has to be done by a shitload of doctors, patients and suchlike and so we are to a certain extent powerless to contribute anything.

  247. Chris L said,

    February 8, 2006 at 10:55 am

    CB (#245): Yes, it would be interesting, but it wouldn’t address the power cord manufacturer’s claim that their cord affects the digital output of the CD transport, which is the hypothesis under test. Any differences people might hear in bit-for-bit identical files cannot have anything to do with any part of the original source of those bits. We’re going to end up with people claiming that the file sounds much better when played from a SCSI drive than from an IDE drive, which won’t help anyone. (I’m not joking, by the way, people do claim such things).

  248. Ian D. said,

    February 8, 2006 at 11:21 am

    Whether the files are bit for bit identical or slightly different, one still needs to be sure that the difference came from the cables, not from anything else. (stray magnetic fields, cosmic rays causing bits to toggle in the chips etc.)

    Controlling the recording conditions is vital.

    It’s also possible to just put up some control files which are identical (bit for bit) to see if people perceive a difference. Perhaps the second time listening to something compared to the first time will have an influence.

    Of course that assumes the people listening to the recordings are listening through one of these cables!!?? – They can always claim that the listening end was so corrupted with rf induced noise that both recordings are identical sounding.

  249. Malcolm White said,

    February 8, 2006 at 12:06 pm

    If you want some really bad Bad Science try this site www.nordost.com

    I quote from their FAQs

    “Question:
    Are the cables directional?

    Answer:
    Nordost cables are not directional when they are manufactured. However once they have been used for 70 to 80 hours in one direction they will sound better when they are hooked up and used in the original direction of break in.”

    Having a degree in electronics I have tried on infrequent occasions to point out to HiFi magazines that they are spouting rubbish about cables but no-one seems to listen. One of my lecturers when I did my degree back in the 80s was a Dr Hawksford whose research was quoted by people like Celestion and International Rectifier and had there been anything special about cables that would improve audio quality I’m sure he would have mentioned it.

    One thing that nobody ever mentions is that the circuit boards and transistor legs in hifi equipment are made of ordinary copper. Yet people insist that having a cable made of “linear crystal oxygen free” copper cable will make a difference.

  250. The Engineer said,

    February 8, 2006 at 12:36 pm

    Don’t have time just now to read the whole thread – so apologies if i’m repeating. I’ll come back and read the whole lot at some point.

    If i was betting i’d say that the experiment will show no difference between cables. However i do see one potential weakness in the method. The point of the cable is to ‘shield RF interference’. In the original experimental design the amount of RF interference is not controlled. To be truly robust we need to have the SAME RF interference on the cable when all 3 recordings are made, and that interference should be at the level normally experienced in the home.

    Alternatively enough separate recordings need to be made to make the variation
    unimportant. As someone said earlier A/B/X may be too simple. Someone who knows alot more about statistical significance and RF than me will have to help out further i feel.

    The Engineer

  251. Ben said,

    February 8, 2006 at 12:47 pm

    What is the hypothesis you are trying to test? That there is a detectable difference between the sound quality with cheap or expensive power cables, or that there is a difference sufficiently large that a large number of people, all listening on different and generally low-quality equipment, will detect the difference?

    Increasing the number of respondents will improve the power of your test if it is the second hypothesis you are interested in. But an audiophile will still be able to claim that some people, with better hearing or more expensive equipment, will be able to appreciate the difference. There is a limit to the amount you can improve an experiment by increasing the statistics without using more sensitive apparatus. Conducting the test in a controlled environment with a smaller number of “expert” listeners would be a more interesting experiment to my mind.

    There are in fact two experiments that could be done this way. One, more similar to Ben Goldacre’s proposal, is use the same equipment (leads and all) to play all sound samples, so that the only difference is in the digital input, which would be produced using the two different leads. The other would be to listen directly to the equipment with the different leads, and would be sensitive to any analogue effects of the power lead on the actual sound produced.

    While it is easier to test the hypothesis that there is a perceptible difference in the digital signal, the most interesting outcome is that a (perhaps rather rash) marketing claim is disproved. It would be more interesting (to me at least) to test the actual sound quality.

    Not that I plan to spend £1800 on a hi-fi lead either way.

  252. Chris L said,

    February 8, 2006 at 2:07 pm

    Jesus wept. Ian D (#248):

    “Whether the files are bit for bit identical or slightly different, one still needs to be sure that the difference came from the cables, not from anything else. (stray magnetic fields, cosmic rays causing bits to toggle in the chips etc.)

    Controlling the recording conditions is vital.”

    If the files are bit-for-bit identical, as we have every right to expect, there will be no difference to explain. It is only if Ben’s experiment demonstrates bit-for-bit differences in the files resulting from repeated recordings over time that we will be justified in starting to hypothesise causes for the difference. We have absolutely no evidence that “controlling the recording conditions is vital”, and every reason to believe that the recording conditions will make absolutely no difference at all.

    This is exactly analogous to arguing over how homeopathy works, without first bothering to prove whether or not it does.

    If it were in any sense common for RF interference or cosmic rays to affect the transmission of an absolutely glacial bit stream (150k/s) along two or three feet of optical fibre, the very computer you’re using to read these words could not operate. This is not a tricky thing to test, it really isn’t.

  253. Steve Phelps said,

    February 8, 2006 at 4:11 pm

    You need to differentiate between the following hypothesis and ensure you sample is controlled correspondingly:

    A. People on average can tell the difference

    B. People with extremely good hearing can tell the difference

    If you want to test B then an anonymous online trial is probably not the best way to go……

  254. Chris L said,

    February 8, 2006 at 4:36 pm

    Did I mention hypothesis C?

    C. There is no difference.

  255. Steve Phelps said,

    February 8, 2006 at 5:23 pm

    Chris L said:

    “1) If I were to rip the same CD onto the same computer more than once (assume ripping to 44.1kHz /16-bit WAVs), would the files be expected to be identical, at the bit level? I would suspect they should be, allowing for the significant error-correction built-in to the CD standard.”

    From what I’ve read this is _not_ the case. Real-time digital-audio decoding as performed by CD players can introduce errors due to jitter, see eg the citation below. I think this is because the original red-book specs for cd audio placed an emphasis on the real-time nature of decoding for audio as compared for eg reading from a CD-ROM where it is absolutely critical that you are bit-perfect.

    Jitter: Specification And Assessment In Digital Audio Equipment
    Abstract: Timing jitter in digital audio equipment can subtley degrade the audio quality or even cause data transmission failure. This paper examines the jitter performance requirements for digital audio equipment in the context of the audibility of sampling jitter modulation effects and the digital audio interface specification. It concludes by presenting techniques for the measurement of jitter performance.

    Julian Dunn – `Jitter: Specification and Assessment in Digital Audio Equipment’. Preprint 3361, presented at the 93rd AES Convention, San Francisco, October 1992.

  256. Steve Phelps said,

    February 8, 2006 at 5:31 pm

    Useful info on jitter:

    www.tc.umn.edu/~erick205/Papers/paper.html#jitter

    I guess the other point this raises is that variations in effect due to different hifi components needs to be controlled for. You should repeat the experiment with a different sample of, eg CD-players.

    For example, given that the manufacturer would probably claim something like “noise in CD players can introduce jitter and by using a higher-quality power signal you can reduce jitter in some CD players” then you need to ensure that you use a CD-player that is prone to jitter as well as a jitter-free CD player.

  257. Andy said,

    February 8, 2006 at 5:52 pm

    That Nordost FAQ has got to be a joke.
    “Question:
    Why are Nordost cables not shielded. Won’t I have a problem with noise?

    Answer:
    The use of flat cable geometry coupled with very precise conductor spacing alleviates the need for conventional shielding. In addition, the use of extruded Teflon provides better shielding than conventional insulation techniques. Nordost cables are not shielded because shielding increases the capitance of the cable by a factor of 55% or more when applied in the conventional manner. If the capacitance of the cable is increased high frequency information is rolled of and you don’t hear all of the musical information. ”

    So if you have a flat cable you don’t need to shield it. Odd. For non-audio use the best way of avoiding noise in an unshielded cable is to twist it.
    However Teflon, the wonder material, somehow manages to provide RF shielding despite being an insulator. So that’s all OK then.

    Also it’s rather amusing that they give the signal speed as a percentage of the speed of light for their cables. Perfect for those who don’t want to wait too long for the sound to get to them.
    But then I’m sure that there are people who claim that they can hear a 1 meter difference in the length of two speaker cables. After all any true audiophile positions their speakers and their head to within 1 micrometer when playing music. ;-)

  258. Ronald said,

    February 8, 2006 at 5:54 pm

    I think that in taking a stance against this “Bad Science” Ben has overlooked to amplify the primary hypothesis of any experiment: “Please prove me wrong”.
    The way experiments should be designed is not to prove anything, but to disprove it (which is supposedly easier).
    So although the standard double blind is the right way to go at it, one should also define what the hypothesis is.It should be (IMHO): “The cables do have an effect” (and not the other way around). Then you define what way to go about it to disprove that supposition (in this case a double blind), but also what the parameters are when you can say it is disproven (in this case standard statistical parameters ie if there is no significant difference from 50% on the unknown file)..

    One question remains: should there not be 2 files that are unknown to have a correct (or fair) experment?

  259. Chris L said,

    February 8, 2006 at 6:15 pm

    Steve Phelps (#255) wrote:

    “Real-time digital-audio decoding as performed by CD players can introduce errors due to jitter”

    Bear in mind that there is no “digital-audio decoding” involved in the creation of Ben’s files. Only when the files are played back is the digital information fed through a DAC to produce an analog signal – jitter may be introduced at that point. See the link in post #256:

    “It occurs in both analog-to-digital and digital-to-analog conversion. The latter instance is the only concern here.”

    and pertinently:

    “A great deal of money has been made by shrewd marketeers preying on the fears of the consumer worried about jitter. Such products marketed include disc stabilizer rings to reduce rotational variations, highly damped rubber feet for the players, and other snake oil remedies.”

    I take it no-one has performed the simple experiment I outlined in post #162, when I was still a young man? I hope to be back at home tomorrow night, when I will try re-ripping some CDs which I have previously ripped; I’ll diff the files and report my results here.

  260. Briantist said,

    February 8, 2006 at 6:16 pm

    There must be some way of using a quantum recording to make the CD in the first place, so there can be some uncertainty for the mains cable to effect.

    Back to the subject of jitter. Look at it this way: 44,100 times a second a sixteen bit sample is taken of the left and right audio channels. This is represented by thirty-two bits of information each time in a bitstream.

    You could use the same data to do 88,200 samples a second of a mono sourse, or 176,400 eight-bit sampes of a mono souce, or 1,411,200 one bit samples.

    Leaving aside the sound quality at one bit mono, how would the power lead effect the transfer of the data at these rates?

    Answer: it will still be 100.000000000%

    Jitter is a red herring.

  261. Briantist said,

    February 8, 2006 at 6:27 pm

    From what I’ve read this is _not_ the case. Real-time digital-audio decoding as performed by CD players can introduce errors due to jitter, see eg the citation below. I think this is because the original red-book specs for cd audio placed an emphasis on the real-time nature of decoding for audio as compared for eg reading from a CD-ROM where it is absolutely critical that you are bit-perfect.

    That’s SO analogue of you! CD audio must be bit perfect too. Just because it’s a 1 and a 0 switching over, it might be representing any one of sixteen values from 1 to 32768.

    Datastreams MUST be bit-perfect, otherwise they are useless.

  262. Andy said,

    February 8, 2006 at 7:11 pm

    Briantist,
    Raw CD data streams are NOT perfect. If the error correction was perfect then 1) there would be no requirement for CD players to interpolate over missing samples if required and 2) They wouldn’t throw away 10% of the space on a CD-ROM by using it for extra error checking.

    Digital computer data must be perfect, digital audio and video does not need to be. People can’t spot small differences and the audio and video standards make use of this to simplify the error correction and/or reduce the bandwidth of the data.
    One obvious example, virtually all audio, video or still image compression systems are lossy (e.g. mp3, oog,mpeg,mov,jpg). On the other hand data compression applications are always lossless, a lossy data compression system would be pointless.

    The CD standard was designed for audio. Yes it is designed to give perfect playback but the same philosophy was applied, it is also designed with the minimum required to reproduce the audio. The standard allowed for non-perfect (but good enough that no human ear could tell) reproduction of the sound in the event that the true data could not be recovered. This fallback wasn’t sufficient for data CDs and so extra error correction was added to the data standard resulting in a 10% reduction in data capacity.

    That said the reduced error correction on an audio CD is sufficient to correct almost all errors assuming the disc isn’t badly damaged. From what I’ve found comes out at an average of about 1 bit wrong every two-ten discs after correction. More than good enough for audio, just not acceptable for data.

    Oh and “one of sixteen values from 1 to 32768″? I think there are roughly 32k values in that range.

  263. Andy said,

    February 8, 2006 at 7:26 pm

    D’oh!
    I see what you mean by 1 of 16 values. With a one bit ambiguity in a 16 bit number there are 16 possible values within that range which the data could represent.
    Sorry, the coffee hasn’t kicked in yet this morning.

  264. Ben Goldacre said,

    February 8, 2006 at 11:42 pm

    www.roger-russell.com/wire/wire.htm

  265. Carl Johnstone said,

    February 9, 2006 at 12:13 am

    One interesting comment made elsewhere in here. Why aren’t these types of cables being recommended for computers?

    You’ve got electronics running at differing frequencies right into Ghz. The pathways the electrons are travelling along are as small as 90nm. Surely it doesn’t take much interference to stop all that from working.

    Yet audio equipment that isn’t made to anywhere near the same levels of precision require such cables?

  266. Ian D. said,

    February 9, 2006 at 4:00 am

    Chris L (#252).
    Chris you’ve missed my point completely. Read my earlier posts. I for one do not believe the cables make any difference. I’m betting that there will be no change. But to do science, you can’t just claim it’s obvious and let it drop at that (even though I know you are correct). You have to carry out the experiment anyway. Homeopathy and audiobullshit are just that.. bullshit. But nevertheless, you have to run the experiment otherwise you’re playing in the same league as them.

    Controlling the recording conditions is absolutely necessary or your experiment is garbage.

    I noticed talk in some posts of doing just a few recordings. If we do just two files (say), and notice a difference, that tells us precisely nothing. A worthless experiment. Basically telling us… back to the drawing board and do it again, this time configure it properly because you need to know that the difference didn’t come from somewhere other than the cables. And doing many recordings to overcome this scenario is what I was implying by my comment on “doing the stats”. As that is the only way to determine if it was the cables if you don’t take particular care to control the conditions. But who wants to hearing test a couple hundred files? Or even 10?

    As for stray fields and cosmic rays affecting glacial bit streams, I was only exaggerating a bit (for drama), but you’re mistaken about that too. The bit stream speed has nothing to do with the matter. Data bits sitting in a chip can flip due to cosmic ray events. This guides the selection of spacecraft microprocessor memory, CMOS chip latchup etc. I work in the space industry and we have to deal with things like this all the time. You can flip data bits in a piece of music and not notice and audible change. And newsflash, there are times when my computer DOESN’T faithfully reproduce what it’s supposed to as bits get flipped in transit. This can happen anywhere the signal to noise ratio is low. All stored data will spontaneously undergo degradation due to entropy.

    If you place data on a chip and leave it there, there is a finite, non-zero probability (small though it may be) that you will flip a bit. Also, stray electromagnetic fields can
    also do that. Stray fields can also induce EM noise via paths OTHER than the cable. Also, if you happend to record one file while there was a huge transient on the powerline with the “ordinary” cable, you may falsely conclude that the bullshit cable indeed does something.

    That was my original concern for the test: I expect that the cable has so negligible an effect that other noise sources become the dominant contribution to any bit errors.

  267. Chris L said,

    February 9, 2006 at 9:38 am

    Hi again Ian,

    I do understand your point about controlling the recording conditions. My point was that, as you say in post #266 above, this only becomes important when one is trying to determine the causes of any differences in the recordings. We do not yet know if there will be any differences. There are two distinct questions to be answered here, with the second contingent on the first: (1) does meddling with the mains power delivery affect the outcome?, and (2) ***if and only if the answer to (1) is yes*** what causes the difference? In Ben’s experiment, question (2) is actually “are the differences audible?”, but the criterion for proceeding to the second step still applies. Only if a difference is detected, either at the bit-level or aurally, does one need to start worrying about putting all the equipment in a Faraday cage. I suspect we both agree that this experiment won’t ever get to question 2, so long as question is actually asked, which is all that I’m arguing for.

    I understand that digital media and digital transmission are not infallible. However, you yourself argue that the errors are at the level of 1 bit per 2 to 10 discs. Such errors are completely irrelevant when dealing with claims that the improved optical digital output leads to a “fuller, more musical sound with less distortion” – to have that effect, we can safely say that there would have to be a underlying error rate of significantly more than one bit per second – at least three orders of magnitude more than you say actually happens.

  268. Chris L said,

    February 9, 2006 at 9:42 am

    Ben,

    Those cable lifters are great. However, you’ve barely scratched the surface of hi-fi ludicrosity yet. Have you been made aware of the “Clever Little Clock” yet?

    See www.machinadynamica.com/machina41.htm, but make sure you’re sitting down first.

  269. Martin Burley said,

    February 9, 2006 at 9:58 am

    Hello again,

    I’m glad to see ‘cashebo effect’ is catching on as a phrase. Andrew’s (#169) ‘cashebo horizon’ would be a fun idea to test, though if it caught on as a testing method, I imagine manufacturers would circumvent it by producing luminous equipment!

    Has anyone else checked out this article: stereophile.com/asweseeit/805awsi/index1.html

    It seems pretty relevant to this discussion because, as fun (?) as all the above technical discussion is, if the hi-fi guys are saying things like, “Let’s start by pointing out again that all of us can learn to improve our scores on blind audio tests,” then they’re just going to claim that the tests proposed above are invalid (if no significant difference is found). We haven’t been using the cables long enough to ‘train our ears’ to hear the difference.

    So whatever the results of the on-line test, the real test would be to take a bunch of people (e.g. hi-fi magazine editors) who claim their ears *are* sufficiently trained to hear the difference between Supercable Ultra7000 with Added Zing, and Kettle Lead, and challenge them to show that they can still hear the difference in a blind test. (Presumably the editor of the article would agree the differences caused by differing sampling rates that he describes are ones he can (with his now-cleverly-trained ears) hear in blind tests, so he should have no reason to quibble with this kind of set-up.)

    The tricky part, though, is finding people who admit to having the expensive cables *and* believe the cables make an audible difference *and* are brave enough to take the blind test. Does the manufacturer do customer satisfaction surveys? Maybe Ben should challenge them to let a sample of their most-satisfied customers (who presumably ‘hear’ the difference) take a blind test? There’d be no good reason for them to refuse.

    Sorry to interrupt the technical stuff… back again in another hundred posts or so…

  270. Martin Burley said,

    February 9, 2006 at 10:07 am

    …or sooner…

    *wow* – that ‘clever little clock’ is a superb link, Chris. And they come with a money-back guarantee! 30 days should be plenty long enough to have some fun with that thing… I wonder if standing it on a cable lifter makes it work even even better?!

  271. stevemosby said,

    February 9, 2006 at 10:08 am

    There’s obviously a lot of very intelligent, very specialised people here and so I’m nervous about adding a comment to a very long thread, but I agree (I think) with Chris L.

    If digital copies of the same piece of music are identical, give or take small errors, then surely all you need to do is look at one done with the fancy power cable, one without, and see if there’s any meaningful difference beyond the error level. Do ten copies with and ten copies without and see if anyone can analyse the files and tell the difference. As someone pointed out, the music comes from the data, and so if the data is the same the music must be too.

    Ian D: “But to do science, you can’t just claim it’s obvious and let it drop at that”.

    To an extent maybe, but you don’t have to reinvent the wheel over and over again. If someone claims they’re Napoleon, you don’t do a background check to see if they are. If the sound files are the same and someone says they can tell the difference, aren’t you moving into more of a paranormal claim? You need to test them. And if the sound files are different then the main objection lying behind all this, which seems to be “a power cable altering the sound – how ridiculous” has surely disappeared?

  272. Chris L said,

    February 9, 2006 at 11:13 am

    Mike (#270) – yes, I like the Clever Little Clock. Ben had better check with his neighbours that they don’t have one, or the Clock’s effects might skew his results.

    Re: your earlier link: the audiophiles’ refusal to accept double-blind testing puts them in a very difficult position regarding things like the Clever Little Clock. As Ben wrote in his original article, they claim that no objective measurement or test can trump their subjective opinions, so they are forced to accept that, if at least one person hears a difference after “installing” a Clever Little Clock, then it must be having an effect in the real world, and not just between the listener’s ears. How they can bend their minds around that sort of doublethink, and yet still retain sufficient grasp on reality to operate cars and computers, is what really interests me.

    By the way, check out the rest of Machina Dynamica’s products; they all operate on similar principles and are all backed up by equally convincing evidence.

  273. Andrew Rose said,

    February 9, 2006 at 11:37 am

    Ben

    Now that this discussion is no long that it’s unreadable I’m not sure whether you’ve sourced your music or not yet. I’d be happy to supply you with a CD-R of classical music in which I hold the copyright for the purposes of this experiment – and suggest you use more than 20s of it.

    The CD(s) I can send you would be written directly from our master files, with CD error rates well below those you would expect from a commerically pressed disc.

    This would have the added advantage that I could compare not just your CD output files against each other, but both against the masters.

    The slight downside is that we’d be dealing with recordings remastered from 1950’s analogue sources. The quality on the best of these is remarkably high, and given the claims made for these cables, should be more than sufficient to do the test. Not that we’re not a tin pot outfit with a couple of record players – we have a number of award-winning CDs under our belt and a string of excellent reviews from the music media.

    What this achieves is high quality audio that side-steps the copyright restictions you’ll be up against by using anything published less than 50 years ago, and the publishing restrictions on anything written by a living composer or one who died less than 70 years ago.

    Details of recordings and contacts here: www.pristineaudiodirect.com

    Let me know if this fits the bill…

    Regards

    Andrew

    P.S. I found this review with reference to some $470 power cables (from a sales page): “Most competent after-market power cords will clean up the sound, usually by lowering the noise floor. Despite that general but important improvement, some cords then take a step backwards by imposing some measure of tonal degradation. Few of the cords I’ve tried seem to inject a life-like force to the music. Fewer than that improve timing and flow. Fewer still will broaden your musical horizons by rewarding you with depth of stage, all the while giving you insights on performer technique by allowing your gear to better resolve low level and inner detail in a musical, organic manner. The powerChords on my CD player and amp delivered the goods. Highly recommended for audition”

  274. Dave Berry said,

    February 9, 2006 at 11:38 am

    Wow, amazingly long thread!

    As a musician, producer, audio engineer (who’s ears are wearing out) and ex-hi-fi afficionado, I would like to add my tuppence…

    Surely the real point is that even if there is any difference whatsoever between bog-standard mains lead and the £1800 one is that there is never going to be EIGHTEEN HUNDRED QUID’S worth of difference?

    Heard the one about directional connecting cables?

  275. Ian D. said,

    February 9, 2006 at 11:39 am

    Chris L. (#267)

    Chris yes we agree on the cable being crap. However let me play devils advocate and point out one unsettling scenario:

    Let’s pretend that the cable really does work. Just pretend. It really does and amazing job at filtering unwanted electromagnetic fields.

    We perform a recording with the ordinary kettle lead when, unknown to us, the background field is very low. Hardly any noise. And also unknown to us, we perform the second recording with the ‘magic’ lead when the background noise is extremely high. We compare recordings and voila! They are identical. We stand up and proclaim that the magic lead doesn’t do anything. We’d be wrong.

    I’m just giving them the benefit of the doubt. (even though I myself already have a strong sense that their crap product doesn’t do anything).

    If we control the noise, they have no case. One, perhaps two recording tests are all we’d need.

    But hey, if Ben or someone wants to do the statistics, the other method will eventually give the same answer… perhaps less convincingly, for the audio’piles’.

  276. GWO said,

    February 9, 2006 at 12:08 pm

    #262 “one of sixteen values from 1 to 32768″

    The original was quite right. A single bit represents one of sixteen possible values, and those values are 1, 2, 4, 8, 16, 32, 64 … etc … 32768

    Suppose my quad was 0000000011111111 = 255
    and my error is to flip a single bit.

    If I flip the last (least significant) bit, 0000000011111110 = 254 and my ear can’t tell the difference.
    If I flip the leading bit, 1000000011111111 = 33023, and it can.

  277. Chris L said,

    February 9, 2006 at 12:22 pm

    Hi Ian,

    Ok, I see your point, and I agree that some sort of control would avoid the scenario that you describe. Perhaps I misunderstood your earlier posts, but I would suggest that, if adding this sort of control to the experiment, it’s important to control for all possible environmental effects, not just mains noise. Hence, rather than taking special pains to ensure a clean or dirty mains supply, it might be better to simply make three recordings via each lead, alternately – normal, magic, normal, magic, normal, magic. This would give a lot of confidence that any environmental variations will affect all recordings equally, on the average.

    Anyway, even that’s starting to sound like overkill… :-)

  278. Ian D. said,

    February 9, 2006 at 1:38 pm

    Agreed. Overkill. I hate to make a mountain out of a molehill. We already know the answer, but…. just in case….

    I can think of a way to reduce interference using a cable, if the cable had embedded circuitry. (actually one can buy filtered lines). . But it still wouldn’t cost 1800 !! …Unless they’re hand-braiding the thing, using actors or NBA basketball players for labour… :-)

    So it’s clear from the cost alone, that this is a scam.

  279. Drew Stephenson said,

    February 9, 2006 at 1:47 pm

    At the risk of getting caught up in something way beyond my technical knowledge, and dragging in a point from about 100 posts ago, wasn’t John Thorpe’s point that the data that comes out of the optical output is more than that which gets put into the recorded file, so that by working from a recorded file we aren’t, in fact, testing the whole of the output and therefore are not performing a valid experiment?
    That was my understanding, flame away if i’m wrong :¬)

  280. Andrew Rose said,

    February 9, 2006 at 1:56 pm

    Drew – the only thing you’re going to hear is the recording – i.e. the recorded file. I don’t quite understand your point. What else is there, musically, to hear – other than the recording?

  281. TRiG said,

    February 9, 2006 at 2:17 pm

    Okay. Three possible claims, the first (and seemingly least likely) of which you are testing.

    1. The fancy cable will effect the digital output to a computer recording device over an optical cable. This claim has been specifically made to you by one reprasentitive of one manufacturer, but has been frequently derided here. It claims that the cable will affect the quality of data kept entierly in the digital domain.

    2. The fancy cable will reduce timing errors (jitter) and so improve audio quality. This is probably impossible to test except in ‘live’ circumstances.

    3. The fancy cable will be less susceptible to sudden interferance from central heating and the like. Now that could be easily tested, surely?

    The first claim is the one you started off with, but the other two, which have come up here, are equally worthy of testing, are they not?

    The assertation that the power cable will affect the quality of digital data sounds to me quite increadible, but it does seem to have something going for it. There are read errors, and presumably a steady background (less noise) could reduce their frequency. And if the error-correction on audio CDs is less exacting than that on data CDs, there could be an argument for using these leads on hi-fis but not on computers.

    But: (1) Is there really that much difference in the fluctuatuon (noise) between cheap and expensive leads? (2) Will the noise have much impact on the number of read errors? (3) Will a couple of extra read errors have any significance on the sound quality detected by the human ear?

    I suspect that the answer to all three questions is no, but the experiment is still worth doing. (I suspect is not a declaration of fact.)

    TRiG.

  282. TRiG said,

    February 9, 2006 at 2:23 pm

    I’d just like to mention that I’d only read as far as post 236 when I typed that comment, but by the time I got around to posting it it was number 281. So it didn’t take account of the last few posts and may duplicate them.

    Sorry.

    TRiG.

  283. Drew Stephenson said,

    February 9, 2006 at 2:52 pm

    Sorry, the point i was failing to make was:
    If the data that is captured in the recorded file is not the whole of the data that is being sent from the optical output (as i understood John’s point to be), then we are not testing the whole of the data.
    Therefore there could be affects on the data part that we are not testing and therefore the experiment is invalid.
    This argument is redundant if i’ve misunderstood John’s point waaaaaay back.

  284. David Hardman said,

    February 9, 2006 at 3:55 pm

    Yes, the idea of the blind trial is good. However, just suppose that it turns out that the sound from the new power cable turns out to be slightly more highly rated than that from a standard cable. Personally, I think this is unlikely, and nobody here seems to think that any difference would be a large difference. But just suppose there’s a small difference… Does this mean people should spend money on the new cable? Perhaps not. After all, when the equipment is installed in your home you will not be in the situation of comparing two different sound outputs; you are just listening to the sound from a single piece of equipment. Thus, any small advantage that was noticeable in the comparison situation may not be noticeable in an everyday listening situation, and hence not worth paying money for.

    In fact, this suggests another experiment: The sound associated with each type of cable is rated separately (and blindly) by different groups of participants. You could even measure the average amount that people would be willing to pay for each cable. I can’t claim originality for this idea: there are several studies that show how people’s preferences can “reverse” depending upon whether they evaluate items separately or jointly. See this article by Chris Hsee:

    gsbwww.uchicago.edu/fac/christopher.hsee/vita/Papers/AttributeEvaluability.pdf

    I also like the idea that someone had of deceiving people as to which cable is being used. I’m reminded of a situation a while back when a friend was playing me a CD of some sixties band on a high-end hi-fi. He was bemoaning the inferior sound that the CD produced compared to the vinyl original. To demonstrate this, he dug out the original vinyl and put it on. Neither of us could tell any difference.

  285. Drew Stephenson said,

    February 9, 2006 at 4:30 pm

    Just to clarify, the post i’m referring to is number 182.
    We’ve had John’s technical opinion on this, is there someone who can verify/repute this?

  286. Chris L said,

    February 9, 2006 at 5:23 pm

    Drew,

    John’s point is a bit subtle. He’s not claiming that there is any extra digital information being sent that is not being recorded. What he’s describing is that, if one regards the stream of ones and zeros as itself being an analog waveform, where, say, a zero is 0 volts and a 1 is 5 volts, then there is extra information encoded in the timing of the transitions from 0 to 1 and vice versa. This is what the engineers call “jitter”.

    For example, if I were transmitting zeros and ones at a nominal 1000 bits per second, then the transitions between 0 volts and 5 volts should occur at exactly 1 millisecond intervals. However, nothing’s perfect, and in reality there will be small variations in the timing. If your digital-to-analog converter relies on the timing of those transitions, then the quality of the resulting analog signal will be affected by the extent of the jitter. If you’re just recording the zeros and ones to be replayed later, any jitter in the signal you’re recording is irrelevant. Similarly, if your DAC has its own internal clock, then again, its the DAC’s jitter characteristics that are important, and not those of the received digital signal.

    I wouldn’t like to comment on how many DACs have their own clocks, but they’re not uncommon – for example, any personal CD player which uses buffering for skip protection will have its own DAC clock.

  287. Drew Stephenson said,

    February 9, 2006 at 8:09 pm

    Ok, got that bit. now tell me, is the experiment still valid?

  288. Drew Stephenson said,

    February 9, 2006 at 8:19 pm

    ‘Cause it’s beginning to sound like it isn’t? If there’s a chance (and i’m just being theoretical here, i have no intention of spending £75 on a lead) that the lead could reduce the jitter at output but that this wouldn’t bve transferred to the recording then surely the experiment is invalid.
    Stop me if i’m being dumb…

  289. Andy said,

    February 9, 2006 at 8:34 pm

    Drew,

    I’ve yet to see any contradiction to my final paragraph in post 219 here (hopefully that worked).

    The experiment is to prove that the raw data from the CD is unchanged or if it is changed that the differences are to small for the human ear to detect.

    It becomes a question of what exactly the manufacturer’s claims mean. They claim it improves the audio on any possible system configuration. Is it possible under normal situations to build a Hi-Fi system where the only possible impact of a CD player on the audio playback of a CD is the raw digital data? If so then this is a valid test of their claim.
    If however it is not possible to build a real world HiFi system where the raw data is the only impact due to electrical interference (radiated or conducted), jitter on the data etc. then this in not a valid test of the claim.

    I think the general consensus here is that under at least some possible reasonable configurations the factors mentioned above can be reduced to the point where they are negligible if not completely eliminated. If that is the case then this is a valid test of the claim. It does however leave a large gap in the experimental practice since while most people feel that these factors are insignificant there are a few who disagree and we have no proof of this either way.

    Which is a very long way of saying that while I feel the experiment is valid but a little pointless and stretching the manufacturers claim a bit. However it leaves enough of a hole that it would be possible for someone to raise objections based on the experimental method.

  290. Andy said,

    February 9, 2006 at 8:37 pm

    Minor correction / clarification to the second paragraph of 289 above…

    Please replace “The experiment is to prove” with “The proposed experiment will prove”

  291. John Ruch said,

    February 9, 2006 at 9:06 pm

    My concern, which has been repeated by several others, remains unaddressed: Namely, what exactly is the claim that is being tested? If we go only by your manufacturer’s statement above–that there is “impact” and “affect,” those are too vague to test. They are entirely subjective. We are all talking a lot about analyzing “data,” but what data, precisely? Many posters seem to filling in the gaps for the manufacturer, speculating on electromagnetic filtration and so forth–but that is all beside the point. The claimaint must define their claim, or the test is meaningless.

    Without having a precise claim to test, you cannot design a meaningful experiment.
    You might as well just plug the cables and see what happens; the most complicated set-up, without a specific claim to test, amounts to the same thing.

    You should request a specific, testable claim from the manufacturer. I doubt you will get one. But that at least strongly implies they have no basis for believing their own claims. That may have to satisfy you.

  292. Drew Stephenson said,

    February 9, 2006 at 10:49 pm

    Ben’s already stated what the manufacturer’s claims are in the first article:
    “Replacing the mains cable to any of your hi-fi or home cinema components means they perform better, giving you less distortion, clearer pictures, and more musicality, so that you enjoy your music and movies even more!”

    “The key to the success of the Russ Andrews mains cables is the unique Kimber cable weave. The woven mains cable has been proven to dramatically reduce sound-degrading radio frequency interference on the mains supply and to reject further pick up of RFI.”

  293. Chris L said,

    February 10, 2006 at 10:24 am

    I agree that the jitter issue does cloud things somewhat – there will always be wiggle room for audiophiles to propose some real-world mechanism to explain their perceived improvements, even though – and this is the crucial point – it is perfectly well known in the audiophile world that many of those perceived improvements aren’t detectable in a blinded test.

    However, going back to Drew’s technical question – the jitter issue only invalidates the experiment to this extent: the manufacturer’s claims for the cable can only apply to systems without a re-clocking DAC. My own system comprises a Sony DVD/CD player connected digitally to a Yamaha DSP-A2 amp. If I were to enquire about buying one of these cables for my system, I would expect the salesman to tell me not to bother changing the power lead on the CD player, because it won’t make any difference. That might actually make a more interesting experiment… :-)

  294. Briantist said,

    February 10, 2006 at 10:49 am

    Let me just reitterate the point about how the digital information is carried.

    It is *NOT* represented using 0V for 0 and +5V for 1.

    The information is encoded using MFM encoding. This ensures that you don’t get lots of 0s or 1s in a row that you can’t clock.

    MFM improves on FM by reducing the number of flux reversals inserted just for the clock. Instead of inserting a clock reversal at the start of every bit, one is inserted only between consecutive zeros. When a 1 is involved there is already a reversal (in the middle of the bit) so additional clocking reversals are not needed. When a zero is preceded by a 1, we similarly know there was recently a reversal and another is not needed. Only long strings of zeros have to be “broken up” by adding clocking reversals.

    See

    www.storagereview.com/guide2000/ref/hdd/geom/dataMFM.html

  295. Briantist said,

    February 10, 2006 at 11:04 am

    I also find it amazing that people can trust computers to do almost everything, but something SO basic as timing, somehow when you put the electronics together to make a stereo that they suddenly can’t manage it.

    Timing is *EASY*. It’s just a vibrating bit of quartz. It’s known to vibrate at a set number of million times a second. Using electronics to county down (say /64 or /256) requires only basic electronic circuits.

    Jitter is easily fixed with a small FIFO buffer.

    But I suppose that, like Bistromathics (“Just as Einstein observed that time was not an absolute but depended on the observer’s movement in space … so it is now realised that numbers are not absolute, but depend on the observer’s movement in restaurants”) binary maths falls apart when used to represent audio data inside hi-fi equipment.

  296. GWO said,

    February 10, 2006 at 11:23 am

    Let me just reitterate the point about how the digital information is carried. It is *NOT* represented using 0V for 0 and +5V for 1. The information is encoded using MFM encoding.

    I thought CD-Audio used EFM, not MFM.

  297. Ben Goldacre said,

    February 10, 2006 at 11:34 am

    just got this from a hifi dealer with a sense of humour:

    Hi Ben!

    That experiment of yours, where you post an audio file and challenge HiFi buffs to detect whether it was recorded from a source which derived its power via a cheap or via a luxury cable; please be advised that I am able to offer your readers a special metacable, only £9999.99, which should be used to connect all test equipment to the mains for the duration of the test.

    #### ####
    ##### ####

  298. Drew Stephenson said,

    February 10, 2006 at 12:26 pm

    Ok Briantist, your corrections accepted, straight yes/no answer, is the test valid?

  299. Hatter said,

    February 10, 2006 at 4:08 pm

    “Despite this audiophiles want to go right up to 192/24″

    There isn’t really good reason to go up to 192, although such demands are amusing coming from a crowd that previously poured scorn on CD while drooling over vinyl, which after a couple of runs on the turntable will lucky to still carry anything over 15kHz.

    There is very good reason to do your initial recordings at a high bit rate – few noticeable errors in the end result after processing. And there is noticeably more dynamic range.

    Love this one – “My point stands though, I do not need affirmation from any one as to the honesty of my own senses”

    OK, I’ve stopped laughing.

    “after all that is what our brains do with the information received by the senses (noisy unclean data which the brain works hard on to re-construct a ‘pure’ original)”

    Our brains perform a fuzzy reconstruction of the information from our senses.

    There is no point arguing with lover’s of overpriced audio products. It’s a very expensive religion. It isn’t based on any verifiable facts.

    People who think science is the source of all problems in the world are quite welcome to eschew the use of all the fruits of science.

    “I find computer fan-noise very annoying, even if its barely audible. Most people don’t notice it at all.”

    That is all in your brain. You’d say the noise is loud, whereas someone whose brain filters the noise would say it is not. A simple scientific test would verify the actual volume level.

    Good microphones are not that expensive. Like speakers you have rapidly diminishing returns – one that costs 10 times as much will be only likely only be better. Most professionals buy specific microphones for specific purposes. Some are used because they colour the sound, others because they can handle a wide range of recording sources.

    “Would £1,800 be too much to solve that problem?”

    An overpriced piece of cable is hardly going to solve the type of power problem you describe.

    “Why buy a Bugatti Veyron (1000hp, 250km/h)”

    There is a significant, real difference. Driving a vehicle with a massive power to weight ratio is a different experience at any speed. The Bugatti Veyron can reach 250mph – there are numerous cars that can manage a mere 250kph. Based on the figures it appears it has the acceleration of a sportbike, and a much higher top speed (there is I recall a superbike built around a gas turbine engine that can reach 250mph).

    This isn’t a valid analogy anyway. A correct analogy would be if the Bugatti actually had the same engine as the Mondeo, but hidden under a shiny covering. That is the difference between for instance audiophile speaker cable and lamp flex.

    Peter it is true you can train your ear, but just as a wine or whisky taster should be able to distinguish varieties without knowing what they’re tasting the same should be true of these trained listeners. You do blind tasting to remove the obvious subjectivity of favouring what is more expensive and hence assumed to be better. What actually happens when you blind test these hi-fi cables is the audiophiles rate highly the one they think is the expensive one.

    Any reasonably-priced modern equipment will provide superb quality. Designing amplifiers is an art, not a science, but it is a well-established art and it is no longer necessary to spend a fortune to get excellent sound quality. The only component that has a major influence is the speakers – these are basically devices that turn the signal you put into them into heat and distortion.

    Alison network delays happen due to traffic contention, something that is not an issue on a point to point connection. Ethernet gets real data throughput rates of about third to half the stated throughput.

    Alison studios use higher bit depths because they run their audio through numerous processing stages and there are calculation errors at every stage – the digital equivalent of signal losses in analog recording equipment. These type of cumulative calculation errors are why graphics processors are working internally at 128 bits, even though they’re only going to output to the screen at 24 bits.

    Pathman there is a limit to how much current you can draw from the mains supply. A thicker cable might be better than a thin one, but we’re still talking off-the-shelf dirt cheap wire here. Even 30 quid would be excessive for a power cable. Same for speaker cable – it does need to be a decent thickness, but the cheap stuff you can pick at your local hardware store will be the equal of the most expensive cable from specialist suppliers.

    Wonder why John Vestman didn’t just do a binary compare to see if the data on the CD was the same as that coming over the optical link? When he copies files onto a flash stick does he listen to them to see if they still sound the same? Hold on I’m just going to listen to this PDF to make it still sounds good.

    You want to hear something really bad take a listen to Depeche Mode’s Playing the Angel CD – whoever mastered it didn’t just normalise away the dynamic range, as has become common, but pushed the entire disc into clipping. Thankfully it came with a DVD containing the same music as AC3 and DTS – these sound really good. Pushing up the volume on CDs is a very unfortunate trend.

  300. Hypolyta said,

    February 10, 2006 at 4:58 pm

    Living with an Audiophile, hi fi bigot I can tell you why your blind test would never work. The whole enjoyment of super hi-fi seems not to reside in the sound but in the sleekness of the black, silver or wooden boxes involved (depending on the ‘fashion’ this year), the colour and thickness of ones speaker cables and the strange electrical trickery involved in your power cables (as you are finding).

    Depressing as this is – and please don’t explain to me again why soft furnishings and MY books interfere with sound quality – its best just to leave the boys with their toys in their white room….. No-one has yet adequately explained to me why a record player (and I loved my dancette) should have a huge heavy glass turntable and manual arm…. or the big rubber band you need to move from sprocket to sprocket when playing LP’s or 45’s. Sorry that’s my girly rant on hi-fi. Thank the deity of your choice for MP3’s……

  301. mike gover said,

    February 10, 2006 at 7:27 pm

    Has anyone ever heard a CD of any kind of acoustic music or singing, that sounded indistinguishable from a live performance? The more I listen to my hifi the more I want to buy tickets for the real thing. You can run a test to see if file A sounds different from file B, and this seems a valid test of the manufacturer’s claim.

    But anyone unaccustomed to listening to live unamplified music is not qualified to say whether A is better than B. FWIW my ‘golden rule’ on hifi is that over a long period you should not spend more on equipment and CDs than you spend on tickets for concerts.

  302. Andy said,

    February 10, 2006 at 8:13 pm

    Briantist,

    Going by your comments here (295) I would take a guess that you have never designed anything requiring a high accuracy clock.
    Yes, generating a clock that if fairly good is trivial. A 32.768kHz standard quartz crystal, an inverter, a couple of caps and maybe a resistor or two. Feed the output into a 16 bit counter and you have a nice accurate 1 second clock.
    It’s accurate enough that the clock only wrong by around one hundred parts per million (PPM). Plus temperature effects. Plus ageing effects. Plus vibration effects (crystals are an electro-mechanical device after all). In other words fine for a watch but far from perfect.
    Have you seen the size of the crystals and support circuit on a high end GPS (I’m talking a >$10k type of system)? They are big and expensive and the error on the clock must still be calculated and removed from the calculations if you want to get any sort of accuracy.

    Generating a truly accurate clock is not a trivial undertaking.

    Also a clock recovered from the data will normally pass through some form of PLL. PLL’s jitter.

    Audiophiles here have claimed to be able to hear a difference in clock jitter in the range of picoseconds. With the data rates we are talking about that is an error of 1.5 PPM. Fortunately jitter is normally far less that the total error of the clock. Still creating a clock with that sort of accuracy and lack of noise while possible is sufficiently expensive and complex that few audio manufacturers will go to the trouble. It’s certainly not something you run off in a couple of minutes.

    OK, on to the second point. Yes a fifo would remove jitter on the input. The output will be as accurate as the clock you can generate locally.
    You seem to think that a digital system is somehow magic and immune to analogue effects. That’s just not true.
    Say we have out bit clock running at around 1.5MHz. That gives us a 666ns clock period. Even if we assume the part creating the clock was perfect the line connecting the clock to the PCB is not. It has capacitance to ground, resistance and inductance. The connection from the PCB to the silicon of the fifo also has all of these. So assuming reasonable quality parts the signal on the silicon of the fifo is going to take anything from 1 to 10 ns to transition from one state to the other.
    Within that window, somewhere near the middle, we will pass through the region where the fifo decides that the input has changed. We’ll assume that internally it is all a perfect digital system.
    OK, let’s make some basic assumptions, they aren’t perfect but they give us a rough feel for the numbers…
    Assume a 1 ns rise time from 0V to 3.3V. Assume a switching threshold at 1.5V.
    The fifo will decide that the clock has changed 455ps after the clock line starts to rise from 0 to 1. So far so good.
    But no power supply is perfect, the voltage and ground will move a bit due to noise. Let’s say that the only noise in the system is from the connecting cable from the CD player. We will get noise at the edges of the recovered incoming data clock as other logic switches clocking things into our fifo. Assuming the clocks aren’t perfectly synced this will by chance line up with our output clock edge some of the time. Secondly if we use copper interconnect then there will be noise coming in on that cable connected directly to our ground plane. All this noise will have the effect of adding or removing a small voltage from our incoming clock signal, technically it’s changing the reference voltages rather than the signal itself but it’s easier to follow if you view it as affecting the signal. Let’s say that this noise it adds up to 0.01V. This means that our clock edge could be detected anywhere between 452 and 458ps after the start of the signal.
    Even assuming perfect digital function of our clock source and of the internals of our fifo we have 6ps of jitter on the output of the fifo due to some fairly miniscule noise in the system.

    OK, this is far more detail than the situation warranted but I wanted to point out that the argument “It’s digital, it is all perfect” just doesn’t work when dealing with people who think they can hear timing changes of stupidly small amounts.

  303. Drew Stephenson said,

    February 10, 2006 at 8:18 pm

    Ok Briantist & Andy
    Same question!
    Straight yes/no answer, is the test valid?

  304. Andy said,

    February 10, 2006 at 10:20 pm

    You want to limit me to just a yes or no? Where’s the fun in that?
    In my opinion yes.
    From an objective scientific point of view I would have say no.

    There are factors which are uncontrolled and/or untested, in my opinion they are sufficently small that they should not have any impact however this has not been proven and so constitutes an invalid assumption or at least a qualification on the final results.

    For even longer yes/no see my comments here(289/290).

  305. Ben Goldacre said,

    February 11, 2006 at 1:38 am

    Ok kids, good to get some issues ironed out, but we’d better get cracking on this now. Just a few practical things to be sorted:

    1. i’m going to build a webpage with links to the files and email links to vote with. if anybody thinks they ought to program a more complicated voting system then do speak now.

    2. for music, we’ve got: some stuff from the label NAIM coming on cd (i think); probably some asian electronic stuff; and some contemporary classical. if anyone can put something big glamorous and commercial in my hand with proof that i can use it without being sued then do so quickly, it’ll be much better.

    3. has anybody got a cd player which takes a kettle lead somewhere in central-ish london that i can borrow for a day?

  306. Briantist said,

    February 11, 2006 at 10:56 am

    Andy,

    I worked on GPS when it was still classified project back in the 1980s, so yes, I do know about high-end clocks.

    I’ve helped design and install digital and analogue TV and radio studios and used monitoring equipment over the whole digital TV network (for BBC, ITV companies, Sky, CNN, you get the idea) so yes, I do understand about the timing issues. And yes, I have had scopes out and looked at waveforms and measured all these things.

    And I’ve seen all sorts of audiovisual installations and seen many people’s reaction to them. I’ve listened to the opinion of many, many technical people in the business

    And I still listen to newly pressed records on a regular basis too.

    My illustration about building a simple quartz-based clock was that it costs only a few pennies, and you don’t even need a soldering iron. The point is that a £30 Freeview box is dealing with clock signals at bitrates that are way above that required for audio, so even a cheap bit of audio kit will be good enough, even for “Audiophiles here have claimed to be able to hear a difference in clock jitter in the range of picoseconds. ”

    This phrase makes as much sense as “Audiophiles here have claimed to be able to hear extra data bits not present in the original recording.”

    Is the test valid? Yes, in as much as any single experiment will prove anything.

    (When it does, it’s called democracy)

  307. Chris L said,

    February 11, 2006 at 8:16 pm

    Just for my own peace of mind, and possibly for the wider edification of Bad Science readers, I’ve done the experiment I outlined in posts 162 and 171 earlier this week. The results are available at www.oview.co.uk/badscience, but to summarise, ripping a CD multiple times results in exactly bit-for-bit identical files, regardless of power source or (within reason) disc surface contamination.

  308. steve mosby said,

    February 11, 2006 at 9:20 pm

    Re (307). So would there be any point in doing an extensive, elaborate listening test on those files? Is it possible they could sound different in any way that could be meaningful with regard to the effect of the original power lead?

  309. Chris L said,

    February 11, 2006 at 9:34 pm

    Steve,

    No, there wouldn’t.

    That was the question I asked, and suggested the likely answer to, in post 162. Clearly, there isn’t any point doing comparative listening tests on identical files – any differences listeners might hear will be entirely down to the quality of their own equipment, changes in their head position or attentiveness, or any number of other things, but cannot be due to differences in the files, because there aren’t any differences. Listening to one file and then an identical one is exactly equivalent to listening to the same file twice.

    I’m not sure if Ben’s experiment will deal with this. I strongly suspect that all three of Ben’s A/B/X files will be similarly identical. Adding a random period of silence to the beginning and/or end of each file would prevent them from showing up as equal using “comp”, but it would still be trivial to show that the non-silent bits are identical. The only other possibility would be to perform some subtle transformation on one of the files such as adding 1 to the value of every sample, but this would completely obscure the results of the experiment.

    As I’ve said somewhere higher up this thread, if the recorded files come out identical, that’s the end of the experiment – no need for a listening test.

  310. steve mosby said,

    February 11, 2006 at 11:40 pm

    Chris – I agree, and have done all along. I reckon that’s as far as the experiment needs to go. If the files are identical, the lead’s clearly had no effect. If there are differences, then further experiments could determine whether the lead caused them. Either way, I’m not sure what a listening test could really achieve.

  311. John Ruch said,

    February 12, 2006 at 6:15 am

    Quote:
    ‘Ben’s already stated what the manufacturer’s claims are in the first article:
    “Replacing the mains cable to any of your hi-fi or home cinema components means they perform better, giving you less distortion, clearer pictures, and more musicality, so that you enjoy your music and movies even more!”

    “The key to the success of the Russ Andrews mains cables is the unique Kimber cable weave. The woven mains cable has been proven to dramatically reduce sound-degrading radio frequency interference on the mains supply and to reject further pick up of RFI.” ‘

    This is all still generic and untestable. Nothing is quantified. “Better,” “less,” “clearer,” “more,” “dramatically reduce” and “reject further” are the adjectives/verbs. If quantifying them is left up to the testers, not those being tested–what, exactly, can possibly be proved?!

    To be completely plain: Better than what, less than what, clearer than what, more than what, dramatically reduce by how much exactly and reject further from what point?

    If the manufacturer will not say, then we will quibble about it ourselves endlessly–which is how the manufacturer has already won in the first place. Forcing the testers to actually invent the details of the claim is a classic looney trick, and this whole discussion looks like we’re falling for it. Lengthy discussions of technological standards and hi-fi inner workings do not, by themselves, constitute sound experimental design.

    This idea of just putting “data” online and letting people vote is meaningless subjectivity. It’s feeding what it’s supposedly debunking. There will be no way to tell what the results signify because you don’t know what you’re looking for (unless it’s something you quantified on your own). Most of these posts have been about inventing things to look for, or even (shudder) listen for. *The person who made the cables* must tell us that, in a quantified, testable way.

    You will have a test, but not a scientific test. The experiment does have the potential (though not at all guaranteed) to indicate whether the cables are doing something or not doing anything, but as currently constituted it will prove nothing and just generate still more arcane quibbling–including among skeptics, just you wait and see.

    At the very, very least–why in the heck are you using something as complex and taste-based as musical recordings, rather than a consistent single note that would be easy to observe and neutral in content?

  312. Rowan said,

    February 12, 2006 at 11:41 am

    Somewhat relevant to the current discussion: Freesound is a source of Creative Commons licensed samples, especially intended for use by researchers.

  313. Graham said,

    February 14, 2006 at 10:03 am

    I agree with Mike at 301. Are the gold plated cables better than the thin bits of wire held into the socket with matchsticks in the sense that they are more like a live performance? All the experiments can show is whether or not there is a difference in sound quality, not whether that sound is “better” unless you have some agreed “criteria of quality” which the listeners score. Listening to HiFi is a self-contained activity, with an ambiguous relationship to live music. And even the quality of live performance is affected by the acoustics of the hall and where you are sitting.

    Maybe HiFi is more like LitCrit than Physics.

  314. Steve Phelps said,

    February 14, 2006 at 12:42 pm

    Chris L:

    Surely it would be interesting to test the effect that the lead has on the CD-player’s DAC. If you ommit this from the test by ripping directly from CD, then a reasonable objection to the experiment would be that you have only tested a small subset of the electronic components that go into the sound-production and that a power-lead might (ok- barely) conceivable have an effect on.

    >any differences listeners might hear will be entirely down to the quality of their own >equipment, changes in their head position or attentiveness, or any number of other >things

    This is easily dealt with by random sampling. However, this is impossible to do with anonymous online voting. An online voting experiment will be seriously flawed because you would have no control over the population of listeners, their equipment, etc.. and moreover you would not even have statistics on these variables. It would be a “bad science” experiment.

    The experiment should be conducted in controlled conditions. If you want to test the hypothesis that audiophiles can tell the difference as opposed to the general public then you need to control the sample of listeners and conduct an objective calibarion test of their hearing prior to the experiment. If you want to control for factors like the fact that the lead might have varying effect on different equipment then you need to repeat the experiment with different samples of equipment. Just because the manufacterer says it will _work_ with _any_ hifi equipment does not mean that we can be any less dilligent here either- as we would not then be taking into account that just because the lead _works_ we could still get different magnitude of effect with different equipment some which will be a lot harder to detect than others. This was a serious flaw in the wine-tasting experiment reported on in one of Ben’s earlier columns- they didn’t control for varying effect with different subjects — the wine was a subject of the experiment just as much as the taster. In this case the hifi equipment and listening environment are variables that need to be controlled- and if they can’t be explicitly or meaninfully controllled then we need to introduce random sampling across different pieces of kit and different listening environments to ensure we have meaninful results.

  315. Chris L said,

    February 14, 2006 at 1:08 pm

    Steve Phelps (#314),

    I can only encourage you to re-read the original article and all the above posts – especially the one from Ben which begins “holy fucking fuck…” etc. This whole experiment was specifically proposed to test the manufacturer’s claim that their special cable would have an effect on the digital outputs of a CD player. By making this claim, they expose themselves to the possibility of an objective test. In testing that specific claim, it is not only legitimate but necessary to omit the DAC from the test.

    At the high end of the hi-fi market, CD players don’t come in a single box. One buys a CD transport, which doesn’t contain a DAC and only has digital outputs, and a separate DAC, which converts the digital signal to analog. The manfacturer’s claim is a transparent ploy to convince audiophiles that, if they want the best sound out of their system, they’ll have to buy a special power lead for the CD transport, as well as every other component.

    Fortunately, this claim is immune from all the subjective problems that your post mentions. This claim can be tested entirely in the digital domain, and answered with a completely objective yes/no result – did the special lead make a difference to the digital output of the CD transport, or not?

    That’s what this entire thread is about. Other tests would indeed be interesting, but they’re not relevant to Ben’s proposed experiment, and would be, as you say, much harder to set up, control and interpret.

    If and when it is established that the special lead makes absolutely no difference to the digital outputs of a CD transport, therefore removing any possibility that it can make any difference to the analog output of a re-clocking DAC, then we can ask the manufacturer what tests they ran, and what their results were.

  316. Ben Goldacre said,

    February 14, 2006 at 1:16 pm

    i’ve basically come to a full stop on this: none of my mates i’ve asked have got cd players which take kettle leads, so i haven’t got a cd player to test it on. anyone in north/central london who wants to help out welcome.

  317. Chris L said,

    February 14, 2006 at 1:34 pm

    Ben,

    You don’t think Russ Andrews will let you avoid spending £1,800 that easily, do you? :-) From www.russandrews.com, choose “Russ Andrews Mains Leads” from the “Mains” menu and check the Q&A panel on the left. Apparently, it’s ok to chop the current crappy power lead off your CD player and wire on a female IEC socket, so that you can then plug in your magin power lead. Female IEC sockets are available from all good hi-fi stores, B&Q, Homebase, etc.

    The two or three inches of crappy power lead that are left won’t affect the sound quality because…. errmmm… hang on, let me get back to you on that one…

    ..oh yeah. Quantum mechanics. That’s it.

  318. Steve Phelps said,

    February 14, 2006 at 4:02 pm

    “This whole experiment was specifically proposed to test the manufacturer’s claim that their special cable would have an effect on the digital outputs of a CD player.”

    IMHO testing this hypothesis would not be satisfactory to refute the manufacter’s broad claim that installing this lead will improve sound quality. If I were the manufacturer then I would simply turn around and say that although the effect on digital outputs is very hard to detect, it has much greater effect on other components and will still improve overall sound quality. I don’t believe that this is actually the case- I just don’t think testing this hypothesis is sufficient to refute the manufacturer’s claims.

  319. GWO said,

    February 14, 2006 at 4:10 pm

    testing this hypothesis would not be satisfactory to refute the manufacter’s broad claim that installing this lead will improve sound quality.

    True, but it would invalidate that the mechanism claimed by the manufacturer to improve sound. Which means, any improvement would be due to luck.

  320. Steve Phelps said,

    February 14, 2006 at 4:14 pm

    Invalidating the claimed mechanism doesn’t invalidate the effect!

    Just because light propogates as a wave-function doesn’t invalidate measurements of the speed-of-light made by people who thought that light was strictly a particle…

  321. Steve Phelps said,

    February 14, 2006 at 4:21 pm

    Ultimately I guess this is all futile though, because people with the audiophile/newage mindset will always retreat into claiming a hypothesis such as:

    “There is an objective effect that only manifests itself in the absence of a double-blind trial”

    Note that this is untestable and hence unrefutable. But just because it is untestable doesn’t mean that it is false. You can call it “pseudo-science” a la Popper- but name-calling isn’t going to change these peoples’ minds- (and remember that there are plenty of irrefutable statements in science that are perfectable sensible- I can see this sparking a flamewar though).

    Empirical evidence — or lack of — is never going to convince these people.

  322. Chris L said,

    February 14, 2006 at 4:52 pm

    “the effect on digital outputs is very hard to detect”

    The point is that this argument is technically invalid. Effects on digital outputs aren’t hard to detect. They’re very easy to detect. As has been discussed to death above, once you show that the stream of 1s and 0s is the same, it’s only things like jitter *at the DAC stage* that can make a difference.

    In any event, I completely agree with you that we will never change an audiophile’s mind with a test like this. The aim is always simply to shine a light on the truth, and try to prevent any fence-sitters from falling on the wrong side.

  323. Steve Phelps said,

    February 14, 2006 at 5:15 pm

    ok – let me rephrase that – how about “We find your results highly interesting. However, in the light of our own subsequent tests, the effect on digital outputs is relatively weak compared to the effect on other outputs, and can also vary signficantly from player to player, and also depends on the amount of interference already present in an installation, and so can vary from customer customer. Nevertheless we acknowledge that the effect on digital outputs is not as great as on other components, and therefore we now recommend to our customers that they prioritise other components such as the DAC, when upgrading their system.”

    Such a claim would be testable- but a lot harder to do so.

    I guess it makes sense to take things a step at a time though.

  324. Max said,

    February 14, 2006 at 6:52 pm

    Ok. So how about we rephrase the question. If we know what bits we lay down on a cd (by using a .bin and .cue file to burn the cd, which will introduce some FEC redundant data) we know exactly what the bitstream output of the cd should be (once error correction has occurred). Forgive me – I don’t know red/orange book all that well, so I don’t know whether the FEC gets passed through in the bitstream, but you get my jist.

    It would be sufficient to show that a CD player with any chosen lead yields back a bit for bit copy of the original to show that it would be impossible for any cable to have an effect (whichever one Ben chooses). Even if there is jitter in the signal, we will have demonstrated that that jitter has no effect in the data being passed to the amplifier (where of course our debate can start all again) which will mean that whether or not it is there, it will necessarily be inaudible (assuming the DAC in the amplifier is independently clocked).

  325. Max said,

    February 14, 2006 at 7:10 pm

    Further to that, as someone who is currently purchasing various bits of hifi kit and trying to discern the nonsense from the audible, is Ben aware of the claims made by e.g. What Hifi, who swear blind that separates stands make a noticeable difference? Whilst I can understand that speaker stands might (what with all the vibration), although I have never tested that so can’t comment, I refuse to believe (although I’d love to be proved wrong) that it could make an audible difference (as is claimed) to an amp or cd player (although audible differences on a record player are much more plausible) whether it is sitting on a shelf or a beautiful glass number.
    The claim that expensive vs cheap optical/coaxial digital cables could make a difference likewise seems nonsense to me (and apparently you should use coax over shorter differences to avoid the distortion caused by the signal being converted to light through the cable), but the one that really amuses me is the extremely expensive HDMI/DVI cables being offered (digital video).
    The fact that most of us spend half our day sitting in front of a hi-res monitor with an el cheapo DVI cable, sitting far closer than we ever do to a TV and have no complaints and yet suddenly notice the difference when sitting 5 times further away when using a TV seems most implausible of all.
    While on the subject, I’d be interested to know the differences in optical/hdmi output between cheapo and expensive cd/dvd players (which magazines swear by), but given the can of worms you’ve opened here, I think that might take some time…

  326. Rachel said,

    February 15, 2006 at 4:25 pm

    Dear All
    Am I missing something here? I have read the original article and lots of your terribly facsinating posts and still will not be calling in my kids for supper tonight using a dog whistle because they won’t bloody hear it!!! I shall use a bell and no-one will go hungry. I want to take part in the experiment at the “lower” end of the spectrum because I can’t even see the differnce in TV picture quality between the image received down a cable and that received via a digibox (and I have got mighty fine eyesight – 20-20 and all that). If my nearest and dearest spend a small fortune on some nobby cable so that they can imagine an infinitesimally small improvement in something that was perfectly fine anyway, someone else around here won’t be able to spend a relatively small amount of money on a piece of beautiful artwork for our hall which everyone will notice and say “Wow! That’s new and makes a huge difference to what were, previously, completely bare walls!” I look forward with eagerness to the post that seeks to convince me that buying a dog whistle is a good buy as a kid-caller because, although no-one will hear it, the noise really is there.

  327. Drew Stephenson said,

    February 15, 2006 at 9:10 pm

    Rachel,
    I’m not going to advocate the dog whistle but it’s all relative; my landlady had a new fireplace put in and i didn’t notice for a week, i guess it just comes down to what’s important to you but don’t expect ‘everyone’ to notice a new picture on the wall… :¬)
    P.S. i’m not going to spend £75 on a kettle lead either

  328. Paul Richmond said,

    February 17, 2006 at 11:53 am

    Rachel,

    If you use a dog-whistle to call your kids to dinner, at least you will be able to use the well-worn line “Your dinner’s in the dog”.

    Ben,

    How about testing the Russell Andrews kettle-lead to see if it can be used to make better tea or coffee ?

  329. Steve said,

    February 20, 2006 at 8:16 pm

    It seems like the general concensus is that HI-FI users are idiots who buy anything if they are told it will increase performance ?

    I thought hearing acuity was varied , and would have guessed that someone who spends a lot of time listening to music would be able to tell if there was a difference, more so than someone who spends all day with an mp3 player ?

    I suppose expensive audio is no better than a cheap midi system either, and that all CD players sound the same ? and if the signal is the same when it reaches the amp then all amps would sound the same ? and all amps draw the same current from the mains, and as speakers are driven by watts and drive units operate in basic terms by vibrating , then there can be no difference between them etc etc ?

    Regarding optic cable, I thought RCA could transfer more data than optical ?
    And an RCA connector is not all that good at preserving a true 75ohm signal.
    And most interference is generated from the mix of mains, interconnect and electrical equipment in close proximity to each other ?

    I would also have thought not all people into music would want to spend money on cable when they could buy more music, yet some feel the cables make a big enough difference to take them home and demo them, and choose one after listening buying what they thought sounded best.

    Its also interesting the amount of claims made about why cables should not make a difference, yet no one questions the acuity of the measuring equipment or can make there mind up on what it is about recorded music that means so much to some.

    If CD was so good why are manufacturers still trying to better it, it does not carry more information or recreate music more than analogue, and due to the nature of digital and jitter, are you sure every time you hear a CD it has the exact same errors as the first time ? or could it be that the measurements and engineering we are applying is too crude compaired to human hearing ?

  330. Ben Goldacre said,

    February 20, 2006 at 8:34 pm

    so, does anybody have a cd player that takes a kettle lead, or some license free music, in london?

  331. Tommy said,

    February 21, 2006 at 9:12 pm

    I have a CD player that takes a kettle lead, and a decent Hi-Fi too (a Quad system). I don’t have any license free music though…..

    I’m in Barons Court in London

  332. El_Cid said,

    February 24, 2006 at 12:56 pm

    As a person who is involved in selling high-end video and audio equipment, I’m personally very annoyed by many of these nonsense claims by accessory manufacturers.

    In practical terms I have learned that at the very highest end (Transparent Cable company sells a pair — One Pair — of speaker wires for $30,000 US) the point really is simply to allow very wealthy people to completely distinguish and dissociate their products and systems from the commoners.

    E.g., why go out and buy a $10,000 DVD player when you’re just going to hook it up with the same cables that a commoner could buy?

    But on the serious side of testing:

    (1) Many arguments by the “tweaker” community (an industry term for those people who think they can milk out more performance not through better actual main products but using accessory / pseudoscientific “tweaks”) say that these special power cables and wires and what not help bring out “more” sound from the recordings they are listening to.

    (2) Recordings are produced in studios using certain equipment.

    (3) The goal of a musical recording is to capture and mix / produce as well as possible the desired sound of the studio engineers and artists.

    (4) If your equipment changes, rather than reproduces their originally captured and produced recording, then you have ruined the product.

    (5) So by their very arguments, all these tweaking solutions are not better reproducing the original intent of the artists, instead (if anything at all is changed) they are creating something new and different than the recording actually purchased.

    (6) If this is the case, then the tweakers need to stop buying pre-recorded CD’s, LP’s, DVD’s, and SACD’s, and go to small live shows, and then tell the artists and performers that the sound is wrong and bring along lots of little gadgets to hang about the performance space to make the artists “sound better.”

    – PS: If you’re aiming for actual fidelity, bear this in mind: the studio engineers who produce the recordings to which you (and the tweakers) listen do not use $30,000 speaker cables or $3,000 power leads. They might use $100,000 mixing boards, but these actually do something and have the computing power needed to produce today’s mixes, but they connect their $100,000 mixer to a $800 speaker using a $20 speaker cable.

  333. Mick James said,

    February 27, 2006 at 9:08 am

    Digital has posed a huge problem for the high-end audio industry, as the components are clearly on the same price/performance curve as the rest of the electronics industry. So as long as they get decent speakers, and to some extent I suppose cables, then the audio quality experienced by the proles must be getting better and cheaper very fast indeed. As someone who spent £280 in 1979 on a crappy music centre, I wonder what an inflation-adjusted sum would buy me know, and how that would compare to the very best equipment available 27 years ago?

    What intrigues me is that (again, with reservations over speakers) the audio quality coming from from a reasonable component system or even a midi system may now have surpassed that from audiophile kit (look at what happens to Bose prices while everything else gets cheaper). Imagine how a maker of handmade valve-driven computers in cherrywoood boxes would have fared against Dell in the last decade. (but I can hear it now…the 0’s are more vacuous, the 1’s are more unified….).

    Somewhere I have a little book of audiophile nonsense from the analogue era which I’ll try to dig out. Highlights I recall are: disconnecting all the lights on your amplifier, only having even numbers of LPs in your room and using channels of mercury to connect your speakers. There was even one audio guru who claimed to be able to tell which channel your television was tuned to–even when it was switched off–just by listening to your hi-fi.

  334. John Manderson said,

    February 27, 2006 at 8:41 pm

    The ear is very easily fooled into thinking it hears some thing. The following story illustrates this.
    I used to to a bit of sound engineering for a few friends who had a group, I hasten to add that I do not consider myself as a sound engineer, jsust some one who roughly knows which knobs to twiddle.
    During the sound check the guitarist decided he wanted his guitar a bit ‘brighter’ so I gave him a bit more top, he decided that that wassn’t quite enough so I gave him the full monty, we both thought it was so bright that it hurt, so I backed it off and he agreed that the sound was what he wanted. No problems, except that a coupleof minutes later I realised that I hadn’t increased the brightness on his channel, but on the unused channel next to his.
    If you think that an expensive piece of kit will sound better than a cheap piece of kit then it probably will, provided you ‘know’ that you are listening to the expensive kit.

  335. Ben Goldacre said,

    March 22, 2006 at 11:05 pm

    sorry i’ve stalled on this, snowed under and couldnt get a cd player which took kettle lead, will do in next few weeks, in the mean time:

    www.ilikejam.dsl.pipex.com/audiophile.htm

    www.positive-feedback.com/Issue21/audioramblings1.htm

  336. Mike said,

    April 18, 2006 at 11:04 pm

    Why don’t you just use your computer as the CD player (I assume your soundcard can output PCM)? Google “bit-perfect”, and you’ll see that it’s not an uncommon thing to have a soundcard (the $20 Chaintech AV710 comes to mind) that outputs bit-perfect PCM signals. The bonus is that if you run the test with the cheap cable first, you don’t even need to get a expensive mains lead, since you can’t possibly get better than bit-perfect (given the procedure of the “experiment”). Hmm… it looks like you’ve already acquired one, though.

  337. Mike said,

    April 18, 2006 at 11:10 pm

    Oh … I’m assuming that these “kettle leads” fit into the same power jacks on computers. In my limited experience, both computers and audio gear use IEC inlets; it didn’t occur to me that it might not be the case.

  338. Nick Grant said,

    September 30, 2006 at 9:47 am

    Have really enjoyed skimming through this and following up some links but didn’t spot any mention of Backmasking which gives a very very powerful demonstration of the power of suggestion over what we hear. Good way to prove that the Voodoo people are right, blind listening doesnt work!!! You need to know what you will hear then it works.

    jeffmilner.com/backmasking.htm

    Is a good start and you can make your own experiments in minutes without expensive leads etc just by playing tracks backwards to people with and without the words showing. Stairway to Heaven backwards is the classic! I saw Simon Singh do this at Hay Festival. Played it backwards, asked if anyone could hear anything, no, anyone hear the word Satan? Couple of hands in big audience went up. Then played it again with words on the screen, wow! Just try it.

    Most web threads argue whether it was Led Zep or the Devil himself that did this, given the limited technology at the time :-). One lecturer at Leeds (from memory) tried it on his students but with different words.

    I have tried it on an opera singer and ‘tone deaf ‘ people with 100% result, no stats analysis needed with this experiment.

    My suggestion is that Hi-Fi shops should have a sign saying ‘this sounds great’ visible when demonstrating expensive kettle leads and oxygen free sexed cables, or they could do as now and just tell people….

    Problem I have is I want to buy a new amp and speakers, how do I avoid self and salesperson deception.

  339. Home Cinema Speaker Wire !! - Page 2 - Alfa Romeo Forum said,

    February 11, 2008 at 7:52 pm

    […] Originally Posted by Nev If you like spending money for no reason, there’s plenty of expensive pseudo-scientific hi-fi carp out there for you. Bad Science Voodoo HiFi Experimental Design Masterclass […]

  340. jiangjiang said,

    December 8, 2009 at 2:10 am

    ed hardy ed hardy
    ed hardy clothing ed hardy clothing
    ed hardy shop ed hardy shop
    christian audigier christian audigier
    ed hardy cheap ed hardy cheap
    ed hardy outlet ed hardy outlet
    ed hardy sale ed hardy sale
    ed hardy store ed hardy store
    ed hardy mens ed hardy mens
    ed hardy womens ed hardy womens
    ed hardy kids ed hardy kids ed hardy kids

  341. EV Speakers: Optimum Audio Quality with EV Powered Speakers said,

    October 19, 2011 at 5:04 am

    […] EV Speakers Selections: The balance and quality of sound from EV speakers are important especially when large spaces are con…kers generally employ transducers that are referred to as drivers. The latter is responsible for […]