Drawing Conclusions…

The point here is that even apparently simple ‘facts’ are surprisingly variable and their value needs to be considered and understood in a far wider context. If there are that many variables in trying to measure how far a speaker sits from the rear wall, how many do you think apply to trying to compare and describe two different products?

Recently, I have actually had a couple of opportunities to make just such a comparison in what I’d consider a meaningful enough scenario to make it worthwhile. Not long ago, I was able to compare the SUPATRAC Night Hawk and Kuzma 4Point 14 tonearms. It was possible because of the availability of two (nominally) identical cartridges and a turntable (the Grand Prix Audio Monaco v3.0) that actually allowed rapid substitution of complete tonearm/armboard assemblies (with mechanical location and all alignment parameters maintained intact). How rare is that? But tonearms are something of a special case. A second, in many ways more representative example occurred just the other day…

The arrival of the Wadax Studio Player has occasioned considerable interest. Promising Wadax performance at something approaching a real-world price, while also embracing the move towards more integrated, multi-functional components, it is a product that begs an awful lot of questions, not least of which are, what exactly is the Wadax sound and how exactly does the Studio Player fit into the wider market context? With that in mind and as an essential part of the ongoing review process, I set about comparing the (not so) compact Wadax to a known benchmark, in this case the CH Precision D1.5. In one sense, this is a direct comparison of two CD/SACD players at near identical prices. In another, I’m not actually comparing like with like: the D1.5 is a straight player, whereas the Wadax offers both a streaming facility and a variable output option, allowing you to connect it directly to a power amp, if the mood takes you or circumstances dictate. So that’s your first big variation (or limitation): customer expectation and application. Even so, the disc replay comparison is still worthwhile, in terms of establishing where the Wadax sits in the great scheme of things, if only because the D1.5 is currently the class of the field.

On the surface, you might think that this should be a pretty straightforward exercise. Connect both units to the same system, play the same tracks and compare away. But before you start, you need to first ask yourself what you are trying to compare. The classic review approach would be to listen to one unit, substitute the second and listen again. Alternatively, you could add the second unit to the existing system and listen to them side-by-side. This latter route has become the dominant methodology, but it is riddled with potential pitfalls.

“A fact is like a half-truth; you must see it whole to appreciate its value.” Jonathon Green

The most serious conceptual problem lies in the ‘change one thing’ mentality that has become so dominant in the reviewing world. The logic is impeccable: if you only change one thing, then any variation in the sonic and musical results must be the result of that change. The problem lies in trying to only change one thing… Here are just a few of the issues with this approach:

Even if you connect the two units with nominally identical cables (same type, same length) there’s no guarantee that the various cables will be the same age, or have the same degree of usage/burn-in.

Even if you connect the two units with nominally identical cables to two different inputs, those two inputs are unlikely to sound identical. Amount of use and location within the circuit can both impact performance. Likewise, different power sockets do sound different, depending on the topology of the AC supply and grounding arrangements.

Simply treating the two units identically rarely works either. Do that and you discover what each unit sounds like under a very specific set of circumstances, limited to that one system alone. What you actually want to do is discover how the two units compare, with each sounding at its best.

Different shelves sound different, as do different shelf positions in the same rack. Put one unit on the top shelf of a four-shelf rack and the other on the shelf below. Listen to the two and then reverse the positions. You might be surprised by the magnitude of the difference in relative performance.

Do the same thing in the same way, every time. That means, operate the equipment manually (remotes don’t just impact sound quality, that impact varies both between instances of the same operation and between the same operation – eg. track select – depending on how it is executed, using the numerical keypad or counting through the tracks.)

Put that lot together and it soon becomes clear not only how hard it is to establish a level playing field, but how those situational differences can collapse or even eliminate performance differences between different products.