In the headlines today, Consumer Reports continues to explore the merits of ghetto coffee: Lack of excellent coffee blends: Consumer Reports | Reuters. Whereas before they chimed in on the McDonald’s vs. Starbucks debate — something we’ve always likened to a beauty contest between Courtney Love and Joan Rivers — this time their “expert” taste buds were disappointed by 37 different coffee blends available at major supermarket chains.

To confuse the story further, Reuters' article shows Turkish coffee from IstanbulOf course, we’re writing this post while sipping a press pot of some freshly ground El Salvador Nueva Granada from Barefoot Coffee Roasters. (Mmmmm.) But this is the same Consumer Reports that lavished untimely praise on Toyota last month.

Our biggest contention with Consumer Reports is that, in recent years, they have over-extended themselves from objective reviewers of consumer appliances towards more subjective arbiters of public taste. It’s one thing to judge a minivan by objectively measurable criteria such as noise levels, cabin space, engine pickup, and fuel economy. It’s an entirely different thing where, to quote the press release:

“Consumer Reports has a rating criteria in which the tasters look for specific characteristics including the flavor and aroma.”

Consumer Reports established itself on unbiased, objective reviews of vacuum cleaners, washing machines, and even home espresso machines. But lately they have been trying to become public taste-makers for coffee. This shift towards subjective analysis calls their credentials into question — particularly since we’ve found a number of dubious conclusions from their previous taste tests. It also makes us question what’s next: wine? Restaurants? Single-malt Scotch?

How is Consumer Reports any different than

Do we claim to be any more qualified as arbiters of coffee taste? Absolutely not, but that’s kind of the issue. What best appeals to your taste buds or our taste buds does not follow the same kind of analysis that you’d give a child’s car seat.

So who makes the taste judgments, and how they make them, become absolutely critical. Transparency is essential, as this is the know-your-coffee-reviewers problem we wrote about three years ago. And Consumer Reports‘ model for expanding into coffee reviews — which is indistinguishable from their legacy of reviewing dishwashers — offers none of that. It’s one thing to recommend a cordless phone for its range and battery life, but it’s an entirely different thing to recommend one to eat for dinner.

What additionally concerns us is a kind of blind (and undeserved) reverence bestowed on Consumer Reports by most media outlets and consumers — many who seem blissfully unaware of their transition from objective review criteria to subjective taste-making. From another take on this survey (Why America Can’t Get a Great Cup of Coffee – DailyFinance), we also learn:

Tasters looking for “smoothness and complexity, with no off-flavors” and beans “neither under-roasted nor charred” and, of all things, “subtle top notes” were left wanting.

Sweet Maria's Moka Kadir blend evaluated on a tasting card/spider graphTo be useful to consumers on subjective criteria, Consumer Reports must frame their standards of coffee tasting to a profile to which we can each relate. What’s written above is perhaps better than no information at all. But reading this, we’ll be damned if we can figure out how our own taste preferences compare to theirs. Who actively seeks out under-roasted, charred, or off-flavored coffee? This doesn’t describe coffee profiling so much as defect-finding, making Consumer Reports less coffee tasters and more meat inspectors.

Furthermore, Consumer Reports has provided no information about their methodology and standards for evaluation. The freshness of their supplies, how they prepare their coffee, how many samples they try at a given seating — all of these factors can make a huge difference in any side-by-side comparison. (UPDATE: We at least learned they do a swish-and-spit.)

We love this stock photo from Consumer Reports because it has everything to do with how the coffee is preparedOf course, in today’s quality coffee world where purveyors and consumers are obsessing over single origin roasts, a survey of supermarket coffee blends seems about as retro and down-market as tuna casserole. Not that there isn’t an audience for down-market coffee reviews. After all, there are people who think of Joan Rivers and Courtney Love as beauty queens. We just ask that if you do these reviews, please bother to do them properly.

And with that, we leave you with one of our favorite quotes about public tastes for coffee:

If I asked all of you, for example, in this room, what you want in a coffee, you know what you’d say? Every one of you would say “I want a dark, rich, hearty roast.” It’s what people always say when you ask them what they want in a coffee. What do you like? Dark, rich, hearty roast! What percentage of you actually like a dark, rich, hearty roast? According to Howard [Moskowitz], somewhere between 25 and 27 percent of you. Most of you like milky, weak coffee. But you will never, ever say to someone who asks you what you want — that “I want a milky, weak coffee.”

— Malcolm Gladwell [video: start at 10:45]