A friend and CoffeeRatings.com reader, Jeremy Nisen, occasionally writes for SFist — which many rightfully recognize as San Francisco’s best local blog. (Jeremy interviewed me for it earlier this year.) Yesterday he pointed me to his most recent article on the site: SFist: Um, Yeah, But What Kind of Grinder Did They Use?.

In it, Jeremy takes issue with a recent Los Angeles Times article (French roast brews, sip for sip – Los Angeles Times) where a panel of coffee tasters reviewed the French roasts of several different roasters. True to an SFist’s core, Jeremy does his best “L.A. sucks” rebuttal when he finds them guilty of talking smack about Graffeo beans. But more to the point, he raises all sorts of legit questions about the validity of their side-by-side comparison taste test to begin with.

Certainly, the L.A. Times was just innocently cranking out a nice, forgettable fluff piece that perhaps influenced a few readers to try out different roasters — and undoubtedly the staff are now all off to thinking about who has the best mojito recipe. But in a taste-making media world now dominated by celebrity chefs who never cook, knowing the quality of the advice you’re getting is at least as important as knowing the rancher who raised your Kobe beef.

The Airing of Grievances

Issue #1: Food editors, like most restaurant chefs, couldn’t tell a good cup of coffee if it scalded their thighs in a McDonald’s drive-thru. They’ve made it their business to know everything about food, but often that has translated into a misguided belief that this somehow confers upon them de facto coffee taster certification. Which partly explains why restaurants notoriously serve some of the worst espresso in any city I visit. And as Jeremy put it, “After all, how often do you see a restaurant review talk about the end-of-the-meal cuppa joe?”

Issue #2: Thirteen cups of coffee at one sitting is bound to make anybody think, “we were struck by how similar so many of them were.” I guarantee you that if you smack your forehead hard enough on your kitchen table a dozen times, that thirteenth smack will feel less painful than the first. When I reviewed espresso for CoffeeRatings.com, four was my limit at any one time (caffeine-induced hallucinations aside). Even at coffee cuppings, my tastebuds go numb before reaching double digits.

Issue #3: How fresh is the coffee? This is a huge determinant to how good any coffee will turn out. If you bought resold Blue Bottle Beans hidden behind the two-year-old Pop Tarts at the corner convenience store, they may taste a lot worse than Costco’s ashy Mt. St. Helens Pyroclastic Flow blend that you happen to catch a mere three-weeks-old that day.

The point of all this is that methodology counts immensely when making any legitimate comparison. While I do not claim to be a professional coffee tastemaster by any means, it is for this reason that I have posted a Tasting Methodology link prominently on the site from day one. If you really want to know methodology, the guide Espresso Italiano Tasting is a good resource for “rules of engagement”.