As we mentioned in our last post a couple days ago, being popular and being good can be two very things. There are few places where this is more apparent than on Web sites that thrive on aggregate public user ratings and reviews. We had intended this article to be about Yelp’s top SF coffee picks — much like our annual round-up on the Best of CitySearch. However, our analysis of Yelp’s rankings became so absurdist so quickly, our story changed dramatically from what we originally intended.

Yelp not only failed at the fundamental task of telling us what other San Franciscans thought was the best coffee in town, in aggregate, but it took us down a dark and strange path into the social motivators behind the site. In essence, we found that Yelp’s rankings are a better measure of a social game among its users than as a measure of the establishments its users supposedly rate.

It’s been a good year since we last poked Yelp in the eye over its major flaws. So this time around we will take a critical look at Yelp’s actual rankings for top coffee in San Francisco based on their rather mad methods.

Arbitrary Standards

Many of you probably know that Internet users were TIME magazine’s choice for the 2006 Person of the Year. You might even be familiar with James Surowiecki’s book, Wisdom of the Crowds, which illustrates many ways in which the collective intelligence of large groups of people is greater than that of individuals. All of this is certainly wishful thinking in an election year. However, any “wisdom of crowds” has also provided us with mob behavior, witch hunts, and mass hysteria.

Fortunately, poorly biased coffee ratings should not inflict any bodily harm on espresso lovers who have been lead astray. Or at least in theory — as there have been times we’ve sampled some pretty awful espresso and still wonder about the long-term health effects. But sites such as Yelp are often utterly useless to us because the rating criteria are often completely arbitrary.

To help illustrate this point, we present a summary of Yelp’s Top 20 SF Coffee & Tea establishments as of June 5, 2008, as determined by Yelp users. (Given that the review rankings are dynamically affected by new user ratings, and that we had something of a Web hosting meltdown since the time we started penning this post, the ranks have changed a little since last month. They are also likely to change after this article is published.)

Next to each ranked establishment, we’ve listed their equivalent rank on CoffeeRatings.com (many of which are tied with others for the same ranking), real quotes from reviewers who gave the establishment a maximum rating of five stars, and our associated thoughts on the reviewers and/or the reviewed. It’s the kind of stuff that makes us think they should change their name from “Yelp” to “Wince”.

Name Yelp’s 2008 rank Our 2008 rank Yes, a 5-star Yelp reviewer really wrote this Our thoughts
Graffeo Coffee Roasting Company 1 N/A N/A Good local roaster. But they don’t do retail coffee.
Bernie’s 2 18 “I personally don’t drink coffee and generally don’t like the smell of it, but I love the aromas emanating from Bernie’s coffee shop.” Amazing how much money you can save when you don’t actually drink any of the stuff.
Trouble Coffee Company 3 7 “Blue Bottle is delicious but I don’t own the appropriate attire to hang out there.” Ball gowns, white gloves, and tiaras are so hard to come by at thrift stores these days.
Double Team Coffee 4 396 “What more could you ask for than a cute little asian dude yelling ‘strooooooong coooofffe’ while he’s completely jacked out his head on his product.” Well, you could ask for coffee that doesn’t rate in the bottom third of the entire city.
L’s Caffé 5 453 “It depends on how YOU like your coffee, but personally I drink coffee because it’s warm and it’s a vehicle for milk.” Which is why personally we drink beer because it’s foamy and it’s a vehicle for drunken, anonymous sex.
Leland Tea Company 6 N/A N/A It was the great President Lincoln who once said, “If this is coffee, please bring me some tea; but if this is tea, please bring me some coffee.”
MotoJava 7 294 “I haven’t tried the coffee yet, but the sandwich I had was excellent, as was the service.” The French Laundry: five stars. We didn’t try any of the food, but the coat check was fantastic.
Cafe Murano 8 240 “They serve great food (Sorry i don’t know about coffee, i don’t drink it)” And we love the softness of their toilet tissue.
Cento 9 6 Two-fer: “Hooray for yet another coffee shop serving good drinks out of a loading bay!” and “I haven’t even been there yet, but am certain it will be fabulous” If we actually stepped over the heroin-addled hookers to get to their loading bay and taste their coffee, we’re certain it would be some of the best in the city.
Danilo Bakery’s BaoNecci 10 159 “A very handsome young man behind the counter flirted with me (as only a European man can do!) and I found they are making the delicious corn bread loaves on Saturday again!” And I’ll give you six stars if you give me your phone number in an accent of whatever language they must speak in Italy.

And the next 10…

Name Yelp’s 2008 rank Our 2008 rank Yes, a 5-star Yelp reviewer really wrote this Our thoughts
Spike’s Coffees & Teas 11 13 “I just started drinking coffee, so maybe I’m not an expert, but I think their coffee is one big cup of awesome!” As long as it’s not one big cup of ashy, watery, over-extracted dreck like most places pull.
Imperial Tea Court – CLOSED 12 N/A N/A Some places are worth more dead than alive.
Philz Coffee @ Castro, 4023 18th St. 13 N/A “Have you ever tasted coffee that didn’t taste like coffee at all?” Philz Coffee: for when you want to pretend you’re drinking coffee when you really aren’t.
Java Detour 14 140 “I wouldn’t say it’s the best coffee but dammit it is convenient to the 101 and the people who work there are so friendly.” When I’m fleeing the scene after committing a felony downtown, I prefer the fast, friendly service of Java Detour.
Lupicia Fresh Tea 15 N/A N/A No retail beverages and proud of it.
Coffee Adventures 16 301 “In all honesty, their ‘regular coffee’ is horrid lol BUT BUT BUT everything else I have had is great.” Speaking of Honest Abe Lincoln, “Our American Cousin” was an awesome play.
Faye’s Video & Espresso Bar 17 301 Two-fer: “THE best place to me in the City, to get great expresso [sic]. They also are always quite generous in their servings as well. [sick]” and “I love stopping by and getting my own movie. I get to look and not be so overwhelmed with technology. plus the cute boy behind the counter, i think it is Mike is cute!” THE best place in the city to go for over-extracted expresso while fawning over Mike, or whatever his name is. At least I think it’s a “he”.
Mamá Art Cafe 18 331 Two-fer: “Best chai I’ve ever had. Don’t know how authentic it is, but it’s spicy creamy goodness in a big, fat over-sized cup.” and “there are never any ‘surprises’ showing up in my food or drink which is always a major bonus.” Any place that doesn’t put live scorpions in my coffee gets five stars!
Jackson Place Cafe 19 65 “Two words: Secret Hideout” Two words: Honeycomb Hideout
Ritual Gardens 20 3 “the lines are much shorter here than on Valencia st, the hipsters not so hip and the coffee and espresso concoctions are just as tasty. I’d rather share my coffee with a cactus any day.” And as soon as I off this *&#$% barista and silence these voices in my head, I can have my frigging espresso in peace!

The Negative Review Head-scratchers

While the above covers the excitable, five-star reviews that contribute to sometimes puzzling accolades for otherwise weak espresso purveyors, there are equally puzzling criteria reviewers use to depress the rankings of otherwise great espresso options. Below are a few more of the more bizarre rankings on the list — optionally accompanied by some classic excerpts from the Yelpers themselves who gave the places one-star reviews (with a couple two-star reviews thrown in for good prose):

  • The Hayes Valley Blue Bottle was ranked #36 of 784, below The Castro Cheesery at #24, and just below the ever-disappointing 595 Mission St. SOMA Peet’s at #34. Overheard: “OVERRATED… this spot can be such a freaking scene, filled with thirty-year-olds holding little dogs on a leash, talking about internet gossip on their iphones.” (Apparently coffee quality is dependent upon age, dogs, and iPhones.) “I don’t like coffee, except for maybe an iced one on a hot day.” “I don’t know how anyone can drink coffee if the[y] say they believe in global warming.” (Note to Al Gore: Solve global warming by shutting down Blue Bottle.)
  • Theatre Too Cafe ranked #59 with an average 4.5-star rating (Overheard: “Tasty. Juicy. Ample. Cheap.”). Meanwhile, CoffeeRatings.com gave it a 1.80 espresso rating, ranking it 584th out of 594 — or the 6th worst in all San Francisco.
  • Blue Bottle Farmer’s Market at #63 of 784 ranked below both nearby purveyors, Frog Hollow Farm at #44 and the 2 Embarcadero Peet’s Coffeeat #60
  • Coffee Bar (tied for #1 on CoffeeRatings.com) at #74 ranked below a Mission Starbucks on 2727 Mariposa St at #71. Overheard: “I wasn’t exactly inspired to try it myself” (So I gave it a one-star rating anyway.) “The MUSIC is way too loud.” “Coffee Bar has been open for about two months, but they haven’t bothered to mention that small fact on their website which still says it’s ‘coming soon.’ What gives? … So you get one star until you get your act together.”
  • Blue Bottle Cafe (tied for #1 on CoffeeRatings.com) ranked at #114. Overheard: “I asked for it to go, they made it to stay. Then they insisted on throwing that away to make it right.” (Those bastards!) “Sure it’s organic, sure it’s shade grown, sure it’s fair trade. But you needlessly spend cash on expensive coffee; but by the same token you won’t give a homeless person change?” (Note to greenies: hardworking homeless people in Nicaragua don’t count.) “I’m really more of a tea guy, myself.”
  • Ritual Roasters, Mission ranked at #145, just below Java Beach Cafe at #143. Overheard: “If you knew me at all you know I don’t drink coffee…I only went into Ritual because I was walking by and I had to pee.” (There’s grounds for a one-star review right there.) “Pffff, how incredibly lame to cover up the plugs… Whose brilliant idea was that?” (Laptop owners speak out on coffee quality.) “In all honesty, I have not tried your coffee because the masses of bug eyed yups and students clinging to their laptops and I pods makes me want to vomit.”

I don’t know what kind of parallel, warped universe some Yelpers are living in. But one that ranks Theater Too Cafe at the 92nd percentile — well above Blue Bottle Cafe, Coffee Bar, and Ritual Roasters in the Mission — must contend with questions such as, “Mommy, why do you have three heads?”

The Yelp Rating Lifecycle

To be sure, there are a few ratings on Yelp that we’d actually agree with. But take Cento, for example. It was only open one month when it was ranked #9 on Yelp — and it had a ways to go before it left the “honeymoon” stage of the Yelp rating lifecycle.

What is the Yelp rating lifecycle, you ask? Typical for a place like Cento, wannabe hipsters race to be the first to review a place they like and pump it up — particularly if it’s obscure or relatively out of the way in some back alley. Apparently, from what we observed, they believe this behavior somehow bestows upon them some form of social currency among other Yelp posters. It’s the “I’m cool because I am an insider and I found it before any of you” badge.

But then as a place gets exposed and more people know about it, this is followed by a wave of professional killjoy reviewers that love nothing more than to say, “OVERRATED! — And they’re so arrogant here” as a form of their own social currency among Yelp posters. It’s the “I’m late, but I’m cool because my tastes are way better than this” badge. Mark our words. Give things a few months and Cento’s Yelp ranking will drop like Starbucks‘ stock price.

Interestingly enough, since we originally wrote this paragraph in June, Cento’s Yelp ranking has dropped from #9 and is now currently hovering around #24. While our repeat visits haven’t produced espresso ratings as strong as our first, we sense there’s more going on than just a subtle weakening of their espresso.

Cento's early stages on the Yelp Rating Lifecycle

So what does any of this have to do with the rating of quality coffee, you ask? Absolutely nothing. But it has everything to do with how Yelp works. Which is precisely our point. Using a site like Yelp only makes us feel dumber than we already are.