One of the things that the big coffee chains do well is consistency. Or at least that’s the conventional wisdom, lifted straight from the fast food chain playbook. A McDonald’s french fry is pretty much the same everywhere, and the coffee served at a big chain is little different. Or so we’ve all been told.

For example, a Long Island Press article cited in our last post quoted local roast master, Greg Heinz: “Starbucks does a lot of things very well. It maintains consistency nationwide.” The article’s author then later goes on to say, “Just like any chain, Starbucks cannot exist without uniformity.”

Or can it? We’ve always felt that the coffee quality can be quite erratic between one Starbucks and another. Lately we’ve been digging into the CoffeeRatings.com data we’ve collected over the years to validate some of our assumptions about coffee and quality. What we found supported our hunch that some of the biggest coffee chains are actually pretty lousy at consistency and uniformity.

Using our ratings data to measure quality consistency among cafés

Below is a table we’ve compiled by keying off some of the fields in our database from thousands of reviewed espresso shots. Each row represents the aggregate espresso shot reviews for a given chain, a given coffee roaster, a given espresso machine manufacturer, or a given cup manufacturer — reflecting a few choices we made for illustration.

Each row (or sample set) shows the number of cafés, high espresso score, low score, average score, median score, and standard deviation for all the associated ratings. For our consistency evaluation here, the key is the standard deviation — which is a simplistic measure of the spread in espresso scores for a given grouping.

For example, the first row represents all reviewed Starbucks. The data suggest that most reviewed Starbucks — about 68 percent, assuming a normal distribution — have an espresso rating score that’s within 1.05 rating points of the average for all Starbucks reviewed (here that’s 5.13).

Now compare this 1.05 with the other example rows in the table. For example, all reviewed Peet’s Coffee & Tea outlets have a standard deviation of 0.74. This suggests a much narrower variation in their espresso scores — and hence better consistency and predictability.

Coffee quality consistency among independent cafés can be stronger than within the big chains themselves

All reviewed cafés using Blue Bottle Coffee beans may have very different owners but score an even lower 0.70 standard deviation. These cafés may only share a bean supplier and some of standards for freshness and access to common consulting and training, but their espresso scores are significantly more consistent among each other than the cafés under a single Starbucks ownership — or even Peet’s.

Surely, a single quality dimension does not represent the breadth of possible flavor profiles, body weights, and crema textures that might also factor into a consistency analysis. But this data refutes the conventional wisdom that Starbucks, for example, provides a consistently dependable and uniform level of beverage quality. Even with their complete supply and delivery chain standardization, Starbucks fails to produce espresso quality that’s as consistent as a number of independent cafés that have only a coffee bean supplier in common.

Our data also suggests that the Starbucks brand is no better a determinant of quality consistency than whether the café uses a La Marzocco espresso machine or Delco cups. This is a critical point to understand, so let’s put this another way:

The Starbucks brand predicts consistency of espresso quality no better than if all the random cafés we surveyed that use Delco cups decided to re-brand themselves as “The Delco-branded Cups Coffeehouse chain.”

Given the investments Starbucks has made in coffee bean and roasting consistency, standardized push-button espresso machines, and standardized training, we don’t see how you can interpret this data as anything short of a complete failure for the company to deliver on the brand expectations of quality consistency.

If Starbucks’ consistency isn’t in their prepared coffee, then it is likely a psychological perception: the consumer’s brand association, the consistency of the Starbucks beverage menu, Starbucks’ own ridiculous names for drinks and their sizes, and the familiar environment of its coffee shops.

Seeking out chains in the hopes of managing risk

We recently came across a blog post by Chris Pirillo, a tech-geek/blogger/ex-TV-show-host who practically wet his underpants because he found a Peet’s Coffee & Tea chain store in his Sheraton Waikiki hotel. This when located in the very same building is an outlet of the Honolulu Coffee Company — which sources beans from Hawaii’s own Greenwell Farms, operates one of the few Mistral Triplette espresso machines in the world, and last scored as high as the best Peet’s we’ve ever been to anywhere. (Famed Baltimore coffee podcaster, Jay Caragay, disputes the quality there as dubious — but we stick by our last, albeit five-year-old, rating.)

Meanwhile, a friend in the Netherlands told us about a Starbucks that opened in Utrecht a week ago. When he passed by over the weekend, he said, “The line was unholy.” In a country where coffee shops legally sell marijuana by the menu, lines are out the door for one that peddles the double-tall, four-pump vanilla caramel macchiato.

What makes people seek out mass-produced mediocrity over independent, higher quality options just as conveniently nearby? What made an old boss of mine seek out a Pizza Hut for dinner when we were in the heart of London together on business travel with time to kill? These questions are woven into why we started CoffeeRatings.com six years ago.

In the case of coffee, you can largely dismiss the argument that chains provide a convenient shorthand to take out the risk in quality versus an unknown. As our data suggests here, big coffee chains can be less consistent among themselves than independent cafés with a common coffee supplier are to each other.

This is the part of the article where we’re supposed to have a snappy, revelatory answer to these questions. But we’ve got nothin’ — other than the consumer comfort with big coffee chains likely says more about their environment and sales practices than it does about the quality of the actual products they sell.

We’ve previously posted about the phenomenon of well-heeled consumers paying for the privilege of performing manual labor in food production that our country could otherwise only get illegal immigrants to do. But here’s a new twist on the continued glamorization of the food service industry — yesterday Microsoft announced a new video game for the Xbox 360 called “Barista Barista Revolution”: Microsoft Unveils Barista Barista Revolution | The Naked Loon.

In case you’ve been hiding out in a cave with Osama Bin Laden for the past decade, the name is a take on an immensely popular video game called Dance Dance Revolution (aka “DDR”). In “BBR”, players apparently take on the roles of highly skilled disco baristas as they “grind, tamp, pull, and steam their way to the perfect espresso, in rhythm to the beat of dozens of today’s top hits.”

The article then goes on to announce:

The game is expected to retail for $120, and thanks to an exclusive partnership with Starbucks, will include a wireless espresso machine game controller—an 80-pound full-scale model of the most common unit found in Starbucks around the world.

…

“Our driving motivation for BBR was to bring the thrill and excitement of the United States Barista Championship into the living room,” explained [the game's lead developer, John] Parker.

We’ve always thought the specialty drink program of the U.S. Barista Championship (USBC) smacked a little too much of Tom Cruise-inspired Cocktail mixology. And the competition’s deficit of real-life barista skills and emphasis on barista gymnastics have always been a puzzle.

But a video game that sports a game controller modeled after the heinous super-automatic Verismo machine so people can play video barista to the sounds of the disco beat? The USBC had better hope this video game does not catch on, or its integrity is going into the toilet faster than you can say “Frappuccino”.

P.s.: Psyche! Continuing our need for blog comedy, Microsoft has no such plans and the The Naked Loon is, er, well… you get the idea.

DDR scene from the movie “Grandma’s Boy”

We really hate doing Starbucks posts if we don’t have to. After all, Starbucks hasn’t been relevant to quality espresso in over a decade. But if you’ve been following some of the Clover brewer posts here, you may be surprised to learn that Starbucks liked them enough to buy the company: Aroma comeback: Starbucks to start grinding coffee in stores. (More details here: Starbucks to Acquire The Coffee Equipment Company, Maker of the Clover – HispanicBusiness.com.)

OK, so the rest of the world seems to be “oohing” and “aahing” over news that Starbucks is returning to grinding beans fresh at their locations — reversing a move to pre-ground, packaged beans from 10 years ago. The media also seem curious about Starbucks’ announced replacement for their horrid Verismo machines: an even more dismal-sounding contraption from the same manufacturer, Swiss-based Thermoplan, called the Mastrena. (More on that in a minute.)

But the news most relevant to quality coffee is their purchase of the fledgling Coffee Equipment Company, makers of the (oft-cited-$11,000-a-pop) Clover brewer. This after Starbucks tried out the device in a couple of Seattle-area cafés for a couple months. For chocolate lovers, this is akin to Hershey’s buying out Scharffen Berger in 2005. (It’s entirely fitting that Starbucks announced Hershey’s as their chocolate partner earlier this month.)

Starbucks coffee in a Clover machine? Who buys a $30,000 sound system to listen to AM talk radio?

But back to the Mastrena, a device that the Seattle Post-Intelligencer described as “a new machine designed to leave a smaller margin for error in pulling shots and steaming milk.” Apparently Starbucks will now be able to hire employees with less skills than trained monkeys — to produce consistently underwhelming and “safe” espresso beverages that taste like they were squirted out of a coin-operated vending machine.

“It’s an unbelievable tool that will provide us with the highest-quality, consistent shot of espresso that will be second to none,” said Starbucks’ chairman, Howard Schultz. However, we’re wondering if by “unbelievable tool” he meant the Mastrena…or if he was referring to himself.

UPDATE: March 20, 2008
If you envy those at The Coffee Equipment Company, who cashed in big with their Starbucks acquisition success, here’s a story for you from today’s Post-Intelligencer: Starbucks deal ‘dream come true’ for manufacturer of coffee maker.

UPDATE: March 26, 2008
The New York Times kicked the tires of a Clover machine in a Starbucks, bringing along George Howell of Acton, MA’s Terroir Coffee as their expert taster: Tasting the Future of Starbucks Coffee From a New Machine – New York Times. His findings? Most of the coffee Starbucks roasted for their Clover machines was over-roasted and destroyed the flavor, reducing the Clover to something no better than a $20 French press could produce with the same beans.

UPDATE: March 28, 2008
And here’s a version of the story today from the Associated Press, highlighting some of the independent cafés that are disowning their Clover machines in response to the buyout: Starbucks acquires pricey coffee maker … and the indies are upset – SGVTribune.com.

What will those wacky Swiss think up next? Apparently, it’s an espresso-tasting machine: A machine with a taste-for espresso.

Yes, it’s the latest invention from the people who brought us yodeling and clandestine overseas bank accounts. But with the Swiss, it’s not all good stuff. They have also brought us brain-dead, monkey-operated, superautomated espresso machines — such as those manufactured by Franke, Nespresso, the Schaerer. (The latter of which is responsible for the dreaded Verismo, part of a secret Swiss plot to bring about the downfall of Starbucks.)

So the nation that has built robots to do everything from stacking wheels of cheese to pumping out sickly cups of espresso has turned its attention to espresso tasting. (More sophisticated coffee industry robots that also wear hoodies and bike messenger bags, collect tatts, and listen to Cat Power on iPods are apparently still in the works.)

The machine operates by analyzing gases released by a heat­ed es­pres­so sam­ple. By evaluating some of the over 1,500 aromatic and flavor compounds in a brewed espresso, the developers hope to replace a lot of the human profiling that still goes on in coffee production. A study on the effectiveness of the device, which is rumored to be nearly as ac­cu­rate as a pan­el of trained hu­man es­pres­so tasters, is planned for publication in the March issue of An­a­lyt­i­cal Chem­is­try.

Robots to brew, robots to drink

The coffee industry still relies heavily on human senses for evaluating what makes a good cup. While some day a Swiss-made, espresso-tasting robot may put CoffeeRatings.com out of business, I was also struck by something James Freeman told me yesterday at his new Blue Bottle Cafe. He took a moment to sample his siphon bar coffee after serving it — noting how he was impressed with how the bartenders at his neighborhood Nopa often sample their cocktails from the ends of stirring straws as a way to keep tabs on the resulting product.

Technology can go a long way towards modeling the physical world. But until coffee is served by robots for robots, chances are that some things just can’t be replaced without the human touch.

UPDATE: Feb. 12, 2008
More news of this story is starting to hit the presses. This includes the Royal Society of Chemistry, which uses espresso-tasting profile charts as illustrated by the Centro Studi e Formazione Assaggiatori: Machines get a taste for espresso. There’s also the actual ACS publication: When Machine Tastes Coffee: Instrumental Approach To Predict the Sensory Profile of Espresso Coffee, where other blogs have excerpted some of its illustrations. Plus a weak, one-minute audio spot from Scientific American: Scientific American: Coffee Tasting Machine Stirs Industry.

While we’re on the subject of humorous opinions on coffee purveyors…

No matter what you might think of Arianna Huffington, you have to admire the cajones to publish today’s op-ed blog piece on The Huffington Post: Beverly Tan: Why I F**cking Hate Starbucks – Living on The Huffington Post. Author Beverly Tan, an admitted and yet reluctant Starbucks customer herself, rips into Starbucks’ super-saturated ubiquity, “cross between drain cleaner and flat Pepsi” quality, “lesbian folk singers and Miles Davis” soundtrack, and misplaced pretentiousness.

Nespresso loves company.

Coincidentally, yesterday Starbucks announced that they are “upgrading” their espresso machines: Starbucks Upgrades Espresso Machines – WSJ.com. It’s a move designed to help stave off customer losses in their U.S. stores — though they’ll deny it’s connected in any way, just like their newly planned national ad campaign. Given the last time Starbucks put their genius towards “upgrading” espresso machines, where they replaced all their La Marzoccos with with superautomated Verismo machines, we can’t wait to see how they’re going to top that.

Starbucks executives bristle at the suggestion that their corporation represents the fast food of coffee. (This despite lamenting memos from the chairman to the contrary.) But when was the last time Starbucks made a corporate decision that favored quality over quantity?

Case and point … the new chocolate partnership Starbucks announced this week: Starbucks, Hershey to offer coffee-flavored chocolate products in the U.S. – International Herald Tribune. Starbucks claims this deal will help them better pursue customers “looking for indulgence”. But indulgence from Hershey’s? Those “artisans” of assembly-line milk chocolate, who brought us the ubiquitous mega packs of fun size candy bars given away for free every Halloween? Well, perhaps if by “indulgence” they mean a Triple Whopper with cheese…

(OK, some of you chocolate aficionados will cite Hershey’s 2005 purchase of Scharffen Berger, but buying quality doesn’t count. Just look at what Starbucks did with the acquisition of Torrefazione Italia. Scharffen Berger fans are still dismayed by the sell-out.)

Starbucks acts as if they’re on a relentless mission to erode the quality promise their brand established over a decade ago — from weaker and weaker coffee standards, to drive-thru outlets found in gas stations, to downmarket partnering with America’s greatest purveyor of quantity-over-quality chocolate. If Starbucks brushes off the threat of coffee competitors in the fast food industry, why do they seem hell bent on becoming indistinguishable from their fast food “non-competitors”? (Archrivals storm Starbucks – BusinessWeek Online – MSNBC.com.)

Meanwhile in the news, financial analysts suspect that Starbucks will need to raise their prices once again to improve their flagging operating margins (and stagnant stock price): Price of your caffeine fix at Starbucks could be headed higher. These are rough times to be a Starbucks shareholder. Perhaps their only hope now is for Verismo to make a superautomatic espresso machine with a giant clown head that will take your double-tall, four-pump vanilla caramel macchiato order.

UPDATE: July 24, 2007
Sure enough, Starbucks announced yesterday that it plans to raise prices 9 cents this month: Starbucks raises beverage prices 9 cents – USATODAY.com. Operating profit margins and same store sales must definitely be hurting at the company.

Today’s Toronto Star featured an article on the growth of Toronto area independent coffee shops (many of which call themselves “espresso bars”, per the article): TheStar.com – living – Toronto’s love affair with espresso bars heats up. Toronto now has a mixture of established independent espresso bars and a growing array of newer ones. (A little over a year ago, we wrote about a Toronto local who lamented the common over-extraction problem. Hopefully this new crop of indie espresso bars has helped.)

While some credit Starbucks with paving the way for consumer interest in these indie coffee shops, the co-owner of one of them notes, “They’ve [Starbucks] taken away the art of the barista with what I call their robo-espresso machines.” The article suggests that much of the success of these independent espresso bars comes from how they cater to a customer’s desire for “individual service and wanting to feel special”.

Me? I don’t need to feel special. I just want good espresso.

101 Ways to Make Your Coffee Unrecognizable

On that note, the rest of the article tours a number of Toronto espresso bars with Susan Zimmer, a Calgary-based author of a new book called I Love Coffee! Over 100 Easy and Delicious Coffee Drinks. Which leads me to another observation. Despite the specialty drink “ring” of the three-ring circus that is the barista championship, I have yet to meet a coffee expert with an opinion I respect who focuses their energies on the variety of coffee drinks rather than the quality of the coffee per se.

Sure, a single espresso every time might sound like a monotonous death sentence to some people. But who in their right mind needs over 100 coffee drinks? That smacks of Starbucks’ coffee-flavored milkshake approach, primarily aimed at people who really don’t like coffee but still want to play along as if they do.

And while I’m sure Ms. Zimmer knows her stuff — even if her other book has the red-flag title of Cappuccino Cocktails & Coffee Martinis — I can’t tell you how many times I’ve been disappointed by coffee books that suddenly turn into pages and pages of recipes by the end. These books are no more about coffee than The Betty Crocker Ultimate Cake Mix Cookbook is a book about sugar.

Is it really so hard to understand? Coffee is not wine. It is not a martini either.

The news of Starbucks‘ death has been greatly exaggerated.

That’s all I could think of when reading the media and blogosphere response over the past week to an executive memo written by Starbucks’ founder and corporate chairman, Howard Schultz (as reported here on Feb. 24). In the memo, the founder lamented the loss of Starbucks’ “romance” for the sake of rampant, metastasis-like growth — citing the infiltration of superautomatic Verismo machines, bagged coffee, and cookie-cutter store designs. The memo was leaked as a sort of PR stunt; Starbucks first floated it out, leaving it up to public interpretation to see if it had the desired effect, and only afterwards did the corporation verify the memo’s authenticity.

But why was this suddenly breaking news to everyone?! Are people so mindless before their morning cup of caffeine that they’ve walked into Starbucks like automatons for the past dozen years — not noticing Starbucks’ relentless execution on their massive growth plan at the expense of quality?

“Yo! This is Mars Blackmon and I’m chilling with my main man, Howard Schultz!
Yo, Howard, what makes Starbucks the biggest coffeehouse in the universe?”

While some see this as reason to start sifting through the Starbucks ashes (kind of like drinking their coffee, in fact), I am not about to feel sorry for Starbucks making money hand-over-fist. It’s not like this “news” has halted any of their plans to double their growth plans to over 40,000 cafés. But for the marketing wonks who try to justify every business success as a deliberate brand strategy, the memo did not sit well with them — there are things to be fixed and professions to justify. (Meanwhile, shareholders keep counting the money.)

Sure, there are those who lament the watering down of the Mercedes-Benz brand when their automotive offerings started going downmarket to entice new customers. But that has hardly hurt their pocketbooks. An elitist brand is worthless without a profitable base of mass consumer appeal. As in The Big Night, while Primo lamented the rape of cuisine going on at a rival Italian restaurant, consumers were passing on the risotto and opening their wallets for spaghetti & meatballs.

“Is it the machines?!” “No, Mars.”

Among these lamenting marketing wonks is a pair of former Starbucks marketing guys who surprisingly knew very little about what made Starbucks successful. They’ve gone so far as to ridiculously suggest that Starbucks replace their Verismo machines with the return of their old La Marzocco lines (“Starbucks should put the Verisimo automated espresso machines in higher-volume stores and put the La Marzocco manual machines in lower-volume stores”).

The genie is already out of the bottle, folks. Starbucks “going back” is akin to asking Gorbachev to roll back Perestroika (Gorbachev being my favorite Starbucks analogy) because too many of Putin’s critics are ending up dead and glowing. Today Starbucks has some 140,000 employees. Can you imagine the complete chaos, and expense, if even just 20,000 of their low-wage, push-button employees were suddenly asked to operate sophisticated machinery? May as well give every bicycle commuter in Ho Chi Minh City a Volvo and tell them to drive to work tomorrow.

“Is it the baristas?! It’s gotta be the baristas!” ” “No, Mars…it’s not the baristas.”

Some of the branding recommendations even include showcasing some of Starbucks’ best baristas (“Starbucks must give permission to store partners to showcase their flair and personality while on the bar”). Really — only a few of us coffee geeks, an unprofitable minority, even care about these things. To the rest of the world, a “barista” is hardly an elevated profession. Due to the likes of Starbucks, the barista has become a modern symbol of downscale jobs, skills, and wages; coffee’s version of the minimum wage McDonald’s fry cook.

More to the point, there just aren’t enough quality baristas to go around — particularly for the volume that a massive chain like Starbucks would need. The best ones wouldn’t be caught dead in a behemoth such as Starbucks … and would work elsewhere for better tips and appreciation.

The other marketing wonk proposal is that Starbucks differentiate between their retail stores. But that would just create a bizarre Starbucks caste system — defeating many of the business goals of their ubiquity. Launch “Starbucks Select” (gag, cough, spew), and the Starbucks brand becomes a liability at the high end of the market. Take a page from The Gap and differentiate between your Banana Republic’s and Old Navy’s (e.g., call it “Howard’s Grand Café”), and these quality efforts become completely dissociated with their flagship brand and add nothing to its eroding value.

“Money, it’s gotta be differentiation!” “No, Mars.”

So Starbucks is naturally under attack from the likes of McDonald’s and Dunkin’ Donuts; that’s to be expected. You’re making a boatload of money, and others are following suit to ride your commercial coattails. You’re going to attract imitators, but that’s the price of success. “Romance” has nothing to do with it. I would argue that if Starbucks didn’t take the low road of mass production and commercialization, their company would have been run roughshod by today’s upstarts who run quality circles around them at the high end of the market.

So here’s my advice, Howard. Forget us coffee elitists. You clearly abandoned all hope of keeping us engaged many years ago, and it’s hardly hurt your bottom line. We fled your brand with no intention of ever turning back. (No hard feelings — we just agreed to part ways a long time ago.)

And the superautomatic machines? So what! — just keep the drinks coming fast and in a “clown-free” environment. Keep on cranking out the double-tall, four-pump vanilla caramel macchiatos and your broad array of coffee-flavored beverages. Note that I did not say “coffee” — as your profits are in coffee-flavored milkshakes, not in single shots of espresso. As I mentioned before, most people really don’t like coffee anyway. A key ingredient to your success has been convincing your customers that they do like coffee — just your version of it.

Stay true to these facts and you’ll be nicknamed “Money” for many years to come.

(Apologies to Mars Blackmon for the preceeding subheads — taken from recent conversations with Howard Schultz about what makes Starbucks what it is today.)

In the news today from Bangalore, India has applied for a Geographic Indication (GI) for Monsooned Malabar coffee: Malabar coffee set to get GI brand : Malabar Coffee, Monsooned Malabars, Geographic Indication mark, GI : IBNLive.com : CNN-IBN. Some love the stuff, others can’t stand its “funkiness” at times. Whatever it is, it makes a mean espresso blend. (Monsooned Malabar is one of the core components, and the only named bean, of the infamous Malabar Gold blend.)

India’s choice for GI instead of trademarks will be sure to please Starbucks, who are preoccupied with Ethiopian Yirgacheffe, Sidamo, and Harar: What’s in a name? Ethiopia’s battle against Starbucks | FP Passport. (Though in unrelated news today, it looks like Starbucks has a better alternative to their super-automatic Verismo machines to consider … robot baristas!: Japan mixes robotics with tea time – Innovation – MSNBC.com.)

Changes are brewing at the Tully’s Coffee chain, as American Express announced Tully’s as their latest ExpressPay partner: American Express signs Tully’s Coffee to contactless payments system – Assodigitale – Associazione Italiana Operatori Industria e Media Digitali. Think of ExpressPay as FasTrak® for the rush hour coffee queue — where customers can swipe credit cards to pay for small transactions without a signature.

I expect Starbucks to quickly follow suit, given how the company’s profits rise and fall with the weather — and their slower turnaround service times to make Frappuccinos. Starbucks publicly cited these service times as the reason for eliminating their remaining La Marzocco machines and replacing them all with their dreadful Verismo super-automatic machines. (It is also a smokescreen for the fact that these machines allow them to open outlets more quickly with cheaper, less skilled labor, making you lower-quality espresso drinks.)

It’s probably only a matter of time before Starbucks exhumes the Automat, firing all employees and replacing them with push-button vending machines for self-service coffee, music CDs, and movie rentals. And nothing says quality quite like “Automat”.