Tag Archives: Brewers Association

World Beer Cup Judging

As I was watching the live stream of the 2014 World Beer Cup awards ceremony the other night on the The Brewing Network, I noticed in the chat room a lot of suppositions about how the competition is run. As a long time judge, I thought I might take a little time to explain how the competition for the WBC (and the GABF) is organized and how the judging process works. Special thanks to Technical Brewing Projects Coordinator and long time Competition Manager Chris Swersey from the Brewers Association, who reviewed this post and added some valuable detail about the process.

First off, the competition is blind, meaning that judges do not ever know what beers they are tasting in any given session. The only information provided is a random identification number and a description of what (if any) special ingredients might be in the beer (about a third of all beers are entered in styles allowing entering brewers to specify fruit, spices, type of wood, etc.). The random numbers assigned to each specific beer change for each round they are judged, making the process truly a blind competition.

Judging sessions are divided into a morning session and an afternoon session for each day of the judging. In each approximately 3 hour session, 6 or 7 judges are assigned to sit at a specific table, and they judge 1, 2 or 3 flights of beer (most often 3). The judges stay at the same table for each half-day session. Each round consists usually of 10-12 beers, so there can be 20-30 beers total in each session, and with 2 sessions per day, that results in 40-50 beers being judged by any particular judge over the day. Unless it is a medal round, the table is usually split in half, and one side of 3-4 judges gets one round of samples, the other side of judges get a different round of samples, though they are always the same style. Morning sessions tend to focus on lower alcohol styles; afternoon sessions tend to include higher alcohol or higher flavor styles. This is not a hard and fast rule, just a general theme. Also, at any given table styles tend to be scheduled as less flavorful followed by more flavorful – for example, golden ale followed by stout.

It is common to have 2 different styles judged in any session, though for each individual flight in a session, they are all the same style. So for example, in one session, a team of judges at a table could have 2 flights of American Pale Ale, then 1 flight of Imperial Stouts (I am not divulging what styles I judged). Categories with 12 or fewer entries are judged in one first and final round, meaning all 6-7 judges taste all the beers, choosing (usually) 3 winners.

Categories with 13-24 entries are judged at one table, but in two flights. In the first flight, the table is split in half. Each group of 3 or 4 judges evaluates half of the entries, passing 3 on to the final round. In the second and final flight, all 6-7 judges taste the 6 finalists, choosing (usually) 3 winners.

Categories with 25-48 entries are judged at two tables, in two flights. In the first flight, half of the total number of entries is assigned to each table, and each table is split in half. Each of the four groups of 3 or 4 judges evaluates their share of entries (never more than 12), passing 3 along to the final round at one table, for a total of 12 finalist entries. In the final round, all 6-7 judges taste the 12 finalists, choosing (usually) 3 winners.

Categories with 49-72 entries are judged at three tables, in three flights. In the first flight, one third of the total number of entries is assigned to each table, and each table is split in half. Each of the six groups of 3 or 4 judges evaluates their share of entries (never more than 12), passing 3 along to the second round at one table, for a total of 18 second round beers. The second round table is split in half, with each group of 3 or 4 judges evaluating 9 beers and passing along 3 finalist entries. In the final round, all 6-7 judges taste the 6 finalists, choosing (usually) 3 winners.

Categories with 73-96 entries are judged at four tables, in three flights. In the first flight, one quarter of the total number of entries is assigned to each table, and each table is split in half. Each of the six groups of 3 or 4 judges evaluates their share of entries (never more than 12), passing 3 along to the second round at one table, for a total of 24 second round beers. The second round table is split in half, with each group of 3 or 4 judges evaluating 12 beers and passing along 3 finalist entries. In the final round, all 6-7 judges taste the 6 finalists, choosing (usually) 3 winners.

Categories with more than 96 entries are judged at tables increasing every time another 24 entries is added.

Most categories have 2-3 rounds. Categories with more than 192 entries like India Pale Are are judged over 4 rounds. For most styles, the tasting flow is structured in multiples of 12 or 24 entries. For certain high alcohol or high BU styles the multiple is 10 or 20 instead of 12 or 24.

During the first round (only) comments are filled out that are returned to the entering breweries:


The beer evaluation form used for beers in the first round of WBC and GABF. Judges who don’t write a lot of comments on this form may not be invited back. It’s important feedback for the entering brewers.

In rounds 2, 3, and sometimes 4, for each category, 3 of the 10-12 samples are again selected for moving on to the next round. By the time the beers make it to the final round, they have been selected and passed through as being one of the top 3 in each previous round. The final round (the medal round) can consists from anywhere from 6-12 samples that have arrived via a process of elimination. If a table is doing a medal round, the table is not split, and every one of the 6 or 7 judges tastes and evaluates the same beers to award the medals. Note that you may taste 2 rounds of a certain style, yet may not judge in the medal round, which can get sent to a different table of judges.

The judging requires consensus on the 3 beers being passed forward. It is not based on scores. No scores are given, unlike in the Beer Judge Certification Program (BJCP) homebrew competitions (see below for their scoresheet). The 3-4 judges at WBC and GABF have to all agree on which 3 beers are the best of the round and are deserving of being passed on. This can take a fair amount of discussion, though the process is helped by the fact that each style has very specific Style Guidelines and each judge is required to use those guidelines for the basis in which they judge the beers. And if a beer is not perfect in any way with respect to the style, it will not be given a gold medal. Which is sometimes why there is no gold medal awarded in a category. It’s not a ranking contest, medals are given based on very specific guidelines for gold, silver and bronze awards.

BJCP Judging Scoresheet includes a detailed scoring system-not used at WBC or GABF

By the time the judges get the remaining beers for the medal round, the beers are, by and large, world class examples of the particular style. And determining which get awarded medals can be tough and at times contentious. The discussions and debates that occur are always respectful, but judges are not always in agreement over which beers deserve to be awarded a medal.

This year there were 94 separate categories that were judged. All the judges have proven skills in taste evaluation of beers and knowledge of beer styles. In an impressive showing, 75% of the judges this year were from outside the United States. And no judge is allowed to judge in a category that they have a beer entered in. It was a pleasure and an honor to sit at the table with some of the best brewers in the world and judge this year’s World Beer Cup. The integrity of the competition is at the highest level, and my congratulations to all the winners this year, many of whom are good friends.




What is quality? Part 2

So after I ranted a bit last week about brewers who don’t invest in quality and how that can harm everyone who makes beer for a living, I thought I’d take some time this week to review a little bit more about what it takes to brew a quality beer (or a “proper pint” as they say in the UK).

I hit on a few elements that I think make a quality beer in the previous post. These are:

1. Consistency from batch to batch. Admittedly, this refers to a brewery brewing and packaging a core lineup of beers. Although a brewpub that makes a few flagship beers or regular offerings should strive for consistency too. Philosophically, I disagree to some extent with the notion that a small brewpub/nanobrewery/farmhouse brewer can successfully embrace variable beer flavor caused by seasonal changes to recipes and variability in ingredient flavors, but I understand the attraction to this kind of brewing. For them the question becomes: “Did I make the beer I set out to make?”

2. Lack of off-flavors. Some people don’t agree or don’t care, and many beer drinkers don’t have a good understanding of this, but the presence of off-flavors like diacetyl (butter) and acetaldhyde (pumpkin seed) is a strong indication that the brewer doesn’t have a good grasp on the basics of managing the fermentation part of the brewing process. Too many brewers focus on recipe formulation and the brewhouse operation, but fermentation is by far the hardest part of the brewing process to manage. The idea of managing a living organism (yeast), and controlling it to the extent that you get proper amounts of growth, the flavors produced by the yeast are flavors you want, and the yeast is healthy and happy though several rounds of repitching to fresh wort is daunting to say the least. It requires the utmost focus every day. And it requires good sensory skills and knowledge, so the brewer can determine if things are going as planned, and if not, take immediate steps to rectify the situation.

Other off-flavors:

The presence of DMS (Cream Corn flavor) is indicative that the boiling process in the brewhouse is not as vigorous as it needs to be, or the wort was held hot for too long in the brewhouse.

The presence of phenolic (Bandaid/medicinal flavor) is most likely indicative of having sanitation or ingredient issues.

The presence of excessive grainy or harsh, wet paper or cardboard character is indicative that the beer is either old, or was packaged with high oxygen levels. As someone pointed out in the last blog, this applies primarily to fresh beers. In high strength beers that are brewed for aging, like Imperial Stouts, Barley Wines, etc., the aging and oxidation can create complex flavors that are highly desirable. But still, the aging process must be controlled by holding the beer at the right temperatures to avoid the development of excessively harsh flavors.

3. Creativity: This is a big part of what is going to make or break brewers that are just now getting started. There are so many brewers out there now, each brewery has to have a unique product and message in order to stand out in the crowd. This being said, if you don’t have the first two elements locked down, creativity and uniqueness won’t mean squat. And be mindful of flavor balance. More hops or more spice, for example, isn’t always better. Learning how to get proper balance of flavors in your beer requires skill and experience.

So what’s a brewer to do? Here are my recommendations for brewers that are just starting out:

1. Every single brewery should have, at the very least, lab equipment that includes a pH meter, a hydrometer, and a microscope that is capable of viewing yeast and bacteria.

A pH meter is used to check brewing water pH (critical for proper and consistent mashing), wort pH and beer pH (changes in beer pH can indicate the presence of acid-producing beer spoiling bacteria). pH meters are also critical in wastewater operations. Most municipalities that receive waste streams from a brewery require a certain pH range for that liquid, so brewers need to be able to measure and adjust the pH of their waste streams.

A hydrometer is a floating, calibrated glass or plastic spindle that is used to measure the specific gravity (or density compared to water) of the wort, which refers to the sugar content of the wort. It’s critical to understand wort specific gravities throughout the brewhouse and fermentation process. This tells the brewer that the targets in the brewhouse are being met, and that fermentation (which causes a reduction of sugar content) is occurring at the proper rate. There is an instrument called a refractometer which also measures the density, but be aware that this instrument, while easier to use, won’t provide accurate numbers in beer (with alcohol). But for the brewhouse gravity checks a refractometer is a great tool.

A microscope is critical for viewing brewers’ yeast. Brewers need to check every fermentation as it progresses for yeast viability and cell counts. This involves taking a sample of fermenting beer, or the yeast, and putting it on a special slide called a hemocytometer. The hemocytometer is equipped with gridlines that assist with counting the cells-important to know as fermentation progresses-should see a 3-5 fold increase in yeast at the height of fermentation. In addition, staining the yeast with methylene blue, trypan blue or methylene violet will allow the brewer to count dead yeast cells vs. live, healthy cells. Yeast viability should be be in the 90+% range if you hope to re-use the yeast with good results.

2. At the second level, brewers should really consider investing in a shaker table, malt sieve screens, an analytical balance and a Dissolved Oxygen Meter.

The shaker table holds Erlenmeyer flasks and is used to agitate samples of wort from each brew in the presence of a lot of yeast. The beer in the flasks will completely ferment out within 24 hours, and by measuring the gravity after this process, the brewer will know where exactly the fermentation should finish in the production fermentor. This is called a “forced fermentation” or “attenuation limit” test. In my opinion, this is one of the most critical quality checks a brewer can do, since chilling a tank of beer that still has residual sugar in it, even though the fermentation appears “done”, can result in a sweeter beer that is very susceptible to microbial spoilage. Don’t ever stop a fermentation before the fermentable sugars are consumed, it’s a recipe for disaster. This test can also be done with a lower price tag using an Erlenmeyer flask of fermenting beer placed on a stir plate with a magnetic stir bar.

A malt sieve shaker is set of circular screens with varying mesh sizes that are stacked on top of each other, from coarsest (most open) sieve at the top to a fine mesh sieve on the bottom. Milled malt is placed on the top screen, and the stack is shaken. As the particles of millled malt fall through the screens, they are held back on a particular screen with a mesh size too small to allow the piece to fall through. Using a scale or analytical balance to measure the amount of milled malt retained on each screen allows the brewer to gauge the performance and consistency of the malt milling operation. There are guidelines for the percentage of total grain that should be held on each screen to optimize extraction of sugars and also get good straining of clear wort from the spent grain. Several quality, taste and efficiency issues arise from having too much coarse material in the milled malt, or too much fine/powder material.

The Dissolved Oxygen (DO) meter is the most costly piece of equipment mentioned so far (usually $3000-$10,000), but is critical to ensure low dissolved oxygen levels in the fermented beer. Oxidation reactions create excessively harsh, papery and grainy flavors, and also a very rapid loss of hop character. A brewer who knows that their process is keeping the beer free of oxygen can be assured the beer will survive better in the treatment it gets out in the trade.

Understand that many of these pieces of equipment can be purchased for very reasonable prices, especially if the brewer is willing to shop around and buy used equipment on eBay or Craigslist.

Most small brewers don’t have the resources to purchase the more expensive equipment used to measure IBUs and alcohol. But there are several companies that provide analytical services, and I highly recommend using them on a regular basis. Small brewers have to rely on calculations and estimates to list abv and IBU levels. Using an analytical service can verify those numbers and make them more accurate, as well as help ensure consistency from batch to batch.

3. A formal sensory program is a must! Sitting at the bar and having a glass of beer does not constitute a sensory program. Sensory programs require formal tasting of the in-progress beer and finished beer to ensure the flavors are consistent and desirable. I recommend tasting daily at a minimum: brewing water, samples from the fermentor (a sniff test is fine here), beer that is ready to package, and packaged beer. Also, a museum stock of packaged beer should be held cold and at room temperature and tasted on a regular basis to ensure the beer isn’t going stale too quickly. A consistent and thorough tasting program helps ensure beer flavor consistency and also helps catch problems very early, when they are easier to fix. And keep records!

4. Don’t accept poor quality ingredients. Have a program to select what you want to use (this refers in particular to hops), and evaluate your ingredients as they come in to make sure they are of proper quality. This is tough for a brewer that is just starting out, because often there is no money available to forward contract the hops or the malt. Just be aware that there are some sub-par ingredients out there, don’t succomb to financial or production requirement pressure to use them if you can avoid it.

5. Have a basic microbiological testing program. There are many ways to do this, but have a formal program to either streak agar plates with samples of wort and beer, or some other method that will tell you if your process is sanitary enough to ensure no spoiled beer. Spoiled beer will turn off beer drinkers very quickly, and most customers won’t come back for another try. And people on the beer internet sites love to spread the word about infected beer.

6. Don’t release sub-par beer. If the beer is spoiled, or has off-flavors, please bite the bullet and dump it. Take the financial hit, because in the long-term, it will save the company’s reputation. Learn something from the experience and move on.

7. Understand beer styles and how to brew them. This doesn’t necessarily mean be part of the “style police” and brew everything strictly to style. But it does help to understand the established styles so you can use them as a basis for your beer and develop well crafted variations on them.

8. For goodness sake, if you are a brewery owner, hire a trained and/or educated brewer to run your operation. Don’t skimp on this so very important part of the business. There is no substitute for experience, but a young person who has just graduated from brewing school (and there are many excellent ones across the country) will be able to understand the brewing process on a commercial brewing scale much more effectively than someone who has only brewed several batches of homebrew. This really is critical. Get someone who on board understands the science behind brewing. A brewing school graduate will demonstrate both passion for the craft and the ability to think critically about the processes and troubleshooting.

9. Rely on the industry for help and advice. The Brewers Association out of Boulder is a wonderful organization for craft brewers. It doesn’t cost much to join, and there are many ways you can ask questions, and get answers from some of the smartest, most experienced brewers in the country. The Master Brewers Association of America is a technical association that has been in existence for over 100 years. And though historically they have been the technical organization for large brewers, they are now also focused on craft brewing. There are regional districts of the Master Brewers all over the country, and it costs typically less than $150/year to join. The district meetings are a great place to network and learn from more experienced brewers, and the annual conference is a nationwide event that presents some of the best brewing technical information I have experienced. As a third option, this industry is still young, and there still a strong sense of brotherhood and camaraderie among craft brewers, and most of the local brewers are likely willing to help a startup brewery with quality or equipment issues and advice. And finally, websites like pro brewer.com have excellent forums where brewers can ask questions and get great answers.

10. Be humble when praised, and respond quickly to complaints. Nothing can salvage a brewer’s reputation quicker than providing a quick response to a customer who has a complaint about the beer. Make a sincere effort to make good on the issue, either by replacing the beer or sending a shirt, hat, or logo glass. Ignoring complaints will come back to hurt the business. More people spread the word about poor experiences with beer and breweries than those who have had enjoyable experiences. And when receiving praise, don’t get a big head. Know that a serious quality issue, infection, bad batch, whatever, is lurking just around the corner. I remember Dan Carey of New Glarus once saying: “There are 2 kinds of brewers: brewers that have had infection issues, and brewers who haven’t had infection issues, yet“. True words.

11. Focus 100% on cleanliness and sanitation. Experienced brewers know this, and often (semi) jokingly call themselves janitors, because 75% of their job is cleaning, scrubbing, and sanitizing. Wort, being 12-25% sugar, is extremely susceptible to unwanted bacterial or wild yeast spoilage. And beer is only a bit more resistant. An unclean brewery cannot produce consistent beer without off flavors.

And now it is time for me to get off my soapbox, at least for a few days!