KFPL Season 2 Sealed Qualifiers - An Event Analysis

On the 23rd January, the KeyForge Premier League hosted their second set of qualifiers for spaces in Season 2 of the league. These were sealed qualifiers, where each player had a choice between 4 decks - 1 deck from each set. Each player who had a record of 4 wins and 2 losses or better after 6 rounds of Swiss was put into a single elimination bracket, with any byes being allocated to those with the best records. The top 3 players after this process (with the 3rd place player determined through a play-off) were invited to join Season 2 of the KeyForge Premier League.

As the deck selection process was much more pragmatic than the archon choices, this analysis is going to be very different from the first article.This write-up is to examine the options available, the deck choices players made from those pools, and the success of the top 16 players. I will also be spotlighting several case studies.

In this write-up, I will be using the following terms to refer to different groups of decks:

  • ‘Pool’ - this was every deck which was an option in the qualifier (4 per player)

  • ‘Decks played’ - this is every deck which was chosen from the pool and played.

  • ‘Top 16’ - this is every deck which made the top 16 decks in the event after the challenger round of the single elimination bracket. Some decks were given a bye into the top 16 due to numbers and individual performance.

One of the criticisms of sealed as a format is that it is reliant upon the player's luck in pulling a strong deck. Primarily the goal of this analysis is to answer the question: "Were your chances of winning completely dashed if you got a low-SAS set of decks?"

Deck Methodology

The decks were drawn from Decks of Keyforge (DoK) by using a date as a filter - the most recent 1000 decks of each set were downloaded, with no other restrictions, so deck quality was completely randomised. Once downloaded, the decks were sorted randomly into pots, which were sent to each player. This ensured an entirely random selection of decks. (Note: random does not mean equally random.)

One of the quirks of doing this is that a bulk listing of Cull The Weak’s Brobnar89’s decks were caught in this - approximately 20, all of which were listed for sale. This was quite funny to discover after the fact, and great free advertising for him! (Perhaps an idea for the future is to only use decks with a For Sale tag to generate the event pool…)

Prior to the event, there were some concerns about drawing random Mass Mutation (MM) decks from DoK. The most concerning of these was the risk of pip manipulation to create broken combos, particularly on cards which recur such as Relentless Creeper, Q-Mechs and Rad Penny. Despite this, the KFPL organisers chose to draw decks randomly, deeming the risk to be very low. I will be examining some of the MM decks which made the top 16 to see if this was an issue in practice.

Note on Analysis

Most of the SAS analysis conducted for this write-up is presented with box-and-whisker plots. Due to me grappling hopelessly with Excel ( >:( ), these don’t have labelled axis, and have an additional item on the key for each type of deck. To read these as intended, please ignore the colours which don’t show up, and read the vertical axis as SAS rating, and horizontal axis as Set. Dots on the charts are outliers.

I would also like to give a disclaimer that I’m not an expert in statistics, although I have checked my methodology with people who are more knowledgeable than me.

Qualifier 3 - 9AM GMT / 4AM EST

Pool Analysis

80 players registered to participate in this event, resulting in a total pool size of 320. The median for each set ranged between 60 - 67 SAS, with Age of Ascension (AoA) being the weakest set on average, and Mass Mutation (MM) the strongest. MM had the highest SAS peak at 79, and Worlds Collide (WC) the lowest value at 43. The top end of all the sets was fairly consistent (76 - 79), but the low end varied significantly (43 - 57), with MM having the highest value (57) of the lowest scores.

Played Decks Analysis

As the MM decks in the pool were on average much better than the other options, it is unsurprising that it was the most popular choice for this event, making up 45% of the field. WC was second-most popular at 22.5%, and CotA third at 18.75%. Although it was least popular, AoA had a respectable showing at 13.75%.

The SAS ranges of the played decks indicated that those who received the top-rated SAS decks almost always chose those, as the peak values are identical to the general pool. AoA was generally rejected unless it was significantly better than other options. Players who used CotA decks were more forgiving towards their SAS ratings. WC had the highest concentration of high-SAS decks (50% were >=73 SAS).

Swiss Performance

There was a significant correlation between the SAS of the deck played and its performance in the Swiss rounds [r(78)= .28, p= .011.].

However, this isn't to say that players who chose a lower-SAS deck were doomed to failure - the lowest SAS deck to qualify for the top cut in this event was 62 SAS. One player who chose a 77 SAS deck from their pool went 0-3 and then dropped from the event. You can see the SAS ranges and their success in the Swiss rounds in the box plot below:

The take-home from this, I think, is - you don't have to have a high-SAS deck to do well in a sealed event, but it helps.

Top 16 analysis

The conversion rate of each set to the top 16 was largely proportional to the set share in the decks played.

SAS was a strong predictor of whether or not a deck would reach the top 16, with the range of SAS being 66 - 79. The peaks for AoA and CotA decks were slightly lower than those across the decks played. The top 3 players' decks were 77, 73 and 70 SAS respectively.

Luck or Skill?

So, were the results of this event down to the strength of the decks? To check this, I calculated the expected wins for each deck based on their statistics, and then checked the performance against the average for each of the top 16 players.

The method for this was the following:

  • On advice from someone with DoK know-how, I used AERC as a metric for measuring CotA and MM decks, and SAS for measuring AoA and WC decks. I was advised that those statistics are better indicators for those sets.

  • I calculated the average Swiss wins for each AERC and SAS score represented, and then used those to calculate an average number of wins in the event per unit, (This ended up being 0.038 for AERC, and 0.037 for SAS.)

  • I used this number to calculate the expected wins for each deck. Then, I compared this with the actual wins, then took away 1 from each result to get the performance of that player/deck above the average.

As a formula, this was:

The results, displayed on the right, show that every player in the top 16 performed well above the level of expected wins for their deck in the Swiss rounds. This suggests that there was an enormous element of player skill which contributed towards individual performance. Interestingly, this factor peaks within the rank 3-8 players and then tapers off in the top 2, suggesting that those in the top spots had a slightly easier time due to the power of their decks. That being said, though, the values of the performance above average in the top 16 are universally high - well done!

Any players who would like to know their individual result for this metric for Qualifier 3 or 4 is welcome to DM me on Discord (Muffins#7150) - I won't be publishing them publicly!

Mass Mutation - The Pip Connection

There was a significant correlation between the pip surplus (i.e. having more pips available than the number of enhanced cards in a deck). [r(34)= .35, p= .039].

However, there was not a significant correlation between SAS or AERC and the number of surplus pips. [SAS; r(34)= .17, p= .33]. This implies that decks which had any number of surplus pips had a significant advantage over those that didn't, regardless of other metrics.

Out of 37 MM decks, 11 had a pip surplus (29.7%), of which 4 made the top 16. MM decks with a pip surplus in the top 16 made up 10.8% of MM decks played, and 36.3% of MM decks played with a pip surplus.

These were:

The question is whether these decks placed the pips in such a way as to give them a large advantage over their opponent. We checked the event footage to verify the location of the pips and see what impact they had upon the games.

Couteeyr the Annoyingly Aegian - 1 surplus pip

  • The surplus here was used to assign 2 capture pips to Observ-u-Max. With the double Transporter Platform, this means recurring the capture effect 4 times in one turn across any friendly creatures, if the stars align. Use of this can be seen at 59 mins on the event Twitch vod.

  • In theory this was highly impactful - however, on the footage I was able to find, the Observ-u-Max was used to upgrade an ANT1-10NY, which cancelled out the additional capture benefit.

Vacuous Pierce of the City - 3 surplus pips

  • The main beneficiary of the surplus icons was a Keyfrog, which was enhanced with 2 damage pips. This is excellent, as it allows the user to forge a key at cost instantly. (See VOD at 5h54m)

  • One point of the surplus was used to enhance a Growth Surge with 1 damage pip and 1 capture pip (see VOD at 5h50m)

  • The last surplus pip is used to upgrade a Securi-Droid with a draw pip and a damage pip. (See VOD at 5h45m)

  • The Keyfrog upgrade is enormously impactful and entirely changes the card - rather than having to time its destruction well, it effectively becomes an action and resolves its destroyed effect immediately as it is played.

Demoledor el Somnoliento - 3 surplus pips

  • This player used the entirety of their surplus on Bo Nithing, which ended up having two aember pips, a capture pip and a draw pip.

  • These upgrades, combined with Safe House, made it a very low risk but high impact choice. This card was never purged during the tournament.

Spitepeace, Spawn of Triumspinax - 3 surplus pips

  • One pip of the surplus is added onto Diametric Charge, which has two damage pips. (See VOD at 7h57m)

  • One surplus pip is attached to Ancient Power, which received a capture pip and a draw pip. (See VOD at 7h47m)

  • The last surplus pip was applied to Axiom of Grisk, which was upgraded with a capture pip and a damage pip. (See VOD at 7h54m)

  • This player appears to have spread out their pips, potentially as a risk mitigation strategy against cards like Infurnace and Mindfire. Placing both capture pips on the Ancient Power would have been extremely effective, but also a massively high-value target.

Given this information, it is clear that players who had a surplus used the opportunity of placing their own pips to create extremely powerful cards which gave them a significant advantage.

In terms of placement, here's how these four examples did:

The caveat to this analysis is that I haven't looked at the individual cases for the MM decks with a pip surplus which did not qualify for the top 16 - there may have been more egregious cases of optimal pip placement in that sample. I don't believe the event was a foregone conclusion for players who drew MM decks with pip surpluses, but I believe the data suggests it was a very influential factor.

The Mistake

Eagle-eyed readers will have noticed that the number of decks in the deck pool stats is 79, while I noted there were 80 players. For a reason I can’t really identify, two players in this qualifier were accidentally assigned the same pool of decks. We’re going to have a deep dive into their choices and journey throughout the event.

These were the decks both players had to choose between:

  • Biggs, Sognatore Curioso: This deck has a huge amount of disruption and 14 aember pips, which should make it consistent enough at high levels.

  • Quill of Hippocart Lair: This deck has similar aember control (A) and expected aember (E) values to the previous deck, but contains two copies of Miasma and a Too Much To Protect to stall and threaten your opponent, as well as Brend the Fanatic and plenty of ways to destroy him for the bonus aember.

  • Tessier le fougueux: While this deck has decent speed and disruption, its A is 3.9 and E is 8, which are both incredibly low.

  • The Virus that Sublimates Dinosaurs: This deck has excellent board control (particularly against non-MM decks with the two Dark Waves) and definite burst potential with the 2 Cleansing Waves. However, there is a significant amount of antisynergy between the board wipes and the capture in the deck.

After reviewing these choices, both players chose Quill of Hippocart Lair for the event.

One of these players went 5-1, and the other 3-3 in the Swiss rounds. This is despite facing approximately the same strength decks, and set breakdown. A possible reason for this is player match-ups. I checked the performance over the average for each opponent that faced either player, which can be viewed in the table below.

After Round 1, player 1 (5-1) faced opponents which consistently performed slightly above average (37.05%). The strength of player 2's opponents oscillated dramatically after Round 1, and despite having a worse record going into later rounds, the opponents they faced performed much better against the average than player 1's. This leads me to believe match-ups are a huge factor in tournament performance, and just another reason why the numbers being big doesn't equal an auto-win.

Next, I'll look at the statistics for Qualifier 4, before gathering my thoughts on both events.

Qualifier 4 - 5PM GMT / 12PM EST

Pool Analysis

98 players participated in this event, which made Qualifier 4 the largest event the KFPL has ever ran. This made the pool a total of 392 decks. As the pool was larger, it was perhaps inevitable that more high-SAS decks would be caught by the net, and this was confirmed by the existence of several 80+ SAS decks in the pool. Despite this, the median SAS in each set was roughly similar to those in Qualifier 3: the AoA and CotA medians were the same, the MM median was lower, and the WC median was higher. One major difference between the pools are the upper and lower SAS limits of each set - the CotA decks have a much wider range of quality (48 - 83, versus 51 - 76 in Qualifier 3), and the WC decks have a larger upper limit (81 versus 77). Asides from the MM decks, which have consistently higher lower limits in both events, the lower limits of each set is roughly the same. MM is consistently more highly-rated by SAS in this pool, but lower upper limits.

Played Decks Analysis

The proportion of each set played is almost identical to those in Qualifier 3. The main difference here is the reduced number of AoA decks played, probably due to being lower in SAS overall and having a much lower upper SAS limit. The popularity of CotA and WC as choices reflects their comparatively high upper limits.

The data confirms this, showing that the highest extremes of every set in the pool were chosen for use in the event. WC decks were mostly chosen if they exceeded 70 SAS, while players who chose decks from other sets were much more forgiving towards the values of their decks. Interestingly, the range of SAS in the MM played decks is identical to the range of those in the general pool, but the median of MM decks played is lower than the pool, meaning these were worse than the average available.

Swiss Performance

Like in Qualifier 3, there was a significant correlation of deck SAS with the number of Swiss wins [r(96)= .29, p= .0036].