Scotland’s unwanted record of being the only team with as many as nine World Cup appearances never to have reached the knockout stage continues. Their fans look on enviously as other nations of comparable and smaller populations have fought on in the competition, some of them doing so consistently.
It’s a familiar feeling for Scotland at a major football tournament; months of anticipation culminating in what can only be described as a damp squib. While many of the fans have been travelling home with their tails between their legs, there are always some who jump to defend the national team, citing a population of just 5.5 million, making Scotland the David to their opponents’ Goliath.
The logic is clear: population determines the size of the talent pool and the likelihood of a country producing top-quality players, all else being equal. Indeed, having crunched the numbers from the group stages, I find that population explains around 18% of the variation in group stage performance when considered alone.
But applying this logic would have you believe that the United States are the runaway favourites to win the tournament. Clearly, all else is not equal. A host of factors determine countries’ abilities to transform a large pool of potential talent into realised superstars.
A far better predictor of performance is squad value, which explains around 60% of the variation in group stage performance. Not only does squad value explain over three times more than population alone, but when considered simultaneously, population becomes statistically insignificant at the 5% level.
Did Scotland overperform or underperform?
So, with this in mind, I aim to answer a simple question: given our squad value and that of our opponents, should Scotland have done better?
Figure 1 plots the difference in the number of points each team earned, relative to the number of points that they’d be expected to earn, based on their squad value and the average value of their opponents’ squads. Clearly, the teams at the top in green exceeded expectations, while teams at the bottom underperformed.
Figure 1. Expected points based on squad value and average opponent squad value
Note: Left = Worse than expected · Right = Better than expected
Squad value source: Transfermarkt
Scotland, highlighted in blue for clarity, underperformed by 0.53 points.
Still, when the draw was made, many accepted that the most probable path to making history by qualifying for the knockout stage was via one of the spots designated for the eight best-performing third-placed teams.
After beating Haiti 1-0 in the first round of matches in Group C, third place looked all but sealed, so attention switched either to getting that elusive additional point from the matches against Brazil or Morocco – or at the very least, to not getting hammered to give us a chance on goal difference.
Heading into the Brazil game, everything was on the line, with our goal difference poised at 0. Most projections indicated that the threshold for qualification was likely to be around the -2 mark for a third-placed team on three points, so in losing the game 3-0, we gave ourselves very little chance of making it. Ultimately, these projections turned out to be wrong in any case, as the eighth-placed team, Senegal, made it through on three points and a goal difference of +2.
Nevertheless, should we have expected to make it through as one of the best-ranked third-placed teams before a ball was kicked? No.
Table 1. Expected third-placed teams, ranked by expected points
| Rank | Country | Expected pts | Expected GD |
|---|---|---|---|
| 1 | Senegal | 4.79 | 0.93 |
| 2 | Ecuador | 4.67 | 0.87 |
| 3 | Japan | 4.35 | 0.37 |
| 4 | Austria | 4.30 | 0.33 |
| 5 | South Korea | 4.16 | 0.49 |
| 6 | Bosnia & Herzegovina | 4.04 | 0.16 |
| 7 | Ghana | 3.85 | -0.72 |
| 8 | Paraguay | 3.71 | -0.70 |
| 9 | Scotland | 3.53 | -1.21 |
| 10 | Congo DR | 3.30 | -1.63 |
| 11 | Cape Verde | 1.71 | -4.63 |
| 12 | New Zealand | 1.66 | -4.33 |
12 third-placed group finishers, ranked by expected points across the group stage. Hover any cell for detail.
As Table 1 shows, among the teams who’d be expected to finish third in their groups based on their respective squad values, Scotland would still find themselves outside the top eight if every team performed to expectations.
So, what can we make of it all?
I’m afraid, if you’re a Scotland fan reading this hoping for some sort of comfort, I haven’t got much to offer unless you take comfort in the fact that even if we’d performed to our expectations (along with everyone else), we still probably wouldn’t have made it through – and most of us would still be (at the time of writing) Norway fans all the same.
I feel similarly helpless in offering any concrete reasons to feel aggrieved. We were deservedly beaten by Brazil and Morocco by scorelines that reflected the quality of chances created.
In fact, we could count ourselves lucky to have won the game against Haiti. According to Opta, both teams created exactly the same expected goals values – luckily for us, John McGinn’s deflected strike was enough for us to edge it on the day.
Debates can, and will, be had around whether the manager Steve Clarke got the best out of his squad from a player selection and tactical perspective. I won’t enter this debate. For now, I’ll try to answer the question set out at the beginning, and hopefully inform some of the debate among fans as they conduct their own debrief of events in the States.
Did Scotland underperform?
Yes, both in terms of points and, even more emphatically, in terms of goal difference.
But we didn’t strictly underperform by not qualifying for the knockout stage, when accounting for our squad value and that of our opponents. The harsh truth is that sometimes, you just aren’t good enough – and that’s okay.
As such, I propose an answer to the initial question in this piece, which may, at first, seem contradictory, following the answer above.
Should Scotland have done better?
Not really.
In truth, it depends on what ‘doing better’ entails. We underperformed our expected points tally by 0.53 points. But of course points don’t come in denominations of 0.53. Therefore, any increased performance with respect to points would require us to overperform expectations. So, as disappointed as I am, I’m reluctant to say we should have done better.
Granted, we could have done better with respect to goal difference, which we underperformed by around 1.8 goals. But this is pretty much academic in hindsight, since, as it turned out, we’d have had to better our goal difference by five goals just to equal Senegal.
Football, as we are so often reminded by players and managers, is a game of fine margins. For us, one extra point would have been enough, as would winning more handsomely against Haiti, and avoiding such a heavy defeat to Brazil (or, possibly, one of the penalty shouts going our way against Morocco).
As it is, we’re left bemoaning these margins once more as the wait for our next chance to end the unwanted record of being the only team with as many as nine World Cup appearances never to have reached the knockout stage continues. At the same time, we look on enviously as other nations of comparable and smaller populations fight on in the competition, some of them doing so consistently.
‘Why can’t it be us?’ will be the question on many Scotland fans’ lips, particularly as tournament debutants Cape Verde (a country with a population smaller than Glasgow’s) stunned the world to make it through a tough group at their first attempt. Ultimately, some things can’t be answered with models. I’d prefer to put that one down to the magic of football.
Nevertheless, it would be nice if, one day, it could be us. Indeed, it would be nice if we didn’t have to wait another 28 years to try again – and that when we do get there again, we could avoid Brazil for once.