England always fails on the big stage! Really?
TL;DR: England and Spain are the worst underperformers in Europe in big tournaments, while Portugal, Germany and Italy are true “tournament teams”. In South America, Peru and Uruguay exceed expectations. In general, it is pretty tough to define what “failure” in big tournaments means.
Before I start, a quick side note: This is not a simple translation of this post, even it’s about the same topic. The frame is very different: There is a (I think very German) idea of a “Turniermannschaft”, literally “tournament team”. This is usually used to admiringly describe the German national team and means that no matter what they always perform well in the big tournaments. In England, this idea is turned on its head, and the question is “Why does England always fail at big tournaments?” (see here, here, here, here, or here. Wow, they really like their doom and gloom.)
Of course, when I read both of these claims (“Germany is a true tournament team, England always fails at big tournaments”), my first reaction as a true skeptic is always “Probably nonsense”, followed by “Hmm, how do I find out if that’s actually true?” and finally “Shit, not as easy as I thought.”
Anyway, it’s worth a try! First, I need to define what constitutes “failure” (or being a “tournament team”, respectively). For me, that’s performing worse (or better) than expected. Now it gets a little more difficult: How do I quantify that? I need some way to measure (reasonable!) expectations, the actual tournament result and a way to compare those two.
Step 1: Expected tournament result
In my last post, I have described 3 possibilities to evaluate national teams: Betting quotes, Elo ranking, FIFA ranking. For our purpose, betting quotes are not useful (thanks Philipp for the comment!) as being a tournament team etc. should already be priced in (under rational expectations). That leaves Elo and FIFA ranking, but I could only get historical data for the FIFA ranking (if anybody knows how to get hostorical Elo data, drop me a line!) The FIFA ranking data is available since 1993 with (almost) monthly rankings. From this rank, I can compute the expected result (champion, runner-up, semi-final and so on). Each team then gets an “expected rank”, with the same rank for all teams who were kicked out in the same round (e.g. quarter final).
An example: Before the 2002 World Cup (May 2002 to be exact), England was 12th in the FIFA ranking. Among the World Cup participants, they were 9th (so several teams with a higher rank did not qualify). Their expected result was thus exit in round of last 16, their expected rank 12.5 (everybody who lost in the round of last 16 gets the same rank of 12.5) .
Step 2: Actual tournament result
That’s quicker: If a rank is fully determined (always winner and runner-up, 3rd and 4th place in World Cups and Copa Americas), this is the actual rank. Otherwise, it’s the average of all ranks, i.e. 6.5 for everybody who lost in the quarter final (average of ranks 5, 6, 7, and 8) and 12.5 for everybody who lost in the round of last 16 (average of ranks 9, 10, 11, 12, 13, 14, 15, 16). In our example, England lost in the quarterfinal to Brazil, so their actual rank is 6.5.
Step: Calculation of deviation
Before I show the results, a comment on the data used: I used data from all World Cups, European Championships, and Copa Americas since 1994. In total 19 tournaments with all national teams that participated in at least one of those tournaments.
Now the calculation of the deviation: My first idea was to just take the difference in ranks (actual minus expected rank). That’s problematic however, as the deviation is limited in both directions. A consistently high ranked team (like Brazil) can never overperform as much as a consistently low ranked team. So I just count how often a team has been better (or worse) than expected (as a share of all participations). This is the result for the European teams which have participated at at least 8 tournaments (WC and Euro) since 1994:
Well, England actually does fail in big tournaments quite often. In 5 of the 9 (56%) big tournaments they have played, they underperformed. Portugal is the best “tournament team”, they have exceeded expectations 5 out of 9 tournaments; followed by Germany and Italy. So the sentiment in both England and Germany seems to be true…
In South America, things are a little more surprising:
Peru has exceeded expectations in 7 of 8 tournaments (all Copa Americas since 1995)! Brazil and Argentina are of course usually top favourites (especially at Copas) so it’s very hard for them to exceed expectations. Still, disappointing overall for them!
Finally a couple of comments on the weaknesses of this analysis:
- The data base (since 1993 only) is pretty thin. I would have liked to go back a little more.
- The FIFA ranking is probably not the ideal tool to measure a team’s strength and through that expectations. I would have preferred Elo, but did not have historical data.
- The deviation measure used does not allow differences in how strong the deviation was; it’s only better, equal, worse. Finishing 4th instead of 3rd is as bad as not getting out of the group stage as the title holder (hello, Spain!).
- If you do not qualify at all, this is not counted as good or bad. Makes sense on the one hand (it’s about tournament performance after all), but kind of favours strong teams who fail in qualification from time to time (hello, Holland!).
Still, I found it super interesting to see how often used cliches hold up when you look at the data. What do you think? Does England always fail on the big stage?