Player Performance Benchmarking
Arne Engels- Celtic's record signing used as a case study to explain this exercise
One of the transitions being made as we launch this Substack is to migrate and consolidate content which was primarily distributed via Alan’s blog and Twitter to this platform.
In addition, this venue also offers the opportunity to provide more robust granularity vs Twitter, which is why we have dedicated a specific section to the player benchmarking exercise I have been sharing on Twitter over the past several years.
As has been stated with these posts in the past, these are intended as an exercise with which to try and ascertain a profile of players, with a goal to look for potential signals of potential relative strengths and weaknesses. Generally speaking, the exercise has yielded pretty good accuracy, but it is just a starting point for analysis.
The basis for the exercise is derived from the concepts within performance benchmarking, contribution, and attribution analysis which is endemic within the investment industry.
I will use Arne Engels to flush out various issues and concepts, as his profile is a complicated one. His tenure at Augsburg was playing at a club with relatively low resources in a top 5 league, and during his age 19 and 20 seasons.
Each metric is sourced from Wyscout or derived from other metrics within that platform:
Let us review how each of the three terms mentioned above are being used and what they mean within the context of these analyses:
Benchmark - the selection and construction of benchmarks are of vital importance if they are to be useful within an analytical process. My mentor taught me to consider this issue with a simple framing - align the benchmark with the question one is looking to answer.
For example, one may be looking to gauge how a player performed relative to a peer group driven by league, age range, position profile, style of play of the team, role within the team, etc. There is no right or wrong benchmark construction - only whether they are logically aligned in order to try and make analytical inferences. In addition, it is important to consider what a peer group DOES NOT measure.
The graphic above addressed the simple question - how did Engels perform relative to his Bundesliga peers that played at least 1,500 minutes in each of his two seasons?
It does not account for his age, where he may have been within his own development curve, both physically and cognitively, etc. The fact that he was 19 for most of the 2022-2023 season is of huge importance, but a different benchmark could be constructed in order to try and answer that different question: how did Engels perform for a player his age playing in a top 5 league?
Contribution - this term can be thought of as taking the level of performance within a team and then looking at how that is arithmetically carved up amongst players. The easiest way to explain this is to use simple counting stats such as minutes played, or passes made.
Contribution-based analysis is anchored in trying to account for objective reality - i.e. accurately measuring what actually happened during a game. If a game is 95 minutes in duration, and a player is on the pitch for 68 minutes, then he will have played 68 out of 1,045 total minutes for his team, or 6.5%. Simple! This is what I see most commonly referenced as “stats” within popular discourse.
Attribution - this is a term which introduces far more of the artistry of critical thinking, problem solving, inference, deductions, etc. that are involved within the domain of analysis. Second, third, and fourth order thinking, similar to the difference between playing chess vs checkers, is where real value can be added to analyses.
For example, the Expected Trophies Revisited column laid out the basics of the broader attribution framework I use to try and analyze football clubs. It takes into account relative financial resources, how they manifest via talent within squads, quality of talent acquisition, managerial decision making, and good old fashioned luck, both within the context of controllable and uncontrollable varieties.
Now back to analyzing Engels’ pre-Celtic performance levels. Prior to his move to Augsburg, he played for Club Brugge II in the 2nd tier Belgium Challenger Pro League.
Dear reader, you will be forgiven for not being acutely aware of the nature of that league, and I am only superficially so after doing the following:
The left hand column was Engels’ season with Club Brugge II for the 2020-2021 COVID season, back when Brugge’s was the only reserve team in the Challenger Pro League. Subsequently, three other larger Belgium clubs now have II’s in that league.
For whatever reason, whether injuries, developmental, etc., Engels’ minutes within Wyscout data were very limited for the 2021-2022 season and in the U19 UEFA Youth League team - hence my ‘skipping’ it.
For the 2020-2021 season, the average age of players in the league was 28, with what would be the bottom of the table Brugge side with an average of 22. As a reminder, Engels played most of that season at age 17, so similar to his subsequent German Bundesliga seasons, when he played at ages 19 and 20, was young for his level.
That qualitative factor is important to keep in mind when considering the benchmarking exercises listed above. The Club Brugge side not only finished bottom of the table, but had a goal difference of 23 scored vs 64 conceded over 28 matches on about 47% possession. The Augsburg teams in which Engels played were also not possession dominant nor front footed in their style of play. All three seasons were playing for sides that were more apt to play ‘long ball.’
So what the hell good is this exercise given all of these complicating factors?
The first obvious issue which stands out via this simple exercise is that Engels just has not played much senior football. In fact, the total amount of minutes over the past four seasons is on par with what Callum McGregor played across all competitions in each of the seasons from 2018-2019 through 2021-2022.
How will Engels’ body and mental strength endure when placed into the meat grinder of twice-a-week games playing an extremely high tempo and demanding style of play at Celtic?
Much of the rest of his profile are more nuanced given the defensively anchored and more long-ball natured style of play for the sides in which he played. On the positive side, for what was still a boy and at best a young man, he appears to have held his own physically playing against full grown men. Particularly, his Pro League season in 2020-2021 benchmarked ok even at age 17.
However, holding one’s own is not the same as performing well on an absolute basis, as was reflected in his elevated foul rate combined with relatively low success rates across the various types of duels.
Now for the potential red flag within the profile - i.e. what could be a signal extracted from all of this noise. Note his relatively poor rankings for passing accuracy for short/medium passes over all three seasons. That is not a metric which should be heavily impacted by the issue of a long-ball style of play, as the forward pass accuracy could be. Speed of thought and transmitting it into effective actions via technical abilities can be broken down into those various elements.
By contrast, a player like Matt O’Riley displayed 90%+ percentiles across the wide breadth of passing-related metrics at a similar age prior to arriving at Celtic. This could be an area of relative weakness for Engels, which could manifest more so against bunkered opposition or as the skill and athleticism of opposition defenders shrink the time and space available to make decisions and execute.
His profile also indicates a player with an appetite for making more aggressive passes, but with a very mixed record on doing so effectively. Much of his attacking utility within his data reflect crosses and being a primary set piece taker, which we have already seen via his prowess during his short stint at Celtic.
But what about attacking output and ball progression from open play?
Those are hugely important aspects of an attacking midfielders’ role playing at Celtic, and particularly on cold winter nights at away grounds versus sides content to play for a nil nil draw. In addition, as we have seen with Reo Hatate at times in Europe, aggressive passing that does not come off can come at the cost of opposition getting chances to attack Celtic in transition via counter attacks.
One of the benefits of conducting this sort of exercise is that it typically generates as many questions as answers, with impetus to conduct further analysis focused upon answering those questions. But I am left with what appear to be unknowable questions given the specific nature of the teams for which Engels played.
For example, given the potential flag with his short/medium passing accuracy, what will be the impact of materially increasing his volume of involvement via receiving passes at Celtic domestically vs what he has experienced to date?
His highest average passes received for a season has been 23, where as Hatate and O’Riley averaged 46 and 37 in league games at Celtic?
Still only 21, while the Scottish Premiership is many levels down from the German Bundesliga, and particularly so for a player at Celtic, it is a very fast paced and physical league. How will Engels perform and endure within that context over what promises to be a very long season?
To place this potential issue within a statistical context, Wyscout has a metric called Challenge Intensity, which they define\ as follows:
A team metric, quantifying how many defensive actions (defensive duels, loose ball duels, interceptions, tackles) a team is doing per minute of opponent ball possession.
Challenge intensity reflects how often the team is actively trying to recover the ball when the opponent is in possession. The higher this number, the more intense the team is in challenges.
The average Challenge Intensity for the Bundesliga teams last season was 5.27. The English Premier League was 5.18, Italian Serie A 4.83, Spanish La Liga 5.04, Dutch Eredivisie 5.03, and Austrian Bundesliga 5.63.
The Scottish Premiership average for the 2023-2024 season was 6.2. In fact, despite the reputation as a comparably fast/physical league, the English Championships average was 5.20. The Scottish league even eclipsed the 6.15 average for English League One on the season.
With Engels being Celtic’s record signing, it is my analytical conclusion that the risks associated with these unknowns are material. This is not to suggest he is incapable of, or unlikely to develop sufficiently, in order to be a high level, and perhaps even dominant, performer at Celtic.
I simply do not see much objective evidence that he has the all-around skillset one may expect for the club’s record signing. He may, and I certainly hope he does…but that would be different than what this similar exercise suggested about another rumored target, Mathias Kvistgaarden at Brondby, for example.
Hopefully, this introduction to this benchmarking exercise will provide readers with the basis and context for analyzing new Celtic signings within this venue. In addition, as someone who spends entirely too much time analyzing the main threat to Celtic in what has effectively become a two-horse race, I’ll also be sharing similar consolidated analysis on Rangers’ respective transfer windows.
He's been impressive so far but in a barnstorming type way. Definitely seems to be the definition of box to box, with very good set piece delivery and some snippets of nice through balls. But the slackness in short passing has been evident, dulled by the fact the opposition have been nowhere near good enough to take advantage. He's lost the ball with his back to goal a few times (as has Bernardo). Are their figures on this like pressing and press duration?
Few questions:
1. On what basis is he valued at £11M
2. Why the feck did we pay it
3. Given O'Reilly's exceptional stats, why were no 'smart' clubs looking at him? He wasn't even Celtic's first choice.