The cautionary tale of creating a framework for draft evaluation
I recently got a question from a Blue 58 listener by the name of James. He write:
“I’ve had this idea of late that the Packers’ modern draft approach (high RAS and little to no ‘off-field issues’ would effectively exclude some all-time greats. Maybe this is not the best case to explore this, but I fed a few into ChatGPT and got the answer below:
That’s a fascinating angle—Green Bay’s modern drafting philosophy does appear to favor high Relative Athletic Scores (RAS) and clean character profiles. Let’s go through each of the players you mentioned, estimating their RAS (Relative Athletic Score) and off-field profile, based on publicly available data and scouting reports. RAS is on a scale of 0–10, where 10 is an elite athlete. [Editor’s note: the list of players the AI spit out is discussed following James’ question.]
There does seem to be something to this, and I end up wondering does their current approach (under Brian Gutekunst) select for probabilistically good but not great?”
I think James is onto something here that’s worth discussing, especially in the wake of a draft weekend that produced a draft class with a couple of strongly defining characteristics, namely high athleticism and sparkling off-field character. (I’ve jokingly dubbed this class “The Good Kids” and I think that’s an accurate description of the guys they selected.)
The AI pulled up a few names as potential discussion points for James’ question:
James Harrison (undrafted in 2002 with a low estimated RAS and no off-field stuff)
Terrell Owens (third round pick in 1996, incredible athletic ability but potential off-field concerns (in addition to coming from a very small school))
Ed Reed (first round pick in 2002, limited athleticism (6.54 RAS), highly respected off the field)
Lawrence Taylor (first round pick in 1981, elite estimated RAS, numerous off-field issues)
Wayne Simmons (first round pick in 1993, elite RAS (9.5) but off-field issues in college)
That group runs the gamut, and to James’ initial point, some of them probably would have had off-field stuff that prevented them from ending up in Green Bay or a lack of athleticism that would have probably taken them off the Packers’ board. Harrison is probably the best example of the latter point: to say he was a lightly regarded prospect dramatically understates it, and at just six feet tall he’d almost certainly have been too small for the 4-3 defense the Packers were running at the time.
But they missed on Harrison, clearly, and who knows how many other good players a rigidly defined approach might have taken off the board throughout the years. But this is the real gist of the question: is the Packers’ approach stopping them from being successful before they’ve even made a pick?
It’s certainly possible. The reality of making a choice is that you’re not just choosing a thing in and of itself, you’re also deciding against its opposites. Every decision in favor of option A is a decision against options B-Z. Or, as Rush eloquently put it, “If you choose not to decide, you still have made a choice.”
I think the Packers’ current approach is generally solid, even if some players do slide through the cracks. Focusing on athleticism has the twofold benefit of putting players in your developmental pipeline with very high ceilings. However skilled a player might be, he’s still going to have lower long-term potential than a less-skilled player who’s an exceptional athlete. Skills can be learned, but athleticism is limited.
Gutekunst’s approach also reflects the realities of the NFL. A recent study (now lost to either me or the internet in general or both) pointed out that the average Relative Athletic Score of the typical NFL player has increased in recent years. Targeting athleticism is no longer about finding the best prospects to develop; it’s a matter of survival. Building your team out of sub-par athletes puts you behind the NFL’s ongoing arms race.
It’s true, though, that this approach has limitations. Focusing on dynamic athletes could cause you to miss out on someone who just happens to be good at football, and Ed Reed is a great example of that. Did his relative lack of athleticism ever limit him on the field? I’d say that’s a firm no.
The real conclusion here, then, is that anyone trying to make a framework for evaluating prospects has to be aware of the limitations of that framework, and they must also be willing to go against that framework at times, too. But having the framework in place is, in itself, a good thing, because it forces you to ask yourself hard questions about those edge cases. Is this guy really exceptional? Are his limitations going to be a problem? Why am I trying to convince myself one way or another.
It’s also worth remembering that these frameworks have always existed. In Vince Lombardi’s 1966 playbook, he lays out strict requirements for players at every position. Here’s what he said for the all-important ends in his offense, who were responsible for a key block in his famed power sweep:
Basic Requirements and Information:
He must have excellent fakes and sure hands - most reliable receiver - considered the shorter pass receiver and best third down man. The split end does not need the speed that the wing must possess, however, his speed should be above average. He needs sufficient intelligence to understand defenses - he must be rugged enough to execute the “crackback” block on the linebacker - occasionally he will tighten and have to perform a head-on block or angle block, naturally the greater his pass catching ability, the less will be his blocking requirements.
Speed: 4.8 seconds for 40 yards in shorts and football shoes.
Now, this isn’t as rigid a scouting report as what you might expect in the modern NFL, and even scouts of the era would probably have some more well-defined characteristics for a given position. But make no mistake: this is still a defined framework for identifying a player that fits with what Lombardi wanted to do, and there would have been some good players who went by the wayside as a result of doing things this way.
Having the framework is the important part, though, as is knowing when to disregard it. As long as the Packers and Brian Gutekunst are able to check both of those boxes, things will probably break in their favor as often as not.