Last week, it was my pleasure to hand plaques to 27 NFL scouts who were voted by their peers last fall to the 2023 BART List, our honors system for the best evaluators working in the league these days. It’s the only way I know to recognize the best people in the scouting industry.

Some, however, may wish there was a more objective measure. I’m one of those people. I’ve even seen that some teams, apparently, do just that for their own scouts. I really like that teams are working to identify the best members of their staffs. 

I’ve often thought of how to measure the effectiveness of scouts across the league. People who believe in analytics feel you can put a number on anything, right? But I keep bumping up to the following reasons that evaluating the evaluators is next to impossible.

Each team scouts for its team, not the league: We would have to have complete records of the grades given by each scout to players drafted and signed, every year, to even get a baseline of his effectiveness. Even at that, we could only hope to determine the best scout for each team, not for the league. Until every team runs the same defense and same offense, this is impossible.

A player’s success is not solely based on where/when he was drafted: To some degree, a draftee is a lump of clay. That lump gets shaped by the team’s coaching staff once he’s drafted. Some teams develop players better than others, and some teams have more “rope” to develop them for various reasons (usually, prior success). Therefore, just because a player worked out doesn’t necessarily mean the scout did his job.

Hungry players play better: If Tom Brady gets drafted in the second round instead of the sixth round, does he still wind up the same player? Maybe. Still, it seems to me unmistakable that players selected lower than they expected tend to play harder, while sometimes, players drafted earlier take the game less seriously. Obviously, this is hard to measure, but seems evident, nonetheless. Either way, it has an effect on a player’s grade. 

Obviously, there are other factors. Teams that have great depth at a position might mean a good player never gets a chance to really prove himself. Players drafted higher than expected may nonetheless get multiple chances in the league due to their draft pedigree. I could go on. The point is, the NFL draft is hopelessly subjective. I guess that’s one reason it’s so fun.

Do you disagree? Feel like it’s easier than I’m making it look? Give me some ideas on Twitter/X at @insidetheleague. Make sure you’re also reading our newsletter to get more insights on scouting and how teams evaluate players.