Wide Receivers are a Pain to Evaluate: Part I

My original plan for a post this week was to talk about college football wide receivers and who I think is having the most productive season in college football right now.  But I ran into a problem.  The problem is that wide receivers are a giant pain to evaluate.  In fact, it gets incredibly frustrating to evaluate and project the performance of wide receivers.

Dependent Variables

The first question is what to use as an evaluation metric.  Before you can begin to predict “performance” in any useful way, one has to settle on what “performance” means.  Do you want yards, touchdowns, yards per reception, yards per target, touchdowns per target, fantasy points, what?

The question of dependent variable is crucial to understanding every analysis that comes afterward.  All the conclusions drawn from analyses done will only be relevant to the dependent variable that one chooses.  Therefore, it is crucial that one choose the “right” dependent variable.  Sadly, there is little consensus regarding what the right variable is to use to evaluate wide receivers.  So you’re stuck having to simply pick one.  I pick a witches-brewed version of yards per target. It works for me, but it may not work for you.

Depth of Target

Once you’ve “solved” your dependent variable problem, you run headlong into another one.  Generally, whatever DV you choose is somehow correlated with depth of target.  Wide receivers that get thrown to farther down the field rack up more yards, generally get more touchdowns (that one is a bit tenuous, but I digress), have more yards per reception, and more yards per target.  So now you’re stuck with a problem of understanding how the wide receiver fits into the offensive system regarding average depth per target.  Does this receiver have low yards per target because they are not a particularly good receiver or because they are consistently being asked to be a “chain-mover” out of the slot?  This calculation is impossible in some circumstances and tricky even with witches-brewed data.

Small Effect Size

You know what actually predicts production at receiver? Targets. End. This is a graphic I made showing the relationship between targets and yards in college football.

Targets accounts for around 75-80% of the variance in yards, which means that there isn’t much variance left for differences in ability to do any work.  You could have a pretty decent receiver buried on a roster and they won’t look like much at all *cough cough Jarius Wright cough cough*  And the reverse is also true.  A relatively poor receiver could get a lot of targets and look like a golden god.

So, I wanted to post about wide receivers.  I ended up getting frustrated at the position and writing about my frustration.

2014 Passing Yardage Predictions – Part II

Categories: Fantasy, NFL, Statistics
Comments: No Comments
Published on: August 19, 2014

Welcome back everyone. I took last week off because some very important things were and still are happening in our country. I couldn’t bring myself to talk about a game overlaid on top of another game. It just seemed a little disconnected from the world at large. I think it’s important that we all stay reminded of the events happening in Ferguson, MO. That being said, it’s time to build an audience and nothing builds an audience like new content.

This week I finished a full set of projections for yardage totals of quarterbacks, wide receivers, and tight ends. This new model makes two major corrections compared to the one I posed a couple weeks ago. First, it corrects for 2013 injuries. You’ll note that Julio Jones has a much higher predicted yardage total in this model compared to the previous one. Second, it corrects for changes to the offensive system. I’ll get to why this is important in a minute. For ease of viewing, I’ve added a new page to the banner so you can easily check these tables whenever you need to. Remember that these yardage totals assume the player in question plays all 16 games and any coaches that have changed jobs do not radically alter the schemes they’ve used in the past. I also want to throw out a big thank you to Jeff over at thefakefootball.com for the offensive coordinator history spreadsheet that made all these projections possible.

One very important caveat before we begin. I don’t have any historical data to check these predictions against. Jeff’s data on targets doesn’t go back far enough for me to do any historical checking on how accurate this model tends to be. So, I have no idea about the uncertainty inherent in this model. We’ll all be learning this together as the season goes on. After the season is over, we’ll check them together. Isn’t science fun?

Top Projected Wide Receivers – Receiving Yards

My first list of projections is for wide receivers, and that list doesn’t come with a lot of surprises. You’ve got your Andre Johnsons, your Dez Bryants, your Brandon Marshalls and your DeSean Jacksons at the top. I don’t really see a surprise on that list until I see Josh Gordan predicted at less than 1,000 yards – assuming he plays all 16 games. And even that is understandable given Cleveland’s quarterback situation. I’ll keep the list updated as depth charts change and injuries occur.

Top Projected Tight Ends – Receiving Yards

Once again, a lot of ho-hum on this list. Jimmy Graham will lead the league in tight end receiving yards, a Detroit Lion will follow him because Detroit will still throw the ball all over the place and defenses will try to lock down Calvin Johnson, blah-blah-blah. You’ll see Levine Toilolo third on that list, but I’m not sure I buy that specific prediction. The model is assuming that Toilolo will step in and take all of Tony Gonzalez’s targets which my human brain tells me isn’t going to happen. I have left that prediction as the model reports it for accuracy’s sake, but on that one, I think we have some justification to adjust it down a bit.

Top Projected Quarterbacks – Passing Yards

I went back and checked the results I’m about to tell you three different times. As I was doing that, I anthropomorphized the mathematical equation and called it a “little dickens” for trying to trick me. But there was no mistake. The inputs I fed into the model were all correct. Furthermore, all the other top five quarterbacks make perfect sense. Most of us expect Carson Palmer, Drew Brees, Tony Romo, and Peyton Manning to have high yardage totals at the end of the season. But I didn’t expect the guy at #1 by a long shot. And so, without further ado, your projected 2014 NFL leader in passing yards – edging Peyton Manning by 98 yards – is…Houston’s Ryan Fitzpatrick.

You’d call a mathematical equation a “little dickens” too if it tried to trick you with such nonsense. After I saw it I looked up the prop bet odds on Ryan Fitzpatrick leading the NFL in passing yards and found that it’s such a ludicrous notion that Vegas isn’t giving action on such a proposition. It seems insane, but let’s keep an open mind and consider this for a second.

Once you think about it, there are several reasons why it makes sense that Ryan Fitzpatrick could lead the league in passing yards this year. First, we know something about what Bill O’Brien likes to do on offense. We know he likes to throw the football and his system is very effective at gaining yards through the air. Any system that makes Matt McGloin look that good has got to have something going for it. We also know that O’Brien provides a lot of opportunities to his best receivers and seems to be able to adapt the passing game around what he has. Second, Houston has the best receiving corps you will find outside of Denver or Chicago. From top to bottom, the wide receivers in Houston know how to get open and know how to get yards after the catch. This will be a second huge bonus to Fitzpatrick’s passing yards. Third, nobody really knows what the status of Arian Foster is. We know he’s busy trying to be the best teammate he can be, but can he still be the productive running back he once was? I have my doubts. And finally, I don’t want to count out the man himself. Fitzpatrick is a serviceable quarterback. He’s not going to take a team on his back or anything, but he’s not horrific either. There’s a reason he’s stuck around in the NFL so long.

So there you go. Lots of fairly boring expectations for receiving yards and one super out-of-left field prediction. Let the season begin!

2014 Draft Class – Wide Receivers and Tight Ends

Summer time is over for us up here in the Deep North. Those two 90 degree days were brutal, let me tell you. I’ve spent the summer cocooned in my office cooking up the latest and greatest that I can offer in predictive football statistics both at the college and professional levels.

If you’re new to the site during a football season, I’ll be talking wide receivers and quarterbacks here, both at the collegiate and professional level.

And to start, let’s talk NFL rookie wide receivers. I made some predictions about rookie wide receivers for the 2013 season and they were…well they were terrible. I spent most of my spring and summer ripping apart the model and figuring out what went wrong. I discovered two very important elements of predicting wide receivers that needed to be addressed.

#1) What’s your Dependent Variable?

One of the trickiest things about doing football analysis is figuring out exactly what you want to measure and what you want your wide receiver or pass catching tight end to do. Many people have tried to deal with this issue when it comes to pass receivers with varied success.

My approach in 2013 was to use an in-house metric I created that measures pass catching ability. This, it turns out, was a horrible mistake. The reason is that NFL teams typically use particular wide receivers in particular roles. One receiver goes deep, another goes across the middle, etc. This creates a problem because it confounds pass catching ability with depth of target. Deep passes are successful less often, but the large impact they create on the game can offset their lower success rate.

For the 2014 predictions, I have changed the DV I use to get around this problem. I start with NFL Yards per Target. Yards per Target is much less susceptible to the depth of target problem. It’s not perfect, but it’s less bad than what I was using last year. I also do a little hoo-doo with the numbers to make them more consistent year-over-year. The first thing I do is to subtract the league average Yards per Target for the given season. This corrects for changes in the passing game across seasons, rule changes, passing tendencies, etc. Next, I use a highly constrained (sort of) structural equation model to pull out the effect of quarterback and offensive system on yards per target. I call it a “sort of SEM” because the model is incredibly constrained due to the realities of the game I’m modeling. It’s so constrained that what I do cannot be called a true SEM. But the technical details are probably not why you’re here. The ultimate result is a metric I call Receiver Influence on Yards per Target (RIYPT; in my ears “ripped”). We’ll use this metric at the NFL level as our DV.

#2) Lower-Level Interactions

Issue #2 I didn’t appreciate when making the 2013 predictions is the importance of interactions. You see, RIYPT at the NFL level predicts productivity. In fact, it’s the only receiver-focused metric that predicts NFL level performance. I made a bad assumption that the same situation would exist at the college level. It doesn’t. At the NFL level, every receiver has great hands. If you don’t have great hands, you don’t get to be a receiver in the NFL. Once we account for depth of target, there’s no meaningful variance in ability to catch a football among NFL pass receivers.

This is not true at the NCAA level. You can be a college receiver without having exceptional hands. As long as you make up for it with lots of long gains, less-than-stellar hands aren’t the handicap that they can be in the NFL. To deal with this, you need an interaction term. Interactions find the receivers that have good enough hands to make it in the NFL while also having the ability to gain useful yardage on an efficient basis.

That’s enough details. Down to brass tacks. Who should we be looking at as far as success at the wide receiver position? Remember, our DV is yards per target, so our definition of success may be different than the actual outcomes obtained on a football field. You can have a large RIYPT, but if you don’t get a lot of targets you won’t gain a lot of yards (see Ladarius Green in 2013).

The full table can be found above in the web page header. My top five rookie wide receivers, according to Predicted NFL RIYPT are…

  1. Brandin Cooks – Oregon State – 2.56
  2. Jalen Saunders – Oklahoma – 2.33
  3. Cody Latimer – Indiana – 1.99
  4. Marqise Lee – USC – 1.97
  5. Cody Hoffman – BYU – 1.93

Here you can see I expect Brandin Cooks to be head and shoulders above the rest of the rookie receivers in the NFL. This is especially true now that he is a member of the New Orleans Saints. Jalen Saunders will be a stickier situation. He’s a lower round draft pick, so it will be more difficult for him to see the field compared to Cooks. Second, he has (at the moment) Geno Smith throwing to him. Past visitors to the site will know I’m not high on Geno Smith. Geno will likely improve next year compared to 2013, but I still expect him to be in the bottom fifth of the league in terms of completion percentage and passing yards.

My top five pass catching tight ends according to Predicted NFL RIYPT are…

  1. Eric Ebron – North Carolina – 1.37
  2. Richard Rodgers – California – 1.12
  3. Marcel Jensen – Fresno State – 1.04
  4. Blake Jackson – Oklahoma State – 1.02
  5. Jace Amaro – Texas Tech – 1.02

Once again, we have one prospect that is head and shoulders above the rest, that being the prospect everyone expected to be on top, Eric Ebron. The second name on that list probably surprises a few people. If you look up scouting profiles of Richard Rogers, they’re not that glowing of him. I guess we’ll see where we end up. Rodgers also ended up in a great place to succeed as a pass catching tight end – Green Bay – so hopefully we get an opportunity to see him succeed.

This wraps up our rookie preview.  Up next, we’ll predict 2014 yardage totals for veterans from 2013 data.

The Analyst has No Clothes

Categories: General Info, Statistics
Comments: No Comments
Published on: September 7, 2013

I follow a lot of scouts on Twitter.  Mostly because the nonsense they spout makes me angry and I use that anger as motivation to write.  Once in a great while, though, you find a scout that does things the right way.  Or at least the way you would do things if you had the wherewithal to actually want to do that job.  Matt Waldman is in this latter group.  When Matt talks about his process, he makes me believe he’s got something valid.  He says all the right things and avoids saying the wrong things about how he goes about his craft.  You can tell there is something important going on under the hood.  Also, he’s a hell of a writer.  I have spent days studying how he constructs such compelling sentences.

The point is I respect the dude’s work, which is why I was a little disappointed to read this article on his blog about the process of scouting wide receivers.  It’s not the wide receiver scouting part that bothers me.  It’s the part when he talks about why he is not a fan of “analytics.”

I believe analytics have value, but the grading of wide receivers based heavily on speed, vertical skill, and production is an ambitious, but misguided idea. Further the application is the torturing of data to fit it into a preconceived idea and making it sound objective and scientific due to the use of quantitative data.

That quote was incredibly depressing to read.  Mostly because the reader can so easily tell what the word “analytics” means to an intelligent, quality focused scout.  The context around the word is dripping with disdain toward the self-serving, self-interested analyst.  It seems as though the people doing “analytics” that this author has met are more interested in notoriety and getting paid than delivering an accurate answer.  He goes on to make this point.

I’m trying to do the same from a different vantage point. The more I watch wide receivers, the less I care about 40 times, vertical results, or broad jumps. Once a player meets the acceptable baselines for physical skills, the rest is about hands, technique, understanding defenses, consistency, and the capacity to improve.

I liked Kenbrell Thompkins, Marlon Brown, Austin Collie, (retired) Steve Smith, several other receivers lacking the headlining “analytical” formulas that use a variety of physical measurements and production to find “viable” prospects. What these players share is some evidence of “craft”. They weren’t perfect technicians at the college level or early in their NFL careers, but you could see evidence of a meticulous attention to detail that continued to get better.

Take a look at that second paragraph.  He talks about headlining analytical formulas in reference to physical measurements like 40 times, vertical jump, and broad jump results.  Here is the heart of the issue.  Several places doing respectable analysis (pdf here) have tested whether or not things like 40 times, vertical jumps, and broad predict wide receiver production.  That sort is test is the exact thing that analytics can bring to the table.  Statistical analysis of 40 times, vertical jump, and broad jump results will tell you very clearly if the number is in any way meaningful.  And the answer that comes back repeatedly is the answer Matt has already arrived at.  They’re not useful.  Anyone that thinks they can predict who will be a quality wide receiver based on a 40 time is wasting their breath and your time.  So are there people out there really running around building predictive formulas on 40 times?  If there are, those people should not be listened to.  Furthermore, the idea that such people exist makes me feel like a biker gang member that sees a non-member wearing his clubs rocker.

There is a right way and a wrong way to do statistical analysis.  Knowing the right way is not a trivial thing that you can just dive into without training.  Somewhere, you need to learn the correct way to do it. There are lessons to learn and dues to pay and, to hear Matt talk about his experiences, there are people walking around pretending to have the cache that simply don’t have a clue.

You can see this when you read the ESPN story about the Jacksonville Jaguars “analytics” department.  From my perspective, anyone with a brain should have been able to shred those conclusions and recognize how ridiculous they actually were.  Thankfully, someone at ESPN has both a brain and the ability to write and did it for us.  It should not have taken someone in the press to recognize how terrible that analysis was.  The basic premise of any good statistical analysis starts with the notion that the analyst is wrong.  It is then the analyst’s responsibility to work through every other possibility to find the holes.  And once you reach a point where you can’t see the holes in your own work, you give it to someone else to find holes you can’t see.

Given what I’ve seen when I hear NFL people discussing the advice “analytic” people have given them, it’s no wonder that analytics is having trouble gaining respect in NFL circles.  It seems there are a bunch of people talking to NFL decision makers whose analytic methods should be severely questioned.  If what the Jaguars and some other NFL teams are doing with numbers is considered “analytics,” I’m not sure I want to be associated with that term.

Data Drop: Before the Draft Edition

Categories: NFL Draft, Statistics
Comments: No Comments
Published on: April 21, 2013

The 2013 NFL Draft is less than a week away.  All the draft analyzers are putting in their final push to get everything in order for the big day.  In the spirit of getting everything in order, I’ve added a load of new data to the site.  Check the Draft Numbers menu for everything.

I’ve added my entire list for wide receivers and pass receiving tight ends.  I’m a bit scared of putting that up, actually.  There was a time when I said I wasn’t going to do that.  I’m scared because I don’t have a good metric for understanding what these numbers predict.  The numbers are an indication of the pass catching talent of that player separate from the quarterback and offensive scheme.  However, I don’t think the numbers will predict receiving yards or touchdowns at the end of a season.  My major project for the summer will be to connect these numbers to something more meaningful and concrete, like I have with quarterbacks.  Until then, we can just look at the nice pretty list.

One note, the tight end data refers to pass catching ability only.  There are many other factors that make a quality tight end other than being able to catch passes, most notably blocking ability.  The tight end list only ranks tight ends on the single measure of pass catching.

Career Wide Receiver Production: 7th Round Darlings

Categories: NFL Draft, Statistics
Comments: No Comments
Published on: March 10, 2013

People say that the quarterback position is the most difficult to project into the NFL.  I respectfully disagree.  Certainly, the quarterback position is difficult to project from the college to professional game, and we’re not very good at it.  The empirical evidence shows decision makers are essentially making a random guess when it comes to projecting quarterbacks.  The only reason first round picks seem to have better numbers is because decision makers expect them to be good and provide them more opportunities – a self-fulfilling prophecy.

But there are worse things in this world than randomly guessing.  You will be right sometimes if you’re randomly guessing.  More than that, if you keep guessing randomly over and over and over again – say once a year every April since 1936 – the number of times you’re right and the number of times you’re wrong should be essentially equal.  Random decisions are not the worst possible outcome.  It is entirely possible that you could be on the other side of random.  It’s possible to make decisions that are worse than what a random process could achieve.  Sometimes we could make better decisions if we started throwing darts while blindfolded.

As an example, consider how most lay people buy stocks.  Most people don’t want to lose their money in the stock market.  They want high performing stocks that are going up.  So they don’t buy a stock until it starts going up.  And the average person doesn’t want to be burned by market fluctuations, so they continue to watch the stock as it goes up more.  Once the stock gets sufficiently high, they buy the stock.  But this is precisely the worst time to buy the stock.  The stock has nowhere to go but down from this point and the investor is more likely to lose money.  The person would be much better off picking stocks at random because the information they are using is leading to worse decisions.

How does this relate to football?  Football decision makers drafting wide receivers and lay people picking stocks have a lot in common.  To be clear, I have no idea what information football decision makers are using when predicting wide receiver success in the NFL.  Others have studied this (that one is a pdf, be aware if you click on it) and found inconsistent results.  However, just because I don’t know the process doesn’t mean I can’t evaluate the outcomes the process generates.  So let’s do that.  Let’s evaluate wide receiver draft picks and compare that with what we would expect given a random process.

The Evidence

I started with another study of the Hall of Fame.  It seemed to work well for the quarterbacks, so why not go back to the well again?  Where were the consensus great wide receivers drafted?  Here is a histogram of all wide receivers in the Hall of Fame that played since 1945.  Dante Lavelli is not on this graph as the U.S. Army drafted him before the NFL got a chance.  Remember, a random process coupled with a self-fulfilling prophecy will create a J-shaped distribution.

If we delete Raymond Berry as an extreme outlier, the histogram looks like this.

Here we have our first strange finding in the wide receiver data.  For picks 1-80, the data show the expected pattern.  A nice J-shaped pattern as we get further on in the draft.  But what is going on with picks 81-120?  There is a strange increase in Hall of Fame receivers drafted at a point corresponding, in modern times, to the late 3rd through the 4th rounds.  This is our first evidence that wide receiver evaluation is not just random, but biased in the wrong direction.  However, it is far from conclusive.

This pattern is strange enough to demand a more complete analysis of wide receiver production.  Perhaps the bump in the late 3rd through 4th rounds represents the perceived value of the wide receiver position.  We can be reasonably sure that everyone wants a quarterback and is willing to draft their favorite quarterback with the highest pick possible.  However, the same might not be true for wide receivers.  It’s possible that wide receivers are not as highly valued compared to quarterbacks and are not taken with the highest draft pick possible.

To attempt to rule out that explanation, I took Berri and Simmons’s methodology that they used for quarterbacks and applied it to wide receivers.  Let’s examine career production for all wide receivers drafted between 1995 and 2009.  We will look at Career Receptions, Career Yards, and Career Touchdowns.  I chose to start at 2009 because the average career length of an NFL wide receiver is just a shade over 3 seasons.  Any receiver drafted in 2009 is already at the average career length for an NFL wide receiver.  I chose to stop at 1995 because it includes 15 years of data, which is a nice round number, and it was midnight when I finished entering the 1995 data and I wanted to go to sleep.  All data was downloaded from pro-football-reference.com.

The first graph shows the Average Career Receptions for a wide receiver drafted during each round of the draft during the 15 year time period we are looking at.  Unless otherwise specified, I deleted all players that were never credited with catching an NFL pass.  I have run these numbers keeping all players, and the pattern is exactly the same.  Remember that this analysis does not control for playing time, so we are expecting a J-shaped distribution.

Here is a similar graph showing Career Yards

And the same graph showing Career Touchdowns

In all three cases, the data from Rounds 1-5 show the expected pattern.  There are still players drafted in the later rounds that are making an impact, but the impacts tend to be less than those drafted earlier.

However, that wasn’t the really weird part.  The really weird part was what happened in the final two rounds.  I might not have thought much of it if I hadn’t run the Hall of Fame data first.  Let’s zoom in on career production only for players drafted in rounds 5, 6, and 7.

Here is Career Receptions

Here is Career Yards

Here is Career Touchdowns

Wide receivers drafted in the 6th round have, on average, less productive careers compared to wide receivers drafted in the 5th and 7th rounds.  This is true if we measure production based on receptions, yards, or touchdowns.  One-way ANOVAs using only players drafted in the 5th-7th round confirms this, all F’s > 15.90.  My home set up is not great for running ANOVAs, which is why I did this slimmed down version.  Next week I will put all players into an ANOVA and run some splashy post-hoc tests.  Until then, we’re stuck with this slimmed down analysis.

This is a crazy finding.  Compared to receivers drafted in the 6th round, receivers drafted in the 7th are working against the self-fulfilling prophecies of decision makers, coaches, and their quarterbacks.  They should, by all accounts, have worse careers.  And yet, they have the more productive careers than players drafted in the 6th round and are just as productive as receivers drafted in the 5th.  My first explanation for this finding was that receivers from non-FBS schools are more likely to be drafted in the 7th round.  That was not the case in this data.  80.95% of 5th round draftees, 72.5% of 6th round draftees, and 81.81% of 7th round draftees come from FBS schools (Remember we deleted receivers that either did not make a team or did not catch an NFL pass).  At the very least, this isn’t a question of source school .

What should we conclude from this?  I think we have some very clear evidence that talent evaluators should stay away from wide receivers.  The pattern at the top of the draft is consistent with a random process.  I can’t conclude it is random right now because I don’t have good numbers on wide receiver playing time.  However, the pattern is consistent with randomness coupled with self-fulfilling prophecies.  Given what we know about the randomness of selecting quarterbacks, there is no reason to assume that wide receivers would be any different.

The pattern at the lower end of the draft is even more compelling.  In this case, talent evaluators and draft decision makers are consistently wrong about players evaluated as 7th round talents.  Those players are, on average, just as good as players evaluated as 5th round talents and better than players evaluated as 6th round talents.

This leads me to believe that we simply do not know what makes a productive wide receiver.  Furthermore, something about how we evaluate wide receivers is leading us to make worse decisions.  If we could control for the self-fulfilling process effect, we might see lower rated receivers doing BETTER than higher rated wide receivers.  (see This Post for explanation of the correction)

Which ultimately leads me back to my two favorite predictions for this wide receiver draft class, Cody Wilson from Central Michigan and Brent Leonard from Louisiana-Monroe.  Both of these receivers had extremely productive college careers at schools that don’t get a lot of media attention.  Both of them are not rated highly by the draft community for physical reasons – Cody Wilson because he is short for an NFL receiver and Brent Leonard because he is “not fast.”  But I ask you, who cares?  Fast receivers do not do any better than slow ones.  And prioritizing fast receivers might be leading us to draft worse receivers in the high rounds.

Remember, when I started this I said I don’t know what process evaluators are using to grade wide receivers.  However, after looking at the data, I am confident that they should stop whatever it is they are doing.  The decisions being made late in the draft are worse than randomly guessing.  At the end of the day, decision makers would be better off throwing darts at the names of draft eligible wide receivers blindfolded.

page 1 of 1
Welcome , today is Monday, October 23, 2017