What Exactly Do You Know, Part II: The “Is He Your Cousin or Something?” Edition

Comments: No Comments
Published on: October 21, 2014

In last week’s post, I discussed the concept of what statistical inference actually tells you and how it’s boring and cumbersome to talk about it accurately, so analysts often shorten the conversation so they can actually talk with real people about something interesting. Today we take a slightly different tack regarding what exactly we know. Our example for this week is Minnesota Vikings quarterback Christian Ponder.

Christian Ponder close-up.jpg

If you’ve been reading for a while, you know that I was actually a fan of Ponder for a long time. Or, at the very least, I didn’t hate him with every fiber of my being like every other Vikings fan seemed to. I called him “not the problem in Minnesota” instead pointing to the largely ineffective receiving corps. I was talking to my neighbor before the season started. I said that Ponder is not the problem. We had this long and somewhat loud conversation about how I have to be wrong about him because everyone was giving up on Christian Ponder. Even Paul Allen – the radio play-by-play announcer for the Vikings – a guy who has never in his life given up on anyone in a purple jersey had given up on Christian Ponder. When I persisted that Ponder wasn’t the problem, my neighbor ended the conversation by saying, “You’re the only guy I know saying nice things about Ponder. Is he your cousin or something?” At the time the comment made me laugh. Then the Thursday night game against the Packers happened. I had to think more about this and examine what I know and what I don’t know about Christian Ponder in particular and the game of football in general.

So why was I so adamant that Ponder wasn’t the problem? Because, for all his faults, Ponder has one singular but important ability. He is rather accurate for an NFL quarterback. He’s not super-star Peyton Manning accurate, but he can get a football into a receiver’s hands slightly better than the average NFL quarterback. And why do I care so much about accuracy and nothing else? Because it’s the only quarterback ability I’ve found at the NFL level that will predict useful outcomes. Nothing else comes back predictive. Not a quantification of arm-strength, not Wonderlich scores, nothing at the combine, nothing but accuracy predicts NFL level outcomes.

And now we have another trap that analysts can fall into, a trap that is particularly present and meaningful for the NFL. I can’t find a predictive effect of my in-house metric that I think measures arm strength (let’s ignore the measurement point of “how do we know this thing is really arm strength” for now. It’s important but not where we’re going here). So I don’t find this effect. There are a couple possibilities why. The first possibility is the one that brings the page views and the loud conversations – that Arm Strength isn’t an important thing. However, another interpretation is that the lack of data at the NFL level makes finding the effect of arm strength insanely difficult.

Think about it like this. Imagine I told you that there was gold to be found in the body of water closest to you. To me that body of water is a river, so for the rest of this example I’ll be talking about a river. But maybe for you it’s a lake or an ocean or your friend’s bathtub. Whatever. You want to find this gold because you think having gold would be better than not having gold. So you go out and buy all the equipment necessary to pan for gold. You get the sorter pieces and the dirt sucker and everything else and you go stand in the river for a few hours and try to find this gold. Now, if you stood in the same spot panning for gold for four hours and didn’t find gold, would it be reasonable for anyone to assume that I’m wrong and that there is no gold in the river?


Crude Drawing of Where Gold is in a Fictional River
Crude Drawing of Where Gold is in a Fictional River

No, it would be ridiculous to say that. Maybe you were panning in the wrong spot. Maybe the screen you were using was too big and all the gold was little and slipping through. There could be many reasons why you didn’t find gold in the river.

Analytical findings are like gold. Just because you don’t find one, doesn’t mean that they aren’t there. This is a concept called “statistical power” and in the NFL it’s a huge problem. Our ability to find effects generally increases the more data we have. Think of it like this – more data makes our gold panning screens smaller. It allows us to find ever smaller nuggets of gold. In the NFL, the data is very sparse. There are only 32 teams playing 16 games each with maybe 30 passing attempts in each game. This pales in comparison to basketball’s 82 games and baseball’s 162. Compared to other sports, an effect in the NFL has to be fairly large before our screens will catch it. There is so little data coming from the NFL that it’s possible an arm-strength effect exists but there just isn’t enough data to find it.

So, after the Thursday night Ponder debacle, I went on a quest for more power. And in football, if you want more statistical power you need to look at the college level. With many many more teams we suddenly have a lot more power in our data set. I spent most of my summer calculating the same arm-strength metric for every NCAA FBS level quarterback and I ran the same model to see if arm-strength, along with accuracy, can predict useful quarterback outcomes. Low and behold, it does (said the amazed analyst and no one else). Ponder fairs very well on accuracy, but he suffers horribly on arm-strength. With this lesson learned, it’s time to quit dying trying to take the Ponder hill. Ponder is a problem for the Vikings offense. One of many, many problems.

Quarterbacks in Minnesota

Categories: Fantasy, NFL, Statistics
Comments: No Comments
Published on: October 16, 2013

The Vikings recently announced that Josh Freeman was going to be their starting quarterback.  This move really shouldn’t surprise anyone given the problems the Vikings have in the passing game.  However, is Josh Freeman “the answer” in Minnesota?  What can we expect from Josh Freeman now that he is the man of the hour on this endless carousel of starting quarterbacks in Minnesota? 

First, you’ve probably seen that Josh Freeman has an NFL low 45.7% completion percentage this year.  We care about completion percentage because it’s one of the few quarterback stats that travels reasonably well.  However, I am confident that Freeman’s completion percentage will go up as a member of the Vikings.  When that happens, people will tell you that it is because of the toxic relationship between Freeman and his coach in Tampa.  They will tell you that sometimes a change is scenery is necessary for a quarterback to get better.  These story lines are all nonsense of course, but people will tell them to you because so many are willing to believe them.  What’s actually going to happen is a much less interesting story called regression to the mean.  Extreme scores tend to be followed up by less extreme scores and performance tends to move toward the averages.  Josh Freeman has a career completion percentage of 58.2% and we would expect his completion percentage this season to move towards that number (This will be important later).    

We can also expect that Freeman’s completion percentage will go up because he moved from Tampa to Minnesota, but not by much.  Given the data we have, we can expect an increase in his completion percentage of about 1%.  This effect is statistically reliable, but not particularly meaningful because it’s going to be swamped by the change due to regression to the mean.

Let’s get to the heart of the question.  Are we going to see more production out of the Viking’s passing game now that Freeman is the starter compared to Matt Cassel or Christian Ponder?  We can actually answer that question better in this specific case than most times.  At one point or another last year, all three of these quarterbacks had starting jobs.  That means we have reasonably good data on production levels for each one.  This really good data gives us some idea of how well each would do in a starting role this year. 

The short answer is no, the Vikings are not better off with Josh Freeman starting.  In fact, I am projecting we will see much much less out the Viking’s already anemic passing attack now that Freeman has been placed in the starting role. 

Here are the numbers.  Pre-season passing yard projections are based on a regression equation that uses statistics in Year A to predict production in Year A+1.  It accounts for 33.5% of the variance in passing yards and has a standard error of the estimate of 314.9 yards.  This model assumes similar production and usage from 2012 to 2013.  Be aware that this model tends to underestimate yardage totals because yardage totals are not normally distributed.   

Some take away points from that table

·         I would not have recommended signing Greg Jennings.  At one time he was a productive wide receiver, but that production has faded with age.

·         Where has Jarius Wright been this season?  He had some good times in 2012.

·         Of the three quarterbacks currently on the Vikings roster, Ponder was the best choice preseason.  I’ve said it before and I’ll say it again.  Ponder is not the problem.  The problem for Minnesota’s passing game for the last three years has been and continues to be a wide-receivers-not-named-Percy-Harvin problem.

The decision to sign Freeman is not looking particularly good here.  However, the previous projections come from 2012 data.  We also have data from 2013 on all three quarterbacks.  Admittedly, at most it’s two or three games for each quarterback and five games for each receiver, but it’s at least something.  Let’s take what we know about each quarterback and receiver’s production and usage for 2013 and input that into our model instead of the 2012 numbers.  Here are the projections showing what I would expect with each of these quarterbacks throwing to these five receivers for a full season.

* Because Freeman’s completion percentage is so abysmal this year, the prediction of Freeman to Wright actually comes out negative. Freeman’s line has been changed to assume he will increase his 2013 completion percentage by 10% as a member of the Vikings, which would be more in line with his historical average.

Here are those same projections pro-rated for the 11 games the Vikings have remaining on their schedule. 

* See note in previous table

So there you go.  According to these projections, the Vikings paid $3 million to lose more than 1,000 yards of production through the air compared to what we would expect if Cassel or Ponder were still the starter.  Go Vikes.

A Viking, a Jet, and a Mountaineer walk into a bar…

As I was watching the Vikings-Packers game last weekend, a strange thing happened.  I agreed completely with Troy Aikman.  He was talking about the Viking’s struggles in the passing game and mentioned that the Viking’s receivers were simply not winning the one-on-one battles.  This meant that Christian Ponder had nowhere to throw the ball and therefore couldn’t complete any passes.

I’ve already mentioned my take on the Viking’s problems, and that assessment hasn’t changed.  With Percy Harvin out for the season, the Viking’s are likely to struggle in the passing game for the rest of the season.  However, the fault won’t be with Christian Ponder, who is still above average in terms of completing passes to the Viking’s set of receivers.

What about the newest quarterback controversy in New York.  The Jets are under considerable fan pressure to bench Mark Sanchez.  What does the model say?  Is Mark Sanchez to blame for the Jets’ struggles?  This question is more difficult to answer than you might think, but only because of historical coincidence.  My model relies on historical data to observe how performance changes when receivers, quarterbacks, and offensive systems change.  Mark Sanchez and the current Jets coaching staff have never been separate from one another.  Rex Ryan’s staff has never had a different quarterback and Mark Sanchez has never worked within a different offensive system in the NFL.  So, while the model can identify that the problem in New York isn’t a receiver problem, it can’t separate the quarterback from the Jets offensive system (if anyone knows of a good site for seeing historical lists of offensive coordinators and play callers, please let me know in the comments).

What does this all have to do with a Mountaineer?  Well, it’s a lot easier to separate offensive system from quarterback in the NCAA because the quarterbacks turn over so quickly.  This turns our attention to Geno Smith, the current consensus #1 pick.  How much of Geno Smith’s success is due to the West Virginia offensive system?  How much is due to Geno’s quarterbacking ability?  Sadly, the answer is that everything that makes Geno Smith an above average NCAA quarterback is attributable to the West Virginia offensive system.  Geno Smith’s Career CAA is only 3.78.  In his entire NCAA career, he has only completed 3.78 passes that an average NCAA quarterback wouldn’t have completed.  The rest of the success comes from the coach.

I find it strange that no one is talking about this, given that Dana Holgorsen made a name for himself as an offensive coaching specialist.  My model says that there is nothing unique about Geno Smith.  Any average NCAA quarterback dropped into the West Virginia offensive system would perform just as well.  I hope someone with decision making power recognizes this before heaping a load of expectations onto him and turning him into the next JaMarcus Russell.  The new rookie wage scale will limit the damage, but Geno Smith is not likely to be the answer for the Kansas City Chiefs or any other team in the market for a quarterback.

A Tale of Two Quarterback Controversies

Categories: NFL, Statistics
Comments: 1 Comment
Published on: November 7, 2012

We’re halfway through the season and quarterback controversies are brewing.  I want to compare and contrast two quarterbacks that have been struggling recently, Michael Vick in Philadelphia and Christian Ponder in Minnesota.

Right off the top, quarterback controversies halfway through the season are wildly overblown.  The differences among NFL quarterbacks are actually very small and not larger than the effects of random chance.   That being said, we have enough data to begin to understand the different reasons these two quarterbacks might be struggling.

Let’s start in Philadelphia.  Michael Vick is taking a lion’s share of the blame for problems in Philadelphia’s passing game.  The question is, how much is Michael Vick to blame for Philadelphia’s problems?  Well, our handy-dandy math equation tells us that Michael Vick is a below average quarterback this year.  However, he’s not that below average.  The model tells us that Michael Vick has failed to complete 3.9 passes that an average quarterback would have completed, less than half a pass per game.

So while Michael Vick is below average, he’s not so below average that a quarterback change is warranted.  He’s certainly not Mark Sanchez below average (who has failed to complete 22 passes an average quarterback would have completed).  Sanchez does have an advantage over Vick in this area though.  Sanchez has the only backup in the league expected to be worse than he is, whereas Vick’s backup, Nick Foles, is projected to be rather good.

Now let’s look at Minnesota.  To say Christian Ponder has struggled the past few weeks is to say Januarys in Vikings Country are a little chilly [Note:  The author is a resident of Vikings Country and a lifelong Vikings fan].  I believe ESPN put his QBR somewhere around 11 for the game last Sunday.  But can we say that Christian Ponder is the problem?  Actually, Christian Ponder is an above average quarterback, completing more passes than an average quarterback would have completed given his stable of receivers.  Where Michael Vick has failed to complete 3.9 passes that an average quarterback would have completed, Christian Ponder has completed 6.5 passes that an average quarterback would not have completed.

Minnesota has a receiver problem.  Specifically, Minnesota has a receivers-not-named-Percy-Harvin problem.  This leaves defenses with a simple solution for stopping the Minnesota offense, load the box to contain Adrian Peterson, double cover Percy Harvin, and sack the quarterback.  Wash, rinse, repeat.

I love sorting through the data that this model puts out because it paints such interesting picture of where one should lay both praise and blame for quarterback and receiver issues.  Michael Vick may deserve a small amount of criticism for his play this year, though probably not as much as is currently being heaped on him.  Christian Ponder, on the other hand, deserves no blame for his struggles and is actually doing amazingly well given the receivers he has at his disposal.

page 1 of 1
Welcome , today is Wednesday, March 21, 2018