Using ICCs to Calculate the Effect of the Quarterback

Categories: NCAA FBS, Statistics
Comments: No Comments
Published on: April 8, 2015

Last time I discussed the importance of correctly modeling the game you are interested if you want to address the problem of data analysis in football. If you are new to the blog, I would suggest reading that post before reading this one. It will give you a good overview of how the analytics are built around here.

For everyone else, a quick refresher. We’re assuming that the correct model of a football offense is shown below.

BasicOffenseModelYards gained on the field begin as a play called by the offensive play caller. They then filter down to the quarterback execution which then filters down to the wide receiver execution. Obviously there are important aspects of the offense we don’t model here, most notably the offensive line and any interactions between the stated roles of the model, but those bridges are a substantial distance down the road.

Now that we have our model, we need two things 1) a question to answer and 2) an analytical tool that can take our question, address the realities of the model, and give us back a number for us to interpret.

The Question

I want to answer one of Bill Connelly’s 45 Reasons to Care about College Football Analytics. These are a set of questions Bill created to drive interest in analyzing college football data. Specifically, I want to begin to address question #5, quantifying how important the quarterback is to the offense. Believe it or not, we can address this question with already existing tools. The first bit of technology we need is the Better Box Score I detailed last week. Here is an example of a Better Box Score.

BetterBoxScoreSo our question is How Important is the Quarterback to the offense. We have our basic model up top and the Better Box Score to help us. Now all we need is the tool. To answer this question we will use Intraclass Correlations (ICC). ICC’s are analytical tools designed to understand similarity across sub-sets of a group. Similarity among group members can be interpreted as the effect of a common factor higher up the hierarchy – in this case the quarterback.

For example, below you see a scatterplot of the four individuals a) played quarterback for Utah State in 2014 and b) targeted two different receivers at least five times over the course of the season. Utah State presents a nice example of this as they had so many quarterbacks and no single quarterback ran away with the team’s attempts. Receptions are on the X axis and Targets are on the Y. Each mark represents a different pass receiver targeted by that quarterback, but the same receiver could be marked on this graph multiple times if they were targeted by multiple quarterbacks. Remember that’s not a problem for us because our model says a pass exists as a connection between quarterback and receiver, not as individual performances. Notice the general patterns in the subsets.

 4-8-2015 - Scatterplot - No Circles

First, Craig Harrison is markedly different from the rest of the quarterbacks on this list. His points are concentrated in the lower right hand corner. The second thing that should be noted is how tightly clustered Darell Garretson’s completion percentage is whereas Kent Myers is more spread out. This may be easier to see if we draw ellipses around each sub-set.

 4-8-2015 - Scatterplot - With Circles

See how Darell Garretson’s ellipse is much more squished compared to Kent Myers’s? That means the receivers targeted by Darell Garretson are more similar to one another than they are with Kent Myers. Likely, this indicates an effect of Darell Garretson getting more consistent performances out of the pass receivers (note that more consistent does not necessarily mean better. One could be consistently throwing balls into the dirt on every pass play and still be consistent).

But we want more than just looking at graphs and guessing if they mean anything. We want to quantify if those circles are meaningfully different from one another. The ICC is a good tool to use here as it returns both a null hypothesis significance test and an effect size of what percentage of variance in a statistic is attributable to a particular “focal person,” in this case the quarterback.

I will be using the terms focal person and partner repeatedly throughout the explanation, so let’s define those terms. A “focal person” is any entity on the higher end of a two-level hierarchy and the “partner” is on the lower level of the hierarchy. So in the hierarchy of our model, the quarterback would be the “focal person” and the receivers would be the “partners.” Note that, in this case, higher up on the hierarchy does not mean “better.” It just means that, when we model the actual game, multiple wide receivers are paired with a single quarterback.

The formula I will use for the ICC is a bit different than what you might find in other sources. Psychologists, most notably David Kenny, have evolved the formula of the ICC so it better matches the questions we care about. The formula we use can be interpreted as an assessment the similarity of the members of a sub-group. It will tell us whether or not receivers targeted by a particular quarterback are more similar to one another than they are to other random points in our data set. Therefore, using the following formula we can assess the percentage of variance explained by having the ball thrown to you by a particular quarterback.

Formula - ICC

  Where k’ = either the number of partners if group size if fixed or, if group size is variable as it is in our case

Formula - K prime

  To calculate our ICCs we first need to choose a dependent variable. I will focus on yards gained (rather than the completion percentage that I showed above). As a teaching example, let’s first calculate the ICC for our Utah State quarterbacks. Here are the data that I have that we’ll be using.

BetterBoxScore - Utah State - 2014

To calculate the ICC, we first run a univariate ANOVA on yards with Quarterback (the focal person) as the independent variable. This returns our Between-subjects and within-subjects variance. In this case those numbers are

Formula - MSbetween MSwithin for Utah StatePlugging those numbers into our formula above, along with calculating k’ gives us an ICC of

Formula - ICC for Utah State

This means that on Utah State during the 2014 season, 18.7% of the variance in passing yards gained can be attributed to the quarterback. Now let’s do this same thing for the entire league, but we have one final wrinkle to overcome, the fact that we have nested hierarchies – receivers within quarterbacks within teams.

To tease all this nonsense apart, we’re going to start at the very top of our hierarchy. Team will be our focal person and quarterback-receiver connections will be he partners. We need to enter more than one season’s worth of data into this analysis because we need to be sure that every team has at least two of the next level down in the hierarchy, in other words quarterbacks. Because of the NCAA’s eligibility rules, this means we need to have at least six seasons of data to guarantee this criteria is met for every single team. So we have data from 2009-2014 in the data set.


Calculating out the Mean Square (MS) between and MS within (a.k.a. between and within groups variance respectively) gives us the following.

Formula - ICC for teams

So 3.1% of the variance in yards can be attributed to the team. This would be anything that is common among all receivers and quarterbacks, so things like the offensive system, facilities, average offensive line ability, average relative defense strength played against, etc.


Now we run the same analysis on the same data but now we change the focal person from team to quarterbacks. Running this analysis gets us the following result.

Formula - ICC for teams and qbs

This result tells us that 6.4% of the variance is attributable to…what? Because it’s not directly true that this results explains everything about the quarterback only. Instead it says 6.4% of the variance is attributable to everything that is held in common among the partners, which would be quarterbacks but would also include, play callers, facilities, etc. So, we need to do a simple subtraction here to get a pure quarterback metric.

Formula - ICC for qbs

And there’s our answer. Quarterbacks in NCAA FBS football have 3.3% of the variance in passing yards attributed directly to them. I also find it very interesting that knowing who the quarterback on a team is will explain almost exactly as much of the variance in passing yards gained as knowing who the play caller is.

NCAA Quarterbacks: 2015 Draft Class

The quarterback situation for the 2015 draft class is looking very murky.  The 2012 draft class was a very unique class in that the highly talented players all managed to get drafted highly and to generate the needed playing time to demonstrate that talent.  I don’t see that happening with the 2015 draft class currently.  I see the 2015 class looking a lot more like the 2013 draft class.  Guys that can play exist in the pool, but who knows if they will get the playing time they need.

Revisiting the quarterbacks I’m watching section from earlier this year, I’m still high on Rakeem Cato.  I saw one feature story on him, but it had to do more with his background than with his ability as a passer.  Hopefully he gets some more attention for the latter.

One player we should keep a close eye on is Bo Wallace at Ole Miss.  I didn’t have him on my original list this year, but he’s having a very good season, both in terms of accuracy and in down the field throws.  That second part is important because he did not have a good season throwing down field last year.  Good to see him demonstrate that he actually has that ability.

Last but not least, someone to keep an eye on is Conner Halliday at Washington State.  Not saying I’m overly excited about him, but he does have something worth looking at.  I don’t know where he’ll end the season in my projections, but his current performance is at least elevating him from where he was.

Last but not least, looking far into the future, is anybody looking at this sophomore from Middle Tennessee?  His name is Austin Grammer.  A name you might want to get to know.

SeaWorld and the NFL

The orcas at SeaWorld are getting a new habitat. The new habitat will cost SeaWorld hundreds of millions of dollars and basically double the size of the habitat the orcas currently have.

Several events have conspired together to move SeaWorld toward building this habitat, but the catalyst can be traced back to a large male orca named Tilikum who murdered Dawn Brancheau, an experienced trainer who was working with him, in 2010.

A key point in the debate about whether or not “murdered” is an acceptable word for the events that transpired rests on whether or not any trainer at any level of experience is safe in the pool with an orca whale. In court, SeaWorld contended that trainers well versed in methods of animal learning and operant conditioning were perfectly capable of controlling a 3 ton whale. The Occupational Safety and Health Administration, on the other hand, contended that no amount of contact with orcas could be considered safe. After some legal wrangling, it was decided that any and all contact between orcas and trainers had to be done with a solid material, such as a concrete barrier, between the trainer and the whale. Direct contact was no longer allowed.

I’ve been returning to my feelings about the SeaWorld incident a lot during this week of terrible football fandom that just won’t end. I wonder about organized football, both in college and the NFL, from the perspective of a workplace. I think about the increased rates of brain damage, along with the recent instances of domestic violence, child abuse, and rape and I wonder what this game that I love to watch is doing to the individuals that get a chance to play it at a high level.

Last week I wrote about one of the great benefits to using statistical methods for employee selection. Basically, using math to find people means it is much easier to find other people should the ones you originally find not work out socially. That’s a great boon to employers in a typical workplace. They become less beholden to talent. Talented people no longer become insulated from the consequences of socially unacceptable acts. And, for any typical place of employment, that works quite well. However, I did not consider in that piece that the NFL is not a typical place of employment. Professional football has some troubling facts associated with it. As a psychology professor, I know all about what brain damage to the prefrontal cortex can do to an individual. Increased emotional reactivity and impulsivity are just the tip of the iceberg. I wonder if this game is destroying the lives of the people that play it. And I worry that anyone that works to identify and project success from one level of football to another, including myself, may be complicit in that destruction.

I am well aware that this blog isn’t particularly popular. As of this writing, I have 45 followers on Twitter and I get about 6 readers a day, 3 of which are spam crawlers. I am, at the moment, insulated from a looming ethical dilemma. No one with actual decision making power is calling me to learn my opinion on whether or not a particular player should be granted the “privilege” of continuing to play the game. The minute that happens, though, I will have an important decision to make. I will need to choose whether or not I should use my intellect to grant someone else the opportunity to potentially destroy theirs in the name of a sporting contest. I will need to decide if I believe the game of football can be played in a way that doesn’t destroy the lives of its players or if I believe my ability to identity talented football players is the same as placing the best trainers in a pool with 3 tons of socially maladjusted rage.

I do know one thing. Talking about football has never seemed more hollow than in these last 10 days. The heart-rune of my fandom is ripped. I’m not sure it will ever heal.

NCAA Quarterbacks to Watch – 2014 Season

Categories: NCAA FBS, NFL Draft
Comments: 1 Comment
Published on: September 2, 2014

Week 1 of the college football season is in the books. I had intended to put together a post of the college quarterbacks I’m watching this year as NFL prospects. I did this last year and saw a lot of search interest in that poorly titled piece. Getting this piece out wasn’t near the top of the priority list, then the new semester started, I started working on a projection model for NFL production, and didn’t get around to putting out this piece until today. Sadly, this means I didn’t strike while the iron was hot in one specific case. I’ll take this as a lesson in timely publication.

Without further ado, the quarterbacks I’m watching for the 2014 NCAA season

Rakeem Cato, Marshall


Cato is still my leader in the clubhouse for NFL quarterback prospects. I actually thought he declared for the draft last year. If he had, he would have been my #3 quarterback prospect then. Things are looking good for him and I’m excited to see what he’s got during his senior season.

Brent Hundley, UCLA

Brett Hundley.jpg

The first player on our list that many scouts will likely agree on. He’s currently second on my list of prospects, but had a really poor game by his standards last weekend. I’ll be watching this one to see which way it goes. Keep watching the season numbers page for more updates.

Shane Carden, East Carolina

Here’s someone you probably haven’t heard of. I’ve said it before, but one of the best things about using numbers to scout prospects is the ability to find gems that don’t get a large amount of T.V. time.  And the numbers say Carden definitely is worth a long look from professional scouts. I think you’ll be pleasantly surprised by what you see.

Sean Mannion, Oregon State

Sean Mannion.jpg

Mannion apparently really impressed some people over the summer at a Manning passing academy. I’m glad to hear it because I’ve been watching him with some interest since the beginning of last season. Hopefully he can continue the trend and keep trending up. I’ll be excited to see where he ends up.

Brandon Doughty, Western Kentucky

Yeah yeah, I know. I’m four days late and six touchdowns short on calling this one. Lesson learned, I suppose. I should probably also note that with last weekend’s performance Doughty launched himself from fifth to third on this list. Keep watching this guy.

So there you have it. My quick list of who I’m watching this season. As always, season numbers are available on the top ribbon.

2014 Draft Class – Wide Receivers and Tight Ends

Summer time is over for us up here in the Deep North. Those two 90 degree days were brutal, let me tell you. I’ve spent the summer cocooned in my office cooking up the latest and greatest that I can offer in predictive football statistics both at the college and professional levels.

If you’re new to the site during a football season, I’ll be talking wide receivers and quarterbacks here, both at the collegiate and professional level.

And to start, let’s talk NFL rookie wide receivers. I made some predictions about rookie wide receivers for the 2013 season and they were…well they were terrible. I spent most of my spring and summer ripping apart the model and figuring out what went wrong. I discovered two very important elements of predicting wide receivers that needed to be addressed.

#1) What’s your Dependent Variable?

One of the trickiest things about doing football analysis is figuring out exactly what you want to measure and what you want your wide receiver or pass catching tight end to do. Many people have tried to deal with this issue when it comes to pass receivers with varied success.

My approach in 2013 was to use an in-house metric I created that measures pass catching ability. This, it turns out, was a horrible mistake. The reason is that NFL teams typically use particular wide receivers in particular roles. One receiver goes deep, another goes across the middle, etc. This creates a problem because it confounds pass catching ability with depth of target. Deep passes are successful less often, but the large impact they create on the game can offset their lower success rate.

For the 2014 predictions, I have changed the DV I use to get around this problem. I start with NFL Yards per Target. Yards per Target is much less susceptible to the depth of target problem. It’s not perfect, but it’s less bad than what I was using last year. I also do a little hoo-doo with the numbers to make them more consistent year-over-year. The first thing I do is to subtract the league average Yards per Target for the given season. This corrects for changes in the passing game across seasons, rule changes, passing tendencies, etc. Next, I use a highly constrained (sort of) structural equation model to pull out the effect of quarterback and offensive system on yards per target. I call it a “sort of SEM” because the model is incredibly constrained due to the realities of the game I’m modeling. It’s so constrained that what I do cannot be called a true SEM. But the technical details are probably not why you’re here. The ultimate result is a metric I call Receiver Influence on Yards per Target (RIYPT; in my ears “ripped”). We’ll use this metric at the NFL level as our DV.

#2) Lower-Level Interactions

Issue #2 I didn’t appreciate when making the 2013 predictions is the importance of interactions. You see, RIYPT at the NFL level predicts productivity. In fact, it’s the only receiver-focused metric that predicts NFL level performance. I made a bad assumption that the same situation would exist at the college level. It doesn’t. At the NFL level, every receiver has great hands. If you don’t have great hands, you don’t get to be a receiver in the NFL. Once we account for depth of target, there’s no meaningful variance in ability to catch a football among NFL pass receivers.

This is not true at the NCAA level. You can be a college receiver without having exceptional hands. As long as you make up for it with lots of long gains, less-than-stellar hands aren’t the handicap that they can be in the NFL. To deal with this, you need an interaction term. Interactions find the receivers that have good enough hands to make it in the NFL while also having the ability to gain useful yardage on an efficient basis.

That’s enough details. Down to brass tacks. Who should we be looking at as far as success at the wide receiver position? Remember, our DV is yards per target, so our definition of success may be different than the actual outcomes obtained on a football field. You can have a large RIYPT, but if you don’t get a lot of targets you won’t gain a lot of yards (see Ladarius Green in 2013).

The full table can be found above in the web page header. My top five rookie wide receivers, according to Predicted NFL RIYPT are…

  1. Brandin Cooks – Oregon State – 2.56
  2. Jalen Saunders – Oklahoma – 2.33
  3. Cody Latimer – Indiana – 1.99
  4. Marqise Lee – USC – 1.97
  5. Cody Hoffman – BYU – 1.93

Here you can see I expect Brandin Cooks to be head and shoulders above the rest of the rookie receivers in the NFL. This is especially true now that he is a member of the New Orleans Saints. Jalen Saunders will be a stickier situation. He’s a lower round draft pick, so it will be more difficult for him to see the field compared to Cooks. Second, he has (at the moment) Geno Smith throwing to him. Past visitors to the site will know I’m not high on Geno Smith. Geno will likely improve next year compared to 2013, but I still expect him to be in the bottom fifth of the league in terms of completion percentage and passing yards.

My top five pass catching tight ends according to Predicted NFL RIYPT are…

  1. Eric Ebron – North Carolina – 1.37
  2. Richard Rodgers – California – 1.12
  3. Marcel Jensen – Fresno State – 1.04
  4. Blake Jackson – Oklahoma State – 1.02
  5. Jace Amaro – Texas Tech – 1.02

Once again, we have one prospect that is head and shoulders above the rest, that being the prospect everyone expected to be on top, Eric Ebron. The second name on that list probably surprises a few people. If you look up scouting profiles of Richard Rogers, they’re not that glowing of him. I guess we’ll see where we end up. Rodgers also ended up in a great place to succeed as a pass catching tight end – Green Bay – so hopefully we get an opportunity to see him succeed.

This wraps up our rookie preview.  Up next, we’ll predict 2014 yardage totals for veterans from 2013 data.

2014 NFL Draft Predictions: Quarterbacks

Categories: NCAA FBS, NFL Draft, Statistics
Comments: No Comments
Published on: February 12, 2014

Welcome back everyone.  I took a couple weeks off to recharge and enjoy the Super Bowl.  Now it’s time to get to the heart of the matter.

This post marks my predictions for the 2014 NFL Draft Quarterback class.  On the Draft Numbers page you will see predictions for both NFL Passer Rating and ANY/A for every draft eligible prospect.  This class is a very interesting one.  It’s very similar to the 2011 class in its number of potential starters, one.  Unlike the 2011 class, though, the one player that has the potential to start in the NFL isn’t getting very much buzz, is unlikely to be drafted highly, and probably won’t be a first year starter.  That player is Keith Price from Washington.  Many of the other potential prospects will get playing time, some have potential to be career backups in the league, but this class will be very short on quality starters.  All the numbers are available here.

Prediction Model Details

These predictions are generated using Bayesian analysis procedures.  If you would like details on the priors, you can ask in the comments.  The data set used to generate the equation includes all quarterbacks that played FBS football for at least one season from 2007-2012 and threw at least one pass in the NFL during the 2008-2013 seasons.

First off, the analysis finds that Career Completions Away from Average effectively predicts both 4-year passer rating and 4-year ANY/A.

When we make predictions like this, it’s important to evaluate the model to see how precise it is.  When I tell you that Aaron Murray is predicted to have a Passer Rating of 77.5 after four years in the NFL, how much uncertainty is there in that prediction?  Below you will see a plot showing how much we can reasonably expect the predictions to be off.

The plot gives you an indication of how much the predictions based on CAA can be expected to be off.  The circles represent one quarterback in the data set that was used to generate the prediction model.  The vertical blue lines represent the region that is 95% likely that the prediction will fall into.  You might look at that plot and rightly say that there is a lot of uncertainty in these predictions.  And you would be right.  There is a lot that isn’t accounted for by this single number.  However, let’s make a comparison.  One of the best ways to gauge the general league’s opinion of a prospect’s chances of being successful is using relative draft rank broken down by position.  In other words, was this quarterback the 1st, 2nd, 3rd, etc. quarterback selected.  The plot below uses the same data set that was used to generate the CAA plot, but uses positional draft rank to generate the regions.

The direction of the effect is reversed, so it might be difficult to see, but the length of those blue lines is almost exactly the same as with CAA.  Which I like to see.  It tells me I’m on the right track with this thing.  For completeness sake, here is the same plot predicting ANY/A.

Player Profile: Derek Carr

Categories: NCAA FBS, NFL Draft, Statistics
Comments: No Comments
Published on: January 22, 2014

Player:  Derek Carr

School:  Fresno State

Year:  Senior

Career CAA:  35.1

Predicted 4-year NFL Passer Rating:  75.3

Predicted 4-year ANY/A:  4.9

The Senior Bowl is just now wrapping up, and by all accounts Derek Carr was the quarterback that everyone came away impressed with.  Certainly there were some detractors, but it seems like overall, everyone had much more positive things to say about Carr than they said about any of the other quarterbacks in the mix.

But then there are always whispers and rumors out there.  One person compared Carr to Christian Ponder.  They meant it to be a frowny face sort of thing.  I would say that’s a better comparison than most people think, but then again I don’t get to be an arbiter of Ponder opinions.

So, what do we think about Derek Carr?  In a word, meh.  At least for the first four years he’ll be okay but not going to set the world on fire.  This is actually going to be a theme as we advance through the player profiles.  So much hype is surrounding the current class of quarterbacks and almost to a man the model shrugs at them and says “Yeah I guess.”  So that’s what I think will happen to Derek Carr.  I guess it could happen.  I wouldn’t expect him to set the world on fire though.

Take Home Point

Cool story, bro

Do you draft him?

Much like Manziel, probably not where he’s going to go.

Player Profile: Johnny Manziel

Categories: NCAA FBS, NFL Draft, Statistics
Comments: No Comments
Published on: January 8, 2014

Player:  Johnny Manziel

School:  Texas A&M

Year:  Sophomore

Career CAA:  41.7

Predicted 4-Year NFL Rating:  76.5

Predicted 4-Year NFL ANY/A:  4.97

The most polarizing prospect in the 2014 draft declared today.  The gloriously nicknamed Johnny Football.  I’m not going to tell you much you don’t already know.  Some people love his skill set.  Others think his skills won’t translate to the next level.  I’m going to come down right in the middle.

My biggest concern is that Manziel has very little experience compared to the rest of the class. By my reckoning, he had a very good season in 2013.  But that doesn’t mean that I think he’s a wonderful prospect worthy of a first round pick.  If he had another season in college that was similar to last season, maybe we would have something.  Alas, we don’t.

But that also doesn’t mean he won’t be a good NFL quarterback.  He showed skills this year – an ability to be accurate and an ability to improve on accuracy from year to year.

Take Home Point

This sounds like the most wishy-washy evaluation.  I don’t love him, I don’t hate him.  He’s just a giant lump of there that refuses to go away.  But that’s where I have him.  A middling prospect that everyone seems to want to talk about.

Do you draft him?

Not where he’s going to go.

Player Profile: Nathan Scheelhaase

Player:  Nathan Scheelhaase

School:  Illinois

Year:  Senior

Career CAA:  +54.9

Predicted 4-Year NFL Rating:  78.4

Predicted 4-Year NFL ANY/A:  5.19

I featured Nate Scheelhaase in my “Quarterbacks to Watch” article at the opening of the season.  In that article, I called him a controversial prospect because he had been so variable during his first three years as a quarterback at the University of Illinois.  I asked for the real Nate Scheelhaase to please stand up.  It appears he stood up.  Statistically, this season has been the best of his career, posting career highs in attempts, completion percentage, yards per attempt, touchdowns, and interceptions (can’t be all good with all those attempts :D).  Prior to the bowl games, I have him as the 15th best quarterback in the nation for the 2013 season-above much loved prospects like Zach Mettenberger, Aaron Murray, and Sean Mannion.

But I don’t see much discussion out there about Scheelhaase.  As far as I know, he hasn’t been invited to any of the off-season all-star games.  Most other websites I’ve seen list him as a player that won’t get drafted and won’t make a team as an undrafted free agent.  I respectfully disagree.  I understand that the level of competition in the Big 10 is less than it has been in the past.  I also understand that Illinois only won four games this season.  But you can’t use that as a knock on Scheelhaase.  Did you see that defense out there?  They gave up 52 points to Indiana and 34 points to Southern Illinois.  No quarterback can be asked to make up for that on his own.

Take Home Point

If you’re not convinced on Scheelhaase, I have to ask.  What you want from an NFL prospect?  Scheelhaase has decent size, is mobile enough to avoid pressure, and can deliver the football with accuracy.  I only care about the last two, honestly, so the size is just a bonus.

Do you draft him?

In a vacuum, yes I would draft him.  However, draft decisions aren’t made in a vacuum.  Given how little attention he’s getting, my sense is that you target him as an undrafted free agent and thank the rest of the league for your free quarterback.

Player Profile: David Fales

Categories: NCAA FBS, NFL Draft, Statistics
Comments: No Comments
Published on: December 18, 2013

Player:  David Fales

School:  San Jose State

Year:  Senior

Career CAA:  29.2

Predicted 4-Year NFL Passer Rating:  74.7

Predicted 4-Year ANY/A:  4.76*

Finally, a player with some actual buzz.  David Fales is a very interesting prospect to me, mostly because he stretches the bounds of what my prediction model can say about a prospect.  Of all the quarterback prospects in this year’s draft, I think Fales is one the model will be the most wrong about.  The thing is, I don’t know in which direction the model will be wrong, which makes Fales even more interesting to watch.  You could tell me that, four years from now David Fales will have a career passer rating of 80 and take a team to the playoffs two years running and I’d believe you.  Or you could tell me that David Fales won’t start a game in the NFL and I’d believe that too.

So why so much uncertainty around David Fales?  A number of reasons come to mind.  First there are the standard problems of not playing a full career at the FBS level.  I’ve only got two years of data on Fales when I have three or four on many other quarterbacks.  But there is an extra problem with the situation that surrounds David Fales that makes his future outcomes even more difficult to predict.  Nothing breaks my prediction model faster than constant changes to the offensive system.  Changing receivers is good, changing quarterbacks in the same offensive system is good, but as soon as the offensive system starts changing rapidly, the numbers go haywire.  And San Jose State has seen its fair share of change in the offensive system during the tenure of David Fales.

During Fales’s first year (2012), a new offensive coordinator was brought in.  This means that most of what we knew about San Jose State’s offense can be thrown out the window.  Not everything of course, because the head coach was still the same.  We can imagine that Mike MacIntyre had a particular philosophy in mind when hiring Brian Lindgren that wouldn’t be much of a change from previous offensive systems.  But we don’t know much about whether this new guy has a system works the same way, features the same personnel, or uses the personnel they have in the same way as the previous system.  This problem gets compounded when the head coach lands a new job for 2013 and takes the offensive coordinator with him.  Now we not only have a new head coach but another new offensive coordinator, which really throws everything out the window.

You can see this in the large variability in Fales’s numbers from 2012 to 2013.  During the 2012 season, Fales is third in the country on my metric (35.6 CAA).  In 2013, he’s 126th (-6.4) and below average (0 is perfectly average).  Part of the reason for this massive change is that the model is trying to account for circumstances like offensive system.  In this particular case it’s having a really difficult time calculating a precise number to put on those circumstances.

At the end of the day, it would be really nice if Fales had another year of eligibility so we could get additional information about his on field performance.  However, NCAA rules being what they are, we just aren’t going to get that.  Without more data, the prediction model can only throw up its hands and say “Might be great, might never work out.  Can’t tell you which one based on what I’ve got.”  It’s possible the 2012 numbers are the true David Fales, or it’s possible the 2013 numbers are the true David Fales, or it’s possible that neither is the true David Fales.  Everywhere we look with this guy, the future is cloudy.

Take Home Point

If I was working for a team, this is one I’d turn the Fales tape over to the highest paid scout they have and say “You deal with this one.  My skills are pretty useless here.”

Do you draft him?

Ask a scout.

* I’ve added a prediction of Adjusted Net Yards Per Attempt (ANY/A) for the advanced stats buffs out there.  Career CAA predicts ANY/A after 4 years in the league according to the following equation Y = 4.27 + 0.0167*Career CAA.  The equation was generated using Bayesian analysis assuming a t-distribution for the dependent variable.

«page 1 of 3
Welcome , today is Thursday, February 22, 2018