“You’d think after 7 or 8 you’d start to go, ‘Maybe it’s me.'”: Franchise Problems

The title of this post is the punch line of my favorite Ron White joke.  He’s describing a woman who claims to have had bad sex with more than 2,000 airmen.

I’m using the joke as a metaphor for NFL teams that seem to be constantly drafting quarterbacks.  How many times will certain teams have to draft an apparently talented quarterback only to see him waste away before they take a hard look at themselves and say, “Maybe it’s us”?

I’ve actually been kicking this idea around for a while now.  I just didn’t have a proper story to couch the numbers in.  But today I was listening to the latest Wages of Wins podcast with Eric Weiss of Sports Aptitude.  He was discussing how some basketball teams seem to draft really well.  He wasn’t talking about teams that are able to recognize talent and drafted accordingly.  He talked about how some NBA teams draft to create an environment.  He cited the San Antonio Spurs as an example of a team that appears to follow this strategy.  The Spurs seem to draft with an eye to the individual player, how that player will fit within the system, and if the player’s personality will respond to the role they are asked to fill.  Most important for our purposes, he talked about what teams can do to ease the transition when new players enter the system.  It isn’t easy to be uprooted from everything you’ve known as a 22 to 25 year old and be placed in an entirely new place with the only constant being that you’re still playing a sport.  There are really important things you have to learn that have nothing to do with the sport you are now playing professionally.  Where is the good grocery store?  Can you still access your bank accounts?  Where are you going to hang out on your off time?  What do you do with all this money you suddenly have?  And now we’re going to add to that the constant scrutiny and publicity that comes with being a professional athlete?  Anyone that can thrive with that sort of uncertainty, uprooting, and pressure certainly deserves our respect.

But are there some NFL teams that could improve in this regard?  Are there teams that constantly seem to be getting less of a return on their drafting investments compared to others?  Who should be looking at easing the transition to improve the outcomes of their newly drafted players?  If you’ve been reading my site, you know that I claim I can separate the impact of the quarterback from the impact of wide receivers.  I can tell you whether or not a quarterback is succeeding because he is a great quarterback (Aaron Rodgers) or because he has a great set of receivers to throw to (Andrew Luck).  Same with struggling quarterbacks.  Do they struggle because they are having trouble putting the skills together (Ryan Lindley) or because their receivers just aren’t up to the task (Christian Ponder).

Using the same methodology, I can also separate the impact of the quarterback from the impact of the franchise.  The problem is, I can only do this for teams that change quarterbacks.  In the NFL, when things are working they don’t change.  I suppose that’s true for a lot of places, but I care most about the NFL.  So I can’t tell you whether the Patriots are really good at creating an environment for Tom Brady to excel or if Tom Brady would succeed in any environment.  The Patriots haven’t changed quarterbacks in a long time so there is no way for me to tell you that.  But I can tell you about the struggling teams.  The immediate, reactionary move for any struggling team is to change the quarterback.  This gives us some very useful information.  I can quantify how much the issue associated with the franchise is hurting the quarterbacks that play there.  So not only can I say, “Yeah, it’s you.”  I can say, “And this is how much it’s you.”

The data included in this analysis are from the 2007 to 2012 seasons.  Please note that I can’t give you exact reasons for why these teams seem to have poor environments.  Also, each team might have a different reason or its poor environment.  I don’t know.  I’m not close enough to these teams.  I just know that the data say that the problems are not entirely at the quarterback position. In reverse order, the top 5 teams that need to improve their environment for quarterbacks.

#5:  New York Jets      

Expected Reduction in Completion Percentage – 3.29%

The Jets love to rabble rabble rabble about their quarterback situations.  Mark Sanchez is the latest victim here.  Jets fans:  It’s not his fault.  At least, not all of it.  Mark Sanchez would likely really benefit from a change in scenery.  The Jets, though, not so much.  I doubt they would fare any better by drafting another quarterback.  These guys they keep bringing in new quarterbacks to “push” Mark Sanchez aren’t doing anything.  I think we need to accept that playing for the Jets just isn’t that great for a quarterback’s career.

#4:  St. Louis Rams    

Expected Reduction in Completion Percentage – 3.46%

We’ve come a long way from the “Greatest Show on Turf.”  Since the Super Bowl years of the early 2000’s, the Rams have had a hard time getting any quarterback to stick.  Sam Bradford was supposed to be this great success, but he’s had his struggles.  Struggles that are only magnified after he signed the last massive rookie contract ever.  But I say the issue is not with Bradford.  Everyone loves to pick on Sam Bradford for not being the huge success he “should” be by now.  Without the St. Louis drag on him, Sam Bradford is as accurate as Tom Brady.  Think about that.

#3:  Carolina Panthers    

Expected Reduction in Completion Percentage – 4.99%

A surprise franchise comes in at #3.  Carolina hasn’t been the greatest team in the last six years, but they haven’t been the worst either.  They seem to have a quarterback now that can get them where they want to go, and even before that they had talented players at the position.  I would really like to dive into the data and study more about what might be pulling Carolina down so low on the list.  Only problem is that would involve me moving to Carolina to work for the Panthers, and I don’t think that’s in the cards.

#2:  Oakland Raiders    

Expected Reduction in Completion Percentage – 5.00%

Our last two entries on our list are teams that go through quarterbacks like Kleenex.  The first is the Oakland Raiders.  Many quarterbacks have tried to turn this team around.  All have failed.  And it would appear that unless something drastic happens, quarterbacks drafted by the Raiders will likely continue to whither.  I’ve been hearing rumblings of Geno Smith being mock drafted to the Oakland Raiders.  It might be kinder just to break the dude’s legs right now.

#1:  Cleveland Browns    

Expected Reduction in Completion Percentage – 5.37%

You knew it was coming, Cleveland fans.  The train wreck that is Cleveland Browns has seen quarterback after quarterback try to save this thing.  None of it is working and I think the franchise really needs to take a hard look at what they do off the field before they bring in yet another quarterback.  They need to help their players in a way other than talent evaluation.  And I truly believe Cleveland has been drafting talented quarterbacks.  Especially the last two.  Brandon Weeden and Colt McCoy are loved by my draft model.  But just like Ron White’s airbase lady, when your partner doesn’t have it going on, every one of them seems like a bad…

Gone Analyzun

Categories: Uncategorized
Tags: No Tags
Comments: No Comments
Published on: March 25, 2013

Need buks
Kant estimate one p’rameter
Bak next week

In all seriousness, I don’t have anything new to report. Amazon willing, I’ll have my useful book by the end of this week. Once I have that, I can report the results of a cool new approach to predicting quarterback success. Until then…Gone Analyzun

The Way-Back Machine: JaMarcus Russell

Categories: NCAA FBS, NFL, Statistics
Comments: No Comments
Published on: March 17, 2013

Blizzard permitting, I will be standing in front of an Introductory Psychology class lecturing on the topic of intelligence on Monday.  The major point of the lesson will be that intelligence is a combination of ability, on the one hand, and the skills and motivation to use that ability on the other.  Both influence outcomes on intelligence tests about equally.  It’s not just intelligence we see this pattern for.  Many outcomes we label “talent” or “God-given ability” can be broken down statistically into an inherited, biological component and a skills and motivation component.

Which brings us to JaMarcus Russell.  The narrative about Russell is that he is a classic case of a highly talented individual lacking the skills and motivation to put that talent to use.  Pundits point to his arm strength and success in college as indicators of his talent, while his numerous issues both on and off the field as indicators of his lack of work ethic, poor motivation, etc.  Even now, four years after his last played down of professional football, we seem to be paying attention to his apparent comeback.  We talk of how great an “untapped” talent he still is.  If only he could get the motivation component under control.  But untapped talent tends to remain so.  Without the skills and motivation necessary to unlock such talent, that talent will remain hidden and useless.

So what do the data say?  Was JaMarcus Russell tapping his potential in college before his unfortunate demise in the NFL?  Before we answer that question, some caveats.  JaMarcus Russell was one of the first players I was interested in looking at once I realized I had a useful model for predicting success at the NFL level.  But there are problems.  First, I cannot analyze anything before 2005.  An important piece of data, incomplete pass targets, was not recorded in any form prior to then.  So there are points in JaMarcus Russell’s career that we simply can’t analyze.  Second, the limited amount of data limits our ability to separate JaMarcus Russell’s ability from the impact of play-calling, skills of the typical defense faced, etc., etc., etc.  Essentially, we are limited how well we can separate Russell from program-level factors.  This is an especially difficult problem for evaluating Russell because Les Miles began his tenure at LSU in 2005.  We can’t correct for the influence of Les Miles in these early data.

So once again, data.  The table shows JaMarcus Russell’s Completions Away from Average for the 2005 and 2006 NCAA season as well as totals.  Remember that 0 CAA is average.

2005JaMarcus Russell-11.703
2006JaMarcus Russell12.825

Even though we only have two seasons of data, note the wild inconsistency across these two seasons.  In 2005, he had a poor season but in 2006, he had a pretty good season.  The combination of the two gives us a prediction of his passer rating after 3 years in the NFL of 65.9.  His actual NFL passer rating after 3 years in the league was 65.2, a phenomenal prediction for this model given the limited data it’s working with.

So, was JaMarcus Russell a tremendous college talent that flamed out because of poor work ethic.  Not really.  He did exactly what we should have expected.  He was inconsistent in college and ultimately should have been labeled as an average NCAA quarterback.  Sound familiar?

The bigger question is, how did evaluators get it so wrong with JaMarcus Russell.  I can only speculate about that, but I think evaluators were seduced by what they saw outside of actual games.  JaMarcus Russell can throw a football a long way and is “naturally gifted,” according to Jeff Garcia, so what if he is somewhat mechanically inconsistent and has some sideline issues?  It would appear that, at least in JaMarcus Russell’s case, it matters a great deal.

Correction to Previous Post

Categories: General Info
Comments: No Comments
Published on: March 12, 2013

I mentioned last time that I was going to run a full ANOVA on the data from my post last time, including some nice confidence intervals and post-hoc tests.  Well, I did that and made these graphs.

This one for Career Receptions

This one for Career Yards

This one for Career Touchdowns

I looked at these graphs and noticed something hinky.  The confidence intervals for rounds 5, 6, and 7 overlap.  That can’t be if I have a significant ANOVA like I reported in the last post.

So I spent some time checking over my calculations and noticed an error.  My sum of squares calculation was pointed at the group standard deviation instead of the group mean.

Upshot of the whole thing?  The points at the end of the last post about the 7th round being better than the 6th are incorrect.  Instead, we see our expected pattern, a J-shaped distribution consistent with random selection of talent coupled with self-fulfilling prophecy associated with draft round.  Once we control for playing time, we should see no differences in performance across rounds, which we can begin to evaluate using Yards per Game Started

Career Wide Receiver Production: 7th Round Darlings

Categories: NFL Draft, Statistics
Comments: No Comments
Published on: March 10, 2013

People say that the quarterback position is the most difficult to project into the NFL.  I respectfully disagree.  Certainly, the quarterback position is difficult to project from the college to professional game, and we’re not very good at it.  The empirical evidence shows decision makers are essentially making a random guess when it comes to projecting quarterbacks.  The only reason first round picks seem to have better numbers is because decision makers expect them to be good and provide them more opportunities – a self-fulfilling prophecy.

But there are worse things in this world than randomly guessing.  You will be right sometimes if you’re randomly guessing.  More than that, if you keep guessing randomly over and over and over again – say once a year every April since 1936 – the number of times you’re right and the number of times you’re wrong should be essentially equal.  Random decisions are not the worst possible outcome.  It is entirely possible that you could be on the other side of random.  It’s possible to make decisions that are worse than what a random process could achieve.  Sometimes we could make better decisions if we started throwing darts while blindfolded.

As an example, consider how most lay people buy stocks.  Most people don’t want to lose their money in the stock market.  They want high performing stocks that are going up.  So they don’t buy a stock until it starts going up.  And the average person doesn’t want to be burned by market fluctuations, so they continue to watch the stock as it goes up more.  Once the stock gets sufficiently high, they buy the stock.  But this is precisely the worst time to buy the stock.  The stock has nowhere to go but down from this point and the investor is more likely to lose money.  The person would be much better off picking stocks at random because the information they are using is leading to worse decisions.

How does this relate to football?  Football decision makers drafting wide receivers and lay people picking stocks have a lot in common.  To be clear, I have no idea what information football decision makers are using when predicting wide receiver success in the NFL.  Others have studied this (that one is a pdf, be aware if you click on it) and found inconsistent results.  However, just because I don’t know the process doesn’t mean I can’t evaluate the outcomes the process generates.  So let’s do that.  Let’s evaluate wide receiver draft picks and compare that with what we would expect given a random process.

The Evidence

I started with another study of the Hall of Fame.  It seemed to work well for the quarterbacks, so why not go back to the well again?  Where were the consensus great wide receivers drafted?  Here is a histogram of all wide receivers in the Hall of Fame that played since 1945.  Dante Lavelli is not on this graph as the U.S. Army drafted him before the NFL got a chance.  Remember, a random process coupled with a self-fulfilling prophecy will create a J-shaped distribution.

If we delete Raymond Berry as an extreme outlier, the histogram looks like this.

Here we have our first strange finding in the wide receiver data.  For picks 1-80, the data show the expected pattern.  A nice J-shaped pattern as we get further on in the draft.  But what is going on with picks 81-120?  There is a strange increase in Hall of Fame receivers drafted at a point corresponding, in modern times, to the late 3rd through the 4th rounds.  This is our first evidence that wide receiver evaluation is not just random, but biased in the wrong direction.  However, it is far from conclusive.

This pattern is strange enough to demand a more complete analysis of wide receiver production.  Perhaps the bump in the late 3rd through 4th rounds represents the perceived value of the wide receiver position.  We can be reasonably sure that everyone wants a quarterback and is willing to draft their favorite quarterback with the highest pick possible.  However, the same might not be true for wide receivers.  It’s possible that wide receivers are not as highly valued compared to quarterbacks and are not taken with the highest draft pick possible.

To attempt to rule out that explanation, I took Berri and Simmons’s methodology that they used for quarterbacks and applied it to wide receivers.  Let’s examine career production for all wide receivers drafted between 1995 and 2009.  We will look at Career Receptions, Career Yards, and Career Touchdowns.  I chose to start at 2009 because the average career length of an NFL wide receiver is just a shade over 3 seasons.  Any receiver drafted in 2009 is already at the average career length for an NFL wide receiver.  I chose to stop at 1995 because it includes 15 years of data, which is a nice round number, and it was midnight when I finished entering the 1995 data and I wanted to go to sleep.  All data was downloaded from pro-football-reference.com.

The first graph shows the Average Career Receptions for a wide receiver drafted during each round of the draft during the 15 year time period we are looking at.  Unless otherwise specified, I deleted all players that were never credited with catching an NFL pass.  I have run these numbers keeping all players, and the pattern is exactly the same.  Remember that this analysis does not control for playing time, so we are expecting a J-shaped distribution.

Here is a similar graph showing Career Yards

And the same graph showing Career Touchdowns

In all three cases, the data from Rounds 1-5 show the expected pattern.  There are still players drafted in the later rounds that are making an impact, but the impacts tend to be less than those drafted earlier.

However, that wasn’t the really weird part.  The really weird part was what happened in the final two rounds.  I might not have thought much of it if I hadn’t run the Hall of Fame data first.  Let’s zoom in on career production only for players drafted in rounds 5, 6, and 7.

Here is Career Receptions

Here is Career Yards

Here is Career Touchdowns

Wide receivers drafted in the 6th round have, on average, less productive careers compared to wide receivers drafted in the 5th and 7th rounds.  This is true if we measure production based on receptions, yards, or touchdowns.  One-way ANOVAs using only players drafted in the 5th-7th round confirms this, all F’s > 15.90.  My home set up is not great for running ANOVAs, which is why I did this slimmed down version.  Next week I will put all players into an ANOVA and run some splashy post-hoc tests.  Until then, we’re stuck with this slimmed down analysis.

This is a crazy finding.  Compared to receivers drafted in the 6th round, receivers drafted in the 7th are working against the self-fulfilling prophecies of decision makers, coaches, and their quarterbacks.  They should, by all accounts, have worse careers.  And yet, they have the more productive careers than players drafted in the 6th round and are just as productive as receivers drafted in the 5th.  My first explanation for this finding was that receivers from non-FBS schools are more likely to be drafted in the 7th round.  That was not the case in this data.  80.95% of 5th round draftees, 72.5% of 6th round draftees, and 81.81% of 7th round draftees come from FBS schools (Remember we deleted receivers that either did not make a team or did not catch an NFL pass).  At the very least, this isn’t a question of source school .

What should we conclude from this?  I think we have some very clear evidence that talent evaluators should stay away from wide receivers.  The pattern at the top of the draft is consistent with a random process.  I can’t conclude it is random right now because I don’t have good numbers on wide receiver playing time.  However, the pattern is consistent with randomness coupled with self-fulfilling prophecies.  Given what we know about the randomness of selecting quarterbacks, there is no reason to assume that wide receivers would be any different.

The pattern at the lower end of the draft is even more compelling.  In this case, talent evaluators and draft decision makers are consistently wrong about players evaluated as 7th round talents.  Those players are, on average, just as good as players evaluated as 5th round talents and better than players evaluated as 6th round talents.

This leads me to believe that we simply do not know what makes a productive wide receiver.  Furthermore, something about how we evaluate wide receivers is leading us to make worse decisions.  If we could control for the self-fulfilling process effect, we might see lower rated receivers doing BETTER than higher rated wide receivers.  (see This Post for explanation of the correction)

Which ultimately leads me back to my two favorite predictions for this wide receiver draft class, Cody Wilson from Central Michigan and Brent Leonard from Louisiana-Monroe.  Both of these receivers had extremely productive college careers at schools that don’t get a lot of media attention.  Both of them are not rated highly by the draft community for physical reasons – Cody Wilson because he is short for an NFL receiver and Brent Leonard because he is “not fast.”  But I ask you, who cares?  Fast receivers do not do any better than slow ones.  And prioritizing fast receivers might be leading us to draft worse receivers in the high rounds.

Remember, when I started this I said I don’t know what process evaluators are using to grade wide receivers.  However, after looking at the data, I am confident that they should stop whatever it is they are doing.  The decisions being made late in the draft are worse than randomly guessing.  At the end of the day, decision makers would be better off throwing darts at the names of draft eligible wide receivers blindfolded.

On Communication

Categories: General Info
Comments: No Comments
Published on: March 6, 2013

It’s a rare weekday post from me.  I’m updating now rather than my normal weekend posts because I think the topic of conversation is fading rapidly.  I want to get my two cents while the iron is still slightly warm.

I mentioned last time that I Twittered into the Sloan Conference last weekend.  Since that time, the panel that seems to have had the most legs beyond the conference is the panel on communication.  Commentators have gone on at length about the good information that was in that panel essentially saying that quantitative people need to be concerned about how they are presenting their ideas.  In classic terms, this was the “thesis.”

Then people started mulling over the thesis and I saw some reactions against that line of thinking.  Some people saying that communication is a two way street and the burden is not solely on the quantitative people in the room.  Front office people also need to be learning rudimentary statistics to help facilitate that communication that they are looking for.  We can call this the “antithesis.”

My goal in this post is to provide some synthesis.  Let’s combine these ideas to see what we can take forward.  I often find that when I’m trying to communicate with non-analytic people, the issue is a difference in mindset.  I don’t mean to imply that every individual within these two general groups thinks in this way.  However, I do believe that the general tendencies exist and are worth mentioning.  What might Front Office People and Quantitative Analysts not know about one another that could be inhibiting this communication?

Front Office People – You need to understand that quantitative analysis is a process of pointing out what’s wrong.  Every day I show up at work and think up some ideas.  I spend the rest of the day trying to find a way to demonstrate that those ideas are wrong. If I don’t do this, my colleagues will do it for me when I present, or my peers will do it when I try to publish my ideas.  I have literally sat in a room while a friend and colleague asked me a series of questions that had the possibility of invalidating six entire years of work.  We don’t do this to each other because we’re assholes.  We do this because it is the quickest way to the best answer.  The idea is that if we spend all of our time, energy, and brain power on ripping down this idea, whatever can’t be ripped down must be worth pursuing.  The entire process of analysis is a process of falsification.  Always remember, when you pay an analyst to run some numbers you are paying them to tell you that you are wrong.

Analysts – There is a reason so many of us are depressed alcoholics.  The constant falsification and focusing on what we’ve done wrong rather than what we’ve done right is not a typically healthy mindset to be in.  Most of the rest of the world does not think this way.  It hurts to be reminded that we’re wrong.  We analysts have a pretty thick callus over that particular spot, but most don’t.  Most people want to be reminded that they are right.  They want to be affirmed and given positive reinforcement.  As analysts, we have to be very concerned with how we frame our responses.  The old clichéd story about the dream interpreter comes to mind.  The “your dream means your children will all die before you vs. your dream means you will outlive all with eyes on your thrown” thing.  The first interpretation gets the interpreter killed while the second gets the interpreter a cushy job and some treasure, but they are both the same interpretation.

Here is the message that I got from the communication panel.  Framing answers is what will truly separate the successful analysts from the unsuccessful ones.  The analysts that will continue to have jobs are the artists who can continuously tell powerful people they are wrong and still keep the powerful people happy.  That is not a trivial skill.  It takes effort and patience to reframe our traditional ways of thinking for those that aren’t as familiar with them.  And from the non-analyst side, analysts aren’t assholes, even though we might talk like we are sometimes.

Attending the Sloan Sports Analytics Conference via Twitter

Categories: Uncategorized
Comments: No Comments
Published on: March 3, 2013

I’ve been participating in a very strange experience lately.  Over the last two days, I’ve been vicariously “attending” the Sloan Sports Analytics Conference via Twitter.  At first I didn’t even mean to.  I just happened to follow enough nerds.  I logged in Friday afternoon and suddenly my Twitter feed was filled with interesting points, great zingers, and summaries of each talk that were surprisingly effective for only being 140 characters long.  I could even ask questions if I knew the right hashtag.  It really was an incredible experience.  Here I am getting a full conference experience – outside of the networking – while sitting halfway across the country correcting student assignments.  Technology, man.  Crazy.

Anyway, the whole experience got me thinking about sports analytics as a field and what I can contribute to it.  Ultimately, I think I can add two pieces of the football analytics puzzle:  1) quantifying interdependent team processes and 2) specifying my methodology for calculating Completions Away from Average.

Part 1:  In Which Jared Plays Under Appreciated Academic

I don’t often jump up and down in my office.  My job as a college professor is fairly low-key.  I don’t have lots of reasons to get really excited during the work day.  On Friday, I did.  There was one panel at Sloan talking about how sports are almost exclusively outcome focused, but the successful teams are the teams that focus more on process.  Basically, they were saying that if a team is following a process that that generally leads to positive outcomes, positive outcomes will follow at a greater than average rate.  You won’t have to rely on luck or poor competition or something else that’s not under the team’s control if the team is process focused.  This piqued my interest.  I spent 10 minutes playing Twitter like a video game, clicking on the new tweets as fast as they pop up.  Everyone was digesting the idea.  There was general agreement that process is important.  In fact, general agreement that process should be the true focus of a well-constructed team.  And then the big question came.  The question that always gets asked when a bunch of data nerds start talking team process.  How do we measure/quantify/work with process?  And now I’m literally out of my chair.  “Ooooh!  Ooooh! I know the answer!  I’m not in Boston, but it’s still cold and snowy around here.  Maybe we can pretend.  Pick me! Pick me!”

Well, of course they didn’t pick me.  We got other answers of “You don’t,”, “Results are the best proxy,” and, the answer that started a whole new round of jumping, “It’s exceptionally difficult in football because the data are interdependent”.  I don’t have a specific problem with any those answers because they are the best generally accepted answers the field has.  But I think I have a better answer.  I tried to engage some of the tweeters in conversation about this topic, but it didn’t work.  The ideas are too big for 140 characters and when I tried to tweet them, I ended up sounding asininely critical.  Why do I think I have a better answer?  To answer that, you need a little background on graduate school.

Let’s imagine that you have earned your master’s degree and want to go for the Ph.D.  Before you get admitted to the program, you have to demonstrate that you have the knowledge and skills necessary to be successful.  There are two general ways that programs will do this, qualifying exams or an area paper.  My program went with the area paper.  An area paper is a project where you go off for a while and find an important, unanswered question you find interesting.  You then read everything ever published regarding that question.  In the end, write a 50-100 page paper describing your answer to that unanswered question.  You then present your paper to three or four tenured professors who try their best to rip down, discredit, or otherwise poopoo on your answer.  If it holds up to a couple hours of questions, you get in.  If it doesn’t, well then you have a problem.

I went off and spent nine months working 12 hours a day, six days a week in my tiny little basement office.  I found the question I thought was interesting, read all that I could about it, and came up with an answer.  In the end, I wrote a paper I titled “Inferring Team Process Using Interdependent Team Data.”  It was good enough to get me admitted to my program and I spent the next two years working the same schedule figuring out a way to test and evaluate my answer.  So you can imagine why I was jumping up and down about the answers about quantifying group process in interdependent teams.  It’s a tough problem.  I took me nine months working nearly 80 hours a week to figure out an answer and another two years working 80 hours a week to figure out how to validate that answer.  But in the end, I think it’s a good answer.

I even got my original idea published as a book chapter.  I had to change the title to make it fit the scope of the book, but the idea is essentially the same.  You can find the book here.  If you read it, you will be part of an exclusive club.  This book is very obscure.  It’s been on the market for a year and Google Scholar doesn’t even know it exists.  When Google can’t find you, you know you’re off the grid.  But there you go.  Inside my chapter in that book, you will find my idea for inferring and quantifying team process when teams are interdependent.  As a short summary, you find out about team process by understanding the distribution of team member expectations.  What does each team member expect will happen as the team completes its task?  Variability in expectations can be used as a proxy for effective (low variability) and ineffective (high variability) team processes.  If everyone is on the same page as it were, things are good.  If everyone is not on the same page, things are bad.  Moreover, there are multiple ways the team could be on the same page or not.  To continue the metaphor, the team members could all be on different pages, or they could all be on the same page but reading different books.  A specific statistical method can be used to figure out in what way the team is or is not on the same page.  There’s a lot more to it than those short sentences, especially on the quantitative end.  If you would like the full argument, read the chapter.  Maybe you can get the book through interlibrary loan or something.

Part 2:  In Which Jared Contemplates Publishing the Methodology

So there was that whole thing.  There was some good to come out of feeling under appreciated.  It got me wrestling with the idea of publishing the methodology I use to calculate my Completions Away from Average metric.  I want you to know that I believe I have a fundamental, scientific duty to publish this methodology.  It’s not scientific to purposefully keep people from verifying, replicating, and questioning my work.  I recognize that publishing the methodology is the quickest route to credibility.

But I’ve been dragging my feet about doing it.  I’m not proud of the reason, but I do feel I have some justification.  You see, I don’t have any job security.  For many reasons, some under my control and some not, I’m not a competitive job candidate for full-time professor jobs.  Instead, I’ve been cobbling together these one-year teaching positions to try and make ends meet.  If you do these non-secure contracts full-time, they let you call yourself a professor, but I don’t have a true professorship in any sense of the word.  My CAA methodology is useful, it’s powerful, and it’s the only thing I’ve ever created outside of academia that I think someone might pay me for.  Which makes me hesitant to publish it.

So what do I do?  I don’t have a job after June, I have something that I think is worth money, but I believe I have a fundamental duty to give it away.  A significant portion of my identity is tied up with being a scientist and educator, but I also enjoy eating.  Basically, the conflict breaks down into whether I think credibility or knowledge is more valuable.  I haven’t answered that question yet, which leads me to drag my feet about publishing. In the meantime, the only way to generate credibility is to take the “FiveThirtyEight approach.” Have a proprietary method, make some forecasts, and wait to see how the results match the predictions.

So, now you know what sort of questions I’m wrestling with right now.  I’d like to thank the Sloan Sports Analytics Conference and Twitter for giving me an excuse to talk about them.  Any advice from the internet would be welcome and appreciated.

page 1 of 1
Welcome , today is Saturday, June 24, 2017