What Exactly Do You Know?

Categories: General Info, Statistics
Comments: 1 Comment
Published on: October 14, 2014

(This post was inspired, in part, by a post Matt Waldman posted on his website a few days ago. If you haven’t already, go check out his site. He does an amazing job over there. The post in question is titled “Deny Emotion and You Only See a Fraction of the Game.”)

The establishment in sports has, for a while now, been telling outsider-sports-analytics types that one of the main barriers to widespread acceptance of analytics rests on the ability of the quants to communicate with the non-quants in ways that the non-quants can understand. I’ve covered the problems of communicating information both to quants and to non-quants in the past. But Matt’s post about emotion made me realize something more.

His general point was about the role emotion plays in sports performance. Specifically, he was contemplating whether or not momentum exists in sports. He cites conversations with analytics experts on the subject of momentum. If you’re familiar with the argument, most analytics experts conclude momentum does not exist. This is a fairly standard finding across many sports – basketball’s hot hand being another classic example. Matt’s reaction to that conclusion is also fairly standard among non-analysts. His argument is that emotion, and the effects of emotion on situations in sports are obviously occurring. Anyone who denies that is missing a huge portion of the signal inherent in the game. At one point he suggests that analysts have possibly never put themselves in physically dangerous situations and felt the impact of emotion first hand.

So, two things with that summation, both having to do with communication. First, assuming experiences that someone has or hasn’t had is a raw nerve for me. The stereotype of the milquetoast academic who simulates experiences rather than having actual experiences looms large over me. I’m guessing it has something to do with my father telling me to put down the video games and go spend some time working on my grandfather’s farm. But, I know the comment wasn’t directed at me and it wasn’t malicious anyway, so let’s put aside any irritation that might shut our brains off. In fact, we need to keep our brains on if we’re going to truly examine the point Matt is trying to get at.

Analysts (myself included) are often guilty of a particular linguistic shorthand. Our job is to find predictive effects. Does changing the way a request is phrased reliably change donations to charity? If I know about your height, can I make an educated guess about your weight? That sort of thing. We can become so familiar, so practiced in that job that, when we talk to other people, we tend to shorten the description of what we’re talking about. When my neighbor asks me what I know about momentum in sports, I say “I’m trying to find out if momentum exists” and he gets excited and engages with the conversation. The problem for analysts, though, is that this gets the conversation off on a disingenuous foot. Because we’re not truly trying to find out if momentum exists. We’re trying to find out if the predictive effect of momentum on some other variable, like scoring, exists. But we repeat the phrasing about studying the existence of momentum so much that we can forget that other people see that collection of words as having a different meaning.

And when we analyze the results in sports, the results a pretty clear. The effect of momentum on any variable we look at is unpredictable. The same is true in the academic literature as well. We cannot predict the effect of emotion on motivation with any reliability. So, I agree that we should probably stop saying that momentum doesn’t exist. The subjective experience of the emotion of the game is a real thing that people feel.  But I will hold fast to the notion that predicting what will initiate a change in momentum and how a change in momentum will impact athletic performance is an unpredictable enterprise.

On Communication

Categories: General Info
Comments: No Comments
Published on: March 6, 2013

It’s a rare weekday post from me.  I’m updating now rather than my normal weekend posts because I think the topic of conversation is fading rapidly.  I want to get my two cents while the iron is still slightly warm.

I mentioned last time that I Twittered into the Sloan Conference last weekend.  Since that time, the panel that seems to have had the most legs beyond the conference is the panel on communication.  Commentators have gone on at length about the good information that was in that panel essentially saying that quantitative people need to be concerned about how they are presenting their ideas.  In classic terms, this was the “thesis.”

Then people started mulling over the thesis and I saw some reactions against that line of thinking.  Some people saying that communication is a two way street and the burden is not solely on the quantitative people in the room.  Front office people also need to be learning rudimentary statistics to help facilitate that communication that they are looking for.  We can call this the “antithesis.”

My goal in this post is to provide some synthesis.  Let’s combine these ideas to see what we can take forward.  I often find that when I’m trying to communicate with non-analytic people, the issue is a difference in mindset.  I don’t mean to imply that every individual within these two general groups thinks in this way.  However, I do believe that the general tendencies exist and are worth mentioning.  What might Front Office People and Quantitative Analysts not know about one another that could be inhibiting this communication?

Front Office People – You need to understand that quantitative analysis is a process of pointing out what’s wrong.  Every day I show up at work and think up some ideas.  I spend the rest of the day trying to find a way to demonstrate that those ideas are wrong. If I don’t do this, my colleagues will do it for me when I present, or my peers will do it when I try to publish my ideas.  I have literally sat in a room while a friend and colleague asked me a series of questions that had the possibility of invalidating six entire years of work.  We don’t do this to each other because we’re assholes.  We do this because it is the quickest way to the best answer.  The idea is that if we spend all of our time, energy, and brain power on ripping down this idea, whatever can’t be ripped down must be worth pursuing.  The entire process of analysis is a process of falsification.  Always remember, when you pay an analyst to run some numbers you are paying them to tell you that you are wrong.

Analysts – There is a reason so many of us are depressed alcoholics.  The constant falsification and focusing on what we’ve done wrong rather than what we’ve done right is not a typically healthy mindset to be in.  Most of the rest of the world does not think this way.  It hurts to be reminded that we’re wrong.  We analysts have a pretty thick callus over that particular spot, but most don’t.  Most people want to be reminded that they are right.  They want to be affirmed and given positive reinforcement.  As analysts, we have to be very concerned with how we frame our responses.  The old clichéd story about the dream interpreter comes to mind.  The “your dream means your children will all die before you vs. your dream means you will outlive all with eyes on your thrown” thing.  The first interpretation gets the interpreter killed while the second gets the interpreter a cushy job and some treasure, but they are both the same interpretation.

Here is the message that I got from the communication panel.  Framing answers is what will truly separate the successful analysts from the unsuccessful ones.  The analysts that will continue to have jobs are the artists who can continuously tell powerful people they are wrong and still keep the powerful people happy.  That is not a trivial skill.  It takes effort and patience to reframe our traditional ways of thinking for those that aren’t as familiar with them.  And from the non-analyst side, analysts aren’t assholes, even though we might talk like we are sometimes.

page 1 of 1
Welcome , today is Tuesday, June 27, 2017