On Analytics, Intangibles, and Why They Both Matter

By Alison Lukan on December 16, 2020 at 1:50 pm
Columbus Blue Jackets forward Oliver Bjorkstrand fights for the puck against Alexander Ovechkin of the Washington Capitals.
Aaron Doster – USA TODAY Sports

Yesterday, I put out a tweet as part of a bigger project I’m working on.

I got very good (and mostly serious) answers that are going to be a big help to me, but I also got plenty of cheeky responses (y’all think you’re funny) saying things like “grit” or “want it more.” “Be good in the room” was in there, too.

You know these phrases. They are supposed hockey cliches. “Intangibles.” Things we “can’t measure” that have been bandied about so much we often mock them, a behavior the analytics community is often assumed to perform the loudest.

But, as the answers kept pouring in, that trend of “intangibles” being mentioned increased in more and more thoughtful ways.

Now, regardless of whether you agree that these concepts (and others like them) are valuable to evaluating hockey players, what most people might all agree on is “sure, but you can’t measure that!”

Intangibles Quote from CBJHAC

This has been a crux of the argument critiquing analytics for years, not just in hockey but in many sports: “you can’t measure heart, but I know it matters.” How do you quantify whatever it was that propelled the Blue Jackets to come back from a 3-0 deficit in Game One of the Stanley Cup 2019 playoffs against Tampa Bay and not just win the game but sweep the series? That wasn’t what the data laid out – there had to be something more.

This is why “watch the game” became a common chirp directed at us hockey nerds – it was presumed that if you didn’t watch players play you couldn’t infer that often things happen that don’t make sense or don’t show up on a spreadsheet. You couldn’t enjoy the magic of sport that goes beyond completing a pass, making a save, or scoring a goal.

But here’s the dirty secret. Or at least, here’s my dirty secret.

EVERYTHING can matter. And just because you can’t measure it now, doesn’t mean it doesn’t matter.

A little bit about me. I graduated with a degree in Leadership Studies from the University of Richmond. And it’s still something I’m very proud of. We learned about the science (emphasis mine) of leading, following, team building, communications, and more. There are methods to the madness. There are ways of studying behavior and quantifying them.

As I cited above, back in 2015, the brilliant Carolyn Wilke was challenging us to change what we called “intangibles.” There’s literally a note on my iPhone from a conversation with her from October of that year (do I take notes when I have conversations with my friends? Yes. I have very smart friends. They teach me things.)

Wilke Notes

And going one step further than understanding that we can quantify things like leadership, there have been important voices that have challenged us with the how.

Someone who will always hold a place as a thought leader in my book is Stefan Wolejszo. In the midst of the 2014-2015 data revolution in hockey, Wolejszo was writing on his now shuttered blog about bringing in quantifiable ways to measure leadership, to measure motivation, to measure grit.

He spoke on the subject at hockey conferences, including a talk “Rank Ordering the ‘Intangibles,’” (Slides HERE) that outlined how data on things like resilience and grit could be collected and evaluated.

Wolejszo also talked about the importance of folding in “intangibles” to player evaluation and integrating it as part of a true analytical approach, something he outlined at the Rochester Hockey Analytics Conference (Slides HERE).

Wolejszo Slide

Slide by Stefan Wolejszo, 2015

So, if we follow this line of thinking, intangibles don’t live in isolation, nor do “analytics,” they meld together for comprehensive evaluation.

The other side of the coin is that some intangibles can tie to measurable data depending on how an organization so chooses to do so. At last year’s Blue Jackets Hockey Analytics conference, Colorado’s Arik Parnass made this point during panel discussions.

So popular to what some might think “analytics” and “intangible” evangelists believe, both analytics and intangibles deserve to have a seat at the table when it comes to evaluation; perhaps the challenge with the latter is that we may not always have a quantifiable way to measure them (yet) – and if we don’t, that is when the path ahead is fraught with peril.

And maybe that’s the problem we’ve been facing. Instead of debating if a big hit or a massive intermission speech matters, we should start to ask not “what matters?” but “how can we define how it matters?” There are tools out there…and it may not be easy going, but it’s possible.

Do we run the risk of taking the magic, or even the joy, out of the game if everything is quantified? I say no. I believe the true caution underlying concerns over unmeasured characteristics isn’t that they may not matter, but rather, that absent true quantifiability, we may over-value things. Is the ability to “lead” worth having a not so good player on the ice? To answer that question, we need scales of measurement because sometimes the value of leadership does compensate “enough” for the “not so good.” But sometimes it does not. And it’s those decisions that we still can’t make with authority.

Maybe in 10 years – perhaps sooner! – dismissing certain behaviors as “intangibles” will be a moot point. But until then, we owe it to ourselves to investigate things that we think matter, recognize that all things can be measured, and continue to evolve the way we evaluate the game.

View 2 Comments