Measure What Matters: Correlation vs Causation
Data is not king. Data analysis is.
Every other week I write an email discussing what I learn launching and growing Reboot Motion. If you would like to receive it directly in your inbox, subscribe below.
I recently wrote a post on the importance of Measuring What Matters. The premise was simple:
We believe momentum-based biomechanics is the superior way to analyze movement.
That belief is rooted in first principles of physics and physiology.
Beliefs are great; data is better.
With this in mind, we tested our proprietary metrics vs key outcomes teams care about- velocity for pitchers and bat speed for hitters.
Our early results saw various body parts’ peak momentums, rotation plane alignment, and ranges of motion (ex. hip shoulder separation) correlating to better performance for pitchers and hitters.
While it would be great if we could stop there, early testing is just the start.
So far, all we have done is shown evidence that, more often than not, two metrics move together.
Our goal is to help coaches help athletes move better, which means the bar must be much higher than correlation.
We are specifically looking for cause and effect. We are looking for causation.
The Difficulty in Proving Causation
Again, our jobs would be easy if every time we looked at data, we could simply pick out the metrics that went up with velocity and bat speed, and tell athletes focusing on these will improve performance.
But that is not how data analysis works, for a few major reasons:
Today’s world has limitless data. The more variables we search, the more likely we are to find two that may appear related simply by chance.
That is, unless we believe in the strong correlation between US Highway Fatality Rate and Lemons Imported from Mexico
Or if we think the Washington Football team can predict the outcome of Presidential Elections.
In baseball today, front offices likely have 100x the data they had a decade ago. In fact, there is probably more data available in a single pitch than an organization used to capture from an entire game.
With so much information available, data is no longer king. Data analysis is.There may be a third variable lurking. Those that read Steven Levitt and Stephen Dubner's bestseller, Freakonomics, may remember the correlation between 1) books in the home and 2) a child’s education.
At the surface, this relationship makes sense: more books → more reading → better academic results.
However, researchers eventually realized actually reading those books had no impact. This is because books in the home were not increasing a child’s aptitude. Rather, as Levitt and Dubner describe it:
"Much more likely is that any family that has 100 children's books in the home is likely to be pretty highly educated to begin with, is starting out with a pretty high IQ, and values or treasures or rewards education to begin with."
In other words, more books did not improve the likelihood of strong academic results. High achieving parents, who are more likely to already have lots of books, did.
For those that want a sports example, here is a fangraphs breakdown of the top 10 teams in terms of innings pitched by relievers in 2021:
And here are the top 10 (11 since there was a tie) records:
Seven of the league’s leaders in innings pitched by relievers also sit atop the standings.
Does relying on relief pitchers lead to more wins? Or, is it possible, each of these metrics is the result of something else?
It is far more likely 1) deep, talented rosters or 2) front offices willing to test new strategies lead to both of the above datapoints.
Causality may be working the other way. Any sports fan has heard about the correlation between running the ball and winning football games.
While some analysts still use this to bury coaches that “didn’t establish the run”, the rest of us can roll our eyes or scream at the TV “that’s because they’re already winning!”.
Baseball analysts may have their flaws, but at least no MLB broadcasts tout the need for a team to put in their closer more- because teams that do, generally win.
Not Your Father’s Biomechanics Reports.
In the sports world, we rarely can do interventional studies- all we can do is observe. Therefore, we need to fully understand what we are looking at before we make any conclusions about training, coaching, roster decisions, and more.
For Training
At Reboot, we worry that a lot of traditional biomechanics reports fall victim to the above traps, blindly trusting r-squared values without a deep understanding of movement.
The process seems simple.
We know what elite looks like. For pitching, higher fastball velocity is universally believed to be the #1 indicator of performance.
Given we can detail nearly every move an athlete makes (thanks motion capture technology!), the logical next step is to run a regression analysis and see what input metric maximizes our key output variable.
We can look at joint angles at foot plant, joint angles at ball release, and more.
We can analyze an athlete’s trunk tilt at a specific time and see where it is relative to the “elite population”.
And we can say, “on average, the best athletes do X, you should do X too.”
Unfortunately, this is flawed. Athletes are different. They have different heights, weights, flexibilities, strength, and more. And, given the complexity of human movement, these differences need to be accounted for.
So how do we account for them? Let’s compare the pitcher’s below.
Visually, we quickly notice the bottom pitcher has a much greater trunk tilt than the top one.
Is the bottom pitcher tilting too much? Is the the top one not tilting enough?
Any model that includes trunk tilt has to pick a side, because any model will hold all else equal. Unfortunately, that is not how the human body works.
In fact, we think both pitchers do a great job aligning their torso with their arm and therefore the ball. The bottom pitcher may be tilting too much for those that are slaves to a model, but we believe their tilt is the result of 1) great alignment and 2) a vertical throwing motion, which has its own advantages- notably induced vertical break.
If a coach told the bottom pitcher to reduce their trunk tilt to get closer to “ideal”, they would likely be 1) reducing their ability to transfer momentum from the torso to the ball or 2) changing their arm slot.
(And yes, we can be confident in this takeaway not just because our data tells us torso alignment is correlated with increased velocity, but because the laws of physics back up a causal relationship.)
In short, no two athletes are the same. We shouldn’t expect their biomechanics to be.
For Analysis
Reboot’s metrics may be different than a lot of others, but that does not mean we are immune to having to think critically about our results.
For example, we capture each pitcher’s maximum lead knee range of motion- or in layman terms, the maximum bend of a pitcher’s lead knee.
When we ran the numbers we noticed less knee bend correlating with higher velocity.
The first thing anyone should realize is- taking it to the extreme- no one is recommending a pitcher do their entire motion with a completely straight lead leg.
In others words, universally saying “less knee bend is better” is simply wrong.
Rather, by combining 1) data 2) physics and 3) lessons from great coaches, we can arrive the following hypothesis:
Each pitcher likely has their own ideal bend. More often than not, pro pitchers are over-bending, not under-bending, which we believe is due to an over emphasis on stride length.
While stride length may be beneficial in a vacuum, it also can create an overly bent knee, making it difficult to extend after foot plant and during rotation.
We are confident that being too bent (or too vertical for that matter) at foot plant inhibits the athlete’s ability to extend the knee, rotate the pelvis, and begin rotating the upper half.
Going into this much detail should be table stakes. We need to explain to our partners what our metrics say…and just as importantly what they do not say.
Our goal is to optimize the leverage we provide coaches, biomechanists, and data scientists. This only happens if we are clear with our metrics, and open with our process.
Proof Plus
In a world where executives and coaches may make a single decision that impacts an athlete’s career, a team’s chance at a championship, and a franchise’s bottom line, everyone needs to be confident in their conclusions.
That is why we subscribe to a “proof plus” model.
Data obviously matters. Being able to show takeaways like these obviously matters.
But so does being able to explain why.
It’s great that a 12.5% gain in torso alignment increases fastball velocity by 1 MPH. But in no way should coaches blindly trust us and maximize torso alignment without fully understanding why it will help.
Rather- this data, plus…
The knowledge that fastball velocity is produced by the pitcher creating ground reaction forces and transferring that energy through the body, AND
The understanding that there is a desired path that energy transfer takes, AND
Knowing that the more an athlete’s body parts are aligned, the more efficient that energy transfer will be,
…allows decision makers to be confident in their conclusions.
