Why We Don't Show "Calories" In the Trackster App

I've been wanting to share these thoughts with our community for a while now. Today, because our community is so rapidly growing, I feel it's important to break it down and give a thoughtful explanation. Here's my best.

There are 3 main reasons why our app, Trackster, does not include calorie data from smart watches (or anywhere else):

1. 'Calories burned' has proven to be the least accurate data point collected from smart watches.

2. No elite runner that we've worked with pays any attention at all to their caloric data from a smartwatch.

3. Platforms with a 'calories burned' data point are actually quite harmful, from anecdotal evidence.

First up: The scientific research on accuracy of smartwatch data points

The first commercial GPS watch

The very first GPS watch was the Casio PRT-1GPJ (Satellite Navi) which was soft launched at the Consumer Electronics Show (CES) in Las Vegas, January '99. Its premiere was well received, so much so that the watch went into full production and was released to consumers in June of that year. Since then, smart watches have gotten faster, more accurate and have had longer battery life - in line with Moore's Law.

For background, most modern GPS watches collect the following categories:

- Coordinates (location) & distance
- Time
- Heart Rate
- Cadence (pedometer)
- Elevation
- Pace
- Calories
- Stride length (newer devices)
- Power (newer devices)

While none of this data from a GPS watch is ever going to be truly 100% accurate beyond a shadow of a doubt, most of these categories have gotten so reliable that they are, if nothing else, valuable markers to use as a guide along the path of one's training journey.

The (mostly) accurate stuff

The heart rate monitors in these smart devices usually work by having a green LED light methodically pulse along the wrist of an athlete. This LED light and accompanying sensor can tell when its reflection and light is being obstructed by coursing blood in the veins in the wrist. It counts that obstruction as a heart beat and continues to build out a strong estimate of an athlete's BPM over more and more blood flow. Believe it or not, this method is relatively accurate as long as the strap on the wrist is fit properly - not too snug and not too loose. Some of the best devices can consistently track heart rate within 5% of a body's true value. (Worth noting - there are some documented issues with a person with a darker skin color as well as a variance in accuracy depending on the LED light that a manufacturer choices to use).

Infographic of heart rate LED system

From what I've found in my digging, as well as using these devices for over 10 years of my own training, Cadence is actually the most accurate set collected. These watches get Cadence values from a built-in pedometer. It makes sense that is the most sound set of values collected since pedometers are said to have been first invented in 1780 (!). So, watch makers have had plenty of time to perfect this technology.

Leonardo Da Vinci's sketch of a pedometer

OK - so just how off is the energy data?

I've taken the time to read through a dozen or so scientific studies published by various scientific orgs - Medicine & Science in Sports & Exercise, Stanford Medical, The National Library of Medicine, MDPI and more. Here I will summarize what I've found but will be sure to link the papers for you to read yourself. Essentially what these documents conclude is that the energy expenditure (EE) is a very difficult estimate to make indeed and that our GPS watches are set to make their best guesses only for a certain sub-group of humans. The further an athlete is away from this target demographic the more unreliable the data will be.

In one study, the calorie monitors first overestimated EE, or Calories, by anywhere from 16%-40% for walking and other physical activity. In the same study, later, for more "household" type activities, the monitors underestimated by 27%-34%.

"The magnitude of just how bad [the energy expenditure measures] were surprised me"

Stanford concluded in a different study that, confirming my earlier point, the GPS devices measured heart rate within 5% of the true value. "None [of the monitors], however, measured energy expenditure well" says Euan Ashley, DPhil, FRCP, professor of cardiovascular medicine, of genetics and of biomedical data science at Stanford. This study even found some of the devices calorie estimates to be off by 93%!

The final study I'll touch on was more specific about the devices they used - Apple Watch Series 4, Polar Vantage V, Garmin Fenix 5 and Fitbit Versa. They concluded the Apple Watch had the most accurate heart rate data with Polar in second. Beyond that, though, the research paper concludes by saying "none of the 4 wrist-worn wearables should be employed to monitor EE at the intensities and durations tested"

Where do we go from here?

Hopefully now it's clear that while some GPS & smart watches do certain things better than their competitors, literally not a single device on the market has proven to estimate energy expenditure with any consistent accuracy.

So, if the calorie data isn't reliably accurate, then why do the companies include it? And why do pro runners use these devices at all?

First, and this is totally speculating, I believe the GPS companies include the inaccurate EE data simply because of peer pressure and competition in the market place... Everyone is doing it! If a customer is choosing between watch A with every feature except calorie estimates and watch B comes with every feature including calorie estimates - they're going to choose watch B! The customer is completely oblivious to the fact that the caloric estimates are mostly garbage.

Second, from talking with pro athletes and elite runners, we've found that top-tier athletes take in all of their data with a proverbial grain of salt - while just altogether ignoring the calorie data completely. Also, some elite coaches don't even like looking at or using GPS watches in any regard whatsoever. (In my opinion that level of abhorrence is dumb TBH). In short: nobody who's an elite athlete cares about the EE data from their watches. They use actual nutritionists and professionals instead. The GPS watches are mainly used for time keeping, heart rate and location tracking.

The final blow to caloric data's legitimacy ( E.D. trigger warning )

Let's imagine that everybody in the world now knows the truth about EE and calorie data on watches. So, during the daily grind of training, an athlete acknowledges their watch's EE estimates with a healthy amount of skepticism - if not just totally ignoring it. This would be great! There's no downside, by definition, in having this poor estimate offered if people are aware of its weaknesses. If everybody knows that the Calories field is simply the watch's best guess, mostly incorrect, and it will get better over time then there's no harm and no foul... right??

MyFitnessPal issues with calorie data; Credit BBC

This is unfortunately far from the reality! Most users take their GPS data and, more generally, their training app's data as inarguable fact. If the watch says X, then X must be true! In regards to caloric data, this has lead to serious offshoot issues. According to an article by the BBC, they were able to easily find more than 20 harmful entries created by MyFitnessPal users that included "dangerous cycles of starving and binging". One user, Holly, stated that she believes calorie counting apps enabled a "cult community" for people suffering with eating disorders. Here's another blurb, direct from the BCC:

After doing this research myself I can say that Trackster will forever be a platform without any caloric data whatsoever (unless something with the smart devices methods & algorithm changes drastically). It's just not worth it.

I will forever be committed to building an app, a community and a brand that makes it easy for athletes & coaches to keep track of all of their training data... so long as the training data is accurate and healthy. Energy expenditure and calorie estimates from the devices currently on the market, in my mind, do much more harm than good. They're just completely inaccurate and misleading!

In conclusion, why would I want to include data that is both harmful and inaccurate on our budding training platform? Because everyone (Strava, MyFitnessPal, etc) are doing it? I don't think so. I'd truly rather lose all of our users than compromise the accuracy and safety of our platform. Let me know what you think and be sure to tell me what additional data you would like to see!