Garmin Running Power App: The good, the bad, and the ugly.

DSC_6740

Today, Garmin released their running power app that they announced back in October.  The functionality, which requires an Garmin RD-POD, Garmin HRM-TRI, or HRM-RUN strap.  It also requires the latest Garmin watches, specifically either a Fenix 5, Chronos or a FR935.

While Garmin might say otherwise, the reality is that it’s meant to compete with both Stryd and RunScribe running power.  And in many ways it does.  If you’ve got the required watches and sensors listed above, then it provides equally as unverifiable numbers as those companies do.

I’ve been using it since October, and more lately also in conjunction with Stryd and RunScribe Plus, both of which provider running power to Garmin watches as well as competitor watches.  So I’ve got a pretty good idea on how things work, but at the same time, it’s really something best visited again after many months of usage on final products – such as next spring.

I’m going to change things up a bit for the format for this post, if for no other reason than I’m feeling the need for speed…err…difference?  Let’s move on.

The Good:

DSC_6718

The first thing you’ve gotta do is get the app installed on your device.  The good news is this is silly simple, and is just like installing any other Connect IQ app.  Technically speaking, the Garmin Running Power is actually a multitude of apps, and even further, it’s actually a data field, not a full blown app.  But that’s actually all good stuff – because it gives you way more flexibility.  All of these are currently published as from an entity called ‘GarminLabs’, as opposed to Garmin itself.  Likely this is Garmin trying to taper your expectations that this is all a bit new.  Here’s the full list of what’s available:

image

The titles might be a bit confusing at first, but it’s pretty easy.  Here’s the five apps you’ve got:

Average Running Power: Running power average for the entire workout
Combo Running Power: Current power, lap power, last lap power and average power on a single data page
Lap Running Power: Will show running power for your current lap
Last Lap Running Power: Will show running power for the previously completed lap
Running Power: Will show instantaneous running power

Like with cycling power, there’s a bit of volatility in the instant power number.  Like others in running power, Garmin has done some smoothing here to make the numbers easier to follow – but I still find it pretty variable.  As such, I kinda recommend one of two options: Lap Running Power, or the Combo Running Power.

With the ‘Lap’ option, you’ll get a single data field that you can place on any data page alongside other data fields (such as heart rate, pace, time…whatever you want).  Anytime you press the lap button like normal, I’ll keep the average for that lap as what you see.  This is great for doing most interval workouts, since your ultimate goal for any interval workout should be the preset target (be it pace, time, cadence, etc…).  It’s no different with power.

2017-12-08 11.30.48 2017-12-08 11.30.32 2017-12-08 12.10.32

Alternatively, you can use the combo data field, which gives you ‘All the powers’ on one field.  This is great if you want a dedicated data field to running power.

image

Now remember that Garmin’s Running devices are limited to two Connect IQ data fields being active at once.  You can install additional fields on your device, but you can only have two active in a workout at any point in time.  And even further, you can’t reuse one of those fields on multiple pages.  That limits you a bit.

For example, in my running setup I’ve historically had heart rate (HR – Current) on almost all my pages.  Some pages are dedicated to lap stats (except HR, which is always current for me), while others are totals (I.e. total run time/distance/etc…).  I can’t setup 4 pages and put power on all of them, it can only be in two spots.

Now remember, before you start running, you MUST put at least one of the Running Power data fields somewhere on your run screens, otherwise it won’t record anything.  It’ll record the same data no matter which one you put on there (since it’s just varying views during the run, the underlying data isn’t impacted).  Simply go into your data field settings on the watch and you’ll find these fields under Connect IQ:

DSC_6734 DSC_6735 DSC_6736

With all that said, we’ll sit for 10 minutes.

Actually, you don’t really have to – unless you’ve been following this guide and truly do plan to go out and run immediately.

See, Garmin actually pulls wind data from nearby weather stations.  In order to get the data to your watch, it needs to ensure it’s been connected to your phone at some point in the last hour for at least 10 minutes with the Garmin Connect Mobile app enabled somewhere in the background.  If you use your watch as a general smartwatch, then there’s no issues here – it’s connected 100% of the time as it uploads steps and other metrics constantly anyway.  If however you don’t wear your running watch all day, then just put it next to your phone while you get ready.

image

With that set, be sure that you’ve got your HRM-TRI, HRM-RUN, or RD-POD connected/enabled.  Remember, you need one of these (the one in the back is the older HRM-RUN with the runner man icon on the front):

DSC_6721

So now we’d head out for a run.  While running we’ll see a power number on the watch as we’d expect.

GOPR0365

In general with Garmin’s running power algorithms this number will be higher than your cycling power for the same effort (by a fair bit).  This is neither right or wrong. It just is.  Despite how much as logical humans we want these two to match, there’s actually no scientific or mathematical basis for them to match.  As with all things matching (clothes, musicians in an orchestra, manhole covers with yellow lines painted atop them reinserted the right way after servicing), nice doesn’t mean real.

Now there are some caveats here.  For example, you can’t set any power targets in a custom workout, though you can create zones to show within the settings (as well as set basic high/low alerts using the CIQ app configuration).  But neither of those have the same flexibility as a native data field like heart rate.  There are other quirks as well like you won’t see running power metrics in the totals at the end of the run, nor in history on the device.

Note that Garmin has confirmed they do leverage secondary speed sources within the algorithms.  So for example all of their current running watches are using wrist based accelerometers to help taper pace against GPS pace oddities (just as Suunto and others do).  Garmin declined to specify to what degree exactly, but this helps against GPS pace blips.  If you have a footpod, it’ll use that as the pace/speed source.

Anyway, you’ll run your run and get power along the way.  Once complete, you’ll see running power on Garmin Connect down in the Connect IQ sections:

image

You’ll also see it on Garmin Connect Mobile too:

2017-12-08 14.48.26

And then that data is stored within the .FIT file so that 3rd party apps can access it.  Though, it won’t show up as ‘power’ in most 3rd party apps.  Instead, those apps have to be coded to understand that specific Connect IQ app and the data from it.

But overall, the concept of running power in Garmin’s apps works fine from a functional standpoint.  But let’s talk a bit about where things are messier.

The Bad:

DSC_6729

It’s probably a bit harsh to label this section ‘The Bad’. But ultimately, that’s sorta what it is.  But if you’re skimming, don’t mistake this section as saying ‘Garmin is bad, Stryd & RunScribe is good’, as that’d be a flawed takeaway.  Instead, it’s me saying “The whole situation is….squishy.’

And by ‘situation’, I’m referring to whether or not these numbers are accurate.  But it’s not just Garmin, rather, everyone.   See, the first thing to understand about running power is that with the exception of Arion, all these companies are using accelerometers and gyros to calculate running power.  Arion is using pressure sensors in your insoles (shoes) to more directly measure it, but even that has caveats around sheer forces.

(Side note: I’m not using Arion running power data here because at present it’s too messy to collect, normalize, and graph. Until they roll out some planned features in that area, I’ll be holding off a bit.)

So anytime we hear the word ‘calculate’, we have to understand that not all calculations are equal.  Just like when different watches produce different calorie calculations, the same is true here.  Different companies all have equally smart people, and those people sometimes disagree on what’s truly correct.  No different than employee disagreements at your work place, or between your company and your competitors.

And in this case, it’s not just the companies disagreeing – but also scientists too.  Depending on which study you use as the basis for your device, you’ll get different results.  All of which temporarily ignores those in the science community that disagree with running power as a concept at all.

So – a more interesting question might be how they compare?  After all, with Stryd, RunScribe, and Garmin all out there producing power data, how might an overlay look?  Glad you asked, and here ya go (sorry, the automated colors are a bit hard to see on this one, I’ll try and get that fixed shortly):

image

What you’re looking at above is a comparing of Garmin Running Power, Stryd Running Power, and RunScribe Plus Running Power.  It doesn’t take a rocket scientist to see they don’t match 100%.  They do trend together though for the most part.

At first, one might presume a given one of them is incorrect (or correct), but here’s the thing – none of us can say that.  In fact, nobody can.  Anyone who tries to say that has an agenda here.  There’s simply no technology at this point that can account for everything can give ‘known good’ outside.

But the key word there is ‘outside’.

See, part of the challenge is account for two very specific items: Wind and terrain (specifically things like gravel, sand, etc…).  Only Garmin can somewhat account for wind, and none of them account for terrain.  So if I’m running into a headwind for an entire one-way marathon, then most of these technologies won’t account for the added wattage – it’s as if the wind doesn’t exist.  Garmin’s running power in theory does by using wind stations nearby, and that likely would have actually worked well for my marathon since it’s mostly wide open terrain.  But in the swirling winds of the city I live in, that falls apart somewhat.

The next is terrain, for which none understand that it requires more power to run on gravel, grass, or sand than it does perfect pavement.  For road runners that won’t be a huge issue, but trail runners it’s mostly a non-starter.

Here’s another longer run, with both hills as well as tunnels (since that shows how Garmin transitions from GPS to wrist-based pace).

image

So how about inside?  Glad you asked.  I tried that out too!  Here’s that:

image

In this simple test I started off and did 2.5 minutes at 10KPH, then 2.5mins at 12KPH, then 2.5mins at 14KPH, then 1 minute at 16KPH and 30 seconds at 18KPH.  You can see that Garmin starts much higher than Stryd, but as I get into my groove (at 14KPH), then RunScribe and Garmin are near identical.  They also track closer up higher.

But wait you say – what about all the studies they reference saying each is right?  It’s true, each company references different studies:

Stryd: Just yesterday, Stryd published this 16 page study showing how they compare to high end force plate treadmill.  And I applaud them for doing that.  Nobody else has yet.  But it’s also highly misleading, since it’s inside.  Most of us don’t compete in races on treadmills, but rather go outside where complexities like wind, terrain, and even just general air resistance are where the issues are with this device.

Garmin: Within their large running power FAQ page they list a huge number of scientific papers that they use for the basis of their calculations.  Like with Stryd, I too appreciate the clarity there.  Though, it’s a bit of a needle and a haystack to sort through as to what exactly they’re doing.

RunScribe: They’ve not only listed published papers, but also even listed the exact algorithm they’re using via a nifty calculator that you can play with.  Further, they’ve provided boatloads more information in their forums.  Out of all the approaches, I’d say this is the closest to what I’d like to see, though I’d love to see them also validate indoors as well (despite the caveats to indoors vs outdoors).

So about now you’re wondering whether or not each device is consistent to itself, and the answer is: Sorta.

I think Stryd is most consistent to itself from what I’ve seen (whether or not it’s accurate I don’t know).  I think Garmin is more volatile than the others, but also seems to more accurately capture sprints in terms of perceived effort (again, regardless of the specific number).  And RunScribe is somewhat in between, it’s tamer than Garmin’s volatility, but still seems closer to Garmin’s numbers.

Could you train by one given set of numbers?  Perhaps.  It brings up the always present issue though of what happens when you shift power meter providers.  Heck, even when I switched from Stryd’s chest strap to the running footpod sensor, my numbers shifted.  Some would argue that’s because placement is different (even between Garmin vs Stryd/RunScribe), and that’s true – but I’d also argue it shouldn’t substantially matter if we want to say running power is anything approaching semi-standardized.

Side note for prospective DIY testers: Do ensure that if you’re comparing multiple running power meters that your weight is correct and identical across all devices/sensors and watches.  It’s critical.  On some of my earlier runs from a month or so ago I think there was some differences in weights which may have impacted things.  All runs in this post are showing identical weights on all devices.

Also, in case you’re wondering how I created these graphs, back a few months ago the DCR Analyzer got updated to handle .FIT developer fields, which is where the data from all these apps gets recorded into.  This allows you to mix and match any developer fields and overlay them.  You can even do the same thing with the DCR Analyzer if you have two or more of the running power devices.  Remember that Connect IQ limits you to two fields active during a workout, so in my case I had to use two watches to get coverage of the three fields. If you only are comparing two devices, you can do it on a single watch.

And the Ugly:

DSC_6727

So what’s ugly?  Well, not actually the accuracy.  Because as I said above – you can’t verify that.  Nobody really can outdoors.  Indoors there’s some possibilities there, but even with that there’s disagreement in the scientific community on how what the definition of running power should be.

But that’s actually not where things get messy.  Rather, it’s on what devices it supports (or doesn’t), as well as what devices it may support in the future.

(Preemptive note #1: Some of this might get geeky, but I think it’s important to understand, and important to call Garmin out on this.)

(Preemptive note #2: Most of my annoyance here isn’t actually the power app itself, but how Garmin is doing things around it.  Effectively, this is a proxy war and the Garmin power app is the bystander.)

See, as you remember, the Garmin Running Power app has three specific requirements:

A) Must have a barometric altimeter
B) Must support Garmin Connect IQ 2.4
C) You must have a compatible Running Dynamics sensor

At first glance, item B is the only one that’s moderately annoying, because that excludes the Fenix 3/3HR, as well as the FR920XT.  The main reason for the CIQ 2.4 requirement basis is that Garmin wanted to leverage the new standard ANT+ Running Dynamics profile that came out earlier this year, which would be baked into CIQ 2.4.

But here’s the thing, the above is actually full of hidden caveats:

A) Must have a barometric altimeter*
B) Must support Garmin Connect IQ 2.4**
C) You must have a compatible Running Dynamics sensor***

Here, let me help you:

A) * – “We actually mean a device with with a barometric altimeter and its name is Fenix 5, Chronos, or FR935”
B) ** – “But only if that CIQ 2.4 device is named Fenix 5, Chronos, or FR935, Vivoactive HR/Vivoactive 3 need not apply even though it has 2.4”
C) *** – “But only if that ANT+ Standard Running Dynamics device is made by Garmin, not Wahoo or Stryd or anyone else who adopts the new standard.”

And items #B and #C are a bit of a backtrack in how Garmin has handled standards previously.  After all, the entire point of having standardized Connect IQ versions is to have uniformity for developers to aim towards.  Garmin says that #B was more of a miscommunication, and I can kinda see where that occurred, but it doesn’t change the resultant: I disagree with how it’s being implemented.

Let’s look at that CIQ 2.4 requirement first.  For that, Garmin had stated this was the CIQ level needed because they wanted to leverage the new Running Dynamics built into it.  That’s a laudable goal, as it forces Garmin to utilize that pipeline of information.  The downside though is that it makes it unavailable to previous Garmin units, specifically the Fenix 3/3HR and FR920XT – despite those units natively supporting Running Dynamics already.  In theory, CIQ 2.4 should have made the running dynamics pieces available to any CIQ2.4 device, so even something like the Vivoactive 3 or Vivoactive HR.

Except Garmin didn’t do that.  They decided not to roll out that feature to the ‘lesser’ watches, despite being totally capable of it. And certainly, that’s their right.  Product differentiation and all.  But it’s also an obvious developer double-standard that we haven’t really seen before.  It says Garmin will only roll out Connect IQ developer pieces to devices as it makes sense for their own business.  Again, that’s also their right.  But it also undercuts the premise of having an app development platform where aside from specific hardware limitations (storage/performance/altimeters/etc…), all devices are treated equally.  That’d be no different than Apple saying certain API’s are only available on the iPhone 8 Plus and not the iPhone 8 (or iPhone 6 vs 6c), merely because you paid more for that device. Rather than because there’s a physical hardware requirement.

But it gets better. Garmin then added that requirement #C:

“C) You must have a compatible Running Dynamics device***”

At first glance, that’s a perfectly understandable requirement.  Today, that basically means a Garmin RD Pod or HRM-RUN/HRM-TRI strap.  It needs the additional non-wrist based accelerometer data, just like other running power meters do.  That’s perfectly fine and logical.

Except for the catch: Garmin’s requiring that it be a Garmin branded device (aka RD-POD/HRM-TRI/HRM-RUN).

Why does that matter?

Because Wahoo has stated their intent to support the previously established ANT+ Running Dynamics standard that Garmin is supposedly leveraging here.  And other companies have shown interest as well.  After all, that’s sorta the entire point of creating a standard – to have other companies adopt it.

But Garmin is saying that doesn’t matter.  Instead, only if the ANT+ Running Dynamics device is made by Garmin will it work with Garmin Running Power app.  Garmin says that’s because it depends on the quality of the data, and needs high quality data from Garmin’s own accessories.  Except, there aren’t any devices sending out that data yet, so it’s a bit premature to say they’ll be any less accurate than Garmin (in fact, they could be more accurate). Furthermore, if that rationale held true, then shouldn’t Garmin block all 3rd party sensors?  After all, it doesn’t block 3rd party ANT+ heart rate straps which are used for lactate threshold estimation in running, nor does it block 3rd party ANT+ power meters which it uses for FTP estimation.  Nor does it block those sensors for feeding data into the recovery metrics.

So why do I care about this?  Because Garmin is basically trying to get credit for using Connect IQ to develop this versus doing it natively.  When in reality they’re leveraging cards they hold elsewhere to influence the end results – specifically which devices this does or doesn’t work with.  None of which impacts the functionality of those lucky enough to have the required hardware, but otherwise prevents plenty of people from trying it out.

Wrap Up:

DSC_6719

Here’s the thing: If you’ve got a Fenix 5, Chronos, or FR935 watch, along with an HRM-TRI/HRM-RUN/RD-POD, there’s no harm in loading up the app and sticking the data field in the background on one of your data pages.  It’ll silently collect running power data for you, even if you never look at it.  Perhaps a few months down the road you’ll look back and figure out some trends.

Or perhaps you’ll find value in it today.  Just as people find value today in Stryd and RunScribe Plus.  All of these have their pros and cons.  Despite how some people might view it – I don’t see it as a case of a tradeoff of price versus accuracy.  Rather, I think it’s more about different options that happen that vary in price with different features.

The real question may be where running power goes from here.  An ANT+ working group has been established, and various players are operating within it to establish standards and definitions.  That may be part of the first step to getting some consistency between the players.

And like with Stryd when they first started – I wouldn’t be surprised to see some shifts in power numbers from all the players over the next year, both in terms of algorithms as well as definitions of different metrics.  Either way, it’s kinda like being in on Bitcoin from the beginning. It’ll be an interesting ride and you’re likely to learn some cool stuff away.  Though, unlike Bitcoin, you’re not gonna get rich here.  Actually, it’ll probably cost ya money.

With that – thanks for reading!  Oh, and if you do some charted comparisons, I’d love to see those dropped below!

Source: dcrainmaker

Leave a Reply

Your email address will not be published. Required fields are marked *