├── screenshots ├── help.png └── token_extraction.png ├── requirements.txt ├── .gitignore ├── LICENSE ├── setup.py ├── README.md └── nrc_exporter.py /screenshots/help.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yasoob/nrc-exporter/HEAD/screenshots/help.png -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | colorama==0.4.3 2 | gpxpy==1.4.2 3 | requests==2.31.0 4 | selenium==3.141.0 5 | selenium-wire==1.2.3 -------------------------------------------------------------------------------- /screenshots/token_extraction.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/yasoob/nrc-exporter/HEAD/screenshots/token_extraction.png -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | geckodriver.log 2 | activities/* 3 | gpx_output/* 4 | website.png 5 | build/* 6 | dist/* 7 | *.egg-info/* 8 | .DS_Store 9 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2020 M.Yasoob Ullah Khalid ☺ 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import setuptools 2 | 3 | with open("README.md", "r") as fh: 4 | long_description = fh.read() 5 | 6 | with open("requirements.txt", "r") as fh: 7 | REQUIRES = [line.strip() for line in fh.readlines()] 8 | 9 | setuptools.setup( 10 | name="nrc-exporter", 11 | version="0.0.7", 12 | author="Yasoob Khalid", 13 | author_email="hi@yasoob.me", 14 | description="This program allows you to export your runs from Nike Run Club and convert them to GPX format", 15 | long_description=long_description, 16 | long_description_content_type="text/markdown", 17 | url="https://github.com/yasoob/nrc-exporter", 18 | install_requires=REQUIRES, 19 | classifiers=[ 20 | "Programming Language :: Python :: 3", 21 | "Programming Language :: Python :: 3.4", 22 | "Programming Language :: Python :: 3.7", 23 | "Programming Language :: Python :: 3.8", 24 | "Programming Language :: Python :: Implementation :: CPython", 25 | "License :: OSI Approved :: MIT License", 26 | "Operating System :: OS Independent", 27 | ], 28 | python_requires=">=3.4", 29 | py_modules=["nrc_exporter"], 30 | entry_points={"console_scripts": ["nrc-exporter = nrc_exporter:main"]}, 31 | ) 32 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # :running: NRC Exporter 2 | 3 | Download your runs from Nike Run Club and convert them to GPX format that can be imported in other running apps. I wrote an article about how I reverse engineered the NRC APK [at my blog](https://yasoob.me/posts/reverse-engineering-nike-run-club-using-frida-android/). You might find it a fun read if you are into that sort of stuff. 4 | 5 | ## :page_with_curl: Introduction 6 | 7 | There was a time when I was a huge fan of Nike Run Club. It was the first running application I got hooked on when I started this sport. Later on down the road I realized that most of my new running friends were using Strava. I wanted to move my old runs from NRC to Strava but couldn't find a way to do it. Nike had recently removed the option to extract your data so I was stuck. 8 | 9 | I did what any programmer would do. I spent a weekend trying to whip up a Nike Run Club data exporter. This program extracts all of your runs which have associated GPS data and converts them into the GPX format that can be imported to Strava. 10 | 11 | I have made this program in a modular way with helpful docstrings for all functions. You are more than welcome to add extra features you need in this program. If you aren't tech-savy and/or want my help please open up an issue and we can figure it out from there. 12 | 13 | ## :wrench: Installation 14 | 15 | You can either install the package from [PyPI](https://pypi.org/project/nrc-exporter/) or [source](https://github.com/yasoob/nrc-exporter). 16 | 17 | ### Using pip 18 | 19 | The PyPI method is the easiest as it installs the dependencies as well (other than geckodriver). Run the following command to install from PyPI: 20 | 21 | ``` 22 | $ pip install nrc-exporter 23 | ``` 24 | 25 | If everything goes well, you should be able to run this command: 26 | 27 | ``` 28 | $ nrc-exporter --help 29 | 30 | _ _ ____ ____ _ 31 | | \ | | _ \ / ___| _____ ___ __ ___ _ __| |_ ___ _ __ 32 | | \| | |_) | | / _ \ \/ / '_ \ / _ \| '__| __/ _ \ '__| 33 | | |\ | _ <| |___ | __/> <| |_) | (_) | | | || __/ | 34 | |_| \_|_| \_\____| \___/_/\_\ .__/ \___/|_| \__\___|_| 35 | |_| 36 | 37 | ~ Yasoob Khalid 38 | https://yasoob.me 39 | 40 | 41 | usage: nrc-exporter [-h] [-e EMAIL] [-p PASSWORD] [-v] [-t TOKEN] [-i INPUT] 42 | 43 | Login to Nike Run Club and download activity data in GPX format 44 | 45 | optional arguments: 46 | -h, --help show this help message and exit 47 | -e EMAIL, --email EMAIL 48 | your nrc email 49 | -p PASSWORD, --password PASSWORD 50 | your nrc password 51 | -v, --verbose print verbose output 52 | -t TOKEN, --token TOKEN 53 | your nrc token 54 | -i INPUT [INPUT ...], --input INPUT [INPUT ...] 55 | A directory or directories containing NRC activities 56 | in JSON format.You can also provide individual NRC 57 | JSON files 58 | ``` 59 | 60 | ### From Source 61 | 62 | Just clone the repo, install the dependencies and run it. 63 | 64 | - Clone it: 65 | 66 | ``` 67 | $ git clone git@github.com:yasoob/nrc-exporter.git 68 | ``` 69 | 70 | - Install dependencies: 71 | 72 | ``` 73 | $ cd nrc-exporter 74 | $ pip install -r requirements.txt 75 | ``` 76 | 77 | - Add Geckodriver in path 78 | 79 | The automated access token extraction makes use of selenium and geckodriver. I used geckodriver instead of Chrome because Chrome (selenium) was being blocked by Nike (Akamai Bot Manager) but geckodriver was not. This is an optional step. If you want to try automated extraction make sure you have downloaded the geckodriver from [here](https://github.com/mozilla/geckodriver/releases) and the binary is in your PATH. 80 | 81 | - Run it: 82 | 83 | ``` 84 | python nrc_exporter.py --help 85 | ``` 86 | 87 | If everything goes well you will see the following text: 88 | 89 | ``` 90 | _ _ ____ ____ _ 91 | | \ | | _ \ / ___| _____ ___ __ ___ _ __| |_ ___ _ __ 92 | | \| | |_) | | / _ \ \/ / '_ \ / _ \| '__| __/ _ \ '__| 93 | | |\ | _ <| |___ | __/> <| |_) | (_) | | | || __/ | 94 | |_| \_|_| \_\____| \___/_/\_\ .__/ \___/|_| \__\___|_| 95 | |_| 96 | 97 | ~ Yasoob Khalid 98 | https://yasoob.me 99 | 100 | 101 | usage: __main__.py [-h] [-e EMAIL] [-p PASSWORD] [-v] [-t TOKEN] [-i INPUT] 102 | 103 | Login to Nike Run Club and download activity data in GPX format 104 | 105 | optional arguments: 106 | -h, --help show this help message and exit 107 | -e EMAIL, --email EMAIL 108 | your nrc email 109 | -p PASSWORD, --password PASSWORD 110 | your nrc password 111 | -v, --verbose print verbose output 112 | -t TOKEN, --token TOKEN 113 | your nrc token 114 | -i INPUT, --input INPUT 115 | A directory containing NRC activities in JSON format 116 | ``` 117 | 118 | ## :rocket: Usage 119 | 120 | You have multiple ways to run this application. You can either provide an email password combination, access tokens for Nike or a directory containing NRC activities in JSON format. 121 | 122 | - Email/Password 123 | 124 | This is probably the easiest way to run the application. The program will try to automatically extract the access_tokens for NRC by logging you in using Selenium and intercepting the requests. You will have to run nrc_exporter like this: 125 | 126 | ``` 127 | $ nrc-exporter -e yasoob@example.com -p sample_password 128 | ``` 129 | 130 | This method will probably be blocked by Nike in the near future. If it doesn't work use the access tokens method described below. 131 | 132 | - Access Tokens 133 | 134 | This is useful for when the program is unable to extract the tokens automatically. You will have to manually provide the access tokens to the program. If you don't know where to get the access tokens from, just run the program without any arguments and it should automatically open the URL where you can log in. For extracting the tokens from that page check out [these instructions](#heavy_dollar_sign-extracting-access-tokens). Once you have the tokens, you can run nrc_extractor like this: 135 | 136 | ``` 137 | $ nrc-exporter -t 138 | ``` 139 | 140 | - Activities folder 141 | 142 | Some of you might have already downloaded your NRC runs data using [this script](https://gist.github.com/niw/858c1ecaef89858893681e46db63db66) and are now wondering how to convert that JSON data to GPX data. Put all of those activity JSON files in a folder and pass that folder's name to nrc_extractor. Let's suppose you put all of those files in the `activities` folder. It should look something like this: 143 | 144 | ``` 145 | $ tree activities 146 | activities 147 | ├── 0019f189-d32f-437f-a4d4-ef4f15304324.json 148 | ├── 0069911c-2cc8-489b-8335-8e613a81124s.json 149 | ├── 01a09869-0a95-49f2-bd84-75065b701c33.json 150 | └── 07e1fa42-a9a9-4626-bbef-60269dc4a111.json 151 | ``` 152 | 153 | Now you can run `nrc_extractor` like this: 154 | 155 | ``` 156 | $ nrc-exporter -i activities 157 | or 158 | $ nrc-exporter -i 07e1fa42-a9a9-4626-bbef-60269dc4a111.json 01a09869-0a95-49f2-bd84-75065b701c33.json 159 | ``` 160 | 161 | ## :heavy_dollar_sign: Extracting access tokens 162 | 163 | Nike uses Akamai Bot Manager which doesn't allow scripts to automatically log users in and extract the access tokens. Sometimes you might be lucky and automated token extraction works but mostly you will find the automated token extraction to be broken. Luckily, manually extracting the access token isn't too hard. 164 | 165 | ### From local storage 166 | 167 | Open up your favorite browser, open developer tools, and head over to this [login url on the main page](https://accounts.nike.com/lookup?client_id=4fd2d5e7db76e0f85a6bb56721bd51df&redirect_uri=https://www.nike.com/auth/login&response_type=code&scope=openid%20nike.digital%20profile%20email%20phone%20flow%20country&state=9cda65dea2c240cb81ed583bf733c122&code_challenge=WbnqwTvTmE-O4jVtcs4GYKvr0dhFgppcM2Hrl1qEEiU&code_challenge_method=S256) and log in. 168 | 169 | Submitting the form will not do much. You will just have a blank page in front of you but you will be logged in. Now in order to extract the access tokens, open up developer tools. You can search online about how to open the developer tools for your specific browser. Once the developer tools are open, click on the `Console` and type this: 170 | 171 | ``` 172 | JSON.parse(window.localStorage.getItem('oidc.user:https://accounts.nike.com:4fd2d5e7db76e0f85a6bb56721bd51df')).access_token 173 | ``` 174 | 175 | This should print your access tokens on screen. If this doesn't work and/or gives you errors, just click on storage and check out local storage. You should be able to find `access_token` as part of the value for a particular key. It should look something like this: 176 | 177 | ![Extract key](https://raw.githubusercontent.com/yasoob/nrc-exporter/master/screenshots/token_extraction.png) 178 | 179 | ### From network tab 180 | 181 | If the local storage doesn't contain any access tokens, open up your favorite browser, *open developer tools*, and head over to this [alternate login url](https://unite.nike.com/s3/unite/mobile.html?androidSDKVersion=3.1.0&corsoverride=https://unite.nike.com&uxid=com.nike.sport.running.droid.3.8&locale=en_US&backendEnvironment=identity&view=login&clientId=WLr1eIG5JSNNcBJM3npVa6L76MK8OBTt&facebookAppId=84697719333&wechatAppId=wxde7d0246cfaf32f7) and log in. 182 | 183 | Then go to the "network" tab in the developer tools. A couple of requests should be visible there assuming you logged in after opening the developer tools: 184 | 185 | ![image](https://user-images.githubusercontent.com/3696393/86953188-0f6be700-c122-11ea-8140-9c1ff135632f.png) 186 | 187 | Click on the `/login` request and check the response. It will contain the required `access_token`: 188 | 189 | ![image](https://user-images.githubusercontent.com/3696393/86953412-696cac80-c122-11ea-851e-a1516f5e302f.png) 190 | 191 | 192 | Now copy these `access_tokens` and provide them to the program. 193 | 194 | ## :heavy_exclamation_mark: Limitations 195 | 196 | This was a weekend project so there are definitely a lot of rough edges to this script. Try it at your own risk. I have extracted my runs successfully with this program so I am hopeful that it will work for you too. In case it fails please open up an issue and I will take a look. 197 | 198 | For now, one major isssue is that the script does not correctly add elevation data to the GPX file. NRC provides us with the ascent and descent data of different runs but I am not sure of the math that is required to convert that into actual elevation data. This data wasn't particularly important for me to maintain for historic runs so I did not spend a lot of time on it. You are more than welcome to open up a PR if you know how to do it. 199 | 200 | 201 | ## :camera: Screenshots 202 | 203 | Who doesn't love screenshots? 204 | 205 | - Initial run 206 | 207 | ![help message](https://raw.githubusercontent.com/yasoob/nrc-exporter/master/screenshots/help.png) 208 | 209 | ## :satellite: Release 210 | 211 | This is for my own documentation. There are three steps involved with releasing a new version to PyPI after updating the code. 212 | 213 | - Increment the version number in `setup.py` 214 | - Build the distribution package 215 | 216 | ``` 217 | python setup.py sdist bdist_wheel 218 | ``` 219 | 220 | - Upload to PyPI: 221 | 222 | ``` 223 | python -m twine upload --skip-existing --repository pypi dist/* 224 | ``` 225 | 226 | ## :scroll: License 227 | 228 | This program is distributed under the MIT license. You are more than welcome to take a look, modify and redistribute it (even for commercial purposes). Just make sure that the LICENSE file stays intact and you redistribute it under the same license. 229 | 230 | ``` 231 | MIT License 232 | 233 | Copyright (c) 2020 M.Yasoob Ullah Khalid ☺ 234 | 235 | Permission is hereby granted, free of charge, to any person obtaining a copy 236 | of this software and associated documentation files (the "Software"), to deal 237 | in the Software without restriction, including without limitation the rights 238 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 239 | copies of the Software, and to permit persons to whom the Software is 240 | furnished to do so, subject to the following conditions: 241 | 242 | The above copyright notice and this permission notice shall be included in all 243 | copies or substantial portions of the Software. 244 | 245 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 246 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 247 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 248 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 249 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 250 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 251 | SOFTWARE. 252 | ``` 253 | -------------------------------------------------------------------------------- /nrc_exporter.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- 3 | """ 4 | nrc_exporter 5 | ~~~~~~~~~~~~ 6 | A program to export and convert NRC activities to GPX. 7 | :copyright: (c) 2020 by Yasoob Khalid. 8 | :license: MIT, see LICENSE for more details. 9 | """ 10 | import os 11 | from xml.etree import ElementTree 12 | import sys 13 | import time 14 | import requests 15 | import argparse 16 | import webbrowser 17 | import logging 18 | import gpxpy.gpx 19 | import json 20 | from json.decoder import JSONDecodeError 21 | import datetime 22 | from colorama import Fore, Style 23 | from seleniumwire import webdriver 24 | from selenium.webdriver.support.ui import WebDriverWait 25 | from selenium.webdriver.support import expected_conditions 26 | from selenium.webdriver.common.by import By 27 | 28 | __version__ = "0.0.1" 29 | __author__ = "Yasoob Khalid" 30 | __license__ = "MIT" 31 | 32 | ACTIVITY_FOLDER = os.path.join(os.getcwd(), "activities") 33 | GPX_FOLDER = os.path.join(os.getcwd(), "gpx_output") 34 | LOGIN_URL = "https://www.nike.com/in/launch?s=in-stock" 35 | MOBILE_LOGIN_URL = ( 36 | "https://unite.nike.com/s3/unite/mobile.html?androidSDKVersion=3.1.0" 37 | "&corsoverride=https://unite.nike.com&uxid=com.nike.sport.running.droid.3.8" 38 | "&locale=en_US&backendEnvironment=identity&view=login" 39 | "&clientId=WLr1eIG5JSNNcBJM3npVa6L76MK8OBTt&facebookAppId=84697719333" 40 | "&wechatAppId=wxde7d0246cfaf32f7" 41 | ) 42 | ACTIVITY_LIST_URL = "https://api.nike.com/plus/v3/activities/before_id/v3/*?limit=30&types=run%2Cjogging&include_deleted=false" 43 | ACTIVITY_LIST_PAGINATION = ( 44 | "https://api.nike.com/plus/v3/activities/before_id/v3/{before_id}?limit=30&types=run%2Cjogging&include_deleted=false" 45 | ) 46 | ACTIVITY_DETAILS_URL = ( 47 | "https://api.nike.com/sport/v3/me/activity/{activity_id}?metrics=ALL" 48 | ) 49 | 50 | LOGIN_BTN_CSS = ( 51 | "button.join-log-in.text-color-grey.prl3-sm.pt2-sm.pb2-sm.fs12-sm.d-sm-b" 52 | ) 53 | EMAIL_INPUT_CSS = "input[data-componentname='emailAddress']" 54 | PASSWORD_INPUT_CSS = "input[data-componentname='password']" 55 | LOGIN_DIALOG_CSS = "div[class='d-md-tc u-full-width u-full-height va-sm-m']" 56 | SUBMIT_BTN_CSS = ( 57 | "div.nike-unite-submit-button.loginSubmit.nike-unite-component input[type='button']" 58 | ) 59 | 60 | 61 | LOGO = """ 62 | _ _ ____ ____ _ 63 | | \ | | _ \ / ___| _____ ___ __ ___ _ __| |_ ___ _ __ 64 | | \| | |_) | | / _ \ \/ / '_ \ / _ \| '__| __/ _ \ '__| 65 | | |\ | _ <| |___ | __/> <| |_) | (_) | | | || __/ | 66 | |_| \_|_| \_\\____| \___/_/\_\ .__/ \___/|_| \__\___|_| 67 | |_| 68 | 69 | ~ Yasoob Khalid 70 | https://yasoob.me 71 | 72 | """ 73 | 74 | 75 | def f_message(msg, level="info"): 76 | """ 77 | Format a message using colors 78 | 79 | Args: 80 | msg: a msg string 81 | Returns: 82 | msg: a colored msg string 83 | """ 84 | 85 | color_map = { 86 | "info": Style.BRIGHT + Fore.CYAN, 87 | "error": Fore.RED, 88 | "debug": Fore.BLUE, 89 | "logo": Fore.YELLOW, 90 | } 91 | 92 | return color_map[level] + msg + Style.RESET_ALL 93 | 94 | 95 | def info(message): 96 | """ 97 | A simple utility class for formatting and logging info messages 98 | 99 | Args: 100 | message: the log string 101 | """ 102 | logger = logging.getLogger(__name__) 103 | logger.info(f_message(f"[+] {message}", level="info")) 104 | 105 | 106 | def error(message): 107 | """ 108 | A simple utility class for formatting and logging info messages 109 | 110 | Args: 111 | message: the log string 112 | """ 113 | logger = logging.getLogger(__name__) 114 | logger.error(f_message(f"[-] ⚠️ {message}", level="error")) 115 | 116 | 117 | def debug(message): 118 | """ 119 | A simple utility class for formatting and logging debug messages 120 | 121 | Args: 122 | message: the log string 123 | """ 124 | logger = logging.getLogger(__name__) 125 | logger.debug(f_message(f"[-] 🐛 {message}", level="debug")) 126 | 127 | 128 | def warning(message): 129 | """ 130 | A simple utility class for formatting and logging warning messages 131 | 132 | Args: 133 | message: the log string 134 | """ 135 | logger = logging.getLogger(__name__) 136 | logger.warning(f_message(f"[-] ⚠️ {message}", level="error")) 137 | 138 | 139 | def login(driver, email, password): 140 | """ 141 | Open the login Page and sign in 142 | 143 | Args: 144 | driver: a Webdriver instance 145 | email: the email for nike website 146 | password: the password associated with the email 147 | 148 | Returns: 149 | Returns a boolean for whether the login was successful or not 150 | """ 151 | 152 | info("Trying to log in") 153 | debug("Opening the login page") 154 | driver.get(LOGIN_URL) 155 | debug("login page opened") 156 | 157 | login_btn = driver.find_element_by_css_selector(LOGIN_BTN_CSS) 158 | login_btn.click() 159 | 160 | email_input = WebDriverWait(driver, 10).until( 161 | expected_conditions.presence_of_element_located( 162 | (By.CSS_SELECTOR, EMAIL_INPUT_CSS) 163 | ) 164 | ) 165 | email_input.send_keys(email) 166 | 167 | password_input = WebDriverWait(driver, 10).until( 168 | expected_conditions.presence_of_element_located( 169 | (By.CSS_SELECTOR, PASSWORD_INPUT_CSS) 170 | ) 171 | ) 172 | password_input.send_keys(password) 173 | 174 | submit_btn = WebDriverWait(driver, 10).until( 175 | expected_conditions.presence_of_element_located( 176 | (By.CSS_SELECTOR, SUBMIT_BTN_CSS,) 177 | ) 178 | ) 179 | submit_btn.click() 180 | 181 | debug("Submitted email and password") 182 | 183 | try: 184 | login_dialog = WebDriverWait(driver, 5).until( 185 | expected_conditions.invisibility_of_element_located( 186 | (By.CSS_SELECTOR, LOGIN_DIALOG_CSS) 187 | ) 188 | ) 189 | info("Seems like login was successful") 190 | return True 191 | 192 | except: 193 | warning( 194 | f"Seems like automated login wasn't successful. You will have to manually provide access tokens" 195 | ) 196 | return False 197 | 198 | 199 | def extract_token(driver): 200 | """ 201 | Extracts access token from the login request 202 | 203 | Args: 204 | driver: the webdriver instance that was used to do the login 205 | 206 | Returns: 207 | access_token: the token extracted from request 208 | """ 209 | for request in driver.requests: 210 | if "login" in request.path: 211 | resp_body = request.response.body 212 | 213 | try: 214 | access_token = json.loads(resp_body)["access_token"] 215 | info(f"💉 Successfully extracted access token") 216 | debug(f"Access Token: {access_token}") 217 | except: 218 | error(f"Unable to extract access token. You will have to manually pass it in") 219 | return access_token 220 | 221 | 222 | def get_access_token(options): 223 | """ 224 | This function opens the login page and extracts access tokens 225 | 226 | Args: 227 | options: the options dict which contains login info 228 | 229 | Returns: 230 | access_token: the bearer token that will be used to extract activities 231 | """ 232 | 233 | login_success = False 234 | 235 | if options["gecko_path"] and not options["manual"]: 236 | info(f"🚗 Starting gecko webdriver") 237 | driver = webdriver.Firefox(executable_path=options["gecko_path"]) 238 | driver.scopes = [ 239 | ".*nike.*", 240 | ] 241 | login_success = login(driver, options["email"], options["password"]) 242 | 243 | if options["debug"]: 244 | debug(f"Saving screenshot from after login") 245 | with open("website.png", "wb") as f: 246 | f.write(driver.get_screenshot_as_png()) 247 | 248 | if login_success: 249 | access_token = extract_token(driver) 250 | else: 251 | info( 252 | f"I will open your web browser and you will have to manually intercept the access tokens.\n" 253 | f" You can find more details on how to do this over here: https://git.io/nrc-exporter\n" 254 | f" Press 'y' to open up the login url" 255 | ) 256 | accept = input() 257 | if not accept == "y": 258 | info("You didn't want to continue. Exiting") 259 | sys.exit(0) 260 | 261 | webbrowser.open_new_tab(MOBILE_LOGIN_URL) 262 | info(f"Please paste access tokens here: \n") 263 | access_token = input() 264 | debug(f"Manually entered access token: {access_token}") 265 | if len(access_token) < 5: 266 | error( 267 | f"You didn't paste access tokens. Please provide them using -t or --token argument" 268 | ) 269 | sys.exit(1) 270 | 271 | info( 272 | f"Closing the webdriver. From here on we will be using requests library instead" 273 | ) 274 | driver.quit() 275 | return access_token 276 | 277 | 278 | def get_activities_list(options): 279 | """ 280 | Gets the list of activity IDs from Nike. For now it only saves the runs and not other activities 281 | 282 | Args: 283 | options: the options dict which contains the access token 284 | 285 | Returns: 286 | activity_ids: a list of running activity ids 287 | """ 288 | 289 | info("🏃‍♀️ Getting activities list") 290 | headers = { 291 | "Authorization": f"Bearer {options['access_token']}", 292 | } 293 | debug(f"headers: {headers}") 294 | activity_ids = [] 295 | page_num = 1 296 | next_page = ACTIVITY_LIST_URL 297 | more = True 298 | while more: 299 | info(f"📃 opening page {page_num} of activities") 300 | debug(f"\tActivities page url: {next_page}") 301 | activity_list = requests.get(next_page, headers=headers) 302 | if "error_id" in activity_list.json().keys(): 303 | error("Are you sure you provided the correct access token?") 304 | sys.exit() 305 | for activity in activity_list.json()["activities"]: 306 | debug(f"Entry type: {activity.get('tags', {}).get('com.nike.running.runtype', 'unknow type')}") 307 | if ( 308 | activity["type"] == "run" 309 | and activity.get("tags", {}).get("com.nike.running.runtype", "") != "manual" 310 | ): 311 | # activity["tags"]["location"].lower() == "outdoors": 312 | activity_ids.append(activity.get("id")) 313 | 314 | if activity_list.json()["paging"].get("before_id"): 315 | page_num += 1 316 | next_page = ACTIVITY_LIST_PAGINATION.format( 317 | before_id=activity_list.json()["paging"]["before_id"] 318 | ) 319 | continue 320 | break 321 | 322 | info( 323 | f"🏃‍♀️ Successfully extracted {len(activity_ids)} running activities from {page_num} pages" 324 | ) 325 | debug(f"ID List: {activity_ids}") 326 | return activity_ids 327 | 328 | 329 | def get_activity_details(activity_id, options): 330 | """ 331 | Extracts details for a specific activity 332 | """ 333 | 334 | info(f"Getting activity details for {activity_id}") 335 | headers = { 336 | "Authorization": f"Bearer {options['access_token']}", 337 | } 338 | html = requests.get( 339 | ACTIVITY_DETAILS_URL.format(activity_id=activity_id), headers=headers 340 | ) 341 | return html.json() 342 | 343 | 344 | def save_activity(activity_json, activity_id): 345 | debug(f"Saving {activity_id}.json to disk") 346 | title = activity_json.get("tags", {}).get("com.nike.name", "") 347 | json_path = os.path.join(ACTIVITY_FOLDER, f"{activity_id}.json") 348 | with open(json_path, "w") as f: 349 | f.write(json.dumps(activity_json)) 350 | 351 | 352 | def generate_gpx(title, latitude_data, longitude_data, elevation_data, heart_rate_data): 353 | """ 354 | Parses the latitude, longitude and elevation data to generate a GPX document 355 | 356 | Args: 357 | title: the title of the GXP document 358 | latitude_data: A list of dictionaries containing latitude data 359 | longitude_data: A list of dictionaries containing longitude data 360 | elevation_data: A list of dictionaries containing elevation data 361 | 362 | Returns: 363 | gpx: The GPX XML document 364 | """ 365 | 366 | gpx = gpxpy.gpx.GPX() 367 | gpx.nsmap["gpxtpx"] = "http://www.garmin.com/xmlschemas/TrackPointExtension/v1" 368 | gpx_track = gpxpy.gpx.GPXTrack() 369 | gpx_track.name = title 370 | gpx.tracks.append(gpx_track) 371 | 372 | # Create first segment in our GPX track: 373 | gpx_segment = gpxpy.gpx.GPXTrackSegment() 374 | gpx_track.segments.append(gpx_segment) 375 | 376 | points_dict_list = [] 377 | 378 | def update_points(points, update_data, update_name): 379 | """ 380 | Update the points dict list so that can easy create GPXTrackPoint 381 | 382 | Args: 383 | points: basic points list 384 | update_data: attr to update points which is a list 385 | update_name: attr name 386 | 387 | Returns: 388 | None (just update the points list) 389 | """ 390 | counter = 0 391 | for p in points: 392 | while p["start_time"] >= update_data[counter]["end_epoch_ms"]: 393 | if counter == len(update_data) - 1: 394 | break 395 | p[update_name] = update_data[counter]["value"] 396 | counter += 1 397 | 398 | 399 | for lat, lon in zip(latitude_data, longitude_data): 400 | if lat["start_epoch_ms"] != lon["start_epoch_ms"]: 401 | error(f"\tThe latitude and longitude data is out of order") 402 | 403 | points_dict_list.append({ 404 | "latitude": lat["value"], 405 | "longitude": lon["value"], 406 | "start_time": lat["start_epoch_ms"], 407 | "time": datetime.datetime.utcfromtimestamp(lat["start_epoch_ms"] / 1000) 408 | }) 409 | 410 | if elevation_data: 411 | update_points(points_dict_list, elevation_data, "elevation") 412 | if heart_rate_data: 413 | update_points(points_dict_list, heart_rate_data, "heart_rate") 414 | 415 | for p in points_dict_list: 416 | # delete useless attr 417 | del p["start_time"] 418 | if p.get("heart_rate") is None: 419 | point = gpxpy.gpx.GPXTrackPoint(**p) 420 | else: 421 | heart_rate_num = p.pop("heart_rate") 422 | point = gpxpy.gpx.GPXTrackPoint(**p) 423 | gpx_extension_hr = ElementTree.fromstring(f""" 424 | {heart_rate_num} 425 | 426 | """) 427 | point.extensions.append(gpx_extension_hr) 428 | gpx_segment.points.append(point) 429 | 430 | return gpx.to_xml() 431 | 432 | 433 | def parse_activity_data(activity): 434 | """ 435 | Parses a NRC activity and returns GPX XML 436 | 437 | Args: 438 | activity: a json document for a NRC activity 439 | 440 | Returns: 441 | gpx: the GPX XML doc for the input activity 442 | """ 443 | 444 | debug(f"Parsing activity: {activity.keys()}") 445 | lat_index = None 446 | lon_index = None 447 | ascent_index = None 448 | heart_rate_index = None 449 | if not activity.get("metrics"): 450 | warning( 451 | f"\tThe activity {activity['id']} doesn't contain metrics information" 452 | ) 453 | return None 454 | for i, metric in enumerate(activity["metrics"]): 455 | if metric["type"] == "latitude": 456 | lat_index = i 457 | if metric["type"] == "longitude": 458 | lon_index = i 459 | if metric["type"] == "ascent": 460 | ascent_index = i 461 | if metric["type"] == "heart_rate": 462 | heart_rate_index = i 463 | 464 | debug( 465 | f"\tActivity {activity['id']} contains the following metrics: {activity['metric_types']}" 466 | ) 467 | if not any([lat_index, lon_index]): 468 | warning( 469 | f"\tThe activity {activity['id']} doesn't contain latitude/longitude information" 470 | ) 471 | return None 472 | 473 | latitude_data = activity["metrics"][lat_index]["values"] 474 | longitude_data = activity["metrics"][lon_index]["values"] 475 | elevation_data = None 476 | heart_rate_data = None 477 | if ascent_index: 478 | elevation_data = activity["metrics"][ascent_index]["values"] 479 | if heart_rate_index: 480 | heart_rate_data = activity["metrics"][heart_rate_index]["values"] 481 | 482 | title = activity.get("tags", {}).get("com.nike.name", "") 483 | 484 | gpx_doc = generate_gpx(title, latitude_data, longitude_data, elevation_data, heart_rate_data) 485 | info(f"✔ Activity {activity['id']} successfully parsed") 486 | return gpx_doc 487 | 488 | 489 | def save_gpx(gpx_data, activity_id): 490 | """ 491 | Saves the GPX data to a file on disk 492 | 493 | Args: 494 | gpx_data: the GPX XML doc 495 | activity_id: the name of the file 496 | """ 497 | 498 | file_path = os.path.join(GPX_FOLDER, activity_id + ".gpx") 499 | with open(file_path, "w", encoding="utf-8") as f: 500 | f.write(gpx_data) 501 | 502 | 503 | def get_gecko_path(): 504 | """ 505 | Check if geckodriver exists in the code directory. If it does, return 506 | the absolute path. If not, exit program with an error 507 | 508 | Returns: 509 | path: the absolute path to geckodriver 510 | """ 511 | debug(f"Checking if geckodriver is in path or not") 512 | if os.path.exists("geckodriver"): 513 | return os.path.join(os.getcwd(), "geckodriver") 514 | else: 515 | error( 516 | "Gecko driver doesn't exist. I will not try to automatically extract access tokens" 517 | ) 518 | return None 519 | 520 | 521 | def arg_parser(): 522 | """ 523 | Parses the input arguments 524 | 525 | Returns: 526 | options: A dictionary containing either the bearer token or email and password 527 | """ 528 | 529 | ap = argparse.ArgumentParser( 530 | description="Login to Nike Run Club and download activity data in GPX format" 531 | ) 532 | ap.add_argument("-e", "--email", help="your nrc email") 533 | ap.add_argument("-p", "--password", help="your nrc password") 534 | ap.add_argument("-v", "--verbose", action="store_true", help="print verbose output") 535 | ap.add_argument("-t", "--token", help="your nrc token", required=False) 536 | ap.add_argument( 537 | "-i", "--input", nargs='+', help="A directory or directories containing NRC activities in JSON format." 538 | "You can also provide individual NRC JSON files" 539 | ) 540 | args = ap.parse_args() 541 | 542 | if args.verbose: 543 | logger = logging.getLogger(__name__) 544 | logger.setLevel(logging.DEBUG) 545 | 546 | debug(f"Passed in arguments:") 547 | debug(f"\t\tEmail: {args.email}") 548 | debug(f"\t\tPassword: {args.password}") 549 | debug(f"\t\tToken: {args.token}") 550 | 551 | options = {} 552 | options["debug"] = args.verbose 553 | options["manual"] = False 554 | if args.input: 555 | if all([os.path.exists(i) for i in args.input]): 556 | options["activities_dirs"] = args.input 557 | if all([i.endswith(".json") for i in args.input]): 558 | options["activities_files"] = args.input 559 | elif args.token: 560 | options["access_token"] = args.token 561 | elif args.email and args.password: 562 | options["email"] = args.email 563 | options["password"] = args.password 564 | else: 565 | options["manual"] = True 566 | info( 567 | "You will have to manually provide the access tokens in a later step because you\n" 568 | " did not provide email/password or access tokens while running the program." 569 | ) 570 | 571 | return options 572 | 573 | 574 | def init_logger(): 575 | """ 576 | Initializes the logger 577 | 578 | Returns: 579 | logger: A logging object 580 | """ 581 | 582 | if not os.path.exists(GPX_FOLDER): 583 | debug("Created a folder for GPX data: {GPX_FOLDER}") 584 | os.mkdir(GPX_FOLDER) 585 | if not os.path.exists(ACTIVITY_FOLDER): 586 | debug("Created a folder for activity data: {ACTIVITY_FOLDER}") 587 | os.mkdir(ACTIVITY_FOLDER) 588 | 589 | logging.basicConfig( 590 | format="%(message)s", level=logging.INFO, 591 | ) 592 | # Set this to True if you want to see the actual requests captured by seleniumwire 593 | logging.getLogger("seleniumwire").propagate = False 594 | 595 | print(f_message(LOGO, level="logo")) 596 | 597 | 598 | def main(): 599 | """ 600 | Main will be called if the script is run directly 601 | """ 602 | 603 | init_logger() 604 | options = arg_parser() 605 | 606 | info("Starting NRC Exporter") 607 | 608 | start_time = time.time() 609 | 610 | if options.get("email") or options.get("manual"): 611 | info("💉 Email and password provided so will try to extract access tokens") 612 | options["gecko_path"] = get_gecko_path() 613 | options["access_token"] = get_access_token(options) 614 | 615 | if options.get("access_token"): 616 | activity_ids = get_activities_list(options) 617 | for activity in activity_ids: 618 | activity_details = get_activity_details(activity, options) 619 | save_activity(activity_details, activity_details["id"]) 620 | 621 | activity_folders = options.get("activities_dirs", [ACTIVITY_FOLDER]) 622 | activity_files = options.get("activities_files", []) 623 | if not activity_files: 624 | for folder in activity_folders: 625 | # add path to every file in folder 626 | activity_files.extend([os.path.join(folder, f) for f in os.listdir(folder)]) 627 | info(f"Parsing activity JSON files from the {','.join(activity_folders)} folders") 628 | 629 | total_parsed_count = 0 630 | for file_path in activity_files: 631 | with open(file_path, "r") as f: 632 | try: 633 | json_data = json.loads(f.read()) 634 | except JSONDecodeError: 635 | error(f"Error occured while parsing file {file_path}") 636 | debug(f"Parsing file: {file_path}") 637 | parsed_data = parse_activity_data(json_data) 638 | if parsed_data: 639 | total_parsed_count += 1 640 | save_gpx(parsed_data, json_data["id"]) 641 | 642 | info( 643 | f"Parsed {total_parsed_count} activities successfully out of {len(activity_files)} total run activities" 644 | ) 645 | info( 646 | f"Total time taken: {time.strftime('%H:%M:%S', time.gmtime(time.time() - start_time))}" 647 | ) 648 | 649 | 650 | if __name__ == "__main__": 651 | main() 652 | --------------------------------------------------------------------------------