├── requirements.txt ├── import.csv ├── sample form data 2020-05-02.txt ├── sample request header 2020-05-02.txt ├── readme.md ├── import.py └── sample mint category menu 2020-05-02.txt /requirements.txt: -------------------------------------------------------------------------------- 1 | DateTime==4.3 2 | requests==2.24.0 3 | urllib3==1.25.10 4 | -------------------------------------------------------------------------------- /import.csv: -------------------------------------------------------------------------------- 1 | Transaction Date,Post Date,Description,Category,Type,Amount 2 | 07/22/2018,07/23/2018,NORTH PARK PRODUCE,Groceries,Sale,-1.04 3 | 07/22/2018,07/23/2018,LITTLE SHEEP MONGOLIAN,Food & Drink,Sale,-58.29 4 | -------------------------------------------------------------------------------- /sample form data 2020-05-02.txt: -------------------------------------------------------------------------------- 1 | cashTxnType: on 2 | mtCheckNo: 3 | tagXXXXXXX: 0 4 | tagXXXXXXX: 0 5 | tagXXXXXXX: 0 6 | task: txnadd 7 | txnId: :0 8 | mtType: cash 9 | mtAccount: XXXXXXXX 10 | symbol: 11 | note: 12 | isInvestment: false 13 | catId: 1405 14 | category: Auto Insurance 15 | merchant: test 16 | date: 05/02/2020 17 | amount: 0.01 18 | mtIsExpense: true 19 | mtCashSplitPref: 2 20 | token: XXXXXXX -------------------------------------------------------------------------------- /sample request header 2020-05-02.txt: -------------------------------------------------------------------------------- 1 | :authority: mint.intuit.com 2 | :method: POST 3 | :path: /updateTransaction.xevent 4 | :scheme: https 5 | accept: */* 6 | accept-encoding: gzip, deflate, br 7 | accept-language: en-US,en;q=0.9,la;q=0.8 8 | adrum: isAjax:true 9 | content-length: 323 10 | content-type: application/x-www-form-urlencoded; charset=UTF-8 11 | cookie: XXXXXXX 12 | origin: https://mint.intuit.com 13 | referer: https://mint.intuit.com/transaction.event?accountId=XXXXXXX 14 | sec-fetch-dest: empty 15 | sec-fetch-mode: cors 16 | sec-fetch-site: same-origin 17 | user-agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36 18 | x-requested-with: XMLHttpRequest -------------------------------------------------------------------------------- /readme.md: -------------------------------------------------------------------------------- 1 | For a complete overview of how the Mint CSV Importer was developed check out - https://nathanielkam.com/import-transactions-to-mint-using-python/ 2 | 3 | # Run: 4 | Run the code by typing the following from the directory the code is located in 5 | python3 import.py 6 | 7 | ## Pre-requisites needed: 8 | 9 | csv 10 | datetime 11 | os 12 | random 13 | requests 14 | time 15 | urllib.parse 16 | 17 | Virtual Environment Setup (from app repo root) 18 | 1. Make sure you have venv on your system, using the following command based on your python version 19 | - python3 -m pip3 install virtualenv 20 | 2. Make sure you are in repo root \ 21 | - (where import.py and requirements.txt are) 22 | 3. Create a virtual environment 23 | - virtualenv venv 24 | 4. Turn on the virtual environment (these should work on both but depends on your version you may need to explicitly run the sh or bat file) 25 | - Mac / Linux in terminal or bash: venv/Scripts/activate 26 | - Windows in powershell: venv\Scripts\activate 27 | 5. Install Requirements 28 | - pip3 install -r requirements.txt 29 | 30 | 31 | ## Overview: ## 32 | Simulates bulk manual transaction adds to mint.com. Mint manual transactions are submitted as "cash transactions" which 33 | will mean it shows in your cash / all accounts transaction list. You cannot submit manual transactions against credit 34 | cards or other integrated bank accounts (even in Mint's UI this is not possible and ends up as cash transction). 35 | 36 | ## Approach: ## 37 | Simulating manual transactions from UI is based on Nate H's proof of concept from https://www.youtube.com/watch?v=8AJ3g5JGmdU 38 | 39 | ## Python: ## 40 | Credit to https://github.com/ukjimbow for his work on Mint imports for UK users in https://github.com/ukjimbow/mint-transactions 41 | 42 | ## Process: ## 43 | 1. Import CSV 44 | 2. Process date for correct format and HTTP encode result 45 | 3. Process merchant for HTTP encode 46 | 4. Process categories. Change your banks category name into a mint category ID (limited in scope based on the categories needed when I wrote this) 47 | 6. Process amount for positive or negative value indicating income or expense 48 | 7. Send POST Request to mint as new transaction. 49 | 8. Force Randomized Wait Time before starting next request 50 | 51 | ## Instructions: ## 52 | 1. Prepare your data to import using import.csv as an example 53 | 2. Edit import.py and replace the variables set to XXXXX's to values in your browser during a live Mint session 54 | - account is mtaccount and approximately 8 digits 55 | - tag1 is in form of tagXXXXXXX 56 | - tag2 is in form of tagXXXXXXX 57 | - tag3 is in form of tagXXXXXXX 58 | - cookie will be an apprimately 2000 character string 59 | - referrer is likely always 'https://mint.intuit.com/transaction.event' 60 | - token is approximately 50 characters 61 | 3. If you have custom categories, they need to go along others in function category_id_switch() 62 | -------------------------------------------------------------------------------- /import.py: -------------------------------------------------------------------------------- 1 | """ 2 | ################################# 3 | Pre-requisites needed 4 | ################################# 5 | 6 | If you are missing any of the following you can install with: 7 | 8 | pip install $name 9 | Example: pip install csv 10 | 11 | OR if you are using pip3 12 | 13 | pip3 install $name 14 | Example: pip3 install csv 15 | """ 16 | 17 | import csv 18 | import datetime 19 | import os 20 | import random 21 | import requests 22 | import time 23 | import urllib.parse 24 | 25 | """ 26 | ################################# 27 | Overview: 28 | ################################# 29 | 30 | Simulates bulk manual transaction adds to mint.com. Mint manual transactions are submitted as "cash transactions" which 31 | will mean it shows in your cash / all accounts transaction list. You cannot submit manual transactions against credit 32 | cards or other integrated bank accounts (even in Mint's UI this is not possible and ends up as cash transction). 33 | 34 | Approach Credits: 35 | Simulating manual transactions from UI is based on Nate H's proof of concept from https://www.youtube.com/watch?v=8AJ3g5JGmdU 36 | 37 | Python Credits: 38 | Credit to https://github.com/ukjimbow for his work on Mint imports for UK users in https://github.com/ukjimbow/mint-transactions 39 | 40 | Process Documentation: 41 | 1. Import CSV 42 | 2. Process date for correct format and HTTP encode result 43 | 3. Process merchant for HTTP encode 44 | 4. Process cateogories change your banks category name into a mint category ID (limited in scope based on the categories 45 | 5 needed when I wrote this) 46 | 6. Process amount for positive or negative value indicating income or expense 47 | 7. Send POST Request to mint as new transaction. 48 | 8. Force Randomized Wait Time before starting next request 49 | 50 | Future Development: 51 | 1. Replace curl command string generation with parametized curl class constructor 52 | 2. Add support for the rest of the manual transaction items 53 | 54 | """ 55 | 56 | """ 57 | ################################# 58 | Settings 59 | ################################# 60 | """ 61 | csv_name = 'import.csv' # name of csv you you want import to mint [string.csv] 62 | verbose_output = 1 # should verbose messages be printed [0,1] 63 | uk_to_us = 0 # do you need to change dates from UK to US format [0,1] 64 | min_wait = 0 # min wait time in seconds between requests, int[0-n] 65 | max_wait = 2 # max wait time in seconds between requests, int[0-n] 66 | 67 | """ 68 | ################################# 69 | Mint Client Credentials 70 | ################################# 71 | 72 | You will need the tags, cookie, and token to simulate a UI form submission. You can get these by opening developer tools > network analysis tab and doing 73 | a test submission in mint.com. From there look for the post request to "updateTransaction.xevent" and grab the credentials from the header and body 74 | """ 75 | account = 'XXXXXXX' # grab from POST request form body in devtools 76 | tag1 = 'tagXXXXXX' # in form of tagXXXXXXX 77 | tag2 = 'tagXXXXXXX' # in form of tagXXXXXXX 78 | tag3 = 'tagXXXXXXX' # in form of tagXXXXXXX 79 | cookie = 'XXXXXXX' # grab from POST request header in devtools 80 | referrer = 'XXXXXXX' # grab from POST request header in devtools 81 | token = 'XXXXXXX' # grab from POST request form body in devtools 82 | 83 | """ 84 | ################################# 85 | Import CSV using the pythons csv reader 86 | ################################# 87 | """ 88 | csv_object = csv.reader(open(csv_name,'rU')) 89 | next(csv_object) 90 | 91 | for row in csv_object: 92 | 93 | # Initialize Variables 94 | date = (row[0]) 95 | postDate = (row[1]) 96 | merchant = (row[2]) 97 | catName = (row[3]) 98 | typeID = (row[4]) 99 | amount = (float(row[5])) 100 | expense = 'true' 101 | curl_input = 'Error: Did not Generate' 102 | curl_output = 'Error: Did not run' 103 | 104 | """ 105 | ################################# 106 | Process Date for format and HTTP Encode 107 | ################################# 108 | """ 109 | 110 | # Convert Commonwealth to US Date System 111 | if uk_to_us == 1: # based on setting 112 | dateconv = time.strptime(date,"%d/%m/%Y") # not needed for US to US 113 | date = (time.strftime("%m/%d/%Y",dateconv)) # converted new US date format from UK 114 | 115 | # Require "/" for date delimiter and HTTP Encode Character, supports "/", ".", "-" 116 | # We are not using url encode library here because we custom map other delimiters 117 | dateoutput = date.replace("/", "%2F") 118 | dateoutput = date.replace(".", "%2F") 119 | dateoutput = date.replace("-", "%2F") 120 | 121 | """ 122 | ################################# 123 | Process Merchant with HTTP Encode 124 | ################################# 125 | """ 126 | merchant = urllib.parse.quote(merchant) 127 | 128 | """ 129 | ################################# 130 | Process Categories 131 | ################################# 132 | 133 | Support is limited to the categories I needed at the time, if you need to map more you can. To get category ids: 134 | 1. Go to mint 135 | 2. Add a transactions 136 | 3. Right click "inspect-element" on the category you want 137 | 4. The ID is in the
  • item that encapsulates the a href 138 | 5. Add mapping here based on string match from your CSV to the catID you got from mint (following existing examples) 139 | """ 140 | 141 | # Category ID Mapping Function 142 | def category_id_switch(import_category): 143 | 144 | # Define mapping of import categories to : Mint Category IDs 145 | switcher={ 146 | #Chase categories - incomplete 147 | 'Gas':1401, 148 | 'Food & Drink':7, 149 | 'Groceries':701, 150 | 'Bills & Utilities':13, 151 | 'Shopping':2, 152 | 'Health & Wellness':5, 153 | 'Personal':4, 154 | 'Credit Card Payment':2101, 155 | 'Travel':15, 156 | 'Entertainment':1, 157 | 'Automotive':14, 158 | 'Education':10, 159 | 'Professional Services':17, 160 | 'Home':12, 161 | 'Fees & Adjustments':16, 162 | 'Gifts & Donations':8, 163 | # American Express Categories - Incomplete 164 | # American Express doesn't provide category information for payments, so I recommend manually changing those to "Payment" 165 | 'Merchandise & Supplies-Groceries':701, 166 | 'Transportation-Fuel':1401, 167 | 'Fees & Adjustments-Fees & Adjustments':16, 168 | 'Merchandise & Supplies-Wholesale Stores':2, 169 | 'Restaurant-Restaurant':707, 170 | 'Payment':2101, 171 | # The following categories are Mint categories. 172 | # Citi does not included categories in downloaded transactions so I added my own categories using the Mint categories. 173 | # These mappings make sure those categories don't get mapped to 'uncategorized:20' when they aren't found in the mappings 174 | # for the other banks above. 175 | # 176 | # If you want to change a category mapping, be mindful that some category names may be repeated because Mint uses 177 | # the same category name as another bank. 178 | 'Auto & Transport':14, 179 | 'Auto Insurance':1405, 180 | 'Auto Payment':1404, 181 | 'Gas & Fuel':1401, 182 | 'Parking':1402, 183 | 'Public Transportation':1406, 184 | 'Service & Parts':1403, 185 | 'Bills & Utilities':13, 186 | 'Home Phone':1302, 187 | 'Internet':1303, 188 | 'Mobile Phone':1304, 189 | 'Television':1301, 190 | 'Utilities':1306, 191 | 'Business Services':17, 192 | 'Advertising':1701, 193 | 'Legal':1705, 194 | 'Office Supplies':1702, 195 | 'Printing':1703, 196 | 'Shipping':1704, 197 | 'Education':10, 198 | 'Books & Supplies':1003, 199 | 'Student Loan':1002, 200 | 'Tuition':1001, 201 | 'Entertainment':1, 202 | 'Amusement':102, 203 | 'Arts':101, 204 | 'Movies & DVDs':104, 205 | 'Music':103, 206 | 'Newspapers & Magazines':105, 207 | 'Fees & Charges':16, 208 | 'ATM Fee':1605, 209 | 'Bank Fee':1606, 210 | 'Finance Charge':1604, 211 | 'Late Fee':1602, 212 | 'Service Fee':1601, 213 | 'Trade Commissions':1607, 214 | 'Financial':11, 215 | 'Financial Advisor':1105, 216 | 'Life Insurance':1102, 217 | 'Food & Dining':7, 218 | 'Alcohol & Bars':708, 219 | 'Coffee Shops':704, 220 | 'Fast Food':706, 221 | 'Groceries':701, 222 | 'Restaurants':707, 223 | 'Gifts & Donations':8, 224 | 'Charity':802, 225 | 'Gift':801, 226 | 'Health & Fitness':5, 227 | 'Dentist':501, 228 | 'Doctor':502, 229 | 'Eyecare':503, 230 | 'Gym':507, 231 | 'Health Insurance':506, 232 | 'Pharmacy':505, 233 | 'Sports':508, 234 | 'Home':12, 235 | 'Furnishings':1201, 236 | 'Home Improvement':1203, 237 | 'Home Insurance':1206, 238 | 'Home Services':1204, 239 | 'Home Supplies':1208, 240 | 'Lawn & Garden':1202, 241 | 'Mortgage & Rent':1207, 242 | 'Income':30, 243 | 'Bonus':3004, 244 | 'Interest Income':3005, 245 | 'Paycheck':3001, 246 | 'Reimbursement':3006, 247 | 'Rental Income':3007, 248 | 'Returned Purchase':3003, 249 | 'Kids':6, 250 | 'Allowance':610, 251 | 'Baby Supplies':611, 252 | 'Babysitter & Daycare':602, 253 | 'Child Support':603, 254 | 'Kids Activities':609, 255 | 'Toys':606, 256 | 'Misc Expenses':70, 257 | 'Personal Care':4, 258 | 'Hair':403, 259 | 'Laundry':406, 260 | 'Spa & Massage':404, 261 | 'Pets':9, 262 | 'Pet Food & Supplies':901, 263 | 'Pet Grooming':902, 264 | 'Veterinary':903, 265 | 'Shopping':2, 266 | 'Books':202, 267 | 'Clothing':201, 268 | 'Electronics & Software':204, 269 | 'Hobbies':206, 270 | 'Sporting Goods':207, 271 | 'Taxes':19, 272 | 'Federal Tax':1901, 273 | 'Local Tax':1903, 274 | 'Property Tax':1905, 275 | 'Sales Tax':1904, 276 | 'State Tax':1902, 277 | 'Transfer':21, 278 | 'Credit Card Payment':2101, 279 | 'Transfer for Cash Spending':2102, 280 | 'Travel':15, 281 | 'Air Travel':1501, 282 | 'Hotel':1502, 283 | 'Rental Car & Taxi':1503, 284 | 'Vacation':1504, 285 | 'Uncategorized':20, 286 | 'Cash & ATM':2001, 287 | 'Check':2002, 288 | 'Hide from Budgets & Trends':40, 289 | } 290 | # Get the mint category ID from the map 291 | return switcher.get(import_category,20) # For all other unmapped cases return uncategorized category "20" 292 | 293 | # typeID payment overrides all categories 294 | if typeID == "Payment": 295 | catID = '2101' # Since I was importing credit cards I have mine set to credit card payment. If you are doing bank accounts you many want to change this to payment general 296 | 297 | # if type is NOT payment then do a category check 298 | else: 299 | 300 | # if there IS no cat it is uncategorized 301 | if len(catName) == 0: 302 | catID = '20' # mint's uncategorized category 303 | 304 | # If there is a category check it against mapping 305 | else : 306 | # Use a switch since there may be MANY category maps 307 | catID = str(category_id_switch(catName)) 308 | 309 | 310 | # Set mint category name by looking up name in ID map 311 | category = catName 312 | category = urllib.parse.quote(category) 313 | 314 | """ 315 | ################################# 316 | Process Amount seeing if transaction is an expense or income. 317 | ################################# 318 | """ 319 | if amount < 0: 320 | expense = 'true' # when amount is less than 0 this is an expense, ie money left your account, ex like buying a sandwich. 321 | else: 322 | expense = 'false' # when amount is greater than 0 this is income, ie money went INTO your account, ex like a paycheck. 323 | amount = str(amount) # convert amount to string so it can be concatenated in POST request 324 | 325 | """ 326 | ################################# 327 | Build CURL POST Request 328 | TODO: Swap command string generation for parametized curl class 329 | ################################# 330 | """ 331 | 332 | # Break curl lines 333 | curl_line = " " 334 | 335 | # fragment curl command 336 | curl_command = "curl -i -s -k -X POST 'https://mint.intuit.com/updateTransaction.xevent'" + curl_line 337 | curl_host = "-H 'Host: mint.intuit.com'" + curl_line 338 | curl_user_agent = "-H 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.129 Safari/537.36'" + curl_line 339 | curl_accept = "-H 'Accept: */*'" + curl_line 340 | curl_accept_language = "-H 'Accept-Language: en-US,en;q=0.5'" + curl_line 341 | curl_compressed = "--compressed" + curl_line 342 | curl_x_requested_with = "-H 'X-Requested-With: XMLHttpRequest'" + curl_line 343 | curl_content_type = "-H 'Content-Type: application/x-www-form-urlencoded; charset=UTF-8'" + curl_line 344 | curl_referer = "-H 'Referer: https://mint.intuit.com/transaction.event?accountId=" + referrer + "'" + curl_line 345 | curl_cookie = "-H 'Cookie: " + cookie + "'" + curl_line 346 | curl_connection = "-H 'Connection: close' " + curl_line 347 | curl_data = "--data" + curl_line 348 | 349 | # Fragment the curl form data 350 | form_p1 = "'cashTxnType=on&mtCheckNo=&" + tag1 + "=0&" + tag2 + "=0&" + tag3 + "=0&" 351 | form_p2 = "task=txnadd&txnId=%3A0&mtType=cash&mtAccount=" + account + "&symbol=¬e=&isInvestment=false&" 352 | form_p3 = "catId="+catID+"&category="+category+"&merchant="+merchant+"&date="+dateoutput+"&amount="+amount+"&mtIsExpense="+expense+"&mtCashSplitPref=1&mtCashSplit=on&" 353 | form_p4 = "token=" + token + "'" 354 | 355 | # Piece together curl form data 356 | curl_form = form_p1 + form_p2 + form_p3 + form_p4 357 | 358 | # Combine all curl fragments together into an entire curl command 359 | curl_input = curl_command + curl_host + curl_user_agent + curl_accept + curl_accept_language + curl_compressed + curl_x_requested_with + curl_content_type + curl_referer + curl_cookie + curl_connection + curl_data + curl_form 360 | 361 | """ 362 | ################################# 363 | Submit CURL POST Request 364 | ################################# 365 | """ 366 | curl_output = str(os.system(curl_input)) # use os sytem to run a curl request submitting the form post 367 | 368 | """ 369 | ################################# 370 | Verbose Output for Debug 371 | ################################# 372 | """ 373 | if verbose_output == 1: 374 | print ('Transaction Date:', dateoutput) # date of transaction 375 | print ('Merchant', merchant) # merchant Description 376 | print ('Category ID:', catID) # category of transaction 377 | print ('Category Name:', category) # category of transaction 378 | print ('Amount:', amount) # amount being processed 379 | print ('Expense:', expense) # in amount expense 380 | print ('CURL Request:', curl_input) # what was sent to mint 381 | print ('CURL Response:', curl_output) # what was returned from mint OR curl ERROR 382 | print ('\n\n==============\n') # new line break 383 | 384 | """ 385 | ################################# 386 | Force a random wait between 2 and 5 seconds per requests to simulate UI and avoid rate limiting 387 | ################################# 388 | """ 389 | time.sleep(random.randint(min_wait, max_wait)) 390 | -------------------------------------------------------------------------------- /sample mint category menu 2020-05-02.txt: -------------------------------------------------------------------------------- 1 | --------------------------------------------------------------------------------