├── README.md
└── code.gs


/README.md:
--------------------------------------------------------------------------------
  1 | # 🏦 plaid-transaction-script
  2 | ## Introduction
  3 | 
  4 | There are a only few platforms that offer to automate bank transaction data into a Google Spreadsheet, however everything I could find required payment. Using the Plaid API and Google App Scripts this process is entirely free. 
  5 | 
  6 | The code for this project can be found on [GitHub](https://github.com/lpg0/plaid-transaction-script).
  7 | 
  8 | Please note that this project is barebones and it could be extended to synchronously update data and format it (see concluding section). 
  9 | 
 10 | ## Setup
 11 | 
 12 | You will need a [Plaid account](https://dashboard.plaid.com/signup). After signing up, go to your [development dashboard](https://dashboard.plaid.com/overview)* and click **Build In Development**. You have 5 live bank account to use, which is plenty for a personal finance project. Here you will have access to the *client_id* API key and the *development_secret*. Keep this tab open since both will be used in the next section.
 13 | 
 14 | *Please note that this project may be first completed in sandbox mode to limit the chance of an accidental data leak, however I am only writing this for development.
 15 | 
 16 | ## Access Token
 17 | 
 18 | The next step is to generate an access token. Via the Plaid founder's [recommendation](https://stackoverflow.com/a/49868230) (thanks Michael!) we will generate an access token by running the Plaid quickstart application locally where we can authenticate our bank. The quickstart project can be found [here](https://github.com/plaid/quickstart). The instructions are described in the README.md file, however due to some discrepancies I will walk through the process on Windows (Linux would be similar). 
 19 | 
 20 | First, clone the repo and cd into quickstart and then the python folder.
 21 | 
 22 | ```bash
 23 | $ git clone https://github.com/plaid/quickstart
 24 | $ cd quickstart
 25 | ```
 26 | 
 27 | In the python folder open the <u>.env</u> file and change the *PLAID_CLIENT_ID=client_id*, *PLAID_SECRET=development_secret*, and *PLAID_ENV=development*. Thee *PLAID_CLIENT_ID* and *PLAID_SECRET* were found from the setup section.
 28 | 
 29 | Then open the server.py file and replace the *"host=plaid.Environment.Sandbox"* line with *"host=plaid.Environment.Development"* on line 73. The sandbox mode is hardcoded into the server and this will overwrite it.
 30 | 
 31 | Next install the requirements and run the python server.
 32 | 
 33 | ```bash
 34 | $ pip install -r requirements.txt
 35 | $ py start.sh
 36 | ```
 37 | 
 38 | In a new terminal cd in quickstart and then into the frontend folder. Run npm install and then start the application.
 39 | 
 40 | ```bash
 41 | $ cd quickstart
 42 | $ cd frontend
 43 | $ npm install
 44 | $ npm start
 45 | ```
 46 | 
 47 | A browser window will open at http://localhost:3000/. From the running application you can authenticate your bank with your Plaid account via a pop up window. The following dashboard will provide an *item_id* and *access_token*. You can also test different http request from the application. Leave this tab open to reference the *access_token* in the next section.
 48 | 
 49 | ## Transaction Scripting
 50 | 
 51 | In this section we will create a script that will import transaction data for one month into a Google Sheet. 
 52 | 
 53 | Go to **[Google Sheets](http://docs.google.com/spreadsheets/ "Google Sheets") > Blank** . Upon creating a new blank spreadsheet, add a title (such as "Transactions"). Then click **Tools > Script Editor**, which will open up a new window with the Script Editor.
 54 | 
 55 | Give the new script project a title (such as "Transactions Script"). 
 56 | 
 57 | In the function *myFunction()* we will need to use the *UrlFetchApp.fetch* function provided to use by the App Script APIs. The following function creates a simple POST request to recieve the JSON transaction data for the past month. Update the *start_date* and *end_date* accordingly (we will change this later).
 58 | 
 59 | ```js
 60 | function myFunction() {
 61 |   var data = {
 62 |     "client_id": "XXXXXXXXXXXXXXXXXXXXXXXXXXX",
 63 |     "secret": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
 64 |     "access_token": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
 65 |     "start_date": "2021-09-13",
 66 |     "end_date": "2021-08-13"
 67 |   };
 68 |   
 69 |   var payload = JSON.stringify(data);
 70 |   var options = {
 71 |     "method" : "POST",
 72 |     "contentType" : "application/json",
 73 |     "payload" : payload
 74 |   };
 75 |   
 76 |   var url = "https://development.plaid.com/transactions/get";
 77 |   var response = UrlFetchApp.fetch(url, options);
 78 |   console.log(response.getContentText());
 79 | }
 80 | ```
 81 | 
 82 | Run the script and fix any errors before proceeding. The transactions should now be displayed in the console.
 83 | 
 84 | The next step is to automate the *start_date* and *end_date*. The following two functions do this in a rather brute force way, but get the job done.
 85 | 
 86 | ```js
 87 | function getStartDate(){
 88 |   const d = new Date();
 89 |   var yyyy = String(d.getFullYear());
 90 |   var s_mm = d.getMonth();
 91 |   
 92 |   if (s_mm == 0) {
 93 |     s_mm = "12"
 94 |   } else {
 95 |     s_mm = String(s_mm)
 96 |   }
 97 |   
 98 |   if (s_mm.length == 1) {
 99 |     s_mm = "0" + s_mm
100 |   }
101 |   
102 |   var s_dd = d.getDate();
103 |   if (s_dd == 29 || s_dd == 30 || s_dd == 31) {
104 |     s_dd = "28"
105 |   } else {
106 |     s_dd = String(s_dd)
107 |   }
108 |   
109 |   if (s_dd.length == 1) {
110 |     s_dd = "0" + s_dd
111 |   }
112 |   
113 |   var start_date = yyyy + "-" + s_mm + "-" + s_dd;
114 |   return start_date;
115 | }
116 | ```
117 | 
118 | ```js
119 | function getEndDate(){
120 |   const d = new Date();
121 |   var yyyy = String(d.getFullYear());
122 |   var mm = String(d.getMonth() + 1);
123 |   
124 |   if (mm.length == 1) {
125 |     mm = "0" + mm
126 |   }
127 |   
128 |   var dd = String(d.getDate());
129 |   if (dd.length == 1) {
130 |     dd = "0" + dd
131 |   }
132 |   
133 |   var end_date = yyyy + "-" + mm + "-" + dd;
134 |   return end_date;
135 | }
136 | ```
137 | 
138 | Update the function *myFunction()* as follows to automate the dates.
139 | 
140 | ```js
141 | function myFunction() {
142 |   var data = {
143 |     "client_id": "XXXXXXXXXXXXXXXXXXXXXXXXXXX",
144 |     "secret": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
145 |     "access_token": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
146 |     "start_date": getStartDate(),
147 |     "end_date": getEndDate()
148 |   };
149 |   
150 |   var payload = JSON.stringify(data);
151 |   var options = {
152 |     "method" : "POST",
153 |     "contentType" : "application/json",
154 |     "payload" : payload
155 |   };
156 |   
157 |   var url = "https://development.plaid.com/transactions/get";
158 |   var response = UrlFetchApp.fetch(url, options);
159 |   console.log(response.getContentText());
160 | }
161 | ```
162 | 
163 | The final step is to export the JSON to the active spreadsheet. The JSON object that the <u>/transactions/get</u> HTTP response returns has the following basic structure. All of this information can be found in the [API docs](https://plaid.com/docs/api/products/#transactionsget).
164 | 
165 | ```json
166 | {
167 |   "accounts": ,
168 |   "item": ,
169 |   "request_id": ,
170 |   "total_transactions": ,
171 |   "transactions": [
172 |     {
173 |       "account_id": ,
174 |       "account_owner": ,
175 |       "amount": ,
176 |       "authorized_date": ,
177 |       "authorized_datetime": ,
178 |       "category": ,
179 |       "category_id": ,
180 |       "check_number": ,
181 |       "date": ,
182 |       "datetime": ,
183 |       "iso_currency_code": ,
184 |       "location": ,
185 |       "merchant_name": ,
186 |       "name": ,
187 |       "payment_channel": ,
188 |       "payment_meta": ,
189 |       "pending": ,
190 |       "pending_transaction_id": ,
191 |       "personal_finance_category": ,
192 |       "transaction_code": ,
193 |       "transaction_id": ,
194 |       "transaction_type": ,
195 |       "unofficial_currency_code":
196 |     }
197 |   ]
198 | }
199 | ```
200 | 
201 | There is a lot here, but for this guide we will just use the date, amount, and name for each transaction.
202 | 
203 | The following function takes a JSON object and appends the transaction date, amount, and name to the active spreadsheet (make sure its open). 
204 | 
205 | ```js
206 | function initializeSheet(response) {
207 |   var sheet = SpreadsheetApp.getActiveSheet();
208 |   const obj = JSON.parse(response.getContentText());
209 |   
210 |   for (let i = 0; i < obj.transactions.length; i++) {
211 |     sheet.appendRow([obj.transactions[i].date,obj.transactions[i].amount,obj.transactions[i].name]);
212 |   }
213 | }
214 | ```
215 | 
216 | Update the function *myFunction()* to account for the new feature.
217 | 
218 | ```js
219 | function myFunction() {
220 |   var data = {
221 |     "client_id": "XXXXXXXXXXXXXXXXXXXXXXXXXXX",
222 |     "secret": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
223 |     "access_token": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX",
224 |     "start_date": getStartDate(),
225 |     "end_date": getEndDate()
226 |   };
227 |   
228 |   var payload = JSON.stringify(data);
229 |   var options = {
230 |     "method" : "POST",
231 |     "contentType" : "application/json",
232 |     "payload" : payload
233 |   };
234 |   
235 |   var url = "https://development.plaid.com/transactions/get";
236 |   var response = UrlFetchApp.fetch(url, options);
237 |   initializeSheet(response);
238 | }
239 | ```
240 | 
241 | Run the script a check that the spreadsheet is populated properly.
242 | 
243 | ## Conclusion
244 | 
245 | At this point the data pipeline from your bank to spreadsheet should be complete. 
246 | 
247 | There are many ways to extend this example. Quicker methods to receive the *access_token* should be possible since running a complete application seems a bit excessive. The App Script could be extended to retrieve different data objects or to do specific data validation and formatting.
248 | 
249 | Please leave a star if this was helpful.
250 | 


--------------------------------------------------------------------------------
/code.gs:
--------------------------------------------------------------------------------
 1 | function myFunction() {
 2 | var url = "https://development.plaid.com/transactions/get";
 3 | var data = {
 4 | "client_id": "62d56ae4200ab2001404a241",
 5 | "secret": "579163fc2e18aa7e91ae14a23b9934",
 6 | "access_token": "access-development-7d485e9d-d150-4d79-8fd5-53846c744b53",
 7 | "start_date": getStartDate(),
 8 | "end_date": getEndDate()
 9 | };
10 | var options = {
11 | "method" : "POST",
12 | "contentType" : "application/json",
13 | "payload" : JSON.stringify(data)
14 | };
15 | 
16 | var response = UrlFetchApp.fetch(url, options);
17 | const obj = JSON.parse(response.getContentText());
18 | 
19 | //parses the response and inserts the data into two sheets in the current active spreadsheet: "Balance" and "Transactions". 
20 | //Then transactions are sorted by date in descending order, and any duplicate transactions are removed using the removeDuplicateRows() function.
21 | 
22 | var balanceSheet = SpreadsheetApp.getActive().getSheetByName("Balance");
23 | balanceSheet.deleteRow(balanceSheet.getLastRow());
24 | balanceSheet.appendRow(["Current Balance:", obj.accounts[0].balances.current]);
25 | 
26 | var transactionsSheet = SpreadsheetApp.getActive().getSheetByName("Transactions");
27 | var data = obj.transactions.map(transaction => [transaction.date, transaction.name, transaction.amount, transaction.category, transaction.transaction_id]);
28 | transactionsSheet.getRange(transactionsSheet.getLastRow() + 1, 1, data.length, data[0].length).setValues(data);
29 | transactionsSheet.sort(1, false);
30 | removeDuplicateRows("Transactions");
31 | }
32 | 
33 | //used to generate the start and end dates for the transactions to be fetched. 
34 | //The start date is determined by taking the current date and setting the month to the previous month, unless it is already January, in which case it is set to January. 
35 | //The day of the month is set to the 28th unless the current day is already later than the 28th, in which case it is left unchanged. The end date is simply the current date.
36 | 
37 | function getStartDate() {
38 |   const d = new Date();
39 |   const yyyy = String(d.getFullYear());
40 |   let s_mm = d.getMonth() + 1;
41 |   if (s_mm === 12) {
42 |     s_mm = 1;
43 |   }
44 |   const s_dd = d.getDate() > 28 ? 28 : d.getDate();
45 |   const start_date = `${yyyy}-${s_mm.toString().padStart(2, "0")}-${s_dd.toString().padStart(2, "0")}`;
46 |   return start_date;
47 | }
48 | function getEndDate() {
49 |   const d = new Date();
50 |   return d.toISOString().split('T')[0];
51 | }
52 | 
53 | function removeDuplicateRows(sheetName) {
54 |   var sheet = SpreadsheetApp.getActiveSpreadsheet().getSheetByName(sheetName);
55 |   var data = sheet.getDataRange().getValues();
56 |   var uniqueData = data.reduce((unique, transaction) => !unique.some(t => t[4] === transaction[4]) ? [...unique, transaction] : unique, []);
57 |   sheet.clearContents();
58 |   sheet.getRange(1, 1, uniqueData.length, uniqueData[0].length).setValues(uniqueData);
59 |   sheet.sort(1, false);
60 | }
61 | 


--------------------------------------------------------------------------------