├── README.md
├── esb-smart-meter-reader.py
└── esb-smart-meter.png


/README.md:
--------------------------------------------------------------------------------
 1 | #  esb-smart-meter-reading-automation
 2 | 
 3 | ![](https://github.com/badger707/esb-smart-meter-reading-automation/blob/main/esb-smart-meter.png)
 4 | <br><br>
 5 | ## How to read your Smart Meter data automatically?
 6 | Simple Python code to download your smart electricy meter readings/data from ESB Networks user portal.
 7 | <br>
 8 | ## Requirements<br>
 9 | * You need to create account with ESBN here https://myaccount.esbnetworks.ie <br>
10 | * In your account, link your electricity meter MPRN
11 | <br><br>
12 | ## Script Setup<br>
13 | * update MPRN, user and password in the script:<br>
14 | ```
15 | meter_mprn = "mprn_number"
16 | esb_user_name = "email@email.com"
17 | esb_password = "password"
18 | ```
19 | 
20 | ## Output Format<br>
21 | * Either CSV of JSON. Select required format by uncommenting one of last 2 rows where it says: <br>
22 | ```
23 | ###/ Select file format of your choice /###
24 | #print(csv_file)
25 | print(json_file)
26 | ```
27 | 
28 | ## Debug Mode for troubleshooting
29 | * Set debug_mode to True if you want to see extended info of what sript is doing and sending/receiving.
30 | ````
31 | ## Debug Mode print messages, set to True or False ##
32 | debug_mode=False
33 | ````
34 | 
35 | ## Error Messages
36 | * When things goes wrong and User Portal starts serving human verification pages, script will stop with one or another error message based on received response content analysis:
37 | ````
38 |  [Script Message] Unable to reach login page -- too many retries (max=2 in 24h) or prior sessions was not closed properly. Please try again after midnight.
39 | ````
40 | ````
41 |  [FAILED] Unable to get full set of required cookies -- too many retries (captcha?) or prior sessions was not closed properly. Please wait 6 hours for server to timeout and try again.
42 | ````
43 | 
44 | ## Known Limitations<br>
45 | * ESBN User Portal have enabled human verification process for logins since around Nov'24, this creates inconvenience/chalenges regardless of what you use -- standard web browser or script like this.
46 | * Server side limit: it does allow you to make only 2 clean logins per one IP per 24 hours without triggering human verification or captcha traps.
47 | * Trying to make 3 or more logins during the day - server will start serving human verification pages or captcha or complain about disabled javascript or disabled cookies on your side. This script will detect this and will provide comments in console/terminal and will terminate/stop.
48 | * Server side timers/blockers/limits resets once a day at midnight - plan your workflow accordingly.
49 |   
50 | 
51 | ## UPDATES: <br>
52 | * 04-Jan-2025 -- Reworked login and file download proccess due to Nov'24 portal changes.
53 | * 24-Jul-2024 -- Python script changes to accomodate ESB Networks user portal changes to download historic usage file. 
54 | * 09-May-2024 -- there was some changes on ESB side and this broke CSV parsing in script, fixed & tested, JSON output works as expected.
55 | 
56 | 
57 | 


--------------------------------------------------------------------------------
/esb-smart-meter-reader.py:
--------------------------------------------------------------------------------
  1 | #!/usr/bin/env python3
  2 | 
  3 | import requests
  4 | from random import randint
  5 | from time import sleep
  6 | from bs4 import BeautifulSoup   # pip install beautifulsoup4
  7 | import re as re
  8 | import json
  9 | import csv
 10 | 
 11 | ## Debug Mode print messages, set to True or False ##
 12 | debug_mode=False
 13 | 
 14 | ###### START OF SCRIPT ###### 
 15 | 
 16 | if debug_mode:print("##### REQUEST 1 -- GET [https://myaccount.esbnetworks.ie/] ######")
 17 | meter_mprn = "mprn_number"
 18 | esb_user_name = "email@email.com"
 19 | esb_password = "password"
 20 | session = requests.Session()
 21 | session.headers.update({
 22 |     'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0',
 23 | })    
 24 | 
 25 | try:
 26 |     request_1_response = session.get('https://myaccount.esbnetworks.ie/', allow_redirects=True, timeout= (10,5))     # timeout -- 10sec for connect and 5sec for response
 27 | except requests.exceptions.Timeout:
 28 |     print("[FAILED] The request timed out, server is not responding. Try again later.")
 29 |     session.close()
 30 |     raise SystemExit(0)
 31 | except requests.exceptions.RequestException as e:
 32 |     print("An error occurred:", e)
 33 |     session.close()
 34 |     raise SystemExit(0)
 35 | 
 36 | result = re.findall(r"(?<=var SETTINGS = )\S*;", str(request_1_response.content))
 37 | settings = json.loads(result[0][:-1])
 38 | tester_soup = BeautifulSoup(request_1_response.content, 'html.parser')
 39 | page_title = tester_soup.find("title")
 40 | request_1_response_cookies = session.cookies.get_dict()
 41 | x_csrf_token = settings['csrf']
 42 | transId = settings['transId']
 43 | 
 44 | if debug_mode:
 45 |     print("[!] Request #1 Page Title :: ", page_title.text)
 46 |     print("[!] Request #1 Status Code ::", request_1_response.status_code)
 47 |     print("[!] Request #1 Response Headers ::", request_1_response.headers)
 48 |     print("[!] Request #1 Cookies Captured ::", request_1_response_cookies)
 49 |     print("x_csrf_token ::", x_csrf_token)
 50 |     print("transId ::", transId)
 51 | 
 52 | x_ms_cpim_sso = request_1_response_cookies.get('x-ms-cpim-sso:esbntwkscustportalprdb2c01.onmicrosoft.com_0')
 53 | x_ms_cpim_csrf = request_1_response_cookies.get('x-ms-cpim-csrf')
 54 | x_ms_cpim_trans = request_1_response_cookies.get('x-ms-cpim-trans')
 55 | 
 56 | if debug_mode:
 57 |     print("##### creating x_ms_cpim cookies ######")
 58 |     print("x_ms_cpim_sso ::", request_1_response_cookies.get('x-ms-cpim-sso:esbntwkscustportalprdb2c01.onmicrosoft.com_0'))
 59 |     print("x_ms_cpim_csrf ::", request_1_response_cookies.get('x-ms-cpim-csrf'))
 60 |     print("x_ms_cpim_trans ::", request_1_response_cookies.get('x-ms-cpim-trans'))
 61 |     print("##### REQUEST 2 -- POST [SelfAsserted] ######")
 62 | 
 63 | sleeping_delay= randint(10,20)
 64 | if debug_mode:print('random sleep for',sleeping_delay,'seconds...')
 65 | sleep(sleeping_delay)
 66 | 
 67 | request_2_response = session.post(
 68 |     'https://login.esbnetworks.ie/esbntwkscustportalprdb2c01.onmicrosoft.com/B2C_1A_signup_signin/SelfAsserted?tx=' + transId + '&p=B2C_1A_signup_signin', 
 69 |     data={
 70 |       'signInName': esb_user_name, 
 71 |       'password': esb_password, 
 72 |       'request_type': 'RESPONSE'
 73 |     },
 74 |     headers={
 75 |       'x-csrf-token': x_csrf_token,
 76 |       'User-Agent': 'Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0',
 77 |       'Accept': 'application/json, text/javascript, */*; q=0.01',
 78 |       'Accept-Language': 'en-US,en;q=0.5',
 79 |       'Accept-Encoding': 'gzip, deflate, br',
 80 |       'Content-Type': 'application/x-www-form-urlencoded; charset=UTF-8',
 81 |       'X-Requested-With': 'XMLHttpRequest',
 82 |       'Origin': 'https://login.esbnetworks.ie',
 83 |       'Dnt': '1',
 84 |       'Sec-Gpc': '1',
 85 |       'Sec-Fetch-Dest': 'empty',
 86 |       'Sec-Fetch-Mode': 'cors',
 87 |       'Sec-Fetch-Site': 'same-origin',
 88 |       'Priority': 'u=0',
 89 |       'Te': 'trailers',
 90 |     },
 91 |     cookies={
 92 |         'x-ms-cpim-csrf':request_1_response_cookies.get('x-ms-cpim-csrf'),
 93 |     #    'x-ms-cpim-sso:esbntwkscustportalprdb2c01.onmicrosoft.com_0':request_1_response_cookies.get('x-ms-cpim-sso:esbntwkscustportalprdb2c01.onmicrosoft.com_0'),
 94 |         'x-ms-cpim-trans':request_1_response_cookies.get('x-ms-cpim-trans'),
 95 |     },
 96 |     allow_redirects=False)
 97 | 
 98 | request_2_response_cookies = session.cookies.get_dict()
 99 | if debug_mode:
100 |     print("[!] Request #2 Status Code ::", request_2_response.status_code)
101 |     print("[!] Request #2 Response Headers ::", request_2_response.headers)
102 |     print("[!] Request #2 Cookies Captured :: ", request_2_response_cookies)
103 |     print("[!] Request #2 text :: ", request_2_response.text)
104 |     print("##### REQUEST 3 -- GET [API CombinedSigninAndSignup] ######")
105 | 
106 | request_3_response = session.get(
107 |     'https://login.esbnetworks.ie/esbntwkscustportalprdb2c01.onmicrosoft.com/B2C_1A_signup_signin/api/CombinedSigninAndSignup/confirmed',
108 |     params={
109 |       'rememberMe': False,
110 |       'csrf_token': x_csrf_token,
111 |       'tx': transId,
112 |       'p': 'B2C_1A_signup_signin',
113 |     },
114 |     headers={
115 |       "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0",
116 |       "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
117 |       "Accept-Language": "en-US,en;q=0.5",
118 |       "Accept-Encoding": "gzip, deflate, br",
119 |       "Dnt": "1",
120 |       "Sec-Gpc": "1",
121 |       "Sec-Fetch-Dest": "document",
122 |       "Sec-Fetch-Mode": "navigate",
123 |       "Sec-Fetch-Site": "same-origin",
124 |       "Priority": "u=0, i",
125 |       "Pragma": "no-cache",
126 |       "Cache-Control": "no-cache",
127 |       "Te": "trailers",
128 |     },
129 |     cookies={
130 |         "x-ms-cpim-csrf":request_2_response_cookies.get("x-ms-cpim-csrf"),
131 |     #    "x-ms-cpim-sso:esbntwkscustportalprdb2c01.onmicrosoft.com_0": request_2_response_cookies.get("x-ms-cpim-sso:esbntwkscustportalprdb2c01.onmicrosoft.com_0"),
132 |         'x-ms-cpim-trans':request_2_response_cookies.get("x-ms-cpim-trans"),
133 |     },
134 |   )
135 | 
136 | tester_soup = BeautifulSoup(request_3_response.content, 'html.parser')
137 | page_title = tester_soup.find("title")
138 | request_3_response_cookies = session.cookies.get_dict()
139 | 
140 | if debug_mode:
141 |     print("[!] Page Title :: ", page_title.text)      # will print "Loading..." if failed
142 |     print("[!] Request #3 Status Code ::", request_3_response.status_code)
143 |     print("[!] Request #3 Response Headers ::", request_3_response.headers)
144 |     print("[!] Request #3 Cookies Captured :: ", request_3_response_cookies)
145 |     print("[!] Request #3 Content :: ", request_3_response.content)
146 |     print("##### TEST IF SUCCESS ######")
147 | 
148 | request_3_response_head_test = request_3_response.text[0:21]
149 | if (request_3_response_head_test == "<!DOCTYPE html PUBLIC"):
150 |     page_title = tester_soup.find("title")
151 |     if debug_mode:
152 |         print("[PASS] SUCCESS -- ALL OK [PASS]")
153 |         print("[!] Page Title :: ", page_title.text)
154 | else: 
155 |     session.close()
156 |     try:
157 |         tester_soup_msg = tester_soup.find('h1')
158 |         tester_soup_msg = tester_soup_msg.text
159 |         print("[FAILED] Page response ::", tester_soup_msg)
160 |     except: tester_soup_msg = ""
161 |     try:
162 |         no_js_msg = tester_soup.find('div', id='no_js')
163 |         no_js_msg = no_js_msg.text
164 |         print("[FAILED] Page response :: ", no_js_msg)
165 |     except: no_js_msg = ""
166 |     try:
167 |         no_cookie_msg = tester_soup.find('div', id='no_cookie')
168 |         no_cookie_msg = no_cookie_msg.text
169 |         print("[FAILED] Page response :: ", no_cookie_msg)
170 |     except: no_cookie_msg = ""
171 |     print("[Script Message] Unable to reach login page -- too many retries (max=2 in 24h) or prior sessions was not closed properly. Please try again after midnight.")
172 |     raise SystemExit(0)
173 |     
174 | if debug_mode:print("##### REQUEST - SOUP - state & client_info & code ######")
175 | soup = BeautifulSoup(request_3_response.content, 'html.parser')
176 | try:
177 |     form = soup.find('form', {'id': 'auto'})
178 |     login_url_ = form['action']
179 |     state_ = form.find('input', {'name': 'state'})['value']
180 |     client_info_ = form.find('input', {'name': 'client_info'})['value']
181 |     code_ = form.find('input', {'name': 'code'})['value']
182 |     if debug_mode:
183 |         print("login url ::" ,login_url_)
184 |         print("state_ ::", state_)
185 |         print("client_info_ ::", client_info_)
186 |         print("code_ ::", code_)
187 | except:
188 |     print("[FAILED] Unable to get full set of required cookies from [request_3_response.content] -- too many retries (captcha?) or prior sessions was not closed properly. Please wait 6 hours for server to timeout and try again.")
189 |     session.close()
190 |     raise SystemExit(0)
191 | 
192 | if debug_mode:print("##### REQUEST 4 -- POST [signin-oidc] ######")
193 | sleeping_delay= randint(2,5)
194 | if debug_mode:print('random sleep for',sleeping_delay,'seconds...')
195 | sleep(sleeping_delay)
196 | 
197 | request_4_headers = {
198 |     "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0",
199 |     "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
200 |     "Accept-Language": "en-US,en;q=0.5",
201 |     "Accept-Encoding": "gzip, deflate, br",
202 |     "Content-Type": "application/x-www-form-urlencoded",
203 |     "Origin": "https://login.esbnetworks.ie",
204 |     "Dnt": "1",
205 |     "Sec-Gpc": "1",
206 |     "Referer": "https://login.esbnetworks.ie/",
207 |     "Upgrade-Insecure-Requests": "1",
208 |     "Sec-Fetch-Dest": "document",
209 |     "Sec-Fetch-Mode": "navigate",
210 |     "Sec-Fetch-Site": "same-site",
211 |     "Priority": "u=0, i",
212 |     "Pragma": "no-cache",
213 |     "Cache-Control": "no-cache",
214 |     "Te": "trailers",
215 | }
216 | 
217 | request_4_response = session.post(
218 |         login_url_,
219 |         allow_redirects=False,
220 |         data={
221 |           'state': state_,
222 |           'client_info': client_info_,
223 |           'code': code_,
224 |         },
225 |         headers=request_4_headers,
226 |     )
227 | 
228 | request_4_response_cookies = session.cookies.get_dict()
229 | if debug_mode:
230 |     print("[!] Request #4 Status Code ::", request_4_response.status_code)  # expect 302
231 |     print("[!] Request #4 Response Headers ::", request_4_response.headers)
232 |     print("[!] Request #4 Cookies Captured ::", request_4_response_cookies)
233 |     print("[!] Request #4 Content ::", request_4_response.content)
234 |     print("##### REQUEST 5 -- GET [https://myaccount.esbnetworks.ie] ######")
235 | 
236 | request_5_url = "https://myaccount.esbnetworks.ie"
237 | request_5_headers = {
238 |     "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0",
239 |     "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
240 |     "Accept-Language": "en-US,en;q=0.5",
241 |     "Accept-Encoding": "gzip, deflate, br",
242 |     "Referer": "https://login.esbnetworks.ie/",
243 |     "Dnt": "1",
244 |     "Sec-Gpc": "1",
245 |     "Upgrade-Insecure-Requests": "1",
246 |     "Sec-Fetch-Dest": "document",
247 |     "Sec-Fetch-Mode": "navigate",
248 |     "Sec-Fetch-Site": "same-site",
249 |     "Priority": "u=0, i",
250 |     "Pragma": "no-cache",
251 |     "Cache-Control": "no-cache",
252 |     "Te": "trailers",
253 | }
254 | request_5_cookies = {
255 |     "ARRAffinity":request_4_response_cookies.get("ARRAffinity"),
256 |     "ARRAffinitySameSite":request_4_response_cookies.get("ARRAffinitySameSite"),
257 | }
258 | 
259 | request_5_response = session.get(request_5_url,headers=request_5_headers,cookies=request_5_cookies)
260 | request_5_response_cookies = session.cookies.get_dict()
261 | 
262 | if debug_mode:
263 |     print("[!] Request #5 Status Code ::", request_5_response.status_code)
264 |     print("[!] Request #5 Response Headers ::", request_5_response.headers)
265 |     print("[!] Request #5 Cookies Captured ::", request_5_response_cookies)
266 |     print("[!] Request #5 Content ::", request_5_response.content)
267 |     print("##### Welcome page block #####")
268 |     
269 | user_welcome_soup = BeautifulSoup(request_5_response.text,'html.parser')
270 | welcome_page_title_ = user_welcome_soup.find('title')
271 | if debug_mode:print("[!] Page Title ::", welcome_page_title_.text)                   # it should print "Customer Portal"
272 | welcome_page_title_ = user_welcome_soup.find('h1', class_='esb-title-h1')
273 | if debug_mode:print("[!] Confirmed User Login ::", welcome_page_title_.text)    # It should print "Welcome, Name Surname"
274 | asp_net_core_cookie = request_5_response_cookies.get(".AspNetCore.Cookies")
275 | 
276 | if debug_mode:print("##### REQUEST 6 -- GET [Api/HistoricConsumption] ######")
277 | sleeping_delay= randint(3,8)
278 | if debug_mode:print('random sleep for',sleeping_delay,'seconds...')
279 | sleep(sleeping_delay)
280 | request_6_url = "https://myaccount.esbnetworks.ie/Api/HistoricConsumption"
281 | request_6_cookies = {
282 |     "ARRAffinity":request_4_response_cookies.get("ARRAffinity"),
283 |     "ARRAffinitySameSite":request_4_response_cookies.get("ARRAffinitySameSite"),
284 |     ".AspNetCore.Cookies":asp_net_core_cookie,
285 | }
286 | request_6_headers = {
287 |     "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0",
288 |     "Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
289 |     "Accept-Language": "en-US,en;q=0.5",
290 |     "Accept-Encoding": "gzip, deflate, br",
291 |     "Dnt": "1",
292 |     "Sec-Gpc": "1",
293 |     "Referer": "https://myaccount.esbnetworks.ie/",
294 |     "Upgrade-Insecure-Requests": "1",
295 |     "Sec-Fetch-Dest": "document",
296 |     "Sec-Fetch-Mode": "navigate",
297 |     "Sec-Fetch-Site": "same-origin",
298 |     "Sec-Fetch-User": "?1",
299 |     "Priority": "u=0, i",
300 |     "Te": "trailers",
301 | }
302 | 
303 | request_6_response = session.get(request_6_url, headers=request_6_headers,cookies=request_6_cookies)
304 | request_6_response_cookies = session.cookies.get_dict()
305 | 
306 | if debug_mode:
307 |     print("[!] Request #6 Status Code ::", request_5_response.status_code)
308 |     print("[!] Request #6 Response Headers ::", request_5_response.headers)
309 |     print("[!] Request #6 Cookies Captured ::", request_6_response_cookies)
310 |     print("##### My Energy Consumption - Customer Portal #####")
311 | consumption_soup = BeautifulSoup(request_6_response.text,'html.parser')
312 | consumption_page_title_ = consumption_soup.find('title')
313 | welcome_page_title_ = consumption_soup.find('h1', class_='esb-title-h1')
314 | if debug_mode:
315 |     print("[!] Page Title ::", consumption_page_title_.text)    # it should print "My Energy Consumption - Customer Portal"
316 |     print("[!] 'esb-title-h1' ::", welcome_page_title_.text)    # It should print "My Energy Consumption"
317 |     print("##### REQUEST 7 -- GET [file download token] ######")
318 | sleeping_delay= randint(2,5)
319 | if debug_mode:print('random sleep for',sleeping_delay,'seconds...')
320 | sleep(sleeping_delay)
321 | 
322 | request_7_url = "https://myaccount.esbnetworks.ie/af/t"
323 | request_7_headers = {
324 |     "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0",
325 |     "Accept": "*/*",
326 |     "Accept-Language": "en-US,en;q=0.5",
327 |     "Accept-Encoding": "gzip, deflate, br",
328 |     "X-Returnurl": "https://myaccount.esbnetworks.ie/Api/HistoricConsumption",
329 |     "Dnt": "1",
330 |     "Sec-Gpc": "1",
331 |     "Referer": "https://myaccount.esbnetworks.ie/Api/HistoricConsumption",
332 |     "Sec-Fetch-Dest": "empty",
333 |     "Sec-Fetch-Mode": "cors",
334 |     "Sec-Fetch-Site": "same-origin",
335 |     "Priority": "u=0",
336 |     "Te": "trailers",
337 | }
338 | request_7_cookies = {
339 |     "ARRAffinity":request_4_response_cookies.get("ARRAffinity"),
340 |     "ARRAffinitySameSite":request_4_response_cookies.get("ARRAffinitySameSite"),
341 | }
342 | request_7_response = session.get(request_7_url,headers=request_7_headers,cookies=request_7_cookies)
343 | request_7_response_cookies = session.cookies.get_dict()
344 | file_download_token = json.loads(request_7_response.text)["token"]
345 | if debug_mode:
346 |     print("[!] Request #7 Status Code ::", request_7_response.status_code)
347 |     print("[!] Request #7 Response Headers ::", request_7_response.headers)
348 |     print("[!] Request #7 Cookies Captured ::", request_7_response_cookies)
349 |     print("[!] Request #7 Content ::", request_7_response.content)
350 |     print("File download token :: ",file_download_token)
351 |     print("##### REQUEST 8 -- GET [/DataHub/DownloadHdfPeriodic] ######")
352 | request_8_url = "https://myaccount.esbnetworks.ie/DataHub/DownloadHdfPeriodic"
353 | request_8_headers = {
354 |     "User-Agent": "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:133.0) Gecko/20100101 Firefox/133.0",
355 |     "Accept": "*/*",
356 |     "Accept-Language": "en-US,en;q=0.5",
357 |     "Accept-Encoding": "gzip, deflate, br",
358 |     "Referer": "https://myaccount.esbnetworks.ie/Api/HistoricConsumption",
359 |     "Content-Type": "application/json",
360 |     "X-Returnurl": "https://myaccount.esbnetworks.ie/Api/HistoricConsumption",
361 |     "X-Xsrf-Token": file_download_token,
362 |     "Origin": "https://myaccount.esbnetworks.ie",
363 |     "Dnt": "1",
364 |     "Sec-Gpc": "1",
365 |     "Sec-Fetch-Dest": "empty",
366 |     "Sec-Fetch-Mode": "cors",
367 |     "Sec-Fetch-Site": "same-origin",
368 |     "Priority": "u=4",
369 |     "Cache-Control": "max-age=0",
370 |     "Te": "trailers",
371 | }
372 | payload_data = {
373 |     "mprn": meter_mprn,
374 |     "searchType": "intervalkw"  ### <<<< !!! THIS IS WHERE YOU SELECT WHICH FILE YOU WANT !!!
375 | }
376 | request_8_response = session.post(request_8_url, headers=request_8_headers, json=payload_data)
377 | if debug_mode:
378 |     print("[!] Request #8 Status Code ::", request_8_response.status_code)
379 |     print("[!] Request #8 Response Headers ::", request_8_response.headers)
380 |     print("[END] HTTP Requests completed. Closing session.")
381 | session.close()
382 | 
383 | if debug_mode:print("#### Getting file attributes ####")
384 | file_size_ = request_8_response.headers.get("Content-Length")
385 | file_name_ = request_8_response.headers.get("Content-Disposition")
386 | file_name_ = file_name_.split(";")
387 | file_name_ = file_name_[1].split("=")
388 | file_name_ = file_name_[1]
389 | if debug_mode:
390 |     print("[!] File size, bytes ::",file_size_)
391 |     print("[!] Disposition ::",file_name_)     # it should print [attachment; filename=HDF_kW_mprn_date.csv; filename*=UTF-8''HDF_kW_mprn_date.csv]
392 |     print("[!] File Name ::",file_name_)
393 |     print("##### Checking/converting received CSV file/object #####")
394 | if (isinstance(request_8_response.content, bytes)):
395 |     if debug_mode:print("[!] Object class is 'bytes', decoding to 'utf-8' and continuing...")
396 |     csv_file = request_8_response.content.decode("utf-8")
397 | elif(isinstance(request_8_response.content, str)):
398 |     if debug_mode:print("[!] Object class is 'string', continuing...")
399 |     csv_file = request_8_response.content
400 | else:
401 |     print("[FAIL] received object is neither 'bytes' nor 'string, stopping here, please check/validate [request_8_response.content]")
402 |     raise SystemExit(0)
403 | if debug_mode:
404 |     print("[!] CSV data sample ::")
405 |     print(csv_file[0:500])
406 | try:
407 |     if debug_mode:print("[!] Converting CSV to JSON...")
408 |     if(csv_file[0:4] == "MPRN"):
409 |         my_json = []
410 |         csv_reader = csv.DictReader(csv_file.split('\n'))
411 |         for row in csv_reader:
412 |             my_json.append(row)
413 |         json_file = json.dumps(my_json, indent=2)
414 |         if debug_mode:print("[COMPLETED] JSON file created. Use [json_file] .")
415 |     else:
416 |         print("[FAIL] Something is wrong with CSV file header structure, cant convert to JSON. Expected [csv_file[0:4] == 'MPRN'] but failed.")
417 | except:
418 |     print("[FAIL] Something is wrong with CSV file structure, cant convert to JSON, check/validate if [csv.DictReader(csv_file.split('\n'))] works.")
419 |     raise SystemExit(0)
420 | 
421 | ###### END OF SCRIPT ###### 
422 | 
423 | ###/ Select file format of your choice /###
424 | #print(csv_file)
425 | print(json_file)
426 | 


--------------------------------------------------------------------------------
/esb-smart-meter.png:
--------------------------------------------------------------------------------
https://raw.githubusercontent.com/badger707/esb-smart-meter-reading-automation/fc6817d74c15e7f0740df07243bd54b0d62e095c/esb-smart-meter.png


--------------------------------------------------------------------------------