├── .gitignore ├── devdata └── env.json ├── robot.yaml ├── tasks.robot ├── conda.yaml ├── LICENSE ├── providers ├── pipedrive.robot ├── namecheap.robot └── digitalocean.robot ├── helpers.robot └── README.md /.gitignore: -------------------------------------------------------------------------------- 1 | output/** 2 | browser 3 | output.xml 4 | playwright-log.txt 5 | report.html 6 | log.html -------------------------------------------------------------------------------- /devdata/env.json: -------------------------------------------------------------------------------- 1 | { 2 | "DIGITAL_OCEAN_SESSION": "Paste the value of '_digitalocean2_session_v4' in the cookies of cloud.digitalocean.com" 3 | } 4 | -------------------------------------------------------------------------------- /robot.yaml: -------------------------------------------------------------------------------- 1 | tasks: 2 | Run all tasks: 3 | shell: python -m robot --report NONE --outputdir output --logtitle "Task log" tasks.robot 4 | 5 | condaConfigFile: conda.yaml 6 | artifactsDir: output 7 | PATH: 8 | - . 9 | PYTHONPATH: 10 | - . 11 | ignoreFiles: 12 | - .gitignore 13 | -------------------------------------------------------------------------------- /tasks.robot: -------------------------------------------------------------------------------- 1 | *** Settings *** 2 | Resource helpers.robot 3 | Resource ./providers/digitalocean.robot 4 | Resource ./providers/pipedrive.robot 5 | Resource ./providers/namecheap.robot 6 | 7 | *** Tasks *** 8 | 9 | Bill Collector 10 | ${provider}= Select Provider 11 | Run Keyword And Return Status ${provider} -------------------------------------------------------------------------------- /conda.yaml: -------------------------------------------------------------------------------- 1 | channels: 2 | # Define conda channels here. 3 | - conda-forge 4 | 5 | dependencies: 6 | # Define conda packages here. 7 | # If available, always prefer the conda version of a package, installation will be faster and more efficient. 8 | # https://anaconda.org/search 9 | - python=3.7.5 10 | 11 | - pip=20.1 12 | - nodejs=14.17.4 13 | - pip: 14 | # Define pip packages here. 15 | # https://pypi.org/ 16 | - robotframework-browser==14.3.0 # https://github.com/MarketSquare/robotframework-browser/releases 17 | - rpaframework==19.4.2 # https://rpaframework.org/releasenotes.html 18 | rccPostInstall: 19 | - rfbrowser init 20 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2021-present Benjamin André-Micolon. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /providers/pipedrive.robot: -------------------------------------------------------------------------------- 1 | *** Settings *** 2 | Library RPA.Browser.Playwright 3 | Library RPA.Dialogs 4 | Library Collections 5 | Library OperatingSystem 6 | Resource ../helpers.robot 7 | 8 | 9 | *** Keywords *** 10 | Collect Page Bills 11 | ${bills}= Create List 12 | ${elements}= Get Elements table tr[data-test] 13 | FOR ${element} IN @{elements} 14 | ${invoiceNumber}= Get Text ${element} [data-test=invoice-number] 15 | ${date}= Get Text ${element} [data-test=invoice-date] 16 | ${downloadURL}= Get Property ${element} > td a href 17 | ${bill}= Create Dictionary invoiceNumber=${invoiceNumber} date=${date} downloadURL=${downloadURL} 18 | Append To List ${bills} ${bill} 19 | END 20 | [Return] ${bills} 21 | 22 | Pipedrive 23 | New Browser chromium headless=false 24 | New Context acceptDownloads=${TRUE} viewport={'width': 1920, 'height': 1080} 25 | New Page url=https://app.pipedrive.com/auth/login 26 | Show dialog title=Login to Pipedrive then click close on_top=true 27 | Wait all dialogs 28 | Wait Until Network Is Idle 29 | 30 | ${origin}= Execute JavaScript window.location.origin 31 | 32 | Go To ${origin}/settings/subscription/invoices 33 | 34 | ${bills}= Pipedrive.Collect Page Bills 35 | 36 | ${selectedBills} ${shouldExit} Display Picker ${bills} FALSE 37 | Download Selected Bills ${bills} ${selectedBills} 38 | Close Browser -------------------------------------------------------------------------------- /providers/namecheap.robot: -------------------------------------------------------------------------------- 1 | *** Settings *** 2 | Library RPA.Browser.Playwright 3 | Library RPA.Dialogs 4 | Library Collections 5 | Library OperatingSystem 6 | Library DateTime 7 | Resource ../helpers.robot 8 | 9 | 10 | *** Keywords *** 11 | Get Last 10 Years Invoices 12 | Click .dropdown .show-calendar 13 | ${currentDate}= Get Current Date 14 | ${10YearsAgo}= Add Time To Date ${currentDate} -3650 day %m/%d/%Y 15 | Fill Text .daterangepicker_start_input .input-mini ${10YearsAgo} 16 | Click .applyBtn 17 | 18 | Select All Pages 19 | Click .pagination-small 20 | Click .pagination-small .dropdown-content li:last-of-type a.top-pagesize-change 21 | 22 | 23 | 24 | Collect Page Bills 25 | ${bills}= Create List 26 | ${elements}= Get Elements table tr.with-extra-row 27 | FOR ${element} IN @{elements} 28 | ${invoiceNumber}= Get Text ${element} > td:nth-of-type(1) 29 | ${date}= Get Text ${element} > td:nth-of-type(3) 30 | ${downloadURL}= Get Property ${element} > td.text-right ul.dropdown li:last-of-type a href 31 | ${bill}= Create Dictionary invoiceNumber=${invoiceNumber} date=${date} downloadURL=${downloadURL} 32 | Append To List ${bills} ${bill} 33 | END 34 | [Return] ${bills} 35 | 36 | Namecheap 37 | New Browser chromium headless=false 38 | New Context acceptDownloads=${TRUE} viewport={'width': 1920, 'height': 1080} 39 | New Page url=https://www.namecheap.com/myaccount/login/ 40 | Show dialog title=Login to Namecheap then click close on_top=true 41 | Wait all dialogs 42 | Wait Until Network Is Idle 43 | 44 | Go To https://ap.www.namecheap.com/Profile/Billing/Orders 45 | 46 | Get Last 10 Years Invoices 47 | Select All Pages 48 | 49 | ${bills}= Namecheap.Collect Page Bills 50 | 51 | ${selectedBills} ${shouldExit} Display Picker ${bills} FALSE 52 | Download Selected Bills ${bills} ${selectedBills} 53 | Close Browser -------------------------------------------------------------------------------- /helpers.robot: -------------------------------------------------------------------------------- 1 | *** Settings *** 2 | Library RPA.Browser.Playwright 3 | Library RPA.Dialogs 4 | Library OperatingSystem 5 | 6 | *** Keywords *** 7 | Get Providers 8 | ${providers}= Create List Digital Ocean Pipedrive Namecheap 9 | 10 | [Return] ${providers} 11 | 12 | Get Invoice Id 13 | [Arguments] ${date} ${invoiceNumber} 14 | 15 | ${invoiceId}= Evaluate "${date}${invoiceNumber}".replace(' ', '_').replace('#', '') 16 | 17 | [Return] ${invoiceId} 18 | 19 | Get Invoice Label 20 | [Arguments] ${date} ${invoiceNumber} 21 | 22 | IF "${invoiceNumber}" == "None" 23 | ${invoiceLabel}= Set Variable ${date} 24 | ELSE 25 | ${invoiceLabel}= Set Variable ${date} - ${invoiceNumber} 26 | END 27 | 28 | [Return] ${invoiceLabel} 29 | 30 | 31 | Display Picker 32 | [Arguments] ${bills} ${hasMorePages} 33 | Add heading Bills to collect 34 | FOR ${bill} IN @{bills} 35 | ${invoiceNumber}= Set Variable ${bill["invoiceNumber"]} 36 | ${billDate}= Set Variable ${bill["date"]} 37 | ${billId}= Get Invoice Id ${billDate} ${invoiceNumber} 38 | ${invoiceLabel}= Get Invoice Label ${billDate} ${invoiceNumber} 39 | Add checkbox name=${billId} label=${invoiceLabel} default=False 40 | END 41 | 42 | IF "${hasMorePages}" == "TRUE" 43 | Add submit buttons buttons=Get older bills, Done 44 | END 45 | 46 | ${selectedBills}= Run dialog 47 | 48 | IF "${selectedBills.submit}" == "Done" 49 | ${shouldExit}= Set Variable TRUE 50 | ELSE 51 | ${shouldExit}= Set Variable FALSE 52 | END 53 | 54 | [Return] ${selectedBills} ${shouldExit} 55 | 56 | Download Selected Bills 57 | [Arguments] ${bills} ${selectedBills} 58 | FOR ${bill} IN @{bills} 59 | ${invoiceNumber}= Set Variable ${bill["invoiceNumber"]} 60 | ${billDate}= Set Variable ${bill["date"]} 61 | ${billId}= Get Invoice Id ${billDate} ${invoiceNumber} 62 | 63 | IF ${selectedBills}[${billId}] 64 | ${file}= Download ${bill}[downloadURL] 65 | ${invoiceLabel}= Get Invoice Label ${billDate} ${invoiceNumber} 66 | Move File ${file}[saveAs] %{ROBOT_ARTIFACTS}/bills/${invoiceLabel}.pdf 67 | END 68 | END 69 | 70 | Select Provider 71 | ${supportedProviders}= Get Providers 72 | 73 | Add drop-down provider ${supportedProviders} 74 | 75 | ${result}= Run dialog 76 | 77 | [Return] ${result.provider} -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # 🤖 Bill Collector 🤖 2 | 3 | ## What is it? 4 | 5 | An automation to collect bills automatically **without providing any access credentials to an untrusted party**. 6 | 7 | Multiple providers, such as `DigitalOcean` do not offer an easy way to retrieve bills. 8 | 9 | Some online services offer to automate the collection of bills, however they require to provide the credentials. 10 | 11 | --- 12 | 13 | ![Levelsio Tweet](https://beautiful-space.fra1.cdn.digitaloceanspaces.com/bill-collector/tweet.png) 14 | 15 | _[Source](https://twitter.com/levelsio/status/1325076943495188481)_ 16 | 17 | --- 18 | 19 | ## Show me! 20 | 21 | _An example with DigitalOcean, the first provider to be supported_ 22 | 23 | ### Run the automation 24 | 25 | _Note that the command-line is not the only option, see the other option below_ 26 | 27 | https://user-images.githubusercontent.com/10613140/144981442-05dec750-8108-4904-9fa8-9388d739ec13.mp4 28 | 29 | ### Login (handover) 30 | 31 | https://user-images.githubusercontent.com/10613140/144982845-b7eb40c4-a080-450b-a6ea-1566d10eabb3.mp4 32 | 33 | ### Choose the bills to collect 34 | 35 | https://user-images.githubusercontent.com/10613140/144982395-3784e654-01ee-4bc4-ab9f-d8ba5d3bfbd5.mp4 36 | 37 | ## Supported Providers 38 | 39 | - [DigitalOcean](https://www.digitalocean.com/) 40 | - [Pipedrive](https://www.pipedrive.com) 41 | - [Namecheap](https://www.namecheap.com/) 42 | - Your provider: **via Pull Request or Issue** 43 | 44 | # How to use the Bill Collector 45 | 46 | ## Via Command-Line (for technical users) 47 | 48 | _Prerequisite_: 49 | 50 | - [RCC](https://github.com/robocorp/rcc) 51 | 52 | Place yourself at the root of this folder and run the following command: 53 | 54 | ```bash 55 | rcc run 56 | ``` 57 | 58 | ## Via User-Interface (for technical and non-technical users) 59 | 60 | _Prerequisites_: 61 | 62 | - [A Robocorp account](https://robocorp.com) – necessary to download the assistant, available under the free plan without needing to provide a credit card 63 | - [Robocorp Assistant](https://robocorp.com/docs/control-room/configuring-assistants/installation) 64 | 65 | Once downloaded and installed, click on _Install a community assistant_ and paste in the URL of this repository: `https://github.com/bendersej/bill-collector`. 66 | 67 | ## Contributing 68 | 69 | ### Via Pull Request 70 | 71 | Feel free to open a new pull request with the added provider 72 | 73 | ### Via Issue 74 | 75 | If you don't have the skills or the time, feel free to open an Issue with the provider you'd like to be supported. 76 | **Note that you should provide an access (username/password) to help with the development** 77 | -------------------------------------------------------------------------------- /providers/digitalocean.robot: -------------------------------------------------------------------------------- 1 | *** Settings *** 2 | Library RPA.Browser.Playwright 3 | Library RPA.Dialogs 4 | Library Collections 5 | Library OperatingSystem 6 | Library String 7 | Resource ../helpers.robot 8 | 9 | *** Keywords *** 10 | Get Bill URL 11 | [Arguments] ${billLink} 12 | 13 | ${billIdWithCustomerPrefix}= Fetch From Right ${billLink} /billing/ 14 | ${billId}= Fetch From Left ${billIdWithCustomerPrefix} ? 15 | 16 | ${customerId}= Execute JavaScript window.currentUser.uuid 17 | 18 | [Return] https://cloud.digitalocean.com/v2/customers/${customerId}/invoices/${billId}/pdf 19 | Move To Next Page 20 | ${nextPageLink}= Get Element .next 21 | ${cssClasses}= Get Attribute ${nextPageLink} class 22 | 23 | ${hasNextPage}= Run Keyword And Return Status Should Not Contain ${cssClasses} is-disabled 24 | 25 | IF ${hasNextPage} 26 | Click .next 27 | ELSE 28 | No Operation 29 | # Fail Should not have been called 30 | END 31 | 32 | [Return] ${hasNextPage} 33 | 34 | Collect Page Bills 35 | ${bills}= Create List 36 | ${elements}= Get Elements table#history tr 37 | FOR ${element} IN @{elements} 38 | ${text}= Get Text ${element} 39 | ${isBillRow}= Run Keyword And Return Status Should Contain ${text} CSV 40 | IF ${isBillRow} 41 | ${date}= Get Text ${element} > .date 42 | ${billLink}= Get Property ${element} > .description > a:first-of-type href 43 | ${downloadURL}= Get Bill URL ${billLink} 44 | ${bill}= Create Dictionary invoiceNumber=None date=${date} downloadURL=${downloadURL} 45 | Append To List ${bills} ${bill} 46 | END 47 | END 48 | [Return] ${bills} 49 | 50 | Digital Ocean 51 | New Browser chromium headless=false 52 | New Context acceptDownloads=${TRUE} viewport={'width': 1920, 'height': 1080} 53 | New Page url=https://cloud.digitalocean.com/account/billing 54 | Show dialog title=Login to DigitalOcean then click close on_top=true 55 | Wait all dialogs 56 | Wait Until Network Is Idle 57 | ${pageCount}= Get Element Count a.page.pagination-button 58 | 59 | FOR ${pageNumber} IN RANGE ${pageCount} 60 | ${bills}= Digital Ocean.Collect Page Bills 61 | 62 | ${selectedBills} ${shouldExit} Display Picker ${bills} TRUE 63 | Download Selected Bills ${bills} ${selectedBills} 64 | 65 | Exit For Loop If "${shouldExit}" == "TRUE" 66 | 67 | Move To Next Page 68 | # TODO: Investigate why the table is not ALWAYS up to date and require this sleep 69 | Sleep 2s 70 | END 71 | 72 | Close Browser 73 | 74 | 75 | --------------------------------------------------------------------------------