├── .gitignore ├── CHANGELOG.md ├── LICENSE.txt ├── MANIFEST.in ├── README.md ├── UPDATING_PYPI.md ├── domo.png ├── examples ├── __init__.py ├── connection.ini ├── dataset.py ├── group.py ├── page.py ├── stream.py └── user.py ├── install.sh ├── pydomo ├── DomoAPIClient.py ├── Transport.py ├── __init__.py ├── accounts │ ├── AccountClient.py │ └── __init__.py ├── common │ ├── DomoObject.py │ └── __init__.py ├── datasets │ ├── DataSetClient.py │ ├── DataSetModel.py │ └── __init__.py ├── groups │ ├── GroupClient.py │ ├── GroupsModel.py │ └── __init__.py ├── pages │ ├── PageClient.py │ └── __init__.py ├── streams │ ├── StreamClient.py │ ├── StreamsModel.py │ └── __init__.py ├── users │ ├── UserClient.py │ ├── UsersModel.py │ └── __init__.py └── utilities │ ├── UtilitiesClient.py │ └── __init__.py ├── run_examples.py ├── setup.cfg └── setup.py /.gitignore: -------------------------------------------------------------------------------- 1 | *.iml 2 | *.ipr 3 | *.iws 4 | *.pyc 5 | math.csv 6 | pydomo/__pycache__ 7 | pydomo/datasets/__pycache__ 8 | pydomo/groups/__pycache__ 9 | pydomo/streams/__pycache__ 10 | pydomo/users/__pycache__ 11 | MANIFEST 12 | dist 13 | pydomo/roles/__pycache__ 14 | pydomo/tests/__pycache__ 15 | publishPypiLive.sh 16 | publishPypiTest.sh 17 | .idea/ 18 | build/ 19 | pydomo.egg-info/ 20 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # PyDomo Changelog 2 | 3 | ### v0.3.0.13 4 | May 22, 2025 5 | 6 | Bug Fixes 7 | * Only save column to `dtype_dict` when not a date column 8 | 9 | Improvements 10 | * Add `connection_file` as a parameter to Domo client init to provide ini file with connection parameters 11 | 12 | ### v0.3.0.12 13 | April 21, 2025 14 | 15 | Updates 16 | * Add scope support when creating `Domo` SDK client 17 | * Add use schema feature to `ds_get` method to convert Domo columns to pandas dtypes 18 | 19 | ### v0.3.0.11 20 | January 31, 2024 21 | 22 | Notes 23 | * Version not uploaded to pypi 24 | 25 | Updates 26 | * Modify the authentication process to conform to new API functionality. 27 | * Add ds_get_dict - this function exports a data set from Domo and returns a list of dictionaries. The list contains one dictionary for each row in Domo. 28 | 29 | ### v0.3.0.10 30 | September 20, 2023 31 | 32 | Bug Fixes 33 | * Remove an errant comma in the dataframe paging logic in ds_create and ds_update 34 | 35 | ### v0.3.0.9 36 | March 20, 2023 37 | 38 | Updates 39 | * Include the name_like parameter for the ds_list method to allow filtering on datasource name 40 | 41 | ### v0.3.0.8 42 | March 16, 2023 43 | 44 | * Fix upsert upload through the ds_create method by using a list of key_column_names rather than a string 45 | 46 | ### v0.3.0.7 47 | January 9, 2023 48 | 49 | * Use request_timeout when renewing access token 50 | 51 | ### v0.3.0.6 52 | October 21, 2022 53 | 54 | Bug Fixes 55 | * Now checks for invalid token before sending requests. If the token is invalid, the SDK now refreshes it automatically. 56 | 57 | ### v0.3.0.3 58 | February 9, 2021 59 | 60 | Bug Fixes 61 | * ds_updated and ds_create did not upload the full dataframe when doing mult-part uploads 62 | * pandas was not required in setup file 63 | 64 | ### v0.3.0.2 65 | January 26, 2021 66 | 67 | Updates 68 | * Virtual user support added to PDP policy 69 | * Pandas added to install.sh 70 | 71 | ### v0.3.0.1 72 | January 21, 2021 73 | 74 | Bug Fixes 75 | * ds_query fixed to accept a data set id and a SQL query 76 | * ds_delete now requires confirmation before deleting a data set 77 | * use prompt_before_delete=False to force delete 78 | * group functions modified to match rdomo, prior to this release these functions did not work correctly 79 | * groups_add_users now adds multiple users at a time 80 | * groups_create now takes a group name and a list of users 81 | * groups_list now pages through all lists automatically 82 | * groups_list_users now pages through all users automatically 83 | * groups_remove_users now removes a list of users 84 | * groups_delete now removes all users from the group and then deletes the group 85 | 86 | ### v0.3.0 87 | January 8, 2021 88 | 89 | The primary objective of this release is to make it easier to interact with Domo via Python and to sync functionality with the R SDK. The following changes have been made. 90 | * Methods were added to the primary object so that the sub-objects don't need to be used. 91 | * Method names sync'd with functions from the R SDK 92 | * Method names follow a specific naming convention for ease of use 93 | * Methods to download data from Domo now return a Pandas dataframe 94 | * Methods to upload data to Domo now take a Pandas dataframe as an input 95 | * Uploading data is now easier as interactions with the streams API are abstracted 96 | * All methods prior to these updates are unchanged, existing code should continue to work w/o issue 97 | 98 | ### v0.2.3 99 | - Apr 18, 2019 100 | - Added GZIP support for Streams (Thank you @ldacey!) 101 | - Added file support for Streams (Thank you @jonemi!) 102 | - Added limit and offset support for listing Users in a Group 103 | 104 | ### v0.2.2.1 105 | - Jan 11, 2018 106 | - Circumventing issue in setup.py preventing installing via pip 107 | 108 | ### v0.2.2 109 | - Jan 11, 2018 110 | - Bug Fixes: 111 | - Fixed unicode error when uploading unicode data via stream client 112 | - Dataset client referenced log instead of logger 113 | 114 | ### v0.2.1 115 | - Oct 16, 2017 116 | - Improved csv downloading to disk, avoiding a potential python memory error 117 | 118 | ### v0.2.0 119 | - Aug 17, 2017 120 | - Added Pages: 121 | - List, create, update, and delete pages 122 | - List, create, update, and delete collections 123 | - Improvements: 124 | - All endpoints now accept and return dictionaries. This will 125 | likely break existing code, but it allows for returned objects 126 | to be accepted as parameters. 127 | - Removed dependency on jsonpickle library 128 | - Eliminated an extra API call that was previously being used to 129 | check if access token is valid 130 | - Enable appending to datasets 131 | - Bug Fixes: 132 | - Fixed unicode error when uploading/downloading unicode data 133 | 134 | ### v0.1.3 135 | - Jul 6, 2017 136 | - Improvements: 137 | - Better error descriptions, http request/response dumps on exception (requires requests_toolbelt) 138 | - Bug Fixes: 139 | - Fixed Stream search (aldifahrezi) 140 | - Fixed DataSet export authentication renewal 141 | 142 | ### v0.1.2 143 | - May 1, 2017 144 | - Added functionality to the Group Client: 145 | - Add a User to a Group 146 | - List Users in a Group 147 | - Remove a User from a Group 148 | - Added the "api host" as a parameter to the SDK initialization 149 | 150 | ### v0.1.1 151 | - April 26, 2017 152 | - Minor improvements for PyPI publishing 153 | - Published the SDK to PyPI for easy Pip installation: `pip3 install pydomo` 154 | 155 | ### v0.1.0 156 | - April 21, 2017 157 | - Initial release 158 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2017 Domo 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. -------------------------------------------------------------------------------- /MANIFEST.in: -------------------------------------------------------------------------------- 1 | # Include the license file 2 | include LICENSE.txt 3 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Python3 - Domo API SDK (pydomo) 2 | [![License](https://img.shields.io/badge/license-MIT-blue.svg?style=flat)](http://www.opensource.org/licenses/MIT) 3 | 4 | Current Release: 0.3.0 5 | 6 | ### Notice - Python 3 Compatibility 7 | 8 | * PyDomo is written for Python3, and is not compatible with Python2 9 | * Execute scripts via 'python3', and updates via 'pip3' 10 | 11 | ### About 12 | 13 | * The Domo API SDK is the simplest way to automate your Domo instance 14 | * The SDK streamlines the API programming experience, allowing you to significantly reduce your written code 15 | * This SDK was written for Python3, and is not compatible with Python2 16 | * PyDomo has been published to [PyPI](https://pypi.org/project/pydomo/). The SDK can be easily installed via `pip3 install pydomo`, and can be updated via `pip3 install pydomo --upgrade` 17 | 18 | ### Features: 19 | - DataSet and Personalized Data Policy (PDP) Management 20 | - Use DataSets for fairly static data sources that only require occasional updates via data replacement 21 | - This SDK automates the use of Domo Streams so that uploads are always as fast as possible 22 | - Add Personalized Data Policies (PDPs) to DataSets (hide sensitive data from groups of users) 23 | - Docs: https://developer.domo.com/docs/domo-apis/data 24 | - User Management 25 | - Create, update, and remove users 26 | - Major use case: LDAP/Active Directory synchronization 27 | - Docs: https://developer.domo.com/docs/domo-apis/users 28 | - Group Management 29 | - Create, update, and remove groups of users 30 | - Docs: https://developer.domo.com/docs/domo-apis/group-apis 31 | - Page Management 32 | - Create, update, and delete pages 33 | - Docs: https://developer.domo.com/docs/page-api-reference/page 34 | 35 | ### Setup 36 | * Install Python3: https://www.python.org/downloads/ 37 | * Linux: 'apt-get install python3' 38 | * MacOS: 'brew install python3' 39 | * Windows: direct download, or use Bash on Windows 10 40 | * Install PyDomo and its dependencies via `pip3 install pydomo` 41 | 42 | ### Updates 43 | * Update your PyDomo package via `pip3 install pydomo --upgrade` 44 | * View the [changelog](CHANGELOG.md) 45 | 46 | ### Usage 47 | Below are examples of how to use the SDK to perform a few common tasks. To run similar code on your system, do the following. 48 | * Create an API Client on the [Domo Developer Portal](https://developer.domo.com/) 49 | * Use your API Client id/secret to instantiate pydomo 'Domo()' 50 | * Multiple API Clients can be used by instantiating multiple 'Domo()' clients 51 | * Authentication with the Domo API is handled automatically by the SDK 52 | * If you encounter a 'Not Allowed' error, this is a permissions issue. Please speak with your Domo Administrator. 53 | ```python 54 | from pydomo import Domo 55 | 56 | domo = Domo('client-id','secret',api_host='api.domo.com') 57 | 58 | # Download a data set from Domo 59 | car_data = domo.ds_get('2f09a073-54a4-4269-8c62-b776e67d59f0') 60 | 61 | # Create a summary data set, taking the mean of dollars by make and model. 62 | car_summary = car_data.groupby(['make','model']).agg({'dollars':'mean'}).reset_index() 63 | 64 | 65 | # Create a new data set in Domo with the result, the return value is the data set id of the new data set. 66 | car_ds = domo.ds_create(car_summary,'Python | Car Summary Data Set','Python | Generated during demo') 67 | 68 | # Modify summary and then upload to the data set we already created. The SDK will update the data set schema automatically. 69 | car_summary2 = car_data.groupby(['make','model'],as_index=False).agg({'dollars':'mean','email':'count'}).reset_index() 70 | car_update = domo.ds_update(car_ds,car_summary2) 71 | 72 | 73 | # Create PDP Policy 74 | from pydomo.datasets import Policy, PolicyFilter, FilterOperator, PolicyType, Sorting 75 | 76 | # Create policy filters 77 | pdp_filter = PolicyFilter() 78 | pdp_filter.column = 'make' # The DataSet column to filter on 79 | pdp_filter.operator = FilterOperator.EQUALS 80 | pdp_filter.values = ['Honda'] # The DataSet row value to filter on 81 | 82 | pdp_request = Policy() 83 | pdp_request.name = 'Python | US East' 84 | pdp_request.filters = [pdp_filter] 85 | pdp_request.type = PolicyType.USER 86 | pdp_request.users = [] 87 | pdp_request.groups = [1631291223] 88 | 89 | domo.pdp_create(car_ds,pdp_request) 90 | 91 | 92 | # Interact with groups 93 | all_groups = domo.groups_list() # List all groups 94 | all_users = domo.users_list() # List all users 95 | 96 | # List all users in US South Division 97 | domo.groups_list_users(328554991) 98 | 99 | added_users = domo.groups_add_users(328554991,2063934980) 100 | domo.groups_list_users(328554991) 101 | ``` 102 | 103 | ### Available Functions 104 | The functions in this package match most parts of the API documented at [developer.domo.com](https://developer.domo.com/) and follow a specific convention. Each set of functions is preceeded by the portion of the API it operates on. The following lists all the sets of functions available in this package. For further help, refer to the help function in Python. 105 | * **Data sets** - This set of functions is designed to transfer data in and out of Domo. 106 | * **ds_get** - downloads data from Domo 107 | * **ds_create** - creates a new data set 108 | * **ds_update** - updates an existing data set, only data sets created by the API can be updated 109 | * **ds_meta** - downloads meta data regarding a single data set 110 | * **ds_list** - downloads a list of data sets in your Domo instance 111 | * **ds_delete** - deletes a data set (be careful) 112 | * **ds_query** - allows you to send a query to a data set, Domo will evaluate that query and sends the results back as a list or a tibble 113 | * **Groups** - This set of functions modifies and creates groups. 114 | * **groups_add_users** - adds users to an existing group 115 | * **groups_create** - create a group 116 | * **groups_delete** - delete an existing group 117 | * **groups_list** - list all groups 118 | * **groups_remove_users** - remove users from a group 119 | * **groups_list_users** - list users in a group 120 | * **Pages** - functions related to managing Domo pages 121 | * **page_update** - update a page 122 | * **page_list** - list all pages 123 | * **page_get_collections** - list all collections on a page 124 | * **page_get** - get information regarding a page 125 | * **page_create** - create a page 126 | * **PDP** - functions to manage PDP 127 | * **pdp_update** - update an existing PDP policy 128 | * **pdp_list** - list all PDP policies 129 | * **pdp_enable** - toggle PDP on and off 130 | * **pdp_delete** - delete a PDP policy 131 | * **pdp_create** - create a PDP policy 132 | * **Users** - functions to manage users 133 | * **users_delete** - delete a user 134 | * **users_update** - update a user 135 | * **users_list** - list all users 136 | * **users_get** - get a single user record 137 | * **users_add** - create a user (or users) 138 | -------------------------------------------------------------------------------- /UPDATING_PYPI.md: -------------------------------------------------------------------------------- 1 | # Updating Pypi Instructions 2 | 3 | When you make changes to this [GitHub repo](https://github.com/domoinc/domo-python-sdk), you also need to update the [Pypi repository](https://pypi.org/project/pydomo/) with your latest changes. 4 | 5 | ## Setup 6 | 7 | There are some Python packages you will need to install (in a Terminal): 8 | 9 | ``` 10 | python3 -m pip install --upgrade build 11 | python3 -m pip install --upgrade twine 12 | ``` 13 | 14 | You will also need a pypi account and access to the package. Domo IT/Ops can provide this for you. Ultimately, you need an API key from Pypi that you will use to upload your latest changes to Pypi. It should begin with `pypi-`. 15 | 16 | ## Steps to update Pypi 17 | 18 | 1. Increment the version inside `setup.py` 19 | 1. Double-check this version matches what's already in Pypi 20 | 2. Run `python3 -m build` from the `domo-python-sdk` directory 21 | 1. This should create a directory called `dist` and add two files: `pydomo-{version}-py3-none-any.whl` and `pydomo-{version}.tar.gz`. 22 | 3. Run `python3 -m twine upload --repository pypi dist/*` 23 | 1. This will ask for your API key. Paste it in. 24 | 2. You should see that your new version was uploaded successfully. 25 | 4. Check Pypi to verify your version was uploaded correctly. 26 | 5. Update `CHANGELOG.md` to include your latest changes. 27 | 6. Delete the `dist` directory (`rm -rf dist`) 28 | 1. First, the `dist` directory is ignored in git (see `.gitignore`). It does not need to be saved once it's uploaded. 29 | 2. Second, re-running the `build` command does not remove existing files from `dist`, it only adds new ones. So if you haven't deleted the `dist` directory, and you upgrade the version and `build` again, the `dist` directory will hold both the old version and the new version. When running `twine upload` you don't want to have both versions in `dist`, hence why you should delete it now. 30 | 31 | ## Sources/More Information 32 | 33 | See [Python's docs for this](https://packaging.python.org/en/latest/tutorials/packaging-projects/). 34 | 35 | -------------------------------------------------------------------------------- /domo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/domoinc/domo-python-sdk/ec735429aecb252fddc2e15192d89d3c7c6ea36f/domo.png -------------------------------------------------------------------------------- /examples/__init__.py: -------------------------------------------------------------------------------- 1 | from .dataset import datasets 2 | from .group import groups 3 | from .page import pages 4 | from .stream import streams 5 | from .user import users 6 | -------------------------------------------------------------------------------- /examples/connection.ini: -------------------------------------------------------------------------------- 1 | [client_auth] 2 | client = replace-me-with-client-id 3 | secret = replace-me-with-client-secret 4 | # optional 5 | api_host = api.domo.com 6 | use_https = True 7 | 8 | [client_create_auth] 9 | # sid = "NONE" 10 | -------------------------------------------------------------------------------- /examples/dataset.py: -------------------------------------------------------------------------------- 1 | from pydomo.datasets import DataSetRequest, Schema, Column, ColumnType, Policy 2 | from pydomo.datasets import PolicyFilter, FilterOperator, PolicyType, Sorting 3 | 4 | 5 | def datasets(domo): 6 | '''DataSets are useful for data sources that only require 7 | occasional replacement. See the docs at 8 | https://developer.domo.com/docs/data-apis/data 9 | ''' 10 | domo.logger.info("\n**** Domo API - DataSet Examples ****\n") 11 | datasets = domo.datasets 12 | 13 | # Define a DataSet Schema 14 | dsr = DataSetRequest() 15 | dsr.name = 'Leonhard Euler Party' 16 | dsr.description = 'Mathematician Guest List' 17 | dsr.schema = Schema([Column(ColumnType.STRING, 'Friend')]) 18 | 19 | # Create a DataSet with the given Schema 20 | dataset = datasets.create(dsr) 21 | domo.logger.info("Created DataSet " + dataset['id']) 22 | 23 | # Get a DataSets's metadata 24 | retrieved_dataset = datasets.get(dataset['id']) 25 | domo.logger.info("Retrieved DataSet " + retrieved_dataset['id']) 26 | 27 | # List DataSets 28 | dataset_list = list(datasets.list(sort=Sorting.NAME)) 29 | domo.logger.info("Retrieved a list containing {} DataSet(s)".format( 30 | len(dataset_list))) 31 | 32 | # Update a DataSets's metadata 33 | update = DataSetRequest() 34 | update.name = 'Leonhard Euler Party - Update' 35 | update.description = 'Mathematician Guest List - Update' 36 | update.schema = Schema([Column(ColumnType.STRING, 'Friend'), 37 | Column(ColumnType.STRING, 'Attending')]) 38 | update.owner = { 39 | "id": 28, 40 | "name": "Carl Friedrich Gauss" 41 | } 42 | 43 | updated_dataset = datasets.update(dataset['id'], update) 44 | domo.logger.info("Updated DataSet {}: {}".format(updated_dataset['id'], 45 | updated_dataset['name'])) 46 | 47 | # Import Data from a string 48 | csv_upload = '"Pythagoras","FALSE"\n"Alan Turing","TRUE"\n' \ 49 | '"George Boole","TRUE"' 50 | datasets.data_import(dataset['id'], csv_upload) 51 | domo.logger.info("Uploaded data to DataSet " + dataset['id']) 52 | 53 | # Export Data to a string 54 | include_csv_header = True 55 | csv_download = datasets.data_export(dataset['id'], include_csv_header) 56 | domo.logger.info("Downloaded data from DataSet {}:\n{}".format( 57 | dataset['id'], csv_download)) 58 | 59 | # Export Data to a file (also returns a readable/writable file object) 60 | csv_file_path = './math.csv' 61 | include_csv_header = True 62 | csv_file = datasets.data_export_to_file(dataset['id'], csv_file_path, 63 | include_csv_header) 64 | csv_file.close() 65 | domo.logger.info("Downloaded data as a file from DataSet {}".format( 66 | dataset['id'])) 67 | 68 | # Import Data from a file 69 | csv_file_path = './math.csv' 70 | datasets.data_import_from_file(dataset['id'], csv_file_path) 71 | domo.logger.info("Uploaded data from a file to DataSet {}".format( 72 | dataset['id'])) 73 | 74 | # Personalized Data Policies (PDPs) 75 | 76 | # Build a Policy Filter (hide sensitive columns/values from users) 77 | pdp_filter = PolicyFilter() 78 | pdp_filter.column = 'Attending' # The DataSet column to filter on 79 | pdp_filter.operator = FilterOperator.EQUALS 80 | pdp_filter.values = ['TRUE'] # The DataSet row value to filter on 81 | 82 | # Build the Personalized Data Policy (PDP) 83 | pdp_request = Policy() 84 | pdp_request.name = 'Only show friends attending the party' 85 | # A single PDP can contain multiple filters 86 | pdp_request.filters = [pdp_filter] 87 | pdp_request.type = PolicyType.USER 88 | # The affected user ids (restricted access by filter) 89 | pdp_request.users = [998, 999] 90 | # The affected group ids (restricted access by filter) 91 | pdp_request.groups = [99, 100] 92 | 93 | # Create the PDP 94 | pdp = datasets.create_pdp(dataset['id'], pdp_request) 95 | domo.logger.info("Created a Personalized Data Policy (PDP): " 96 | "{}, id: {}".format(pdp['name'], pdp['id'])) 97 | 98 | # Get a Personalized Data Policy (PDP) 99 | pdp = datasets.get_pdp(dataset['id'], pdp['id']) 100 | domo.logger.info("Retrieved a Personalized Data Policy (PDP):" 101 | " {}, id: {}".format(pdp['name'], pdp['id'])) 102 | 103 | # List Personalized Data Policies (PDP) 104 | pdp_list = datasets.list_pdps(dataset['id']) 105 | domo.logger.info("Retrieved a list containing {} PDP(s) for DataSet {}" 106 | .format(len(pdp_list), dataset['id'])) 107 | 108 | # Update a Personalized Data Policy (PDP) 109 | # Negate the previous filter (logical NOT). Note that in this case you 110 | # must treat the object as a dictionary - `pdp_filter.not` is invalid 111 | # syntax. 112 | pdp_filter['not'] = True 113 | pdp_request.name = 'Only show friends not attending the party' 114 | # A single PDP can contain multiple filters 115 | pdp_request.filters = [pdp_filter] 116 | pdp = datasets.update_pdp(dataset['id'], pdp['id'], pdp_request) 117 | domo.logger.info("Updated a Personalized Data Policy (PDP): {}, id: {}" 118 | .format(pdp['name'], pdp['id'])) 119 | 120 | # Delete a Personalized Data Policy (PDP) 121 | datasets.delete_pdp(dataset['id'], pdp['id']) 122 | domo.logger.info("Deleted a Personalized Data Policy (PDP): {}, id: {}" 123 | .format(pdp['name'], pdp['id'])) 124 | 125 | # Delete a DataSet 126 | datasets.delete(dataset['id']) 127 | domo.logger.info("Deleted DataSet {}".format(dataset['id'])) 128 | -------------------------------------------------------------------------------- /examples/group.py: -------------------------------------------------------------------------------- 1 | from random import randint 2 | from pydomo.groups import CreateGroupRequest 3 | 4 | 5 | def groups(domo): 6 | '''Group Docs: 7 | https://developer.domo.com/docs/domo-apis/group-apis 8 | ''' 9 | domo.logger.info("\n**** Domo API - Group Examples ****\n") 10 | groups = domo.groups 11 | 12 | # Build a Group 13 | group_request = CreateGroupRequest() 14 | group_request.name = 'Groupy Group {}'.format(randint(0, 10000)) 15 | group_request.active = True 16 | group_request.default = False 17 | 18 | # Create a Group 19 | group = groups.create(group_request) 20 | domo.logger.info("Created Group '{}'".format(group['name'])) 21 | 22 | # Get a Group 23 | group = groups.get(group['id']) 24 | domo.logger.info("Retrieved Group '{}'".format(group['name'])) 25 | 26 | # List Groups 27 | group_list = groups.list(10, 0) 28 | domo.logger.info("Retrieved a list containing {} Group(s)".format( 29 | len(group_list))) 30 | 31 | # Update a Group 32 | group_update = CreateGroupRequest() 33 | group_update.name = 'Groupy Group {}'.format(randint(0, 10000)) 34 | group_update.active = False 35 | group_update.default = False 36 | group = groups.update(group['id'], group_update) 37 | domo.logger.info("Updated Group '{}'".format(group['name'])) 38 | 39 | # Add a User to a Group 40 | user_list = domo.users.list(10, 0) 41 | user = user_list[0] 42 | groups.add_user(group['id'], user['id']) 43 | domo.logger.info("Added User {} to Group {}".format(user['id'], 44 | group['id'])) 45 | 46 | # List Users in a Group 47 | user_list = groups.list_users(group['id']) 48 | domo.logger.info("Retrieved a User list from a Group containing {} User(s)" 49 | .format(len(user_list))) 50 | 51 | # Remove a User from a Group 52 | groups.remove_user(group['id'], user['id']) 53 | domo.logger.info("Removed User {} from Group {}".format(user['id'], 54 | group['id'])) 55 | 56 | # Delete a Group 57 | groups.delete(group['id']) 58 | domo.logger.info("Deleted group '{}'".format(group['name'])) 59 | -------------------------------------------------------------------------------- /examples/page.py: -------------------------------------------------------------------------------- 1 | 2 | 3 | def pages(domo): 4 | '''Page Docs: https://developer.domo.com/docs/domo-apis/pages 5 | ''' 6 | domo.logger.info("\n**** Domo API - Page Examples ****\n") 7 | 8 | # Create a page 9 | page = domo.pages.create('New Page') 10 | domo.logger.info("Created Page {}".format(page['id'])) 11 | 12 | # Create a subpage 13 | subpage = domo.pages.create('Sub Page', parentId=page['id']) 14 | domo.logger.info("Created Subpage {}".format(subpage['id'])) 15 | 16 | # Update the page using returned page 17 | page['name'] = 'Updated Page' 18 | domo.pages.update(**page) 19 | domo.logger.info("Renamed Page {}".format(page['id'])) 20 | 21 | # Turn subpage into to top-level page using keyword argument 22 | domo.pages.update(subpage['id'], parentId=None) 23 | domo.logger.info("Moved Page to top level {}".format(subpage['id'])) 24 | 25 | # Get the page 26 | page = domo.pages.get(page['id']) 27 | 28 | # List pages 29 | page_list = list(domo.pages.list()) 30 | domo.logger.info("Retrieved a list of {} top-level page(s)".format( 31 | len(page_list))) 32 | 33 | # Create a few collections 34 | collections = [ 35 | domo.pages.create_collection(page['id'], 'First Collection'), 36 | domo.pages.create_collection(page['id'], 'Second Collection'), 37 | ] 38 | domo.logger.info("Created two collections on page {}".format(page['id'])) 39 | 40 | # Get collections 41 | collection_list = domo.pages.get_collections(page['id']) 42 | domo.logger.info("Retrieved a list of {} collections".format( 43 | len(collection_list))) 44 | 45 | # Update collection 46 | collections[1]['title'] = 'Last Collection' 47 | domo.pages.update_collection(page['id'], **collections[1]) 48 | domo.logger.info("Updated collection {}: {}".format(collections[1]['id'], 49 | collections[1]['title'])) 50 | 51 | # Clean up 52 | for collection in collections: 53 | domo.pages.delete_collection(page['id'], collection['id']) 54 | domo.pages.delete(page['id']) 55 | domo.pages.delete(subpage['id']) 56 | domo.logger.info("Deleted collections and pages") 57 | -------------------------------------------------------------------------------- /examples/stream.py: -------------------------------------------------------------------------------- 1 | from pydomo.datasets import DataSetRequest, Schema, Column, ColumnType 2 | from pydomo.streams import UpdateMethod, CreateStreamRequest 3 | 4 | 5 | def streams(domo): 6 | '''Streams are useful for uploading massive data sources in 7 | chunks, in parallel. They are also useful with data sources that 8 | are constantly changing/growing. 9 | Streams Docs: https://developer.domo.com/docs/data-apis/data 10 | ''' 11 | domo.logger.info("\n**** Domo API - Stream Examples ****\n") 12 | streams = domo.streams 13 | 14 | # Define a DataSet Schema to populate the Stream Request 15 | dsr = DataSetRequest() 16 | dsr.name = 'Leonhard Euler Party' 17 | dsr.description = 'Mathematician Guest List' 18 | dsr.schema = Schema([Column(ColumnType.STRING, 'Friend'), 19 | Column(ColumnType.STRING, 'Attending')]) 20 | 21 | # Build a Stream Request 22 | stream_request = CreateStreamRequest(dsr, UpdateMethod.APPEND) 23 | 24 | # Create a Stream w/DataSet 25 | stream = streams.create(stream_request) 26 | domo.logger.info("Created Stream {} containing the new DataSet {}" 27 | .format(stream['id'], stream['dataSet']['id'])) 28 | 29 | # Get a Stream's metadata 30 | retrieved_stream = streams.get(stream['id']) 31 | domo.logger.info("Retrieved Stream {} containing DataSet {}".format( 32 | retrieved_stream['id'], retrieved_stream['dataSet']['id'])) 33 | 34 | # List Streams 35 | limit = 1000 36 | offset = 0 37 | stream_list = streams.list(limit, offset) 38 | domo.logger.info("Retrieved a list containing {} Stream(s)".format( 39 | len(stream_list))) 40 | 41 | # Update a Stream's metadata 42 | stream_update = CreateStreamRequest(dsr, UpdateMethod.REPLACE) 43 | updated_stream = streams.update(retrieved_stream['id'], stream_update) 44 | domo.logger.info("Updated Stream {} to update method: {}".format( 45 | updated_stream['id'], updated_stream['updateMethod'])) 46 | 47 | # Search for Streams 48 | stream_property = 'dataSource.name:' + dsr.name 49 | searched_streams = streams.search(stream_property) 50 | domo.logger.info("Stream search: there are {} Stream(s) with the DataSet " 51 | "title: {}".format(len(searched_streams), dsr.name)) 52 | 53 | # Create an Execution (Begin an upload process) 54 | execution = streams.create_execution(stream['id']) 55 | domo.logger.info("Created Execution {} for Stream {}".format( 56 | execution['id'], stream['id'])) 57 | 58 | # Get an Execution 59 | retrieved_execution = streams.get_execution(stream['id'], 60 | execution['id']) 61 | domo.logger.info("Retrieved Execution with id: {}".format( 62 | retrieved_execution['id'])) 63 | 64 | # List Executions 65 | execution_list = streams.list_executions(stream['id'], limit, offset) 66 | domo.logger.info("Retrieved a list containing {} Execution(s)".format( 67 | len(execution_list))) 68 | 69 | # Upload Data: Multiple Parts can be uploaded in parallel 70 | part = 1 71 | csv = '"Pythagoras","FALSE"\n"Alan Turing","TRUE"' 72 | execution = streams.upload_part(stream['id'], execution['id'], 73 | part, csv) 74 | 75 | part = 2 76 | csv = '"George Boole","TRUE"' 77 | execution = streams.upload_part(stream['id'], execution['id'], 78 | part, csv) 79 | 80 | # Commit the execution (End an upload process) 81 | # Executions/commits are NOT atomic 82 | committed_execution = streams.commit_execution(stream['id'], 83 | execution['id']) 84 | domo.logger.info("Committed Execution {} on Stream {}".format( 85 | committed_execution['id'], stream['id'])) 86 | 87 | # Abort a specific Execution 88 | execution = streams.create_execution(stream['id']) 89 | aborted_execution = streams.abort_execution(stream['id'], 90 | execution['id']) 91 | domo.logger.info("Aborted Execution {} on Stream {}".format( 92 | aborted_execution['id'], stream['id'])) 93 | 94 | # Abort any Execution on a given Stream 95 | streams.create_execution(stream['id']) 96 | streams.abort_current_execution(stream['id']) 97 | domo.logger.info("Aborted Executions on Stream {}".format( 98 | stream['id'])) 99 | 100 | # Delete a Stream 101 | streams.delete(stream['id']) 102 | domo.logger.info("Deleted Stream {}; the associated DataSet must be " 103 | "deleted separately".format(stream['id'])) 104 | 105 | # Delete the associated DataSet 106 | domo.datasets.delete(stream['dataSet']['id']) 107 | -------------------------------------------------------------------------------- /examples/user.py: -------------------------------------------------------------------------------- 1 | from random import randint 2 | 3 | from pydomo.users import CreateUserRequest 4 | 5 | 6 | def users(domo): 7 | '''User Docs: https://developer.domo.com/docs/domo-apis/users 8 | ''' 9 | domo.logger.info("\n**** Domo API - User Examples ****\n") 10 | 11 | # Build a User 12 | user_request = CreateUserRequest() 13 | user_request.name = 'Leonhard Euler' 14 | user_request.email = 'leonhard.euler{}@domo.com'.format(randint(0, 10000)) 15 | user_request.role = 'Privileged' 16 | send_invite = False 17 | 18 | # Create a User 19 | user = domo.users.create(user_request, send_invite) 20 | domo.logger.info("Created User '{}'".format(user['name'])) 21 | 22 | # Get a User 23 | user = domo.users.get(user['id']) 24 | domo.logger.info("Retrieved User '" + user['name'] + "'") 25 | 26 | # List Users 27 | user_list = domo.users.list(10, 0) 28 | domo.logger.info("Retrieved a list containing {} User(s)".format( 29 | len(user_list))) 30 | 31 | # Update a User 32 | user_update = CreateUserRequest() 33 | user_update.name = 'Leo Euler' 34 | user_update.email = 'leo.euler{}@domo.com'.format(randint(0, 10000)) 35 | user_update.role = 'Privileged' 36 | user = domo.users.update(user['id'], user_update) 37 | domo.logger.info("Updated User '{}': {}".format(user['name'], 38 | user['email'])) 39 | 40 | # Delete a User 41 | domo.users.delete(user['id']) 42 | domo.logger.info("Deleted User '{}'".format(user['name'])) 43 | -------------------------------------------------------------------------------- /install.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | pip3 install requests requests_toolbelt jsonpickle pandas 3 | -------------------------------------------------------------------------------- /pydomo/DomoAPIClient.py: -------------------------------------------------------------------------------- 1 | import json 2 | import requests 3 | 4 | from pydomo.Transport import HTTPMethod 5 | 6 | class DomoAPIClient(object): 7 | """DomoAPIClient 8 | Each API client class inherits from this superclass 9 | """ 10 | 11 | def __init__(self, transport, logger): 12 | self.transport = transport 13 | self.logger = logger 14 | 15 | def _create(self, url, request, params, obj_desc): 16 | response = self.transport.post(url=url, params=params, body=request) 17 | if response.status_code in (requests.codes.CREATED, requests.codes.OK): 18 | obj = response.json() 19 | self.print_json("Created", obj) 20 | return obj 21 | else: 22 | self.logger.debug("Error creating " + obj_desc + ": " 23 | + self.transport.dump_response(response)) 24 | raise Exception("Error creating " + obj_desc + ": " + response.text) 25 | 26 | def _get(self, url, obj_desc): 27 | response = self.transport.get(url=url, params={}) 28 | if response.status_code == requests.codes.OK: 29 | return response.json() 30 | else: 31 | self.logger.debug("Error retrieving " + obj_desc + ": " 32 | + self.transport.dump_response(response)) 33 | raise Exception("Error retrieving " + obj_desc + ": " 34 | + response.text) 35 | 36 | def _update(self, url, method, success_code, obj_update, obj_desc): 37 | if method == HTTPMethod.PUT: 38 | response = self.transport.put(url, obj_update) 39 | elif method == HTTPMethod.PATCH: 40 | response = self.transport.patch(url, obj_update) 41 | if response.status_code == success_code: 42 | if str(response.text) == '': 43 | return 44 | else: 45 | obj = response.json() 46 | self.print_json("Updated", obj) 47 | return obj 48 | else: 49 | self.logger.debug("Error updating " + obj_desc + ": " 50 | + self.transport.dump_response(response)) 51 | raise Exception("Error updating " + obj_desc + ": " 52 | + response.text) 53 | 54 | def _list(self, url, params, obj_desc): 55 | response = self.transport.get(url=url, params=params) 56 | if response.status_code == requests.codes.OK: 57 | return response.json() 58 | else: 59 | self.logger.debug(obj_desc + " Error: " 60 | + self.transport.dump_response(response)) 61 | raise Exception(obj_desc + " Error: " + response.text) 62 | 63 | def _delete(self, url, obj_desc): 64 | response = self.transport.delete(url=url) 65 | if response.status_code == requests.codes.NO_CONTENT: 66 | return 67 | else: 68 | self.logger.debug("Error deleting " + obj_desc + ": " 69 | + self.transport.dump_response(response)) 70 | raise Exception("Error deleting " + obj_desc + ": " 71 | + response.text) 72 | 73 | def _upload_csv(self, url, success_code, csv, obj_desc): 74 | response = self.transport.put_csv(url=url, body=csv) 75 | if response.status_code == success_code: 76 | if str(response.text) == '': 77 | return 78 | else: 79 | return response.json() 80 | else: 81 | self.logger.debug("Error uploading " + obj_desc + ": " + self.transport.dump_response(response)) 82 | raise Exception("Error uploading " + obj_desc + ": " 83 | + response.text) 84 | 85 | def _upload_gzip(self, url, success_code, csv, obj_desc): 86 | response = self.transport.put_gzip(url=url, body=csv) 87 | if response.status_code == success_code: 88 | if str(response.text) == '': 89 | return 90 | else: 91 | return response.json() 92 | else: 93 | self.logger.debug("Error uploading " + obj_desc + ": " + self.transport.dump_response(response)) 94 | raise Exception("Error uploading " + obj_desc + ": " 95 | + response.text) 96 | 97 | def _download_csv(self, url, include_csv_header): 98 | params = { 99 | 'includeHeader': str(include_csv_header) 100 | } 101 | return self.transport.get_csv(url=url, params=params) 102 | 103 | def _validate_params(self, params, accepted_keys): 104 | bad_keys = list(set(params.keys()).difference(accepted_keys)) 105 | if bad_keys: 106 | raise TypeError('Unexpected keyword arguments: {}'.format(bad_keys)) 107 | 108 | def _base(self, obj_id): 109 | return self.urlBase + str(obj_id) 110 | 111 | def print_json(self, message, json_obj): 112 | self.logger.info(message + ": " + json.dumps(json_obj, indent=4)) 113 | -------------------------------------------------------------------------------- /pydomo/Transport.py: -------------------------------------------------------------------------------- 1 | import requests 2 | import json 3 | import base64 4 | from collections import namedtuple 5 | from requests.auth import HTTPBasicAuth 6 | from requests_toolbelt.utils import dump 7 | from datetime import datetime, timezone 8 | 9 | 10 | class DomoAPITransport: 11 | """Essentially a wrapper around the 'requests' library to make 12 | it easier to interact with the Domo API. 13 | 14 | OAuth2 authentication is handled automatically, as well as the 15 | serialization and deserialization of objects. 16 | """ 17 | 18 | def __init__(self, client_id, client_secret, api_host, use_https, logger, request_timeout, scope): 19 | self.apiHost = self._build_apihost(api_host, use_https) 20 | self.clientId = client_id 21 | self.clientSecret = client_secret 22 | self.logger = logger 23 | self.request_timeout = request_timeout 24 | self.scope = scope 25 | self._renew_access_token() 26 | 27 | @staticmethod 28 | def _build_apihost(host, use_https): 29 | if use_https: 30 | host = 'https://' + host 31 | else: 32 | host = 'http://' + host 33 | return host 34 | 35 | def get(self, url, params): 36 | headers = self._headers_default_receive_json() 37 | return self.request(url, HTTPMethod.GET, headers, params) 38 | 39 | def get_csv(self, url, params): 40 | headers = self._headers_receive_csv() 41 | return self.request(url, HTTPMethod.GET, headers, params) 42 | 43 | def post(self, url, body, params): 44 | headers = self._headers_send_json() 45 | return self.request(url, HTTPMethod.POST, headers, params, 46 | self._obj_to_json(body)) 47 | 48 | def put(self, url, body): 49 | headers = self._headers_send_json() 50 | return self.request(url, HTTPMethod.PUT, headers, {}, 51 | self._obj_to_json(body)) 52 | 53 | def put_csv(self, url, body): 54 | headers = self._headers_send_csv() 55 | return self.request(url, HTTPMethod.PUT, headers, {}, body) 56 | 57 | def put_gzip(self, url, body): 58 | headers = self._headers_send_gzip() 59 | return self.request(url, HTTPMethod.PUT, headers, {}, body) 60 | 61 | def patch(self, url, body): 62 | headers = self._headers_send_json() 63 | return self.request(url, HTTPMethod.PATCH, headers, {}, 64 | self._obj_to_json(body)) 65 | 66 | def delete(self, url): 67 | headers = self._headers_default_receive_json() 68 | return self.request(url, HTTPMethod.DELETE, headers) 69 | 70 | def request(self, url, method, headers, params=None, body=None): 71 | url = self.apiHost + url 72 | self.logger.debug('{} {} {}'.format(method, url, body)) 73 | request_args = {'method': method, 'url': url, 'headers': headers, 74 | 'params': params, 'data': body, 'stream': True} 75 | if self.request_timeout: 76 | request_args['timeout'] = self.request_timeout 77 | 78 | # Expiration date should be in UTC 79 | if datetime.now(timezone.utc).timestamp() > self.token_expiration: 80 | self.logger.debug("Access token is expired") 81 | self._renew_access_token() 82 | headers['Authorization'] = 'bearer ' + self.access_token 83 | 84 | return requests.request(**request_args) 85 | 86 | def _renew_access_token(self): 87 | self.logger.debug("Renewing Access Token") 88 | # scope == None means use all scopes from client 89 | scope = ' '.join(self.scope) if self.scope else None 90 | 91 | request_args = { 92 | 'method': HTTPMethod.POST, 93 | 'url': self.apiHost + '/oauth/token', 94 | 'data': {'grant_type': 'client_credentials', 'scope': scope}, 95 | 'auth': HTTPBasicAuth(self.clientId, self.clientSecret) 96 | } 97 | if self.request_timeout: 98 | request_args['timeout'] = self.request_timeout 99 | 100 | response = requests.request(**request_args) 101 | if response.status_code == requests.codes.OK: 102 | self.access_token = response.json()['access_token'] 103 | self.token_expiration = self._extract_expiration(self.access_token) 104 | else: 105 | self.logger.debug('Error retrieving access token: ' + self.dump_response(response)) 106 | raise Exception("Error retrieving a Domo API Access Token: " + response.text) 107 | 108 | def _extract_expiration(self, access_token): 109 | expiration_date = 0 110 | try: 111 | decoded_payload_dict = self._decode_payload(access_token) 112 | 113 | if 'exp' in decoded_payload_dict.keys(): 114 | expiration_date = decoded_payload_dict['exp'] 115 | self.logger.debug('Token expiration: {}' 116 | .format(expiration_date)) 117 | except Exception as err: 118 | # If an Exception is raised, log and continue. expiration_date will 119 | # either be 0 or set to the value in the JWT. 120 | self.logger.debug('Ran into error parsing token for expiration. ' 121 | 'Setting expiration date to 0. ' 122 | '{}: {}'.format(type(err).__name__, err)) 123 | return expiration_date 124 | 125 | def _decode_payload(self, access_token): 126 | token_parts = access_token.split('.') 127 | 128 | # Padding required for the base64 library 129 | payload_bytes = bytes(token_parts[1], 'utf-8') + b'==' 130 | decoded_payload_bytes = base64.urlsafe_b64decode(payload_bytes) 131 | payload_string = decoded_payload_bytes.decode('utf-8') 132 | return json.loads(payload_string) 133 | 134 | def dump_response(self, response): 135 | data = dump.dump_all(response) 136 | return str(data.decode('utf-8')) 137 | 138 | @staticmethod 139 | def _obj_to_json(obj): 140 | return json.dumps(obj, default=str) 141 | 142 | def _headers_default_receive_json(self): 143 | return { 144 | 'Authorization': 'bearer ' + self.access_token, 145 | 'Accept': 'application/json' 146 | } 147 | 148 | def _headers_send_json(self): 149 | headers = self._headers_default_receive_json() 150 | headers['Content-Type'] = 'application/json' 151 | return headers 152 | 153 | def _headers_send_csv(self): 154 | headers = self._headers_default_receive_json() 155 | headers['Content-Type'] = 'text/csv' 156 | return headers 157 | 158 | def _headers_send_gzip(self): 159 | headers = self._headers_default_receive_json() 160 | headers['Content-Type'] = 'text/csv' 161 | headers['Content-Encoding'] = 'gzip' 162 | return headers 163 | 164 | def _headers_receive_csv(self): 165 | headers = self._headers_default_receive_json() 166 | headers['Accept'] = 'text/csv' 167 | return headers 168 | 169 | 170 | class HTTPMethod: 171 | GET = 'GET' 172 | POST = 'POST' 173 | PUT = 'PUT' 174 | PATCH = 'PATCH' 175 | DELETE = 'DELETE' 176 | -------------------------------------------------------------------------------- /pydomo/__init__.py: -------------------------------------------------------------------------------- 1 | from pydomo.Transport import DomoAPITransport 2 | from pydomo.datasets import DataSetClient 3 | from pydomo.datasets import DataSetRequest 4 | from pydomo.datasets import Schema 5 | from pydomo.datasets import Column 6 | from pydomo.datasets import ColumnType 7 | from pydomo.groups import GroupClient 8 | from pydomo.pages import PageClient 9 | from pydomo.streams import StreamClient 10 | from pydomo.users import UserClient 11 | from pydomo.users import CreateUserRequest 12 | from pydomo.accounts import AccountClient 13 | from pydomo.utilities import UtilitiesClient 14 | from pandas import read_csv 15 | from pandas import DataFrame 16 | from io import StringIO 17 | import logging 18 | import json 19 | import csv 20 | 21 | DOMO = """#################################################################################################### 22 | #################################################################################################### 23 | #################################################################################################### 24 | #################################################################################################### 25 | #################################################################################################### 26 | #################################################################################################### 27 | #################################################################################################### 28 | #################################################################################################### 29 | #################################################################################################### 30 | #################################################################################################### 31 | #################################################################################################### 32 | #################################################################################################### 33 | #################################################################################################### 34 | #################################################################################################### 35 | #################################################################################################### 36 | #################################################################################################### 37 | #################################################################################################### 38 | #################################################################################################### 39 | #################################################################################################### 40 | ``.############.` `.###### `#######################``######.` ``.##### 41 | .#######.` `.### .###################` ###.` `### 42 | #######..` .####` ..######.` `## .###############` ##` ..######.` `# 43 | ###########` .## .############. `# .###########` # .############. 44 | ############. #` `################ ` .#######. ` `################ 45 | #############` #################. `` .###. `` #################. 46 | #############` ################## .##` ` `##` ################## 47 | #############` #################. .####` `####` #################. 48 | ############. #` `################ ` .######` `######` ` `################ 49 | ###########` .## .############. # .########`########` # .############. 50 | #######..` .####` ########.` `## .#################` ##` ..######.` `# 51 | .#######.` `.### .#################` ###.` `.## 52 | .############.` `.###### .#################` ######.` `.##### 53 | #################################################################################################### 54 | #################################################################################################### 55 | #################################################################################################### 56 | #################################################################################################### 57 | #################################################################################################### 58 | #################################################################################################### 59 | #################################################################################################### 60 | #################################################################################################### 61 | #################################################################################################### 62 | #################################################################################################### 63 | #################################################################################################### 64 | #################################################################################################### 65 | #################################################################################################### 66 | #################################################################################################### 67 | #################################################################################################### 68 | #################################################################################################### 69 | #################################################################################################### 70 | #################################################################################################### 71 | ####################################################################################################""" 72 | 73 | parent_logger = logging.getLogger('pydomo') 74 | parent_logger.setLevel(logging.WARNING) 75 | 76 | 77 | class Domo: 78 | def __init__(self, client_id=None, client_secret=None, connection_file=None, api_host='api.domo.com', **kwargs): 79 | if 'logger_name' in kwargs: 80 | self.logger = parent_logger.getChild(kwargs['logger_name']) 81 | else: 82 | self.logger = parent_logger 83 | 84 | timeout = kwargs.get('request_timeout', None) 85 | scope = kwargs.get('scope') 86 | use_https = kwargs.get('use_https', True) 87 | 88 | if kwargs.get('log_level'): 89 | self.logger.setLevel(kwargs['log_level']) 90 | self.logger.debug("\n" + DOMO + "\n") 91 | 92 | # Validate credential inputs 93 | if connection_file: 94 | if client_id or client_secret: 95 | raise ValueError("Cannot specify both connection_file and client_id/client_secret") 96 | try: 97 | import configparser 98 | config = configparser.ConfigParser() 99 | config.read(connection_file) 100 | if 'client_auth' not in config: 101 | raise ValueError("connection_file must contain a [client_auth] section") 102 | client_auth = config['client_auth'] 103 | client_id = client_auth.get('client') 104 | client_secret = client_auth.get('secret') 105 | 106 | if not client_id or not client_secret: 107 | raise ValueError("connection_file must contain both client and secret in [client_auth] section") 108 | # Optional overrides from config 109 | api_host = client_auth.get('api_host', api_host) 110 | if 'use_https' in client_auth: 111 | use_https = client_auth.get('use_https').lower() == 'true' 112 | except Exception as e: 113 | raise ValueError(f"Error reading connection_file: {str(e)}") 114 | elif not client_id or not client_secret: 115 | raise ValueError("Must provide either connection_file or both client_id and client_secret") 116 | 117 | self.transport = DomoAPITransport(client_id, client_secret, api_host, use_https, self.logger, request_timeout = timeout, scope = scope) 118 | self.datasets = DataSetClient(self.transport, self.logger) 119 | self.groups = GroupClient(self.transport, self.logger) 120 | self.pages = PageClient(self.transport, self.logger) 121 | self.streams = StreamClient(self.transport, self.logger) 122 | self.users = UserClient(self.transport, self.logger) 123 | self.accounts = AccountClient(self.transport, self.logger) 124 | self.utilities = UtilitiesClient(self.transport, self.logger) 125 | 126 | 127 | ######### Datasets ######### 128 | def ds_meta(self, dataset_id): 129 | """ 130 | Get a DataSet metadata 131 | 132 | :Parameters: 133 | - `dataset_id`: id of a dataset (str) 134 | 135 | :Returns: 136 | - A dict representing the dataset meta-data 137 | """ 138 | return self.datasets.get(dataset_id) 139 | 140 | def ds_delete(self, dataset_id, prompt_before_delete=True): 141 | """ 142 | Delete a DataSet naming convention equivalent with rdomo 143 | 144 | :Parameters: 145 | - `dataset_id`: id of a dataset (str) 146 | """ 147 | 148 | del_data = 'Y' 149 | if prompt_before_delete: 150 | del_data = input("Permanently delete this data set? This is destructive and cannot be reversed. (Y/n)") 151 | 152 | out = 'Data set not deleted' 153 | if del_data == 'Y': 154 | out = self.datasets.delete(dataset_id) 155 | 156 | return out 157 | 158 | def ds_list(self, df_output = True, per_page=50, offset=0, limit=0, name_like=""): 159 | """ 160 | List DataSets 161 | 162 | >>> l = domo.ds_list(df_output=True) 163 | >>> print(l.head()) 164 | 165 | :Parameters: 166 | - `df_output`: should the result be a dataframe. Default True (Boolean) 167 | - `per_page`: results per page. Default 50 (int) 168 | - `offset`: offset if you need to paginate results. Default 0 (int) 169 | - `limit`: max ouput to return. If 0 then return all results on page. Default 0 (int) 170 | 171 | :Returns: 172 | list or pandas dataframe depending on parameters 173 | 174 | """ 175 | datasources = self.datasets.list(per_page=per_page, 176 | offset=offset, 177 | limit=limit, 178 | name_like=name_like) 179 | if df_output == False: 180 | out = list(datasources) 181 | else: 182 | out = DataFrame(list(datasources)) 183 | return out 184 | 185 | def ds_query(self, dataset_id, query, return_data=True): 186 | """ 187 | Evaluate query and return dataset in a dataframe 188 | 189 | >>> query = {"sql": "SELECT * FROM table LIMIT 2"} 190 | >>> ds = domo.ds_query('80268aef-e6a1-44f6-a84c-f849d9db05fb', query) 191 | >>> print(ds.head()) 192 | 193 | :Parameters: 194 | - `dataset_id`: id of a dataset (str) 195 | - `query`: query object (dict) 196 | - `return_data`: should the result be a dataframe. Default True (Boolean) 197 | 198 | :Returns: 199 | dict or pandas dataframe depending on parameters 200 | """ 201 | output = self.datasets.query(dataset_id, query) 202 | if(return_data == True): 203 | output = DataFrame(output['rows'], columns = output['columns']) 204 | return output 205 | 206 | 207 | def ds_get(self, dataset_id, use_schema=True) -> DataFrame: 208 | """ 209 | Export data to pandas Dataframe 210 | 211 | >>> df = domo.ds_get('80268aef-e6a1-44f6-a84c-f849d9db05fb') 212 | >>> print(df.head()) 213 | 214 | :Parameters: 215 | - `dataset_id`: id of a dataset (str) 216 | - `use_schema`: whether to use the dataset schema to determine column types (bool, default True) 217 | if false, let pandas dynamically determine types from the data 218 | :Returns: 219 | pandas dataframe 220 | """ 221 | csv_download = self.datasets.data_export(dataset_id, include_csv_header=True) 222 | content = StringIO(csv_download) 223 | 224 | if use_schema: 225 | try: 226 | schema_dict = self.ds_meta(dataset_id) 227 | 228 | if "schema" in schema_dict and "columns" in schema_dict["schema"]: 229 | 230 | dtype_dict = {} 231 | date_columns = [] 232 | 233 | for column in schema_dict["schema"]["columns"]: 234 | col_name = column["name"] 235 | col_type = column["type"] 236 | 237 | 238 | if self.utilities.is_date_type(col_type): 239 | date_columns.append(col_name) 240 | else: 241 | dtype_dict[col_name] = self.utilities.convert_domo_type_to_pandas_type(col_type) 242 | 243 | return read_csv( 244 | content, dtype=dtype_dict, parse_dates=date_columns 245 | ) 246 | 247 | except Exception as err: 248 | print(f"""An error occurred while converting domo schema to pandas dtypes. 249 | Letting pandas attempt to read the data. Error={err}""") 250 | 251 | content.seek(0) 252 | 253 | return self.utilities.read_content_to_dataframe(content) 254 | 255 | 256 | def ds_get_dict(self,ds_id): 257 | my_data = self.datasets.data_export(ds_id,True) 258 | dr = csv.DictReader(StringIO(my_data)) 259 | data_list = list(dr) 260 | return(data_list) 261 | 262 | def ds_create(self, df_up, name, description='', 263 | update_method='REPLACE', key_column_names=[]): 264 | new_stream = self.utilities.stream_create(df_up, 265 | name, 266 | description, 267 | update_method, 268 | key_column_names) 269 | if "dataSet" in new_stream: 270 | ds_id = new_stream['dataSet']['id'] 271 | self.utilities.stream_upload(ds_id, df_up, 272 | warn_schema_change=False) 273 | return ds_id 274 | else: 275 | raise Exception(("Stream creation didn't work as expected. " 276 | "Response: {}").format(new_stream)) 277 | 278 | def ds_update(self, ds_id, df_up): 279 | return self.utilities.stream_upload(ds_id, df_up) 280 | 281 | ######### PDP ######### 282 | 283 | def pdp_create(self, dataset_id, pdp_request): 284 | """ 285 | Create a PDP policy 286 | 287 | >>> policy = { 288 | "name": "Only Show Attendees", 289 | "filters": [ { 290 | "column": "Attending", 291 | "values": [ "TRUE" ], 292 | "operator": "EQUALS" 293 | } ], 294 | "users": [ 27 ] 295 | } 296 | >>> domo.pdp_create('4405ff58-1957-45f0-82bd-914d989a3ea3', policy) 297 | {"id" : 8, "type": "user", "name": "Only Show Attendees" 298 | , "filters": [{"column": "Attending", "values": [ "TRUE" ], "operator": "EQUALS" 299 | , "not": false } ], "users": [ 27 ],"groups": [ ]} 300 | 301 | :Parameters: 302 | - `dataset_id`: id of the dataset PDP will be applied to (String) Required 303 | Policy Object: 304 | - `name`: Name of the Policy (String) Required 305 | - `filters[].column`: Name of the column to filter on (String) Required 306 | - `filters[].not`: Determines if NOT is applied to the filter operation (Boolean) Required 307 | - `filters[].operator`: Matching operator (EQUALS) (String) Required 308 | - `filters[].values[]`: Values to filter on (String) Required 309 | - `type`: Type of policy (user or system) (String) Required 310 | - `users`: List of user IDs the policy applies to (array) Required 311 | - `groups`: List of group IDs the policy applies to (array) Required 312 | """ 313 | return self.datasets.create_pdp(dataset_id, pdp_request) 314 | 315 | def pdp_delete(self, dataset_id, policy_id): 316 | """ 317 | Delete PDP Policy 318 | 319 | >>> domo.pdp_delete('4405ff58-1957-45f0-82bd-914d989a3ea3', 35) 320 | 321 | :Parameters: 322 | - `dataset_id`: id of the dataset PDP will be applied to (String) Required 323 | - `policy_id`: id of the policy to delete (String) Required 324 | """ 325 | return self.datasets.delete_pdp(dataset_id, policy_id) 326 | 327 | def pdp_list(self, dataset_id, df_output = True): 328 | """ 329 | List PDP policies 330 | 331 | >>> l = domo.pdp_list(df_output=True) 332 | >>> print(l.head()) 333 | 334 | :Parameters: 335 | - `dataset_id`: id of dataset with PDP policies (str) Required 336 | - `df_output`: should the result be a dataframe. Default True (Boolean) 337 | 338 | :Returns: 339 | list or pandas dataframe depending on parameters 340 | 341 | """ 342 | output = self.datasets.list_pdps(dataset_id) 343 | if(df_output == True): 344 | output = DataFrame(output) 345 | return output 346 | 347 | def pdp_update(self, dataset_id, policy_id, policy_update): 348 | 349 | """ 350 | Update a PDP policy 351 | 352 | >>> policy = { 353 | "name": "Only Show Attendees", 354 | "filters": [ { 355 | "column": "Attending", 356 | "values": [ "TRUE" ], 357 | "operator": "EQUALS" 358 | } ], 359 | "users": [ 27 ] 360 | } 361 | >>> domo.pdp_create('4405ff58-1957-45f0-82bd-914d989a3ea3', 4, policy) 362 | {"id" : 8, "type": "user", "name": "Only Show Attendees" 363 | , "filters": [{"column": "Attending", "values": [ "TRUE" ], "operator": "EQUALS" 364 | , "not": false } ], "users": [ 27 ],"groups": [ ]} 365 | 366 | :Parameters: 367 | - `dataset_id`: id of the dataset PDP will be applied to (String) Required 368 | - `policy_id`: id of the PDP pollicy that will be updated (String) Required 369 | Policy Object: 370 | - `name`: Name of the Policy (String) Required 371 | - `filters[].column`: Name of the column to filter on (String) Required 372 | - `filters[].not`: Determines if NOT is applied to the filter operation (Boolean) Required 373 | - `filters[].operator`: Matching operator (EQUALS) (String) Required 374 | - `filters[].values[]`: Values to filter on (String) Required 375 | - `type`: Type of policy (user or system) (String) Required 376 | - `users`: List of user IDs the policy applies to (array) Required 377 | - `groups`: List of group IDs the policy applies to (array) Required 378 | """ 379 | return self.datasets.update_pdp(dataset_id, policy_id, policy_update) 380 | 381 | 382 | ######### Pages ######### 383 | 384 | def page_create(self, name, **kwargs): 385 | """Create a new page. 386 | 387 | >>> page = {'name':'New Page'} 388 | >>> new_page = domo.pages.create(**page) 389 | >>> print(new_page) 390 | {'id': 123456789, 'parentId': 0, 'name': 'My Page', 391 | 'locked': False, 'ownerId': 12345, 'cardIds': [], 392 | 'visibility': {'userIds': 12345}} 393 | 394 | :Parameters: 395 | - `name`: The name of the new page 396 | - `parentId`: (optional) If present create page as subpage 397 | - `locked`: (optional) whether to lock the page 398 | - `cardIds`: (optional) cards to place on the page 399 | - `visibility`: (optional) dict of userIds and/or groupIds to 400 | give access to 401 | 402 | :Returns: 403 | - A dict representing the page 404 | """ 405 | return self.pages.create(name, **kwargs) 406 | 407 | 408 | def page_get(self, page_id): 409 | """Get a page. 410 | 411 | >>> page = domo.pages.get(page_id) 412 | >>> print(page) 413 | {'id': 123456789, 'parentId': 0, 'name': 'My Page', 414 | 'locked': False, 'ownerId': 12345, 'cardIds': [], 415 | 'visibility': {'userIds': 12345}} 416 | 417 | :Parameters: 418 | - `page_id`: ID of the page to get 419 | 420 | :returns: 421 | - A dict representing the page 422 | """ 423 | return self.pages.get(page_id) 424 | 425 | 426 | def page_delete(self, page_id): 427 | """Delete a page. 428 | 429 | :Parameters: 430 | - `page_id`: ID of the page to delete 431 | """ 432 | return self.pages.delete(page_id) 433 | 434 | 435 | def collections_create(self, page_id, title, **kwargs): 436 | """Create a collection on a page. 437 | 438 | >>> collection = domo.pages.create_collection(page_id, 439 | 'Collection') 440 | >>> print(collection) 441 | {'id': 1234321, 'title': 'Collection', 'description': '', 442 | 'cardIds': []} 443 | 444 | :Parameters: 445 | - `page_id`: ID of the page to create a collection on 446 | - `title`: The title of the collection 447 | - `description`: (optional) The description of the collection 448 | - `cardIds`: (optional) cards to place in the collection 449 | 450 | :Returns: 451 | - A dict representing the collection 452 | """ 453 | return self.pages.create_collection(page_id, title, **kwargs) 454 | 455 | 456 | def page_get_collections(self, page_id): 457 | """Get a collections of a page 458 | 459 | >>> print(domo.pages.get_collections(page_id)) 460 | [{'id': 1234321, 'title': 'Collection', 'description': '', 461 | 'cardIds': []}] 462 | 463 | :Parameters: 464 | - `page_id`: ID of the page 465 | 466 | :Returns: 467 | - A list of dicts representing the collections 468 | """ 469 | return self.pages.get_collections(page_id) 470 | 471 | 472 | def collections_update(self, page_id, collection_id=None, **kwargs): 473 | """Update a collection of a page. 474 | 475 | >>> collections = domo.pages.get_collections(page_id) 476 | >>> print(collections) 477 | [{'id': 1234321, 'title': 'Collection', 'description': '', 478 | 'cardIds': []}] 479 | >>> collection_id = collections[0]['id'] 480 | >>> domo.pages.update_collection(page_id, collection_id, 481 | description='description', 482 | cardIds=[54321, 13579]) 483 | >>> print(domo.pages.get_collections(page_id)) 484 | [{'id': 1234321, 'title': 'Collection', 485 | 'description': 'description', 'cardIds': [54321, 13579]}] 486 | 487 | # Using **kwargs: 488 | >>> collections = domo.pages.get_collections(page_id) 489 | >>> collections[0]['description'] = 'Description' 490 | >>> domo.pages.update_collection(page_id, **collections[0]) 491 | 492 | :Parameters: 493 | - `page_id`: ID of the page the collection is on 494 | - `collection_id`: ID of the collection. Can also be provided 495 | by supplying `id` to **kwargs. This allows for calling 496 | get_collections, updating one of the returned collections, 497 | then passing it to update_collection. 498 | - `title`: (optional) update the title 499 | - `description`: (optional) update the description 500 | - `cardIds`: (optional) update cards in the collection 501 | 502 | :Returns: 503 | - A dict representing the collection 504 | """ 505 | return self.pages.update_collection(page_id, collection_id, **kwargs) 506 | 507 | 508 | def collections_delete(self, page_id, collection_id): 509 | """Delete a collection from a page. 510 | 511 | :Parameters: 512 | - `page_id`: ID of the page the collection is on 513 | - `collection_id`: ID of the collection to delete 514 | """ 515 | return self.pages.delete_collection(page_id, collection_id) 516 | 517 | 518 | def page_list(self, per_page=50, offset=0, limit=0): 519 | """List pages. 520 | Returns a list of dicts (with nesting possible) 521 | If limit is supplied and non-zero, returns up to limit pages 522 | """ 523 | return list(self.pages.list()) 524 | 525 | 526 | def page_update(self, page_id=None, **kwargs): 527 | """Update a page. 528 | 529 | >>> print(domo.pages.get(page_id)) 530 | {'id': 123456789, 'parentId': 0, 'name': 'My Page', 531 | 'locked': False, 'ownerId': 12345, 'cardIds': [], 532 | 'visibility': {'userIds': 12345}} 533 | >>> domo.pages.update(page_id, locked=True, 534 | cardIds=[54321, 13579]) 535 | >>> print(domo.pages.get(page_id)) 536 | {'id': 123456789, 'parentId': 0, 'name': 'My Page', 537 | 'locked': True, 'ownerId': 12345, 'cardIds': [54321, 13579], 538 | 'visibility': {'userIds': 12345}} 539 | 540 | # Using **kwargs: 541 | >>> page = domo.pages.get(page_id) 542 | >>> page['cardIds'].append(new_card_id) 543 | >>> domo.pages.update(**page) 544 | 545 | :Parameters: 546 | - `page_id`: ID of the page to update. Can also be provided 547 | by supplying `id` to **kwargs. This allows for calling get, 548 | updating the returned object, then passing it to update. 549 | - `name`: (optional) rename the page 550 | - `parentId`: (optional) turn page into subpage, or subpage 551 | into top-level page if parentId is present and falsey 552 | - `ownerId`: (optional) change owner of the page 553 | - `locked`: (optional) lock or unlock the page 554 | - `collectionIds`: (optional) reorder collections on page 555 | - `cardIds`: (optional) specify which cards to have on page 556 | - `visibility`: (optional) change who can see the page 557 | """ 558 | return self.pages.update(page_id, **kwargs) 559 | 560 | 561 | ######### Groups ######### 562 | 563 | def groups_add_users(self, group_id, user_id): 564 | """ 565 | Add a User to a Group 566 | """ 567 | 568 | if isinstance(user_id,list): 569 | for x in user_id: 570 | self.groups.add_user(group_id, x) 571 | else: 572 | self.groups.add_user(group_id, user_id) 573 | 574 | return 'success' 575 | 576 | 577 | 578 | def groups_create(self, group_name, users=-1, active='true'): 579 | """ 580 | Create a Group 581 | """ 582 | req_body = {'name':group_name,'active':active} 583 | grp_created = self.groups.create(req_body) 584 | if (not isinstance(users,list) and users > 0) or isinstance(users,list): 585 | self.groups_add_users(grp_created['id'],users) 586 | 587 | return grp_created 588 | 589 | 590 | 591 | def groups_delete(self, group_id): 592 | """ 593 | Delete a Group 594 | """ 595 | existing_users = self.groups_list_users(group_id) 596 | self.groups_remove_users(group_id,existing_users) 597 | return self.groups.delete(group_id) 598 | 599 | 600 | 601 | def groups_get(self, group_id): 602 | """ 603 | Get a Group Definition 604 | """ 605 | return self.groups.get(group_id) 606 | 607 | 608 | 609 | def groups_list(self): 610 | """ 611 | List all groups in Domo instance in a pandas dataframe. 612 | """ 613 | grps = [] 614 | n_ret = 1 615 | off = 0 616 | batch_size = 500 617 | while n_ret > 0: 618 | gg = self.groups.list(batch_size,off*batch_size) 619 | grps.extend(gg) 620 | n_ret = gg.__len__() 621 | off += 1 622 | return DataFrame(grps) 623 | 624 | 625 | def groups_list_users(self, group_id): 626 | """ 627 | List Users in a Group 628 | """ 629 | user_list = [] 630 | n_ret = 1 631 | off = 0 632 | batch_size=500 633 | while n_ret > 0: 634 | i_users = self.groups.list_users(group_id,limit=batch_size,offset=off*batch_size) 635 | user_list.extend(i_users) 636 | n_ret = i_users.__len__() 637 | off += 1 638 | 639 | return user_list 640 | 641 | 642 | 643 | def groups_remove_users(self, group_id, user_id): 644 | """ 645 | Remove a User to a Group 646 | """ 647 | if isinstance(user_id,list): 648 | for x in user_id: 649 | self.groups.remove_user(group_id, x) 650 | else: 651 | self.groups.remove_user(group_id, user_id) 652 | 653 | return 'success' 654 | 655 | 656 | ######### Accounts ######### 657 | def accounts_list(self): 658 | """List accounts. 659 | Returns a generator that will call the API multiple times 660 | If limit is supplied and non-zero, returns up to limit accounts 661 | 662 | >>> list(domo.accounts.list()) 663 | [{'id': '40', 'name': 'DataSet Copy Test', ...}, 664 | {'id': '41', 'name': 'DataSet Copy Test2', ...}] 665 | 666 | :Parameters: 667 | - `per_page`: results per page. Default 50 (int) 668 | - `offset`: offset if you need to paginate results. Default 0 (int) 669 | - `limit`: max ouput to return. If 0 then return all results on page. Default 0 (int) 670 | 671 | 672 | :returns: 673 | - A list of dicts (with nesting possible) 674 | """ 675 | return list(self.accounts.list()) 676 | 677 | def accounts_get(self, account_id): 678 | """Get a account. 679 | 680 | >>> account = domo.accounts.get(account_id) 681 | >>> print(account) 682 | {'id': '40', 'name': 'DataSet Copy Test', 'valid': True, 'type': {'id': 'domo-csv', 'properties': {}}} 683 | 684 | :Parameters: 685 | - `account_id`: ID of the account to get (str) 686 | 687 | :returns: 688 | - A dict representing the account 689 | """ 690 | return self.accounts.get(account_id) 691 | 692 | def accounts_delete(self, account_id): 693 | """Delete a account. 694 | 695 | :Parameters: 696 | - `account_id`: ID of the account to delete 697 | """ 698 | return self.accounts.delete(account_id) 699 | 700 | def accounts_create(self, **kwargs): 701 | """Create a new account. 702 | 703 | >>> account = { 'name': 'DataSet Copy Test', 'valid': True, 'type': {'id': 'domo-csv', 'properties': {}}} 704 | >>> new_account = domo.accounts.create(**account) 705 | >>> print(new_account) 706 | {'name': 'DataSet Copy Test', 'valid': True, 'type': {'id': 'domo-csv', 'properties': {}}} 707 | 708 | 709 | :Returns: 710 | - A dict representing the account 711 | """ 712 | return self.accounts.create(**kwargs) 713 | 714 | def accounts_update(self, account_id, **kwargs): 715 | """Update a account. 716 | 717 | >>> print(domo.accounts.get(account_id)) 718 | {'id': '40', 'name': 'DataSet Copy Test', 'valid': True, 'type': {'id': 'domo-csv', 'properties': {}}} 719 | updatedAccount = {'name': 'DataSet Copy Test2, 'valid': True, 'type': {'id': 'domo-csv', 'properties': {}}} 720 | >>> domo.accounts.update(account_id, **updatedAccount) 721 | >>> print(domo.accounts.get(account_id)) 722 | {'id': '40', 'name': 'DataSet Copy Test2, 'valid': True, 'type': {'id': 'domo-csv', 'properties': {}}} 723 | 724 | 725 | :Parameters: 726 | - `account_id`: ID of the account to update. 727 | - `kwargs`: New account object 728 | """ 729 | return self.accounts.update(account_id, **kwargs) 730 | 731 | ######### Users ######### 732 | def users_add(self, x_name, x_email, x_role, x_sendInvite=False): 733 | uu = CreateUserRequest() 734 | uu.name = x_name 735 | uu.email = x_email 736 | uu.role = x_role 737 | return self.users.create(uu,x_sendInvite) 738 | 739 | def users_get(self, user_id): 740 | return self.users.get(user_id) 741 | 742 | def users_list(self,df_output=True): 743 | return self.users.list_all(df_output) 744 | 745 | def users_update(self, user_id, user_def): 746 | return self.users.update(user_id, user_def) 747 | 748 | def users_delete(self, user_id): 749 | return self.users.delete(user_id) 750 | -------------------------------------------------------------------------------- /pydomo/accounts/AccountClient.py: -------------------------------------------------------------------------------- 1 | import requests 2 | 3 | from pydomo.DomoAPIClient import DomoAPIClient 4 | from pydomo.Transport import HTTPMethod 5 | 6 | ACCOUNT_DESC = "Account" 7 | URL_BASE = '/v1/accounts' 8 | 9 | ACCOUNT_CREATION_KWARGS = [ 10 | 'name', 11 | 'type', 12 | 'valid' 13 | ] 14 | 15 | 16 | class AccountClient(DomoAPIClient): 17 | """ 18 | accounts 19 | - Programmatically manage Domo accounts 20 | - Docs: https://developer.domo.com/docs/accounts-api-reference/account-api-reference 21 | """ 22 | 23 | def create(self, **kwargs): 24 | """Create a new account. 25 | 26 | >>> account = { 'name': 'DataSet Copy Test', 'valid': True, 'type': {'id': 'domo-csv', 'properties': {}}} 27 | >>> new_account = domo.accounts.create(**account) 28 | >>> print(new_account) 29 | {'name': 'DataSet Copy Test', 'valid': True, 'type': {'id': 'domo-csv', 'properties': {}}} 30 | 31 | 32 | :Returns: 33 | - A dict representing the account 34 | """ 35 | 36 | self._validate_params(kwargs, ACCOUNT_CREATION_KWARGS) 37 | 38 | account_request = kwargs 39 | print(account_request) 40 | return self._create(URL_BASE, account_request, {}, ACCOUNT_DESC) 41 | 42 | def get(self, account_id): 43 | """Get a account. 44 | 45 | >>> account = domo.accounts.get(account_id) 46 | >>> print(account) 47 | {'id': '40', 'name': 'DataSet Copy Test', 'valid': True, 'type': {'id': 'domo-csv', 'properties': {}}} 48 | 49 | :Parameters: 50 | - `account_id`: ID of the account to get (str) 51 | 52 | :returns: 53 | - A dict representing the account 54 | """ 55 | url = '{base}/{account_id}'.format(base=URL_BASE, account_id=account_id) 56 | return self._get(url, ACCOUNT_DESC) 57 | 58 | def list(self, per_account=50, offset=0, limit=0): 59 | """List accounts. 60 | Returns a generator that will call the API multiple times 61 | If limit is supplied and non-zero, returns up to limit accounts 62 | 63 | >>> list(domo.accounts.list()) 64 | [{'id': '40', 'name': 'DataSet Copy Test', ...}, 65 | {'id': '41', 'name': 'DataSet Copy Test2', ...}] 66 | 67 | :Parameters: 68 | - `per_page`: results per page. Default 50 (int) 69 | - `offset`: offset if you need to paginate results. Default 0 (int) 70 | - `limit`: max ouput to return. If 0 then return all results on page. Default 0 (int) 71 | 72 | 73 | :returns: 74 | - A list of dicts (with nesting possible) 75 | """ 76 | # API uses pagination with a max of 50 per account 77 | if per_account not in range(1, 51): 78 | raise ValueError('per_account must be between 1 and 50 (inclusive)') 79 | # Don't pull 50 values if user requests 10 80 | if limit: 81 | per_account = min(per_account, limit) 82 | 83 | params = { 84 | 'limit': per_account, 85 | 'offset': offset, 86 | } 87 | account_count = 0 88 | 89 | accounts = self._list(URL_BASE, params, ACCOUNT_DESC) 90 | while accounts: 91 | for account in accounts: 92 | yield account 93 | account_count += 1 94 | if limit and account_count >= limit: 95 | return 96 | 97 | params['offset'] += per_account 98 | if limit and params['offset'] + per_account > limit: 99 | # Don't need to pull more than the limit 100 | params['limit'] = limit - params['offset'] 101 | accounts = self._list(URL_BASE, params, ACCOUNT_DESC) 102 | 103 | def update(self, account_id=None, **kwargs): 104 | """Update a account. 105 | 106 | >>> print(domo.accounts.get(account_id)) 107 | {'id': '40', 'name': 'DataSet Copy Test', 'valid': True, 'type': {'id': 'domo-csv', 'properties': {}}} 108 | updatedAccount = {'name': 'DataSet Copy Test2, 'valid': True, 'type': {'id': 'domo-csv', 'properties': {}}} 109 | >>> domo.accounts.update(account_id, **updatedAccount) 110 | >>> print(domo.accounts.get(account_id)) 111 | {'id': '40', 'name': 'DataSet Copy Test2, 'valid': True, 'type': {'id': 'domo-csv', 'properties': {}}} 112 | 113 | 114 | :Parameters: 115 | - `account_id`: ID of the account to update. 116 | - `kwargs`: New account object 117 | """ 118 | 119 | url = '{base}/{account_id}'.format(base=URL_BASE, account_id=account_id) 120 | return self._update(url, 121 | HTTPMethod.PATCH, 122 | requests.codes.accepted, 123 | kwargs, 124 | ACCOUNT_DESC) 125 | 126 | def delete(self, account_id): 127 | """Delete a account. 128 | 129 | :Parameters: 130 | - `account_id`: ID of the account to delete 131 | """ 132 | url = '{base}/{account_id}'.format(base=URL_BASE, account_id=account_id) 133 | return self._delete(url, ACCOUNT_DESC) 134 | -------------------------------------------------------------------------------- /pydomo/accounts/__init__.py: -------------------------------------------------------------------------------- 1 | from .AccountClient import AccountClient 2 | -------------------------------------------------------------------------------- /pydomo/common/DomoObject.py: -------------------------------------------------------------------------------- 1 | class DomoObject(dict): 2 | '''Superclass for DomoAPI objects. These are dict-like objects with 3 | predefined allowed fields. 4 | ''' 5 | def __getattribute__(self, name): 6 | if name in super().__getattribute__('accepted_attrs'): 7 | return self.__getitem__(name) 8 | return super().__getattribute__(name) 9 | 10 | def __setattr__(self, name, value): 11 | self.__setitem__(name, value) 12 | 13 | def __setitem__(self, name, value): 14 | if name not in self.accepted_attrs: 15 | raise AttributeError("'{}' object has no attribute '{}'".format( 16 | type(self).__name__, name)) 17 | super().__setitem__(name, value) 18 | -------------------------------------------------------------------------------- /pydomo/common/__init__.py: -------------------------------------------------------------------------------- 1 | from .DomoObject import DomoObject 2 | -------------------------------------------------------------------------------- /pydomo/datasets/DataSetClient.py: -------------------------------------------------------------------------------- 1 | import os 2 | import requests 3 | from pandas import read_csv 4 | from pandas import DataFrame 5 | from io import StringIO 6 | 7 | from pydomo.datasets import Sorting, UpdateMethod 8 | from pydomo.DomoAPIClient import DomoAPIClient 9 | from pydomo.Transport import HTTPMethod 10 | 11 | """ 12 | DataSets 13 | - Programmatically manage Domo DataSets 14 | - Use DataSets for fairly static data sources that only require occasional updates via data replacement 15 | - Use Streams if your data source is massive, constantly changing, or rapidly growing 16 | - Docs: https://developer.domo.com/docs/data-apis/data 17 | """ 18 | 19 | DATA_SET_DESC = "DataSet" 20 | PDP_DESC = "Personalized Data Policy (PDP)" 21 | URL_BASE = '/v1/datasets' 22 | 23 | 24 | class DataSetClient(DomoAPIClient): 25 | def __init__(self, transport, logger): 26 | super(DataSetClient, self).__init__(transport, logger) 27 | 28 | """ 29 | Create a DataSet 30 | """ 31 | def create(self, dataset_request): 32 | return self._create(URL_BASE, dataset_request, {}, DATA_SET_DESC) 33 | 34 | """ 35 | Get a DataSet 36 | """ 37 | def get(self, dataset_id): 38 | url = '{base}/{dataset_id}'.format( 39 | base=URL_BASE, dataset_id=dataset_id) 40 | return self._get(url, DATA_SET_DESC) 41 | 42 | """ 43 | List DataSets 44 | Returns a generator that will call the API multiple times 45 | If limit is supplied and non-zero, returns up to limit datasets 46 | """ 47 | def list(self, sort=Sorting.DEFAULT, per_page=50, 48 | offset=0, limit=0, name_like=""): 49 | # API uses pagination with a max of 50 per page 50 | if per_page not in range(1, 51): 51 | raise ValueError('per_page must be between 1 and 50 (inclusive)') 52 | 53 | # Don't pull 50 values if user requests 10 54 | if limit: 55 | per_page = min(per_page, limit) 56 | 57 | params = { 58 | 'sort': sort, 59 | 'limit': per_page, 60 | 'offset': offset, 61 | 'nameLike': name_like 62 | } 63 | dataset_count = 0 64 | 65 | datasets = self._list(URL_BASE, params, DATA_SET_DESC) 66 | while datasets: 67 | for dataset in datasets: 68 | yield dataset 69 | dataset_count += 1 70 | if limit and dataset_count >= limit: 71 | return 72 | 73 | params['offset'] += per_page 74 | if limit and params['offset'] + per_page > limit: 75 | # Don't need to pull more than the limit 76 | params['limit'] = limit - params['offset'] 77 | datasets = self._list(URL_BASE, params, DATA_SET_DESC) 78 | 79 | """ 80 | Update a DataSet 81 | """ 82 | def update(self, dataset_id, dataset_update): 83 | url = '{base}/{dataset_id}'.format( 84 | base=URL_BASE, dataset_id=dataset_id) 85 | return self._update(url, HTTPMethod.PUT, requests.codes.ok, 86 | dataset_update, DATA_SET_DESC) 87 | 88 | """ 89 | Import data from a CSV string 90 | """ 91 | def data_import(self, dataset_id, csv, update_method=UpdateMethod.REPLACE): 92 | return self._data_import(dataset_id, str.encode(csv), update_method) 93 | 94 | """ 95 | Import data from a CSV file 96 | """ 97 | def data_import_from_file(self, dataset_id, filepath, 98 | update_method=UpdateMethod.REPLACE): 99 | with open(os.path.expanduser(filepath), 'rb') as csvfile: 100 | # passing an open file to the requests library invokes http 101 | # streaming (uses minimal system memory) 102 | self._data_import(dataset_id, csvfile, update_method) 103 | 104 | def _data_import(self, dataset_id, csv, update_method): 105 | url = '{base}/{dataset_id}/data?updateMethod={method}'.format( 106 | base=URL_BASE, dataset_id=dataset_id, method=update_method) 107 | return self._upload_csv(url, requests.codes.no_content, csv, 108 | DATA_SET_DESC) 109 | 110 | """ 111 | Export data to a CSV string (in-memory) 112 | """ 113 | def data_export(self, dataset_id, include_csv_header): 114 | url = '{base}/{dataset_id}/data'.format( 115 | base=URL_BASE, dataset_id=dataset_id) 116 | response = self._download_csv(url, include_csv_header) 117 | if response.status_code == requests.codes.ok: 118 | return bytes.decode(response.content) 119 | else: 120 | self.logger.debug("Error downloading data from DataSet: " + self.transport.dump_response(response)) 121 | raise Exception("Error downloading data from DataSet: " + response.text) 122 | 123 | """ 124 | Export data to a CSV file (streams to disk) 125 | """ 126 | def data_export_to_file(self, dataset_id, file_path, include_csv_header): 127 | url = '{base}/{dataset_id}/data'.format( 128 | base=URL_BASE, dataset_id=dataset_id) 129 | response = self._download_csv(url, include_csv_header) 130 | file_path = str(file_path) 131 | if not file_path.endswith('.csv'): 132 | file_path += '.csv' 133 | with open(file_path, 'wb') as csv_file: 134 | for chunk in response.iter_content(chunk_size=1024): 135 | if chunk: 136 | csv_file.write(chunk) 137 | return open(file_path, 'r+') # return the file object as readable and writable 138 | 139 | """ 140 | Delete a DataSet 141 | """ 142 | def delete(self, dataset_id): 143 | url = '{base}/{dataset_id}'.format( 144 | base=URL_BASE, dataset_id=dataset_id) 145 | return self._delete(url, DATA_SET_DESC) 146 | 147 | """ 148 | Create a Personalized Data Policy (PDP) 149 | """ 150 | def create_pdp(self, dataset_id, pdp_request): 151 | url = '{base}/{dataset_id}/policies'.format( 152 | base=URL_BASE, dataset_id=dataset_id) 153 | return self._create(url, pdp_request, {}, PDP_DESC) 154 | 155 | """ 156 | Get a specific Personalized Data Policy (PDP) for a given DataSet 157 | """ 158 | def get_pdp(self, dataset_id, policy_id): 159 | url = '{base}/{dataset_id}/policies/{policy_id}'.format( 160 | base=URL_BASE, dataset_id=dataset_id, policy_id=policy_id) 161 | return self._get(url, PDP_DESC) 162 | 163 | """ 164 | List all Personalized Data Policies (PDPs) for a given DataSet 165 | """ 166 | def list_pdps(self, dataset_id): 167 | url = '{base}/{dataset_id}/policies'.format( 168 | base=URL_BASE, dataset_id=dataset_id) 169 | return self._list(url, {}, DATA_SET_DESC) 170 | 171 | """ 172 | Update a specific Personalized Data Policy (PDP) for a given DataSet 173 | """ 174 | def update_pdp(self, dataset_id, policy_id, policy_update): 175 | url = '{base}/{dataset_id}/policies/{policy_id}'.format( 176 | base=URL_BASE, dataset_id=dataset_id, policy_id=policy_id) 177 | return self._update(url, HTTPMethod.PUT, requests.codes.ok, 178 | policy_update, PDP_DESC) 179 | 180 | """ 181 | Delete a specific Personalized Data Policy (PDPs) for a given DataSet 182 | """ 183 | def delete_pdp(self, dataset_id, policy_id): 184 | url = '{base}/{dataset_id}/policies/{policy_id}'.format( 185 | base=URL_BASE, dataset_id=dataset_id, policy_id=policy_id) 186 | return self._delete(url, PDP_DESC) 187 | 188 | """ 189 | Query's Dataset 190 | """ 191 | def query(self, dataset_id, query): 192 | url = '{base}/query/execute/{dataset_id}'.format( 193 | base=URL_BASE, dataset_id=dataset_id) 194 | req_body = {'sql':query} 195 | return self._create(url, req_body, {}, 'query') -------------------------------------------------------------------------------- /pydomo/datasets/DataSetModel.py: -------------------------------------------------------------------------------- 1 | from ..common import DomoObject 2 | 3 | 4 | class Column(DomoObject): 5 | accepted_attrs = [ 6 | 'type', 7 | 'name' 8 | ] 9 | 10 | def __init__(self, column_type, name): 11 | super().__init__() 12 | self.type = column_type 13 | self.name = name 14 | 15 | 16 | class ColumnType: 17 | STRING = 'STRING' 18 | DECIMAL = 'DECIMAL' 19 | LONG = 'LONG' 20 | DOUBLE = 'DOUBLE' 21 | DATE = 'DATE' 22 | DATETIME = 'DATETIME' 23 | 24 | 25 | class DataSetRequest(DomoObject): 26 | accepted_attrs = [ 27 | 'name', 28 | 'description', 29 | 'schema', 30 | 'owner' 31 | ] 32 | 33 | 34 | class FilterOperator: 35 | EQUALS = 'EQUALS' 36 | LIKE = 'LIKE' 37 | GREATER_THAN = 'GREATER_THAN' 38 | LESS_THAN = 'LESS_THAN' 39 | GREATER_THAN_EQUAL = 'GREATER_THAN_EQUAL' 40 | LESS_THAN_EQUAL = 'LESS_THAN_EQUAL' 41 | BETWEEN = 'BETWEEN' 42 | BEGINS_WITH = 'BEGINS_WITH' 43 | ENDS_WITH = 'ENDS_WITH' 44 | CONTAINS = 'CONTAINS' 45 | 46 | 47 | class Policy(DomoObject): 48 | accepted_attrs = [ 49 | 'id', 50 | 'type', 51 | 'name', 52 | 'filters', 53 | 'users', 54 | 'virtualUsers', 55 | 'groups' 56 | ] 57 | 58 | 59 | class PolicyType: 60 | USER = 'user' 61 | SYSTEM = 'system' 62 | 63 | 64 | class PolicyFilter(DomoObject): 65 | accepted_attrs = [ 66 | 'column', 67 | 'values', 68 | 'operator', 69 | 'not' 70 | ] 71 | 72 | 73 | class Schema(DomoObject): 74 | accepted_attrs = [ 75 | 'columns' 76 | ] 77 | def __init__(self, columns): 78 | self.columns = columns 79 | 80 | 81 | class Sorting: 82 | CARD_COUNT = 'cardCount' 83 | DEFAULT = None 84 | NAME = 'name' 85 | STATUS = 'errorState' 86 | UPDATED = 'lastUpdated' 87 | 88 | 89 | class UpdateMethod: 90 | APPEND = 'APPEND' 91 | REPLACE = 'REPLACE' 92 | -------------------------------------------------------------------------------- /pydomo/datasets/__init__.py: -------------------------------------------------------------------------------- 1 | from .DataSetModel import Column, ColumnType, DataSetRequest, FilterOperator 2 | from .DataSetModel import Policy, PolicyType, PolicyFilter, Schema, Sorting 3 | from .DataSetModel import UpdateMethod 4 | from .DataSetClient import DataSetClient 5 | -------------------------------------------------------------------------------- /pydomo/groups/GroupClient.py: -------------------------------------------------------------------------------- 1 | from pydomo.DomoAPIClient import DomoAPIClient 2 | from pydomo.Transport import HTTPMethod 3 | import requests 4 | 5 | 6 | """ 7 | Group Client 8 | - Programmatically manage Domo User Groups 9 | - Docs: https://developer.domo.com/docs/domo-apis/group-apis 10 | """ 11 | 12 | 13 | class GroupClient(DomoAPIClient): 14 | def __init__(self, transport, logger): 15 | super(GroupClient, self).__init__(transport, logger) 16 | self.urlBase = '/v1/groups/' 17 | self.groupDesc = "Group" 18 | 19 | """ 20 | Create a Group 21 | """ 22 | def create(self, group_request): 23 | return self._create(self.urlBase, group_request, {}, self.groupDesc) 24 | 25 | """ 26 | Get a Group 27 | """ 28 | def get(self, group_id): 29 | return self._get(self._base(group_id), self.groupDesc) 30 | 31 | """ 32 | List Groups 33 | """ 34 | def list(self, limit, offset): 35 | params = { 36 | 'limit': str(limit), 37 | 'offset': str(offset), 38 | } 39 | return self._list(self.urlBase, params, self.groupDesc) 40 | 41 | """ 42 | Update a Group 43 | """ 44 | def update(self, group_id, group_update): 45 | return self._update(self._base(group_id), HTTPMethod.PUT, requests.codes.ok, group_update, self.groupDesc) 46 | 47 | """ 48 | Delete a Group 49 | """ 50 | def delete(self, group_id): 51 | return self._delete(self._base(group_id), self.groupDesc) 52 | 53 | """ 54 | Add a User to a Group 55 | """ 56 | def add_user(self, group_id, user_id): 57 | url = self._base(group_id) + '/users/' + str(user_id) 58 | desc = "a User in a Group" 59 | return self._update(url, HTTPMethod.PUT, requests.codes.no_content, {}, desc) 60 | 61 | """ 62 | List Users in a Group 63 | """ 64 | def list_users(self, group_id, limit, offset): 65 | url = self._base(group_id) + '/users' 66 | desc = "a list of Users in a Group" 67 | params = { 68 | 'limit': str(limit), 69 | 'offset': str(offset), 70 | } 71 | return self._list(url, params, desc) 72 | 73 | """ 74 | Remove a User to a Group 75 | """ 76 | def remove_user(self, group_id, user_id): 77 | url = self._base(group_id) + '/users/' + str(user_id) 78 | desc = "a User in a Group" 79 | return self._delete(url, desc) 80 | -------------------------------------------------------------------------------- /pydomo/groups/GroupsModel.py: -------------------------------------------------------------------------------- 1 | from ..common import DomoObject 2 | 3 | 4 | class CreateGroupRequest(DomoObject): 5 | accepted_attrs = [ 6 | 'name', 7 | 'active', 8 | 'default', 9 | 'memberCount' 10 | ] 11 | -------------------------------------------------------------------------------- /pydomo/groups/__init__.py: -------------------------------------------------------------------------------- 1 | from .GroupsModel import CreateGroupRequest 2 | from .GroupClient import GroupClient 3 | -------------------------------------------------------------------------------- /pydomo/pages/PageClient.py: -------------------------------------------------------------------------------- 1 | import requests 2 | 3 | from pydomo.DomoAPIClient import DomoAPIClient 4 | from pydomo.Transport import HTTPMethod 5 | 6 | PAGE_DESC = "Page" 7 | COLLECTION_DESC = "Collection" 8 | URL_BASE = '/v1/pages' 9 | 10 | PAGE_CREATION_KWARGS = [ 11 | 'parentId', 12 | 'locked', 13 | 'cardIds', 14 | 'visibility' 15 | ] 16 | 17 | PAGE_UPDATE_KWARGS = [ 18 | 'id', 19 | 'name', 20 | 'parentId', 21 | 'ownerId', 22 | 'locked', 23 | 'collectionIds', 24 | 'cardIds', 25 | 'visibility' 26 | ] 27 | 28 | COLLECTION_CREATION_KWARGS = [ 29 | 'description', 30 | 'cardIds' 31 | ] 32 | 33 | COLLECTION_UPDATE_KWARGS = [ 34 | 'id', 35 | 'description', 36 | 'cardIds', 37 | 'title' 38 | ] 39 | 40 | 41 | class PageClient(DomoAPIClient): 42 | """ 43 | Pages 44 | - Programmatically manage Domo Pages 45 | - Docs: https://developer.domo.com/docs/data-apis/pages 46 | """ 47 | 48 | def create(self, name, **kwargs): 49 | """Create a new page. 50 | 51 | >>> page = {'name':'New Page'} 52 | >>> new_page = domo.pages.create(**page) 53 | >>> print(new_page) 54 | {'id': 123456789, 'parentId': 0, 'name': 'My Page', 55 | 'locked': False, 'ownerId': 12345, 'cardIds': [], 56 | 'visibility': {'userIds': 12345}} 57 | 58 | :Parameters: 59 | - `name`: The name of the new page 60 | - `parentId`: (optional) If present create page as subpage 61 | - `locked`: (optional) whether to lock the page 62 | - `cardIds`: (optional) cards to place on the page 63 | - `visibility`: (optional) dict of userIds and/or groupIds to 64 | give access to 65 | 66 | :Returns: 67 | - A dict representing the page 68 | """ 69 | 70 | self._validate_params(kwargs, PAGE_CREATION_KWARGS) 71 | 72 | page_request = {'name': name} 73 | page_request.update(kwargs) 74 | return self._create(URL_BASE, page_request, {}, PAGE_DESC) 75 | 76 | def get(self, page_id): 77 | """Get a page. 78 | 79 | >>> page = domo.pages.get(page_id) 80 | >>> print(page) 81 | {'id': 123456789, 'parentId': 0, 'name': 'My Page', 82 | 'locked': False, 'ownerId': 12345, 'cardIds': [], 83 | 'visibility': {'userIds': 12345}} 84 | 85 | :Parameters: 86 | - `page_id`: ID of the page to get 87 | 88 | :returns: 89 | - A dict representing the page 90 | """ 91 | url = '{base}/{page_id}'.format(base=URL_BASE, page_id=page_id) 92 | return self._get(url, PAGE_DESC) 93 | 94 | def list(self, per_page=50, offset=0, limit=0): 95 | """List pages. 96 | Returns a generator that will call the API multiple times 97 | If limit is supplied and non-zero, returns up to limit pages 98 | 99 | >>> list(domo.pages.list()) 100 | [{'id': 123456789, 'name': 'My Page', 'children': []}, ...] 101 | 102 | :returns: 103 | - A list of dicts (with nesting possible) 104 | """ 105 | # API uses pagination with a max of 50 per page 106 | if per_page not in range(1, 51): 107 | raise ValueError('per_page must be between 1 and 50 (inclusive)') 108 | # Don't pull 50 values if user requests 10 109 | if limit: 110 | per_page = min(per_page, limit) 111 | 112 | params = { 113 | 'limit': per_page, 114 | 'offset': offset, 115 | } 116 | page_count = 0 117 | 118 | pages = self._list(URL_BASE, params, PAGE_DESC) 119 | while pages: 120 | for page in pages: 121 | yield page 122 | page_count += 1 123 | if limit and page_count >= limit: 124 | return 125 | 126 | params['offset'] += per_page 127 | if limit and params['offset'] + per_page > limit: 128 | # Don't need to pull more than the limit 129 | params['limit'] = limit - params['offset'] 130 | pages = self._list(URL_BASE, params, PAGE_DESC) 131 | 132 | def update(self, page_id=None, **kwargs): 133 | """Update a page. 134 | 135 | >>> print(domo.pages.get(page_id)) 136 | {'id': 123456789, 'parentId': 0, 'name': 'My Page', 137 | 'locked': False, 'ownerId': 12345, 'cardIds': [], 138 | 'visibility': {'userIds': 12345}} 139 | >>> domo.pages.update(page_id, locked=True, 140 | cardIds=[54321, 13579]) 141 | >>> print(domo.pages.get(page_id)) 142 | {'id': 123456789, 'parentId': 0, 'name': 'My Page', 143 | 'locked': True, 'ownerId': 12345, 'cardIds': [54321, 13579], 144 | 'visibility': {'userIds': 12345}} 145 | 146 | # Using **kwargs: 147 | >>> page = domo.pages.get(page_id) 148 | >>> page['cardIds'].append(new_card_id) 149 | >>> domo.pages.update(**page) 150 | 151 | :Parameters: 152 | - `page_id`: ID of the page to update. Can also be provided 153 | by supplying `id` to **kwargs. This allows for calling get, 154 | updating the returned object, then passing it to update. 155 | - `name`: (optional) rename the page 156 | - `parentId`: (optional) turn page into subpage, or subpage 157 | into top-level page if parentId is present and falsey 158 | - `ownerId`: (optional) change owner of the page 159 | - `locked`: (optional) lock or unlock the page 160 | - `collectionIds`: (optional) reorder collections on page 161 | - `cardIds`: (optional) specify which cards to have on page 162 | - `visibility`: (optional) change who can see the page 163 | """ 164 | 165 | self._validate_params(kwargs, PAGE_UPDATE_KWARGS) 166 | if page_id is None and 'id' not in kwargs: 167 | raise TypeError("update() missing required argument: 'page_id'") 168 | elif 'id' in kwargs: 169 | if page_id is not None and page_id != kwargs['id']: 170 | raise ValueError('ambiguous page ID - page_id and id both ' 171 | 'supplied but do not match') 172 | page_id = kwargs['id'] 173 | del kwargs['id'] 174 | if not kwargs.get('parentId', True): 175 | # Check if parentId is present and falsey 176 | kwargs['parentId'] = 0 177 | 178 | url = '{base}/{page_id}'.format(base=URL_BASE, page_id=page_id) 179 | return self._update(url, 180 | HTTPMethod.PUT, 181 | requests.codes.NO_CONTENT, 182 | kwargs, 183 | PAGE_DESC) 184 | 185 | def delete(self, page_id): 186 | """Delete a page. 187 | 188 | :Parameters: 189 | - `page_id`: ID of the page to delete 190 | """ 191 | url = '{base}/{page_id}'.format(base=URL_BASE, page_id=page_id) 192 | return self._delete(url, PAGE_DESC) 193 | 194 | def create_collection(self, page_id, title, **kwargs): 195 | """Create a collection on a page. 196 | 197 | >>> collection = domo.pages.create_collection(page_id, 198 | 'Collection') 199 | >>> print(collection) 200 | {'id': 1234321, 'title': 'Collection', 'description': '', 201 | 'cardIds': []} 202 | 203 | :Parameters: 204 | - `page_id`: ID of the page to create a collection on 205 | - `title`: The title of the collection 206 | - `description`: (optional) The description of the collection 207 | - `cardIds`: (optional) cards to place in the collection 208 | 209 | :Returns: 210 | - A dict representing the collection 211 | """ 212 | 213 | self._validate_params(kwargs, COLLECTION_CREATION_KWARGS) 214 | collection_request = {'title': title} 215 | collection_request.update(kwargs) 216 | 217 | url = '{base}/{page_id}/collections'.format( 218 | base=URL_BASE, page_id=page_id) 219 | return self._create(url, collection_request, {}, COLLECTION_DESC) 220 | 221 | def get_collections(self, page_id): 222 | """Get a collections of a page 223 | 224 | >>> print(domo.pages.get_collections(page_id)) 225 | [{'id': 1234321, 'title': 'Collection', 'description': '', 226 | 'cardIds': []}] 227 | 228 | :Parameters: 229 | - `page_id`: ID of the page 230 | 231 | :Returns: 232 | - A list of dicts representing the collections 233 | """ 234 | url = '{base}/{page_id}/collections'.format(base=URL_BASE, 235 | page_id=page_id) 236 | return self._get(url, COLLECTION_DESC) 237 | 238 | def update_collection(self, page_id, collection_id=None, **kwargs): 239 | """Update a collection of a page. 240 | 241 | >>> collections = domo.pages.get_collections(page_id) 242 | >>> print(collections) 243 | [{'id': 1234321, 'title': 'Collection', 'description': '', 244 | 'cardIds': []}] 245 | >>> collection_id = collections[0]['id'] 246 | >>> domo.pages.update_collection(page_id, collection_id, 247 | description='description', 248 | cardIds=[54321, 13579]) 249 | >>> print(domo.pages.get_collections(page_id)) 250 | [{'id': 1234321, 'title': 'Collection', 251 | 'description': 'description', 'cardIds': [54321, 13579]}] 252 | 253 | # Using **kwargs: 254 | >>> collections = domo.pages.get_collections(page_id) 255 | >>> collections[0]['description'] = 'Description' 256 | >>> domo.pages.update_collection(page_id, **collections[0]) 257 | 258 | :Parameters: 259 | - `page_id`: ID of the page the collection is on 260 | - `collection_id`: ID of the collection. Can also be provided 261 | by supplying `id` to **kwargs. This allows for calling 262 | get_collections, updating one of the returned collections, 263 | then passing it to update_collection. 264 | - `title`: (optional) update the title 265 | - `description`: (optional) update the description 266 | - `cardIds`: (optional) update cards in the collection 267 | 268 | :Returns: 269 | - A dict representing the collection 270 | """ 271 | 272 | self._validate_params(kwargs, COLLECTION_UPDATE_KWARGS) 273 | if collection_id is None and 'id' not in kwargs: 274 | raise TypeError("update() missing required argument: " 275 | "'collection_id'") 276 | elif 'id' in kwargs: 277 | if collection_id is not None and collection_id != kwargs['id']: 278 | raise ValueError('ambiguous collection ID - collection_id and' 279 | ' id both supplied but do not match') 280 | collection_id = kwargs['id'] 281 | del kwargs['id'] 282 | 283 | url = '{base}/{page_id}/collections/{collection_id}'.format( 284 | base=URL_BASE, page_id=page_id, collection_id=collection_id) 285 | return self._update(url, 286 | HTTPMethod.PUT, 287 | requests.codes.NO_CONTENT, 288 | kwargs, 289 | COLLECTION_DESC) 290 | 291 | def delete_collection(self, page_id, collection_id): 292 | """Delete a collection from a page. 293 | 294 | :Parameters: 295 | - `page_id`: ID of the page the collection is on 296 | - `collection_id`: ID of the collection to delete 297 | """ 298 | url = '{base}/{page_id}/collections/{collection_id}'.format( 299 | base=URL_BASE, page_id=page_id, collection_id=collection_id) 300 | return self._delete(url, COLLECTION_DESC) 301 | -------------------------------------------------------------------------------- /pydomo/pages/__init__.py: -------------------------------------------------------------------------------- 1 | from .PageClient import PageClient 2 | -------------------------------------------------------------------------------- /pydomo/streams/StreamClient.py: -------------------------------------------------------------------------------- 1 | import io 2 | import os 3 | import requests 4 | 5 | from pydomo.DomoAPIClient import DomoAPIClient 6 | from pydomo.Transport import HTTPMethod 7 | 8 | """ 9 | Streams 10 | - Programmatically manage Domo Streams 11 | - A Domo Stream is a specialized upload pipeline pointing to a single Domo DataSet 12 | - Use Streams for massive, constantly changing, or rapidly growing data sources 13 | - Streams support uploading data sources in parts, in parallel 14 | - Use plain DataSets for data sources that only require occasional updates via replacement 15 | - Usage: Create a Stream, create an Execution, upload data via the Execution, then 'commit' the Execution 16 | - Docs: https://developer.domo.com/docs/data-apis/data 17 | """ 18 | 19 | 20 | class StreamClient(DomoAPIClient): 21 | def __init__(self, transport, logger): 22 | super(StreamClient, self).__init__(transport, logger) 23 | self.urlBase = '/v1/streams/' 24 | self.streamDesc = "Stream" 25 | self.executionDesc = "Execution" 26 | 27 | """ 28 | Create a Stream and DataSet 29 | - A Stream is an upload pipeline attached to the newly created DataSet 30 | - Currently, Streams cannot be applied to existing DataSets 31 | - Deleting a Stream does not delete the associated DataSet 32 | """ 33 | def create(self, stream_request): 34 | return self._create(self.urlBase, stream_request, {}, self.streamDesc) 35 | 36 | """ 37 | Get a Stream 38 | """ 39 | def get(self, stream_id): 40 | return self._get(self._base(stream_id), self.streamDesc) 41 | 42 | """ 43 | List Streams 44 | """ 45 | def list(self, limit, offset): 46 | params = { 47 | 'limit': str(limit), 48 | 'offset': str(offset), 49 | 'fields': 'all' 50 | } 51 | return self._list(self.urlBase, params, self.streamDesc) 52 | 53 | """ 54 | Update a Stream's metadata 55 | """ 56 | def update(self, stream_id, stream_update): 57 | return self._update(self._base(stream_id), HTTPMethod.PATCH, requests.codes.ok, stream_update, self.streamDesc) 58 | 59 | """ 60 | Search for Streams by property 61 | """ 62 | def search(self, stream_property): 63 | params = { 64 | 'q': str(stream_property), 65 | 'fields': 'all' 66 | } 67 | return self._list(self.urlBase + 'search', params, self.streamDesc) 68 | 69 | """ 70 | Create a Stream Execution (begin a multi-part upload process for a single Domo DataSet) 71 | - A Stream Execution is an upload pipeline that supports uploading data in chunks, or parts 72 | - Create a new Execution each time you would like to update your Domo DataSet 73 | - Be sure to 'commit' the Execution once all data parts have been uploaded 74 | - Do not create multiple Executions on a single Stream 75 | 76 | update_method - update method to use for this execution. If None, 77 | the execution will use the stream's current setting. 78 | """ 79 | def create_execution(self, stream_id, update_method=None): 80 | url = self._base(stream_id) + '/executions' 81 | body = {'updateMethod': update_method} 82 | return self._create(url, body, {}, self.executionDesc) 83 | 84 | """ 85 | Get a Stream Execution 86 | """ 87 | def get_execution(self, stream_id, execution_id): 88 | url = self._base(stream_id) + '/executions/' + str(execution_id) 89 | return self._get(url, self.streamDesc) 90 | 91 | """ 92 | List Stream Executions 93 | """ 94 | def list_executions(self, stream_id, limit, offset): 95 | url = self._base(stream_id) + '/executions' 96 | params = { 97 | 'limit': str(limit), 98 | 'offset': str(offset) 99 | } 100 | return self._list(url, params, self.executionDesc) 101 | 102 | """ 103 | Upload a data part (String or CSV) 104 | - Data sources should be broken into parts and uploaded in parallel 105 | - Parts should be around 50MB 106 | - Parts can file-like objects 107 | - Parts can be compressed 108 | """ 109 | def upload_part(self, stream_id, execution_id, part_num, csv): 110 | url = self._base(stream_id) + '/executions/' + str(execution_id) + '/part/' + str(part_num) 111 | desc = "Data Part on Execution " + str(execution_id) + " on Stream " + str(stream_id) 112 | if not isinstance(csv, io.IOBase): 113 | csv = str.encode(csv) 114 | return self._upload_csv(url, requests.codes.ok, csv, desc) 115 | 116 | """ 117 | Upload a data part CSV file 118 | - Data sources should be broken into parts and uploaded in parallel 119 | - Parts should be around 50MB 120 | - Parts can file-like objects 121 | - Parts can be compressed 122 | """ 123 | def upload_csv_part_from_file(self, stream_id, execution_id, part_num, filepath, compression): 124 | 125 | url = self._base(stream_id) + '/executions/' + str(execution_id) + '/part/' + str(part_num) 126 | desc = "Data Part on Execution " + str(execution_id) + " on Stream " + str(stream_id) 127 | 128 | if compression == 'gzip': 129 | 130 | import gzip 131 | 132 | if filepath.endswith('.gz'): 133 | with gzip.open(os.path.expanduser(filepath), 'rb') as gzipfile: 134 | return self._upload_csv(url, requests.codes.ok, gzipfile, desc) 135 | 136 | else: 137 | raise ValueError("Valid gzip extension is '.gz'") 138 | 139 | else: 140 | with open(os.path.expanduser(filepath), 'rb') as csvfile: 141 | return self._upload_csv(url, requests.codes.ok, csvfile, desc) 142 | 143 | """ 144 | Upload a data part GZIP file 145 | - Data sources should be broken into parts and uploaded in parallel 146 | - Parts should be around 50MB 147 | - Parts can file-like objects 148 | - Parts can be compressed 149 | """ 150 | def upload_gzip_part_from_file(self, stream_id, execution_id, part_num, filepath, stream_file, chunk_size): 151 | 152 | import gzip 153 | 154 | url = self._base(stream_id) + '/executions/' + str(execution_id) + '/part/' + str(part_num) 155 | desc = "Data Part on Execution " + str(execution_id) + " on Stream " + str(stream_id) 156 | 157 | if stream_file: 158 | 159 | import io 160 | 161 | compressed_body = io.BytesIO() 162 | compressed_body.name = url 163 | compressor = gzip.open(compressed_body, mode='wb') 164 | 165 | with open(filepath, 'rb') as csvfile: 166 | while True: 167 | chunk = csvfile.read(chunk_size) 168 | if not chunk: 169 | break 170 | compressor.write(chunk) 171 | 172 | compressor.flush() 173 | compressor.close() 174 | compressed_body.seek(0, 0) 175 | return self._upload_gzip(url, requests.codes.ok, compressed_body, desc) 176 | 177 | else: 178 | 179 | with open(os.path.expanduser(filepath), 'rb') as csvfile: 180 | compressed_body = gzip.compress(csvfile.read()) 181 | return self._upload_gzip(url, requests.codes.ok, compressed_body, desc) 182 | 183 | 184 | """ 185 | Commit an Execution (finalize a multi-part upload process) 186 | - Finalize a multi-part upload process by committing the execution 187 | - The execution/upload/commit process is NOT atomic; this is a RESTful approach to multi-part uploading 188 | """ 189 | def commit_execution(self, stream_id, execution_id): 190 | url = self._base(stream_id) + '/executions/' + str(execution_id) + '/commit' 191 | return self._update(url, HTTPMethod.PUT, requests.codes.ok, {}, self.executionDesc) 192 | 193 | """ 194 | Abort an Execution on a given Stream 195 | """ 196 | def abort_execution(self, stream_id, execution_id): 197 | url = self._base(stream_id) + '/executions/' + str(execution_id) + '/abort' 198 | return self._update(url, HTTPMethod.PUT, requests.codes.ok, {}, self.executionDesc) 199 | """ 200 | Abort the current Execution on a given Stream 201 | """ 202 | def abort_current_execution(self, stream_id): 203 | url = self._base(stream_id) + '/executions/abort' 204 | return self._update(url, HTTPMethod.PUT, requests.codes.no_content, {}, self.executionDesc) 205 | 206 | """ 207 | Delete a Stream 208 | - Deleting a Stream does not delete the associated DataSet 209 | """ 210 | def delete(self, stream_id): 211 | return self._delete(self._base(stream_id), self.streamDesc) 212 | -------------------------------------------------------------------------------- /pydomo/streams/StreamsModel.py: -------------------------------------------------------------------------------- 1 | from ..common import DomoObject 2 | 3 | 4 | class CreateStreamRequest(DomoObject): 5 | accepted_attrs = [ 6 | 'dataSet', 7 | 'updateMethod' 8 | ] 9 | def __init__(self, data_set_request, update_method): 10 | super().__init__() 11 | self.dataSet = data_set_request 12 | self.updateMethod = update_method 13 | 14 | 15 | class UpdateMethod: 16 | APPEND = 'APPEND' 17 | REPLACE = 'REPLACE' 18 | -------------------------------------------------------------------------------- /pydomo/streams/__init__.py: -------------------------------------------------------------------------------- 1 | from .StreamsModel import CreateStreamRequest, UpdateMethod 2 | from .StreamClient import StreamClient 3 | -------------------------------------------------------------------------------- /pydomo/users/UserClient.py: -------------------------------------------------------------------------------- 1 | from pydomo.DomoAPIClient import DomoAPIClient 2 | from pydomo.Transport import HTTPMethod 3 | import pandas as pd 4 | import requests 5 | 6 | """ 7 | User Client 8 | - Programmatically manage Domo users 9 | - Docs: https://developer.domo.com/docs/domo-apis/users 10 | """ 11 | 12 | 13 | class UserClient(DomoAPIClient): 14 | def __init__(self, transport, logger): 15 | super(UserClient, self).__init__(transport, logger) 16 | self.urlBase = '/v1/users/' 17 | self.userDesc = "User" 18 | 19 | """ 20 | Create a User 21 | """ 22 | def create(self, user_request, send_invite): 23 | params = {'sendInvite': str(send_invite)} 24 | return self._create(self.urlBase, user_request, params, self.userDesc) 25 | 26 | """ 27 | Get a User 28 | """ 29 | def get(self, user_id): 30 | return self._get(self._base(user_id), self.userDesc) 31 | 32 | """ 33 | List Users 34 | """ 35 | def list(self, limit, offset): 36 | params = { 37 | 'limit': str(limit), 38 | 'offset': str(offset), 39 | } 40 | return self._list(self.urlBase, params, self.userDesc) 41 | 42 | """ 43 | Update a User 44 | """ 45 | def update(self, user_id, user_update): 46 | return self._update(self._base(user_id), HTTPMethod.PUT, requests.codes.ok, user_update, self.userDesc) 47 | 48 | """ 49 | Delete a User 50 | """ 51 | def delete(self, user_id): 52 | return self._delete(self._base(user_id), self.userDesc) 53 | 54 | """ 55 | List all users 56 | """ 57 | def list_all(self, df_output=True,batch_size=500): 58 | i = 0 59 | n_ret = 1 60 | while n_ret > 0: 61 | ll = self.list(batch_size,batch_size*i) 62 | if( i == 0 ): 63 | all = ll 64 | else: 65 | all.extend(ll) 66 | i = i + 1 67 | n_ret = ll.__len__() 68 | 69 | out = all 70 | 71 | if( df_output ): 72 | out = pd.DataFrame(all) 73 | 74 | return(out) 75 | -------------------------------------------------------------------------------- /pydomo/users/UsersModel.py: -------------------------------------------------------------------------------- 1 | from ..common import DomoObject 2 | 3 | 4 | class CreateUserRequest(DomoObject): 5 | accepted_attrs = [ 6 | 'name', 7 | 'email', 8 | 'role' 9 | ] 10 | -------------------------------------------------------------------------------- /pydomo/users/__init__.py: -------------------------------------------------------------------------------- 1 | from pydomo.users.UsersModel import CreateUserRequest 2 | from pydomo.users.UserClient import UserClient 3 | -------------------------------------------------------------------------------- /pydomo/utilities/UtilitiesClient.py: -------------------------------------------------------------------------------- 1 | 2 | import json 3 | import math 4 | import sys 5 | import json 6 | from pandas import read_csv 7 | from pandas import to_datetime 8 | 9 | from pydomo.DomoAPIClient import DomoAPIClient 10 | from pydomo.datasets import DataSetClient 11 | from pydomo.streams import StreamClient 12 | 13 | class UtilitiesClient(DomoAPIClient): 14 | def __init__(self, transport, logger): 15 | super(UtilitiesClient, self).__init__(transport, logger) 16 | self.ds = DataSetClient(self.transport, self.logger) 17 | self.stream = StreamClient(self.transport, self.logger) 18 | self.transport = transport 19 | 20 | def domo_schema(self, ds_id): 21 | this_get = self.ds.get(ds_id) 22 | return this_get['schema']['columns'] 23 | 24 | def data_schema(self, df): 25 | col_types = dict(df.dtypes) 26 | output_list = [] 27 | for key, value in col_types.items(): 28 | output_list.append({'type': self.typeConversionText(value), 'name': key}) 29 | return output_list 30 | 31 | def typeConversionText(self, dt): 32 | result = 'STRING' 33 | if dt == ' targetSize: 90 | ch_size = math.floor(data_rows*(targetSize) / (sz/1000)) 91 | return(ch_size) 92 | 93 | def stream_upload(self, ds_id, df_up, warn_schema_change=True): 94 | domoSchema = self.domo_schema(ds_id) 95 | dataSchema = self.data_schema(df_up) 96 | 97 | stream_id = self.get_stream_id(ds_id) 98 | 99 | if self.identical(domoSchema,dataSchema) == False: 100 | new_schema = {'schema': {'columns': dataSchema}} 101 | url = '/v1/datasets/{ds}'.format(ds=ds_id) 102 | change_result = self.transport.put(url,new_schema) 103 | if warn_schema_change: 104 | print('Schema Updated') 105 | 106 | exec_info = self.stream.create_execution(stream_id) 107 | exec_id = exec_info['id'] 108 | 109 | chunksz = self.estimate_chunk_rows(df_up) 110 | start = 0 111 | df_rows = len(df_up.index) 112 | end = df_rows 113 | if df_rows > chunksz: 114 | end = chunksz 115 | 116 | for i in range(math.ceil(df_rows/chunksz)): 117 | df_sub = df_up.iloc[start:end] 118 | csv = df_sub.to_csv(header=False,index=False) 119 | self.stream.upload_part(stream_id, exec_id, start, csv) 120 | start = end 121 | end = end + chunksz 122 | if end > df_rows: 123 | end = df_rows 124 | 125 | result = self.stream.commit_execution(stream_id, exec_id) 126 | 127 | return result 128 | 129 | def stream_create(self, up_ds, name, description, updateMethod='REPLACE', keyColumnNames=[]): 130 | df_schema = self.data_schema(up_ds) 131 | req_body = {'dataSet': {'name': name, 'description': description, 'schema': {'columns': df_schema}}, 'updateMethod': updateMethod} 132 | if( updateMethod == 'UPSERT' ): 133 | req_body['keyColumnNames'] = keyColumnNames 134 | # return req_body 135 | st_created = self.transport.post('/v1/streams/', req_body, {}) 136 | return json.loads(st_created.content.decode('utf-8')) 137 | -------------------------------------------------------------------------------- /pydomo/utilities/__init__.py: -------------------------------------------------------------------------------- 1 | from .UtilitiesClient import UtilitiesClient -------------------------------------------------------------------------------- /run_examples.py: -------------------------------------------------------------------------------- 1 | ''' 2 | 3 | Domo Python SDK Usage 4 | 5 | !!! NOTICE !!!: This SDK is written for Python3. Python2 is not compatible. Execute all scripts via 'python3 myFile.py' 6 | 7 | - To run this example file: 8 | -- Copy and paste the contents of this file 9 | -- Plug in your CLIENT_ID and CLIENT_SECRET (https://developer.domo.com/manage-clients), and execute "python3 run_examples.py" 10 | 11 | - These tests clean up after themselves; several objects are created and deleted on your Domo instance 12 | - If you encounter a 'Not Allowed' error, this is a permissions issue. Please speak with your Domo Administrator. 13 | - If you encounter a 'Forbidden' error, your OAuth client is likely missing the scope required for that endpoint 14 | 15 | - Note that the Domo objects used in these examples are dictionaries that prevent you from accidentally setting the wrong fields. 16 | - Standard dictionaries may be supplied instead of the defined objects (Schema, DataSetRequest, etc) 17 | - All returned objects are dictionaries 18 | 19 | ''' 20 | 21 | import logging 22 | import random 23 | from pydomo import Domo 24 | from pydomo.datasets import DataSetRequest, Schema, Column, ColumnType, Policy 25 | from pydomo.datasets import PolicyFilter, FilterOperator, PolicyType, Sorting 26 | from pydomo.users import CreateUserRequest 27 | from pydomo.datasets import DataSetRequest, Schema, Column, ColumnType 28 | from pydomo.streams import UpdateMethod, CreateStreamRequest 29 | from pydomo.groups import CreateGroupRequest 30 | 31 | 32 | # My Domo Client ID and Secret (https://developer.domo.com/manage-clients) 33 | CLIENT_ID = 'MY_CLIENT_ID' 34 | CLIENT_SECRET = 'MY_CLIENT_SECRET' 35 | 36 | # The Domo API host domain. This can be changed as needed - for use with a proxy or test environment 37 | API_HOST = 'api.domo.com' 38 | 39 | 40 | class DomoSDKExamples: 41 | def __init__(self): 42 | domo = self.init_domo_client(CLIENT_ID, CLIENT_SECRET) 43 | self.datasets(domo) 44 | self.streams(domo) 45 | self.users(domo) 46 | self.groups(domo) 47 | self.pages(domo) 48 | 49 | ############################ 50 | # INIT SDK CLIENT 51 | ############################ 52 | def init_domo_client(self, client_id, client_secret, **kwargs): 53 | ''' 54 | 55 | - Create an API client on https://developer.domo.com 56 | - Initialize the Domo SDK with your API client id/secret 57 | - If you have multiple API clients you would like to use, simply initialize multiple Domo() instances 58 | - Docs: https://developer.domo.com/docs/domo-apis/getting-started 59 | 60 | ''' 61 | handler = logging.StreamHandler() 62 | handler.setLevel(logging.INFO) 63 | formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s') 64 | handler.setFormatter(formatter) 65 | logging.getLogger().addHandler(handler) 66 | 67 | return Domo(client_id, client_secret, logger_name='foo', log_level=logging.INFO, api_host=API_HOST, **kwargs) 68 | 69 | 70 | ############################ 71 | # DATASETS 72 | ############################ 73 | def datasets(self, domo): 74 | ''' 75 | 76 | DataSets are useful for data sources that only require occasional replacement. See the docs at 77 | https://developer.domo.com/docs/data-apis/data 78 | 79 | ''' 80 | domo.logger.info("\n**** Domo API - DataSet Examples ****\n") 81 | datasets = domo.datasets 82 | 83 | # Define a DataSet Schema 84 | dsr = DataSetRequest() 85 | dsr.name = 'Leonhard Euler Party' 86 | dsr.description = 'Mathematician Guest List' 87 | dsr.schema = Schema([Column(ColumnType.STRING, 'Friend')]) 88 | 89 | # Create a DataSet with the given Schema 90 | dataset = datasets.create(dsr) 91 | domo.logger.info("Created DataSet " + dataset['id']) 92 | 93 | # Get a DataSets's metadata 94 | retrieved_dataset = datasets.get(dataset['id']) 95 | domo.logger.info("Retrieved DataSet " + retrieved_dataset['id']) 96 | 97 | # List DataSets 98 | dataset_list = list(datasets.list(sort=Sorting.NAME, limit=10)) 99 | domo.logger.info("Retrieved a list containing {} DataSet(s)".format( 100 | len(dataset_list))) 101 | 102 | # Update a DataSets's metadata 103 | update = DataSetRequest() 104 | update.name = 'Leonhard Euler Party - Update' 105 | update.description = 'Mathematician Guest List - Update' 106 | update.schema = Schema([Column(ColumnType.STRING, 'Friend'), 107 | Column(ColumnType.STRING, 'Attending')]) 108 | updated_dataset = datasets.update(dataset['id'], update) 109 | domo.logger.info("Updated DataSet {}: {}".format(updated_dataset['id'], 110 | updated_dataset['name'])) 111 | 112 | # Import Data from a string 113 | csv_upload = '"Pythagoras","FALSE"\n"Alan Turing","TRUE"\n' \ 114 | '"George Boole","TRUE"' 115 | datasets.data_import(dataset['id'], csv_upload) 116 | domo.logger.info("Uploaded data to DataSet " + dataset['id']) 117 | 118 | # Export Data to a string 119 | include_csv_header = True 120 | csv_download = datasets.data_export(dataset['id'], include_csv_header) 121 | domo.logger.info("Downloaded data from DataSet {}:\n{}".format( 122 | dataset['id'], csv_download)) 123 | 124 | # Export Data to a file (also returns a readable/writable file object) 125 | csv_file_path = './math.csv' 126 | include_csv_header = True 127 | csv_file = datasets.data_export_to_file(dataset['id'], csv_file_path, 128 | include_csv_header) 129 | csv_file.close() 130 | domo.logger.info("Downloaded data as a file from DataSet {}".format( 131 | dataset['id'])) 132 | 133 | # Import Data from a file 134 | csv_file_path = './math.csv' 135 | datasets.data_import_from_file(dataset['id'], csv_file_path) 136 | domo.logger.info("Uploaded data from a file to DataSet {}".format( 137 | dataset['id'])) 138 | 139 | ######################################### 140 | # Personalized Data Policies (PDPs) 141 | ######################################### 142 | 143 | # Build a Policy Filter (hide sensitive columns/values from users) 144 | pdp_filter = PolicyFilter() 145 | pdp_filter.column = 'Attending' # The DataSet column to filter on 146 | pdp_filter.operator = FilterOperator.EQUALS 147 | pdp_filter.values = ['TRUE'] # The DataSet row value to filter on 148 | 149 | # Build the Personalized Data Policy (PDP) 150 | pdp_request = Policy() 151 | pdp_request.name = 'Only show friends attending the party' 152 | # A single PDP can contain multiple filters 153 | pdp_request.filters = [pdp_filter] 154 | pdp_request.type = PolicyType.USER 155 | # The affected user ids (restricted access by filter) 156 | pdp_request.users = [998, 999] 157 | # The affected group ids (restricted access by filter) 158 | pdp_request.groups = [99, 100] 159 | 160 | # Create the PDP 161 | pdp = datasets.create_pdp(dataset['id'], pdp_request) 162 | domo.logger.info("Created a Personalized Data Policy (PDP): " 163 | "{}, id: {}".format(pdp['name'], pdp['id'])) 164 | 165 | # Get a Personalized Data Policy (PDP) 166 | pdp = datasets.get_pdp(dataset['id'], pdp['id']) 167 | domo.logger.info("Retrieved a Personalized Data Policy (PDP):" 168 | " {}, id: {}".format(pdp['name'], pdp['id'])) 169 | 170 | # List Personalized Data Policies (PDP) 171 | pdp_list = datasets.list_pdps(dataset['id']) 172 | domo.logger.info("Retrieved a list containing {} PDP(s) for DataSet {}" 173 | .format(len(pdp_list), dataset['id'])) 174 | 175 | # Update a Personalized Data Policy (PDP) 176 | # Negate the previous filter (logical NOT). Note that in this case you 177 | # must treat the object as a dictionary - `pdp_filter.not` is invalid 178 | # syntax. 179 | pdp_filter['not'] = True 180 | pdp_request.name = 'Only show friends not attending the party' 181 | # A single PDP can contain multiple filters 182 | pdp_request.filters = [pdp_filter] 183 | pdp = datasets.update_pdp(dataset['id'], pdp['id'], pdp_request) 184 | domo.logger.info("Updated a Personalized Data Policy (PDP): {}, id: {}" 185 | .format(pdp['name'], pdp['id'])) 186 | 187 | # Delete a Personalized Data Policy (PDP) 188 | datasets.delete_pdp(dataset['id'], pdp['id']) 189 | domo.logger.info("Deleted a Personalized Data Policy (PDP): {}, id: {}" 190 | .format(pdp['name'], pdp['id'])) 191 | 192 | # Delete a DataSet 193 | datasets.delete(dataset['id']) 194 | domo.logger.info("Deleted DataSet {}".format(dataset['id'])) 195 | 196 | 197 | ############################ 198 | # STREAMS 199 | ############################ 200 | def streams(self, domo): 201 | '''Streams are useful for uploading massive data sources in 202 | chunks, in parallel. They are also useful with data sources that 203 | are constantly changing/growing. 204 | Streams Docs: https://developer.domo.com/docs/data-apis/data 205 | ''' 206 | domo.logger.info("\n**** Domo API - Stream Examples ****\n") 207 | streams = domo.streams 208 | 209 | # Define a DataSet Schema to populate the Stream Request 210 | dsr = DataSetRequest() 211 | dsr.name = 'Leonhard Euler Party' 212 | dsr.description = 'Mathematician Guest List' 213 | dsr.schema = Schema([Column(ColumnType.STRING, 'Friend'), 214 | Column(ColumnType.STRING, 'Attending')]) 215 | 216 | # Build a Stream Request 217 | stream_request = CreateStreamRequest(dsr, UpdateMethod.APPEND) 218 | 219 | # Create a Stream w/DataSet 220 | stream = streams.create(stream_request) 221 | domo.logger.info("Created Stream {} containing the new DataSet {}" 222 | .format(stream['id'], stream['dataSet']['id'])) 223 | 224 | # Get a Stream's metadata 225 | retrieved_stream = streams.get(stream['id']) 226 | domo.logger.info("Retrieved Stream {} containing DataSet {}".format( 227 | retrieved_stream['id'], retrieved_stream['dataSet']['id'])) 228 | 229 | # List Streams 230 | limit = 1000 231 | offset = 0 232 | stream_list = streams.list(limit, offset) 233 | domo.logger.info("Retrieved a list containing {} Stream(s)".format( 234 | len(stream_list))) 235 | 236 | # Update a Stream's metadata 237 | stream_update = CreateStreamRequest(dsr, UpdateMethod.REPLACE) 238 | updated_stream = streams.update(retrieved_stream['id'], stream_update) 239 | domo.logger.info("Updated Stream {} to update method: {}".format( 240 | updated_stream['id'], updated_stream['updateMethod'])) 241 | 242 | # Search for Streams 243 | stream_property = 'dataSource.name:' + dsr.name 244 | searched_streams = streams.search(stream_property) 245 | domo.logger.info("Stream search: there are {} Stream(s) with the DataSet " 246 | "title: {}".format(len(searched_streams), dsr.name)) 247 | 248 | # Create an Execution (Begin an upload process) 249 | execution = streams.create_execution(stream['id']) 250 | domo.logger.info("Created Execution {} for Stream {}".format( 251 | execution['id'], stream['id'])) 252 | 253 | # Get an Execution 254 | retrieved_execution = streams.get_execution(stream['id'], 255 | execution['id']) 256 | domo.logger.info("Retrieved Execution with id: {}".format( 257 | retrieved_execution['id'])) 258 | 259 | # List Executions 260 | execution_list = streams.list_executions(stream['id'], limit, offset) 261 | domo.logger.info("Retrieved a list containing {} Execution(s)".format( 262 | len(execution_list))) 263 | 264 | # Upload Data: Multiple Parts can be uploaded in parallel 265 | part = 1 266 | csv = '"Pythagoras","FALSE"\n"Alan Turing","TRUE"' 267 | execution = streams.upload_part(stream['id'], execution['id'], 268 | part, csv) 269 | 270 | part = 2 271 | csv = '"George Boole","TRUE"' 272 | execution = streams.upload_part(stream['id'], execution['id'], 273 | part, csv) 274 | 275 | # Commit the execution (End an upload process) 276 | # Executions/commits are NOT atomic 277 | committed_execution = streams.commit_execution(stream['id'], 278 | execution['id']) 279 | domo.logger.info("Committed Execution {} on Stream {}".format( 280 | committed_execution['id'], stream['id'])) 281 | 282 | # Abort a specific Execution 283 | execution = streams.create_execution(stream['id']) 284 | aborted_execution = streams.abort_execution(stream['id'], 285 | execution['id']) 286 | domo.logger.info("Aborted Execution {} on Stream {}".format( 287 | aborted_execution['id'], stream['id'])) 288 | 289 | # Abort any Execution on a given Stream 290 | streams.create_execution(stream['id']) 291 | streams.abort_current_execution(stream['id']) 292 | domo.logger.info("Aborted Executions on Stream {}".format( 293 | stream['id'])) 294 | 295 | # Delete a Stream 296 | streams.delete(stream['id']) 297 | domo.logger.info("Deleted Stream {}; the associated DataSet must be " 298 | "deleted separately".format(stream['id'])) 299 | 300 | # Delete the associated DataSet 301 | domo.datasets.delete(stream['dataSet']['id']) 302 | 303 | 304 | ############################ 305 | # USERS 306 | ############################ 307 | def users(self, domo): 308 | '''User Docs: https://developer.domo.com/docs/domo-apis/users 309 | ''' 310 | domo.logger.info("\n**** Domo API - User Examples ****\n") 311 | 312 | # Build a User 313 | user_request = CreateUserRequest() 314 | user_request.name = 'Leonhard Euler' 315 | user_request.email = 'leonhard.euler{}@domo.com'.format(random.randint(0, 10000)) 316 | user_request.role = 'Privileged' 317 | send_invite = False 318 | 319 | # Create a User 320 | user = domo.users.create(user_request, send_invite) 321 | domo.logger.info("Created User '{}'".format(user['name'])) 322 | 323 | # Get a User 324 | user = domo.users.get(user['id']) 325 | domo.logger.info("Retrieved User '" + user['name'] + "'") 326 | 327 | # List Users 328 | user_list = domo.users.list(10, 0) 329 | domo.logger.info("Retrieved a list containing {} User(s)".format( 330 | len(user_list))) 331 | 332 | # Update a User 333 | user_update = CreateUserRequest() 334 | user_update.name = 'Leo Euler' 335 | user_update.email = 'leo.euler{}@domo.com'.format(random.randint(0, 10000)) 336 | user_update.role = 'Privileged' 337 | user = domo.users.update(user['id'], user_update) 338 | domo.logger.info("Updated User '{}': {}".format(user['name'], 339 | user['email'])) 340 | 341 | # Delete a User 342 | domo.users.delete(user['id']) 343 | domo.logger.info("Deleted User '{}'".format(user['name'])) 344 | 345 | 346 | ############################ 347 | # GROUPS 348 | ############################ 349 | def groups(self, domo): 350 | '''Group Docs: 351 | https://developer.domo.com/docs/domo-apis/group-apis 352 | ''' 353 | domo.logger.info("\n**** Domo API - Group Examples ****\n") 354 | groups = domo.groups 355 | 356 | # Build a Group 357 | group_request = CreateGroupRequest() 358 | group_request.name = 'Groupy Group {}'.format(random.randint(0, 10000)) 359 | group_request.active = True 360 | group_request.default = False 361 | 362 | # Create a Group 363 | group = groups.create(group_request) 364 | domo.logger.info("Created Group '{}'".format(group['name'])) 365 | 366 | # Get a Group 367 | group = groups.get(group['id']) 368 | domo.logger.info("Retrieved Group '{}'".format(group['name'])) 369 | 370 | # List Groups 371 | group_list = groups.list(10, 0) 372 | domo.logger.info("Retrieved a list containing {} Group(s)".format( 373 | len(group_list))) 374 | 375 | # Update a Group 376 | group_update = CreateGroupRequest() 377 | group_update.name = 'Groupy Group {}'.format(random.randint(0, 10000)) 378 | group_update.active = False 379 | group_update.default = False 380 | group = groups.update(group['id'], group_update) 381 | domo.logger.info("Updated Group '{}'".format(group['name'])) 382 | 383 | # Add a User to a Group 384 | user_list = domo.users.list(10, 0) 385 | user = user_list[0] 386 | groups.add_user(group['id'], user['id']) 387 | domo.logger.info("Added User {} to Group {}".format(user['id'], 388 | group['id'])) 389 | 390 | # List Users in a Group 391 | limit = 50 392 | offset = 0 393 | user_list = groups.list_users(group['id'], limit, offset) 394 | domo.logger.info("Retrieved a User list from a Group containing {} User(s)" 395 | .format(len(user_list))) 396 | 397 | # Remove a User from a Group 398 | groups.remove_user(group['id'], user['id']) 399 | domo.logger.info("Removed User {} from Group {}".format(user['id'], 400 | group['id'])) 401 | 402 | # Delete a Group 403 | groups.delete(group['id']) 404 | domo.logger.info("Deleted group '{}'".format(group['name'])) 405 | 406 | 407 | ############################ 408 | # PAGES 409 | ############################ 410 | def pages(self, domo): 411 | '''Page Docs: https://developer.domo.com/docs/page-api-reference/page 412 | ''' 413 | domo.logger.info("\n**** Domo API - Page Examples ****\n") 414 | 415 | # Create a page 416 | page = domo.pages.create('New Page') 417 | domo.logger.info("Created Page {}".format(page['id'])) 418 | 419 | # Create a subpage 420 | subpage = domo.pages.create('Sub Page', parentId=page['id']) 421 | domo.logger.info("Created Subpage {}".format(subpage['id'])) 422 | 423 | # Update the page using returned page 424 | page['name'] = 'Updated Page' 425 | domo.pages.update(**page) 426 | domo.logger.info("Renamed Page {}".format(page['id'])) 427 | 428 | # Turn subpage into to top-level page using keyword argument 429 | domo.pages.update(subpage['id'], parentId=None) 430 | domo.logger.info("Moved Page to top level {}".format(subpage['id'])) 431 | 432 | # Get the page 433 | page = domo.pages.get(page['id']) 434 | 435 | # List pages 436 | page_list = list(domo.pages.list()) 437 | domo.logger.info("Retrieved a list of {} top-level page(s)".format( 438 | len(page_list))) 439 | 440 | # Create a few collections 441 | collections = [ 442 | domo.pages.create_collection(page['id'], 'First Collection'), 443 | domo.pages.create_collection(page['id'], 'Second Collection'), 444 | ] 445 | domo.logger.info("Created two collections on page {}".format(page['id'])) 446 | 447 | # Get collections 448 | collection_list = domo.pages.get_collections(page['id']) 449 | domo.logger.info("Retrieved a list of {} collections".format( 450 | len(collection_list))) 451 | 452 | # Update collection 453 | collections[1]['title'] = 'Last Collection' 454 | domo.pages.update_collection(page['id'], **collections[1]) 455 | domo.logger.info("Updated collection {}: {}".format(collections[1]['id'], 456 | collections[1]['title'])) 457 | 458 | # Clean up 459 | for collection in collections: 460 | domo.pages.delete_collection(page['id'], collection['id']) 461 | domo.pages.delete(page['id']) 462 | domo.pages.delete(subpage['id']) 463 | domo.logger.info("Deleted collections and pages") 464 | 465 | DomoSDKExamples() # Execute the script 466 | -------------------------------------------------------------------------------- /setup.cfg: -------------------------------------------------------------------------------- 1 | [metadata] 2 | description-file = README.md -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | from setuptools import setup, find_packages 2 | from os import path 3 | 4 | here = path.abspath(path.dirname(__file__)) 5 | description = 'The official Python3 Domo API SDK - Domo, Inc.' 6 | long_description = 'See https://github.com/domoinc/domo-python-sdk for more details.' 7 | 8 | setup( 9 | name='pydomo', 10 | version='0.3.0.13', 11 | description=description, 12 | long_description=long_description, 13 | author='Jeremy Morris', 14 | author_email='jeremy.morris@domo.com', 15 | url='https://github.com/domoinc/domo-python-sdk', 16 | download_url='https://github.com/domoinc/domo-python-sdk/tarball/0.2.2.1', 17 | keywords='domo api sdk', 18 | license='MIT', 19 | packages=find_packages(exclude=['examples']), 20 | install_requires=[ 21 | 'pandas', 22 | 'requests', 23 | 'requests_toolbelt', 24 | ], 25 | python_requires='>=3', 26 | ) 27 | --------------------------------------------------------------------------------