├── README.md ├── UserListExample.csv ├── addOfflineBasemaps.py ├── addUsersToGroups.py ├── backupContent.py ├── batchMigrateAccounts.py ├── changeOwnership.py ├── copyContent.py ├── createCsvOfGroupUsers.py ├── createCsvOfUsers.py ├── createWebMap.py ├── migrateAccount.py ├── migrateRoles.py ├── publishFiles.py ├── registerServices.py ├── removeTag.py ├── updateItemMetadata.py └── updateWebmapServices.py /README.md: -------------------------------------------------------------------------------- 1 | ## Use Cases 2 | 3 | #### [addOfflineBasemaps.py](addOfflineBasemaps.py) 4 | This example registers all of the new offline-ready basemaps [from this public group in ArcGIS Online](http://www.arcgis.com/home/group.html?owner=esri&title=Tiled%20Basemaps) and burns in the Portal admin's user credentials so that Portal users may use these maps without needing to authenticate in ArcGIS Online. These maps are required for disconnected editing support in Collector for ArcGIS. 5 | ###### Sample Usage 6 | Add all offline-ready basemaps to the folder "Offline Basemaps" in the Portal admin's account and burn in the ArcGIS Online credentials. 7 | 8 | `python addOfflineBasemaps.py addOfflineBasemaps.py -a ago_admin -p pass.word -u https://webadaptor.domain.com/arcgis -o admin -s pass.word -f 'Offline Basemaps'` 9 | 10 | #### [addUsersToGroups.py](addUsersToGroups.py) 11 | This example adds members to specific groups within the organization. This is useful if you want new members to immediately have access to the organization's relevant content when they first sign into Portal for ArcGIS. 12 | ###### Sample Usage 13 | Add `user1`, `user2`, and `user3` to all groups with the keyword "Operations" 14 | 15 | `python addUsersToGroups.py https://portal.domain.com:7443/arcgis admin password Operations user1,user2,user3` 16 | 17 | #### [batchMigrateAccounts.py](addUsersToGroups.py) 18 | This script automates the usage of [migrateAccount.py](migrateAccount.py) by reading the old account and new account names from a spreadsheet (see [UserListExample.csv](UserListExample.csv) for an example spreadsheet layout). 19 | 20 | ###### Sample Usage 21 | Migrate all content and group memberships and ownerships for the users in `UserList.csv`. 22 | 23 | `python batchMigrateAccounts.py -u https://www.arcgis.com -o admin -s adminPassword -c UserList.csv True` 24 | 25 | #### [changeOwnership.py](changeOwnership.py) 26 | This example transfers the ownership of all of the Portal for ArcGIS content owned by a member to another member. You may need to transfer ownership if you are attempting to remove a member (a member cannot be removed if they own content or groups). 27 | ###### Sample Usage 28 | Transfer ownership of all content from user `johndoe` to user `janedoe` 29 | 30 | `python changeOwnership.py https://portal.domain.com:7443/arcgis admin password johndoe janedoe` 31 | 32 | #### [copyContent.py](copyContent.py) 33 | This example copies an item from one Portal for ArcGIS (A) into another Portal for ArcGIS (B). The item ownership from Portal for ArcGIS A is transfered to the account specified in Portal for ArcGIS B. This is useful in organizations that have two portals. For example, one for internal and external use or an organization that implements a development and production environment. This script can also be used to move items from Portal for ArcGIS to ArcGIS Online and vice versa. 34 | ###### Sample Usage 35 | Copy all content owned by `johndoe` in Portal A to new user `janedoe` in Portal B 36 | 37 | `python copyContent.py https://portalA.domain.com:7443/arcgis admin password owner:johndoe https://portalB.domain.com:7443/arcgis admin password janedoe /` 38 | 39 | #### [createCsvOfGroupUsers.py](createCsvOfGroupUsers.py) 40 | This example creates a spreadsheet listing the groups and members that match the supplied group search string. The default filename is `group_details.csv` and will be created in the script directory. Optionally specify a filepath with the `-f` argument. 41 | ###### Sample Usage 42 | Create a CSV of all `public` groups with the keyword `fire`. 43 | 44 | `python createCsvOfGroupUsers.py -u https://portalA.domain.com:7443/arcgis -o admin -s password -q 'fire access:public'` 45 | 46 | #### [createCsvOfUsers.py](createCsvOfUsers.py) 47 | This example creates a spreadsheet of all the users in a portal including their full name, username, email, and role. The default filename is `portal_users.csv` and will be created in the script directory. Optionally specify a filepath with the `-f` argument. 48 | ###### Sample Usage 49 | Create a CSV of my portal's users. 50 | 51 | `python createCsvOfUsers.py -u https://portalA.domain.com:7443/arcgis -o admin -s password` 52 | 53 | #### [migrateAccount.py](migrateAccount.py) 54 | This example migrates an entire account from one user to another including their content, group memberships, and group ownerships. This may be useful if you are migrating users to enterprise logins and wish to transfer their existing account to their new enterprise account username. 55 | ###### Sample Usage 56 | Migrate all content, group memberships, and group ownerships from account `john` to account `john_enterprise`, retaining exact folder names in the new account. 57 | 58 | `python migrateAccount.py https://portal.domain.com:7443/arcgis admin password john john_enterprise True` 59 | 60 | #### [migrateRoles.py](migrateRoles.py) 61 | This example changes the role of all users with Role A to Role B. This may be useful if you are setting up custom roles and wish to transfer all users with the built-in Publisher role to the new custom role. 62 | ###### Sample Usage 63 | Migrate all users in the Portal with Role A to Role B. 64 | 65 | `python migrateRoles.py -u https://portal.domain.com:7443/arcgis -o admin -s password` 66 | 67 | #### [publishFiles.py](publishFiles.py) 68 | This example bulk publishes the files in a directory as services in you ArcGIS Online org or Portal. The status of each publishing job is written out to a spreadsheet called `results.csv` in the directory containing the files. This is useful when you have many spatial files (e.g. shapefiles) that you want to publish and share, such as in an ArcGIS Open Data site. 69 | ###### Sample Usage 70 | Upload and publish all of the zipped up shapefiles in `C:\Shapefiles` as Hosted Services in ArcGIS Online. 71 | 72 | `python publishFiles.py -u https://www.arcgis.com -o admin -s password -p 'C:\Shapefiles` 73 | 74 | #### [registerServices.py](registerServices.py) 75 | This script registers all of the services from an ArcGIS for Server REST endpoint in the user's portal. 76 | ###### Sample Usage 77 | Register the services at https://server.domain.com/arcgis/rest/services. 78 | 79 | `python registerServices.py -u https://portal.domain.com:7443/arcgis -o admin -s password -l https://server.domain.com/arcgis/rest/services` 80 | 81 | #### [updateItemMetadata.py](updateItemMetadata.py) 82 | This example updates an item's description page using an xml file with the ArcGIS metadata format. These files can be generated in ArcGIS Desktop using the [Export Metadata tool](http://desktop.arcgis.com/en/arcmap/latest/tools/conversion-toolbox/export-metadata.htm) with the `FGDC2ESRI_ISO.xml` translator. This is useful when you have detailed metadata associated with a dataset that you would like carried over when you publish it to your portal. 83 | ###### Sample Usage 84 | Update the item details for portal item `abc123` using the metadata in `new_metadata.xml`. 85 | 86 | `python updateItemMetadata.py -u https://www.arcgis.com -u admin -s password -i abc123 -p new_metadata.xml` 87 | 88 | #### [updateWebmapServices.py](updateWebmapServices.py) 89 | This example updates the URL of a map service referenced in a web map in Portal for ArcGIS. This is useful if the map service URL has changed and you don't want users to require users to remove and re-add the service to the web map. There are many reasons a service URL may change. For example, the service may have been migrated to a new server, the name of the service was changed, or the service was moved to a different folder on the server. 90 | ###### Sample Usage 91 | Replace the domain prefix `http://server.domainA.com` with `http://server.domainB.com` for all web maps in the Portal 92 | 93 | `python updateWebmapServices.py https://portal.domain.com:7443/arcgis admin password "type: Web Map" http://server.domainA.com http://server.domainB.com` 94 | -------------------------------------------------------------------------------- /UserListExample.csv: -------------------------------------------------------------------------------- 1 | Old Names,New Names user1,user1_enterprise user2,user2_enterprise -------------------------------------------------------------------------------- /addOfflineBasemaps.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python addOfflineBasemaps.py -u -o 6 | # -s -f 7 | # -a -p 8 | 9 | import urllib 10 | import json 11 | import argparse 12 | 13 | def generateToken(username, password, portalUrl): 14 | '''Retrieves a token to be used with API requests.''' 15 | parameters = urllib.urlencode({'username' : username, 16 | 'password' : password, 17 | 'client' : 'referer', 18 | 'referer': portalUrl, 19 | 'expiration': 60, 20 | 'f' : 'json'}) 21 | response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?', 22 | parameters).read() 23 | try: 24 | jsonResponse = json.loads(response) 25 | if 'token' in jsonResponse: 26 | return jsonResponse['token'] 27 | elif 'error' in jsonResponse: 28 | print jsonResponse['error']['message'] 29 | for detail in jsonResponse['error']['details']: 30 | print detail 31 | except ValueError, e: 32 | print 'An unspecified error occurred.' 33 | print e 34 | 35 | def searchPortal(portal, query=None, totalResults=None, sortField='numviews', 36 | sortOrder='desc', token=''): 37 | ''' 38 | Search the portal using the specified query and search parameters. 39 | Optionally provide a token to return results visible to that user. 40 | ''' 41 | # Default results are returned by highest 42 | # number of views in descending order. 43 | allResults = [] 44 | if not totalResults or totalResults > 100: 45 | numResults = 100 46 | else: 47 | numResults = totalResults 48 | results = __search__(portal, query, numResults, sortField, sortOrder, 0, 49 | token) 50 | 51 | if not 'error' in results.keys(): 52 | if not totalResults: 53 | totalResults = results['total'] # Return all of the results. 54 | allResults.extend(results['results']) 55 | while (results['nextStart'] > 0 and 56 | results['nextStart'] < totalResults): 57 | # Do some math to ensure it only 58 | # returns the total results requested. 59 | numResults = min(totalResults - results['nextStart'] + 1, 100) 60 | results = __search__(portal=portal, query=query, 61 | numResults=numResults, sortField=sortField, 62 | sortOrder=sortOrder, token=token, 63 | start=results['nextStart']) 64 | allResults.extend(results['results']) 65 | return allResults 66 | else: 67 | print results['error']['message'] 68 | return results 69 | 70 | def __search__(portal, query=None, numResults=100, sortField='numviews', 71 | sortOrder='desc', start=0, token=None): 72 | '''Retrieve a single page of search results.''' 73 | params = { 74 | 'q': query, 75 | 'num': numResults, 76 | 'sortField': sortField, 77 | 'sortOrder': sortOrder, 78 | 'f': 'json', 79 | 'start': start 80 | } 81 | if token: 82 | # Adding a token provides an authenticated search. 83 | params['token'] = token 84 | request = portal + '/sharing/rest/search?' + urllib.urlencode(params) 85 | results = json.loads(urllib.urlopen(request).read()) 86 | return results 87 | 88 | def groupSearch(query, portalUrl, token=''): 89 | '''Search for groups matching the specified query.''' 90 | # Example 1: query all groups owned by a user. 91 | # 'owner:johndoe' 92 | # Example 2: query groups with Operations in the name. 93 | # 'Operations' 94 | # Example 3: query all groups with public access. 95 | # 'access:public' 96 | parameters = urllib.urlencode({'q': query, 'token': token, 'f': 'json'}) 97 | request = (portalUrl + '/sharing/rest/community/groups?' + parameters) 98 | groups = json.loads(urllib.urlopen(request).read()) 99 | return groups['results'] 100 | 101 | def getUserContent(username, portalUrl, token): 102 | '''''' 103 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 104 | request = (portalUrl + '/sharing/rest/content/users/' + username + 105 | '?' + parameters) 106 | content = urllib.urlopen(request).read() 107 | return json.loads(content) 108 | 109 | def getItemDescription(itemId, portalUrl, token=''): 110 | '''Returns the description for a Portal for ArcGIS item.''' 111 | parameters = urllib.urlencode({'token' : token, 112 | 'f' : 'json'}) 113 | response = urllib.urlopen(portalUrl + "/sharing/rest/content/items/" + 114 | itemId + "?" + parameters).read() 115 | return json.loads(unicode(response, 'utf-8')) 116 | 117 | def createFolder(username, title, portalUrl, token): 118 | '''Creates a new folder in a user's content.''' 119 | parameters = urllib.urlencode({'title': title, 120 | 'token' : token, 121 | 'f' : 'json'}) 122 | response = urllib.urlopen(portalUrl + '/sharing/rest/content/users/' + 123 | username + '/createFolder?', parameters).read() 124 | return json.loads(response) 125 | 126 | def addServiceItem(username, folder, description, serviceUrl, portalUrl, 127 | token, thumbnailUrl='', serviceUsername=None, 128 | servicePassword=None): 129 | '''Creates a new item in a user's content.''' 130 | # Update the description with unicode safe values. 131 | descriptionJSON = __decodeDict__(json.loads(description)) 132 | parameters = {'item': descriptionJSON['title'], 133 | 'url': serviceUrl, 134 | 'thumbnailurl': thumbnailUrl, 135 | 'overwrite': 'false', 136 | 'token' : token, 137 | 'f' : 'json'} 138 | 139 | if serviceUsername and servicePassword: 140 | # Store the credentials with the service. 141 | parameters.update({'serviceUsername': serviceUsername, 142 | 'servicePassword': servicePassword}) 143 | 144 | # Add the item's description (with safe values for unicode). 145 | parameters.update(descriptionJSON) 146 | 147 | # Encode and post the item. 148 | postParameters = urllib.urlencode(parameters) 149 | response = urllib.urlopen(portalUrl + '/sharing/rest/content/users/' + 150 | username + '/' + folder + '/addItem?', 151 | postParameters).read() 152 | return json.loads(response) 153 | 154 | # Helper functions for decoding the unicode values in the response json. 155 | def __decodeDict__(dct): 156 | newdict = {} 157 | for k, v in dct.iteritems(): 158 | k = __safeValue__(k) 159 | v = __safeValue__(v) 160 | newdict[k] = v 161 | return newdict 162 | 163 | def __safeValue__(inVal): 164 | outVal = inVal 165 | if isinstance(inVal, unicode): 166 | outVal = inVal.encode('utf-8') 167 | elif isinstance(inVal, list): 168 | outVal = __decode_list__(inVal) 169 | return outVal 170 | 171 | def __decode_list__(lst): 172 | newList = [] 173 | for i in lst: 174 | i = __safeValue__(i) 175 | newList.append(i) 176 | return newList 177 | 178 | # Run the script. 179 | if __name__ == '__main__': 180 | parser = argparse.ArgumentParser() 181 | parser.add_argument('-u', '--portal', required=True, 182 | help=('url of the Portal (e.g. ' 183 | 'https://portal.domain.com:7443/arcgis)')) 184 | parser.add_argument('-o', '--portalAdmin', required=True, 185 | help='Portal admin username') 186 | parser.add_argument('-s', '--portalPassword', required=True, 187 | help='Portal admin password') 188 | parser.add_argument('-a', '--agoAdmin', required=True, 189 | help='ArcGIS Online admin username') 190 | parser.add_argument('-p', '--agoPassword', required=True, 191 | help='ArcGIS Online admin password') 192 | parser.add_argument('-f', '--folder', required=False, 193 | help='Optional destination folder') 194 | # Read the command line arguments. 195 | args = parser.parse_args() 196 | agoAdmin = args.agoAdmin 197 | agoPassword = args.agoPassword 198 | portal = args.portal 199 | portalAdmin = args.portalAdmin 200 | portalPassword = args.portalPassword 201 | folderTitle = args.folder 202 | 203 | # Get a token for the Portal for ArcGIS. 204 | print 'Getting token for ' + portal 205 | token = generateToken(username=portalAdmin, password=portalPassword, 206 | portalUrl=portal) 207 | 208 | # Get the destination folder ID. 209 | folderId = '' 210 | if folderTitle == None: 211 | # No folder specified. Folder is root. 212 | print 'Using the root folder...' 213 | folderId = '/' 214 | else: 215 | # Check if the folder already exists. 216 | userContent = getUserContent(portalAdmin, portal, token) 217 | for folder in userContent['folders']: 218 | if folder['title'] == folderTitle: 219 | folderId = folder['id'] 220 | # Create the folder if it was not found. 221 | if folderId == '': 222 | print 'Creating folder ' + args.folder + '...' 223 | newFolder = createFolder(portalAdmin, folderTitle, portal, token) 224 | folderId = newFolder['folder']['id'] 225 | print 'Using folder ' + folderTitle + ' (id:' + folderId + ')' 226 | 227 | # Get the ArcGIS Online group ID. 228 | query = 'owner:esri title:Tiled Basemaps' 229 | # Search for the public ArcGIS Online group (no token needed). 230 | sourceGroup = groupSearch(query, 'https://www.arcgis.com')[0]['id'] 231 | 232 | # Get the items in the ArcGIS Online group specified above. 233 | basemaps = searchPortal('https://www.arcgis.com', 'group:' + sourceGroup) 234 | 235 | # Add the basemaps as new items in the Portal. 236 | for basemap in basemaps: 237 | # Get the item description. 238 | description = getItemDescription(basemap['id'], 239 | 'https://www.arcgis.com') 240 | serviceUrl = description['url'] 241 | thumbUrl = ('https://www.arcgis.com' + 242 | '/sharing/rest/content/items/' + description['id'] + 243 | '/info/' + description['thumbnail']) 244 | 245 | newDescription = json.dumps( 246 | {'title': description['title'], 247 | 'type': description['type'], 248 | 'snippet': description['snippet'], 249 | 'description': description['description'], 250 | 'licenseInfo': description['licenseInfo'], 251 | 'tags': ','.join(description['tags']), 252 | 'typeKeywords': ','.join(description['typeKeywords']), 253 | 'accessInformation': description['accessInformation']} 254 | ) 255 | 256 | try: 257 | result = addServiceItem(portalAdmin, folderId, newDescription, 258 | serviceUrl, portal, token, thumbUrl, 259 | agoAdmin, agoPassword) 260 | if 'success' in result: 261 | print 'Successfully added ' + basemap['title'] 262 | elif 'error' in result: 263 | print 'Error copying ' + basemap['title'] 264 | print result['error']['message'] 265 | for detail in result['error']['details']: 266 | print detail 267 | else: 268 | print 'Error copying ' + basemap['title'] 269 | print 'An unhandled error occurred.' 270 | except: 271 | print 'Error copying ' + basemap['title'] 272 | print 'An unhandled exception occurred.' 273 | 274 | print 'Copying complete.' -------------------------------------------------------------------------------- /addUsersToGroups.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python addUsersToGroups.py 6 | # 7 | # should be entered as a comma separated string 8 | 9 | import urllib 10 | import json 11 | import argparse 12 | 13 | def generateToken(username, password, portalUrl): 14 | '''Retrieves a token to be used with API requests.''' 15 | parameters = urllib.urlencode({'username' : username, 16 | 'password' : password, 17 | 'client' : 'referer', 18 | 'referer': portalUrl, 19 | 'expiration': 60, 20 | 'f' : 'json'}) 21 | response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?', 22 | parameters).read() 23 | try: 24 | jsonResponse = json.loads(response) 25 | if 'token' in jsonResponse: 26 | return jsonResponse['token'] 27 | elif 'error' in jsonResponse: 28 | print jsonResponse['error']['message'] 29 | for detail in jsonResponse['error']['details']: 30 | print detail 31 | except ValueError, e: 32 | print 'An unspecified error occurred.' 33 | print e 34 | 35 | def groupSearch(query, token, portalUrl): 36 | '''Search for groups matching the specified query.''' 37 | # Example 1: query all groups owned by a user. 38 | # 'owner:johndoe' 39 | # Example 2: query groups with Operations in the name. 40 | # 'Operations' 41 | # Example 3: query all groups with public access. 42 | # 'access:public' 43 | parameters = urllib.urlencode({'q': query, 'token': token, 'f': 'json'}) 44 | request = (portalUrl + '/sharing/rest/community/groups?' + parameters) 45 | groups = json.loads(urllib.urlopen(request).read()) 46 | return groups['results'] 47 | 48 | def addUsersToGroups(users, groups, token, portalUrl): 49 | ''' 50 | REQUIRES ADMIN ACCESS. 51 | Add users to multiple groups and return a list of the status. 52 | ''' 53 | # Provide one or more usernames in a list. 54 | # e.g. ['john_doe', 'jane_doe'] 55 | # Provide one or more group IDs in a list. 56 | # e.g. ['d93aabd856f8459a8905a5bd434d4d4a', 57 | # 'f84c841a3dfc4591b1ff83281ea5025f'] 58 | 59 | toolSummary = [] 60 | 61 | # Assign users to the specified group(s). 62 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 63 | for group in groups: 64 | response = urllib.urlopen(portalUrl + 65 | '/sharing/rest/community/groups/' + 66 | group + '/addUsers?', 67 | 'users=' + ','.join(users) + "&" + 68 | parameters).read() 69 | # The response will only report back users that 70 | # were NOT successfully added. 71 | toolSummary.append({'id': group, 72 | 'results': json.loads(response)}) 73 | 74 | return toolSummary 75 | 76 | # Run the script. 77 | if __name__ == '__main__': 78 | parser = argparse.ArgumentParser() 79 | parser.add_argument('portal', 80 | help=('url of the Portal (e.g. ' 81 | 'https://portal.domain.com:7443/arcgis)')) 82 | parser.add_argument('username', help='username') 83 | parser.add_argument('password', help='password') 84 | parser.add_argument("query", help="group search string") 85 | parser.add_argument("users", help="a comma-separated string of users") 86 | # Read the command line arguments. 87 | args = parser.parse_args() 88 | portal = args.portal 89 | username = args.username 90 | password = args.password 91 | query = args.query 92 | users = (args.users).split(',') # Create a list from the input users. 93 | 94 | token = generateToken(username=username, password=password, 95 | portalUrl=portal) 96 | 97 | # Get a list of the groups matching the query. 98 | groupIDs = [] 99 | groups = groupSearch(query, token, portal) 100 | for group in groups: 101 | groupIDs.append(group['id']) 102 | 103 | results = addUsersToGroups(users=users, groups=groupIDs, token=token, 104 | portalUrl=portal) 105 | 106 | for group in results: 107 | if len(group['results']['notAdded']) > 0: 108 | # Get the group name. 109 | groupName = groupSearch('id:' + group['id'], token, 110 | portal)[0]['title'] 111 | print 'The following users were not added to ' + groupName 112 | for user in group['results']['notAdded']: 113 | print ' ' + user 114 | 115 | print 'Process complete.' -------------------------------------------------------------------------------- /backupContent.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python backupContent.py -u -o -s -q -f 6 | 7 | import urllib 8 | import json 9 | import argparse 10 | 11 | def generateToken(username, password, portalUrl): 12 | '''Retrieves a token to be used with API requests.''' 13 | parameters = urllib.urlencode({'username' : username, 14 | 'password' : password, 15 | 'client' : 'referer', 16 | 'referer': portalUrl, 17 | 'expiration': 60, 18 | 'f' : 'json'}) 19 | response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?', 20 | parameters).read() 21 | try: 22 | jsonResponse = json.loads(response) 23 | if 'token' in jsonResponse: 24 | return jsonResponse['token'] 25 | elif 'error' in jsonResponse: 26 | print jsonResponse['error']['message'] 27 | for detail in jsonResponse['error']['details']: 28 | print detail 29 | except ValueError, e: 30 | print 'An unspecified error occurred.' 31 | print e 32 | 33 | def searchPortal(portal, query=None, totalResults=None, sortField='numviews', 34 | sortOrder='desc', token=None): 35 | ''' 36 | Search the portal using the specified query and search parameters. 37 | Optionally provide a token to return results visible to that user. 38 | ''' 39 | # Default results are returned by highest 40 | # number of views in descending order. 41 | allResults = [] 42 | if not totalResults or totalResults > 100: 43 | numResults = 100 44 | else: 45 | numResults = totalResults 46 | results = __search__(portal, query, numResults, sortField, sortOrder, 0, 47 | token) 48 | 49 | if not 'error' in results.keys(): 50 | if not totalResults: 51 | totalResults = results['total'] # Return all of the results. 52 | allResults.extend(results['results']) 53 | while (results['nextStart'] > 0 and 54 | results['nextStart'] < totalResults): 55 | # Do some math to ensure it only 56 | # returns the total results requested. 57 | numResults = min(totalResults - results['nextStart'] + 1, 100) 58 | results = __search__(portal=portal, query=query, 59 | numResults=numResults, sortField=sortField, 60 | sortOrder=sortOrder, token=token, 61 | start=results['nextStart']) 62 | allResults.extend(results['results']) 63 | return allResults 64 | else: 65 | print results['error']['message'] 66 | return results 67 | 68 | def __search__(portal, query=None, numResults=100, sortField='numviews', 69 | sortOrder='desc', start=0, token=None): 70 | '''Retrieve a single page of search results.''' 71 | params = { 72 | 'q': query, 73 | 'num': numResults, 74 | 'sortField': sortField, 75 | 'sortOrder': sortOrder, 76 | 'f': 'json', 77 | 'start': start 78 | } 79 | if token: 80 | # Adding a token provides an authenticated search. 81 | params['token'] = token 82 | request = portal + '/sharing/rest/search?' + urllib.urlencode(params) 83 | results = json.loads(urllib.urlopen(request).read()) 84 | return results 85 | 86 | def getUserContent(username, portalUrl, token): 87 | '''''' 88 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 89 | request = (portalUrl + '/sharing/rest/content/users/' + username + 90 | '?' + parameters) 91 | content = json.loads(urllib.urlopen(request).read()) 92 | return content 93 | 94 | def getItemDescription(itemId, portalUrl, token): 95 | '''Returns the description for a Portal for ArcGIS item.''' 96 | parameters = urllib.urlencode({'token' : token, 97 | 'f' : 'json'}) 98 | response = urllib.urlopen(portalUrl + "/sharing/rest/content/items/" + 99 | itemId + "?" + parameters).read() 100 | return response 101 | 102 | def getItemData(itemId, portalUrl, token): 103 | '''Returns the description for a Portal for ArcGIS item.''' 104 | parameters = urllib.urlencode({'token' : token, 105 | 'f' : 'json'}) 106 | response = urllib.urlopen(portalUrl + "/sharing/rest/content/items/" + 107 | itemId + "/data?" + parameters).read() 108 | return response 109 | 110 | def backupItem(itemId, portalUrl, token, folder): 111 | description = getItemDescription(itemId, portal, token) 112 | data = getItemData(itemId, portal, token) 113 | filePath = '{}/{}'.format(folder, itemId) 114 | try: 115 | with open('{}_desc'.format(filePath), 'w') as backupDesc: 116 | backupDesc.write(description) 117 | with open('{}_data'.format(filePath), 'w') as backupData: 118 | backupData.write(data) 119 | return({'success': itemId}) 120 | except: 121 | return({'error': itemId}) 122 | 123 | # Run the script. 124 | if __name__ == '__main__': 125 | parser = argparse.ArgumentParser() 126 | parser.add_argument('-u', '--portal', 127 | help=('url of the portal (e.g. ' 128 | 'https://portal.domain.com:7443/arcgis)')) 129 | parser.add_argument('-o', '--admin', help='admin username') 130 | parser.add_argument('-s', '--password', help='admin password') 131 | parser.add_argument('-q', '--query', help='search string to find content') 132 | parser.add_argument('-f', '--folder', help='local folder to backup to') 133 | # Read the command line arguments. 134 | args = parser.parse_args() 135 | portal = args.portal 136 | admin = args.admin 137 | password = args.password 138 | query = args.query 139 | folder = args.folder 140 | 141 | # Get a token for the source Portal for ArcGIS. 142 | token = generateToken(username=admin, password=password, portalUrl=portal) 143 | 144 | # Get a list of the content matching the query. 145 | content = searchPortal(portal=portal, query=query, token=token) 146 | 147 | # Backup the content to a local folder. 148 | for item in content: 149 | try: 150 | result = backupItem(item['id'], portal, token, folder) 151 | if 'success' in result: 152 | print('backed up {}: {}'.format(item['type'], item['title'])) 153 | elif 'error' in result: 154 | print('couldn\'t backup {}: {}'.format(item['type'], item['title'])) 155 | else: 156 | print('error backing up {}: {}'.format(item['type'], item['title'])) 157 | except: 158 | print('error backing up {}: {}'.format(item['type'], item['title'])) 159 | 160 | print('Backup complete.') -------------------------------------------------------------------------------- /batchMigrateAccounts.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python batchMigrateAccounts.py -u https://www.arcgis.com -o adminUsername 6 | # -s adminPassword -c UserList.csv 7 | 8 | import argparse 9 | import csv 10 | import migrateAccount 11 | 12 | # Run the script. 13 | if __name__ == '__main__': 14 | parser = argparse.ArgumentParser() 15 | parser.add_argument('-u', '--portal', 16 | help=('url of the Portal (e.g. ' 17 | 'https://portal.domain.com:7443/arcgis)')) 18 | parser.add_argument('-o', '--username', help='admin username') 19 | parser.add_argument('-s', '--password', help='admin password') 20 | parser.add_argument('-c', '--csv', help='path to the csv') 21 | parser.add_argument('-r', '--retain', help='true or false: retain exact folder names') 22 | # Read the command line arguments. 23 | args = parser.parse_args() 24 | portal = args.portal[:-1] if args.portal[-1:] == '/' else args.portal 25 | username = args.username 26 | password = args.password 27 | csvPath = args.csv 28 | retainExactFolderNames = args.retain 29 | 30 | with open(csvPath, 'rU') as usersCsv: 31 | users = csv.reader(usersCsv, delimiter=',') 32 | next(users) # Skip the header row. 33 | for user in users: 34 | print 'Migrating {} to {}.'.format(user[0], user[1]) 35 | migrateAccount.migrateAccount(portal, username, password, 36 | user[0], user[1], retainExactFolderNames=False) 37 | -------------------------------------------------------------------------------- /changeOwnership.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python changeOwnership.py 6 | # 7 | 8 | import urllib 9 | import json 10 | import argparse 11 | 12 | def generateToken(username, password, portalUrl): 13 | '''Retrieves a token to be used with API requests.''' 14 | parameters = urllib.urlencode({'username' : username, 15 | 'password' : password, 16 | 'client' : 'referer', 17 | 'referer': portalUrl, 18 | 'expiration': 60, 19 | 'f' : 'json'}) 20 | response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?', 21 | parameters).read() 22 | try: 23 | jsonResponse = json.loads(response) 24 | if 'token' in jsonResponse: 25 | return jsonResponse['token'] 26 | elif 'error' in jsonResponse: 27 | print jsonResponse['error']['message'] 28 | for detail in jsonResponse['error']['details']: 29 | print detail 30 | except ValueError, e: 31 | print 'An unspecified error occurred.' 32 | print e 33 | 34 | def changeOwnership(itemId, newOwner, newFolder, token, portalUrl): 35 | ''' 36 | REQUIRES ADMIN ACCESS. 37 | Transfers ownership of all content from one user to another. 38 | Use '/' as the destination folder when moving content into root. 39 | ''' 40 | itemInfo = getItemInfo(itemId, token, portalUrl) 41 | params = urllib.urlencode({'targetUsername': newOwner, 42 | 'targetFoldername': newFolder, 43 | 'token' : token, 44 | 'f' : 'json'}) 45 | if not itemInfo['ownerFolder']: 46 | itemInfo['ownerFolder'] = '/' 47 | reqUrl = (portalUrl + '/sharing/rest/content/users/' + 48 | itemInfo['owner'] + '/' + itemInfo['ownerFolder'] + 49 | '/items/' + itemId + '/reassign?') 50 | response = urllib.urlopen(reqUrl, params).read() 51 | try: 52 | jsonResponse = json.loads(response) 53 | if 'success' in jsonResponse: 54 | print 'Item ' + itemId + ' has been transferred.' 55 | elif 'error' in jsonResponse: 56 | print 'Error transferring item ' + itemId + '.' 57 | for detail in jsonResponse['error']['details']: 58 | print detail 59 | except ValueError, e: 60 | print 'An unspecified error occurred.' 61 | print e 62 | 63 | def getUserContent(username, folder, token, portalUrl): 64 | '''Returns a list of all folders for the specified user.''' 65 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 66 | request = (portalUrl + '/sharing/rest/content/users/' + username + 67 | '/' + folder + '?' + parameters) 68 | userContent = json.loads(urllib.urlopen(request).read()) 69 | return userContent 70 | 71 | def getItemInfo(itemId, token, portalUrl): 72 | '''Returns general information about the item.''' 73 | params = urllib.urlencode({'token' : token, 74 | 'f' : 'json'}) 75 | itemInfo = json.loads(urllib.urlopen(portalUrl + 76 | '/sharing/content/items/' + 77 | itemId + '?' + params).read()) 78 | return itemInfo 79 | 80 | # Run the script. 81 | if __name__ == '__main__': 82 | parser = argparse.ArgumentParser() 83 | parser.add_argument('portal', 84 | help=('url of the Portal (e.g. ' 85 | 'https://portal.domain.com:7443/arcgis)')) 86 | parser.add_argument('username', help='username') 87 | parser.add_argument('password', help='password') 88 | parser.add_argument("oldOwner", help="source account to migrate from") 89 | parser.add_argument("newOwner", help="destination account to migrate to") 90 | # Read the command line arguments. 91 | args = parser.parse_args() 92 | portal = args.portal 93 | username = args.username 94 | password = args.password 95 | oldOwner = args.oldOwner 96 | newOwner = args.newOwner 97 | 98 | # Sample usage 99 | token = generateToken(username=username, password=password, 100 | portalUrl=portal) 101 | 102 | # Get a list of the oldOwner's folders and any items in root. 103 | userContent = getUserContent(oldOwner, '/', token, 104 | portalUrl=portal) 105 | 106 | # *** CAUTION *** 107 | # The following code will transfer ownership of ALL CONTENT 108 | # from oldOwner to newOwner. 109 | # Be sure you are absolutely sure you want to do this before proceeding. 110 | if not 'items' in userContent: 111 | print oldOwner + ' doesn\'t have any content visible to this account.' 112 | print 'Be sure you are signed in as admin.' 113 | else: 114 | for item in userContent['items']: 115 | changeOwnership(item['id'], newOwner, '/', token=token, 116 | portalUrl=portal) 117 | for folder in userContent['folders']: 118 | folderContent = getUserContent(oldOwner, folder['id'], 119 | token=token, portalUrl=portal) 120 | for item in folderContent['items']: 121 | changeOwnership(item['id'], newOwner, folder['title'], 122 | token=token, portalUrl=portal) 123 | print 'Migration completed.' 124 | print 'All items transferred from {0} to {1}'.format(oldOwner, 125 | newOwner) 126 | 127 | -------------------------------------------------------------------------------- /copyContent.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python copyContent.py 6 | # 7 | # 8 | 9 | import urllib 10 | import json 11 | import argparse 12 | 13 | def generateToken(username, password, portalUrl): 14 | '''Retrieves a token to be used with API requests.''' 15 | parameters = urllib.urlencode({'username' : username, 16 | 'password' : password, 17 | 'client' : 'referer', 18 | 'referer': portalUrl, 19 | 'expiration': 60, 20 | 'f' : 'json'}) 21 | response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?', 22 | parameters).read() 23 | try: 24 | jsonResponse = json.loads(response) 25 | if 'token' in jsonResponse: 26 | return jsonResponse['token'] 27 | elif 'error' in jsonResponse: 28 | print jsonResponse['error']['message'] 29 | for detail in jsonResponse['error']['details']: 30 | print detail 31 | except ValueError, e: 32 | print 'An unspecified error occurred.' 33 | print e 34 | 35 | def searchPortal(portal, query=None, totalResults=None, sortField='numviews', 36 | sortOrder='desc', token=None): 37 | ''' 38 | Search the portal using the specified query and search parameters. 39 | Optionally provide a token to return results visible to that user. 40 | ''' 41 | # Default results are returned by highest 42 | # number of views in descending order. 43 | allResults = [] 44 | if not totalResults or totalResults > 100: 45 | numResults = 100 46 | else: 47 | numResults = totalResults 48 | results = __search__(portal, query, numResults, sortField, sortOrder, 0, 49 | token) 50 | 51 | if not 'error' in results.keys(): 52 | if not totalResults: 53 | totalResults = results['total'] # Return all of the results. 54 | allResults.extend(results['results']) 55 | while (results['nextStart'] > 0 and 56 | results['nextStart'] < totalResults): 57 | # Do some math to ensure it only 58 | # returns the total results requested. 59 | numResults = min(totalResults - results['nextStart'] + 1, 100) 60 | results = __search__(portal=portal, query=query, 61 | numResults=numResults, sortField=sortField, 62 | sortOrder=sortOrder, token=token, 63 | start=results['nextStart']) 64 | allResults.extend(results['results']) 65 | return allResults 66 | else: 67 | print results['error']['message'] 68 | return results 69 | 70 | def __search__(portal, query=None, numResults=100, sortField='numviews', 71 | sortOrder='desc', start=0, token=None): 72 | '''Retrieve a single page of search results.''' 73 | params = { 74 | 'q': query, 75 | 'num': numResults, 76 | 'sortField': sortField, 77 | 'sortOrder': sortOrder, 78 | 'f': 'json', 79 | 'start': start 80 | } 81 | if token: 82 | # Adding a token provides an authenticated search. 83 | params['token'] = token 84 | request = portal + '/sharing/rest/search?' + urllib.urlencode(params) 85 | results = json.loads(urllib.urlopen(request).read()) 86 | return results 87 | 88 | def getUserContent(username, portalUrl, token): 89 | '''''' 90 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 91 | request = (portalUrl + '/sharing/rest/content/users/' + username + 92 | '?' + parameters) 93 | content = json.loads(urllib.urlopen(request).read()) 94 | return content 95 | 96 | def getItemDescription(itemId, portalUrl, token): 97 | '''Returns the description for a Portal for ArcGIS item.''' 98 | parameters = urllib.urlencode({'token' : token, 99 | 'f' : 'json'}) 100 | response = urllib.urlopen(portalUrl + "/sharing/rest/content/items/" + 101 | itemId + "?" + parameters).read() 102 | return response 103 | 104 | def getItemData(itemId, portalUrl, token): 105 | '''Returns the description for a Portal for ArcGIS item.''' 106 | parameters = urllib.urlencode({'token' : token, 107 | 'f' : 'json'}) 108 | response = urllib.urlopen(portalUrl + "/sharing/rest/content/items/" + 109 | itemId + "/data?" + parameters).read() 110 | return response 111 | 112 | def addItem(username, folder, description, data, portalUrl, token, 113 | thumbnailUrl=''): 114 | '''Creates a new item in a user's content.''' 115 | parameters = urllib.urlencode({'item': json.loads(description)['title'], 116 | 'text': data, 117 | 'overwrite': 'false', 118 | 'thumbnailurl': thumbnailUrl, 119 | 'token' : token, 120 | 'f' : 'json'}) 121 | postParameters = (urllib.urlencode(json.loads(description)) 122 | .replace('None', '') + '&' + parameters) 123 | response = urllib.urlopen(portalUrl + '/sharing/rest/content/users/' + 124 | username + '/' + folder + '/addItem?', 125 | postParameters).read() 126 | return response 127 | 128 | def copyItem(itemId, destinationOwner, destinationFolder, sourcePortal, 129 | sourceToken, destinationPortal=None, destinationToken=None): 130 | ''' 131 | Copies an item into a user's account in the specified folder. 132 | Use '/' as the destination folder when moving content into root. 133 | ''' 134 | # This function defaults to copying into the same Portal 135 | # and with the same token permissions as the source account. 136 | # If the destination is a different Portal then specify that Portal url 137 | # and include a token with permissions on that instance. 138 | if not destinationPortal: 139 | destinationPortal = sourcePortal 140 | if not destinationToken: 141 | destinationToken = sourceToken 142 | description = getItemDescription(itemId, sourcePortal, sourceToken) 143 | data = getItemData(itemId, sourcePortal, sourceToken) 144 | thumbnail = json.loads(description)['thumbnail'] 145 | if thumbnail: 146 | thumbnailUrl = (sourcePortal + '/sharing/rest/content/items/' + 147 | itemId + '/info/' + thumbnail + 148 | '?token=' + sourceToken) 149 | else: 150 | thumbnailUrl = '' 151 | status = addItem(destinationOwner, destinationFolder, description, 152 | data, destinationPortal, destinationToken, 153 | thumbnailUrl) 154 | return json.loads(status) 155 | 156 | # Run the script. 157 | if __name__ == '__main__': 158 | parser = argparse.ArgumentParser() 159 | parser.add_argument('sourcePortal', 160 | help=('url of the source Portal (e.g. ' 161 | 'https://portal.domain.com:7443/arcgis)')) 162 | parser.add_argument('sourceAdmin', help='source admin username') 163 | parser.add_argument('sourcePassword', help='source admin password') 164 | parser.add_argument("query", help='search string to find content') 165 | parser.add_argument('destPortal', 166 | help=('url of the destination Portal (e.g. ' 167 | 'https://portal.domain.com:7443/arcgis)')) 168 | parser.add_argument('destAdmin', help='destination admin username') 169 | parser.add_argument('destPassword', help='destination admin password') 170 | parser.add_argument('owner', help='destination account to copy to') 171 | parser.add_argument('folder', help='destination folder to copy to') 172 | # Read the command line arguments. 173 | args = parser.parse_args() 174 | sourcePortal = args.sourcePortal 175 | sourceAdmin = args.sourceAdmin 176 | sourcePassword = args.sourcePassword 177 | query = args.query 178 | destPortal = args.destPortal 179 | destAdmin = args.destAdmin 180 | destPassword = args.destPassword 181 | owner = args.owner 182 | folder = args.folder 183 | 184 | # Get a token for the source Portal for ArcGIS. 185 | sourceToken = generateToken(username=sourceAdmin, password=sourcePassword, 186 | portalUrl=sourcePortal) 187 | 188 | # Get a token for the destination Portal for ArcGIS. 189 | destToken = generateToken(username=destAdmin, password=destPassword, 190 | portalUrl=destPortal) 191 | 192 | # Get the destination folder ID. 193 | folderId = '' 194 | destUser = getUserContent(owner, destPortal, destToken) 195 | for folder in destUser['folders']: 196 | if folder['title'] == folder: 197 | folderId = folder['id'] 198 | 199 | # Get a list of the content matching the query. 200 | content = searchPortal(portal=sourcePortal, 201 | query=query, 202 | token=sourceToken) 203 | 204 | # Copy the content into the destination user's account. 205 | for item in content: 206 | try: 207 | result = copyItem(item['id'], owner, folderId, sourcePortal, 208 | sourceToken, destPortal, destToken) 209 | if 'success' in result: 210 | print 'successfully copied ' + item['type'] + ': ' + item['title'] 211 | elif 'error' in result: 212 | print 'error copying ' + item['type'] + ': ' + item['title'] 213 | print result['error']['message'] 214 | for detail in result['error']['details']: 215 | print detail 216 | else: 217 | print 'error copying ' + item['type'] + ': ' + item['title'] 218 | except: 219 | 'error copying ' + item['type'] + ': ' + item['title'] 220 | 221 | print 'Copying complete.' -------------------------------------------------------------------------------- /createCsvOfGroupUsers.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python createCsvOfGroupUsers.py -o -u -s 6 | # -q -f 7 | 8 | import urllib 9 | import json 10 | import argparse 11 | import csv 12 | 13 | def generateToken(username, password, portalUrl): 14 | '''Retrieves a token to be used with API requests.''' 15 | parameters = urllib.urlencode( 16 | {'username' : username, 'password' : password, 'client' : 'referer', 17 | 'referer': portalUrl, 'expiration': 60, 'f' : 'json'} 18 | ) 19 | response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?', 20 | parameters).read() 21 | try: 22 | jsonResponse = json.loads(response) 23 | if 'token' in jsonResponse: 24 | return jsonResponse['token'] 25 | elif 'error' in jsonResponse: 26 | print(jsonResponse['error']['message']) 27 | for detail in jsonResponse['error']['details']: 28 | print(detail) 29 | except ValueError, e: 30 | print('An unspecified error occurred.') 31 | print(e) 32 | 33 | def searchGroups(query, token, portalUrl): 34 | '''Returns a list of all folders for the specified user.''' 35 | ## Currently does more than 100 groups. 36 | parameters = urllib.urlencode( 37 | {'q': query, 'num': 100, 'token': token, 'f': 'json'}) 38 | request = '{}/sharing/rest/community/groups?{}'.format( 39 | portalUrl, parameters 40 | ) 41 | groups = json.loads(urllib.urlopen(request).read()) 42 | return groups 43 | 44 | def getGroupUsers(groupId, token, portalUrl): 45 | '''Lists the users, owner, and administrators of a given group.''' 46 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 47 | request = '{}/sharing/rest/community/groups/{}/users?{}'.format( 48 | portalUrl, groupId, parameters 49 | ) 50 | users = json.loads(urllib.urlopen(request).read()) 51 | return users 52 | 53 | def getRoles(token, portalUrl): 54 | '''Get the custom roles available in the portal.''' 55 | ## Does not handle more than 100 roles. 56 | parameters = urllib.urlencode({ 57 | 'token': token, 'f': 'json', 'num': 100 58 | }) 59 | request = '{}/sharing/rest/portals/self/roles?{}'.format( 60 | portalUrl, parameters 61 | ) 62 | roles = json.loads(urllib.urlopen(request).read()) 63 | return roles 64 | 65 | def getUser(username, token, portalUrl): 66 | '''A user resource representing a registered user of the portal.''' 67 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 68 | request = '{}/sharing/rest/community/users/{}?{}'.format( 69 | portalUrl, username, parameters 70 | ) 71 | user = json.loads(urllib.urlopen(request).read()) 72 | return user 73 | 74 | # Run the script. 75 | if __name__ == '__main__': 76 | parser = argparse.ArgumentParser() 77 | parser.add_argument('-u', '--portal', required=True, 78 | help=('url of the portal ' 79 | '(e.g. https://portal.domain.com:7443/arcgis)')) 80 | parser.add_argument('-o', '--username', required=True, help='username') 81 | parser.add_argument('-s', '--password', required=True, help='password') 82 | parser.add_argument('-q', '--query', required=True, 83 | help='group query to search') 84 | parser.add_argument('-f', '--filename', required=False, 85 | default='group_details.csv', 86 | help=('the path to the file to create ' 87 | '(default: group_details.csv')) 88 | # Read the command line arguments. 89 | args = parser.parse_args() 90 | portal = args.portal[:-1] if args.portal[-1:] == '/' else args.portal 91 | username = args.username 92 | password = args.password 93 | query = args.query 94 | filename = args.filename 95 | 96 | token = generateToken(username, password, portal) 97 | 98 | groups = searchGroups(query, token, portal) 99 | 100 | print('Found {} group(s) with search "{}"'.format(groups['total'], query)) 101 | 102 | customRoles = getRoles(token, portal) 103 | roles = {} 104 | for role in customRoles['roles']: 105 | roles[role['id']] = role['name'] 106 | 107 | outputFile = filename 108 | with open(outputFile, 'wb') as output: 109 | dataWriter = csv.writer( 110 | output, 111 | delimiter=',', 112 | quotechar='"', 113 | quoting=csv.QUOTE_MINIMAL 114 | ) 115 | # Write header row. 116 | dataWriter.writerow( 117 | ['Group Name', 'Full Name', 'Email', 'Username', 'Role'] 118 | ) 119 | 120 | for group in groups['results']: 121 | users = getGroupUsers(group['id'], token, portal) 122 | print('Processing {}'.format(group['title'])) 123 | groupMembers = [] 124 | for user in users['users']: 125 | groupMembers.append(user) 126 | for admin in users['admins']: 127 | groupMembers.append(admin) 128 | for member in groupMembers: 129 | userInfo = getUser(member, token, portal) 130 | userData = { 131 | 'fullName': '', 132 | 'username': '', 133 | 'email': '', 134 | 'role': '' 135 | } 136 | for attr in ['fullName', 'username', 'email', 'role']: 137 | try: 138 | userData[attr] = userInfo[attr].encode('utf-8') 139 | except: 140 | pass 141 | if 'roleId' in userInfo: 142 | userData['role'] = roles[userInfo['roleId']] 143 | dataWriter.writerow([ 144 | group['title'], 145 | userData['fullName'], 146 | userData['email'], 147 | userData['username'], 148 | userData['role'] 149 | ]) 150 | 151 | print('\nFinished creating {}'.format(filename)) -------------------------------------------------------------------------------- /createCsvOfUsers.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python createCsvOfUsers.py -u -o -s 6 | # -f 7 | 8 | import urllib 9 | import json 10 | import argparse 11 | import csv 12 | 13 | def generateToken(username, password, portalUrl): 14 | '''Retrieves a token to be used with API requests.''' 15 | parameters = urllib.urlencode( 16 | {'username' : username, 'password' : password, 'client' : 'referer', 17 | 'referer': portalUrl, 'expiration': 60, 'f' : 'json'} 18 | ) 19 | response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?', 20 | parameters).read() 21 | try: 22 | jsonResponse = json.loads(response) 23 | if 'token' in jsonResponse: 24 | return jsonResponse['token'] 25 | elif 'error' in jsonResponse: 26 | print(jsonResponse['error']['message']) 27 | for detail in jsonResponse['error']['details']: 28 | print(detail) 29 | except ValueError, e: 30 | print('An unspecified error occurred.') 31 | print(e) 32 | 33 | def getRoles(token, portalUrl): 34 | '''Get the custom roles available in the portal.''' 35 | ## Does not handle more than 100 roles. 36 | parameters = urllib.urlencode({ 37 | 'token': token, 'f': 'json', 'num': 100 38 | }) 39 | request = '{}/sharing/rest/portals/self/roles?{}'.format( 40 | portalUrl, parameters 41 | ) 42 | roles = json.loads(urllib.urlopen(request).read()) 43 | return roles 44 | 45 | def getUsers(token, portalUrl): 46 | ''' 47 | Returns a list of all users in the organization (requires admin access). 48 | ''' 49 | def __portalId__(token, portalUrl): 50 | ''' 51 | Return the id of the portal. If null, then this is the 52 | default portal for anonymous and non-organizational users. 53 | ''' 54 | parameters = urllib.urlencode( 55 | {'token': token, 'f': 'json'} 56 | ) 57 | request = '{}/sharing/rest/portals/self?{}'.format( 58 | portalUrl, parameters 59 | ) 60 | response = json.loads(urllib.urlopen(request).read()) 61 | return response['id'] 62 | 63 | def __users__(portalId, start=0): 64 | '''Retrieve a single page of users.''' 65 | parameters = urllib.urlencode( 66 | {'token': token, 'f': 'json', 'start': start, 'num': 100} 67 | ) 68 | request = '{}/sharing/rest/portals/{}/users?{}'.format( 69 | portalUrl, portalId, parameters 70 | ) 71 | users = json.loads(urllib.urlopen(request).read()) 72 | return users 73 | 74 | portalId = __portalId__(token, portalUrl) 75 | allUsers = [] 76 | users = __users__(portalId) 77 | for user in users['users']: 78 | allUsers.append(user) 79 | while users['nextStart'] > 0: 80 | users = __users__(portalId, users['nextStart']) 81 | for user in users['users']: 82 | allUsers.append(user) 83 | return allUsers 84 | 85 | def getUser(username, token, portalUrl): 86 | '''A user resource representing a registered user of the portal.''' 87 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 88 | request = '{}/sharing/rest/community/users/{}?{}'.format( 89 | portalUrl, username, parameters 90 | ) 91 | user = json.loads(urllib.urlopen(request).read()) 92 | return user 93 | 94 | # Run the script. 95 | if __name__ == '__main__': 96 | parser = argparse.ArgumentParser() 97 | parser.add_argument('-u', '--portal', required=True, 98 | help=('url of the portal ' 99 | '(e.g. https://portal.domain.com:7443/arcgis)')) 100 | parser.add_argument('-o', '--username', required=True, help='username') 101 | parser.add_argument('-s', '--password', required=True, help='password') 102 | parser.add_argument('-f', '--filename', required=False, 103 | default='portal_users.csv', 104 | help=('the path to the file to create ' 105 | '(default: portal_users.csv')) 106 | # Read the command line arguments. 107 | args = parser.parse_args() 108 | portal = args.portal[:-1] if args.portal[-1:] == '/' else args.portal 109 | username = args.username 110 | password = args.password 111 | filename = args.filename 112 | 113 | token = generateToken(username, password, portal) 114 | # Create a dictionary of the portal's roles. 115 | customRoles = getRoles(token, portal) 116 | roles = {} 117 | for role in customRoles['roles']: 118 | roles[role['id']] = role['name'] 119 | # Add the built-in roles to the list. 120 | 121 | roles.update({ 122 | 'account_user': 'User (built-in)', 123 | 'account_publisher': 'Publisher (built-in)', 124 | 'account_admin': 'Admin (built-in)', 125 | 'org_user': 'User (built-in)', 126 | 'org_publisher': 'Publisher (built-in)', 127 | 'org_admin': 'Admin (built-in)' 128 | }) 129 | 130 | users = getUsers(token, portal) 131 | 132 | outputFile = filename 133 | with open(outputFile, 'wb') as output: 134 | dataWriter = csv.writer( 135 | output, 136 | delimiter=',', 137 | quotechar='"', 138 | quoting=csv.QUOTE_MINIMAL 139 | ) 140 | # Write header row. 141 | dataWriter.writerow( 142 | ['Full Name', 'Username', 'Email', 'Role'] 143 | ) 144 | 145 | print('Found {} users.'.format(len(users))) 146 | 147 | for user in users: 148 | userData = {} 149 | for attr in ['fullName', 'username', 'email', 'role']: 150 | try: 151 | userData[attr] = user[attr].encode('utf-8') 152 | except: 153 | pass 154 | userData['role'] = roles[user['role']] 155 | dataWriter.writerow([ 156 | userData['fullName'], 157 | userData['username'], 158 | userData['email'], 159 | userData['role'] 160 | ]) 161 | 162 | print('\nFinished creating {}'.format(filename)) -------------------------------------------------------------------------------- /createWebMap.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python createWebMap.py -p -u -s -a 6 | 7 | import urllib 8 | import json 9 | import argparse 10 | 11 | def generateToken(username, password, portalUrl): 12 | '''Retrieves a token to be used with API requests.''' 13 | parameters = urllib.urlencode({'username' : username, 14 | 'password' : password, 15 | 'client' : 'referer', 16 | 'referer': portalUrl, 17 | 'expiration': 60, 18 | 'f' : 'json'}) 19 | response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?', 20 | parameters).read() 21 | try: 22 | jsonResponse = json.loads(response) 23 | if 'token' in jsonResponse: 24 | return jsonResponse['token'] 25 | elif 'error' in jsonResponse: 26 | print jsonResponse['error']['message'] 27 | for detail in jsonResponse['error']['details']: 28 | print detail 29 | except ValueError, e: 30 | print 'An unspecified error occurred.' 31 | print e 32 | 33 | def addItem(username, folder, description, data, portalUrl, token, 34 | thumbnailUrl=''): 35 | '''Creates a new item in a user's content.''' 36 | parameters = urllib.urlencode({'item': description['title'], 37 | 'text': data, 38 | 'overwrite': 'false', 39 | 'thumbnailurl': thumbnailUrl, 40 | 'token' : token, 41 | 'f' : 'json'}) 42 | postParameters = (urllib.urlencode(description) 43 | .replace('None', '') + '&' + parameters) 44 | response = urllib.urlopen(portalUrl + '/sharing/rest/content/users/' + 45 | username + '/' + folder + '/addItem?', 46 | postParameters).read() 47 | return response 48 | 49 | # Run the script. 50 | if __name__ == '__main__': 51 | parser = argparse.ArgumentParser() 52 | parser.add_argument('-p', '--portal', required=True, help=('url of the Portal')) 53 | parser.add_argument('-u', '--username', required=True, help='username') 54 | parser.add_argument('-s', '--password', required=True, help='password') 55 | parser.add_argument('-a', '--services', required=True, help='comma separated string of services') 56 | # Read the command line arguments. 57 | args = parser.parse_args() 58 | portal = args.portal 59 | username = args.username 60 | password = args.password 61 | services = args.services 62 | 63 | webmapDescription = { 64 | "title": "test", 65 | "type": "Web Map", 66 | "tags": ["test"] 67 | } 68 | 69 | webmapJson = """{ 70 | "operationalLayers": [], 71 | "baseMap": { 72 | "baseMapLayers": [ 73 | { 74 | "id": "defaultBasemap", 75 | "layerType": "ArcGISTiledMapServiceLayer", 76 | "url": "http://services.arcgisonline.com/ArcGIS/rest/services/World_Topo_Map/MapServer", 77 | "visibility": "true", 78 | "opacity": 1, 79 | "title": "Topographic" 80 | } 81 | ], 82 | "title": "Topographic" 83 | }, 84 | "spatialReference": { 85 | "wkid": 102100, 86 | "latestWkid": 3857 87 | }, 88 | "version": "2.4" 89 | }""" 90 | 91 | webmap = json.loads(webmapJson) 92 | token = token = generateToken(username, password, portal) 93 | 94 | serviceList = services.split(',') 95 | for service in serviceList: 96 | layer = json.loads('{}') 97 | layer['url'] = service 98 | layer['title'] = "service 1" 99 | layer['opacity'] = 1 100 | 101 | webmap['operationalLayers'].append(layer) 102 | 103 | webmapJson = json.dumps(webmap) 104 | 105 | print(addItem(username, '/', webmapDescription, webmapJson, portal, token)) 106 | -------------------------------------------------------------------------------- /migrateAccount.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python migrateAccount.py 6 | 7 | import urllib 8 | import json 9 | import argparse 10 | 11 | def generateToken(username, password, portalUrl): 12 | '''Retrieves a token to be used with API requests.''' 13 | parameters = urllib.urlencode({'username' : username, 14 | 'password' : password, 15 | 'client' : 'referer', 16 | 'referer': portalUrl, 17 | 'expiration': 60, 18 | 'f' : 'json'}) 19 | response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?', 20 | parameters).read() 21 | try: 22 | jsonResponse = json.loads(response) 23 | if 'token' in jsonResponse: 24 | return jsonResponse['token'] 25 | elif 'error' in jsonResponse: 26 | print jsonResponse['error']['message'] 27 | for detail in jsonResponse['error']['details']: 28 | print detail 29 | except ValueError, e: 30 | print 'An unspecified error occurred.' 31 | print e 32 | 33 | def changeOwnership(itemId, newOwner, newFolder, token, portalUrl): 34 | ''' 35 | REQUIRES ADMIN ACCESS. 36 | Transfers ownership of all content from one user to another. 37 | Use '/' as the destination folder when moving content into root. 38 | ''' 39 | itemInfo = getItemInfo(itemId, token, portalUrl) 40 | params = urllib.urlencode({'targetUsername': newOwner, 41 | 'targetFoldername': newFolder, 42 | 'token' : token, 43 | 'f' : 'json'}) 44 | if not itemInfo['ownerFolder']: 45 | itemInfo['ownerFolder'] = '/' 46 | reqUrl = (portalUrl + '/sharing/rest/content/users/' + 47 | itemInfo['owner'] + '/' + itemInfo['ownerFolder'] + 48 | '/items/' + itemId + '/reassign?') 49 | response = urllib.urlopen(reqUrl, params).read() 50 | try: 51 | jsonResponse = json.loads(response) 52 | if 'success' in jsonResponse: 53 | print 'Item ' + itemId + ' has been transferred.' 54 | elif 'error' in jsonResponse: 55 | print 'Error transferring item ' + itemId + '.' 56 | for detail in jsonResponse['error']['details']: 57 | print detail 58 | except ValueError, e: 59 | print 'An unspecified error occurred.' 60 | print e 61 | 62 | def addUsersToGroups(users, groups, token, portalUrl): 63 | ''' 64 | REQUIRES ADMIN ACCESS. 65 | Add users to multiple groups and return a list of the status. 66 | ''' 67 | # Provide one or more usernames in a list. 68 | # e.g. ['john_doe', 'jane_doe'] 69 | # Provide one or more group IDs in a list. 70 | # e.g. ['d93aabd856f8459a8905a5bd434d4d4a', 71 | # 'f84c841a3dfc4591b1ff83281ea5025f'] 72 | summary = [] 73 | 74 | # Assign users to the specified group(s). 75 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 76 | for group in groups: 77 | response = urllib.urlopen(portalUrl + 78 | '/sharing/rest/community/groups/' + 79 | group + '/addUsers?', 80 | 'users=' + ','.join(users) + '&' + 81 | parameters).read() 82 | # The response will only report back users that 83 | # were NOT successfully added. 84 | summary.append({'id': group, 'results': json.loads(response)}) 85 | 86 | return summary 87 | 88 | def reassignGroups(newOwner, groups, token, portalUrl): 89 | ''' 90 | REQUIRES ADMIN ACCESS. 91 | Changes the owner of a group. 92 | ''' 93 | # Provide one or more group IDs in a list. 94 | # e.g. ['d93aabd856f8459a8905a5bd434d4d4a', 95 | # 'f84c841a3dfc4591b1ff83281ea5025f'] 96 | summary = [] 97 | 98 | # Assign users to the specified group(s). 99 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 100 | for group in groups: 101 | request = (portalUrl + '/sharing/rest/community/groups/' + 102 | group + '/reassign?' + parameters) 103 | postData = {'targetUsername': newOwner} 104 | response = urllib.urlopen(request, urllib.urlencode(postData)).read() 105 | summary.append({'id': group, 'results': json.loads(response)}) 106 | 107 | return summary 108 | 109 | def getUserContent(username, folder, token, portalUrl): 110 | '''Returns a list of all folders for the specified user.''' 111 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 112 | request = (portalUrl + '/sharing/rest/content/users/' + username + 113 | '/' + folder + '?' + parameters) 114 | userContent = json.loads(urllib.urlopen(request).read()) 115 | return userContent 116 | 117 | def getItemInfo(itemId, token, portalUrl): 118 | '''Returns general information about the item.''' 119 | params = urllib.urlencode({'token' : token, 120 | 'f' : 'json'}) 121 | itemInfo = json.loads(urllib.urlopen(portalUrl + 122 | '/sharing/rest/content/items/' + 123 | itemId + '?' + params).read()) 124 | return itemInfo 125 | 126 | def getUserInfo(username, token, portalUrl): 127 | '''Returns information about the specified user.''' 128 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 129 | request = (portalUrl + '/sharing/rest/community/users/' + username + 130 | '?' + parameters) 131 | userInfo = json.loads(urllib.urlopen(request).read()) 132 | return userInfo 133 | 134 | def updateUserInfo(username, info, token, portalUrl): 135 | '''Updates a user's info.''' 136 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 137 | request = (portalUrl + '/sharing/rest/community/users/' + username + 138 | '/update?' + parameters) 139 | status = json.loads( 140 | urllib.urlopen(request, urllib.urlencode(info)).read() 141 | ) 142 | return status 143 | 144 | def updateUserRole(username, role, token, portalUrl): 145 | '''Updates a user's info.''' 146 | parameters = urllib.urlencode({'token': token, 147 | 'f': 'json', 148 | 'user': username, 149 | 'role': role}) 150 | request = (portalUrl + '/sharing/rest/portals/self/updateUserRole?') 151 | status = json.loads( 152 | urllib.urlopen(request, parameters).read() 153 | ) 154 | return status 155 | 156 | def createFolder(username, newFolderName, token, portalUrl): 157 | '''Creates a new folder in a User's content.''' 158 | parameters = urllib.urlencode( 159 | {'token': token, 160 | 'title': newFolderName, 161 | 'f': 'json'} 162 | ) 163 | request = (portalUrl + '/sharing/rest/content/users/' + username + 164 | '/createFolder?' + parameters) 165 | status = json.loads( 166 | urllib.urlopen(request, parameters).read() 167 | ) 168 | return status 169 | 170 | def migrateAccount(portal, username, password, oldOwner, newOwner, retainExactFolderName): 171 | # Get an admin token. 172 | token = generateToken(username=username, password=password, 173 | portalUrl=portal) 174 | 175 | # Get a list of the oldOwner's folders and any items in root. 176 | userContent = getUserContent(oldOwner, '/', token, portalUrl=portal) 177 | newUserContent = getUserContent(newOwner, '/', token, portalUrl=portal) 178 | 179 | userInfo = getUserInfo(oldOwner, token, portalUrl=portal) 180 | newInfo = {'fullName': userInfo['fullName'], 181 | 'description': userInfo['description'], 182 | 'units': userInfo['units']} 183 | if userInfo.has_key('firstName'): 184 | newInfo['firstName'] = userInfo['firstName'] 185 | if userInfo.has_key('firstName'): 186 | newInfo['lastName'] = userInfo['lastName'] 187 | ## Thumbnail migration not working 188 | #if userInfo.has_key('thumbnail'): 189 | #newInfo['thumbnail'] = (portal + '/sharing/rest/community/users/' + 190 | #oldOwner + '/info/' + userInfo['thumbnail']) 191 | 192 | # Check if the user is assigned a custom role. 193 | if 'roleId' in userInfo.keys(): 194 | newRole = userInfo['roleId'] 195 | else: 196 | newRole = userInfo['role'] 197 | 198 | print 'Updating {}\'s profile'.format(newOwner) 199 | updateUserInfo(newOwner, newInfo, token, portal) 200 | print 'Updating {}\'s role'.format(newOwner) 201 | updateUserRole(newOwner, newRole, token, portal) 202 | ownedGroups = [] 203 | memberGroups = [] 204 | for group in userInfo['groups']: 205 | if group['owner'] == oldOwner: 206 | ownedGroups.append(group['id']) 207 | else: 208 | memberGroups.append(group['id']) 209 | print 'Reassigning ' + str(len(ownedGroups)) + ' groups to ' + newOwner 210 | reassignGroups(newOwner, ownedGroups, token, portal) 211 | print 'Adding ' + newOwner + ' to ' + str(len(memberGroups)) + ' groups' 212 | addUsersToGroups([newOwner], memberGroups, token, portal) 213 | 214 | # *** CAUTION *** 215 | # The following code will transfer ownership of ALL CONTENT 216 | # from oldOwner to newOwner. 217 | # Be sure you are absolutely sure you want to do this before proceeding. 218 | if retainExactFolderName == 'True': 219 | retainExactFolderName = True 220 | if not ('items' in userContent or len(userContent['items']) == 0) and (len(userContent['folders']) == 0): 221 | print oldOwner + ' doesn\'t have any content visible to this account or has no items.' 222 | print 'Be sure you are signed in as admin.' 223 | else: 224 | for item in userContent['items']: 225 | changeOwnership(item['id'], newOwner, '/', token=token, 226 | portalUrl=portal) 227 | for folder in userContent['folders']: 228 | if len(getUserContent(oldOwner, folder['id'], token, portal)['items']) == 0: 229 | continue 230 | if retainExactFolderName is True: 231 | if folder['title'] not in [newfolder['title'] for newfolder in newUserContent['folders']]: 232 | print 'trying to put item into new folder, but no folder exists, creating new folder...' 233 | try: 234 | createFolder(newOwner, folder['title'], token, portal) 235 | print 'created folder: ' + folder['title'] 236 | except: 237 | print 'failed to create folder: ' + folder['title'] 238 | folderContent = getUserContent(oldOwner, folder['id'], 239 | token=token, portalUrl=portal) 240 | for item in folderContent['items']: 241 | changeOwnership(item['id'], newOwner, folder['title'], 242 | token=token, portalUrl=portal) 243 | print 'Migration completed.' 244 | print 'All items transferred from {0} to {1}'.format(oldOwner, 245 | newOwner) 246 | 247 | return 'Migration from {0} to {1} complete.'.format(oldOwner, newOwner) 248 | 249 | # Run the script. 250 | if __name__ == '__main__': 251 | parser = argparse.ArgumentParser() 252 | parser.add_argument('portal', 253 | help=('url of the Portal (e.g. ' 254 | 'https://portal.domain.com:7443/arcgis)')) 255 | parser.add_argument('username', help='username') 256 | parser.add_argument('password', help='password') 257 | parser.add_argument('oldOwner', help='source account to migrate from') 258 | parser.add_argument('newOwner', help='destination account to migrate to') 259 | parser.add_argument('retainExactFolderName', help='True or False. True: exact folder name recreated. \ 260 | False: username_foldername format followed') 261 | # Read the command line arguments. 262 | args = parser.parse_args() 263 | portal = args.portal[:-1] if args.portal[-1:] == '/' else args.portal 264 | username = args.username 265 | password = args.password 266 | oldOwner = args.oldOwner 267 | newOwner = args.newOwner 268 | retainExactFolderName = args.retainExactFolderName 269 | 270 | migrateAccount(portal, username, password, oldOwner, newOwner, retainExactFolderName) 271 | -------------------------------------------------------------------------------- /migrateRoles.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python migrateRoles.py -u -o -s 6 | 7 | import urllib 8 | import json 9 | import argparse 10 | 11 | def generateToken(username, password, portalUrl): 12 | '''Retrieves a token to be used with API requests.''' 13 | parameters = urllib.urlencode({'username' : username, 14 | 'password' : password, 15 | 'client' : 'referer', 16 | 'referer': portalUrl, 17 | 'expiration': 60, 18 | 'f' : 'json'}) 19 | response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?', 20 | parameters).read() 21 | try: 22 | jsonResponse = json.loads(response) 23 | if 'token' in jsonResponse: 24 | return jsonResponse['token'] 25 | elif 'error' in jsonResponse: 26 | print jsonResponse['error']['message'] 27 | for detail in jsonResponse['error']['details']: 28 | print detail 29 | except ValueError, e: 30 | print 'An unspecified error occurred.' 31 | print e 32 | 33 | def getPortalInfo(token, portalUrl): 34 | '''''' 35 | parameters = urllib.urlencode({'token': token, 'f': 'json'}) 36 | request = (portalUrl + '/sharing/rest/portals/self' + '?' + parameters) 37 | portalInfo = json.loads(urllib.urlopen(request).read()) 38 | return portalInfo 39 | 40 | def getUsers(portalId, token, portalUrl): 41 | '''Returns a list of all users in the Portal.''' 42 | 43 | # Iterable function to gather all pages of users. 44 | def __users__(portal, portalId, token, num=100, start=0): 45 | '''Retrieve a single page of search results.''' 46 | parameters = urllib.urlencode({'num': num, 47 | 'start': start, 48 | 'token': token, 49 | 'f': 'json'}) 50 | request = (portalUrl + '/sharing/rest/portals/' + portalId + 51 | '/users?' + parameters) 52 | results = json.loads(urllib.urlopen(request).read()) 53 | return results 54 | 55 | allUsers = [] 56 | users = __users__(portalUrl, portalId, token) 57 | allUsers.extend(users['users']) 58 | 59 | while 'nextStart' in users.iterkeys() and users['nextStart'] != -1: 60 | users = __users__(portalUrl, portalId, token, 61 | start=users['nextStart']) 62 | allUsers.extend(users['users']) 63 | 64 | return allUsers 65 | 66 | def getRoles(token, portalUrl): 67 | '''Get the custom roles available in the Portal.''' 68 | ## Currently does not handle for than 100 roles. 69 | parameters = urllib.urlencode({'token': token, 70 | 'f': 'json', 71 | 'num': 100}) 72 | request = (portalUrl + '/sharing/rest/portals/self/roles' + 73 | '?' + parameters) 74 | roles = json.loads(urllib.urlopen(request).read()) 75 | return roles 76 | 77 | def updateUserRole(username, role, token, portalUrl): 78 | '''Updates a user's info.''' 79 | parameters = urllib.urlencode({'token': token, 80 | 'f': 'json', 81 | 'user': username, 82 | 'role': role}) 83 | request = (portalUrl + '/sharing/rest/portals/self/updateUserRole?') 84 | status = json.loads( 85 | urllib.urlopen(request, parameters).read() 86 | ) 87 | return status 88 | 89 | # Run the script. 90 | if __name__ == '__main__': 91 | parser = argparse.ArgumentParser() 92 | parser.add_argument('-u', '--portal', required=True, 93 | help=('url of the Portal (e.g. ' 94 | 'https://portal.domain.com:7443/arcgis)')) 95 | parser.add_argument('-o', '--portalAdmin', required=True, 96 | help='Portal admin username') 97 | parser.add_argument('-s', '--portalPassword', required=True, 98 | help='Portal admin password') 99 | # Read the command line arguments. 100 | args = parser.parse_args() 101 | portal = args.portal[:-1] if args.portal[-1:] == '/' else args.portal 102 | username = args.portalAdmin 103 | password = args.portalPassword 104 | 105 | # Sample usage 106 | token = generateToken(username=username, password=password, 107 | portalUrl=portal) 108 | 109 | # Get the Portal ID. 110 | portalId = getPortalInfo(token, portal)['id'] 111 | 112 | # Get a list of the Portal's roles. 113 | roles = getRoles(token, portal) 114 | 115 | # Add the built-in roles to the list. 116 | roles['roles'].extend([{'id': 'account_user', 117 | 'name': 'User (built-in)'}, 118 | {'id': 'account_publisher', 119 | 'name': 'Publisher (built-in)'}, 120 | {'id': 'account_admin', 'name': 121 | 'Admin (built-in)'}]) 122 | 123 | print 'Available roles' 124 | for key, role in enumerate(roles['roles']): 125 | # Add a key to let the user select by number. 126 | role['key'] = key 127 | print '{0}. {1}'.format(role['key'], role['name']) 128 | selection = input('Enter the number of the old role: ') 129 | oldRole = roles['roles'][selection] 130 | 131 | selection = input('Enter the number of the new role: ') 132 | newRole = roles['roles'][selection] 133 | 134 | confirm = raw_input('Confirm migration of ' + oldRole['name'] + ' to ' + 135 | newRole['name'] + ' (y/n): ') 136 | 137 | if confirm == 'y': 138 | # Get a list of the Portal's users. 139 | users = getUsers(portalId, token, portalUrl=portal) 140 | 141 | count = 0 142 | for user in users: 143 | if user['role'] == oldRole['id']: 144 | print 'Migrating ' + user['username'] 145 | updateUserRole(user['username'], newRole['id'], token, portal) 146 | count += 1 147 | 148 | print 'Migrated {} users from {} to {}.'.format(str(count), 149 | oldRole['name'], 150 | newRole['name']) 151 | elif confirm == 'n': 152 | print 'Canceling' 153 | exit() -------------------------------------------------------------------------------- /publishFiles.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | # Sample Usage 4 | # python publishShapefiles.py -u https://www.arcgis.com -o username -s password -p 'path/to/my/shapefiles' 5 | 6 | import argparse 7 | import json 8 | import os 9 | import time 10 | import csv 11 | import urllib 12 | import requests 13 | import webbrowser 14 | 15 | portal = 'https://www.arcgis.com' 16 | authMethod = 'Built-In' # SAML | Built-In 17 | verify = True # Set to False to ignore certificate validation warnings. 18 | ######## Complete for SAML login ############ 19 | appId = '' 20 | appSecret = '' 21 | redirectUri = 'urn:ietf:wg:oauth:2.0:oob' 22 | ############################################# 23 | 24 | maxJobs = 5 # Set the max number of concurrent jobs allowed. 25 | sleepTime = 60 # Time in seconds to wait between status checks. 26 | 27 | def generateToken(username, password, portalUrl): 28 | '''Retrieves a token to be used with API requests.''' 29 | parameters = {'username': username, 30 | 'password': password, 31 | 'client': 'referer', 32 | 'referer': portalUrl, 33 | 'expiration': 60, # token life in minutes 34 | 'f': 'json'} 35 | url = '{}/sharing/rest/generateToken'.format(portalUrl) 36 | response = requests.post(url, data=parameters) 37 | return response.json() 38 | 39 | def oAuthAuthorize(clientId, responseType, redirectUri, portalUrl): 40 | '''Retrieves a token to be used with API requests.''' 41 | parameters = urllib.urlencode({'client_id': clientId, 42 | 'response_type': responseType, 43 | 'redirect_uri': redirectUri, 44 | 'f' : 'json'}) 45 | response = urllib.urlopen(portalUrl + '/sharing/rest/oauth2/authorize?', 46 | parameters).read() 47 | try: 48 | jsonResponse = json.loads(response) 49 | if 'access_token' in jsonResponse: 50 | return jsonResponse['access_token'] 51 | elif 'error' in jsonResponse: 52 | print(jsonResponse['error']['message']) 53 | for detail in jsonResponse['error']['details']: 54 | print(detail) 55 | except ValueError, e: 56 | print 'An unspecified error occurred.' 57 | print e 58 | 59 | def oAuthToken(clientId, clientSecret, redirectUri, grantType, code, portalUrl): 60 | '''Retrieves a token to be used with API requests.''' 61 | parameters = urllib.urlencode({'client_id': clientId, 62 | 'client_secret': clientSecret, 63 | 'redirect_uri': redirectUri, 64 | 'grant_type': grantType, 65 | 'code': code, 66 | 'f' : 'json'}) 67 | response = urllib.urlopen(portalUrl + '/sharing/rest/oauth2/token?', 68 | parameters).read() 69 | try: 70 | jsonResponse = json.loads(response) 71 | if 'access_token' in jsonResponse: 72 | return jsonResponse['access_token'] 73 | elif 'error' in jsonResponse: 74 | print jsonResponse['error']['message'] 75 | for detail in jsonResponse['error']['details']: 76 | print detail 77 | except ValueError, e: 78 | print 'An unspecified error occurred.' 79 | print e 80 | 81 | def userContent(username, portalUrl, token): 82 | '''Returns the user's folders and content in the root folder.''' 83 | parameters = {'token': token, 84 | 'f': 'json'} 85 | url = '{}/sharing/rest/content/users/{}'.format(portalUrl, username) 86 | response = requests.get(url, params=parameters, verify=verify) 87 | return response.json() 88 | 89 | def createFolder(title, username, portalUrl, token): 90 | '''Creates a new folder in the user's account.''' 91 | parameters = {'title': title, 92 | 'token': token, 93 | 'f': 'json'} 94 | url = '{}/sharing/rest/content/users/{}/createFolder'.format(portalUrl, username) 95 | response = requests.post(url, data=parameters, verify=verify) 96 | return response.json() 97 | 98 | def addFileItem(username, folder, file, type, portalUrl, token): 99 | '''Creates a new item in the user's content.''' 100 | fileName = os.path.split(file)[-1].split('.')[0] 101 | # need to break out handlers for files over 10mb 102 | parameters = {'file': file, 103 | 'type': type, 104 | 'title': fileName, 105 | # 'multipart': 'true', 106 | 'token': token, 107 | 'f': 'json'} 108 | files = {'file': open(file, 'rb')} 109 | url = '{}/sharing/rest/content/users/{}/{}/addItem'.format(portalUrl, username, folder) 110 | response = requests.post(url, data=parameters, files=files, verify=verify) 111 | return response.json() 112 | 113 | def updateItemMetadata(username, folder, itemId, file, portalUrl, token): 114 | '''Uploads an ArcCatalog xml file containing metadata to an item.''' 115 | parameters = {'token': token, 116 | 'f': 'json'} 117 | files = {'metadata': open(file, 'rb')} 118 | url = '{}/sharing/rest/content/users/{}/{}/items/{}/update'.format(portalUrl, username, folder, itemId) 119 | response = requests.post(url, data=parameters, files=files, verify=verify) 120 | return response.json() 121 | 122 | def publishItem(username, itemId, name, fileType, portalUrl, token, hasStaticData='true', maxRecordCount=2000, capabilities='Query', overwrite='false'): 123 | '''Publishes a hosted service based on an existing source item.''' 124 | parameters = {'itemId': itemId, 125 | 'filetype': fileType, 126 | 'publishParameters': json.dumps({'hasStaticData': hasStaticData, 127 | 'name': name, 128 | 'maxRecordCount': maxRecordCount, 129 | 'layerInfo': {'capabilities': capabilities}}), 130 | 'overwrite': overwrite, 131 | 'token': token, 132 | 'f': 'json'} 133 | url = '{}/sharing/rest/content/users/{}/publish'.format(portalUrl, username) 134 | response = requests.post(url, data=parameters, verify=verify) 135 | return response.json() 136 | 137 | def checkStatus(username, folder, serviceItemId, portalUrl, token, jobId=None, jobType=None): 138 | '''Creates a new item in a user's content.''' 139 | parameters = {'jobId': jobId, 140 | 'jobType': jobType, 141 | 'token': token, 142 | 'f': 'json'} 143 | if jobId != None: 144 | parameters['jobId'] = jobId 145 | if jobType != None: 146 | parameters['jobType'] = jobType 147 | url = '{}/sharing/rest/content/users/{}/{}/items/{}/status'.format(portalUrl, username, folder, serviceItemId) 148 | response = requests.get(url, params=parameters) 149 | return response.json() 150 | 151 | def getDescription(itemId, portalUrl, token): 152 | '''Retrieves an item's description object.''' 153 | parameters = {'token': token, 154 | 'f': 'json'} 155 | url = '{}/sharing/rest/content/items/{}'.format(portalUrl, itemId) 156 | response = requests.get(url, params=parameters) 157 | return response.json() 158 | 159 | def serviceRecordCount(serviceUrl, token): 160 | '''Retrieves a token to be used with API requests.''' 161 | parameters = {'where': '1=1', 162 | 'returnCountOnly': 'true', 163 | 'token': token, 164 | 'f': 'json'} 165 | url = '{}/0/query'.format(serviceUrl) 166 | response = requests.get(url, params=parameters) 167 | return response.json() 168 | 169 | def updateResult(results, matchK, matchV, k, v, outfile=None): 170 | '''Update the specified item in the list of results by matching on a provided key.''' 171 | result = filter(lambda result: result[matchK] == matchV, results) 172 | result[0][k] = v 173 | if outfile: 174 | writeResults(outfile, results) 175 | 176 | def writeResults(outfile, results, portalUrl='https://www.arcgis.com'): 177 | with open(outfile, 'wb') as output: 178 | dataWriter = csv.writer( 179 | output, 180 | delimiter=',', 181 | quotechar='"', 182 | quoting=csv.QUOTE_MINIMAL 183 | ) 184 | # Write header row. 185 | dataWriter.writerow( 186 | ['Dataset', 'Size', 'Folder', 'Item Id', 'Item URL', 'Service Id', 'Service URL', 'Status', 'Service Record Count'] 187 | ) 188 | for result in results: 189 | dataWriter.writerow( 190 | [result['shortname'], 191 | result['size'], 192 | result['foldername'], 193 | result['itemId'], 194 | '{}/home/item.html?id={}'.format(portalUrl, result['itemId']) if result['itemId'] else '', 195 | result['serviceItemId'], 196 | '{}/home/item.html?id={}'.format(portalUrl, result['serviceItemId']) if result['serviceItemId'] else '', 197 | result['status'] if result['status'] else '', 198 | result['count'] if result['count'] else ''] 199 | ) 200 | 201 | def currentJobs(): 202 | return len(jobs) 203 | 204 | 205 | # Run the script. 206 | if __name__ == '__main__': 207 | parser = argparse.ArgumentParser() 208 | parser.add_argument('-u', '--portal', 209 | help=('url of the portal (e.g. ' 210 | 'https://webadaptor.domain.com/arcgis)')) 211 | parser.add_argument('-o', '--username', required=True, help='username') 212 | parser.add_argument('-s', '--password', required=False, help='password') 213 | parser.add_argument('-p', '--path', help='path to the shapefiles') 214 | # Read the command line arguments. 215 | args = parser.parse_args() 216 | portal = args.portal 217 | username = args.username 218 | password = args.password 219 | path = args.path 220 | 221 | # Authenticate with the portal. 222 | if authMethod == 'SAML': 223 | # Use SAML 224 | parameters = urllib.urlencode({'client_id': appId, 225 | 'response_type': 'code', 226 | 'redirect_uri': redirectUri}) 227 | authUrl = '{}/sharing/rest/oauth2/authorize?{}'.format(portal, parameters) 228 | webbrowser.open(authUrl) 229 | 230 | # Get the user's authentication code. 231 | authCode = raw_input('Enter the code you received: ') 232 | 233 | # Get a token with the authentication code. 234 | token = oAuthToken(clientId=appId, clientSecret=appSecret, redirectUri=redirectUri, grantType='authorization_code', code=authCode, portalUrl=portal) 235 | else: 236 | # Use built-in auth. 237 | token = generateToken(username, password, portal)['token'] 238 | 239 | # Gather up the shapefiles in the directory (and subdirectories). 240 | shapefiles = [] # keep track of all the shapefiles to upload. 241 | jobs = [] # keep track of active publishing jobs. 242 | 243 | for root, dirs, files in os.walk(path): 244 | for f in files: 245 | if f[-4:] == '.zip': 246 | description = {'shortname': f[:-4].replace(' ', '_'), 247 | 'filename': f, 248 | 'foldername': os.path.split(os.path.dirname(os.path.join(root, f)))[1], 249 | 'path': os.path.join(root, f), 250 | 'size': round(float(os.path.getsize(os.path.join(root, f))) / 1048576, 3), # bytes to megabytes 251 | 'metadata': os.path.join(root, '{}.xml'.format(f[:-4])), 252 | 'itemId': None, 253 | 'itemUrl': None, 254 | 'serviceItemId': None, 255 | 'serviceItemUrl': None, 256 | 'status': None, 257 | 'count': None} 258 | shapefiles.append(description) 259 | 260 | results = list(shapefiles) # Copy the list of shapefiles to track progress. 261 | resultsFile = '{}/results.csv'.format(path) 262 | 263 | # Check if the destination folders exist in the user's account. 264 | content = userContent(username, portal, token) 265 | folders = content['folders'] 266 | 267 | while len(shapefiles) > 0 or len(jobs) > 0: 268 | if len(shapefiles) > 0 and len(jobs) < maxJobs: 269 | # Continue pushing new jobs. 270 | shapefile = shapefiles.pop(0) 271 | 272 | # Get the destination folder id and create it if it doesn't exist. 273 | folder = filter(lambda folder: folder['title'] == shapefile['foldername'], folders) 274 | if not folder: 275 | print('Creating folder {}'.format(shapefile['foldername'])) 276 | newFolder = createFolder(shapefile['foldername'], username, portal, token) 277 | folder = {'title': newFolder['folder']['title'], 278 | 'id': newFolder['folder']['id']} 279 | folders.append(folder) 280 | else: 281 | folder = folder[0] 282 | 283 | # Create the item in ArcGIS Online. 284 | print('') 285 | print('Uploading {} {} mb'.format(shapefile['filename'], shapefile['size'])) 286 | try: 287 | newItem = addFileItem(username=username, folder=folder['id'], file=shapefile['path'], type='Shapefile', portalUrl=portal, token=token) 288 | if 'success' in newItem: 289 | itemId = newItem['id'] 290 | print(' created item {}'.format(itemId)) 291 | updateResult(results, 'shortname', shapefile['shortname'], 'itemId', itemId, outfile=resultsFile) 292 | 293 | # Update the item's metadata. 294 | print(' updating the metadata') 295 | update = updateItemMetadata(username=username, folder=folder['id'], itemId=itemId, file=shapefile['metadata'], portalUrl=portal, token=token) 296 | else: 297 | print(' error - {}'.format(newItem['error']['message'])) 298 | updateResult(results, 'shortname', shapefile['shortname'], 'status', newItem['error']['message'], outfile=resultsFile) 299 | except: 300 | print(' error adding {}'.format(shapefile['filename'])) 301 | updateResult(results, 'shortname', shapefile['shortname'], 'status', 'unhandled item upload error', outfile=resultsFile) 302 | continue 303 | 304 | # Publish it. 305 | print(' publishing {}'.format(shapefile['shortname'])) 306 | try: 307 | publishing = publishItem(username, newItem['id'], shapefile['shortname'], 'Shapefile', portal, token, overwrite='false') 308 | job = publishing['services'][0] 309 | if 'jobId' in job: 310 | job['folderId'] = folder['id'] 311 | job['shortname'] = shapefile['shortname'] 312 | jobs.append(job) 313 | updateResult(results, 'shortname', job['shortname'], 'serviceItemId', job['serviceItemId']) 314 | updateResult(results, 'shortname', job['shortname'], 'status', 'publishing started', outfile=resultsFile) 315 | else: 316 | print(' error - {}'.format(job['error']['message'])) 317 | updateResult(results, 'shortname', shapefile['shortname'], 'status', job['error']['message'], outfile=resultsFile) 318 | except: 319 | print(' error publishing {}'.format(shapefile['shortname'])) 320 | updateResult(results, 'shortname', shapefile['shortname'], 'status', 'unhandled publishing error', outfile=resultsFile) 321 | continue 322 | else: 323 | # Check the status of existing jobs. 324 | currentJobs = len(jobs) 325 | while len(jobs) > 0: 326 | i = -1 327 | print('') 328 | print('Checking the status of {} publishing job(s)'.format(currentJobs)) 329 | for job in jobs: 330 | i += 1 331 | status = checkStatus(username, job['folderId'], job['serviceItemId'], portal, token, jobId=job['jobId']) 332 | job['status'] = status['status'] 333 | if job['status'] in ['completed', 'failed']: 334 | description = getDescription(job['serviceItemId'], portal, token) 335 | serviceUrl = description['url'] 336 | count = serviceRecordCount(serviceUrl, token)['count'] 337 | updateResult(results, 'shortname', job['shortname'], 'count', count) 338 | updateResult(results, 'shortname', job['shortname'], 'status', 'publishing {}'.format(job['status']), outfile=resultsFile) 339 | jobs.pop(i) 340 | else: 341 | updateResult(results, 'shortname', job['shortname'], 'status', 'publishing {}'.format(job['status'])) 342 | continue 343 | if len(jobs) != 0: 344 | print('{} of {} service(s) still publishing'.format(len(jobs), currentJobs)) 345 | print('Next check in {} seconds'.format(sleepTime)) 346 | time.sleep(sleepTime) 347 | break 348 | 349 | print('') 350 | print('All services finished publishing') 351 | print('Check {} for full details on each item.'.format(resultsFile)) -------------------------------------------------------------------------------- /registerServices.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python registerServices.py -u -o -s -l 6 | 7 | import urllib 8 | import json 9 | import argparse 10 | 11 | def generateToken(username, password, portalUrl): 12 | '''Retrieves a token to be used with API requests.''' 13 | parameters = urllib.urlencode({ 14 | 'username': username, 15 | 'password': password, 16 | 'client': 'referer', 17 | 'referer': portalUrl, 18 | 'expiration': 60, 19 | 'f': 'json' 20 | }) 21 | response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?', parameters).read() 22 | try: 23 | jsonResponse = json.loads(response) 24 | if 'token' in jsonResponse: 25 | return jsonResponse['token'] 26 | elif 'error' in jsonResponse: 27 | print(jsonResponse['error']['message']) 28 | for detail in jsonResponse['error']['details']: 29 | print(detail) 30 | except ValueError, e: 31 | print('An unspecified error occurred.') 32 | print(e) 33 | 34 | def userFolders(portalUrl, username, token): 35 | '''Return a list of the user's folders.''' 36 | parameters = { 37 | 'token': token, 38 | 'f': 'json' 39 | } 40 | request = portalUrl + '/sharing/rest/content/users/' + username + '/?' + urllib.urlencode(parameters) 41 | response = urllib.urlopen(request).read() 42 | return json.loads(response)['folders'] 43 | 44 | def registerService(username, folder, name, description, serviceUrl, portalUrl, token, thumbnailUrl=''): 45 | '''Register the service in the user's portal.''' 46 | parameters = description.copy() 47 | parameters.update({ 48 | 'title': name, 49 | 'type': 'Map Service', 50 | 'url': serviceUrl, 51 | 'overwrite': 'false', 52 | 'thumbnailurl': thumbnailUrl, 53 | 'token' : token, 54 | 'f': 'json' 55 | }) 56 | parameters_safe = __encode_dict__(parameters) 57 | request = portalUrl + '/sharing/rest/content/users/' + username + '/' + folder + '/addItem?' 58 | response = urllib.urlopen(request, urllib.urlencode(parameters_safe)).read() 59 | return json.loads(response) 60 | 61 | def getServices(restUrl): 62 | '''Return the services and folders at the root of the Services directory.''' 63 | request = restUrl + '?f=json' 64 | response = urllib.urlopen(request).read() 65 | return json.loads(response) 66 | 67 | def serviceName(serviceUrl): 68 | '''Return the service's basic info.''' 69 | request = serviceUrl + '?f=json' 70 | response = urllib.urlopen(request).read() 71 | return json.loads(response) 72 | 73 | def serviceInfo(serviceUrl): 74 | '''Return the description object for the service.''' 75 | request = serviceUrl + '/info/iteminfo?f=json' 76 | response = urllib.urlopen(request).read() 77 | return json.loads(response) 78 | 79 | def __encode_dict__(in_dict): 80 | out_dict = {} 81 | for k, v in in_dict.iteritems(): 82 | if isinstance(v, unicode): 83 | v = v.encode('utf8') 84 | elif isinstance(v, str): 85 | # Encode it in UTF-8. 86 | v.decode('utf8') 87 | out_dict[k] = v 88 | return out_dict 89 | 90 | # Run the script. 91 | if __name__ == '__main__': 92 | parser = argparse.ArgumentParser() 93 | parser.add_argument('-u', '--portal', required=True, 94 | help=('url of the portal ' 95 | '(e.g. https://portal.domain.com:7443/arcgis)')) 96 | parser.add_argument('-o', '--username', required=True, help='username') 97 | parser.add_argument('-s', '--password', required=True, help='password') 98 | parser.add_argument('-l', '--rest', required=True, 99 | help=('ArcGIS for Server REST endpoint')) 100 | # Read the command line arguments. 101 | args = parser.parse_args() 102 | portal = args.portal[:-1] if args.portal[-1:] == '/' else args.portal 103 | username = args.username 104 | password = args.password 105 | restUrl = args.rest 106 | 107 | # Get a token to use with subsequent requests. 108 | token = generateToken(username=username, password=password, portalUrl=portal) 109 | 110 | restInfo = getServices(restUrl) 111 | folders = restInfo['folders'] 112 | services = restInfo['services'] 113 | 114 | availableFolders = userFolders(portal, username, token) 115 | availableFolders.insert(0, {'title': 'Root (Top Level)', 'id': ''}) 116 | 117 | print('User folders') 118 | for key, folder in enumerate(availableFolders): 119 | # Add a key to let the user select by number. 120 | folder['key'] = key 121 | print('{0}. {1}'.format(folder['key'], folder['title'])) 122 | selection = input('Enter the number for the destination folder: ') 123 | userFolder = availableFolders[selection]['id'] 124 | 125 | print('Harvesting services from {}'.format(restUrl)) 126 | for folder in folders: 127 | folderServices = getServices(restUrl + '/' + folder)['services'] 128 | services.extend(folderServices) 129 | 130 | for service in services: 131 | serviceUrl = '{}/{}/{}'.format(restUrl, service['name'], service['type']) 132 | print('Registering {}/{}'.format(service['name'], service['type'])) 133 | description = serviceInfo(serviceUrl) 134 | name = description['name'] if 'name' in description else service['name'] 135 | thumbnail = '{}/info/thumbnail/{}'.format(serviceUrl, description['thumbnail']) if 'thumbnail' in description else None 136 | # Convert arrays to comma separated strings for posting. 137 | for k, v in description.iteritems(): 138 | if k == 'extent': 139 | newExtent = '{}, {}, {}, {}'.format(v[0][0], v[0][1], v[1][0], v[1][1]) 140 | description[k] = newExtent 141 | elif isinstance(v, list): 142 | description[k] = ', '.join([str(x) for x in v]) 143 | 144 | result = registerService(username, userFolder, name, description, serviceUrl, portal, token, thumbnail) 145 | if 'success' in result: 146 | print('OK') 147 | elif 'error' in result: 148 | print('Failed to register {}'.format(name)) 149 | -------------------------------------------------------------------------------- /removeTag.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python removeTag.py 6 | # 7 | import urllib 8 | import json 9 | import argparse 10 | 11 | def generateToken(username, password, portalUrl): 12 | '''Retrieves a token to be used with API requests.''' 13 | parameters = urllib.urlencode({'username' : username, 14 | 'password' : password, 15 | 'client' : 'referer', 16 | 'referer': portalUrl, 17 | 'expiration': 60, 18 | 'f' : 'json'}) 19 | response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?', 20 | parameters).read() 21 | try: 22 | jsonResponse = json.loads(response) 23 | if 'token' in jsonResponse: 24 | return jsonResponse['token'] 25 | elif 'error' in jsonResponse: 26 | print jsonResponse['error']['message'] 27 | for detail in jsonResponse['error']['details']: 28 | print detail 29 | except ValueError, e: 30 | print 'An unspecified error occurred.' 31 | print e 32 | 33 | def searchPortal(portalUrl, query=None, totalResults=None, sortField='numviews', 34 | sortOrder='desc', token=None): 35 | ''' 36 | Search the portal using the specified query and search parameters. 37 | Optionally provide a token to return results visible to that user. 38 | ''' 39 | # Default results are returned by highest 40 | # number of views in descending order. 41 | allResults = [] 42 | if not totalResults or totalResults > 100: 43 | numResults = 100 44 | else: 45 | numResults = totalResults 46 | results = __search__(portalUrl, query, numResults, sortField, sortOrder, 0, 47 | token) 48 | 49 | if not 'error' in results.keys(): 50 | if not totalResults: 51 | totalResults = results['total'] # Return all of the results. 52 | allResults.extend(results['results']) 53 | while (results['nextStart'] > 0 and 54 | results['nextStart'] < totalResults): 55 | # Do some math to ensure it only 56 | # returns the total results requested. 57 | numResults = min(totalResults - results['nextStart'] + 1, 100) 58 | results = __search__(portalUrl=portalUrl, query=query, 59 | numResults=numResults, sortField=sortField, 60 | sortOrder=sortOrder, token=token, 61 | start=results['nextStart']) 62 | allResults.extend(results['results']) 63 | return allResults 64 | else: 65 | print results['error']['message'] 66 | return results 67 | 68 | def __search__(portalUrl, query=None, numResults=100, sortField='numviews', 69 | sortOrder='desc', start=0, token=None): 70 | '''Retrieve a single page of search results.''' 71 | params = { 72 | 'q': query, 73 | 'num': numResults, 74 | 'sortField': sortField, 75 | 'sortOrder': sortOrder, 76 | 'f': 'json', 77 | 'start': start 78 | } 79 | if token: 80 | # Adding a token provides an authenticated search. 81 | params['token'] = token 82 | request = portal + '/sharing/rest/search?' + urllib.urlencode(params) 83 | results = json.loads(urllib.urlopen(request).read()) 84 | return results 85 | 86 | 87 | def removeTag(itemId, unwantedTag, token, portalUrl): 88 | '''Removes the unwanted tag from an item.''' 89 | try: 90 | params = urllib.urlencode({'token' : token, 91 | 'f' : 'json'}) 92 | print 'Found item ' + itemId + ' with unwanted tag...' 93 | 94 | # Get the item info 95 | itemInfo = json.loads(urllib.urlopen(portalUrl + "/sharing/rest/content/items/" + itemId + "?" + params).read()) 96 | 97 | # Find items with the unwanted tag and remove it 98 | if unwantedTag in itemInfo['tags']: 99 | tags = itemInfo['tags'] 100 | print 'Tags before: ' + ', '.join(tags) 101 | tags.remove(unwantedTag) 102 | print 'Tags after: ' + ', '.join(tags) 103 | strTags = ', '.join(tags) 104 | ownerFolder = itemInfo['ownerFolder'] if itemInfo['ownerFolder'] else '/' 105 | modRequest = urllib.urlopen(portalUrl + 106 | '/sharing/content/users/' + 107 | itemInfo['owner'] + 108 | '/' + ownerFolder + 109 | '/items/' + itemId + 110 | '/update?' + params , 111 | urllib.urlencode( 112 | {'tags' : strTags} 113 | )) 114 | 115 | # Evaluate the results to make sure it happened 116 | modResponse = json.loads(modRequest.read()) 117 | if modResponse.has_key('error'): 118 | raise AGOPostError(itemId, modResponse['error']['message']) 119 | else: 120 | print 'Successfully removed the unwanted tag from ' + itemInfo['id'] 121 | 122 | except ValueError as e: 123 | print 'Error - no item specified' 124 | except AGOPostError as e: 125 | print e.item 126 | print 'Error updating item ' + e.item + ': ' + e.msg 127 | 128 | class AGOPostError(Exception): 129 | def __init__(self, item, msg): 130 | print 'ok' 131 | self.item = item 132 | self.msg = msg 133 | 134 | # Run the script. 135 | if __name__ == '__main__': 136 | parser = argparse.ArgumentParser() 137 | parser.add_argument('portal', 138 | help=('url of the Portal (e.g. ' 139 | 'https://portal.domain.com:7443/arcgis)')) 140 | parser.add_argument('username', help='username') 141 | parser.add_argument('password', help='password') 142 | parser.add_argument('query', help='search string to find content') 143 | parser.add_argument('unwantedTag', help='the tag to remove') 144 | # Read the command line arguments. 145 | args = parser.parse_args() 146 | portal = args.portal 147 | username = args.username 148 | password = args.password 149 | query = args.query 150 | unwantedTag = args.unwantedTag 151 | 152 | # Get a token for the source Portal for ArcGIS. 153 | token = generateToken(username=username, password=password, 154 | portalUrl=portal) 155 | 156 | # Get a list of the content matching the query. 157 | content = searchPortal(portalUrl=portal, 158 | query=query, 159 | token=token) 160 | if len(content) == 0: 161 | print 'No items returned with query \'' + query + '\'' 162 | for item in content: 163 | removeTag(item['id'], unwantedTag, token=token, portalUrl=portal) 164 | 165 | print 'Update complete.' -------------------------------------------------------------------------------- /updateItemMetadata.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | # Sample Usage 4 | # python updateItemMetadata.py -u https://www.arcgis.com -o username -s password -i itemId -p 'path/to/metadata.xml' 5 | 6 | import argparse 7 | import json 8 | import os 9 | import urllib 10 | import requests 11 | import webbrowser 12 | 13 | portal = 'https://www.arcgis.com' 14 | authMethod = 'Built-In' # SAML | Built-In 15 | verify = True # Set to False to ignore certificate validation warnings. 16 | ######## Complete for SAML login ############ 17 | appId = '' 18 | appSecret = '' 19 | redirectUri = 'urn:ietf:wg:oauth:2.0:oob' 20 | ############################################# 21 | 22 | def generateToken(username, password, portalUrl): 23 | '''Retrieves a token to be used with API requests.''' 24 | parameters = {'username': username, 25 | 'password': password, 26 | 'client': 'referer', 27 | 'referer': portalUrl, 28 | 'expiration': 60, # token life in minutes 29 | 'f': 'json'} 30 | url = '{}/sharing/rest/generateToken'.format(portalUrl) 31 | response = requests.post(url, data=parameters) 32 | return response.json() 33 | 34 | def oAuthAuthorize(clientId, responseType, redirectUri, portalUrl): 35 | '''Retrieves a token to be used with API requests.''' 36 | parameters = urllib.urlencode({'client_id': clientId, 37 | 'response_type': responseType, 38 | 'redirect_uri': redirectUri, 39 | 'f' : 'json'}) 40 | response = urllib.urlopen(portalUrl + '/sharing/rest/oauth2/authorize?', 41 | parameters).read() 42 | print(response) 43 | try: 44 | jsonResponse = json.loads(response) 45 | if 'access_token' in jsonResponse: 46 | return jsonResponse['access_token'] 47 | elif 'error' in jsonResponse: 48 | print(jsonResponse['error']['message']) 49 | for detail in jsonResponse['error']['details']: 50 | print(detail) 51 | except ValueError, e: 52 | print 'An unspecified error occurred.' 53 | print e 54 | 55 | def oAuthToken(clientId, clientSecret, redirectUri, grantType, code, portalUrl): 56 | '''Retrieves a token to be used with API requests.''' 57 | parameters = urllib.urlencode({'client_id': clientId, 58 | 'client_secret': clientSecret, 59 | 'redirect_uri': redirectUri, 60 | 'grant_type': grantType, 61 | 'code': code, 62 | 'f' : 'json'}) 63 | response = urllib.urlopen(portalUrl + '/sharing/rest/oauth2/token?', 64 | parameters).read() 65 | try: 66 | jsonResponse = json.loads(response) 67 | if 'access_token' in jsonResponse: 68 | return jsonResponse['access_token'] 69 | elif 'error' in jsonResponse: 70 | print jsonResponse['error']['message'] 71 | for detail in jsonResponse['error']['details']: 72 | print detail 73 | except ValueError, e: 74 | print 'An unspecified error occurred.' 75 | print e 76 | 77 | def itemDescription(itemId, portalUrl, token): 78 | '''Retrieves an item's description object.''' 79 | parameters = {'token': token, 80 | 'f': 'json'} 81 | url = '{}/sharing/rest/content/items/{}'.format(portalUrl, itemId) 82 | response = requests.get(url, params=parameters) 83 | print(response) 84 | return response.json() 85 | 86 | def updateItemMetadata(username, folder, itemId, file, portalUrl, token): 87 | '''Uploads an ArcCatalog xml file containing metadata to an item.''' 88 | parameters = {'token': token, 89 | 'f': 'json'} 90 | files = {'metadata': open(file, 'rb')} 91 | url = '{}/sharing/rest/content/users/{}/{}/items/{}/update'.format(portalUrl, username, folder, itemId) 92 | response = requests.post(url, data=parameters, files=files, verify=verify) 93 | return response.json() 94 | 95 | 96 | # Run the script. 97 | if __name__ == '__main__': 98 | parser = argparse.ArgumentParser() 99 | parser.add_argument('-u', '--portal', 100 | help=('url of the portal (e.g. ' 101 | 'https://webadaptor.domain.com/arcgis)')) 102 | parser.add_argument('-o', '--username', required=True, help='username') 103 | parser.add_argument('-s', '--password', required=False, help='password') 104 | parser.add_argument('-i', '--itemId', help='the item to update') 105 | parser.add_argument('-p', '--path', help='path to the shapefiles') 106 | # Read the command line arguments. 107 | args = parser.parse_args() 108 | portal = args.portal 109 | username = args.username 110 | password = args.password 111 | itemId = args.itemId 112 | path = args.path 113 | 114 | # Authenticate with the portal. 115 | if authMethod == 'SAML': 116 | # Use SAML 117 | parameters = urllib.urlencode({'client_id': appId, 118 | 'response_type': 'code', 119 | 'redirect_uri': redirectUri}) 120 | authUrl = '{}/sharing/rest/oauth2/authorize?{}'.format(portal, parameters) 121 | webbrowser.open(authUrl) 122 | 123 | # Get the user's authentication code. 124 | authCode = raw_input('Enter the code you received: ') 125 | 126 | # Get a token with the authentication code. 127 | token = oAuthToken(clientId=appId, clientSecret=appSecret, redirectUri=redirectUri, grantType='authorization_code', code=authCode, portalUrl=portal) 128 | else: 129 | # Use built-in auth. 130 | token = generateToken(username, password, portal)['token'] 131 | 132 | # Get the item's folder. 133 | description = itemDescription(itemId=itemId, portalUrl=portal, token=token) 134 | 135 | # Update the item's metadata. 136 | print('Updating the metadata') 137 | update = updateItemMetadata(username=username, folder=description['ownerFolder'], itemId=itemId, file=path, portalUrl=portal, token=token) 138 | 139 | print('Finished.') -------------------------------------------------------------------------------- /updateWebmapServices.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # Requires Python 2.7+ 3 | 4 | # Sample Usage: 5 | # python updateWebmapServices.py 6 | # 7 | 8 | import urllib 9 | import json 10 | import argparse 11 | 12 | def generateToken(username, password, portalUrl): 13 | '''Retrieves a token to be used with API requests.''' 14 | parameters = urllib.urlencode({'username' : username, 15 | 'password' : password, 16 | 'client' : 'referer', 17 | 'referer': portalUrl, 18 | 'expiration': 60, 19 | 'f' : 'json'}) 20 | response = urllib.urlopen(portalUrl + '/sharing/rest/generateToken?', 21 | parameters).read() 22 | try: 23 | jsonResponse = json.loads(response) 24 | if 'token' in jsonResponse: 25 | return jsonResponse['token'] 26 | elif 'error' in jsonResponse: 27 | print jsonResponse['error']['message'] 28 | for detail in jsonResponse['error']['details']: 29 | print detail 30 | except ValueError, e: 31 | print 'An unspecified error occurred.' 32 | print e 33 | 34 | def searchPortal(portalUrl, query=None, totalResults=None, sortField='numviews', 35 | sortOrder='desc', token=None): 36 | ''' 37 | Search the portal using the specified query and search parameters. 38 | Optionally provide a token to return results visible to that user. 39 | ''' 40 | # Default results are returned by highest 41 | # number of views in descending order. 42 | allResults = [] 43 | if not totalResults or totalResults > 100: 44 | numResults = 100 45 | else: 46 | numResults = totalResults 47 | results = __search__(portalUrl, query, numResults, sortField, sortOrder, 0, 48 | token) 49 | 50 | if not 'error' in results.keys(): 51 | if not totalResults: 52 | totalResults = results['total'] # Return all of the results. 53 | allResults.extend(results['results']) 54 | while (results['nextStart'] > 0 and 55 | results['nextStart'] < totalResults): 56 | # Do some math to ensure it only 57 | # returns the total results requested. 58 | numResults = min(totalResults - results['nextStart'] + 1, 100) 59 | results = __search__(portalUrl=portal, query=query, 60 | numResults=numResults, sortField=sortField, 61 | sortOrder=sortOrder, token=token, 62 | start=results['nextStart']) 63 | allResults.extend(results['results']) 64 | return allResults 65 | else: 66 | print results['error']['message'] 67 | return results 68 | 69 | def __search__(portalUrl, query=None, numResults=100, sortField='numviews', 70 | sortOrder='desc', start=0, token=None): 71 | '''Retrieve a single page of search results.''' 72 | params = { 73 | 'q': query, 74 | 'num': numResults, 75 | 'sortField': sortField, 76 | 'sortOrder': sortOrder, 77 | 'f': 'json', 78 | 'start': start 79 | } 80 | if token: 81 | # Adding a token provides an authenticated search. 82 | params['token'] = token 83 | request = portal + '/sharing/rest/search?' + urllib.urlencode(params) 84 | results = json.loads(urllib.urlopen(request).read()) 85 | return results 86 | 87 | def updateWebmapService(webmapId, oldUrl, newUrl, token, portalUrl): 88 | '''Replaces the URL for a specified map service in a web map.''' 89 | try: 90 | params = urllib.urlencode({'token' : token, 91 | 'f' : 'json'}) 92 | print 'Getting Info for: ' + webmapId 93 | # Get the item data. 94 | reqUrl = (portalUrl + '/sharing/content/items/' + webmapId + 95 | '/data?' + params) 96 | itemDataReq = urllib.urlopen(reqUrl).read() 97 | itemString = str(itemDataReq) 98 | 99 | # Find the service URL to be replaced. 100 | if itemString.find(oldUrl) > -1: 101 | newString = itemString.replace(oldUrl, newUrl) 102 | # Get the item's info for the addItem parameters 103 | itemInfoReq = urllib.urlopen(portalUrl + 104 | '/sharing/content/items/' + 105 | webmapId + '?' + params) 106 | itemInfo = json.loads(itemInfoReq.read(), 107 | object_hook=__decodeDict__) 108 | print 'Updating ' + itemInfo['title'] 109 | 110 | # Post back the changes overwriting the old map 111 | modRequest = urllib.urlopen(portalUrl + 112 | '/sharing/content/users/' + 113 | itemInfo['owner'] + 114 | '/' + itemInfo['ownerFolder'] + 115 | '/items/' + webmapId + 116 | '/update?' + params , 117 | urllib.urlencode( 118 | {'text' : newString} 119 | )) 120 | 121 | # Evaluate the results to make sure it happened 122 | modResponse = json.loads(modRequest.read()) 123 | if modResponse.has_key('error'): 124 | raise AGOPostError(webmapId, modResponse['error']['message']) 125 | else: 126 | print 'Successfully updated the urls' 127 | else: 128 | print 'Didn\'t find any services with ' + oldUrl 129 | except ValueError as e: 130 | print 'Error - no web map specified' 131 | except AGOPostError as e: 132 | print e.webmap 133 | print 'Error updating web map ' + e.webmap + ': ' + e.msg 134 | 135 | # Helper functions for decoding the unicode values in the webmap json. 136 | def __decodeDict__(dct): 137 | newdict = {} 138 | for k, v in dct.iteritems(): 139 | k = __safeValue__(k) 140 | v = __safeValue__(v) 141 | newdict[k] = v 142 | return newdict 143 | 144 | def __safeValue__(inVal): 145 | outVal = inVal 146 | if isinstance(inVal, unicode): 147 | outVal = inVal.encode('utf-8') 148 | elif isinstance(inVal, list): 149 | outVal = __decode_list__(inVal) 150 | return outVal 151 | 152 | def __decode_list__(lst): 153 | newList = [] 154 | for i in lst: 155 | i = __safeValue__(i) 156 | newList.append(i) 157 | return newList 158 | 159 | class AGOPostError(Exception): 160 | def __init__(self, webmap, msg): 161 | print 'ok' 162 | self.webmap = webmap 163 | self.msg = msg 164 | 165 | # Run the script. 166 | if __name__ == '__main__': 167 | parser = argparse.ArgumentParser() 168 | parser.add_argument('portal', 169 | help=('url of the Portal (e.g. ' 170 | 'https://portal.domain.com:7443/arcgis)')) 171 | parser.add_argument('username', help='username') 172 | parser.add_argument('password', help='password') 173 | parser.add_argument('query', help='search string to find content') 174 | parser.add_argument('oldUrl', help='the URL to replace') 175 | parser.add_argument('newUrl', help='the new URL') 176 | # Read the command line arguments. 177 | args = parser.parse_args() 178 | portal = args.portal 179 | username = args.username 180 | password = args.password 181 | query = args.query 182 | oldUrl = args.oldUrl 183 | newUrl = args.newUrl 184 | 185 | # Get a token for the source Portal for ArcGIS. 186 | token = generateToken(username=username, password=password, 187 | portalUrl=portal) 188 | 189 | # Get a list of the content matching the query. 190 | content = searchPortal(portalUrl=portal, 191 | query=query, 192 | token=token) 193 | 194 | for item in content: 195 | if item['type'] == 'Web Map': 196 | updateWebmapService(item['id'], oldUrl, newUrl, token=token, 197 | portalUrl=portal) 198 | 199 | print 'Update complete.' --------------------------------------------------------------------------------