├── Demos-Power-BI-Activity-Log-And-APIs.ipynb └── README.md /Demos-Power-BI-Activity-Log-And-APIs.ipynb: -------------------------------------------------------------------------------- 1 | { 2 | "metadata": { 3 | "kernelspec": { 4 | "name": ".net-powershell", 5 | "display_name": ".NET (PowerShell)", 6 | "language": "PowerShell" 7 | }, 8 | "language_info": { 9 | "name": "PowerShell", 10 | "version": "7.0", 11 | "mimetype": "text/x-powershell", 12 | "file_extension": ".ps1", 13 | "pygments_lexer": "powershell" 14 | } 15 | }, 16 | "nbformat_minor": 2, 17 | "nbformat": 4, 18 | "cells": [ 19 | { 20 | "cell_type": "markdown", 21 | "source": [ 22 | "# What You Can Learn from the Power BI Activity Log and REST APIs \n", 23 | "\n", 24 | "This set of PowerShell scripts is from a community presentation focused on thePower BI Activity Log and the Power BI REST APIs. These scripts are focused on understanding the kind of data which is available for understanding usage patterns and activities in the Power BI Service, for the purpose of  Power BI governance, security, management, and adoption efforts. The primary emphasis is awareness of the data which is available, more so than the exact PowerShell syntax.  \n", 25 | "\n", 26 | "**Accompanying presentation materials:** [https://www.coatesdatastrategies.com/presentations/#What-You-Can-Learn-From-Power-BI-Activity-Log](https://www.coatesdatastrategies.com/presentations/#What-You-Can-Learn-From-Power-BI-Activity-Log)  \n", 27 | "\n", 28 | "**Author**: Melissa Coates  \n", 29 | "\n", 30 | "**Last updated**: May 6, 2021  \n", 31 | "\n", 32 | "**Last tested with**: PowerShell 7.1 on Windows 10  \n", 33 | "\n", 34 | "**Code status**: These scripts are considered demo snippets (not production-ready) for learning purposes. The examples are realistic, but highly simplified.  \n", 35 | "\n", 36 | "**Notes**:\n", 37 | "\n", 38 | "- This is a Jupyter notebook (formerly iPython notebook, hence the ipynb file extension).\n", 39 | "- If using the PowerShell extension for Azure Data Studio as shown in this presentation: it requires the PowerShell extension. It also requires use of .NET (PowerShell) kernel (aka .NET interactive) instead of regular PowerShell kernel in order to use PowerShell Core in the notebook cells. (This requirement was last verified Nov. 2020.) See [this video](https://www.youtube.com/watch?v=W-F0gO7dVOE) on .NET interactive notebooks for more details.\n", 40 | "- The Power BI Management module for PowerShell is required to be installed to run most of these scripts:  [https://www.powershellgallery.com/packages/MicrosoftPowerBIMgmt](https://www.powershellgallery.com/packages/MicrosoftPowerBIMgmt)\n", 41 | "- The Data Gateway module is required to be installed to run the last few scripts:  [https://www.powershellgallery.com/packages/DataGateway](https://www.powershellgallery.com/packages/DataGateway)\n", 42 | "- Additional information is available in the accompanying presentation materials." 43 | ], 44 | "metadata": { 45 | "azdata_cell_guid": "252eeaf8-2bd2-44a4-8e7e-31ceac86b0c7" 46 | } 47 | }, 48 | { 49 | "cell_type": "markdown", 50 | "source": [ 51 | "## **Demo Series #1: Power BI Management Module**" 52 | ], 53 | "metadata": { 54 | "azdata_cell_guid": "b5d759ef-5d23-44b5-af5f-6f73ce13b7e0" 55 | } 56 | }, 57 | { 58 | "cell_type": "markdown", 59 | "source": [ 60 | "## 1-1. Check PowerShell version\n", 61 | "- The Power BI Management module is supported on Windows PowerShell or PowerShell Core.  \n", 62 | "- However, PowerShell Core (7.0+) is required for the Data Gateway module." 63 | ], 64 | "metadata": { 65 | "azdata_cell_guid": "34a16c67-23a8-4610-850e-845423d08ac7" 66 | } 67 | }, 68 | { 69 | "cell_type": "code", 70 | "source": [ 71 | "$PSVersionTable" 72 | ], 73 | "metadata": { 74 | "azdata_cell_guid": "ea211d46-1cbb-4fb7-b2f2-598e1a193231" 75 | }, 76 | "outputs": [], 77 | "execution_count": null 78 | }, 79 | { 80 | "cell_type": "markdown", 81 | "source": [ 82 | "## 1-2. Check Power BI Management module version(s)\n", 83 | "- This script checks which version(s) of the Power BI Management Module are installed & where." 84 | ], 85 | "metadata": { 86 | "azdata_cell_guid": "dca10acf-094e-4af6-af41-a0b98caca949" 87 | } 88 | }, 89 | { 90 | "cell_type": "code", 91 | "source": [ 92 | "Get-Module MicrosoftPowerBIMgmt* -ListAvailable | Sort-Object Version, Name " 93 | ], 94 | "metadata": { 95 | "azdata_cell_guid": "78db3a46-b4ab-497c-b69b-c3fc11265d78", 96 | "tags": [] 97 | }, 98 | "outputs": [], 99 | "execution_count": null 100 | }, 101 | { 102 | "cell_type": "markdown", 103 | "source": [ 104 | "## 1-3 Check Power BI Management module commands\n", 105 | "- The first script lists cmdlets, sorted by name.\n", 106 | "- The second script retrieves just the Profile module commands. Note there are several aliases for Connect-PowerBIServiceAccount." 107 | ], 108 | "metadata": { 109 | "azdata_cell_guid": "5b8685f8-6521-4245-91ce-a224237493c1" 110 | } 111 | }, 112 | { 113 | "cell_type": "code", 114 | "source": [ 115 | "$Version = '1.0.974'\r\n", 116 | "Get-Command -Module MicrosoftPowerBIMgmt* -Version $Version | Sort-Object Name " 117 | ], 118 | "metadata": { 119 | "azdata_cell_guid": "f17965a6-2154-470a-a3ca-0ccd79969014" 120 | }, 121 | "outputs": [], 122 | "execution_count": null 123 | }, 124 | { 125 | "cell_type": "code", 126 | "source": [ 127 | "$Version = '1.0.974'\r\n", 128 | "Get-Command -Module MicrosoftPowerBIMgmt* | Where-Object {$PSItem.Source -eq \"MicrosoftPowerBIMgmt.Profile\" -and $PSItem.Version -eq $Version} | Sort-Object Source, Name" 129 | ], 130 | "metadata": { 131 | "azdata_cell_guid": "1e38d2f6-f19a-490a-b1ed-94ac336ad20c", 132 | "tags": [] 133 | }, 134 | "outputs": [], 135 | "execution_count": null 136 | }, 137 | { 138 | "cell_type": "markdown", 139 | "source": [ 140 | "## 1-4. Get help for a cmdlet\n", 141 | "- The first script displays the standard help information for the Get-PowerBIWorkspace cmdlet.\n", 142 | "- The second script displays example syntax." 143 | ], 144 | "metadata": { 145 | "azdata_cell_guid": "87859d51-1d1a-4db0-9730-739e69f726d7" 146 | } 147 | }, 148 | { 149 | "cell_type": "code", 150 | "source": [ 151 | "Get-Help Get-PowerBIWorkspace" 152 | ], 153 | "metadata": { 154 | "azdata_cell_guid": "545ae63c-2cd9-4cf8-a6fb-41fa392c0b40", 155 | "tags": [] 156 | }, 157 | "outputs": [], 158 | "execution_count": null 159 | }, 160 | { 161 | "cell_type": "code", 162 | "source": [ 163 | "Get-Help Get-PowerBIWorkspace -Examples" 164 | ], 165 | "metadata": { 166 | "azdata_cell_guid": "088e43a3-86a4-4e0c-a630-62817c4f99f3" 167 | }, 168 | "outputs": [], 169 | "execution_count": null 170 | }, 171 | { 172 | "cell_type": "markdown", 173 | "source": [ 174 | "## 1-5. Authenticate to the Power BI Service with a Domain Account\n", 175 | "**Use \\*\\*either\\*\\* this option for sign-in, or the next option below (service principal).** Using a domain account for authentication is useful when scripts are being run interactively by a person.\n", 176 | "- This script interactively prompts for the user email address and the password to be passed into a credential object.\n", 177 | " \n", 178 | "- For simplicity, a domain user is being used for authentication. The domain user must possess Power BI Administrator permissions to execute some of the scripts in this notebook (i.e., the the admin cmdlets & APIs which rely on accessing organization-wide metadata for the entire tenant). For automated / scheduled processes, a better practice is to use an Azure AD application (service principal) -- this became supported for the admin cmdlets & APIs as of Dec 2020. If you do use a domain user for this purpose, make sure multi-factor authentication (MFA) is not enabled for it & that it's either a cloud-only account (in Azure AD) or the password hash has been allowed to synchronize from AD to Azure AD. \n", 179 | " \n", 180 | "- The Power BI Management Module is used for all authentication throughout all scripts in this notebook (including when APIs are being called because they're being called by the Management Module)..\n", 181 | " \n", 182 | "- All scripts below this point in the notebook require authentication. After signing in, the token acquired is active for one hour." 183 | ], 184 | "metadata": { 185 | "azdata_cell_guid": "398bcc30-159e-4541-8e6c-7cfe405c875a" 186 | } 187 | }, 188 | { 189 | "cell_type": "code", 190 | "source": [ 191 | "#Prompt for account name:\r\n", 192 | "[string]$DomainUserEmailAddr = Read-Host -Prompt \"Input domain user email address\"\r\n", 193 | "\r\n", 194 | "#Prompt for account password: \r\n", 195 | "[securestring]$DomainUserPW = Read-Host -Prompt \"Input password for $DomainUserEmailAddr\" -AsSecureString\r\n", 196 | "\r\n", 197 | "#Create temporary credential object: \r\n", 198 | "[pscredential]$CredentialObj = New-Object System.Management.Automation.PSCredential($DomainUserEmailAddr, $DomainUserPW) \r\n", 199 | "\r\n", 200 | "#Log into Power BI Service with credential:\r\n", 201 | "Connect-PowerBIServiceAccount -Credential $CredentialObj" 202 | ], 203 | "metadata": { 204 | "azdata_cell_guid": "34d98b29-eb6d-4458-9235-e9f75323e90a" 205 | }, 206 | "outputs": [], 207 | "execution_count": null 208 | }, 209 | { 210 | "cell_type": "markdown", 211 | "source": [ 212 | "## 1-6. Authenticate to the Power BI Service with a Service Principal\n", 213 | "**Use \\*\\*either\\*\\* this option for sign-in, or the previous option above (domain account).** Using a service principal (Azure Active Directory application) is useful for executing scheduled, unattended, operations.  **NOTE**: not all of the scripts in this notebook work with service principal authentication. See [https://docs.microsoft.com/en-us/power-bi/admin/read-only-apis-service-principal-authentication#supported-apis](https://docs.microsoft.com/en-us/power-bi/admin/read-only-apis-service-principal-authentication#supported-apis)  --AND-- [https://docs.microsoft.com/en-us/rest/api/power-bi/admin](https://docs.microsoft.com/en-us/rest/api/power-bi/admin)\n", 214 | "- This script interactively prompts for the service principal secret to be passed into a credential object. When operationalizing this script, the prompt will need to be removed. Consider using the new Secret Management Module ([https://www.powershellgallery.com/packages/Microsoft.PowerShell.SecretManagement](https://www.powershellgallery.com/packages/Microsoft.PowerShell.SecretManagement)) published by Microsoft for managing the secret value securely in either a local vault or in Azure Key Vault. Do not store the secret directly in the script, or in a separate file unencrypted. \n", 215 | " \n", 216 | "- The service principal (Azure AD application) requires:  (1) An Azure AD app to be registered.  (2) A secret created for the app. (Watch out for expiration dates - it defaults to 1 year.)   (3) Azure AD app membership in an Azure AD group. Suggested group name: Power BI Admin Service Principals.   (4) Power BI tenant setting of \"Allow service principals to use read-only Power BI admin APIs enabled for the group referenced in item (4). There's no need to set up API permissions in Azure AD for this app; in fact that can cause an unauthorized error. Use of the group & the tenant setting per (3) and (4) are what controls tenant-wide read permissions for Power BI. More info: [https://docs.microsoft.com/en-us/power-bi/admin/read-only-apis-service-principal-authentication#supported-apis](https://docs.microsoft.com/en-us/power-bi/admin/read-only-apis-service-principal-authentication#supported-apis)\n", 217 | " \n", 218 | "- The Power BI Management Module is used for all authentication throughout all scripts in this notebook (including when APIs are being called because they're being called by the Management Module)..\n", 219 | " \n", 220 | "- All scripts below this point in the notebook require authentication. After signing in, the token acquired is active for one hour." 221 | ], 222 | "metadata": { 223 | "azdata_cell_guid": "f9694c76-9658-4818-bba2-66b5ec263db5" 224 | } 225 | }, 226 | { 227 | "cell_type": "code", 228 | "source": [ 229 | "[string]$AzureADAppID = 'InputIDHere'\r\n", 230 | "[string]$AzureTenantID = 'InputIDHere'\r\n", 231 | "\r\n", 232 | "#Prompt for Azure AD App secret: \r\n", 233 | "[securestring]$AzureADAppSecret = Read-Host -Prompt \"Input secret for $AzureADAppID\" -AsSecureString\r\n", 234 | "\r\n", 235 | "#Create temporary credential object: \r\n", 236 | "[pscredential]$CredentialObj = New-Object System.Management.Automation.PSCredential($AzureADAppID, $AzureADAppSecret) \r\n", 237 | "\r\n", 238 | "#Log into Power BI Service with credential:\r\n", 239 | "Connect-PowerBIServiceAccount -Credential $CredentialObj -ServicePrincipal -Tenant $AzureTenantID " 240 | ], 241 | "metadata": { 242 | "azdata_cell_guid": "9c54479e-a658-46e8-a574-db71f558384d" 243 | }, 244 | "outputs": [], 245 | "execution_count": null 246 | }, 247 | { 248 | "cell_type": "markdown", 249 | "source": [ 250 | "## 1-7. View workspace info - based on user scope\n", 251 | "- The first script returns the first 2 workspaces it finds, using permissions levels of the signed-in user. If -All is not specified, by default it returns the first 100 workspaces.\n", 252 | "- The second script returns one workspace, using permissions of the signed-in user. If you're logged in with a service principal, nothing will be returned from these initial user scope queries unless workspace permissions have been assigned to the service principal." 253 | ], 254 | "metadata": { 255 | "azdata_cell_guid": "732da842-811f-44a9-93d0-440d63824d74" 256 | } 257 | }, 258 | { 259 | "cell_type": "code", 260 | "source": [ 261 | "Get-PowerBIWorkspace -First 2" 262 | ], 263 | "metadata": { 264 | "azdata_cell_guid": "f30f7707-6bd7-425e-9982-e2e631c87192", 265 | "tags": [] 266 | }, 267 | "outputs": [], 268 | "execution_count": null 269 | }, 270 | { 271 | "cell_type": "code", 272 | "source": [ 273 | "Get-PowerBIWorkspace -Name 'Sales Analytics'" 274 | ], 275 | "metadata": { 276 | "azdata_cell_guid": "23a181f5-8ada-4f0b-80b1-eb0e713278b8", 277 | "tags": [] 278 | }, 279 | "outputs": [], 280 | "execution_count": null 281 | }, 282 | { 283 | "cell_type": "markdown", 284 | "source": [ 285 | "## 1-8. View workspace info - based on organization (admin) scope\n", 286 | "- The first script returns one workspace. It requires Power BI admin permissions due to the organization scope (thus this metadata can be returned whether or not the administrator possesses workspace permissions).\n", 287 | "- The second script returns all active workspaces for the organization & reformats certain columns for display. It also requires Power BI admin permissions due to the organization scope." 288 | ], 289 | "metadata": { 290 | "azdata_cell_guid": "e39993bf-b894-4770-92a1-2a98fd375fa1" 291 | } 292 | }, 293 | { 294 | "cell_type": "code", 295 | "source": [ 296 | "Get-PowerBIWorkspace -Scope Organization -Name 'Sales Analytics'" 297 | ], 298 | "metadata": { 299 | "azdata_cell_guid": "77dbaa76-0f5b-411b-9b74-adc882f82c1e", 300 | "tags": [] 301 | }, 302 | "outputs": [], 303 | "execution_count": null 304 | }, 305 | { 306 | "cell_type": "code", 307 | "source": [ 308 | "Get-PowerBIWorkspace -Scope Organization | Where-Object State -eq 'Active' | Format-Table Name, Type, IsOnDedicatedCapacity" 309 | ], 310 | "metadata": { 311 | "azdata_cell_guid": "c75e0763-a2af-405e-b964-8c8aebfed512", 312 | "tags": [] 313 | }, 314 | "outputs": [], 315 | "execution_count": null 316 | }, 317 | { 318 | "cell_type": "markdown", 319 | "source": [ 320 | "## 1-9. View workspace artifact info\n", 321 | "- The first script returns a list of all dashboards in the tenant. Note that the results do not indicate which workspace the object resides in.\n", 322 | "- The second script returns a list of dashboards for a workspace. In this example, the parent workspace info is available in addition to the dashboards info. The results are displayed as a JSON object.\n", 323 | "- The third script incorporates the -Include All parameter to retrieve all objects (reports, dashboards, workbooks, dataflows, and datasets) for one workspace." 324 | ], 325 | "metadata": { 326 | "azdata_cell_guid": "81834746-1c44-41dd-b68e-064da7a3753a" 327 | } 328 | }, 329 | { 330 | "cell_type": "code", 331 | "source": [ 332 | "Get-PowerBIDashboard -Scope Organization | Sort-Object Name" 333 | ], 334 | "metadata": { 335 | "azdata_cell_guid": "63d03e89-28db-40b6-831c-b7add307e8d7" 336 | }, 337 | "outputs": [], 338 | "execution_count": null 339 | }, 340 | { 341 | "cell_type": "code", 342 | "source": [ 343 | "Get-PowerBIWorkspace -Scope Organization -Name 'Sales Analytics' -Include Dashboards | ConvertTo-JSON -Depth 10" 344 | ], 345 | "metadata": { 346 | "azdata_cell_guid": "da9b40bf-ee19-4046-aa5f-a14147c441b0" 347 | }, 348 | "outputs": [], 349 | "execution_count": null 350 | }, 351 | { 352 | "cell_type": "code", 353 | "source": [ 354 | "Get-PowerBIWorkspace -Scope Organization -Name 'Sales Analytics' -Include All | ConvertTo-JSON -Depth 10" 355 | ], 356 | "metadata": { 357 | "azdata_cell_guid": "521c1589-b8b5-4e60-974e-8a6db137f77b" 358 | }, 359 | "outputs": [], 360 | "execution_count": null 361 | }, 362 | { 363 | "cell_type": "markdown", 364 | "source": [ 365 | "## 1-10. Export workspace inventory\n", 366 | "- This script extracts all objects for all workspace, and exports them to one JSON file, which is considered a snapshot at that point in time.\n", 367 | "- This approach is useful for larger tenant which have a lot of data, as it minimizes the # of API calls to be made. The limit is 200 API calls per hour.\n", 368 | "- This technique is useful when you want to store a snapshot of workspace inventory on a regular basis, and/or your want to see changes over time with point-in-time snapshots. \n", 369 | "- The JSON format is used because, over time, new data elements will be introduced and a flexible schema is necessary to accomodate that." 370 | ], 371 | "metadata": { 372 | "azdata_cell_guid": "be4bd188-9ae4-4026-add2-04bc998918ab" 373 | } 374 | }, 375 | { 376 | "cell_type": "code", 377 | "source": [ 378 | "[string]$ExportFileLocation = 'C:\\Demos\\Demo-Output'\r\n", 379 | "[string]$DateTimeFileWrittenUTCLabel = ([datetime]::Now.ToUniversalTime()).ToString(\"yyyyMMddHHmm\") #Snapshot date when data was extracted\r\n", 380 | "[string]$ExportFileName = 'AllWorkspaceObjects-FromCmdlet-' + $DateTimeFileWrittenUTCLabel + '.json' #FromCmdlet in the name since the next example does the same thing using the API directly\r\n", 381 | "\r\n", 382 | "Get-PowerBIWorkspace -Scope Organization -Include All | ConvertTo-JSON -Depth 10 | Out-File \"$ExportFileLocation\\$ExportFileName\"\r\n", 383 | "\r\n", 384 | "Write-Verbose \"File written: $ExportFileName\" -Verbose " 385 | ], 386 | "metadata": { 387 | "azdata_cell_guid": "bf7a0608-62f5-4860-8346-fb84223d8782", 388 | "tags": [] 389 | }, 390 | "outputs": [], 391 | "execution_count": null 392 | }, 393 | { 394 | "cell_type": "markdown", 395 | "source": [ 396 | "## **Demo Series #2: Power BI REST APIs**" 397 | ], 398 | "metadata": { 399 | "azdata_cell_guid": "7a34df41-fac8-4e50-8de1-04d74ea0991d" 400 | } 401 | }, 402 | { 403 | "cell_type": "markdown", 404 | "source": [ 405 | "## 2-1. View workspace artifact info for one workspace\n", 406 | "- This script calls the admin groups API.\n", 407 | "- It uses the $expand parameter to retrieve metadata about all types of artifacts for the workspace. \n", 408 | "- It includes a filter parameter on workspace name." 409 | ], 410 | "metadata": { 411 | "azdata_cell_guid": "c541c71c-1c85-466a-acd5-21ac44f535ce" 412 | } 413 | }, 414 | { 415 | "cell_type": "code", 416 | "source": [ 417 | "[int32]$NbrOfRecordsToFetchPerBatch = 1000 \r\n", 418 | "[int32]$NbrOfRecordsToSkip = 0 \r\n", 419 | "[string]$WorkspaceName = '''Sales Data'''\r\n", 420 | "\r\n", 421 | "[string]$URL = \"https://api.powerbi.com/v1.0/myorg/admin/Groups?%24top=$NbrOfRecordsToFetchPerBatch&%24skip=$NbrOfRecordsToSkip&%24expand=datasets,dataflows,reports,dashboards,workbooks,users&%24filter=name eq $WorkspaceName\"\r\n", 422 | "\r\n", 423 | "Invoke-PowerBIRestMethod -Url $URL -Method GET" 424 | ], 425 | "metadata": { 426 | "azdata_cell_guid": "7f4ae86d-158c-477b-9237-1d1177f968b3" 427 | }, 428 | "outputs": [], 429 | "execution_count": null 430 | }, 431 | { 432 | "cell_type": "markdown", 433 | "source": [ 434 | "## 2-2. Export workspace inventory - using REST API\n", 435 | "- This script accomplishes the same thing as the earlier 'Export workspace inventory' script; this one just uses the REST API instead.\n", 436 | "- It calls the Power BI Groups REST API via the 'Invoke-PowerBIRestMethod' cmdlet from the Power BI Management Module. Therefore, the Power BI Management Module is still acting like a 'wrapper' around the API call.\n", 437 | "- This technique allows us to use authentication with the Power BI Management Module, which is simpler than OAuth authentication.\n", 438 | "- The $expand parameter specifies which objects are included.\n", 439 | "- The %24 shown in the URL is because the $ sign is used by PowerShell for variables & parameters. For this reason, the $ sign is replaced in PowerShell scripts with the %24." 440 | ], 441 | "metadata": { 442 | "azdata_cell_guid": "4b4a48a4-3afb-4add-a71c-fb213cec070e" 443 | } 444 | }, 445 | { 446 | "cell_type": "code", 447 | "source": [ 448 | "[int32]$NbrOfRecordsToFetchPerBatch = 1000 \r\n", 449 | "[int32]$NbrOfRecordsToSkip = 0 \r\n", 450 | "\r\n", 451 | "[string]$ExportFileLocation = 'C:\\Demos\\Demo-Output'\r\n", 452 | "[string]$DateTimeFileWrittenUTCLabel = ([datetime]::Now.ToUniversalTime()).ToString(\"yyyyMMddHHmm\") #Snapshot date when data was extracted\r\n", 453 | "[string]$ExportFileName = 'AllWorkspaceObjects-FromAPI-' + $DateTimeFileWrittenUTCLabel + '.json' #FromAPI in the name since the previous example did the same thing using the cmdlet\r\n", 454 | "\r\n", 455 | "[string]$URL = \"https://api.powerbi.com/v1.0/myorg/admin/Groups?%24top=$NbrOfRecordsToFetchPerBatch&%24skip=$NbrOfRecordsToSkip&%24expand=datasets,dataflows,reports,dashboards,workbooks,users\"\r\n", 456 | "\r\n", 457 | "Invoke-PowerBIRestMethod -Url $URL -Method GET | Out-File \"$ExportFileLocation\\$ExportFileName\"\r\n", 458 | "\r\n", 459 | "Write-Verbose \"File written: $ExportFileName\" -Verbose " 460 | ], 461 | "metadata": { 462 | "azdata_cell_guid": "fccdde65-871b-417b-ba1f-724994365959", 463 | "tags": [] 464 | }, 465 | "outputs": [], 466 | "execution_count": null 467 | }, 468 | { 469 | "cell_type": "markdown", 470 | "source": [ 471 | "## 2-3. View apps info\n", 472 | "- This script returns apps published for the entire tenant. \n", 473 | "- It converts the JSON results into a PowerShell object for viewing." 474 | ], 475 | "metadata": { 476 | "azdata_cell_guid": "1ac3e6f6-d7d2-4679-b6fe-b97ca9f627fb" 477 | } 478 | }, 479 | { 480 | "cell_type": "code", 481 | "source": [ 482 | "$URL = \"https://api.powerbi.com/v1.0/myorg/apps\"\r\n", 483 | "$AppsInfo = Invoke-PowerBIRestMethod -Url $URL -Method GET\r\n", 484 | "$AppsObject = $AppsInfo | ConvertFrom-Json\r\n", 485 | "$AppsObject.value " 486 | ], 487 | "metadata": { 488 | "azdata_cell_guid": "79c48bb5-76be-4b02-b2b1-57e0b8acc886" 489 | }, 490 | "outputs": [], 491 | "execution_count": null 492 | }, 493 | { 494 | "cell_type": "markdown", 495 | "source": [ 496 | "## 2-4. View dataset and report info for PBIX files in a workspace\n", 497 | "- This script displays the dataset and report metadata for each imported PBIX file in a workspace, such as when the PBIX file was last updated. (The file update date is separate from when it was refreshed--see the next example below for getting refresh date.)\n", 498 | "- Step 1 gets the workspace ID based on the name entered in the variable. The API requires single quotes to be sent around the workspace name, which is why the $WorkspaceName variable is constructed the way it is. This is an admin API which requires administrator permissions.\n", 499 | "- Step 2 gets the PBIX file info. The imports API does not extend to related objects or lineage beyond a single PBIX." 500 | ], 501 | "metadata": { 502 | "azdata_cell_guid": "5766382c-8888-4b27-88d7-5bbfa597ec67" 503 | } 504 | }, 505 | { 506 | "cell_type": "code", 507 | "source": [ 508 | "[int32]$NbrOfRecordsToFetchPerBatch = 1000 \r\n", 509 | "[int32]$NbrOfRecordsToSkip = 0 \r\n", 510 | "[string]$WorkspaceName = '''Sales Data'''\r\n", 511 | "\r\n", 512 | "#Step 1:\r\n", 513 | "[string]$URL = \"https://api.powerbi.com/v1.0/myorg/admin/Groups?%24top=$NbrOfRecordsToFetchPerBatch&%24skip=$NbrOfRecordsToSkip&%24filter=name eq $WorkspaceName\"\r\n", 514 | "$Workspace = Invoke-PowerBIRestMethod -Url $URL -Method GET\r\n", 515 | "$WorkspaceObject = $Workspace | ConvertFrom-Json \r\n", 516 | "$WorkspaceID = $WorkspaceObject.value.id \r\n", 517 | "\r\n", 518 | "#Step 2:\r\n", 519 | "$ImportFileURL = \"https://api.powerbi.com/v1.0/myorg/groups/$WorkspaceID/imports\"\r\n", 520 | "$ImportInfo = Invoke-PowerBIRestMethod -Url $ImportFileURL -Method GET\r\n", 521 | "$ImportObject = $ImportInfo | ConvertFrom-Json\r\n", 522 | "$ImportObject.value " 523 | ], 524 | "metadata": { 525 | "azdata_cell_guid": "2de13c17-61bc-415b-84d8-b3752c5a80dc", 526 | "tags": [] 527 | }, 528 | "outputs": [], 529 | "execution_count": null 530 | }, 531 | { 532 | "cell_type": "markdown", 533 | "source": [ 534 | "## 2-5. View dataset last refresh date\n", 535 | "- This script displays the last refresh date for a dataset, as well as other information such as how it was refreshed and the status.\n", 536 | "- Step 1 gets the workspace info because the ID will be needed in step 2.\n", 537 | "- Step 2 gets the datasets in the workspace which was specified. Each time the loop runs, the results are added to an array which is referenced in step 3.\n", 538 | "- Step 3 calls the refresh history API to retrieve the latest (top 1) refresh for each dataset in the workspace which was specified. It is converted from JSON into a PowerShell object for easier viewing in the console. Additional properties for the DatasetID and DatasetName are added to the resultset to help identify the lineage of the refresh information. Also shown is aliasing the ID column so its meaning is more clear. \n", 539 | "This technique is shown to be able to illustrate the logic. However, it does issue quite a few API calls in the looping structure. In large environments with a lot of artifacts, a different technique should be used (such as the workspace with the $expand property shown earlier)." 540 | ], 541 | "metadata": { 542 | "azdata_cell_guid": "e0e74f38-f8f7-44be-b60b-d7696248bdd3" 543 | } 544 | }, 545 | { 546 | "cell_type": "code", 547 | "source": [ 548 | "$WorkspaceName = 'Sales Data'\r\n", 549 | "$NbrOfRefreshes = 1 \r\n", 550 | "\r\n", 551 | "#Step 1:\r\n", 552 | "$Workspaces = Get-PowerBIWorkspace -Scope Organization -Name $WorkspaceName | Select-Object Id \r\n", 553 | "\r\n", 554 | "#Step 2:\r\n", 555 | "$Datasets = @()\r\n", 556 | " foreach ($Workspace in $Workspaces) \r\n", 557 | " {\r\n", 558 | " $Dataset = Get-PowerBIDataset -Scope Organization -WorkspaceId $Workspace.Id | Select-Object Id, Name\r\n", 559 | " $Datasets += $Dataset\r\n", 560 | " }\r\n", 561 | "\r\n", 562 | "#Step 3:\r\n", 563 | "foreach ($Dataset in $Datasets)\r\n", 564 | " { \r\n", 565 | " $DatasetID = $Dataset.Id\r\n", 566 | " $URL = \"https://api.powerbi.com/v1.0/myorg/datasets/$DatasetID/refreshes?%24top=$NbrOfRefreshes\"\r\n", 567 | " $RefreshInfo = Invoke-PowerBIRestMethod -Url $URL -Method GET\r\n", 568 | " $RefreshObject = $RefreshInfo | ConvertFrom-Json \r\n", 569 | " $DatasetRefreshInfo = $RefreshObject.value\r\n", 570 | " $DatasetRefreshInfo | Add-Member -MemberType 'NoteProperty' -Name 'DatasetID' -Value $Dataset.Id -Force \r\n", 571 | " $DatasetRefreshInfo | Add-Member -MemberType 'NoteProperty' -Name 'DatasetName' -Value $Dataset.Name -Force \r\n", 572 | " $DatasetRefreshInfo | Add-Member -MemberType 'AliasProperty' -Name 'RefreshID' -Value id\r\n", 573 | " $DatasetRefreshInfo \r\n", 574 | " }" 575 | ], 576 | "metadata": { 577 | "azdata_cell_guid": "c8cc60ee-ad1b-4a86-bb13-46b39fa78d9c", 578 | "tags": [] 579 | }, 580 | "outputs": [], 581 | "execution_count": null 582 | }, 583 | { 584 | "cell_type": "markdown", 585 | "source": [ 586 | "## **Demo Series #3: Power BI Activity Events**" 587 | ], 588 | "metadata": { 589 | "azdata_cell_guid": "7fcbc167-ab8a-47c4-8cb2-bb06d8472e66" 590 | } 591 | }, 592 | { 593 | "cell_type": "markdown", 594 | "source": [ 595 | "## 3-1. View one type of activity events for one day\n", 596 | "\n", 597 | "View data using cmdlet from the Power BI Management Module: Get-PowerBIActivityEvent.\n", 598 | "\n", 599 | "- This script displays the specific events, such as the \"ShareReport\" events, for one day. \n", 600 | "- It interactively prompts for the date. Data up to 30 days in the past may be retrieved.\n", 601 | "- All activity events are based on UTC datetime." 602 | ], 603 | "metadata": { 604 | "azdata_cell_guid": "13ee5967-a1ea-47ed-ad68-caec811e6c77" 605 | } 606 | }, 607 | { 608 | "cell_type": "code", 609 | "source": [ 610 | "[string]$ActivityType = 'ShareReport'\r\n", 611 | "[string]$DateToExtractLabel = Read-Host -Prompt \"Input date to extract in yyyy-mm-dd format\" #Maximum supported is 30 days back\r\n", 612 | "\r\n", 613 | "Get-PowerBIActivityEvent -StartDateTime ($DateToExtractLabel+'T00:00:00.000') -EndDateTime ($DateToExtractLabel+'T23:59:59.999') -ActivityType $ActivityType" 614 | ], 615 | "metadata": { 616 | "azdata_cell_guid": "000f39c1-ac79-45a4-9af4-3f04a5aa6d5f" 617 | }, 618 | "outputs": [], 619 | "execution_count": null 620 | }, 621 | { 622 | "cell_type": "markdown", 623 | "source": [ 624 | "## 3-2. Export activity events for a range of days\n", 625 | "- This script exports raw data for all Power BI events.\n", 626 | "- It exports one JSON file per day (based on UTC time), which is useful when you want to store raw data over time. \n", 627 | "- The range of time is specified with the first variable. It counts backwards from yesterday to determine how many days it should loop through.\n", 628 | "- A maximum of one day is allowed to be extracted at once, which is why a loop is introduced per day. \n", 629 | "- The JSON format is used because, over time, new data elements will be introduced and a flexible schema is necessary to accomodate that. It is a best practice to store the raw data as-is, in an immutable location when possible, so it can be used for auditing when necessary." 630 | ], 631 | "metadata": { 632 | "azdata_cell_guid": "88781de9-fdab-4524-82b4-ffa82597ceab" 633 | } 634 | }, 635 | { 636 | "cell_type": "code", 637 | "source": [ 638 | "[int32]$NbrDaysDaysToExtract = 7 #Maximum is 30 days back\r\n", 639 | "[string]$ExportFileLocation = 'C:\\Demos\\Demo-Output'\r\n", 640 | "\r\n", 641 | "[datetime]$YesterdayUTC = (([datetime]::Today.ToUniversalTime()).Date).AddDays(-1) #Begin with yesterday, rather than today, to ensure full day results are obtained\r\n", 642 | "[string]$DateTimeFileWrittenUTCLabel = ([datetime]::Now.ToUniversalTime()).ToString(\"yyyyMMddHHmm\")\r\n", 643 | "\r\n", 644 | "#Loop through each of the days to be extracted ( ; ; ):\r\n", 645 | "for($Loop=0 ; $Loop -lt $NbrDaysDaysToExtract ; $Loop++)\r\n", 646 | "{\r\n", 647 | " [datetime]$DateToExtractUTC=$YesterdayUTC.AddDays(-$Loop).ToString(\"yyyy-MM-dd\")\r\n", 648 | "\r\n", 649 | " [string]$DateToExtractLabel=$DateToExtractUTC.ToString(\"yyyy-MM-dd\")\r\n", 650 | " \r\n", 651 | " [string]$ExportFileName = 'AllActivityEvents-' + ($DateToExtractLabel -replace '-', '') + '-' + $DateTimeFileWrittenUTCLabel + '.json' \r\n", 652 | "\r\n", 653 | " [psobject]$Events=Get-PowerBIActivityEvent -StartDateTime ($DateToExtractLabel+'T00:00:00.000') -EndDateTime ($DateToExtractLabel+'T23:59:59.999')\r\n", 654 | " \r\n", 655 | " $Events | Out-File \"$ExportFileLocation\\$ExportFileName\"\r\n", 656 | "\r\n", 657 | " Write-Verbose \"File written: $ExportFileName\" -Verbose \r\n", 658 | "}" 659 | ], 660 | "metadata": { 661 | "azdata_cell_guid": "3970c288-b201-4268-a87b-70d32f4641b8", 662 | "tags": [] 663 | }, 664 | "outputs": [], 665 | "execution_count": null 666 | }, 667 | { 668 | "cell_type": "markdown", 669 | "source": [ 670 | "## **Demo Series #4: Data Gateway Module**" 671 | ], 672 | "metadata": { 673 | "azdata_cell_guid": "76b0cb1b-aecf-4a82-b27d-de22da7dd484" 674 | } 675 | }, 676 | { 677 | "cell_type": "markdown", 678 | "source": [ 679 | "## 4-1. Check Data Gateway module version(s)\n", 680 | "- This script checks which version(s) of the Data Gateway Module are installed. Unlike the Power BI Management Module, the Data Gateway Module requires PowerShell Core (7.0+)." 681 | ], 682 | "metadata": { 683 | "azdata_cell_guid": "4190c47a-4ba9-486b-aad1-ba3fdeb103f5" 684 | } 685 | }, 686 | { 687 | "cell_type": "code", 688 | "source": [ 689 | "Get-Module DataGateway* -ListAvailable | Sort-Object Version, Name " 690 | ], 691 | "metadata": { 692 | "azdata_cell_guid": "83e67c3a-f244-4ed3-aff1-1053f017600f" 693 | }, 694 | "outputs": [], 695 | "execution_count": null 696 | }, 697 | { 698 | "cell_type": "markdown", 699 | "source": [ 700 | "## 4-2. Check Data Gateway module commands\n", 701 | "- The first script lists all cmdlets & aliases, for all versions.\n", 702 | "- The second script retrieves just the Profile module commands. Similar to the Power BI Mgmt Module, there are several aliases for Connect-DataGatewayServiceAccount." 703 | ], 704 | "metadata": { 705 | "azdata_cell_guid": "3a5d6e91-9063-49a6-ab88-9d6a432625b0" 706 | } 707 | }, 708 | { 709 | "cell_type": "code", 710 | "source": [ 711 | "Get-Command -Module DataGateway*" 712 | ], 713 | "metadata": { 714 | "azdata_cell_guid": "a247ca3e-281f-4b44-a258-8c1b593a387a" 715 | }, 716 | "outputs": [], 717 | "execution_count": null 718 | }, 719 | { 720 | "cell_type": "code", 721 | "source": [ 722 | "Get-Command -Module DataGateway* | Where-Object {$PSItem.Source -eq \"DataGateway.Profile\"} | Sort-Object Source, Name" 723 | ], 724 | "metadata": { 725 | "azdata_cell_guid": "fbffce5a-cad0-4187-a48c-60cbfa032ddc" 726 | }, 727 | "outputs": [], 728 | "execution_count": null 729 | }, 730 | { 731 | "cell_type": "markdown", 732 | "source": [ 733 | "## 4-3 Authenticate to the Data Gateway service\n", 734 | "- Note this is a \\*\\*separate\\*\\* login cmdlet from the Power BI Management Module. \n", 735 | "- This interactive login will complete in your default web browser." 736 | ], 737 | "metadata": { 738 | "azdata_cell_guid": "b307f4c2-30e2-465b-98fd-388089a3a7ac" 739 | } 740 | }, 741 | { 742 | "cell_type": "code", 743 | "source": [ 744 | "Connect-DataGatewayServiceAccount " 745 | ], 746 | "metadata": { 747 | "azdata_cell_guid": "ba2e3e78-5add-47c4-a207-d329eaf19546" 748 | }, 749 | "outputs": [], 750 | "execution_count": null 751 | }, 752 | { 753 | "cell_type": "markdown", 754 | "source": [ 755 | "## 4-4. View gateway cluster info\n", 756 | "- This script uses the data gateway module to view metadata about each gateway cluster." 757 | ], 758 | "metadata": { 759 | "azdata_cell_guid": "21012092-f63a-48a9-9747-20980c088645" 760 | } 761 | }, 762 | { 763 | "cell_type": "code", 764 | "source": [ 765 | "Get-DataGatewayCluster -Scope Organization" 766 | ], 767 | "metadata": { 768 | "azdata_cell_guid": "d83eb208-619f-4f77-994e-d648030a2913" 769 | }, 770 | "outputs": [], 771 | "execution_count": null 772 | }, 773 | { 774 | "cell_type": "markdown", 775 | "source": [ 776 | "## 4-5. View gateway data source info\n", 777 | "- This script displays each data source registered to the gateway cluster, including the type. Server name and database name are included in the connection details." 778 | ], 779 | "metadata": { 780 | "azdata_cell_guid": "5ddd1c6c-e517-43cf-a065-88a6f662e100" 781 | } 782 | }, 783 | { 784 | "cell_type": "code", 785 | "source": [ 786 | "$GWClusters = Get-DataGatewayCluster -Scope Organization\r\n", 787 | "\r\n", 788 | "foreach ($GWCluster in $GWClusters) \r\n", 789 | "{\r\n", 790 | " Get-DataGatewayClusterDatasource -GatewayClusterId $GWCluster.Id\r\n", 791 | "}" 792 | ], 793 | "metadata": { 794 | "azdata_cell_guid": "9a4e1761-c91a-422f-b395-bb70253774b8" 795 | }, 796 | "outputs": [], 797 | "execution_count": null 798 | }, 799 | { 800 | "cell_type": "markdown", 801 | "source": [ 802 | "## 4-6. View data source users for each gateway data source\n", 803 | "- This script displays the users which have been added to each data source.\n", 804 | "- Step 1 gets the gateway cluster info. It passes the cluster ID to step 2.\n", 805 | "- Step 2 retrieves each data source for the cluster. It passes the ID, name, and cluster ID to step 3.\n", 806 | "- Step 3 retrieves each user assigned to the data source. Additional properties (Cluster ID, Data Source Name, and Data Source ID) are added to the resultset to help identify the lineage of the user information." 807 | ], 808 | "metadata": { 809 | "azdata_cell_guid": "996c0755-82ff-4cd0-866b-4bde44ff84f6" 810 | } 811 | }, 812 | { 813 | "cell_type": "code", 814 | "source": [ 815 | "#Step 1:\r\n", 816 | "$GWClusters = Get-DataGatewayCluster -Scope Organization\r\n", 817 | "\r\n", 818 | "#Step 2:\r\n", 819 | "$GWClusterSources = @()\r\n", 820 | "foreach ($GWCluster in $GWClusters) \r\n", 821 | "{\r\n", 822 | " $GWClusterSource = Get-DataGatewayClusterDatasource -GatewayClusterId $GWCluster.Id | Select-Object Id, DatasourceName, ClusterId\r\n", 823 | " $GWClusterSources += $GWClusterSource\r\n", 824 | "}\r\n", 825 | "\r\n", 826 | "#Step 3:\r\n", 827 | "foreach ($GWClusterSource in $GWClusterSources)\r\n", 828 | "{ \r\n", 829 | " $Users = Get-DataGatewayClusterDatasourceUser -GatewayClusterId $GWClusterSource.ClusterId -GatewayClusterDatasourceId $GWClusterSource.Id\r\n", 830 | " $Users | Add-Member -MemberType 'NoteProperty' -Name 'ClusterID' -Value $GWClusterSource.ClusterId -Force \r\n", 831 | " $Users | Add-Member -MemberType 'NoteProperty' -Name 'DataSourceName' -Value $GWClusterSource.DatasourceName -Force\r\n", 832 | " $Users | Add-Member -MemberType 'NoteProperty' -Name 'DataSourceID' -Value $GWClusterSource.Id -Force\r\n", 833 | " $Users \r\n", 834 | "}" 835 | ], 836 | "metadata": { 837 | "azdata_cell_guid": "d4923b2b-8b73-4cdf-9972-805a48026a1e" 838 | }, 839 | "outputs": [], 840 | "execution_count": null 841 | }, 842 | { 843 | "cell_type": "markdown", 844 | "source": [ 845 | "## 4-7. Disconnect from the Power BI service & Data Gateway service\n", 846 | "These cmdlets are useful to place at the end of automation scripts, as a matter of good housekeeping." 847 | ], 848 | "metadata": { 849 | "azdata_cell_guid": "5de9e3f5-5e25-44d1-b7c5-ca8e4f524d5f" 850 | } 851 | }, 852 | { 853 | "cell_type": "code", 854 | "source": [ 855 | "Disconnect-PowerBIServiceAccount" 856 | ], 857 | "metadata": { 858 | "azdata_cell_guid": "63cb9f40-f3f9-44bc-a2b1-67e031759b66" 859 | }, 860 | "outputs": [], 861 | "execution_count": null 862 | }, 863 | { 864 | "cell_type": "code", 865 | "source": [ 866 | "Disconnect-DataGatewayServiceAccount" 867 | ], 868 | "metadata": { 869 | "azdata_cell_guid": "6a800229-5412-457b-931a-100ec13d08f7" 870 | }, 871 | "outputs": [], 872 | "execution_count": null 873 | } 874 | ] 875 | } -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Presentation Materials 2 | 3 | This repository includes select materials from community (public) presentations. 4 | 5 | Accompanying materials, including slide decks, can be found at: [coatesdatastrategies.com/presentations](https://www.coatesdatastrategies.com/presentations). 6 | --------------------------------------------------------------------------------