├── README.md ├── examples └── sas-microsoft365-example.sasnb ├── images ├── azure_access_code.png ├── creds-in-studio.png ├── example-listings.png ├── example-listings2.png └── list-drives-output.png └── ms-graph-macros.sas /README.md: -------------------------------------------------------------------------------- 1 | # Using the Microsoft Graph API from SAS 2 | 3 | The SAS code and macros in this project are designed to make common tasks 4 | easier when using SAS to access Microsoft 365 content. This includes OneDrive and 5 | SharePoint Online (including content in Microsoft Teams). 6 | 7 | To use SAS or any scripting language with Microsoft 365, you must first register a 8 | client app, authenticate with your identity to grant it permissions, and obtain an 9 | authorization code. With the auth code in hand, you can then use the code routines in this 10 | project to get an access token and invoke API methods to accomplish tasks such as: 11 | 12 | * List available drives in OneDrive 13 | * List folders and files within OneDrive and SharePoint folders (include files within Microsoft Teams) 14 | * Download files from OneDrive or SharePoint into your SAS session 15 | * Upload files from SAS to a folder on OneDrive or SharePoint 16 | 17 | For more guidance about how to register a client app for use with the Microsoft Graph API, 18 | see [Using SAS with Microsoft 365](https://blogs.sas.com/content/sasdummy/2020/07/09/sas-programming-office-365-onedrive/). 19 | 20 | **Watch**: [Demo: SAS Viya Workbench and SAS code to access Microsoft 365](https://communities.sas.com/t5/SAS-Viya-Workbench-Getting/Demo-SAS-Viya-Workbench-and-SAS-code-to-access-Microsoft-365/ta-p/952476) -- see this macro library in action. Note that these macros work from any SAS environment: PC SAS, SAS Enterprise Guide, SAS 9 and SAS Viya. 21 | 22 | ## Working within a firewall: Preparing your environment 23 | 24 | These methods use APIs from Microsoft to access your Microsoft 365 content. Microsoft 365 services are hosted in the cloud by Microsoft, and so your SAS session needs to be able to access these Internet services. 25 | 26 | If your SAS session is in a hosted environment or running behind a firewall that does not have direct access to the Internet, you need to perform a few additional steps to enable these methods. 27 | 28 | **Read:** [How to test PROC HTTP and Internet access in your environment](https://blogs.sas.com/content/sasdummy/2018/01/23/check-json-and-http/) 29 | 30 | ### Working with an HTTP proxy 31 | 32 | The code in this project uses PROC HTTP without proxy options. If your organization requires a proxy gateway to access the internet, 33 | specify the proxy value in the [special PROCHTTP_PROXY macro variable](https://go.documentation.sas.com/doc/en/pgmsascdc/v_063/proc/p0m87fzxykv1vyn14rftls0mbrkm.htm): 34 | ``` 35 | %let PROCHTTP_PROXY=proxyhost.company.com:889; 36 | ``` 37 | Add this line before calling any other actions. PROC HTTP will apply this proxy value for all methods. 38 | 39 | ### Modify the Allow List (whitelist) for Microsoft 365 endpoints 40 | 41 | If the network rules for your SAS environment block all Internet traffic except for endpoints or IP addresses that are explicitly permitted, then you will need to add at least the following endpoints to the allow list (_whitelist_). 42 | 43 | * `login.microsoftonline.com` - for authentication and token refresh 44 | * `graph.microsoft.com` - for API calls to the Microsoft Graph API 45 | * _your-tenant-site_`.sharepoint.com` - for downloadable files from your SharePoint and Teams sites. Example: `contoso.sharepoint.com`. 46 | * _your-tenant-site_`-my.sharepoint.com` - for OneDrive files folders (those with /personal in the path). Example: `contoso-my.sharepoint.com` The naming convention may vary, so check how your organization differentiates Teams and SharePoints site from OneDrive locations. 47 | 48 | **Note:** Micrososoft [publishes a complete list of IP ranges](https://learn.microsoft.com/en-us/microsoft-365/enterprise/urls-and-ip-address-ranges?view=o365-worldwide) to enable Microsoft 365 clients within a firewall, but the list is extensive and only a subset of these are needed for most SAS use cases. 49 | 50 | ## Create the config.json file with your client app details 51 | 52 | These macros use a file named config.json to reference your client app details, including the app ID and your Azure tenant ID. The file has this format: 53 | 54 | ```json 55 | { 56 | "tenant_id": "your-azure-tenant", 57 | "client_id": "your-app-client-id", 58 | "redirect_uri": "https://login.microsoftonline.com/common/oauth2/nativeclient", 59 | "resource" : "https://graph.microsoft.com" 60 | } 61 | ``` 62 | 63 | Designate a secure location for this file and for your token.json file (to be created in a later step). The information within these files is sensitive and specific to you and should be protected. See [How to protect your REST API credentials in SAS programs for guidance](https://blogs.sas.com/content/sasdummy/2018/01/16/hide-rest-api-tokens/). 64 | 65 | If you are using SAS Viya, you can optionally create this folder and file in your SAS Content area that is private to you. For example, create a ".creds" folder within "/Users/your.account/My Folder". By default, only your account will be able to read content you place there. 66 | 67 | ## Download and include ms-graph-macros.sas code 68 | 69 | This repository contains a SAS program (named [ms-graph-macros.sas](./ms-graph-macros.sas)) with all of the macro routines you need for the remaining tasks. Download this file to a local folder and use %INCLUDE to submit in SAS. 70 | 71 | ```sas 72 | %let src=\sas-microsoft-graph-api; 73 | %include "&src./ms-graph-macros.sas"; 74 | ``` 75 | 76 | You can also include directly from GitHub: 77 | ```sas 78 | /* Run just once in your session */ 79 | options dlcreatedir; 80 | %let repopath=%sysfunc(getoption(WORK))/sas-microsoft-graph-api; 81 | libname repo "&repopath."; 82 | data _null_; 83 | rc = git_clone( 84 | "https://github.com/sascommunities/sas-microsoft-graph-api", 85 | "&repoPath." 86 | ); 87 | put 'Git repo cloned ' rc=; 88 | run; 89 | %include "&repopath./ms-graph-macros.sas"; 90 | ``` 91 | 92 | ## Initialize the config folder 93 | 94 | The macro routines need to know where your config.json and token.json file are located. The ```initConfig``` macro initializes this. 95 | 96 | ```sas 97 | /* This path must contain your config.json, and will also */ 98 | /* be the location of your token.json */ 99 | %initConfig(configPath=/u/yourId/Projects/ms365); 100 | ``` 101 | If you are using SAS Viya and you would like to store your config and token files in the SAS Content folders (instead of the file system), this is supported with a boolean flag on ```initConfig```. For example, if you store config.json in a folder named *.creds* within your SAS Content user home, this tells the macro to look in that folder: 102 | 103 | ``` 104 | %initConfig(configPath=/Users/your.account/My Folder/.creds, sascontent=1); 105 | ``` 106 | 107 | **Note:** This ```sascontent``` flag is needed to tell the macro to use the FILENAME FILESVC method to access the SAS Content area. It requires a different file access method than traditional file systems. 108 | 109 | ## DO ONCE: Get an auth code 110 | 111 | **Note:** you need this step only if you haven't already generated an auth code and stored in token.json. See [Step 2 in this article](https://blogs.sas.com/content/sasdummy/2020/07/09/sas-programming-office-365-onedrive/). 112 | 113 | This helper macro will generate the URL you can use to generate an auth code. 114 | 115 | ```sas 116 | %generateAuthUrl(); 117 | ``` 118 | 119 | The SAS log will contain a URL that you should copy and paste into your browser. After authenticating to Microsoft 365 and granting permissions, the URL address bar will change to include a ```code=``` value that you need for the next step. **Copy only the code= value, not any other values that follow in the URL.** (Again, this is covered in [Step 2 of this article](https://blogs.sas.com/content/sasdummy/2020/07/09/sas-programming-office-365-onedrive/) -- reference for the specific steps to follow!) 120 | 121 | ![authcode in URL](./images/azure_access_code.png) 122 | 123 | ## DO ONCE: Generate the first access token 124 | 125 | If you just generated your auth code for the first time or needed to get a new one because the old one was revoked or expired, then you need to use the auth code to get an initial access token. 126 | ```sas 127 | /* Note: this code can be quite long -- 700+ characters. */ 128 | %let auth_code=PASTE-YOUR-AUTH-CODE-HERE; 129 | 130 | /* 131 | Now that we have an authorization code we can get the access token 132 | This step will write the token.json file that we can use in our 133 | production programs. 134 | */ 135 | %get_access_token(&auth_code.); 136 | 137 | ``` 138 | When successful, token.json will be created/updated in the config directory you specified. 139 | 140 | You should now have both config.json and token.json in your designated config folder. This screenshot shows an example of these files in a hidden folder named "~/.creds". 141 | 142 | ![example of config folder](./images/creds-in-studio.png) 143 | 144 | ## Refresh access token and connect to Microsoft 365 145 | 146 | Use the ```%initSessionMS365``` macro routine to exchange the refresh-token stored in token.json for an active non-expired access token. 147 | 148 | ```sas 149 | %initSessionMS365; 150 | ``` 151 | 152 | When this is successful, you will see notes similar to these in the SAS log: 153 | 154 | ``` 155 | M365: Reading token info from token.json 156 | M365: Token expires on 26JUL2024:10:04:22 157 | ``` 158 | 159 | The Microsoft Graph API session token is stored in the macro variable ```&access_token```, which is referenced implicitly in the other macro routines in this package. 160 | 161 | ## Methods to list content, download files, upload files 162 | 163 | With a valid access token to connect to Microsoft 365, we can now use various methods to discover and list content within OneDrive and SharePoint (including Teams), and also copy files from these sources into your SAS session, and copy files from SAS into Microsoft 365. 164 | 165 | The flow for file discovery is iterative. Each method creates an output data set that can be queried/filtered to a selection of interest, and that will result in an identifier for a folder or file that feeds into the next method. 166 | 167 | ### Example: List OneDrive contents 168 | 169 | This sequence lists your OneDrive "root" drives (you may have more than one), and then lists the contents of the "Documents" drive. 170 | ```sas 171 | 172 | %listMyDrives(out=work.drives); 173 | 174 | /* store the ID value for the drive in a macro variable, where "Documents" is at root */ 175 | proc sql noprint; 176 | select id into: driveId from work.drives where driveDisplayName="Documents"; 177 | quit; 178 | 179 | %listFolderItems(driveId=&driveId, folderId=root, out=work.folderItems); 180 | ``` 181 | 182 | Example output: 183 | 184 | ![OneDrive documents with file listings](./images/list-drives-output.png) 185 | 186 | ### Example: List SharePoint folders files 187 | 188 | Here's an example code flow: 189 | ```sas 190 | /* this macro fetches the root IDs for document libraries in your site */ 191 | %listSiteLibraries( 192 | siteHost=mysite.sharepoint.com, 193 | sitePath=/sites/Department, 194 | out=libraries); 195 | 196 | /* store the ID value for the library in a macro variable, where "Documents" is at root */ 197 | proc sql noprint; 198 | select id into: libraryId from libraries where name="Documents"; 199 | quit; 200 | 201 | /* LIST TOP LEVEL FOLDERS/FILES */ 202 | 203 | /* special macro to pull ALL items from root folder */ 204 | %listFolderItems(driveId=&libraryId., folderId=root, out=work.paths); 205 | ``` 206 | Example output: 207 | 208 | ![example data set with file listings](./images/example-listings.png) 209 | ```sas 210 | 211 | /* LIST ITEMS IN A SPECIFIC FOLDER */ 212 | 213 | /* 214 | At this point, if you want to act on any of the items, you just replace "root" 215 | with the ID of the item. So to list the items in the "General" folder I have: 216 | - find the ID for that folder 217 | - list the items within using %listFolderItems and passing that folder ID 218 | */ 219 | 220 | /* Find the ID of the folder I want */ 221 | proc sql noprint; 222 | select id into: folder from paths 223 | where name="General"; 224 | quit; 225 | 226 | /* Pull ALL items from a folder */ 227 | %listFolderItems(driveId=&libraryId., folderId=&folder., out=work.folderItems); 228 | ``` 229 | 230 | Example output (data set): 231 | 232 | ![example file listing](./images/example-listings2.png) 233 | 234 | ### Example: Download a file from SharePoint to your SAS session 235 | 236 | ```sas 237 | /* 238 | With a valid source folderId and knowledge of the items in this folder, 239 | we can download any file of interest. 240 | 241 | This example downloads a file named "ScoreCard2022.xlx" from a known 242 | folder on SharePoint (obtained in previous steps) and places it in a 243 | file location on the SAS session. 244 | */ 245 | %downloadFile(driveId=&driveId., 246 | folderId=&folder., 247 | sourceFilename=ScoreCard2022.xlsx, 248 | destinationPath=/tmp); 249 | 250 | /* Downloaded an Excel file into SAS? Now we can PROC IMPORT if we want */ 251 | proc import file="/tmp/ScoreCard2022.xlsx" 252 | out=xldata 253 | dbms=xlsx replace; 254 | run; 255 | ``` 256 | 257 | ### Example: Upload a file from SAS to SharePoint 258 | 259 | ```sas 260 | /* Create a sample file to upload */ 261 | %let targetFile=iris.xlsx; 262 | filename tosave "%sysfunc(getoption(WORK))/&targetFile."; 263 | ods excel(id=upload) file=tosave; 264 | proc print data=sashelp.iris; 265 | run; 266 | ods excel(id=upload) close; 267 | 268 | /* Upload to the "General" folder, the folder ID from previous step */ 269 | %uploadFile(driveId=&libraryId., 270 | folderId=&folder., 271 | sourcePath=%sysfunc(getoption(WORK)), 272 | sourceFilename=&targetFile.); 273 | ``` 274 | 275 | Notes: 276 | 277 | * The "list" methods (such as ```listFolderItems```) have special handling to use multiple API calls to gather a complete list of results. The Microsoft Graph API methods return a max of 200 items in a response with an indicator if there are more. These SAS macros will follow through and gather the complete list. 278 | 279 | * The ```uploadFile``` method uses the special "large file upload" handling to create an upload session that can accommodate files larger than the 4MB size that is the default size limit. 280 | 281 | ### Use any Microsoft Graph API endpoint 282 | 283 | With the authenticated session established, you can use PROC HTTP to execute any API endpoint that your app permissions allow. 284 | For example, with User.Read (most apps have this), you can download your own account profile photo: 285 | 286 | ```sas 287 | filename img "c:/temp/profile.jpg"; 288 | proc http url="&msgraphApiBase./me/photo/$value" 289 | method='GET' 290 | oauth_bearer="&access_token" 291 | out = img; 292 | run; 293 | ``` 294 | 295 | The `msgraphApiBase` and `access_token` macro variables are set during ```%initSessionMS365``` macro routine. 296 | 297 | This example shows how to retrieve the SharePoint Lists that are defined at the site root. The */sites/root/lists* endpoint requires Sites.Read.All permission. 298 | 299 | ```sas 300 | filename resp temp; 301 | proc http url="&msgraphApiBase./sites/root/lists" 302 | method='GET' 303 | oauth_bearer="&access_token" 304 | out = resp; 305 | run; 306 | 307 | libname lists JSON fileref=resp; 308 | proc sql; 309 | create table work.list_names as 310 | select t1.name, 311 | t1.displayname, 312 | t1.weburl, 313 | t2.template 314 | from lists.value t1 315 | inner join lists.value_list t2 on 316 | (t1.ordinal_value = t2.ordinal_list); 317 | quit; 318 | ``` 319 | 320 | All APIs are documented in the [Microsoft Graph API reference](https://learn.microsoft.com/en-us/graph/api/overview?view=graph-rest-1.0). -------------------------------------------------------------------------------- /examples/sas-microsoft365-example.sasnb: -------------------------------------------------------------------------------- 1 | [{"kind":1,"language":"markdown","value":"# SAS to Microsoft 365 example\r\n\r\nThe first step is to bring in the SAS macros to simplify access to Microsoft 365. In this example, we'll download directly from GitHub. You may decide to download to a local folder and include from there.","outputs":[]},{"kind":2,"language":"sas","value":"/* Allocate a folder for this SAS program */ \r\n/* and download from GitHub */\r\noptions dlcreatedir;\r\nlibname src \"%sysfunc(getoption(WORK))/src\";\r\nlibname src clear;\r\n%let src = %sysfunc(getoption(WORK))/src;\r\nfilename src \"&src./ms-graph-macros.sas\";\r\n\r\nproc http\r\n method='GET'\r\n url=\"https://raw.githubusercontent.com/sascommunities/sas-microsoft-graph-api/main/ms-graph-macros.sas\"\r\n out=src;\r\nrun;\r\n\r\n%include src;","outputs":[]},{"kind":1,"language":"markdown","value":"Next, we need to establish the path that will hold our config settings and eventually the access token to connect to Microsoft 365. The folder should be in a location that is private to you, as it will hold sensitive information that should be protected. Once generated, the access token/refresh token can be used as credentials for access to your Microsoft 365 content. \r\n\r\nThe config settings should be in a file that you create named `config.json`, with a format like this (of course, substituting your own tenant_id and client_id values that you get from your registered app in the Azure portal):\r\n```\r\n{\r\n \"tenant_id\": \"206db638-6adb-41b9-b20c-95d8d04abcbe\",\r\n \"client_id\": \"8fb7804a-8dfd-40d8-bf5b-d02c2cbc56f3\",\r\n \"redirect_uri\": \"https://login.microsoftonline.com/common/oauth2/nativeclient\",\r\n \"resource\" : \"https://graph.microsoft.com\"\r\n}\r\n```\r\n\r\nWe'll use the `%initConfig` macro to read this file and establish this folder as the location for your token information. It supports a traditional file path (SAS 9 or SAS Viya) or a SAS Content folder in SAS Viya.\r\n\r\nExample for local files:\r\n```\r\n%initConfig(configPath=C:\\Projects\\ms-graph-examples);\r\n```\r\n\r\nOr in a SAS Viya content folder:\r\n```\r\n%initConfig(configPath=/Users/&_clientuserid/My Folder/.creds,sascontent=1);\r\n```","outputs":[]},{"kind":2,"language":"sas","value":"%initConfig(configPath=C:\\Projects\\ms-graph-examples);","outputs":[]},{"kind":1,"language":"markdown","value":"## First connection: sign in and get auth code\r\n\r\nBefore we can make this a repeatable process, we need to get an access token that we can use and refresh as needed. The first step is to get an authorization code that affirms our app connection and permission for this Microsoft 365 user.\r\n\r\nTo accomplish this, we must visit a URL in our browser to sign in and consent to the permissions needed to read and write content in OneDrive and SharePoint. The `%generateAuthUrl` macro helps with that. Run the macro to build this URL (which will be based on your config data). Copy the generated URL from the SAS log and paste the URL into your web browser. Complete the signin and consent steps, then copy the auth code (the `code=` portion) from the URL for use in the next step.","outputs":[]},{"kind":2,"language":"sas","value":"%generateAuthUrl();","outputs":[]},{"kind":1,"language":"markdown","value":"With the auth code in hand, paste it into a macro assignment and use the `%generate_access_token` method to get that first token. This will generate the token.json file and store it in your config folder.","outputs":[]},{"kind":2,"language":"sas","value":"%let auth_code=;\r\n\r\n/*\r\n Now that we have an authorization code we can get the access token\r\n This step will write the token.json file that we can use in our\r\n production programs.\r\n*/\r\n\r\n%get_access_token(&auth_code.);","outputs":[]},{"kind":1,"language":"markdown","value":"With the token.json created in our config folder, we can initiate a connection to Microsoft 365 and get an active access token.","outputs":[]},{"kind":2,"language":"sas","value":"/* Initialize the session by refreshing the access token */\r\n%initSessionMS365;","outputs":[]},{"kind":1,"language":"markdown","value":"The active token value is stored in the `&access_token` macro variable. We can use this in REST API calls for the Microsoft Graph API. This sample call should return the JSON data about your Microsoft 365 user profile.","outputs":[]},{"kind":2,"language":"sas","value":"filename resp temp;\r\nproc http url=\"https://graph.microsoft.com/v1.0/me\" \r\n oauth_bearer=\"&access_token\" \r\n out=resp;\r\nrun;\r\ndata _null_;\r\n rc=jsonpp('resp','log');\r\nrun;\r\nfilename resp clear;","outputs":[]},{"kind":1,"language":"markdown","value":"## Example workflow with SharePoint Online/Teams folders\r\n\r\nLet's look at content in SharePoint Online. Remember that files that you store in Microsoft Teams channels are also SharePoint behind the scenes, so the process is the same.\r\n\r\nFor this we'll start with the `%listSiteLibraries` macro routine.","outputs":[]},{"kind":2,"language":"sas","value":"/* Let's look at SharePoint / Teams folders */\r\n%let siteHost = sasoffice365.sharepoint.com;\r\n%let sitePath = /sites/SASandMicrosoft365APIdemo;\r\n\r\n%listSiteLibraries( siteHost=&siteHost.,sitepath=&sitePath.,out=libs);\r\n/* store the ID value for the library in a macro variable, where \"Documents\" is at root */\r\nproc sql noprint;\r\n select id into: libraryId from libs where name=\"Documents\";\r\nquit;\r\n\r\n/* LIST TOP LEVEL FOLDERS/FILES */\r\n\r\n/* special macro to pull ALL items from root folder */\r\n%listFolderItems(driveId=&libraryId., folderId=root, out=work.paths);\r\nproc print data=paths (obs=10);\r\nrun;","outputs":[]},{"kind":1,"language":"markdown","value":"Drill into that top folder to see the subfolders.","outputs":[]},{"kind":2,"language":"sas","value":"proc sql noprint;\r\n select id into: folderId from work.paths where name=\"Team Content\";\r\nquit;\r\n\r\n%listFolderItems(driveId=&libraryId., folderId=&folderId., out=work.paths);\r\nproc print data=paths (obs=10);\r\nrun;","outputs":[]},{"kind":1,"language":"markdown","value":"Now drill into ***that*** folder to see its subfolders and content.","outputs":[]},{"kind":2,"language":"sas","value":"proc sql noprint;\r\n select id into: folderId from work.paths where name=\"Reports\";\r\nquit;\r\n\r\n%listFolderItems(driveId=&libraryId., folderId=&folderId., out=work.paths);\r\nproc print data=paths (obs=10);\r\nrun;","outputs":[]},{"kind":1,"language":"markdown","value":"Now that we found the folder that contains the file we want, we can use the ```%downloadFile``` macro to bring it into our SAS session.","outputs":[]},{"kind":2,"language":"sas","value":"%let localFolder = %sysfunc(getoption(WORK));\r\n\r\n%downloadFile(driveId=&libraryId., \r\n folderId=&folderId., \r\n sourceFilename=SciFi-AI.xlsx, \r\n destinationPath=&localFolder.);","outputs":[]},{"kind":1,"language":"markdown","value":"**Note:** We now have the IDs for the SharePoint library (drive) and the folder for this content, we can store them and use in future code, since they will not change. Then next time we want to do this download, we can skip the \"query and discover\" steps and go directly to the download step.","outputs":[]},{"kind":2,"language":"sas","value":"%put &=libraryId.;\r\n%put &=folderId.;\r\n\r\n%let reportLibrary=&libraryId;\r\n%let reportFolder=&folderId;","outputs":[]},{"kind":1,"language":"markdown","value":"With that file downloaded and in a local folder, we can now PROC IMPORT as an Excel file.","outputs":[]},{"kind":2,"language":"sas","value":"/* Downloaded an Excel file into SAS? Now we can PROC IMPORT if we want */\r\nproc import file=\"&localFolder./SciFi-AI.xlsx\" \r\n out=scifi\r\n dbms=xlsx replace;\r\nrun;\r\n\r\n/* Preview the contents of this imported file */\r\nproc print data=scifi (obs=10);\r\nrun;","outputs":[]},{"kind":1,"language":"markdown","value":"Now let's use the same iterative 'list folders' techniques to find a destination folder named \"Examples\", and then use the `%uploadFile` macro to send a SAS-generated report file to this Teams/SharePoint folder.\r\n\r\nFirst, let's generate a report in Excel format to share.","outputs":[]},{"kind":2,"language":"sas","value":"proc sql;\r\n create table withProfit as \r\n select t1.*,\r\n (t1.BoxOfficeReceipts - t1.CostToMake) * 1000000 as Profit_Loss format=dollar12.\r\n from scifi t1\r\n order by Profit_Loss desc;\r\nquit;\r\n\r\nfilename report \"&localFolder./SciFi-Summary.xlsx\";\r\n\r\nods excel(id=xl) file=report\r\n options(sheet_interval='none' sheet_name='MOVIES');\r\nods graphics on / imagefmt=png;\r\n\r\nTitle \"Top 10 Profitable Sci-Fi Movies\";\r\nproc print data=withProfit(obs=10);\r\n var YearProduced MovieTitle Profit_Loss;\r\nrun;\r\n\r\ntitle height=3 \"Profit amounts by Year\";\r\nproc sgpie data=withProfit;\r\n styleattrs datacolors=(cXfbb4ae cXb3cde3 cXccebc5 cXdecbe4);\r\n pie YearProduced / response=Profit_Loss \r\n datalabelattrs=(Size = 22pt) \r\n datalabeldisplay=(category percent);\r\nrun;\r\n\r\nods excel(id=xl) close;","outputs":[]},{"kind":1,"language":"markdown","value":"Next, we'll use PROC SQL and subsquent `%listFolderItems` calls to find the ID of the destination folder. Then use `%uploadFile` to send it to our Teams folder.","outputs":[]},{"kind":2,"language":"sas","value":"\r\n/*\r\n Uploading to:\r\n https://sasoffice365.sharepoint.com/:f:/r/sites/SASandMicrosoft365APIdemo/Shared%20Documents/Team%20Content/Reports?csf=1&web=1&e=ThVf00\r\n*/\r\n\r\n%uploadFile(driveId=&reportLibrary., folderId=&reportFolder.,\r\n sourcePath=&localFolder.,\r\nsourcefilename=SciFi-Summary.xlsx);","outputs":[]},{"kind":1,"language":"markdown","value":"## Example workflow with OneDrive\r\n\r\nMost of the actions that we can perform with SharePoint folders, we can also do with OneDrive. The main difference is that the API to list OneDrive root content is different, so we have a special routine for that. ","outputs":[]},{"kind":2,"language":"sas","value":"/* Exploration: get the list of top-level drives in OneDrive */\r\n%listMyDrives(out=work.drives);\r\nproc print data=drives;\r\nrun;","outputs":[]},{"kind":1,"language":"markdown","value":"From the available \"drives\", we can use PROC SQL to single out the ID for the main \"Documents\" folder and then list its contents.","outputs":[]},{"kind":2,"language":"sas","value":"/*\r\n If you have multiple drives you can filter \r\n the set with a where clause on the name value.\r\n Note that the system may track additional drives behind-the-scenes, so\r\n don't assume you have just one!\r\n\r\n In my case, the main drive is labeled \"Documents\".\r\n*/\r\n/* store the ID value for the drive in a macro variable */\r\nproc sql noprint;\r\n select id into: driveId from drives where driveDisplayName=\"Documents\";\r\nquit;\r\n\r\n/* LIST TOP LEVEL FOLDERS/FILES */\r\n\r\n/* special macro to pull ALL items from root folder */\r\n/* Note that the APIs return only 200 items at a time */\r\n/* This macro will iterate through multiple times to get the full */\r\n/* collection. */\r\n%listFolderItems(driveId=&driveId., folderId=root, out=work.paths);\r\n/* preview a few of these */\r\nproc print data=paths (obs=10);\r\nrun;","outputs":[]},{"kind":1,"language":"markdown","value":"Supposing that we have a folder in OneDrive named \"Projects\", we can now drill into the listing of that folder. Once again we use PROC SQL to single out its identifier. Then we can select a file from that folder to \"download\" into our SAS session. Then it's available to use as input into another step, such as PROC IMPORT for an Excel file. In this example we have an XLSX file named \"ScoreCard2022.xlsx\".","outputs":[]},{"kind":2,"language":"sas","value":"/* LIST ITEMS IN A SPECIFIC FOLDER */\r\n/*\r\n At this point, if you want to act on any of the items, you just replace \"root\" \r\n with the ID of the item. So to list the items in the \"Projects\" folder I have:\r\n - find the ID for that folder\r\n - list the items within and save to a data set of items\r\n*/\r\n\r\n/* Find the ID of the folder I want */\r\nproc sql noprint;\r\n select id into: folder from paths\r\n where name=\"Projects\";\r\nquit;\r\n\r\n/* special macro to list ALL items from a folder */\r\n%listFolderItems(driveId=&driveId., folderId=&folder., out=work.folderItems);\r\n\r\n\r\n/* DOWNLOAD A FILE FROM ONEDRIVE TO SAS SESSION */\r\n/*\r\n With a list of the items in this folder, we can download\r\n any file of interest\r\n*/\r\n%downloadFile(driveId=&driveId., \r\n folderId=&folder., \r\n sourceFilename=ScoreCard2022.xlsx, \r\n destinationPath=%sysfunc(getoption(WORK)));\r\n\r\n/* Downloaded an Excel file into SAS? Now we can PROC IMPORT if we want */\r\nproc import file=\"%sysfunc(getoption(WORK))/ScoreCard2022.xlsx\" \r\n out=xldata\r\n dbms=xlsx replace;\r\nrun;\r\n\r\n/* Preview the contents of this imported file */\r\nproc print data=xldata (obs=10);\r\nrun;","outputs":[]}] -------------------------------------------------------------------------------- /images/azure_access_code.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sascommunities/sas-microsoft-graph-api/42773c9061435e944605107014ab0ab8cd8f536a/images/azure_access_code.png -------------------------------------------------------------------------------- /images/creds-in-studio.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sascommunities/sas-microsoft-graph-api/42773c9061435e944605107014ab0ab8cd8f536a/images/creds-in-studio.png -------------------------------------------------------------------------------- /images/example-listings.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sascommunities/sas-microsoft-graph-api/42773c9061435e944605107014ab0ab8cd8f536a/images/example-listings.png -------------------------------------------------------------------------------- /images/example-listings2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sascommunities/sas-microsoft-graph-api/42773c9061435e944605107014ab0ab8cd8f536a/images/example-listings2.png -------------------------------------------------------------------------------- /images/list-drives-output.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sascommunities/sas-microsoft-graph-api/42773c9061435e944605107014ab0ab8cd8f536a/images/list-drives-output.png -------------------------------------------------------------------------------- /ms-graph-macros.sas: -------------------------------------------------------------------------------- 1 | /* -------------------------------------------------------------------------- 2 | Macros for managing the access tokens for the MS Graph API. 3 | Also helpful macros for discovering, downloading/reading, and uploading 4 | file content to OneDrive and SharePoint Online. 5 | 6 | Authors: Joseph Henry, SAS 7 | Chris Hemedinger, SAS 8 | Copyright 2022, SAS Institute Inc. 9 | 10 | See: 11 | https://blogs.sas.com/content/sasdummy/sas-programming-office-365-onedrive 12 | ----------------------------------------------------------------------------*/ 13 | 14 | /* Reliable way to check whether a macro value is empty/blank */ 15 | %macro isBlank(param); 16 | %sysevalf(%superq(param)=,boolean) 17 | %mend; 18 | 19 | /* Check to see if base URLs for services are */ 20 | /* initialized/overridden. */ 21 | /* If not, then define them to the common defaults */ 22 | %macro initBaseUrls(); 23 | %if %symexist(msloginBase) = 0 %then %do; 24 | %global msloginBase; 25 | %end; 26 | %if %isBlank(&msloginBase.) %then %do; 27 | %let msloginBase = https://login.microsoftonline.com; 28 | %end; 29 | %if %symexist(msgraphApiBase) = 0 %then %do; 30 | %global msgraphApiBase; 31 | %end; 32 | %if %isBlank(&msgraphApiBase.) %then %do; 33 | %let msgraphApiBase = https://graph.microsoft.com/v1.0; 34 | %end; 35 | %mend; 36 | %initBaseUrls(); 37 | 38 | /* We need this function for large file uploads, to telegraph */ 39 | /* the file size in the API. */ 40 | /* Get the file size of a local file in bytes. */ 41 | %macro getFileSize(localFile=); 42 | %local rc fid fidc; 43 | %local File_Size; 44 | %let File_Size = -1; 45 | %let rc=%sysfunc(filename(_lfile,&localFile)); 46 | %if &rc. = 0 %then %do; 47 | %let fid=%sysfunc(fopen(&_lfile)); 48 | %if &fid. > 0 %then %do; 49 | %let File_Size=%sysfunc(finfo(&fid,File Size (bytes))); 50 | %let fidc=%sysfunc(fclose(&fid)); 51 | %end; 52 | %let rc=%sysfunc(filename(_lfile)); 53 | %end; 54 | %sysevalf(&File_Size.) 55 | %mend; 56 | 57 | /* 58 | Set the variables that will be needed through the code 59 | We'll need these for authorization and also for runtime 60 | use of the service. 61 | 62 | Reading these from a config.json file so that the values 63 | are easy to adapt for different users or projects. The config.json 64 | can be in a file system or in SAS Content folders (SAS Viya only). 65 | 66 | Usage: 67 | %initConfig(configPath=/path-to-your-config-folder); 68 | 69 | If using SAS Content folders on SAS Viya, specify the content 70 | folder and SASCONTENT=1. 71 | 72 | %initConfig(configPath=/Users/&_clientuserid/My Folder/.creds,sascontent=1); 73 | 74 | configPath should contain the config.json for your app. 75 | This path will also contain token.json once it's generated 76 | by the authentication steps. 77 | */ 78 | %macro initConfig(configPath=,sascontent=0); 79 | %global config_root m365_usesascontent; 80 | %let m365_usesascontent = &sascontent.; 81 | %let config_root=&configPath.; 82 | %if &m365_usesascontent = 1 %then %do; 83 | filename config filesrvc 84 | folderpath="&configPath." 85 | filename="config.json"; 86 | %end; 87 | %else %do; 88 | filename config "&configPath./config.json"; 89 | %end; 90 | %put NOTE: Establishing Microsoft 365 config root to &config_root.; 91 | %if (%sysfunc(fexist(config))) %then %do; 92 | libname config json fileref=config; 93 | data _null_; 94 | set config.root; 95 | call symputx('tenant_id',tenant_id,'G'); 96 | call symputx('client_id',client_id,'G'); 97 | call symputx('redirect_uri',redirect_uri,'G'); 98 | call symputx('resource',resource,'G'); 99 | run; 100 | libname config clear; 101 | filename config clear; 102 | %end; 103 | %else %do; 104 | %put ERROR: You must create the config.json file in your configPath.; 105 | %put The file contents should be:; 106 | %put {; 107 | %put "tenant_id": "your-azure-tenant",; 108 | %put "client_id": "your-app-client-id",; 109 | %put "redirect_uri": "&msloginBase./common/oauth2/nativeclient",; 110 | %put "resource" : "https://graph.microsoft.com"; 111 | %put }; 112 | %end; 113 | %mend; 114 | 115 | /* 116 | Generate a URL that you will use to obtain an authentication code in your browser window. 117 | Usage: 118 | %initConfig(configPath=/path-to-config.json); 119 | %generateAuthUrl(); 120 | */ 121 | %macro generateAuthUrl(); 122 | %if %symexist(tenant_id) %then 123 | %do; 124 | /* Run this line to build the authorization URL */ 125 | %let authorize_url=&msloginBase./&tenant_id./oauth2/authorize?client_id=&client_id.%nrstr(&response_type)=code%nrstr(&redirect_uri)=&redirect_uri.%nrstr(&resource)=&resource.; 126 | %let _currLS = %sysfunc(getoption(linesize)); 127 | 128 | /* LS=MAX so URL will not have line breaks for easier copy/paste */ 129 | options nosource ls=max; 130 | %put Paste this URL into your web browser:; 131 | %put -- START -------; 132 | %put &authorize_url; 133 | %put ---END ---------; 134 | options source ls=&_currLS.; 135 | %end; 136 | %else 137 | %do; 138 | %put ERROR: You must use the initConfig macro first.; 139 | %end; 140 | %mend; 141 | 142 | /* 143 | Utility macro to process the JSON token 144 | file that was created at authorization time. 145 | This will fetch the access token, refresh token, 146 | and expiration datetime for the token so we know 147 | if we need to refresh it. 148 | */ 149 | %macro read_token_file(file); 150 | %put M365: Reading token info from %sysfunc(pathname(&file.)); 151 | libname oauth json fileref=&file.; 152 | 153 | data _null_; 154 | set oauth.root; 155 | call symputx('access_token', access_token,'G'); 156 | call symputx('refresh_token', refresh_token,'G'); 157 | /* convert epoch value to SAS datetime */ 158 | call symputx('expires_on',(input(expires_on,best32.)+'01jan1970:00:00'dt),'G'); 159 | run; 160 | %put M365: Token expires on %left(%qsysfunc(putn(%sysevalf(&expires_on.+%sysfunc(tzoneoff() )),datetime20.))); 161 | 162 | libname oauth clear; 163 | %mend; 164 | 165 | /* Assign the TOKEN fileref to location that */ 166 | /* depends on whether we're using SAS Content */ 167 | %macro assignTokenFileref(); 168 | %if &m365_usesascontent = 1 %then %do; 169 | filename token filesrvc 170 | folderpath="&config_root." 171 | filename="token.json"; 172 | %end; 173 | %else %do; 174 | filename token "&config_root./token.json"; 175 | %end; 176 | %mend; 177 | 178 | 179 | /* 180 | Utility macro that retrieves the initial access token 181 | by redeeming the authorization code that you're granted 182 | during the interactive step using a web browser 183 | while signed into your Microsoft OneDrive / Azure account. 184 | 185 | This step also creates the initial token.json that will be 186 | used on subsequent steps/sessions to redeem a refresh token. 187 | */ 188 | %macro get_access_token(auth_code, debug=0); 189 | 190 | %assignTokenFileref(); 191 | 192 | proc http url="&msloginBase./&tenant_id./oauth2/token" 193 | method="POST" 194 | in="%nrstr(&client_id)=&client_id.%nrstr(&code)=&auth_code.%nrstr(&redirect_uri)=&redirect_uri%nrstr(&grant_type)=authorization_code%nrstr(&resource)=&resource." 195 | out=token; 196 | %if &debug>=0 %then 197 | %do; 198 | debug level=&debug.; 199 | %end; 200 | %else %if &_DEBUG_. ge 1 %then 201 | %do; 202 | debug level=&_DEBUG_.; 203 | %end; 204 | run; 205 | 206 | %if (&SYS_PROCHTTP_STATUS_CODE. = 200) %then %do; 207 | %read_token_file(token); 208 | %end; 209 | %else %do; 210 | %put ERROR: &sysmacroname. failed: HTTP result - &SYS_PROCHTTP_STATUS_CODE. &SYS_PROCHTTP_STATUS_PHRASE.; 211 | %end; 212 | 213 | filename token clear; 214 | 215 | %mend; 216 | 217 | /* 218 | Utility macro to redeem the refresh token 219 | and get a new access token for use in subsequent 220 | calls to the MS Graph API service. 221 | */ 222 | %macro refresh_access_token(debug=0); 223 | 224 | %put M365: Refreshing access token for M365; 225 | %assignTokenFileref(); 226 | 227 | proc http url="&msloginbase./&tenant_id./oauth2/token" 228 | method="POST" 229 | in="%nrstr(&client_id)=&client_id.%nrstr(&refresh_token=)&refresh_token%nrstr(&redirect_uri)=&redirect_uri.%nrstr(&grant_type)=refresh_token%nrstr(&resource)=&resource." 230 | out=token; 231 | %if &debug. ge 0 %then 232 | %do; 233 | debug level=&debug.; 234 | %end; 235 | %else %if %symexist(_DEBUG_) AND &_DEBUG_. ge 1 %then 236 | %do; 237 | debug level=&_DEBUG_.; 238 | %end; 239 | run; 240 | 241 | %if (&SYS_PROCHTTP_STATUS_CODE. = 200) %then %do; 242 | %read_token_file(token); 243 | %end; 244 | %else %do; 245 | %put ERROR: &sysmacroname. failed: HTTP result - &SYS_PROCHTTP_STATUS_CODE. &SYS_PROCHTTP_STATUS_PHRASE.; 246 | %end; 247 | 248 | filename token clear; 249 | 250 | %mend; 251 | 252 | 253 | /* 254 | Use the token information to refresh and gain an access token for this session 255 | Usage: 256 | %initSessionMS365; 257 | 258 | Assumes you have already defined config.json and token.json with 259 | the authentication steps, and set the config path with %initConfig. 260 | */ 261 | 262 | %macro initSessionMS365; 263 | 264 | %if (%isBlank(&config_root.)) %then %do; 265 | %put You must use initConfig first to set the configPath; 266 | %return; 267 | %end; 268 | /* 269 | Our json file that contains the oauth token information 270 | */ 271 | %assignTokenFileref(); 272 | 273 | %if (%sysfunc(fexist(token)) eq 0) %then %do; 274 | %put ERROR: &config_root./token.json not found. Run the setup steps to create the API tokens.; 275 | %end; 276 | %else %do; 277 | /* 278 | If the access_token expires, we can just use the refresh token to get a new one. 279 | 280 | Some reasons the token (and refresh token) might not work: 281 | - Explicitly revoked by the app developer or admin 282 | - Password change in the user account for Microsoft Office 365 283 | - Time limit expiration 284 | 285 | Basically from this point on, user interaction is not needed. 286 | 287 | We assume that the token will only need to be refreshed once per session, 288 | and right at the beginning of the session. 289 | 290 | If a long running session is needed (>3600 seconds), 291 | then check API calls for a 401 return code 292 | and call %refresh_access_token if needed. 293 | */ 294 | 295 | %read_token_file(token); 296 | 297 | filename token clear; 298 | 299 | /* If this is first use for the session, we'll likely need to refresh */ 300 | /* the token. This will also call read_token_file again and update */ 301 | /* our token.json file. */ 302 | %refresh_access_token(); 303 | %end; 304 | %mend; 305 | 306 | /* For SharePoint Online, list the main document libraries in the root of a SharePoint site */ 307 | /* Using the /sites methods in the Microsoft Graph API */ 308 | /* May require the Sites.ReadWrite.All permission for your app */ 309 | /* See https://docs.microsoft.com/en-us/graph/api/resources/sharepoint?view=graph-rest-1.0 */ 310 | /* Set these values per your SharePoint Online site. 311 | Ex: https://yourcompany.sharepoint.com/sites/YourSite 312 | breaks down to: 313 | yourcompany.sharepoint.com -> hostname 314 | /sites/YourSite -> sitepath 315 | 316 | This example uses the /drive method to access the files on the 317 | Sharepoint site -- works just like OneDrive. 318 | API also supports a /lists method for SharePoint lists. 319 | Use the Graph Explorer app to find the correct APIs for your purpose. 320 | https://developer.microsoft.com/en-us/graph/graph-explorer 321 | 322 | Usage: 323 | %listSiteLibraries(siteHost=yoursite.company.com, 324 | sitePath=/sites/YourSite, 325 | out=work.OutputListData); 326 | */ 327 | %macro listSiteLibraries(siteHost=,sitePath=,out=work.siteLibraries); 328 | filename resp TEMP; 329 | proc http url="&msgraphApiBase./sites/&siteHost.:&sitepath.:/drive" 330 | oauth_bearer="&access_token" 331 | out = resp; 332 | run; 333 | %if (&SYS_PROCHTTP_STATUS_CODE. = 200) %then %do; 334 | libname jresp json fileref=resp; 335 | data &out.; 336 | set jresp.root(drop=ordinal:); 337 | run; 338 | libname jresp clear; 339 | %end; 340 | %else %do; 341 | %put ERROR: &sysmacroname. failed: HTTP result - &SYS_PROCHTTP_STATUS_CODE. &SYS_PROCHTTP_STATUS_PHRASE.; 342 | %end; 343 | 344 | filename resp clear; 345 | %mend; 346 | 347 | /* 348 | For OneDrive, fetch the list of Drives available to the current user. 349 | 350 | Output is a data set with the list of available Drives and IDs, for use in later 351 | routines. 352 | 353 | This creates a data set with the one record for each drive. 354 | Note that even if you think you have just one drive, the system 355 | might track others behind-the-scenes. 356 | 357 | Usage: 358 | %listMyDrives(out=work.DriveData); 359 | */ 360 | %macro listMyDrives(out=work.drives); 361 | filename resp TEMP; 362 | proc http url="&msgraphApiBase./me/drives/" 363 | oauth_bearer="&access_token" 364 | out = resp; 365 | run; 366 | 367 | %if (&SYS_PROCHTTP_STATUS_CODE. = 200) %then %do; 368 | libname jresp json fileref=resp; 369 | 370 | proc sql; 371 | create table &out. as 372 | select t1.id, 373 | t1.name, 374 | scan(t1.webUrl,-1,'/') as driveDisplayName, 375 | t1.createdDateTime, 376 | t1.description, 377 | t1.driveType, 378 | t1.lastModifiedDateTime, 379 | t2.displayName as lastModifiedName, 380 | t2.email as lastModifiedEmail, 381 | t2.id as lastModifiedId, 382 | t1.webUrl 383 | from jresp.value t1 inner join jresp.lastmodifiedby_user t2 on 384 | (t1.ordinal_value=t2.ordinal_lastModifiedBy); 385 | quit; 386 | libname jresp clear; 387 | %end; 388 | %else %do; 389 | %put ERROR: &sysmacroname. failed: HTTP result - &SYS_PROCHTTP_STATUS_CODE. &SYS_PROCHTTP_STATUS_PHRASE.; 390 | %end; 391 | filename resp clear; 392 | %mend; 393 | 394 | /* 395 | List items in a folder in OneDrive or SharePoint 396 | The Microsoft Graph API returns maximum 200 items, so if the collection 397 | contains more we need to iterate through a list. 398 | 399 | The API response contains a URL endpoint to fetch the next 400 | batch of items, if there is one. 401 | 402 | Use folderId=root to list the root items of the "Drive" (OneDrive or SharePoint library), 403 | else use the folder ID of the folder you discovered in a previous call. 404 | */ 405 | %macro listFolderItems(driveId=, folderId=root, out=work.folderItems); 406 | 407 | %local driveId nextLink batchnum; 408 | 409 | /* endpoint for initial list of items */ 410 | %let nextLink = &msgraphApiBase./me/drives/&driveId./items/&folderId./children; 411 | %let batchnum = 1; 412 | data _folderItems0; 413 | length name $ 500; 414 | stop; 415 | run; 416 | 417 | %do %until (%isBlank(%str(&nextLink))); 418 | filename resp TEMP; 419 | proc http url="&nextLink." 420 | oauth_bearer="&access_token" 421 | out = resp; 422 | run; 423 | 424 | libname jresp json fileref=resp; 425 | 426 | /* holding area for attributes that might not exist */ 427 | data _value; 428 | length name $ 500 429 | size 8 430 | webUrl $ 500 431 | lastModifiedDateTime $ 20 432 | createdDateTime $ 20 433 | id $ 50 434 | eTag $ 50 435 | cTag $ 50 436 | _microsoft_graph_downloadUrl $ 2000 437 | fileMimeType $ 75 438 | isFolder 8 439 | folderItemsCount 8; 440 | %if %sysfunc(exist(JRESP.VALUE)) %then 441 | %do; 442 | set JRESP.VALUE; 443 | %end; 444 | run; 445 | 446 | data _value_file; 447 | length ordinal_value 8 mimeType $ 75 ; 448 | %if %sysfunc(exist(JRESP.VALUE_FILE)) %then %do; 449 | set JRESP.VALUE_FILE; 450 | %end; 451 | run; 452 | 453 | data _value_folder; 454 | length ordinal_value 8 ordinal_folder 8 childCount 8; 455 | %if %sysfunc(exist(JRESP.VALUE_FOLDER)) %then %do; 456 | set JRESP.VALUE_FOLDER; 457 | %end; 458 | run; 459 | 460 | proc sql; 461 | create table _folderItems&batchnum. as 462 | select t1.name, t1.size, t1.webUrl length=500, 463 | t1.lastModifiedDateTime, 464 | t1.createdDateTime, 465 | t1.id, 466 | t1.eTag, 467 | t1.cTag, 468 | t1._microsoft_graph_downloadUrl, 469 | t3.mimeType as fileMimeType, 470 | case 471 | when t2.ordinal_folder is missing then 0 472 | else 1 473 | end 474 | as isFolder, 475 | t2.childCount as folderItemsCount 476 | from _value t1 left join _value_folder t2 477 | on (t1.ordinal_value=t2.ordinal_folder) 478 | left join _value_file t3 on (t1.ordinal_value=t3.ordinal_value) 479 | ; 480 | quit; 481 | 482 | /* clear placeholder attributes */ 483 | proc delete data=work._value_folder work._value_file work._value ; run; 484 | 485 | %put NOTE: Batch &batchnum: Gathered &sysnobs. items; 486 | /* check for a next link for more entries */ 487 | %let nextLink=; 488 | data _null_; 489 | set jresp.alldata(where=(p1='@odata.nextLink')); 490 | call symputx('nextLink',value); 491 | run; 492 | %let batchnum = %sysevalf(&batchnum. + 1); 493 | 494 | libname jresp clear; 495 | filename resp clear; 496 | %end; 497 | 498 | data &out; 499 | set _folderItems:; 500 | run; 501 | 502 | proc datasets nodetails nolist; 503 | delete _folderItems:; 504 | run; 505 | 506 | %mend; 507 | 508 | /* Download a OneDrive or SharePoint file */ 509 | /* Each file has a specific download URL that works with the API */ 510 | /* This macro routine finds that URL and use PROC HTTP to GET */ 511 | /* the content and place it in the local destination path */ 512 | %macro downloadFile(driveId=,folderId=,sourceFilename=,destinationPath=); 513 | %local driveId folderId dlUrl _opt; 514 | %let _opt = %sysfunc(getoption(quotelenmax)); 515 | options noquotelenmax; 516 | 517 | %listFolderItems(driveId=&driveId., folderId=&folderId., out=__tmpLst); 518 | 519 | /* Use DATA step functions here to escape & to avoid warnings for unresolved symbols */ 520 | data _null_; 521 | set __tmpLst; 522 | length resURL $ 2000; 523 | where name="&sourceFilename"; 524 | resURL = tranwrd(_microsoft_graph_downloadUrl,'&','%str(&)'); 525 | call symputx('dlURL',resURL); 526 | run; 527 | 528 | proc delete data=work.__tmpLst; run; 529 | 530 | %if %isBlank(&dlUrl) %then %do; 531 | %put ERROR: No file named &sourceFilename. found in folder.; 532 | %end; 533 | %else %do; 534 | filename dlout "&destinationPath./&sourceFilename."; 535 | 536 | proc http url="&dlUrl." 537 | oauth_bearer="&access_token" 538 | out = dlOut; 539 | run; 540 | 541 | %put NOTE: Download file HTTP result - &SYS_PROCHTTP_STATUS_CODE. &SYS_PROCHTTP_STATUS_PHRASE.; 542 | 543 | %if (&SYS_PROCHTTP_STATUS_CODE. = 200) %then %do; 544 | %put NOTE: File downloaded to &destinationPath./&sourceFilename., %getFilesize(localFile=&destinationPath./&sourceFilename) bytes; 545 | %end; 546 | %else %do; 547 | %put WARNING: Download file NOT successful.; 548 | %end; 549 | 550 | filename dlout clear; 551 | %end; 552 | options &_opt; 553 | %mend; 554 | 555 | /* 556 | Split a file into same-size chunks, often needed for HTTP uploads 557 | of large files via an API 558 | 559 | Sample use: 560 | %splitFile(sourceFile=c:\temp\register-hypno.gif, 561 | maxSize=327680, 562 | metadataOut=work.chunkMeta, 563 | chunkLoc=c:\temp\chunks); 564 | */ 565 | 566 | %macro splitFile(sourceFile=, 567 | maxSize=327680, 568 | metadataOut=, 569 | /* optional, will default to WORK */ 570 | chunkLoc=); 571 | 572 | %local filesize maxSize numChunks buffsize ; 573 | %let buffsize = %sysfunc(min(&maxSize,4096)); 574 | %let filesize = %getFileSize(localFile=&sourceFile.); 575 | %let numChunks = %sysfunc(ceil(%sysevalf( &filesize / &maxSize. ))); 576 | %put NOTE: Splitting &sourceFile. (size of &filesize. bytes) into &numChunks parts; 577 | 578 | %if %isBlank(&chunkLoc.) %then %do; 579 | %let chunkLoc = %sysfunc(getoption(WORK)); 580 | %end; 581 | 582 | /* This DATA step will do the chunking. */ 583 | /* It's going to read the original file in segments sized to the buffer */ 584 | /* It's going to write that content to new files up to the max size */ 585 | /* of a "chunk", then it will move on to a new file in the sequence */ 586 | /* All resulting files should be the size we specified for chunks */ 587 | /* except for the last one, which will be a remnant */ 588 | /* Along the way it will build a data set with the metadata for these */ 589 | /* chunked files, including the file location and byte range info */ 590 | /* that will be useful for APIs that need that later on */ 591 | data &metadataOut.(keep=original originalsize chunkpath chunksize byterange); 592 | length 593 | filein 8 fileid 8 chunkno 8 currsize 8 buffIn 8 rec $ &buffsize fmtLength 8 outfmt $ 12 594 | bytescumulative 8 595 | /* These are the fields we'll store in output data set */ 596 | original $ 250 originalsize 8 chunkpath $ 500 chunksize 8 byterange $ 50; 597 | original = "&sourceFile"; 598 | originalsize = &filesize.; 599 | rc = filename('in',"&sourceFile."); 600 | filein = fopen('in','S',&buffsize.,'B'); 601 | bytescumulative = 0; 602 | do chunkno = 1 to &numChunks.; 603 | currsize = 0; 604 | chunkpath = catt("&chunkLoc./chunk_",put(chunkno,z4.),".dat"); 605 | rc = filename('out',chunkpath); 606 | fileid = fopen('out','O',&buffsize.,'B'); 607 | do while ( fread(filein)=0 ) ; 608 | call missing(outfmt, rec); 609 | rc = fget(filein,rec, &buffsize.); 610 | buffIn = fcol(filein); 611 | if (buffIn - &buffsize) = 1 then do; 612 | currsize + &buffsize; 613 | fmtLength = &buffsize.; 614 | end; 615 | else do; 616 | currsize + (buffIn-1); 617 | fmtLength = (buffIn-1); 618 | end; 619 | /* write only the bytes we read, no padding */ 620 | outfmt = cats("$char", fmtLength, "."); 621 | rcPut = fput(fileid, putc(rec, outfmt)); 622 | rcWrite = fwrite(fileid); 623 | if (currsize >= &maxSize.) then leave; 624 | end; 625 | chunksize = currsize; 626 | bytescumulative + chunksize; 627 | byterange = cat("bytes ",bytescumulative-chunksize,"-",bytescumulative-1,"/",originalsize); 628 | output; 629 | rc = fclose(fileid); 630 | end; 631 | rc = fclose(filein); 632 | run; 633 | %mend; 634 | 635 | /* Upload a single file segment as part of an upload session */ 636 | %macro uploadFileChunk( 637 | uploadURL=, 638 | chunkFile=, 639 | byteRange= 640 | ); 641 | 642 | filename hdrout temp; 643 | filename resp temp; 644 | 645 | filename _tosave "&chunkFile."; 646 | proc http url= "&uploadURL" 647 | method="PUT" 648 | in=_tosave 649 | out=resp 650 | oauth_bearer="&access_token" 651 | headerout=hdrout 652 | ; 653 | headers 654 | "Content-Range"="&byteRange." 655 | ; 656 | run; 657 | 658 | %put NOTE: Upload segment &byteRange., HTTP result - &SYS_PROCHTTP_STATUS_CODE. &SYS_PROCHTTP_STATUS_PHRASE.; 659 | 660 | /* HTTP 200 if success, 201 if new file was created */ 661 | %if (%sysfunc(substr(&SYS_PROCHTTP_STATUS_CODE.,1,1)) ne 2) %then 662 | %do; 663 | %put WARNING: File upload failed!; 664 | %if (%sysfunc(fexist(resp))) %then 665 | %do; 666 | data _null_; 667 | rc=jsonpp('resp','log'); 668 | run; 669 | %end; 670 | 671 | %if (%sysfunc(fexist(hdrout))) %then 672 | %do; 673 | data _null_; 674 | infile hdrout; 675 | input; 676 | put _infile_; 677 | run; 678 | %end; 679 | %end; 680 | 681 | filename _tosave clear; 682 | filename hdrout clear; 683 | filename resp clear; 684 | 685 | %mend; 686 | 687 | /* 688 | Use an UploadSession in the Microsoft Graph API to upload a file. 689 | 690 | This can handle large files, greater than the 4MB limit used by 691 | PUT to the :/content endpoint. 692 | The Graph API doc says you need to split the file into chunks. 693 | 694 | We do need to know the total file size in bytes before using the API, so 695 | this code includes a file-size check. 696 | 697 | It also uses a splitFile macro to create a collection of file segments 698 | for upload. These must be in multiples of 320K size according to the doc 699 | (except for the last segment, which is a remainder size). 700 | 701 | Credit to Muzzammil Nakhuda at SAS for figuring this out. 702 | 703 | Usage: 704 | %uploadFile(driveId=&driveId.,folderId=&folder., 705 | sourcePath=, 706 | sourceFilename=); 707 | */ 708 | %macro uploadFile(driveId=,folderId=,sourcePath=,sourceFilename=) ; 709 | %local driveId folderId fileSize _opt uploadURL; 710 | %let _opt = %sysfunc(getoption(quotelenmax)); 711 | options noquotelenmax; 712 | filename resp_us temp; 713 | 714 | /* Create an upload session to upload the file. */ 715 | /* If a file of the same name exists, we will REPLACE it. */ 716 | /* The API doc says this should be POST, but since we provide a body with conflict directives, */ 717 | /* it seems we must use PUT. */ 718 | proc http url="&msgraphApiBase./me/drives/&driveId./items/&folderId.:/%sysfunc(urlencode(&sourceFilename.)):/createUploadSession" 719 | method="PUT" 720 | in='{ "item": {"@microsoft.graph.conflictBehavior": "replace" }, "deferCommit": false }' 721 | out=resp_us 722 | ct="application/json" 723 | oauth_bearer="&access_token"; 724 | run; 725 | %put NOTE: Create Upload Session: HTTP result - &SYS_PROCHTTP_STATUS_CODE. &SYS_PROCHTTP_STATUS_PHRASE.; 726 | 727 | %if (&SYS_PROCHTTP_STATUS_CODE. = 200) %then %do; 728 | libname resp_us JSON fileref=resp_us; 729 | data _null_; 730 | set resp_us.root; 731 | call symputx('uploadURL',uploadUrl); 732 | run; 733 | 734 | %let fileSize=%getFileSize(localfile=&sourcePath./&sourceFilename.); 735 | 736 | %put NOTE: Uploading &sourcePath./&sourceFilename., file size of &fileSize bytes.; 737 | 738 | /* split the file into segments for upload */ 739 | %splitFile( 740 | sourceFile=&sourcePath./&sourceFilename., 741 | maxSize = 1310720, /* 327680 * 4, must be multiples of 320K per doc */ 742 | metadataOut=work._fileSegments 743 | ); 744 | 745 | /* upload each segment file in this upload session */ 746 | data _null_; 747 | set work._fileSegments; 748 | call execute(catt('%nrstr(%uploadFileChunk(uploadURL = %superq(uploadURL),chunkFile=',chunkPath,',byteRange=',byteRange,'));')); 749 | run; 750 | proc delete data=work._fileSegments; 751 | %end; 752 | /* Failed to create Upload Session */ 753 | %else %do; 754 | %put WARNING: Upload session not created!; 755 | %if (%sysfunc(fexist(resp_us))) %then %do; 756 | data _null_; rc=jsonpp('resp_us','log'); run; 757 | %end; 758 | %end; 759 | filename resp_us clear; 760 | options &_opt; 761 | %mend; 762 | --------------------------------------------------------------------------------