├── CHANGELOG.md ├── LICENSE.txt ├── README.md ├── assets └── images │ ├── SystemLogsGUI_ArcGISServer_web.png │ ├── SystemLogsGUI_ArcGISServer_web_small.png │ └── by-nc-sa.png └── binaries ├── latest └── SystemLogParser.zip ├── v0.13.0.0 └── SystemLogParser.zip ├── v0.13.0.1 └── SystemLogParser.zip └── v0.13.1.0 └── SystemLogParser.zip /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # System Log Parser 2 | 3 | ## CHANGELOG 4 | 5 | Build 0.13.2.0 6 | 1. Added Resource statistics by user to the Optimized report; this shows up as the "Statistics By User" worksheet 7 | 8 | Build 0.13.1.0 9 | 1. Added fix to safely remove the local binstore directory (temp folder) 10 | 2. Removed the VerboseMode mode report option from the GUI as some of its functionality was not being maintained 11 | 3. Fixed an issue that prevented some error messages from being listed in the ErrorsOnly report 12 | 4. After consideration, the "Server Timeout" and "Wait Time Expired" worksheets have been removed from the WithOverviewCharts report; this move helps simplify some of the internal parsing logic 13 | 5. Added support for ArcGIS Server Info log entries (ArcGIS Server File System only); Log Level Info log entries breakdown the request performance at the framework-level and includes times for request that are not just ArcSOC-based (e.g., hosted feature services); these performance times are close to a "response time" (minus an Web-tier overhead and transport time overhead); captured times at this level may appear higher which is because the instance creation time, wait time and elapsed time durations are all included in the value; ArcGIS Server 11.3 or higher recommended for using System Log Parser Info Analysis Type; The Info Analysis Type report is based on the Optimized report logic 14 | 6. Adjusted the Optimized report now include the folder name which preceeds the service name under Resource/Source listings 15 | 7. When generating a ServiceDetails report, JavaHeapSize is now listed; JavaHeapSize will display 64 (MB) unless a specific value is used to override the default setting; for more information see: https://enterprise.arcgis.com/en/server/10.8/publish-services/windows/geoprocessing-service-settings-advanced.htm 16 | 17 | Build 0.13.0.1 18 | 1. Removed support from System Log Parser for querying errors out of Portal for ArcGIS; this helps simplify the querying process as an error is returned if the Server Url is pointing to a Web Adaptor "portal" instance instead of a Web Adaptor "server" instance 19 | 2. Removed the "System Log Parser Installation Guide.docx" from the zip file; this document had been removed at the start of the 0.12.25.x release 20 | 3. ChangeLog correction, System Log Parser has been moved to: https://github.com/Esri/SystemLogParser 21 | 22 | Build 0.13.0.0 23 | 1. System Log Parser is now available under the Creative Commons CC BY-NC-SA license 24 | - https://creativecommons.org/licenses/by-nc-sa/4.0/ 25 | - https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode.en 26 | - See License.txt 27 | 2. System Log Parser is no longer Prerelease 28 | 3. System Log Parser has been moved to: [https://github.com/ArcGIS/SystemLogParser](https://github.com/Esri/SystemLogParser) 29 | 30 | Build 0.12.25.3 (Prerelease) 31 | 1. Corrections on the internal SymbolServer support 32 | 2. Some additional debug log verbosity 33 | 34 | Build 0.12.25.2 (Prerelease) 35 | 1. More checks for the service provider and configured state properties 36 | 2. The internal System Log Parser default configured state for a service is now "STOPPED" 37 | 3. For a ServiceDetails Analysis Type report (txt, json, excel), the Platform property has been removed from the Site Details worksheet to remove any potential ambiguity from the OS property 38 | 4. For a ServiceDetails Analysis Type report (txt, json, excel), the OS machine property is now listed immediately after Machine Name 39 | 5. In cases where the application debug log is enabled, the .NET Framework version is now listed 40 | 6. The Service Provider Stats table for ServiceDetails Analysis Type reports are now derived using an iterative summary approach 41 | 42 | Build 0.12.25.1 (Prerelease) 43 | 1. Added some checks to better ensure the proper presentation of the Service Provider Stats table data when a ServiceDetails Analysis Type report is generated 44 | 45 | Build 0.12.25.0 (Prerelease) 46 | 1. Fixed an issue where requests for Linear Referencing services would have their Capability set as MapServer instead of LRServer 47 | 2. Added support to pickup the WPSServer service capability so the report can group accordingly for improved analysis 48 | 3. Fixed an issue where Wait Timeouts could get counted as an elapsed time for ArcGIS reports with an Analysis Type of: Simple, WithOverviewCharts and Complete 49 | 4. Fixed an issue when running slp.exe that was preventing the command-line option of "-validate" to be true by default; setting "-validate true" is necessary to see console messages printed to the screen 50 | 5. Added some more throughput initialization to the ArcGIS Site object to help prevent the Provider summary in the ServiceDetails report from being empty 51 | 6. Removed the Cloud Front screen from the System Log Parser GUI; cloudfront can still be selected from the command-line (e.g., slp.exe) 52 | 7. Added an Application Log Level option to the ArcGIS Server (Web) screen; this provides the same capability as using "-apploglevel" from the command-line; this only creates a log for ArcGIS Server (web) log queries 53 | 8. Removed the sanity check for End and Start times when selecting an Analysis Type of ServiceDetails; The ServiceDetails report does not make use of End or Start times 54 | 9. Removed the "System Log Parser Installation Guide.docx" from the zip file; this doc was outdated and from an older release of System Log Parser 55 | 10. In the GUI, the label of "Elapsed Time Minimum" has been changed to "Elapsed Time Minimum (Complete Analysis Only)" for clarity 56 | 11. Enhanced the optional regular expression pattern to search against the source and operation fields; previous releases only search the source field (e.g., the resource or service name) 57 | 12. Fixed an issue where using a regular expression for a ArcGIS Server (FS) log query could cause System Log Parser to crash 58 | 13. For WithOverviewCharts, Complete and Optimized reports, a Wait Pct column has been added to the "Wait Time (Queue Time)" worksheet 59 | 14. Fixed an issue with Optimized reports, where the Service Requests count could be listed as 0 60 | 15. For ArcGIS Server Web log queries using an Optimized report, the Server Version and Server LogLevel fields were added to the Summary page 61 | 16. For Optimized, Simple, WithOverviewCharts and Complete (spreadsheet) reports, the Max column has been moved immediately after the Min column for "Statistics By Resource", "Statistics By User", "Wait Time (Queue Time)" and "Instance Creation Time" worksheets 62 | 17. Fixed an issue that would leave the "Arrival Rate Per Service" worksheet blank for Optimized reports using an ArcGIS Server log queries 63 | 18. Removed the "Server LogLevel (FileAge)" properties listing for spreadsheet reports generated through ArcGIS Server (FS) log queries 64 | 19. Enhanced some of the internal reporting of the ArcGIS Site object's contents 65 | 66 | Build 0.12.24.0 (Prerelease) 67 | 1. For (Analysis Type) Complete reports, the Path column has been added to the table on the "Elapsed Time - All Resources" worksheet 68 | 2. Fixed an issue with the (new) "Arrival Rate Per Service" worksheet to group specifically by service name; the previous build used the service name with capability so instead of counting requests for "PostgreSQL/Branch_Spatial1", the table would separate it by "PostgreSQL/Branch_Spatial1/FeatureServer", "PostgreSQL/Branch_Spatial1/MapServer" and "PostgreSQL/Branch_Spatial1/VersionManagementServer"; while this was correct, that extra separation was not originally intended 69 | 3. In addition to "Total Arrival Request Entries" counts on the arrival worksheets, "Data Rows Captured" are now listed as well 70 | 4. For Simple, WithOverviewCharts, Complete and Optimized reports, the data Start Time and End Time (in Coordinated Universal Time) are now listed 71 | 5. Improved request breakdown analysis has been added to IIS log parsing for capabilities such as NetworkDiagramServer 72 | 73 | Build 0.12.23.0 (Prerelease) 74 | 1. For Spreadsheet reports that pull log data from a local or network path, a "Log Location" field containing the passed in value has been added 75 | 2. The command-line option of validate is now true by default; when running slp.exe, for previous versions, this had to be set to true in order to see any messages or warnings that were printed to the console 76 | 3. Added a wait time minimum option to slp.exe for ignoring entries less than the passed in value; the default value is 1 (ms); this can be changed with the "-wtm" flag and the passed in value is assumed to be in milliseconds; for example: "-wtm 2" would be 0.002 seconds and "-wtm 2000" would be 2.0 seconds 77 | 4. Fixed issue for ArcGIS Server File System log queries where intermediate elapsed time entries were being included in the statistics by resource summary 78 | 5. For ArcGIS Server log source queries using an Analysis Type of Optmized, the worksheet "Arrival Rate" has been changed to "Arrival Rate Per User" 79 | 6. For ArcGIS Server log source queries using an Analysis Type of Optmized, a worksheet called "Arrival Rate Per Service" has been added which shows the amount of requests for services (per second)...this table does not additionally break down the data by user and instead just focuses on what was being requested at the service-level 80 | 81 | Build 0.12.22.0 (Prerelease) 82 | 1. For reports of type ServiceDetails, the internal counting logic had been adjusted to better differentiate between the service instance types (e.g. dedicated, hosted, shared). 83 | This was done to help validate the same counts on the "Site Details" worksheet and the "Service Provider Stats" table and the bottom of the "Summary" worksheet. 84 | A few things to keep in mind, to help keep things simple, even though it is a dedicated service, System Log Parser labels "System/DynamicMappingHost.MapServer" as "Shared" since it is responsible for handling requests for shared services. 85 | Instances minimiums and maximums for Shared and Hosted services are set to 0 in the report...for some ArcGIS Server releases, the REST Admin API may list values greater than 0 (in such cases, these values are an artifact that do *not* have a functional impact to the service). 86 | Going forward, System services will be listed as System_Dedicated and Utility services will be listed as Utilities_Dedicated. 87 | 2. Several columns in the "Site Details" worksheet have been group closer together for easier analysis (e.g. "Provider", "Instance Type", "Min Instances (Node)", "Max Instances (Node)", "Configured State") 88 | 3. In the GUI, the size of the LogsLocation field (for the AGSFS) has been increased from 300 characters to 3000 89 | 4. For listing the help in the command-line from slp.exe, the following commands can now be used: slp.exe, slp.exe -h, slp.exe --h and slp.exe -help 90 | 5. File signing now includes System Log Parser EXEs *and* DLLs 91 | 6. Corrected typo where the Header comments for the "Server Timeout" worksheet table lists the column as wait times 92 | 7. Adjusted the Header comments for the Service Requests column (in the "Wait Time (Queue Time)", "Instance Creation Time", "Server Timeout" and "Wait Time Expired" worksheets) to denote the values are for successful requests; in some situations, a higher number of wait times for a services can be listed than service requests; in such a case, the higher count of wait time suggests than many requests failed and were never logged as "successful" 93 | 8. Added a Report Name text field in the GUI for ArcGIS Server log queries (File System); when scanning large sites (multiple backend machines) over the network (e.g., several log locations), the report name that is automatically generated can become quite large and may have issues in Windows; this allows a convenient way to shorten it from the GUI 94 | 9. Fixed issue when performing ArcGIS Server log queries (File System) where the passed in regular expression string would remove all services from the "Wait Time (Queue Time)" worksheet instead of only showing values for the specific service of interest 95 | 96 | Build 0.12.21.1 (Prerelease) 97 | 1. Fixed issue where not all EXEs were properly signed 98 | 99 | Build 0.12.21.0 (Prerelease) 100 | 1. For the "Arrival Rate" worksheet (ArcGIS Server log sources), the total number of request arrivals is now listed at the top; this is an important statistic since the tabular view of the data will only show 200,000 rows (for performance and practical reasons) so understanding the original total is valuable; this total is available for the WithOverviewCharts, Complete and Optimized reports 101 | 2. For the ArcGIS Server ErrorsOnly reports (SpreadSheet or Text file output), the word "Errors" is not inserted into the filename of the generated report 102 | 3. Fixed an issue with the resource regular expression where you could not capture entries for a particular service name when using the Optimized report with an ArcGIS Server Web log source 103 | 4. Unified the tooltip text for the Regular Expression Pattern input box in the GUI 104 | 5. Changed the worksheet "Arrival Rate" to "Arrival Rate Per User" 105 | 6. Added a worksheet called "Arrival Rate Per Service" which shows the amount of requests for services (per second) but does not additionally break it down by user 106 | 7. Fixed issue in the GUI where toggling the "Add Service Details to Report (Enhanced Site Report)" option in the "ArcGIS Server Log Query (Web)" panel would force the Analysis Type to reset to Simple 107 | 8. Consolidated messages listed on ErrorsOnly reports by removing the user field from the internal error message dictionary (the user column was not being shown) 108 | 9. Updated the Copyright to 2023 for System Log Parser DLLs and EXEs 109 | 110 | Build 0.12.20.0 (Prerelease) 111 | 1. Fixed an issue where a malformed requestID GUID in an ArcGIS Server log entry could cause System Log Parser to stop reading the logs over the web 112 | 2. For the "Elapsed Time - All Resources" worksheet in the Complete report, the auto-fit column width for all columns has been removed to make filtering easier (specifically for the "Query String (Decoded)" and "Request Url" columns); several columns like "Date Time (Local Time)", "Epoch Time" and "Resource" still have the auto-fit sizing 113 | 3. For IIS, Azure and Tomcat, the default "ArcGIS Type" value has been changed from "-" to "" 114 | 4. For the "Elapsed Time - All Resources" worksheet in the Complete report, the "?-" string has been removed from the end of the "Request Url" column 115 | 5. For the "Elapsed Time - All Resources" worksheet in the Complete report, if a token key/value pair was detected in the query string, it is now seperated out into its own column, this makes it easier to read the other data in the query string 116 | 6. For (some) performance and load testing support, the Complete report has been exteneded with the following additional columns: Test_TestPlanName, Test_ThreadNumber, Test_PortalMember, Test_TransactionName; many testing frameworks have the ability to add these informative (but benign) components to requests in a test; if these key/value pairs are detected in the querystring (IIS, ALB/ELB and Azure log sources), they are extracted out into their own column to assist with filtering and analysis 117 | 7. For the "Elapsed Time - All Resources" worksheet in the Complete report, the "ALB TargetGroup" and "CloudFront Edge Result Type" columns have been removed due to low usage 118 | 119 | Build 0.12.19.0 (Prerelease) 120 | 1. Improved request breakdown analysis has been added to IIS log parsing for capabilities such as VersionManagementServer, UtilityNetworkServer and LRServer 121 | 2. Added a command-line option to slp.exe called "fsp" to force server parsing of logs even if the passed in URL has indications of a Portal instance; the default is false; active with: "-fsp true" 122 | 3. For the "Site Details" worksheet, the Service Name has been changed from the REST Admin link to just the name of the service to prevent Excel from blocking all the contents when it is in read-only mode 123 | 4. When using a pre-generated token with an ArcGIS Server (Web) log query from the GUI, the tool tip added clarification to create one that is Webapp URL based: "Manually provide an ArcGIS Server Token or ArcGIS Portal Token (Webapp URL based)"; for example, from https://myserver.domain.com/portal/sharing/rest/generateToken, select "Webapp URL" (where the URL would be something like: https://myserver.domain.com/portal); the command-line help has also been clarified 124 | 125 | Build 0.12.18.0 (Prerelease) 126 | 1. Added the build number to the title bar of the window's GUI 127 | 2. Changed the System Log Parser URL link on the About screen to https://github.com/AaronPLopez/SystemLogParser 128 | 3. Updated the Title and Product Assembly Information for all SLP DLLs and Exes 129 | 130 | Build 0.12.17.0 (Prerelease) 131 | 1. Added a listing of the IsSchemaLocking property for each service (where is exists) on the "Site Details" worksheet; This worksheet is found through: the ServiceDetails Analysis Type, enabling the "Add Service Details to Report (Enhanced Site Report)" option in the GUI, or setting the "-esr" parameter to true on the command-line; This worksheet is not available with the Optimized Analysis Type 132 | 133 | Build 0.12.16.0 (Prerelease) 134 | 1. Added Wait Time and Instance Creation Time statistics to the TextFile report 135 | 2. Fixed an issue with the JSON Report Type where the complete output would not be available 136 | 3. Corrected a typo on the command line help where the JSON Report Type was listed as an Analysis Type 137 | 4. Added Wait Time and Instance Creation Time statistics to the JSON report 138 | 139 | Build 0.12.15.0 (Prerelease) 140 | 1. Added the Optimized Analysis Type as an option for AGS -- ArcGIS Server Log Query (Web); this is the recommended Analysis Type as the memory savings over Simple, WithOverviewCharts and Complete reports is significant; an increase in performance might be minimal as the ArcGIS Server Admin API will most likely be a bottleneck in the speed 141 | 2. Updates to the command line help (show help with "slp.exe -h" or "slp.exe -help") 142 | 3. Changed the GUI button "Analyze Logs" to just "Analyze" 143 | 144 | Build 0.12.14.2 (Prerelease) 145 | 1. When generated an enhanced site report, SLP skips over the examination of service datasources that cannot be parsed 146 | 2. Added several internal checks to avoid crashes by SLP when the siteDetails.ArcGISServiceDatasourceDetails object contains unexpected data 147 | 3. Increased the apploglevel verbosity of several internal methods 148 | 149 | Build 0.12.14.1 (Prerelease) 150 | 1. Package was unable to include signed binary files created by Esri; this has been corrected 151 | 152 | Build 0.12.14.0 (Prerelease) 153 | 1. If System Log Parser was generating "enhanced site report" worksheet, the reading of the site configuration data could be interrupted if "corrupt service" was encountered; slp now tries to continue if such a condition is met 154 | 2. Added some details to the application debug log on timezone and utc offset from the machine running System Log Parser 155 | 3. Added an Analysis Type of ServiceDetails which only connects to the ArcGIS web endpoint to retrieve service and datasource information (e.g. no log statistics) 156 | 4. Added comments to the Min Instances (Node) and Max Instances (Node) colummn headers on the "Site Details" worksheet; comments are to clarify that although some DMaps (Shared) and SDS (Hosted) services may contain a configuration with an unexpected instances value, it has not impact to the respective service 157 | 5. Provided an update to the Get-SLPReport.ps1 PowerShell automation script; this script can be used to quickly download large amounts (e.g. 1 month) of (ELB) log data using multiple copy threads in parallel (outside of SLP), then SLP is invoked to read all the log data from the local disk and quickly create an Optimized report 158 | 159 | Build 0.12.13.0 (Prerelease) 160 | 1. Added the version of System Log Parser to the "Summary" worksheet 161 | 2. Added "Start Time for Query (Coordinated Universal Time)" and "End Time for Query (Coordinated Universal Time)" to the "Summary" worksheet 162 | 3. For clarity, the "local time" string in the "Summary" worksheet was changed in "Start Time for Query (local time)" and "End Time for Query (local time)" to the actual local zone name of where System Log Parser is being run 163 | 4. In the Service Provider Stats table ("Summary" worksheet) for the "Hosted (SDS)" row, the "Minimum Instances Per Node (Started)" and "Maximum Instances Per Node (Started)" values have been set to 0 despite their actual definition; this was done to avoid confusion; in some cases, a hosted feature service can take a value for this option but in actuality, it has no affect on the service 164 | 5. In the Service Provider Stats table "Summary" worksheet for the "Shared ArcGIS Pro (DMaps)", the "Minimum Instances Per Node (Started)" and "Maximum Instances Per Node (Started)" values have been set to Site value for the Shared Instance Setting (e.g. Number of shared instances per machine) 165 | 6. Corrected an incorrect labeling of the file and product version for the SystemLogsGUI executable 166 | 167 | Build 0.12.12.0 (Prerelease) 168 | 1. Added a Service Provider Stats table to the "Summary" worksheet 169 | 170 | Build 0.12.11.2 (Prerelease) 171 | 1. Fixed issue when parsing ArcGIS Server logs from the file system where incomplete entry would stop SLP from reading the rest of the files 172 | 173 | Build 0.12.11.1 (Prerelease) 174 | 1. Fixed issue when parsing IIS logs where SLP could crash if the header and data column count did not match on a line and if an Optimized report was being created 175 | 176 | Build 0.12.11.0 (Prerelease) 177 | 1. For ArcGIS Server (web) log queries that have the "Add Service Details to Report" option selected (also known as the enhanced site report option), the ArcGIS "provider" field is listed for each service on the "Site Details" worksheet 178 | 179 | Build 0.12.10.2 (Prerelease) 180 | 1. Fixed a performance issue with the "Arrival Rate" worksheet for WithOverviewCharts and Complete reports 181 | 2. Fixed an issue the "Arrival Rate" worksheet for WithOverviewCharts and Complete reports (ArcGIS Server log queries) where under some conditions the displayed data object could still exceed the maximum of rows allowed in the table 182 | 3. To help keep the file size more manageable for WithOverviewCharts and Complete reports (ArcGIS Server log queries), the maximum table size of the "Arrival Rate" worksheet is now set to 200,000 rows 183 | 4. Minor fixes to application debug log statements 184 | 185 | Build 0.12.10.1 (Prerelease) 186 | 1. Fixed an issue with the Optimized report on "Statistics By Resource" worksheet where if the listed Methods ended the word "Server", the word "Server" would be omitted 187 | 2. Fixed a presentation issue with the "Elapsed Time - All Resources" worksheet where the Epoch Time values would be formatted with a comma 188 | 3. Fixed issue with the "Arrival Rate" worksheet for WithOverviewCharts and Complete reports (ArcGIS Server log queries) where the number of items in the table could exceed the maximum allowable for a spreadsheet and crash System Log Parser; the table will now automatically cutoff the remaining data at 1,000,000 rows 189 | 4. Adjusted the "Request Throughput" worksheet for optimized reports to show Minutes if the available data time span is 24 hours or less; if over 24 hours, the worksheet will instead show throughput across Hours 190 | 191 | Build 0.12.10.0 (Prerelease) 192 | 1. Fixed an issue with the Optimized report for ArcGIS Server file system log queries where long timespans (or very busy Sites) would contain enough data for the "Arrival Rate" worksheet that exceeded the maximum table size in Excel; the maximum number of rows for tables in the Optimized report is set to 200,000 193 | 2. For the Optimized report, the "Client IP", "Users", "Wait Time (Queue Time)", "Instance Creation Time" and "Request Throughput" worksheets have a maximum number of rows for tables set to 200,000 194 | 3. slp.exe now prints an error message if an ArcGIS Server (web) log query is used with the Optimized Analysis Type; currently the only "ArcGIS Server" log query that is supported with the Optimized Analysis Type is "File System" 195 | 4. Fixed issue with the "Request Throughput" worksheet where it would show Minutes insteads of Hours; this was greatly increasing the size of the report; Minutes are only supposed to be shown for query spans less than or equal to 3 hours 196 | 5. Added option to slp.exe called "-optimizedextras" which can toggle the inclusion of some of the worksheets; the specific worksheets are: Client IP, Users, Arrival Rate, Wait Time (Queue Time), Instance Creation Time; some of the table contents for these worksheets can grow very large as the presentation is not a statistic view of the data; this option allows you to omit them in order to use less memory when performing large queries 197 | 6. Increased the application debug log verbosity for some functions 198 | 7. Updated the slp.exe command-line Help 199 | 8. Adjusted the logic for handling the log source directory path(s) for ArcGIS Server file system log queries (Optimized, Simple, WithOverviewCharts, or Complete reports) 200 | 9. Minor formatting adjustments of the "Request Throughput" worksheet 201 | 202 | Build 0.12.9.0 (Prerelease) 203 | 1. Removed the elb.exe and elb.exe.config files from the System Log Parser package; these files were kept for backwards compatibility to support early versions of ArcGIS Monitor extensions; the slp.exe (and slp.exe.config) provides the same functionality and support 204 | 205 | Build 0.12.8.0 (Prerelease) 206 | 1. Minor adjustments to the reading concurrency logic for Optimized Analysis Type IIS log queries 207 | 2. Added the Optimized Analysis Type as an option for AGSFS (ArcGIS Server File System) log queries; this is the recommended Analysis Type as the memory savings and performance increase over Simple, WithOverviewCharts and Complete reports are significant 208 | 3. Minor adjustments to the column descriptions on the "Wait Time (Queue Time)" and "Instance Creation Time" worksheets for IIS log queries using an Analysis Type of "WithOverviewCharts" or "Complete" 209 | 4. Added column description detail to the "Arrival Rate" worksheet for AGS (web) and AGSFS log queries (Analysis Type of WithOverviewCharts and Complete); this was done to match the "Arrival Rate" worksheet from the new Optimized Analysis Type 210 | 5. For AGS (web) and AGSFS log queries, the "Arrival Rate" worksheet, a label is appended to the end of the table noting that the arrival of hosted service requests can be pulled from the logs but their respective elapsed time information is not found on the "Statistics By Resource" worksheet 211 | 6. The Optimized Analysis Type is now the default for ArcGIS Server (FS) log queries from the GUI 212 | 213 | Build 0.12.7.1 (Prerelease) 214 | 1. Removed several command-line options that were utilized to support ArcGIS Monitor (Classic) functionality 215 | 2. Added logic for elbfs log queries to skip files of zero size 216 | 3. Fixed an issue where querying IIS logs from the GUI did not fully take advantage of the all the memory improvements of the Optimized Analysis Type; slp.exe (and elb.exe) were not affected by this issue 217 | 218 | Build 0.12.7.0 (Prerelease) 219 | 1. Fixed issue with Simple, WithOverviewCharts and Complete Analysis Types where HTTP Codes would appear in a non-sorted order 220 | 2. Added some small memory savings for Simple, WithOverviewCharts and Complete Analysis Types generated from IIS log queries 221 | 3. Added the Optimized Analysis Type as an option for IIS log queries; this is the recommended Analysis Type as the memory savings and performance increase over Simple, WithOverviewCharts and Complete reports are significant 222 | 4. The Optimized Analysis Type is now the default for ELB and IIS log queries from the GUI 223 | 5. Improved error handling and message verbosity for various log query methods 224 | 6. Added fix to automatically remove "info.log" if it was added to the path for ArcGIS Server file system log queries 225 | 7. ArcGIS Monitor (Classic) support officially dropped 226 | 8. System Log Parser now includes a PowerShell script (Get-SLPReport.ps1) to assist downloading ELB/ALB logs and uncompressing gzip logs outsite of SLP; this adds benefit of using Get-S3Object to quickly download multiple logs simultaneously and then call 7zip directly to uncompress the logs from a script (as oppose having SLP call an external process) 227 | In this script, the environment inherits credentials from an IAM Role and downloads 30 days worth of logs locally for analysis by SLP 228 | 9. Support for calling 7zip as an external process has been removed; see the Get-SLPReport.ps1 script as an example of more securely utilizing a specific release of 7zip for *.log.gz file decompression 229 | 10. Added several updates to the command-line Help 230 | 231 | Build 0.12.6.0 (Prerelease) 232 | 1. For IIS log queries that use an Analysis Type of WithOverviewCharts or Complete, a worksheet called "Service Referer" has been added which lists requested services and the associated referer header along with a count; static "REST", "SOAP", "Portal" and "OtherRequests" are not listed in this worksheet table 233 | 2. Added automatic filtering support for the TopographicProductionServer (Topographic Mapping Server) capability; requests for such resources are placed in the (ArcGIS) Server group 234 | 235 | Build 0.12.5.1 (Prerelease) 236 | 1. System Log Parser executable files (SystemLogsGUI.exe, slp.exe, and elb.exe) include Digitial Signatures with "Environmental Systems Research Institute, Inc." as the Name of signer 237 | 238 | Build 0.12.5.0 (Prerelease) 239 | 1. Added option to decompress ELB/ALB GNU zip (*.gz) logs with a "local" instance of 7-zip (must be previously downloaded and installed); this feature is only available from the command line (slp.exe) and can be utilized with the "-7zippath [PATH_TO_7ZIP.EXE]" option 240 | 2. Added data labels to all charts generated from reports with the WithOverviewCharts or Complete Analysis Type 241 | 3. Adjusted the data accuracy presented on the Response and Request Sizes worksheet to 3 decimal places (was 6) 242 | 243 | Build 0.12.4.0 (Prerelease) 244 | 1. Removed a runtime check that prevented some methods from executing for non-spreadsheet report types 245 | 2. Included milliseconds in the app log date timestamps 246 | 3. Added the Source Server field to the Optimized report 247 | 4. Improved the query statistic collection and display for the Optmized report; a worksheet called, "Query Statistics" is appended to the report if any logs cannot be read 248 | 5. Added TRACE as an app log level 249 | 6. Some internal code cleanup 250 | 7. Added addtional checks for ELB/ALB queries (simple, withoverviewcharts, complete, optimized, downloadlogsonly) to help prevent SLP from stopping in the event it could not read certain logs 251 | 8. Added app log verbosity statements to ELB/ALB log queries 252 | 9. Added a fix for the command-line (slp.exe) to allow the Report Type of DownloadLogsOnly to override and download logs even if the Analysis Type of Optimized was passed in as an argument 253 | 10. Minor updates to the command-line help 254 | 255 | Build 0.12.3.0 (Prerelease) 256 | 1. Added a new Report Type of "JSON"; this report type option is only available from the command-line (using the "-r json" switch) as the output is printed to the screen (STDOUT) 257 | 2. Add the Response Time Histogram summary to the Text Report 258 | 3. Minor formatting adjustments to the Text Report 259 | 4. Added a fix where System Log Parser would stop reading the logs if entries with a code of 9000 were detected that did not have a service name in the message 260 | 5. Adjusted the log filters to better catch errors SEVERE (errors) with a code of 9000 261 | 6. Removed the Server IP field as a required column of data for IIS logs; although recommended, it is no longer required 262 | 7. Added a fix to address a NullReferenceException that could be encounted when generating an ErrorsOnly report against log entries that were of type SEVERE with a code of 50000 263 | 264 | Build 0.12.2.0 (Prerelease) 265 | 1. Corrected a display error on the "Wait Time (Queue Time)" and "Instance Creation Time" worksheets where the overall values in the bottom table did not reflect a recent column reordering change 266 | 2. Added the "Server Timeout" worksheet for ArcGIS log queries (web and fs, not ageok...no values will be retrieved for ageok reports) 267 | 3. Added the "Wait Time Expired" worksheet for ArcGIS log queries (web and fs, not ageok...no values will be retrieved for ageok reports) 268 | 4. Fixed some internal reporting logic for ArcGIS Enterprise for Kubernetes (ageok) log queries 269 | 270 | Build 0.12.1.0 (Prerelease) 271 | 1. Added support for ArcGIS Enterprise on Kubernetes (AGEOK); require version and built of at least: 10.9.1.1357 272 | Currently, only command-line support exists; to utilize this functionality, pass in "ageok" to the -f command-line switch parameter 273 | Note: Not all functionality available from AGS is available yet from AGEOK (e.g. "Windows Authentication" and the "Enhanced Site Report") 274 | 2. Minor updates to the command-line help 275 | 276 | Build 0.12.0.0 (Prerelease) 277 | 1. Updated AWS libraries (AWSSDK.Core, AWSSDK.S3, AWSSDK.IdentityManagement) 278 | 2. Updated Azure libraries (Microsoft.Azure.KeyVault.Core, Microsoft.Rest.ClientRuntime, Microsoft.Azure.Storage.Common, Microsoft.Azure.Storage.Blob 279 | 3. Removed EPPlus library; added PerfectXL.EPPlus library 280 | 4. Updated Newtonsoft.Json library 281 | 5. Updated the CommonServiceLocator library 282 | 6. Updated the Unity library 283 | 7. Updated the NLog library 284 | 8. Added support for TLS 1.3 285 | 9. The System Log Parser "package" no longer includes the xml file associated with each dll 286 | 287 | Build 0.11.3.0 (Prerelease) 288 | 1. Added command-line support for AWS ELB/ALB connections through Inherited Role Permissions; under this authentication scenario the Access Key, Secret Key and Region can be omitted from the command line as long as permission is granted to a specific machine using an IAM role and Instance Profile. 289 | For example, the following could be granted for S3 bucket access (this is not performed from System Log Parser): 290 | { 291 | "Effect": "Allow", 292 | "Action": [ 293 | "s3:ListBucket", 294 | "s3:GetObject" 295 | ], 296 | "Resource": [ 297 | "arn:aws:s3:::", 298 | "arn:aws:s3:::/*" 299 | ] 300 | } 301 | 302 | For more information: 303 | https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_use_switch-role-ec2_instance-profiles.html 304 | https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html 305 | 2. Fixed a formatting issue with the "Request Performance" worksheet 306 | 3. Fixed a formatting issue with the "Request Status (HTTP 500)" worksheet 307 | 4. For Optimized Analysis Type reports, a "Client IPs" worksheet has been added that lists the IPv4 Addresses and occurrences of connecting machines 308 | 5. For Optimized Analysis Type reports, a "Request Throughput" worksheet has been added that lists the number of requests per hour (or per minute if query time span hour component is 3 hours or less) 309 | 310 | 311 | Build 0.11.2.1 (Prerelease) 312 | 1. Fixed an issue when parsing Azure Application Gateway v2 logs where the resource string was taken from requestUri instead of originalRequestUriWithArgs; originalRequestUriWithArgs is favored as it contains the url path that is closer to what the user called 313 | 2. Fixed an issue when parsing Azure Application Gateway v2 logs where the resource string included the url parameters; in some cases this would make the resource extremely long and could the log parsing to fail with certain Tile requests 314 | 3. Optimized Azure Application Gateway v2 resource analyzer 315 | 4. Optimized the VectorTileServer service type definitions 316 | 317 | Build 0.11.2.0 (Prerelease) 318 | 1. For IIS log queries, any encountered malformed log entries are now just counted and the process is allowed to continue; the malformed log entry count is displayed in the report on the "Summary" worksheet under "Total Malformed Log Entries" 319 | 2. Added "Arrival Rate" worksheet (and Request Arrival Time table) for ArcGIS Server (web and file system) log source queries; The Arrival table can help highlight which services might be consistently receiving more requests simultaneously than the available hardware can support; if the service is a tradition one (e.g. dedicated) it may be a good candidate for increasing the ArcSOC instances and/or adding more hardware resources 320 | 3. Fixed several formatting issues with the "Throughput per Minute" worksheet 321 | 4. The "Statistics By Resource", "Statistics By User", "Wait Time (Queue Time)", and "Instance Creation Time" worksheets are now sorted by Sum; Categorizing the values by Sum helps show which resources are taking the most time overall to work on by the server 322 | 323 | Build 0.11.1.1 (Prerelease) 324 | 1. Improved support for Azure Application Gateway v2 logs 325 | 326 | Build 0.11.1.0 (Prerelease) 327 | 1. Added support for Azure Application Gateway log v2 328 | 2. Support for ArcGIS Monitor is now deprecated within System Log Parser. The functionality to upload log queries into ArcGIS Monitor will be removed in the near future. 329 | 3. When using slp.exe (the command-line version), the ability has been added to override the auto-generated filename of the report through the "-n" switch (e.g. -n "myreport.xlsx") 330 | 4. Expanded the regular expressions for ELB and IIS log sources to include group by tilemap and portal content listing requests 331 | 5. For AGSFS log queries, any encountered malformed log entries are now just counted and the process is allowed to continue; the malformed log entry count is displayed in the report on the "Summary" worksheet under "Total Malformed Log Entries" 332 | 6. For AGSFS log queries, log entries containing the requestID field are now parsed in a safer manner 333 | 7. Changed the default spreadsheet column formatting from "auto" to a min and max characters of 5 and 75, respectively; this was done as some columns of data for some sites can have values that are extraordinarily long making the manual adjustment for screen readability cumbersome 334 | 8. Several updates to the command-line Help 335 | 336 | Build 0.11.0.0 (Prerelease) 337 | 1. In the interest of greatly reducing memory requirements (querying busy sites or executing a query with a long time duration) the Analysis Type of "Optimized" has been introduced. 338 | This new "Optimized" Analysis Type is currently only available for ELB log queries. 339 | Memory savings with "Optimized" have been shown to be up to 120 times less than "WithOverviewCharts"! This allows for running "bigger" log queries which cover a longer time span duration (note: the time to download load the logs from the remote source remains the same). 340 | The analysis with "Optimized" is also significantly faster than "WithOverviewCharts". 341 | The "Optimized" report output will be similar to the "WithOverviewCharts" report but not all all worksheets are available. 342 | The speed and memory savings of the new "Optimized" Analysis Type is achieved by internally transforming the raw log data into a compressed format that is more efficient on system resources to analyze. 343 | 2. Added an automatic filter for tile cache requests for traditional ELB log queries were the resource is rewritten to (more efficiently) just count the calls per map scale 344 | 345 | Build 0.10.6.1 (Prerelease) 346 | 1. Minor code cleanup and removal of some unused logic 347 | 348 | Build 0.10.6.0 (Prerelease) 349 | 1. Added additional memory utilization and performance enhancements 350 | 2. Adjusted the formatting of the HTTP Method section in the "Summary" portion of the report 351 | 3. Fixed issue where some IIS log queries against many large files would result in an "Unhandled Exception: OutOfMemoryException" 352 | 4. Added improvements to security for working with passwords in memory at "rest" 353 | 5. Adjusted the formatting of the CloudFront section in the "Summary" portion of the report 354 | 6. Adjusted the formatting of the HTTP Code section in the "Summary" portion of the report 355 | 356 | Build 0.10.5.0 (Prerelease) 357 | 1. Added some memory utilization enhancements to various System Log Parser functions 358 | 2. Added some format adjustments to the HTTP Code summary sections for both the spreadsheet and textfile reports 359 | 3. Added an HTTP Method section to the "Summary" worksheet in the report 360 | 4. Added a CloudFront section to the "Summary" worksheet in the report which details some CDN information 361 | 362 | Build 0.10.4.2 (Prerelease) 363 | 1. Adding numerical formatting to the integer values of the "Capabilities" worksheet which was missing 364 | 2. Fixed an issue that corrected the reporting on the actual number of HTTP 500 requests for the "Request Status" worksheet if the total exceeded the allowed maximum of 1,000,000 365 | 3. The "Throughput per Minute" worksheet has been moved back the "WithOverviewCharts" Analysis Type after additional testing revealed the increase in file size is not drastic; this worksheet enhances the "WithOverviewCharts" analysis while leaving "Complete" to show details on every request 366 | 367 | Build 0.10.4.1 (Prerelease) 368 | 1. To help keep the file size down for most reports, the "Throughput per Minute" worksheet has been moved back to the "Complete" Analysis Type 369 | 2. Removed some unnecessary formatting in the "Request Status" worksheet 370 | 371 | Build 0.10.4.0 (Prerelease) 372 | 1. Fixed an issue where floating point numbers would not display properly in a Culture other than "en-US" 373 | 2. Increased the verbosity of the System Log Parser application debug log for several methods and objects 374 | 3. Fixed an issue where the service requests for the corresponding item in the "Wait Time" workbook would always be listed as 0 375 | 4. Fixed a KML request identification issue for System Log Parser to ArcGIS Monitor uploads 376 | 5. Fixed an issue that could cause System Log Parser to stop reading malformed ALB logs 377 | 6. Added an optimization to elb log queries with a Report Type of "Downloadlogsonly" that the logs are just saved to disk and not also read into memory 378 | 7. Added optimizations to elbfs to read all the logs files in the specified directory regardless of file timestamps and to read the log files in parallel 379 | 8. Removed the "ArcGIS Capability" stats on the "Summary" worksheet of the spreadsheet and text report 380 | 9. Extended the Analysis Type of "WithOverviewCharts" and "Complete" to include worksheets summarizing requests based on ArcGIS Server Service Types and ArcGIS Server Capabilities 381 | This new analysis assigns a capability to every request that is based on the requested resource 382 | For example: MapServer, ImageServer, NAServer, WMSServer, FeatureServer, HostedFeatureServer, VectorTileServer, MapServerDefinition, IMAGESERVERDEFINITION, PortalContentData, REST, OtherRequests, etc... 383 | These requests are also shown with respect to an ArcGIS Server function capability grouping (availability depends on selected log source) 384 | For example: ArcGIS Server, ArcGIS Server Extensions, OGC, Tiles, Portal for ArcGIS, Server Definition, REST and SOAP, Other Requests 385 | 10. Extended the Analysis Type of "WithOverviewCharts" and "Complete" to include worksheets summarizing the top 500 slowest requests as well as all HTTP 500 errors (up to the 1,000,000 entries) for the given time span of the query (availability depends on selected log source) 386 | 11. Enhanced the HTTP Status information on the Summary worksheet to include details such the specific code as well as its specific definition and number of occurences 387 | 12. For easier reading of the spreadsheet report, some worksheet groups were assigned a specific code 388 | 13. The "Throughput per Minute" worksheet has been moved the "WithOverviewCharts" Analysis Type 389 | 390 | Build 0.10.3.2 (Prerelease) 391 | 1. Increased the warning message verbosity when an IIS log is read that does not meet the required criteria (e.g. header format, necessary fields) 392 | 2. Fixed an issue in the System Log Parser application debug log that would incorrectly report the number of messages per page (pageSize) returned in the response from ArcGIS Server 393 | 3. Fixed an issue with ArcGIS Server (web) queries where an error would be thrown if the hostname of the server contained the same word as the instance name (e.g. arcgis1.mydomain.com:6443/arcgis) 394 | 395 | Build 0.10.3.1 (Prerelease) 396 | 1. Provided clarification of the stated Security Protocol requirement; the Security Protocol requirement only pertains to connections over HTTPS and not queries where the logs are consumed directly from the file system (e.g. reading the local disk or via CIFS share) 397 | 398 | Build 0.10.3.0 (Prerelease) 399 | 1. When using slp.exe (the command-line version), the ability has been added for an ArcGIS Server (FS) query to read from multiple server log directory sources (presumably from the same Site) all at once; the capability can be achieved by wrapping the specified input paths in quotes and separating each ArcGIS Server log folder by a comma, for example: '-i "\\myserver1.domain.com\c$\arcgisserver\logs\MYSERVER1.DOMAIN.COM,\\myserver2.domain.com\c$\arcgisserver\logs\MYSERVER2.DOMAIN.COM"' 400 | 2. For ArcGIS Server (FS) queries, fixed an issue with reading certain ArcGIS Server 10.8 log files where the file would contain an xml header but no data; this is a regression bug from versions 0.8.x.x 401 | 3. For TimeSpan queries that exceed 7days with an Analysis Type of "WithOverviewCharts" or "Complete", the "Time" worksheet will now generate charts for the most recent 7days and omit the rest, previous builds will only generate charts of the TimeSpan was 7days or less 402 | 4. When using slp.exe (the command-line version), the ability has been added for an IIS query to read from multiple server log directory sources (presumably from the same Site) all at once; the capability can be achieved by wrapping the specified input paths in quotes and separating each IIS log folder by a comma, for example: '-i "\\myserver1.domain.com\c$\inetpub\logs\LogFiles\W3SVC1,\\myserver2.domain.com\c$\inetpub\logs\LogFiles\W3SVC1"' 403 | 5. When using slp.exe (the command-line version), the ability has been added for a Tomcat query to read from multiple server logs directory sources (presumably from the same Site) all at once; the capability can be achieved by wrapping the specified input paths in quotes and separating each Tomcat log folder by a comma, for example: '-i "\\myserver1.domain.com\c$\tomcat\logs,\\myserver2.domain.com\c$\tomcat\logs"' 404 | 6. Newer System Log Parser builds only support System.Net.SecurityProtocolType of Tls11 and Tls12 405 | 7. For backward compatibility with older deployments, support for TLS 1.0 was readded; this had been automatically removed with the 0.10.x.x branches migration to .NET Framework 4.8; although System Log Parser supports connecting to deployments that are running older security protocols such as TLS 1.0 or TLS 1.1, it is *highly* recommended to use TLS 1.2 instead 406 | 407 | Build 0.10.2.0 (Prerelease) 408 | 1. Added the ability of a Tomcat query to read server logs even if they are in use (by ArcGIS Server); in the case of reading logs that are in use, the data can be read by System Log Parser as soon as it has been flushed to the disk by ArcGIS Server 409 | 2. Added internal optimization with ArcGIS Server (web) queries by adjusting the Start time and End time ordering 410 | 3. Added additional application debug logging for several query meta data elements and adjusted the details of several existing logging events 411 | 412 | Build 0.10.1.0 (Prerelease) 413 | 1. Added additional application debug logging for ArcGIS Server (web) log queries; the application debug logging can only be utilized when using slp.exe (the command-line version) and passing in the "-apploglevel INFO" or "-apploglevel DEBUG" command-line option; the default log level is OFF 414 | 2. Added the ability of an ArcGIS Server (FS) query to read service logs even if they are in use (by ArcGIS Server); in the case of reading logs that are in use, the data can be read by System Log Parser as soon as it has been flushed to the disk by ArcGIS Server 415 | 3. Added the ability of an ArcGIS Server (FS) query to read server logs even if they are in use (by ArcGIS Server); in the case of reading logs that are in use, the data can be read by System Log Parser as soon as it has been flushed to the disk by ArcGIS Server 416 | 4. Fixed issue where some error messages upon startup of System Log Parser were silently discarded 417 | 5. Added the ability to pass in a value to specify the folder where the application debug log is written; the default location is "C:\Users\[USERNAME]\Documents\System Log Parser\Logs"; this can now be overridden with the already existing "-downloadlocation" command-line option; so '-downloadlocation "D:\some\folder\with\write\access\"' would save all logs to D:\some\folder\with\write\access\ 418 | 6. Added logic to skip any instance creation time log entries where the elapsed time is 0; although it is not anticipate to exist, adding in zeros would throw off the "accuracy" of the statistical representation of this data 419 | 7. Clarified the description on the "Wait Time" and "Instance Creation Time" worksheets that the statistical representation does not include entries where the value was 0 (e.g. 0 seconds) 420 | 8. Changed the page title of the "Instance Creation Time" worksheet from "Instance Creation Statistics" to "Instance Creation Time Statistics" 421 | 9. Changed the tab title of "Wait Time" worksheet from "Wait Time" to "Wait Time (Queue Time)" 422 | 10. For a better user experience, the System Requirements for RAM was increased from 4GB to 8GB 423 | 11. Added the ability of an IIS query to read the logs even if they are in use (by IIS); in the case of reading logs that are in use, the data can be read by System Log Parser as soon as it has been flushed to the disk by IIS 424 | 12. Added additional application debug logging for ArcGIS Server (file system) log queries 425 | 426 | Build 0.10.0.0 (Prerelease) 427 | 1. Added enhanced statistics to the generated reports from System Log Parser; enhanced stats included additional information between the 95th and Max percentiles 428 | 2. Added some application debug logging for Azure 429 | 3. Fixed issue where System Log Parser would not read in any wait time and instance creation time log entries with an ArcGIS Server file system query 430 | 4. Added some application debug logging for the "Datasource Information" and "Datasource Analysis" worksheets (ArcGIS Server web queries) 431 | 5. Updated the System Log Parser URL in About System Log Parser to point back to the original System Log Parser "home page": https://www.arcgis.com/home/item.html?id=90134fb0f1c148a48c65319287dde2f7 432 | 6. Moved System Log Parser to .NET 4.8 Framework 433 | 7. Updated AWS libraries; System Log Parser now supports the following AWS regions: 434 | SLP Input Name Summary 435 | useast1 Amazon.RegionEndpoint.USEast1 The US East (Virginia) endpoint 436 | useast2 Amazon.RegionEndpoint.USEast2 The US East (Ohio) endpoint 437 | uswest1 Amazon.RegionEndpoint.USWest1 The US West (N. California) endpoint 438 | uswest2 Amazon.RegionEndpoint.USWest2 The US West (Oregon) endpoint 439 | eunorth1 Amazon.RegionEndpoint.EUNorth1 The EU North (Stockholm) endpoint 440 | euwest1 Amazon.RegionEndpoint.EUWest1 The EU West (Ireland) endpoint 441 | euwest2 Amazon.RegionEndpoint.EUWest2 The EU West (London) endpoint 442 | euwest3 Amazon.RegionEndpoint.EUWest3 The EU West (Paris) endpoint 443 | eucentral1 Amazon.RegionEndpoint.EUCentral1 The EU Central (Frankfurt) endpoint 444 | apeast1 Amazon.RegionEndpoint.APEast1 The Asia Pacific (Hong Kong) endpoint 445 | apnortheast1 Amazon.RegionEndpoint.APNortheast1 The Asia Pacific (Tokyo) endpoint 446 | apnortheast2 Amazon.RegionEndpoint.APNortheast2 The Asia Pacific (Seoul) endpoint 447 | apnortheast3 Amazon.RegionEndpoint.APNortheast3 The Asia Pacific (Osaka-Local) endpoint 448 | apsouth1 Amazon.RegionEndpoint.APSouth1 The Asia Pacific (Mumbai) endpoint 449 | apsoutheast1 Amazon.RegionEndpoint.APSoutheast1 The Asia Pacific (Singapore) endpoint 450 | apsoutheast2 Amazon.RegionEndpoint.APSoutheast2 The Asia Pacific (Sydney) endpoint 451 | saeast1 Amazon.RegionEndpoint.SAEast1 The South America (Sao Paulo) endpoint 452 | usgovcloudeast1 Amazon.RegionEndpoint.USGovCloudEast1 The US GovCloud East (Virginia) endpoint 453 | usgovcloudwest1 Amazon.RegionEndpoint.USGovCloudWest1 The US GovCloud West (Oregon) endpoint 454 | cnnorth1 Amazon.RegionEndpoint.CNNorth1 The China (Beijing) endpoint 455 | cnnorthwest1 Amazon.RegionEndpoint.CNNorthWest1 The China (Ningxia) endpoint 456 | cacentral1 Amazon.RegionEndpoint.CACentral1 The Canada (Central) endpoint 457 | mesouth1 Amazon.RegionEndpoint.MESouth1 The Middle East (Bahrain) endpoint 458 | 8. Updated EPPlus library 459 | 9. Updated Newtonsoft.Json library 460 | 10. Removed dependency of WindowsAzure.Storage library 461 | 11. Added support for Microsoft.Azure.Storage.Blob and Microsoft.Azure.Storage.Common libraries 462 | 12. Added support for Microsoft.Rest.ClientRuntime and Microsoft.Rest.ClientRuntime.Azure libraries 463 | 13. Updated Microsoft.Azure.KeyVault.Core library 464 | 14. Minor code clean-up 465 | 15. This version (0.10.x.x) will be last System Log Parser release to support log query uploads into ArcGIS Monitor 10.7.x/10.8.x 466 | 467 | Build 0.9.6.0 (Prerelease) 468 | 1. Fixed some additional issues with the ArcGIS Server file system log queries where an Analysis Type of "ErrorsOnly" could not parse certain entries 469 | 2. Fixed an issue with ArcGIS Server file system log queries where under certain conditions an "ErrorsOnly" Analysis Type report could not be created with either a Text file or Spreadsheet file output 470 | 3. For AWS environments, added support and command line help documentation for the "-printfolders" boolean option 471 | 4. Fixed an issue where some error messages where silenced when run from the GUI 472 | 5. For ArcGIS Server web log queries, the "Add Service Details to Report" option has been slightly optimized to issue less requests 473 | 6. Added P95 and P99 (95th and 99th percentile statistics) to the "Throughput per Minute" worksheet table for reports with an Analysis Type of "Complete" 474 | 7. Enhanced the table header descriptions for the "Throughput per Minute" worksheet table 475 | 8. Added a notification warning if no log entries were found for a report with an Analysis Type of "ErrorsOnly" 476 | 9. For IIS log queries, support has been added for the X-Forwarded-For field to use in place of the c-ip field for the Client IP value; If the X-Forwarded-For field is not present or only contains an empty value, the "origial" c-ip field will be used instead 477 | 10. Fixed an issue with ArcGIS Server file system log queries where the elapsed time of some entries where numerically presented based on the locale of the machine running System Log Parser; from the file system, all ArcGIS Server elapsed time entries are consumed and analyzed in a consistent manner 478 | 11. For ArcGIS Server file system log queries from the GUI, the maximum Start Time has been increased from 90day(s) to 120day(s) 479 | 12. Added an option to toggle the inclusion of the "Statistics By User" worksheet for ArcGUS Server Web, ArcGIS Server file system and IIS log queries (by default this is now false) 480 | 13. Added a command line switch (e.g. the -sbu option) to toggle the inclusion of the "Statistics By User" worksheet (by default this is now false) 481 | 14. From the System Log Parser GUI, the name "Output Folder" has been changed to "Report Folder" for all log queries 482 | 15. Corrected a minor tab ordering issue in the System Log Parser GUI 483 | 484 | Build 0.9.5.0 (Prerelease) 485 | 1. Fixed an issue with ArcGIS Server file system log queries where certain entries could not parsed 486 | 2. Fixed an issue with the ArcGIS Server (web and file system) log query where an Analysis Type of "ErrorsOnly" could crash System Log Parser when attempting to generate the report 487 | 3. Added support to the "Site Details" worksheet for SceneServer and WorkspaceServer service types 488 | 4. Fixed an issue that prevented the System Log Parser GUI interface from allowing the ability to select an Analysis Type ErrorsOnly report with a Text file output for ArcGIS Server web and ArcGIS Server file system log queries 489 | 5. For ArcGIS Server log query reports with an Analysis Type of "WithOverviewCharts" or "Complete", the "Queue Time" worksheet has been renamed to "Wait Time" to better match the naming convention of ArcGIS Server 490 | 6. For ArcGIS Server log query reports that upload to ArcGIS Monitor the Counter Instances of "Tr/Interval Service Queueing" and "Tr/Interval Service Queueing Histogram (%)" have been changed to "Tr/Interval Service Wait Time" and "Tr/Interval Service Wait Time Histogram (%)", respectively; their associated Counter Names have also been adjusted to reflect this change 491 | 492 | Build 0.9.4.0 (Prerelease) 493 | 1. Added some memory clean enhancements to SLP to help improve resource consumption after repeated GUI runs 494 | 2. Added ELBFS support to the interval SetSourceTypeFolder method 495 | 3. Enhanced the LogPrefix logic to automatically append "/AWSLogs" if this was omitted 496 | 4. Altered ELB log query logic by removing the requirement of the AWS ListBuckets method 497 | 5. Added command line help documentation for the "-listbuckets" boolean option 498 | 6. Separated the logic for the listbuckets routine. This is no longer part of the log query for a specific bucket. 499 | 7. Added a command line option boolean switch called "-listaccountid" (default is false) to list the AWS account id associated with the given awsAccessKeyId, awsSecretAccessKey and RegionEndpoint 500 | 8. Added command line help documentation for the "-listaccountid" boolean option 501 | 9. Expanded the supported security protocols for ELB/ALB log queries to include TLS 1.1 and TLS 1.2 502 | 10. Expanded the supported security protocols for CFRONT (CloudFront) log queries to include TLS 1.1 and TLS 1.2 503 | 11. Expanded the supported security protocols for Azure log queries to include TLS 1.1 and TLS 1.2 504 | 12. Added an "Instance Creation Time" worksheet to ArcGIS Server log query reports with an Analysis Type of "WithOverviewCharts" or "Complete". The "Instance Creation Time" worksheet list the times required for an service to start up if a request was made and an instance was not available (and the maximum number of concurrent instance was not yet reached). 505 | 13. Corrected an issue where some POST requests for ArcGIS Server log queries may not get counted in the statistical summaries 506 | 14. Removed the ClientIPs worksheet from ArcGIS Server log query reports with an Analysis Type of "WithOverviewCharts" or "Complete" 507 | 15. Added the Users worksheet in addition to the ClientIPs worksheet to IIS log query reports with an Analysis Type of "WithOverviewCharts" or "Complete" 508 | 16. Fixed an issue where Windows Authentication could not be used for some ArcGIS Server log queries 509 | 17. For a report generated from all log queries, the "Statistics" worksheet has been renamed to "Statistics By Resource" and an additional worksheet named "Statistics By User" has been added (For ArcGIS and IIS log queries) which lists resources with respect to the user who requested them 510 | 18. Improved the accuracy of the service type auto detection of Hosted, Dedicated and Shared services for the "Site Details" worksheet (for ArcGIS Server Web based log queries) 511 | 19. Added an ArcGIS Server capability count table to the Summary worksheet 512 | 20. Bring the Summary content of both the Text and Spreadsheet output closer together 513 | 21. Reduced the number of returned ArcGIS Server log entries chunks per web query from 9999 to 1000 to help improve accuracy and consistency 514 | 22. Added the "Domain" and "Server Machine" columns to the "Elapsed Time - All Resources" worksheet 515 | 23. Expanded the supported security protocols for ArcGIS Monitor log uploads to include TLS 1.1 and TLS 1.2 516 | 517 | Build 0.9.3.0 (Prerelease) 518 | 1. Added support to run ELB log queries against gz compressed log files residing on the local disk through the -f ELBFS command line switch 519 | 2. Added support to run ELB log queries against plaintext log files residing on the local disk through the -f ELBFS command line switch 520 | 3. Expanded the command line help to include information on the "esr" switch; the "esr" switch toggles the "IsEnhancedSiteReport" option; In the GUI, this boolean option is presented as "Add Service Details to Report" 521 | 4. Added support for manually setting the ArcGIS Server Token Expiration and/or Request Timeout value from the command line 522 | 5. Expanded the command line help to include information on manually setting the an ArcGIS Server Token Expiration or Request Timeout value 523 | 6. The Vector Tile request signature categorization for ArcGIS Monitor has been changed from "vectortileserver" to "vectortileserver/tile" 524 | 7. Expanded verbosity of error details if a specific log file or log line cannot be read from an elb or alb source 525 | 8. Improved the robustness of reading elb and alb logs originating from HTTP/2 gzip sources 526 | 9. Expanded verbosity of error details if a specific log file or log line cannot be read from an iis source 527 | 10. Improved the robustness of detecting the required iis w3c headers that could be missing or incomplete 528 | 11. Removed iis log source restriction: the specified target directory no longer needs to contain "w3svc" or "inetpub" 529 | 12. Expanded verbosity of error details if a specific log file or log line cannot be read from an azure source 530 | 13. Expanded verbosity of error details if a specific log file or log line cannot be read from a tomcat source 531 | 14. Expanded verbosity of error details if a specific log file or log line cannot be read from a cloudfront source 532 | 15. Expanded verbosity of error details if a specific log file or log line cannot be read from an arcgis server web source 533 | 16. Expanded verbosity of error details if a specific log file or log line cannot be read from an arcgis server file system source 534 | 17. Fixed several issues with the "Throughput per Minute" worksheet which could cause the report generation to fail 535 | 18. Changed the default ArcGIS Server URL from "https://gisserver.domain.com:6443/arcgis" to "https://gisserver.domain.com/server" 536 | 537 | Build 0.9.2.0 (Prerelease) 538 | 1. For log query uploads to ArcGIS Monitor, a "Response Time per Service Type (Sec)" counter has been added 539 | 540 | Build 0.9.1.0 (Prerelease) 541 | 1. Further enhanced the resource id/GUID name consolidation to support additional raster type requests 542 | 2. Fixed an issue with ELB and ALB log queries where a "bad" log entry could stop the progress of the read 543 | 3. In addition to working with the existing Azure CloudBlockBlob storage, SLP now supports the newer CloudAppendBlob storage format (for Azure log queries) 544 | 4. Due to the CloudAppendBlob support, SLP is also now compatible with the newer JSON Lines log format (for Azure log queries) 545 | 5. Added several checks to help avoid a potential "division by zero" error for reports with an Analysis Type of "WithOverviewCharts" or "Complete" 546 | 6. Fixed issue with some log queries where certain services would incorrectly be marked as a "Hosted service" if the web adaptor instances of the site was called "hosted" 547 | 548 | Build 0.9.0.0 (Prerelease) 549 | 1. Fixed issue where "Client IP" column in (from a Complete Analysis report) may not be listed with a CloudFront log query in the "Elapsed Time of All Resources" worksheet 550 | 2. Fixed issue where the "Users" worksheet (from a Complete Analysis report) may be listed instead of the the "ClientIPs" worksheet with a CloudFront log query 551 | 3. With a Complete Analysis report, the "ArcGIS Log File" column was changed to "Log Source" 552 | 4. With a Complete Analysis report, the "Elapsed Time of All Resources" worksheet has been renamed to "Elapsed Time - All Resources" 553 | 5. Fixed some of the ELB and CloudFront tooltips which did not provide the correct information 554 | 6. Added the "-help" option in addition to the "-h" option for displaying "help" from the command line (e.g. with slp.exe) 555 | 7. Added ability to read Azure logs and perform queries 556 | 8. Added timespan details in the generated report for both the query time range and log data time range 557 | 9. Improved log datetime filtering accuracy for certain sources where the some files may contain a few enteries of the previous hour 558 | 10. Changed the maximum allowable lines of generated log data (per worksheet) in the spreadsheet report to be 1,000,000 559 | 11. Added the ability of SLP to automatically create a text file of the data for a worksheet (e.g. "Throughput per Minute", Elapsed Time - All Resources) that exceeds the maximum allowable lines 560 | 12. Added some report generation progress feedback to the SLP GUI 561 | 13. Added a check in the SLP GUI to help ensure that only the Analysis Type of "Simple" can be used with a Report Type of "TextFile" 562 | 14. Increased the logs detail of information for traditional Feature Server requests from a VERBOSE report 563 | 15. For ArcGIS Monitor log uploads, items like tiles and service definition requests are now automatically saved to the repository 564 | 16. Optimized the VerboseMode and ErrorsOnly Report column formatting to improve creation times 565 | 17. System Log Parser now consolidates jobid GUIDs within specific ArcGIS-type requests in order to help make the generated statistics easier to read. The exception to this are requests to ArcGIS Portal resources (e.g. "/sharing/rest/content/items/[guid]")...these are left in their original form. 566 | 18. Added ALB/ELB fixes for malformed requests 567 | 19. Added improvements to multistream gzip decompression for incomplete files 568 | 20. Added a fix to ArcGISFS log queries to improve accuracy of read based on certain entries 569 | 21. Added a fix to ArcGISFS log queries for an Analysis Type of ErrorsOnly...previous builds would not find any WARNING or SEVERE log entries, under this condition 570 | 22. Fixed an issue when creating the Statistics worksheet where it could take a long time if the log query resulted in many different resources 571 | 23. Added a "Response|Request Sizes" worksheet for reports of Analysis Type "WithOverviewCharts and "Complete" which show the top resources for the largest response and request sizes 572 | 573 | Build 0.8.19.0 (Prerelease) 574 | 1. Adjusted the queueing counts in the ArcGIS Server report to not include entries with an elapsed time of 0. With this adjustment the count now reflect requests that queued (instead of any request). 575 | 2. Added "Service Requests" count column to the "Queue Time" worksheet to given an improved perspective of the impact of the queue time entries 576 | 3. Fixed issue with the SiteDetail worksheet where some services would not be listed if no extensions were enabled 577 | 4. Added an "Is Enabled" column to the SiteDetail worksheet to help filter out extensions that are enabled or disabled 578 | 5. For ArcGIS Server (Web) log queries that are uploaded to ArcGIS Monitor, the following metrics have been added under "Tr/Interval Service Queueing": 579 | TotalSecondsQueued 580 | TotalRequestsQueued 581 | TotalServicesQueued 582 | Tr/Interval Service Queueing Histogram (%) 583 | 6. For IIS log queries that are uploaded to ArcGIS Monitor, the domain and username information is reported on 584 | IIS logs will contains this information if the "User Name (cs-username)" is enabled (in IIS) and the ArcGIS site's Web Adaptor is configured for Windows Authentication and authenticated users made requests during the collection interval 585 | 7. Fixed issue with ArcGIS Server (Web) log queries uploaded to ArcGIS Monitor where the values of the "Tr/Interval Response Time Histogram (%)" counters were the counts instead of the percentage of counts 586 | 8. Added ability to read Tomcat access logs and perform queries (command line only). To enable Tomcat access logs (Note: with this setting, the logs will rotate every hour): 587 | Find the server.xml file within the appropriate Tomcat installation 588 | Make backup copy of original server.xml file 589 | Open server.xml in text editor 590 | Replace the following line: 591 | 595 | With the following line: 596 | 597 | Save edits 598 | Restart Tomcat or associated product service 599 | Note: in this example, the suffix is set to ".txt" but using ".log" will also work with slp.exe 600 | 9. Fixed an issue with ArcGIS Server File System log parsing when a "partial" log is encountered. A "partial" log would have only one line and typically resemble the following: 601 | 602 | 10. Fixed an issue with ArcGIS Server (web) log queries when the "Add Service Details to Report" option was enabled and the operation was Cancelled. 603 | If System Log Parser was in the middle of collecting the service manifest information, the process would continue until all services were read. 604 | This background process now stops much more quickly. 605 | 11. For ArcGIS Server (web) log queries, there is now the option to manually provide an ArcGIS Server or ArcGIS Portal token, instead Windows Authentication or passing in a username/ 606 | . 607 | Examples of generating a token to use with System Log Parser: 608 | To generate an ArcGIS Server token... 609 | For a Server URL such as: https://[HOSTNAME]:6443/arcgis 610 | Connect to the ArcGIS Server Generate Token resource: https://[HOSTNAME]:6443/arcgis/admin/generateToken 611 | Enter in the username and password for the PSA account, use "Request IP" for Client 612 | Copy the generated token to use with System Log Parser 613 | To generate a Portal Server token (with a Portal Federation Login)... 614 | For a Server URL such as: https://[HOSTNAME]/arcgis 615 | Connect to the ArcGIS Adminstrator Directory login resource: https://[HOSTNAME]/arcgis/admin/login 616 | Copy the listed 'Webapp URL' from the Portal Federation Login section 617 | Connect to the ArcGIS Portal Generate Token resource: https://[HOSTNAME]/arcgis/sharing/generateToken (assuming resource is accessible) 618 | Enter in the username and password for the appropriate account, select 'Webapp URL' for Client and provide the 'Webapp URL' from the previous resource 619 | Copy the generated token to use with System Log Parser 620 | 12. Minor formatting adjustments and column comment corrections for spreadsheet generated reports 621 | 13. Fixed an issue where if the "Add Service Details to Report" option was selected, System Log Parser would not read the logs and return a warning of "Could not collect logs. Exception: Object reference not set to an instance of an object" 622 | 14. Improved the clarity of the warning message for ArcGIS Server (web) log queries, if the supplied Server URL was invalid or could not be utilized 623 | 624 | Build 0.8.18.0 (Prerelease) 625 | 1. Included the Web Server Certificate Alias field in the Site Details worksheet 626 | 2. Added support for several ArcGIS Server 10.6.1 features: 627 | Expanded the Site Details worksheet to include hardware information on each machine in the Site 628 | Added a worksheet called Queue Time which statistically breaks down the wait time collected from each service 629 | 3. Fixes for gzip decompression of the multistream ALB logs (as well as CloudFront logs) 630 | 4. Changes for (ELB/IIS/CloudFront) log collection uploads into ArcGIS Monitor: 631 | The Counter name for the Instance "Network Sent MB/Interval" has been changed from "TotalMegabytes" to "TotalMegabytesSent" 632 | The Instance "Network Received MB/Interval" has been added with Counter name "TotalMegabytesReceived" 633 | 5. If a ReportType of "DownloadLogsOnly" was selected and the log source is ELB or CloudFront the log names now have a suffix of ".txt" 634 | 6. If an AnalysisType of "Complete" was selected, the data column of "CloudFront Edge Result Type" in the "Elapsed Time of All Resources" worksheet will default to "-" for log source where this value does not apply 635 | 7. If an AnalysisType of "Complete" was selected against an ALB log source, the "Elapsed Time of All Resources" worksheet will populate the "TargetGroup" column...otherwise this value will default to "-" 636 | 8. Added statistics on the resource request total for spreadsheet or text file generated reports 637 | 9. Provided a notification on the "Statistics" section of a spreadsheet or text file generated report if no resource requests were found for the given period of time 638 | 10. Increased the precision accuracy of the Count Pct and Sum Pct columns in the "Statistics" worksheet of the Spreadsheet report 639 | 11. Changes for ArcGIS (Web/File System) log collection uploads into ArcGIS Monitor: 640 | The Counter name for "Users Tr/Sample" has been changed to "Users Tr/Interval Total" 641 | The Counter name for "TotalGBSent Tr/Sample" has been changed to "TotalMBSent Tr/Interval Total" 642 | The Counter name for "TotalRequests Type" has been changed to "TotalRequests Tr/Interval Total" 643 | The Counter "RequestAverage Type" has been added 644 | The Counter "< 3 sec Tr/Interval Response Time Histogram (%)" has been added 645 | The Counter ">= 10 and < 20 sec Tr/Interval Response Time Histogram (%)" has been added 646 | The Counter ">= 20 and < 30 sec Tr/Interval Response Time Histogram (%)" has been added 647 | The Counter ">= 3 and < 10 sec Tr/Interval Response Time Histogram (%)" has been added 648 | The Counter ">= 30 and < 60 sec Tr/Interval Response Time Histogram (%)" has been added 649 | The Counter ">= 60 sec Tr/Interval Response Time Histogram (%)" has been added 650 | 12. If a Report Type of "DownloadLogsOnly" is selected for an ELB or CloudFront log query and the logs are stored compressed, the raw log (*.log.gz) and decompressed log (*.log.txt) will be saved locally 651 | 652 | Build 0.8.17.0 (Prerelease) 653 | 1. Remove background spreadsheet grid lines from the Statistics worksheet 654 | 2. Expanded the Datasource Details worksheet 655 | This has been broken up into two worksheets: 656 | Datasource Information (formally known as Datasource Details) -- which lists all observable services and their respective datasource information 657 | Datasource Analysis -- which lists datasets by their occurence in services as well as map documents by their occurence in services 658 | 3. Added wide character support for ArcGIS Monitor 659 | 660 | Build 0.8.16.0 (Prerelease) 661 | 1. Initial support for CloudFront log parsing (command-line and GUI) 662 | 2. Added new Report Type, "DownloadLogsOnly". This report only downloads the logs available from the remote source from the queried time window as saved them to the specified location. 663 | 3. Added command-line switch "-downloadlocation" to specify custom location to save downloaded logs 664 | 4. Added GUI support for the new Report Type, "DownloadLogsOnly" 665 | 5. Added checks to validate that the Token Service URL is defined when querying the /rest/info resource page. In some deployments, this may not be set for certain access points to the installation. 666 | 6. Added a "Datasource Details" worksheet to the "Add Service Details to Report" option. This worksheet lists information on the backend datasources for each service, if available. 667 | 7. Expanded the "Site Detail" worksheet to include information on enabled extensions for each service. The capabilities of each extension are also listed. 668 | 8. Added code "100002" to the built-in ArcGIS Server log parser query filter to include GPServer.ExecuteJob.[ServiceName] Method times for geoprocessing services. This log entry reflects the total time taken by geoprocessing service to fullfil a job request. 669 | 9. Fixed issue in the report where the ELB name would appear empty, although there are conditions where this is valid, the blank andn empty string has been replaced with "[empty]" 670 | 10. For ELB log query uploads to ArcGIS Monitor the Validation counter called "TotalELBNames" has been renamed to "TotalNames" 671 | 11. For generated reports from ELB logs the "ELB Name (Total)" field has been renamed to "Names (Total)" 672 | 12. Minor updates to the command-line help 673 | 13. Fixed issue where System Log Parser would crash if the number of lines within a spreadsheet's worksheet exceeded 1,048,576 674 | 14. Fixed issue where System Log Parser could not retrieve the ArcGIS Server Site Details and generate a report if a service did not have a cluster set (e.g. "default") in its configuration 675 | 15. The System Log Parser extension folder now includes an "elb.exe.config" file to accompany the elb.exe executable 676 | 677 | Build 0.8.15.1 (Prerelease) 678 | 1. Updated build number 679 | 680 | Build 0.8.15.0 (Prerelease) 681 | 1. For ELB/IIS log query uploads to ArcGIS Monitor, the counterCategory for items containing "/arcgis/directories/" have been changed from OTHER to REST 682 | 2. For ELB/IIS log query uploads to ArcGIS Monitor, generic REST and SOAP requests (e.g. non-specific ArcGIS capability requests) are now always uploaded. Previous builts uploaded these items if the "-uploadstatic" option was true. 683 | 3. Fixed issue where the "-uploadother" was not true by default. With this fix, all non-ArcGIS requests (that are not considered static requests like *.html, *.js, *.png, etc...) will be uploaded to ArcGIS Monitor. 684 | 4. Fixed issue where System Log Parser would crash creating an Excel report if the line entry in the queried ELB log was partial and not complete 685 | 5. When a line entry for a queried ELB log is found to be partial and not complete, System Log Parser will identify the item with an "[incomplete log entry]" designation for its resource 686 | 6. When a line entry for a queried ELB log is found to be incomplete, System Log Parser will attempt to read as Datetime, Status Code or Elapsed Time, if available 687 | 7. For queried ELB log entries, System Log Parser by default will assume an HTTP Status code of 500 instead of 0 (the previous default) 688 | 8. For queried ELB log entries, System Log Parser by default will initialize the datetime to Epoch 689 | 9. Fixed issue where System Log Parser could incorrectly identify a resource's ArcGIS Capability 690 | 10. The following labels (e.g. counter instances) have either been renamed or how they report the information has been slightly altered: 691 | HTTP 400 Codes/Interval --> HTTP 400 tr/Interval 692 | HTTP Error/Interval --> HTTP 500 tr/Interval 693 | HTTP OK/Interval --> HTTP 200 tr/Interval (and "HTTP 300 tr/Interval") 694 | 11. The following counter names have been moved: 695 | HTTP400s --> from HTTP Error/Interval to HTTP 400 tr/Interval 696 | 12. The following counter names have been removed from the HTTP Error/Interval label 697 | FailedRequest (%) 698 | FailedRequests 699 | 13. The following label has been created 700 | "HTTP Code (%)" 701 | 14. The following counter names have been moved: 702 | Http 500 (%) --> from HTTP Error/Interval to HTTP Code (%) 703 | 15. The following counters have been added: 704 | Http 200 (%) 705 | Http 300 (%) 706 | Http 400 (%) 707 | 708 | Build 0.8.14.0 (Prerelease) 709 | 1. The following labels (e.g. counter instances) have either been renamed or how they report the information has been slightly altered: 710 | GB Sample Sent --> Network Sent MB/Interval 711 | HTTP 400 Codes/Sample --> HTTP 400 Codes/Interval 712 | HTTP Error/Sample --> HTTP Error/Interval 713 | HTTP OK/Sample --> HTTP OK/Interval 714 | IPs/Sample --> IPs/Interval 715 | Response Time (Seconds) --> Response Time (Sec) 716 | Tr/Minute Peak --> Tr/Min Peak 717 | Tr/Sample ArcGIS Endpoint Definition --> Tr/Interval ArcGIS Endpoint Definition 718 | Tr/Sample ArcGIS Server --> Tr/Interval ArcGIS Server 719 | Tr/Sample ArcGIS Server Extension --> Tr/Interval ArcGIS Server Extension 720 | Tr/Sample Non-ArcGIS --> Tr/Interval Non-ArcGIS 721 | Tr/Sample OGC --> Tr/Interval OGC 722 | Tr/Sample Portal --> Tr/Interval Portal 723 | Tr/Sample Response Time Histogram --> Tr/Interval Response Time Histogram 724 | Tr/Sample Tile --> Tr/Interval Tile 725 | Tr/Sample Total --> Tr/Interval Total 726 | 2. The following counter names have been renamed: 727 | TotalGigabytes --> TotalMegabytes 728 | FailedRequest% --> FailedRequest (%) 729 | 3. The following counters have been added: 730 | Http 500 (%) 731 | 4. For uploads into ArcGIS Monitor, a fix was added to ensure the counterCategory type is properly set for either an extension or task collection 732 | 733 | Build 0.8.13.0 (Prerelease) 734 | 1. Added support for ELB log query uploads (into ArcGIS Monitor) to be run as an extension (every 5min or 15min, in addition to a task which is 1hr). By design, the log rotation mechanism of IIS mechanism only supports a resolution up to 1hr. 735 | 2. In addition to the "HTTP400s (Tr/hour)" counter total for ELB and IIS log queries, the HTTP 400 codes have been broken up further into a dedicated group called "HTTP 400 Codes/Hour". This includes the following HTTP 400 codes: 400, 401, 403, 408, 498, 499 and 4xx (all other 400 codes). 736 | 3. Fixed issue where ArcGIS (Web) log queries (into ArcGIS Monitor) may not properly count WorkflowManagerServer and SteamServer requests 737 | 4. Added additional columns to the Excel Statistics worksheet to assist with filtering. The Capability column was added for ArcGIS log query reports. The Instance and Capability column was added for ELB and IIS log query reports. 738 | 5. Added a Failed Request Percentage counter to ELB and IIS uploads (into ArcGIS Monitor). This counter will appear under the "HTTP Error/Hour" instance. 739 | 6. Added Failed Request Total counter to ELB and IIS uploads (into ArcGIS Monitor). This counter will appear under the "HTTP Error/Hour" instance. 740 | 7. Fixed issue where some failed requests (from ELB log queries) were not reported as HTTP 500 Errors 741 | 8. In order to provide more clarity on the "Server" counter totals for ELB and IIS log queries (found under the instance names of "Tr/Hour ArcGIS Server" and "Tr/Hour ArcGIS Server" and "Tr/Hour ArcGIS Server Extension") an additional instance called "Tr/Hour ArcGIS Endpoint Definition" has been created. These totals count hits to ArcGIS services that do NOT contain function calls like (export, query or find). 742 | 9. Added the version number to the output of the command-line Help (slp.exe -h) 743 | 10. Added some missing choices for several of the command-line parameter options 744 | 11. The labels for all (hour based) log query counters uploaded to ArcGIS Monitor have been renamed to from "Tr/Hour" to "Tr/Sample" 745 | 12. Added SampleInterval (seconds) as a Validation counter for ELB/IIS log query uploads (into ArcGIS Monitor) 746 | 747 | Build 0.8.12.0 (Prerelease) 748 | 1. Enabled ArcGIS Server (File System) to create a Report with an Analysis Type of VerboseMode 749 | 2. Minor formatting changes on the Elapsed Time of All Resources worksheet for a Complete Analysis Type Report 750 | 3. Added .NET Regular Expression support to slp.exe for all log queries. Note: this feature is for Advanced Use Only! 751 | a. Use the Regular Expression Pattern (-regexpattern) to pass in a string containing the regex rule (the default is an empty pattern...which is to match everything) 752 | b. Use the Regular Expression IgnoreCase Boolean (-regexignorecase) to toggle case-sensitivity (the default is true...which is to the ignore case) 753 | For ELB, the regular expression pattern will match against the URL field 754 | For IIS, the regular expression pattern will match against the uri-stem field 755 | For ArcGIS Server, the regular expression pattern will match against the source field 756 | For a quick reference on the Regular Expression Language in .NET see the following online resource: https://docs.microsoft.com/en-us/dotnet/standard/base-types/regular-expression-language-quick-reference 757 | 4. Added .NET Regular Expression support to System Log Parser GUI for all log queries 758 | 5. Added the ability for slp.exe to toggle the ability of uploading "Other" (-uploadother) non-ArcGIS Server requests into ArcGIS Monitor (default is true...which is to upload) 759 | 6. Fixed issue with IIS log query where the Summary worksheet in the report, under some circumstances, was not able obtain Source Server (IIS IP Address) from the logs 760 | 7. Fixed issue with System Log Parser GUI where the Analysis Type of ErrorsOnly would show up as an option from the drop down list for ELB and IIS log queries 761 | 8. Minor formatting changes to the System Log Parser GUI 762 | 763 | Build 0.8.11.0 (Prerelease) 764 | 1. Fixed a data formatting display issue in the Site Details worksheet 765 | 2. For ArcGIS Server WorkflowManager log query uploads to ArcGIS Monitor, the counter name has been changed from WorkflowManagerServerRequests to WorkflowManagerServer 766 | 3. Added StreamServer (e.g. GeoEvent Server) support to the Site Details worksheet 767 | 4. Added StreamServer (e.g. GeoEvent Server) support to log data uploaded into ArcGIS Monitor 768 | 5. For improved group and analysis, there are Counter changes for ELB and IIS log query uploads to ArcGIS Monitor: 769 | a. The "Tr/hour" instance has been added to hold all counters related to throughput 770 | b. All counters in the "ArcGIS Server", "ArcGIS Server Extension", "OGC", "Other" and "Tiles" instances have been moved to the "Tr/hour" instance 771 | c. Moved AvgRequestResponseTime from "Requests" Instance to "Response Time (second)" Instance 772 | d. Moved TotalRequests (Tr/hour) from "Requests" Instance to "Tr/minute" Instance 773 | e. Moved HTTP200s (Tr/hour) from "Code" Instance to "HTTP OK/hour" Instance 774 | f. Moved HTTP300s (Tr/hour) from "Code" Instance to "HTTP OK/hour" Instance 775 | g. Moved HTTP400s (Tr/hour) from "Code" Instance to "HTTP Errors/hour" Instance 776 | h. Moved HTTP500s (Tr/hour) from "Code" Instance to "HTTP Errors/hour" Instance 777 | i. The description of "(Tr/hour)" has been moved from all counter names to the respective parent folder 778 | j. TotalIPs (IPs/hour) has been renamed to TotalIPs; the parent folder has been renamed to IPs/hour 779 | k. Moved Tr/minute Statistics from "Requests" Instance to "Peak Tr/minute" Instance 780 | l. The Tr/minute Max counter has been renamed to Max and moved to "Peak Tr/minute" Instance 781 | m. The Tr/minute P95 counter has been renamed to P95 and moved to "Peak Tr/minute" Instance 782 | n. The Tr/minute TotalSamples counter has been renamed to TotalSamples and moved to "Peak Tr/minute" Instance 783 | o. Removed the Tr/minute Avg counter 784 | p. Removed the Tr/minute Min counter 785 | q. Removed the Tr/minute P5 counter 786 | r. Removed the Tr/minute P25 counter 787 | s. Removed the Tr/minute P50 counter 788 | t. Removed the Tr/minute P75 counter 789 | u. Removed the Tr/minute P99 counter 790 | All data from previously collected counters, under the previous Instance names, will be lost 791 | 792 | Build 0.8.10.0 (Prerelease) 793 | 1. System Log Parser now supports Portal Federation Logins 794 | 795 | Build 0.8.9.0 (Prerelease) 796 | 1. Request details for "OtherRequests" are no longer uploaded by default for ArcGIS Monitor logEntry uploads. 797 | The ELB or IIS extension for ArcGIS Monitor will have to be configured to specifically enable the uploading of this information. 798 | 2. GUI input fields such as Server URL from ArcGIS Server (Web) or Bucket Name from ELB have been "trimmed" to remove the white-spaces and the end and beginning in order to prevent unintended "blank characters" from being passed in 799 | 3. Various, but minor changes to the command-line Help descriptions and GUI tooltips for some input options 800 | 4. Update the System Log Parser's Elapsed Time Minimum field in the GUI to reflect the same default value of the command-line option, -etminimum. This default value (for the command-line) was recently changed from 3 to 0. 801 | 802 | Build 0.8.8.0 (Prerelease) 803 | 1. For the Site Details worksheet, Min Instances (Cluster) and Max Instances (Cluster) columns have been added to show the total number of configured instances of the service based on how many machines are participating in its respective cluster. 804 | This is in addition to the original columns of Min Instances (Node) and Max Instances (Node) which list the total number of configured instances with respect to each node (or machine). 805 | 2. For the Site Details worksheet, the name of the Cluster that each service is participating in has been added 806 | 3. For the Site Details worksheet, Max Startup Time for each service has been added 807 | 4. For the Site Details worksheet, the configured Capabilities of each service, if available, has been added 808 | 5. For the Site Details worksheet, the total number of published services (running or not) has been added to the bottom of the Service Name column 809 | 6. Changed the source type field in the report summary from AGS to ArcGISServer where ArcGIS Server (Web) was the log source type. This is also reflected in the file name of the report. 810 | 7. Changed the source type field in the report summary from AGSFS to ArcGISServerFS where ArcGIS Server (File System) was the log source type. This is also reflected in the file name of the report. 811 | 812 | Build 0.8.7.1 (Prerelease) 813 | 1. Fixed a calculation of the Service Requests column on the Site Details worksheet where the number reported was not the correct representation of "the total number of requests for that service" 814 | 815 | Build 0.8.7.0 (Prerelease) 816 | 1. Changed the default value for the Elapsed Time Minimum (-etminimum) from 3 to 0. This will include all requests by default in the Complete Report (All Resources Elapsed Times worksheet). 817 | 2. Add an Response Time Histogram grouping on the Summary worksheet to classify All requests (regardless of the passed in Elapsed Time Minimum value) based on their respective time (e.g. less than 3 seconds; equal to 3 seconds but less than 10, etc...) 818 | 3. Add an Response Time Histogram grouping to the ArcGIS Monitor upload for ELB and IIS log queries 819 | 4. Added a Throughput per Minute worksheet to the Complete report. The Throughput per Minute worksheet is a compliment to the Time worksheet chart and breakdown the throughput of all requests by the minute in tabular form 820 | 5. For non-ArcGIS Server log queries a "Path" column was added to assist with making spreadsheet filtering more convenient. This column is derived from the last part of URL Path. 821 | 6. Added *ALL* Report Types to the special time frame case of "End Time: 1hr (ago) Start Time 2hr (ago)". In this case, the previous full hour time range is taken and all report types will now read and present the same information. In previous builds only a Report Type of ArcGIS Monitor followed this rule. 822 | 7. Removed "Server Version (Build)" and "Server LogLevel (FileAge)" from ELB and IIS reports as such information is not applicable 823 | 8. In Addition to the "TotalLogsRead" Validation counter (ELB and IIS), an "TotalELBNames" Validation counter (ELB only) has been include to assist with QA 824 | 9. ELB queries will no longer create reports or upload data to ArcGIS Monitor if logs from multiple AWS Account Ids are discovered within the configured bucketname. Such a scenario will require additional input in order to filter out the "unwanted" logs. This was done for improved data integrity to ensure that the appropriate and expected logs are being analyzed. 825 | 10. Statistics are now performed on *ALL* requests (successful or not). Note: All elapsed times as reported from ArcGIS Server are successful. 826 | 11. ELB and IIS ArcGIS Monitor uploads now include an Average Response Time per Request as well as a listing of Throughput per Minute statistics 827 | 12. ELB requests that have an incomplete log entry (e.g. "backend HTTP Status = 0" or "elapsed time = -1"), now have their HTTP Status taken from ELB and their elapsed time changed to 0 828 | 13. Added a check to prevent an ArcGIS Server (Web) log query from starting when the "Use Windows Authentication" connection was enabled but the Server URL was not point to a Web Adaptor deployment 829 | 14. Added a check to prevent the specified End Time from creating a DateTime younger than the DateTime created from the specified Start Time 830 | 831 | Build 0.8.6.0 (Prerelease) 832 | 1. Enhanced the Complete Report (All Resources Elapsed Times worksheet) by adding the HTTP Status field 833 | 834 | Build 0.8.5.3 (Prerelease) 835 | 1. Minor correction of the System Log Parser library versioning 836 | 837 | Build 0.8.5.2 (Prerelease) 838 | 1. Fixed rendering issue where the password field for the ArcGIS Server (Web) window was not visible 839 | 840 | Build 0.8.5.1 (Prerelease) 841 | 1. Fixed EndTime hour issue with Analysis Type "VerboseMode". With "VerboseMode", only an EndTime "Now" can be used. 842 | 2. Synchronized all dll and exe versions to the release number 843 | 844 | Build 0.8.5.0 (Prerelease) 845 | 1. Added an Analysis Type called "VerboseMode" to ArcGIS Server (Web) log queries. If the ArcGIS Server instance contains VERBOSE log entries, this type will generate a special report that can breaks down each type of request with detailed information. 846 | 847 | Build 0.8.4.0 (Prerelease) 848 | 1. Fixed issue where under certain circumstances, the Minimum statistical value for a ELB resource would get reported as -1 849 | 2. Fixed issue with the ELB log query where the optional "ELB Name Match" input would only be read in if the optional "Log Prefix" input was populated 850 | 3. Addd a Client IP chart detailing the most popular IPs based on the number of respective requests. This chart can be found for an ELB or IIS log search with an Analysis Type of WithOverviewCharts or Complete. 851 | 4. Enhanced the Complete Report (All Resources Elapsed Times worksheet) for an IIS log query by constructing the requested url 852 | 5. Added an ArcGIS Server User chart (Total Requests per User) 853 | 6. Fixed issue with the Complete Report (All Resources Elapsed Times worksheet) where the Content Length was zero for an ArcGIS Server FS (File System) log query 854 | 7. Clarified response time minimum to be Elapsed Time Minimum (-etminimum) 855 | 8. Fixed issue where some selected log queries did not honor the Elapsed time minimum if it was changed from the default value 856 | 9. Fixed issue with ArcGIS Server FS (File System) log query where the log file's datetime entry was read in and internally (and incorrectly) converted to UTC. The datetime entry of each line is in LocalTime. 857 | 10. Added a fix to workaround an ArcGIS Server FS (File System) datetime precision issue. The datetime entry of each line include milliseconds but stated precision can be ambiguous ('2017-07-20T16:50:01,72' is meant to be '2017-07-20T16:50:01,072' but on conversion is interpreted as '2017-07-20T16:50:01,720'). 858 | This resulted in chance of some of the requests listed in the Complete Report (All Resources Elapsed Times worksheet) to be slightly out of order chronologically (for each unique minute listed). 859 | 11. For clarity, the Date Time column in the All Resources Elapsed Times worksheet (of the Complete Report) now includes milliseconds 860 | 12. Enhanced the Complete Report (All Resources Elapsed Times worksheet) to now include the same decimal precision (e.g. 5) found in the ArcGIS Server logs 861 | 13. Enhanced the Complete Report (All Resources Elapsed Times worksheet) to now include the name of the source log file where the request entry occurred. This column is empty for an ArcGIS Server (Web) log query as the API does not provide this information. 862 | 14. Enhanced the Complete Report (All Resources Elapsed Times worksheet) to now include the ArcGIS Server log entry Code and Type for ArcGIS Server (Web and File System) log queries 863 | 15. Log entries of Type 'INFO' and code '10138' were listed as a request in the statistics table and as a request in the Complete Report (All Resources Elapsed Times worksheet). These have now been removed and are no longer included in the report. 864 | 865 | Build 0.8.3.0 (Prerelease) 866 | 1. Reset the default output directory to be based on "[Username's Documents Folder]\System Log Parser\Reports" instead of "[Username's Documents Folder]\Documents" 867 | 2. Fixed bug with IIS uploads into ArcGIS Monitor where the collection would run at the top of the hour but IIS had not yet rotated its logs (from the previous hour). SLP now intentionally waits for several minutes until IIS has rotated its logs. 868 | 869 | Build 0.8.2.0 (Prerelease) 870 | 1. Added the ability to filter/search ELB logs based on the load balancer name for the case where multiple sites are writing ELB logs to the same folder 871 | 2. Enhanced the Complete Report to just display request details for response times greater than a configurable amount of seconds (default is 3). This is adjusted from the "Response Time Minimum" field in the GUI. 872 | 3. Greatly improved the generation performance of the Complete Report 873 | 4. "OtherRequests" are now uploaded for ELB and IIS ArcGIS Monitor logEntry uploads 874 | 5. "OtherRequests" are now uploaded with Source as UriStemResource instead of Asset 875 | 6. logEntry upload now contain the requestURL, if it was available in the log (e.g. ELB) 876 | 7. The Complete Report now contains the requestURL, if it was available in the log (e.g. ELB) 877 | 8. The ArcGIS Server (Web) log parsing GUI section now disables the "Add Service Details to Report" option if an Analysis Type of "ErrorsOnly" is selected 878 | 9. Fixed issue with ArcGIS Server FS (File System) and IIS where the log query would stop with "Sequence contained no elements" if a log was encountered that had file size of 0 (zero) 879 | 880 | Build 0.8.1.2 (Prerelease) 881 | 1. When using ArcGIS Server FS (File System) log parsing, the time ranges in the GUI have been increased to match ArcGIS Server (File System) 882 | 2. Internal code clean-up; removed unused libraries from core SLP logic 883 | 3. The Third_Party_Software_README.txt has been updated to include information on all the third party support libraries used by System Log Parser 884 | 885 | Build 0.8.1.1 (Prerelease) 886 | 1. Fixed problem in the GUI where the Elastic Load Balancing (ELB) log parsing Secret Key field was disabled and unable to accept input text 887 | 888 | Build 0.8.1.0 (Prerelease) 889 | 1. Added timestamps to the report listing the date range (oldest entry and newest entry) of the data discovered during the query 890 | 2. Added Site Details output if the "Add Service Details to Report" option was selected for a Report Type of TextFile 891 | 3. Corrected the Title of TextFile Report to list "System Log Parser" at the top of the page instead of "Elastic Load Balancer" 892 | 4. Enhanced the readability of the TextFile report file name 893 | 5. When using ArcGIS Server FS (File System) log parsing, requests to the root page of a service (e.g. '/') now show up as "/" under Method; In the previous version the Method was empty. 894 | 6. When using ArcGIS Server FS (File System) log parsing, the maximum time ranges in the GUI are now represented in days instead of weeks 895 | 7. Fixed the ArcGIS Server (Web) log parsing GUI where the Analyze button was still disabled if the "Use Windows Authentication" button was selected even when the Server Url and Output Directory fields were populated 896 | 897 | Build 0.8.0.0 (Prerelease) 898 | 1. A redesigned GUI (SystemLogsGUI.exe) has now been added to support the SLP log query functions of: ArcGIS Server (via the web), ArcGIS Server (via the file system), Elastic Load Balancing, Internet Information Services (via the file system). 899 | ArcGIS Monitor extensions and tasks should still refer to slp.exe in their respective configurations. 900 | 901 | Build 0.7.4.0 (Prerelease) 902 | 1. Added support for the command line version of System Log Parser (slp.exe) to accept inputs from ArcGIS Monitor in non-plaintext format 903 | 904 | Build 0.7.3.1 (Prerelease) 905 | 1. Corrected the count of Total Requests (IIS and ELB) where the total was not matching the sum of all HTTP 200, HTTP 300, HTTP 400 and HTTP 500 requests 906 | 2. Corrected console output counter totals (MapServer, ImageServer, SOAP, etc...) where some 4xx/5xx requests could be included in sums 907 | 908 | Build 0.7.3.0 (Prerelease) 909 | 1. Changed webTrack uploads to utilize the same Epoch time value for gmtMinute and gmtHour. This was done to improve the report functionality in ArcGIS Monitor. 910 | 2. Changed webTrack uploads to utilize a counterType of 'slp-web-ip' instead of 'slp-iis-ip'. This was done to improve the report functionality in ArcGIS Monitor. 911 | 3. Removed WebTrackIndex and LogEntryIndex uploads. This is now done in ArcGIS Monitor. 912 | 4. Removed the console output totals for static resources (e.g. TXT, CSS, PNG, etc...). This was done to save on optimize performance and storage space. 913 | 5. Various internal changes on how error exceptions are handled 914 | 915 | Build 0.7.2.0 (Prerelease) 916 | 1. Expanded the detection of an WMTS tile request for IIS/ELB 917 | 2. Added a count of the source being tracked in the new logentry index upload 918 | 3. Added ability to be detected by ArcGIS Monitor as being run as a task (ArcGIS Logs, ArcGIS Errors, IIS Logs, ELB Logs) or extension (ArcGIS Errors) 919 | 920 | Build 0.7.1.1 (Prerelease) 921 | 1. Fixed case-sensitivity issue with uploads into logentry for the resource and method properties 922 | 2. Fixed issue with uploads into logentry where code was not reflecting the httpstatus code of the response 923 | 924 | Build 0.7.1.0 (Prerelease) 925 | 1. Logentry and webtracks optimizations (index uploads) 926 | 2. Added floor function property to logentry uploads 927 | 3. Fixed lastmodified logfile datetime issue with IIS 928 | 4. Added optimizations for ArcGIS request uploads into logentry 929 | 930 | Build 0.7.0.0 (Prerelease) 931 | 1. New SLP engine released 932 | 933 | Build 0.6.3.1 (Prerelease) 934 | 1. Fixed an issue where some items involving a date time conversion might run into problems with international deployments. Where appropriate, SLP now utilizes the ISO 8601 standard when working with date and time-related data. 935 | 2. Fixed a minor issue in Site Details where the Max Record for a Globe Service would be blank...it should be "n/a" 936 | 937 | Build 0.6.3.0 (Prerelease) 938 | 1. Corrected a small issue as well as expanded on the summarized log data that is uploaded to ArcGIS Monitor for IIS traffic 939 | 940 | Build 0.6.2.0 (Prerelease) 941 | -------------- 942 | 1. The example URL has been changed to "https://gisserver.domain.com:6443/arcgis" instead of "http://gisserver.domain.com:6080/arcgis". This was to let users know that System Log Parser supports HTTPS. Using HTTP to connect to an ArcGIS Server host on port 6080 will still work. 943 | 2. WMTSTile support has been added for IIS log uploads into ArcGIS Monitor 944 | 3. Fixed a timing issue to help prevent the potential for log file read overlaps when uploading data into ArcGIS Monitor 945 | 946 | Build 0.6.1.3 (Prerelease) 947 | -------------- 948 | 1. When setting up System Log Parser as a ArcGIS Monitor Task, the "Test" operation would incorrectly fail with "System.InvalidOperationException: Sequence contains no elements" if there were no IIS or ArcGIS server entries found for the default time frames. 949 | This output has now been corrected to return the appropriate JSON body. 950 | 951 | Build 0.6.1.2 (Prerelease) 952 | -------------- 953 | 1. The default log verbosity setting in the Preferences.xml file has been changed from 'Warn' to 'Off' 954 | 955 | Build 0.6.1.1 (Prerelease) 956 | -------------- 957 | 1. Fixed an issue where the report could not be created and would return the message "The given path's format is not supported" due to an illegal character 958 | 2. Fixed an issue in the ErrorsOnly Analysis Type report where some messages would still get truncated if they were longer than 150 characters 959 | 960 | Build 0.6.1.0 (Prerelease) 961 | -------------- 962 | 1. Added enhancements for the integration of System Log Parser to ArcGIS Monitor for both IIS and ArcGIS payloads 963 | 2. Added a new command line version of System Log Parser called slp.exe that optimizes System Log Parser to ArcGIS Monitor communication. The existing command line features of "System Logs.exe" remain for backward compatibility. 964 | 3. Added ability to temporally show errors (errors with respect to time) with the ErrorsOnly Analysis Type report. Open the Preferences.xml file and set the false configuration to true. Restart System Log Parser. 965 | 4. Fix errors messages longer than 150 characters from being truncated in the ErrorsOnly Analysis Type report 966 | 5. Upgraded the internal EPPlus spreadsheet library to version 4.1 in order to increase the speed of creating large tables (errors/elapsed time tables) 967 | 6. Updated/simplified the command line help 968 | 7. The SystemMonitorConfiguration section in the Preferences file is a now deprecated feature and will no longer be supported with future releases of System Log Parser since newer installations of ArcGIS Monitor (v3.0+) handle the passing in of this information automatically 969 | 8. Added a "busy cursor" for ArcGIS Server and Web queries for improved application feedback 970 | 971 | Build 0.6.0.0 (Prerelease) 972 | -------------- 973 | 1. Removed Log Level from GUI. This setting had no functionality impact 974 | 2. Removed Warnings and Error worksheet from Complete report in order to improve speed 975 | 3. Warnings and Error worksheet is now its own report. This report is created when the new Analyst Type called 'ErrorsOnly' is selected 976 | 4. Redesigned Warnings and Error report to be faster, more detailed and more accurate 977 | 5. Redesigned the individual service worksheets in the Complete report into one comprehensive worksheet called "All Service Elapsed Times" which has a table containing all ArcSOC elapsed times. Users can then filter for specific service and just create the chart(s) they need. This can help reduce the Excel generation time and keeps the file size down of the report. 978 | 6. The option in the Preferences.xml file is now obsolete due to the table in the "All Service Elapsed Times" worksheet 979 | 7. Removed 'WithOverviewCharts' from the Analysis Type list. For backwards compatibility in command-line scripts, selecting this as an Analysis Type will default to 'Complete'. 980 | 8. Fixed incorrect exit code when uploading to ArcGIS Monitor when an error was encountered 981 | 9. Improved verbosity (log and screen) if System Log Parser encounters any http errors 982 | 10. System Log Parser now treats most of the supporting http calls (which help provide additional but not critical information in the report) as non-fatal. Prior builds would stop if an error was encountered. 983 | 11. Fixed issue where some supporting http calls would not honor the timeout settings of the System Log Parser Preferences.xml file 984 | 12. Added ServerURL to the Preferences.xml file. For convenience, this can be "preset" to one specific, frequently used URL 985 | 13. Fixed issues that can crash System Log Parser when some command line options were specified but their input values omitted 986 | 14. Increased display time of the command line Help window before auto closing 987 | 15. SLP IIS to SM improvements such as immediate console feedback (used internally by ArcGIS Monitor) and individual IIS request uploads 988 | 16. Fixed issue where SLP now no longer examines each service for the EnhancedSiteReport feature if SystemMonitor is selected as the ReportType. This helps speed up the SystemMonitor upload execution. 989 | 17. Added the "Requests over Time" worksheet to the 'Complete' Analysis Type which charts the requests as they occurred over either 24hrs or 1week (depending on the given timeframe) 990 | 18. Fixed an issue where the IsEnhancedSiteReport routine would honor the cancellation of a report generation 991 | 19. In the report, the ArcGIS Server Build number was added along with the listing of the ArcGIS Server Version number 992 | 20. Various internal code cleanup/optimizations 993 | 994 | Build 0.5.1.1 (Prerelease) 995 | -------------- 996 | 1. Fixed an issue with the -o flag which under certain conditions would not honor the value passed in and would open up the report even if the user passed in the command line argument of "-o false" 997 | 998 | Build 0.5.1.0 (Prerelease) 999 | -------------- 1000 | 1. Fixed issue with the new IsEnhancedSiteReport feature (e.g. the Site Service Details worksheet) where the report would not generate if this process encountered a service type of WMServer, GlobeServer or FeatureServer. IsEnhancedSiteReport property is now set to true in Preferences file for new installations...a simple workaround for this issue was to just set the IsEnhancedSiteReport property to false and restart System Log Parser. 1001 | 2. Fixed issue attempting to open the License Agreement (the E204_2014-06.docx file) from the About System Log Parser window 1002 | 3. Added the MaxRecordCount listing to the Site Service Details worksheet for services that support this property 1003 | 1004 | Build 0.5.0.0 (Prerelease) 1005 | -------------- 1006 | 1. System Log Parser is now Prerelease! Refer to the license page during installation or the E204_2014-06.docx file post-installation. 1007 | 2. Added an enchancement to the Web (IIS) report to list counts for HTTP 2XX - HTTP 5xx requests 1008 | 3. Fixed Web report labeling to not recommend checking ArcGIS Server if no log entries were found in IIS 1009 | 1010 | Build 0.4.3.0 1011 | -------------- 1012 | 1. The IsEnhancedSiteReport option is now set to true for newly created Preference files. The IsEnhancedSiteReport performs additional queries against ArcGIS Server (not against the logs) in order to determine several specific service configuration settings. This information is also displayed in the report. 1013 | 2. Full support for the ltag parameter when parsing a Web log and sending it to ArcGIS Monitor. The ltag parameter is "log tag" option which helps ArcGIS Monitor internally with the grouping the specific log data. 1014 | 1015 | Build 0.4.2.1 1016 | -------------- 1017 | 1. Adjusted the column header comments and arraingment for the Web (IIS) report 1018 | 2. Fixed small bug in the ArcGIS Server log report where Identify requests would show up as two different items if a Complete or WithOverviewCharts Analysis Type report was selected 1019 | 3. Fixed issue where the Web (IIS) report would always count the requests in the logs even if they were not a status code of 200 or 300. 1020 | W3C field of Protocol Status (sc-status) is not required but highly recommended to include as it improves this accuracy in the report. 1021 | With this version, the following are required in the IIS log in order to be parsed by System Log Parser: 1022 | i. Date (date) 1023 | ii. Time (time) 1024 | iii. Client IP Address (c-ip) 1025 | iv. User Name (cs-username) 1026 | v. Server IP Address (s-ip) 1027 | vi. URI Stem (cs-uri-stem) 1028 | vii. Time Taken (time-taken) 1029 | The following fields are recommended to also be included in the IIS log: 1030 | i. Protocol Status (sc-status) 1031 | 4. Improved the warning message clarity in text report if the ArcGIS Server logs did not contain any service requests for the given time duration 1032 | 5. Updated the System Log Parser Help 1033 | 1034 | Build 0.4.2.0 1035 | -------------- 1036 | 1. Added new feature to auto-detect the IIS log column availability matchup...no need to select the exact 15 specific set of fields in the W3C format. 1037 | * However, System Log Parser will require that at least the following fields are available in the IIS log: 1038 | i. Date (date) 1039 | ii. Time (time) 1040 | iii. Client IP Address (c-ip) 1041 | iv. User Name (cs-username) 1042 | v. Server IP Address (s-ip) 1043 | vi. URI Stem (cs-uri-stem) 1044 | vii. Time Taken (time-taken) 1045 | 2. System Log Parser does not require that the "Use local time for file naming and rollover" option In the IIS Log File Rollover section be selected 1046 | 3. Added Preference to group the map cache tiles requests in the Web log based on the Level of Detail...this can help shorten the Web (IIS) report 1047 | 4. Improved the warning message clarity in Excel report if the ArcGIS Server logs did not contain any service requests for the given time duration 1048 | 5. Filtered the warning/error worksheet in the Excel report to be more specific to just warnings and errors 1049 | 1050 | Build 0.4.1.0 1051 | -------------- 1052 | 1. Fix axis labeling for several charts from the Analysis Type of WithOverviewCharts. 1053 | 2. Added command line option to validate the ArcGIS Server logs or Web logs upload to Esri ArcGIS Monitor 3.0. With this option specified the ArcGIS Monitor Server URL is only validated...the log payload is not persisted (by design). 1054 | 3. Corrected a few minor bugs for the Web Logs payload sent to Esri ArcGIS Monitor 3.0 1055 | 4. If System Log Parser is run from the command line and encounters and error, the runtime will issue an Exit of 1 instead of 0 (which generally signifies that the operation completed successfully) 1056 | 1057 | Build 0.4.0.0 1058 | -------------- 1059 | 1. System Log Parser now has the ability to parse web logs. Currently this is limited to IIS and has the following requirements 1060 | a. The Log File is in the W3C format 1061 | b. Use local time for file naming and rollover is checked 1062 | c. The Log File format is W3C and has the following fields selected: 1063 | i. Date (date) 1064 | ii. Time (time) 1065 | iii. Client IP Address (c-ip) 1066 | iv. User Name (cs-username) 1067 | v. Server IP Address (s-ip) 1068 | vi. Server Port (s-port) 1069 | vii. Method (cs-method) 1070 | viii. URI Stem (cs-uri-stem) 1071 | ix. URI Query (cs-uri-query) 1072 | x. Protocol Status (sc-status) 1073 | xi. Protocol Substatus (sc-substatus) 1074 | xii. Win32 Status (sc-win32-status) 1075 | xiii. Time Taken (time-taken) 1076 | xiv. User Agent (cs(User-Agent)) 1077 | xv. Referer (cs(Referer)) 1078 | 2. The ArcGIS Server charts in the report (when used with the Analyst Type of WithOverviewCharts or Complete) have been reworded in the interest of simplification and clarity 1079 | 3. Support for sending the output of the ArcGIS Server logs or Web logs to Esri ArcGIS Monitor 3.0 (see ArcGIS Monitor 3.0 Help for details) 1080 | 1081 | Build 0.3.6.0 1082 | -------------- 1083 | 1. Set 'Simple' to the default Analysis Type 1084 | 2. In order to improve on log query performance, the QueryFilter used in the Preferences is overridden if 'Simple' is used an only log items with a code of 100004 (e.g. ArcSOC elapsed times) are retrieved. This can greatly shorten the query time. 1085 | 3. Removed 'WithServiceCharts' from the Analysis Type list. For backwards compatibility in command-line scripts, selecting this as an Analysis Type will default to 'Complete'. 1086 | 4. Increased application logging to aide with troubleshooting. Set the AppLogLevel tag in the Preferences to Info to increase application log verbosity. 1087 | 5. Added ArcGIS Server's LogLevel and FileAge to the report 1088 | 6. Moved the 'Max' column of the ArcSOC Elapsed time tabular view to the right so it is displayed immediately after P99 1089 | 1090 | Build 0.3.5.4 1091 | -------------- 1092 | 1. Added some initial support for application logging 1093 | 2. Fixed issue where the throughput chart (if enabled) would not render correctly and the times would be off by hour 1094 | 3. Fixed issue where clicking cancel during a log query would not stop the query 1095 | 4. Added better handling of an invalid Preferences file or invalid markup in the Preferences file 1096 | 5. Enabled chart throughput rendering (up to 1 day) if a Complete Analysis Type is selected. Prior versions needed this option to also be set in the Preferences. 1097 | 1098 | Build 0.3.5.2 1099 | -------------- 1100 | 1. Altered the internal storage format of the elapsed time floating point numbers from the logs to better account for international deployments. This now helps put the presentation format of Service Summary table on the viewing Spreadsheet program which is better equipped to honor the client environment's locale settings. 1101 | 1102 | Build 0.3.5.1 1103 | -------------- 1104 | 1. Increased the error verbosity within the GUI when attempting to parse the ArcGIS Server ArcSOC elapsed time entries within the logs 1105 | 1106 | Build 0.3.5.0 1107 | -------------- 1108 | 1. Added support to optionally create and store encrypted username and password Credential Sets within System Log Parser. Credential Sets allow you to reference a username and password by a string name that you can create through the GUI. For increased security, the username and password are encrypted before being saved. Automative scripts can then just reference the Credential Set by name. 1109 | 2. Added '103800' to the default query filter 1110 | 3. Enhanced the "Warning and Error Message Statistics" and "Map Server Request Statistics" tables so they are more informative 1111 | 4. Updated the Help to cover the two new authentication options (Windows Authentication and Credential Set Authentication) 1112 | 5. Fixed a bug that might cause System Log Parser to crash when run from the command line for certain versions of Windows 1113 | 1114 | Build 0.3.4.0 1115 | -------------- 1116 | 1. Added support to optionally use Windows Authentication to a Web Adaptor endpoint (as opposed to passing in a username and password to the 6080/6443 endpoint). This can aide deployments that do not want to store the username/password in automative scripts but does require the Web Adaptor and ArcGIS Server to be configured for Windows Authentication at the Web tier. 1117 | 2. Added support to the GUI to auto create the specified Output Directory if it does not exist 1118 | 3. Added support to the GUI to check the specified output directory for write permission by the current user before starting the log query 1119 | 4. Increased the size of the command line window to make the (command line) help easier to read. The command line help can be seen when System Log Parser is passed the "-h" option (from the command line). 1120 | 5. Fixed the command line defaults to better match the GUI defaults 1121 | 1122 | Build 0.3.3.3 1123 | -------------- 1124 | 1. Simplified the System Log Parser Help. The ability to configure the optional System Log Parser repository storage feature for use as a ArcGIS Monitor Extension is now performed through Esri Professional Services. If you are interested in this optional feature, please contact Esri Professional Services. 1125 | 1126 | Build 0.3.3.2 1127 | -------------- 1128 | 1. Installation now includes template bat script referenced in Help documentation for easily sending log information to ArcGIS Monitor as needed. 1129 | 1130 | Build 0.3.3.1 1131 | -------------- 1132 | 1. Fixed condition where selecting the Analysis Type of Simple still performed the logic for the other types but just did not add them to the report. Selecting Simple now does not do any of the other logic which can (slightly) speed up the report generation and potential avoid warnings from the tool (if warnings happened to be encountered with the other analysis types). 1133 | 2. Clarified some references of the System Log Parser tool by name in the documentation 1134 | 1135 | Build 0.3.3.0 1136 | -------------- 1137 | 1. Fixed User Overview worksheet...User Count per Unique "Service Method Machine User" chart was plotting number of user occurrences, it now plots the "service method machine user" metric correctly 1138 | 2. Separated Service chart from SMMU (Service Method Machine User) chart 1139 | 3. Made Overview charts easier to read and added verbal descriptions 1140 | 4. Enhanced the "User Count per 'Service Method Machine User'" chart so its more informative 1141 | 5. Added Count Percentages and Sum Percentages to the Service Summary table. These new columns let analyst quickly see which items are being requested the most as well as taking the most ArcSOC CPU. 1142 | 6. Add a Preferences option to plot throughput (requests/sec) along with ArcSOC Elapsed Time if an Analysis Type of "Complete" was selected and the log query time period is 2 hours or less. 1143 | 7. Fixed condition where date time data going into a ArcGIS Monitor could be off 20 minutes with respect to the local time. 1144 | 8. Fixed case where the Request Statistics worksheet could encounter problems attempting to parse the map scales 1145 | 9. Added more support (detailed documentation) for sending captured ArcGIS Server log data to a ArcGIS Monitor repository 1146 | 10. Increased the "Log Messages per Query Page" default setting from 1000 to 9999 in order to help increase the performance of heavy log queries. Instead of using many smaller queries to generate a report System Log Parser now uses fewer, larger queries. 1147 | 1148 | Build 0.3.2.0 1149 | -------------- 1150 | 1. Added some support to send captured ArcGIS Server log data to a ArcGIS Monitor repository 1151 | 2. Fixed datetime representation of log data such that when viewed in ArcGIS Monitor it is in the timezone of the machine running System Log Parser 1152 | 3. Added '0' and '8000' to the default query filter 1153 | 1154 | Build 0.3.1.0 1155 | -------------- 1156 | 1. Added a human readable DateTime column to the individual service charts that's dynamically formatted based on the given timespan and specifically used for making the charts easier to read 1157 | 2. Added an additional Statistics page in the report called Request Statistics that lists information on the top 3 most requested map sizes and map scales. Note: currently, this only reports on REST export map and WMS requests. 1158 | 3. Allowed System Log Parser Installer to automatically upgrade itself over an older version 1159 | 4. Added '80014' to the default query filter 1160 | 5. Fixed sizing on Back To Service Summary link 1161 | 1162 | Build 0.3.0.0 1163 | -------------- 1164 | 1. Added log error analysis which displays grouped error messages and a numeric count of their occurrence 1165 | 2. Added the Machine Overview chart which breaks down the activity of serviced requests by machine (and cluster) 1166 | 3. Added support for a Preferences file 1167 | a) Added the option to specify a default value (in seconds) for the request timeout. This had a fixed value of 100 seconds which was encountered for some environments and caused the log query to stop prematurely. 1168 | b) Added the option to specify a default value (in minutes) for the token expiration. The default ArcGIS Server value was encountered for some environments and could also cause the log query to stop prematurely. 1169 | c) Added the option to specify the default column to sort by in the main Service table (e.g. on the Service Summary worksheet) 1170 | d) Added the option to customize a specific list of ArcGIS Server codes to search when querying the logs 1171 | e) Note: There are additional options in the Preferences file, but these other settings are currently unsupported 1172 | 4. Changed default Analysis Type to WithOverviewChart to improve the generation speed of default report 1173 | 5. Fixed an issue where "Add/Remove Programs" in Windows would show the incorrect installed version of System Log Parser 1174 | 6. Corrected Starttime initialization value. Previous version based Starttime on "now", but it should be based on "now - the specified Endtime". 1175 | 7. Added an "About System Log Parser" button (the blue exclamation point) on the graphical user interface to easily find the current version of the program 1176 | 1177 | Build 0.2.4.0 1178 | -------------- 1179 | 1. In the interest of improved scalability and stability, the individual service chart worksheet naming convention was changed to use a GUID instead of the "partial service name" and "occurance number". 1180 | 2. Added ArcGIS Server Version to one of the informational items stored at the top of report 1181 | 3. Added the option of a text file to the list of report outputs 1182 | 4. Increased the maximum number of allowable service items to report on 1183 | 5. Added an additional hyperlink pointing back to the Service Summary above the chart of each individual service worksheet 1184 | 1185 | Build 0.2.3.0 1186 | -------------- 1187 | 1. Fixed issue where System Log Parser could not parse certain log query results from deployments of where the locale of ArcGIS Server was not set to 'US'. 1188 | 2. Added option to adjust the Log Messages per Query Page. 1189 | 3. Added weighted average column at the end of the Service/Source summary which calculates the weighted average of the average ArcSOC processing time. 1190 | 4. Renamed some of the connection and log query statistics section headers (e.g. Average Log Inquiry Time Total Inquiry Time), so these items are less likely to be confused information pertaining to the Service/Source summary items (e.g. Count, Avg, Stdev, etc...). 1191 | 5. Split the Overview worksheet into two parts for easier viewing: Services Overview and Users Overview. 1192 | 6. Renamed and clarified the chart titles for the Services Overview and Users Overview charts. 1193 | 7. Added the 'Request Count per "Service"' chart to the Services Overview page which lists the most popular service based on all the Methods that users requested. Alternatively, the 'Request Count per "Service Method Machine User"' chart lists the most popular service based on the most hits to each Service-Method-Machine-User item. 1194 | 8. Increased the verbosity of the error message if System Log Parser was not able to deserialize the ArcGIS Server log contents. 1195 | 1196 | Build 0.2.2.0 1197 | -------------- 1198 | Initial Public Release 1199 | 1200 | -------------------------------------------------------------------------------- /LICENSE.txt: -------------------------------------------------------------------------------- 1 | Attribution-NonCommercial-ShareAlike 4.0 International 2 | 3 | ======================================================================= 4 | 5 | Creative Commons Corporation ("Creative Commons") is not a law firm and 6 | does not provide legal services or legal advice. Distribution of 7 | Creative Commons public licenses does not create a lawyer-client or 8 | other relationship. Creative Commons makes its licenses and related 9 | information available on an "as-is" basis. Creative Commons gives no 10 | warranties regarding its licenses, any material licensed under their 11 | terms and conditions, or any related information. Creative Commons 12 | disclaims all liability for damages resulting from their use to the 13 | fullest extent possible. 14 | 15 | Using Creative Commons Public Licenses 16 | 17 | Creative Commons public licenses provide a standard set of terms and 18 | conditions that creators and other rights holders may use to share 19 | original works of authorship and other material subject to copyright 20 | and certain other rights specified in the public license below. The 21 | following considerations are for informational purposes only, are not 22 | exhaustive, and do not form part of our licenses. 23 | 24 | Considerations for licensors: Our public licenses are 25 | intended for use by those authorized to give the public 26 | permission to use material in ways otherwise restricted by 27 | copyright and certain other rights. Our licenses are 28 | irrevocable. Licensors should read and understand the terms 29 | and conditions of the license they choose before applying it. 30 | Licensors should also secure all rights necessary before 31 | applying our licenses so that the public can reuse the 32 | material as expected. Licensors should clearly mark any 33 | material not subject to the license. This includes other CC- 34 | licensed material, or material used under an exception or 35 | limitation to copyright. More considerations for licensors: 36 | wiki.creativecommons.org/Considerations_for_licensors 37 | 38 | Considerations for the public: By using one of our public 39 | licenses, a licensor grants the public permission to use the 40 | licensed material under specified terms and conditions. If 41 | the licensor's permission is not necessary for any reason--for 42 | example, because of any applicable exception or limitation to 43 | copyright--then that use is not regulated by the license. Our 44 | licenses grant only permissions under copyright and certain 45 | other rights that a licensor has authority to grant. Use of 46 | the licensed material may still be restricted for other 47 | reasons, including because others have copyright or other 48 | rights in the material. A licensor may make special requests, 49 | such as asking that all changes be marked or described. 50 | Although not required by our licenses, you are encouraged to 51 | respect those requests where reasonable. More considerations 52 | for the public: 53 | wiki.creativecommons.org/Considerations_for_licensees 54 | 55 | ======================================================================= 56 | 57 | Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International 58 | Public License 59 | 60 | By exercising the Licensed Rights (defined below), You accept and agree 61 | to be bound by the terms and conditions of this Creative Commons 62 | Attribution-NonCommercial-ShareAlike 4.0 International Public License 63 | ("Public License"). To the extent this Public License may be 64 | interpreted as a contract, You are granted the Licensed Rights in 65 | consideration of Your acceptance of these terms and conditions, and the 66 | Licensor grants You such rights in consideration of benefits the 67 | Licensor receives from making the Licensed Material available under 68 | these terms and conditions. 69 | 70 | 71 | Section 1 -- Definitions. 72 | 73 | a. Adapted Material means material subject to Copyright and Similar 74 | Rights that is derived from or based upon the Licensed Material 75 | and in which the Licensed Material is translated, altered, 76 | arranged, transformed, or otherwise modified in a manner requiring 77 | permission under the Copyright and Similar Rights held by the 78 | Licensor. For purposes of this Public License, where the Licensed 79 | Material is a musical work, performance, or sound recording, 80 | Adapted Material is always produced where the Licensed Material is 81 | synched in timed relation with a moving image. 82 | 83 | b. Adapter's License means the license You apply to Your Copyright 84 | and Similar Rights in Your contributions to Adapted Material in 85 | accordance with the terms and conditions of this Public License. 86 | 87 | c. BY-NC-SA Compatible License means a license listed at 88 | creativecommons.org/compatiblelicenses, approved by Creative 89 | Commons as essentially the equivalent of this Public License. 90 | 91 | d. Copyright and Similar Rights means copyright and/or similar rights 92 | closely related to copyright including, without limitation, 93 | performance, broadcast, sound recording, and Sui Generis Database 94 | Rights, without regard to how the rights are labeled or 95 | categorized. For purposes of this Public License, the rights 96 | specified in Section 2(b)(1)-(2) are not Copyright and Similar 97 | Rights. 98 | 99 | e. Effective Technological Measures means those measures that, in the 100 | absence of proper authority, may not be circumvented under laws 101 | fulfilling obligations under Article 11 of the WIPO Copyright 102 | Treaty adopted on December 20, 1996, and/or similar international 103 | agreements. 104 | 105 | f. Exceptions and Limitations means fair use, fair dealing, and/or 106 | any other exception or limitation to Copyright and Similar Rights 107 | that applies to Your use of the Licensed Material. 108 | 109 | g. License Elements means the license attributes listed in the name 110 | of a Creative Commons Public License. The License Elements of this 111 | Public License are Attribution, NonCommercial, and ShareAlike. 112 | 113 | h. Licensed Material means the artistic or literary work, database, 114 | or other material to which the Licensor applied this Public 115 | License. 116 | 117 | i. Licensed Rights means the rights granted to You subject to the 118 | terms and conditions of this Public License, which are limited to 119 | all Copyright and Similar Rights that apply to Your use of the 120 | Licensed Material and that the Licensor has authority to license. 121 | 122 | j. Licensor means the individual(s) or entity(ies) granting rights 123 | under this Public License. 124 | 125 | k. NonCommercial means not primarily intended for or directed towards 126 | commercial advantage or monetary compensation. For purposes of 127 | this Public License, the exchange of the Licensed Material for 128 | other material subject to Copyright and Similar Rights by digital 129 | file-sharing or similar means is NonCommercial provided there is 130 | no payment of monetary compensation in connection with the 131 | exchange. 132 | 133 | l. Share means to provide material to the public by any means or 134 | process that requires permission under the Licensed Rights, such 135 | as reproduction, public display, public performance, distribution, 136 | dissemination, communication, or importation, and to make material 137 | available to the public including in ways that members of the 138 | public may access the material from a place and at a time 139 | individually chosen by them. 140 | 141 | m. Sui Generis Database Rights means rights other than copyright 142 | resulting from Directive 96/9/EC of the European Parliament and of 143 | the Council of 11 March 1996 on the legal protection of databases, 144 | as amended and/or succeeded, as well as other essentially 145 | equivalent rights anywhere in the world. 146 | 147 | n. You means the individual or entity exercising the Licensed Rights 148 | under this Public License. Your has a corresponding meaning. 149 | 150 | 151 | Section 2 -- Scope. 152 | 153 | a. License grant. 154 | 155 | 1. Subject to the terms and conditions of this Public License, 156 | the Licensor hereby grants You a worldwide, royalty-free, 157 | non-sublicensable, non-exclusive, irrevocable license to 158 | exercise the Licensed Rights in the Licensed Material to: 159 | 160 | a. reproduce and Share the Licensed Material, in whole or 161 | in part, for NonCommercial purposes only; and 162 | 163 | b. produce, reproduce, and Share Adapted Material for 164 | NonCommercial purposes only. 165 | 166 | 2. Exceptions and Limitations. For the avoidance of doubt, where 167 | Exceptions and Limitations apply to Your use, this Public 168 | License does not apply, and You do not need to comply with 169 | its terms and conditions. 170 | 171 | 3. Term. The term of this Public License is specified in Section 172 | 6(a). 173 | 174 | 4. Media and formats; technical modifications allowed. The 175 | Licensor authorizes You to exercise the Licensed Rights in 176 | all media and formats whether now known or hereafter created, 177 | and to make technical modifications necessary to do so. The 178 | Licensor waives and/or agrees not to assert any right or 179 | authority to forbid You from making technical modifications 180 | necessary to exercise the Licensed Rights, including 181 | technical modifications necessary to circumvent Effective 182 | Technological Measures. For purposes of this Public License, 183 | simply making modifications authorized by this Section 2(a) 184 | (4) never produces Adapted Material. 185 | 186 | 5. Downstream recipients. 187 | 188 | a. Offer from the Licensor -- Licensed Material. Every 189 | recipient of the Licensed Material automatically 190 | receives an offer from the Licensor to exercise the 191 | Licensed Rights under the terms and conditions of this 192 | Public License. 193 | 194 | b. Additional offer from the Licensor -- Adapted Material. 195 | Every recipient of Adapted Material from You 196 | automatically receives an offer from the Licensor to 197 | exercise the Licensed Rights in the Adapted Material 198 | under the conditions of the Adapter's License You apply. 199 | 200 | c. No downstream restrictions. You may not offer or impose 201 | any additional or different terms or conditions on, or 202 | apply any Effective Technological Measures to, the 203 | Licensed Material if doing so restricts exercise of the 204 | Licensed Rights by any recipient of the Licensed 205 | Material. 206 | 207 | 6. No endorsement. Nothing in this Public License constitutes or 208 | may be construed as permission to assert or imply that You 209 | are, or that Your use of the Licensed Material is, connected 210 | with, or sponsored, endorsed, or granted official status by, 211 | the Licensor or others designated to receive attribution as 212 | provided in Section 3(a)(1)(A)(i). 213 | 214 | b. Other rights. 215 | 216 | 1. Moral rights, such as the right of integrity, are not 217 | licensed under this Public License, nor are publicity, 218 | privacy, and/or other similar personality rights; however, to 219 | the extent possible, the Licensor waives and/or agrees not to 220 | assert any such rights held by the Licensor to the limited 221 | extent necessary to allow You to exercise the Licensed 222 | Rights, but not otherwise. 223 | 224 | 2. Patent and trademark rights are not licensed under this 225 | Public License. 226 | 227 | 3. To the extent possible, the Licensor waives any right to 228 | collect royalties from You for the exercise of the Licensed 229 | Rights, whether directly or through a collecting society 230 | under any voluntary or waivable statutory or compulsory 231 | licensing scheme. In all other cases the Licensor expressly 232 | reserves any right to collect such royalties, including when 233 | the Licensed Material is used other than for NonCommercial 234 | purposes. 235 | 236 | 237 | Section 3 -- License Conditions. 238 | 239 | Your exercise of the Licensed Rights is expressly made subject to the 240 | following conditions. 241 | 242 | a. Attribution. 243 | 244 | 1. If You Share the Licensed Material (including in modified 245 | form), You must: 246 | 247 | a. retain the following if it is supplied by the Licensor 248 | with the Licensed Material: 249 | 250 | i. identification of the creator(s) of the Licensed 251 | Material and any others designated to receive 252 | attribution, in any reasonable manner requested by 253 | the Licensor (including by pseudonym if 254 | designated); 255 | 256 | ii. a copyright notice; 257 | 258 | iii. a notice that refers to this Public License; 259 | 260 | iv. a notice that refers to the disclaimer of 261 | warranties; 262 | 263 | v. a URI or hyperlink to the Licensed Material to the 264 | extent reasonably practicable; 265 | 266 | b. indicate if You modified the Licensed Material and 267 | retain an indication of any previous modifications; and 268 | 269 | c. indicate the Licensed Material is licensed under this 270 | Public License, and include the text of, or the URI or 271 | hyperlink to, this Public License. 272 | 273 | 2. You may satisfy the conditions in Section 3(a)(1) in any 274 | reasonable manner based on the medium, means, and context in 275 | which You Share the Licensed Material. For example, it may be 276 | reasonable to satisfy the conditions by providing a URI or 277 | hyperlink to a resource that includes the required 278 | information. 279 | 3. If requested by the Licensor, You must remove any of the 280 | information required by Section 3(a)(1)(A) to the extent 281 | reasonably practicable. 282 | 283 | b. ShareAlike. 284 | 285 | In addition to the conditions in Section 3(a), if You Share 286 | Adapted Material You produce, the following conditions also apply. 287 | 288 | 1. The Adapter's License You apply must be a Creative Commons 289 | license with the same License Elements, this version or 290 | later, or a BY-NC-SA Compatible License. 291 | 292 | 2. You must include the text of, or the URI or hyperlink to, the 293 | Adapter's License You apply. You may satisfy this condition 294 | in any reasonable manner based on the medium, means, and 295 | context in which You Share Adapted Material. 296 | 297 | 3. You may not offer or impose any additional or different terms 298 | or conditions on, or apply any Effective Technological 299 | Measures to, Adapted Material that restrict exercise of the 300 | rights granted under the Adapter's License You apply. 301 | 302 | 303 | Section 4 -- Sui Generis Database Rights. 304 | 305 | Where the Licensed Rights include Sui Generis Database Rights that 306 | apply to Your use of the Licensed Material: 307 | 308 | a. for the avoidance of doubt, Section 2(a)(1) grants You the right 309 | to extract, reuse, reproduce, and Share all or a substantial 310 | portion of the contents of the database for NonCommercial purposes 311 | only; 312 | 313 | b. if You include all or a substantial portion of the database 314 | contents in a database in which You have Sui Generis Database 315 | Rights, then the database in which You have Sui Generis Database 316 | Rights (but not its individual contents) is Adapted Material, 317 | including for purposes of Section 3(b); and 318 | 319 | c. You must comply with the conditions in Section 3(a) if You Share 320 | all or a substantial portion of the contents of the database. 321 | 322 | For the avoidance of doubt, this Section 4 supplements and does not 323 | replace Your obligations under this Public License where the Licensed 324 | Rights include other Copyright and Similar Rights. 325 | 326 | 327 | Section 5 -- Disclaimer of Warranties and Limitation of Liability. 328 | 329 | a. UNLESS OTHERWISE SEPARATELY UNDERTAKEN BY THE LICENSOR, TO THE 330 | EXTENT POSSIBLE, THE LICENSOR OFFERS THE LICENSED MATERIAL AS-IS 331 | AND AS-AVAILABLE, AND MAKES NO REPRESENTATIONS OR WARRANTIES OF 332 | ANY KIND CONCERNING THE LICENSED MATERIAL, WHETHER EXPRESS, 333 | IMPLIED, STATUTORY, OR OTHER. THIS INCLUDES, WITHOUT LIMITATION, 334 | WARRANTIES OF TITLE, MERCHANTABILITY, FITNESS FOR A PARTICULAR 335 | PURPOSE, NON-INFRINGEMENT, ABSENCE OF LATENT OR OTHER DEFECTS, 336 | ACCURACY, OR THE PRESENCE OR ABSENCE OF ERRORS, WHETHER OR NOT 337 | KNOWN OR DISCOVERABLE. WHERE DISCLAIMERS OF WARRANTIES ARE NOT 338 | ALLOWED IN FULL OR IN PART, THIS DISCLAIMER MAY NOT APPLY TO YOU. 339 | 340 | b. TO THE EXTENT POSSIBLE, IN NO EVENT WILL THE LICENSOR BE LIABLE 341 | TO YOU ON ANY LEGAL THEORY (INCLUDING, WITHOUT LIMITATION, 342 | NEGLIGENCE) OR OTHERWISE FOR ANY DIRECT, SPECIAL, INDIRECT, 343 | INCIDENTAL, CONSEQUENTIAL, PUNITIVE, EXEMPLARY, OR OTHER LOSSES, 344 | COSTS, EXPENSES, OR DAMAGES ARISING OUT OF THIS PUBLIC LICENSE OR 345 | USE OF THE LICENSED MATERIAL, EVEN IF THE LICENSOR HAS BEEN 346 | ADVISED OF THE POSSIBILITY OF SUCH LOSSES, COSTS, EXPENSES, OR 347 | DAMAGES. WHERE A LIMITATION OF LIABILITY IS NOT ALLOWED IN FULL OR 348 | IN PART, THIS LIMITATION MAY NOT APPLY TO YOU. 349 | 350 | c. The disclaimer of warranties and limitation of liability provided 351 | above shall be interpreted in a manner that, to the extent 352 | possible, most closely approximates an absolute disclaimer and 353 | waiver of all liability. 354 | 355 | 356 | Section 6 -- Term and Termination. 357 | 358 | a. This Public License applies for the term of the Copyright and 359 | Similar Rights licensed here. However, if You fail to comply with 360 | this Public License, then Your rights under this Public License 361 | terminate automatically. 362 | 363 | b. Where Your right to use the Licensed Material has terminated under 364 | Section 6(a), it reinstates: 365 | 366 | 1. automatically as of the date the violation is cured, provided 367 | it is cured within 30 days of Your discovery of the 368 | violation; or 369 | 370 | 2. upon express reinstatement by the Licensor. 371 | 372 | For the avoidance of doubt, this Section 6(b) does not affect any 373 | right the Licensor may have to seek remedies for Your violations 374 | of this Public License. 375 | 376 | c. For the avoidance of doubt, the Licensor may also offer the 377 | Licensed Material under separate terms or conditions or stop 378 | distributing the Licensed Material at any time; however, doing so 379 | will not terminate this Public License. 380 | 381 | d. Sections 1, 5, 6, 7, and 8 survive termination of this Public 382 | License. 383 | 384 | 385 | Section 7 -- Other Terms and Conditions. 386 | 387 | a. The Licensor shall not be bound by any additional or different 388 | terms or conditions communicated by You unless expressly agreed. 389 | 390 | b. Any arrangements, understandings, or agreements regarding the 391 | Licensed Material not stated herein are separate from and 392 | independent of the terms and conditions of this Public License. 393 | 394 | 395 | Section 8 -- Interpretation. 396 | 397 | a. For the avoidance of doubt, this Public License does not, and 398 | shall not be interpreted to, reduce, limit, restrict, or impose 399 | conditions on any use of the Licensed Material that could lawfully 400 | be made without permission under this Public License. 401 | 402 | b. To the extent possible, if any provision of this Public License is 403 | deemed unenforceable, it shall be automatically reformed to the 404 | minimum extent necessary to make it enforceable. If the provision 405 | cannot be reformed, it shall be severed from this Public License 406 | without affecting the enforceability of the remaining terms and 407 | conditions. 408 | 409 | c. No term or condition of this Public License will be waived and no 410 | failure to comply consented to unless expressly agreed to by the 411 | Licensor. 412 | 413 | d. Nothing in this Public License constitutes or may be interpreted 414 | as a limitation upon, or waiver of, any privileges and immunities 415 | that apply to the Licensor or You, including from the legal 416 | processes of any jurisdiction or authority. 417 | 418 | ======================================================================= 419 | 420 | Creative Commons is not a party to its public 421 | licenses. Notwithstanding, Creative Commons may elect to apply one of 422 | its public licenses to material it publishes and in those instances 423 | will be considered the “Licensor.” The text of the Creative Commons 424 | public licenses is dedicated to the public domain under the CC0 Public 425 | Domain Dedication. Except for the limited purpose of indicating that 426 | material is shared under a Creative Commons public license or as 427 | otherwise permitted by the Creative Commons policies published at 428 | creativecommons.org/policies, Creative Commons does not authorize the 429 | use of the trademark "Creative Commons" or any other trademark or logo 430 | of Creative Commons without its prior written consent including, 431 | without limitation, in connection with any unauthorized modifications 432 | to any of its public licenses or any other arrangements, 433 | understandings, or agreements concerning use of licensed material. For 434 | the avoidance of doubt, this paragraph does not form part of the 435 | public licenses. 436 | 437 | Creative Commons may be contacted at creativecommons.org. 438 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # System Log Parser 2 | ## Version 0.13.2.0 3 | ### Date: 2025/01/06 4 | 5 | ------------------------------- 6 | ### Download Latest Release 7 | [**System Log Parser Download**](../../raw/main/binaries/latest/SystemLogParser.zip) 8 | 9 | ![System Log Parser GUI](/assets/images/SystemLogsGUI_ArcGISServer_web_small.png) 10 | 11 | ------------------------------- 12 | #### Description 13 | System Log Parser is an ArcGIS for Server (10.1+) log query and analyzer tool to 14 | help you quickly quantify the GIS usage in your deployment. 15 | When run, it connects to an ArcGIS for Server instance on port 443/6443/6080/80 as a publisher 16 | (or an administrator), retrieves the logs from a time duration (specified as an input), 17 | analyzes the information then produces a spreadsheet version of the data that 18 | summarizes the service statistics. 19 | However, System Log Parser can also read access logs from other sources such as: 20 | Microsoft IIS, Apache Tomcat, Amazon ELB/ALB, Amazon CloudFront and Microsoft Azure. 21 | 22 | #### System Requirements 23 | - 64bit Windows Operating System: 24 | - Windows (Workstation): 7, 8*, 8.1**, 10**, 11** 25 | - Windows (Server): 2008, 2012, 2016, 2019, 2022 26 | - Processor: AMD64/Intel64 27 | - RAM: 8GB 28 | - Disk Space: 2GB free when using the new Optimized Analysis Type 29 | - Security Protocol***: TLS 1.0, TLS 1.1, TLS 1.2, or TLS 1.3 30 | - Microsoft .NET Framework 4.8 (Full) 31 | - Publisher (or administrative) access to ArcGIS Server's REST API Admin endpoint in order to query the logs 32 | - ArcGIS Server log level set to FINE (from within the ArcGIS Server Manager) before using this tool 33 | - A deployment running ArcGIS Server 10.1 or higher 34 | - If performing Web (IIS) log analysis, it is recommended to have the Log File Rollover Schedule set to Hourly or Daily (from the Internet Information Services Manager) 35 | 36 | - *If using Windows 8, .NET Framework 3.5 (in addition to 4.8) must be enabled per Microsoft documentation 37 | - **If using Windows 8.1 or greater, the .NET Framework 3.5 (in addition to 4.8) must be enabled per Microsoft documentation 38 | - ***Not relevant if logs are consumed directly from the file system (e.g. reading the local disk or via CIFS share) 39 | 40 | #### System Recommendations: 41 | - Microsoft Excel 2010 or higher (or appropriate xlsx viewer) 42 | - RAM: 16GB 43 | - Disk Space: 6GB free when using the new Optimized Analysis Type 44 | - Security Protocol*: TLS 1.2 or TLS 1.3 45 | - If performing Microsoft IIS log analysis, it is recommended to have the Log File Rollover Schedule set to Hourly (from the Internet Information Services Manager); Hourly creates more, smaller files but allows for a finer grain search 46 | - If querying through the web, an user with administrative access is recommended as they can gather details on all service types 47 | - *Not relevant if logs are consumed directly from the file system (e.g. reading the local disk or via CIFS share) 48 | 49 | ------------------------------- 50 | #### How To Download System Log Parser 51 | - Download the latest release here: 52 | [**System Log Parser Download**](../../raw/main/binaries/latest/SystemLogParser.zip) 53 | and unzip it to your local workstation 54 | - System Log Parser does not include an installer. 55 | 56 | ------------------------------- 57 | #### How To Use System Log Parser 58 | Once the System Log Parser is downloaded and unzipped, launch the GUI by double-clicking on SystemLogsGUI.exe 59 | The most common scenario for using System Log Parser is to point it at an ArcGIS Server to analyze the logs: 60 | - Once the this has launched, select the ArcGIS Server (Web) tab from the top 61 | - Fill out the necessary information like Server URL and Authentication details 62 | - Click Analyze 63 | - System Log Parser will connect to ArcGIS Server to retrieve and analyze the logs, then produce a spreadsheet report 64 | - The log retrieval make take a few moments, depending on the size of the ArcGIS Server logs 65 | - The generated report can be used to help quantify the GIS usage of the Site by providing a breakdown of what services users are requesting and how long they are waiting for the response 66 | 67 | Note: the ArcGIS Server LogLevel must be set to Fine when performing ArcGIS Server (Web) analysis 68 | 69 | ------------------------------- 70 | #### Esri Community Articles on System Log Parser for Specific Scenarios 71 | - [ArcGIS Enterprise Analysis with System Log Parser's Optimized Analysis Type](https://community.esri.com/t5/implementing-arcgis-blog/arcgis-enterprise-analysis-with-system-log-parser/ba-p/1189005) 72 | - [Automating System Log Parser from the Windows Command Line](https://community.esri.com/t5/implementing-arcgis-blog/automating-system-log-parser-from-the-windows/ba-p/1195294) 73 | - [ArcGIS Enterprise Analysis with System Log Parser's ServiceDetails Analysis Type](https://community.esri.com/t5/implementing-arcgis-blog/arcgis-enterprise-analysis-with-system-log-parser/ba-p/1198115) 74 | - [System Log Parser - Statistics and Service Optimization](https://community.esri.com/t5/implementing-arcgis-blog/system-log-parser-statistics-and-service/ba-p/886389) 75 | - [ArcGIS Server Tuning and Optimization with System Log Parser 76 | ](https://community.esri.com/t5/implementing-arcgis-blog/arcgis-server-tuning-and-optimization-with-system/ba-p/886361) 77 | 78 | ------------------------------- 79 | ##### License 80 | System Log Parser is released under: CC BY-NC-SA 4.0 81 | - [https://creativecommons.org/licenses/by-nc-sa/4.0/](https://creativecommons.org/licenses/by-nc-sa/4.0/) 82 | - [https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode.en](https://creativecommons.org/licenses/by-nc-sa/4.0/legalcode.en) 83 | 84 | 85 | ------------------------------- 86 | ##### Support Status 87 | System Log Parser is not a supported tool 88 | 89 | ------------------------------- 90 | ##### ChangeLog 91 | [ChangeLog details](https://github.com/ArcGIS/SystemLogParser/blob/main/CHANGELOG.md) 92 | -------------------------------------------------------------------------------- /assets/images/SystemLogsGUI_ArcGISServer_web.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Esri/SystemLogParser/6642f4aef3777837f0acc150294dd0597331d76a/assets/images/SystemLogsGUI_ArcGISServer_web.png -------------------------------------------------------------------------------- /assets/images/SystemLogsGUI_ArcGISServer_web_small.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Esri/SystemLogParser/6642f4aef3777837f0acc150294dd0597331d76a/assets/images/SystemLogsGUI_ArcGISServer_web_small.png -------------------------------------------------------------------------------- /assets/images/by-nc-sa.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Esri/SystemLogParser/6642f4aef3777837f0acc150294dd0597331d76a/assets/images/by-nc-sa.png -------------------------------------------------------------------------------- /binaries/latest/SystemLogParser.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Esri/SystemLogParser/6642f4aef3777837f0acc150294dd0597331d76a/binaries/latest/SystemLogParser.zip -------------------------------------------------------------------------------- /binaries/v0.13.0.0/SystemLogParser.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Esri/SystemLogParser/6642f4aef3777837f0acc150294dd0597331d76a/binaries/v0.13.0.0/SystemLogParser.zip -------------------------------------------------------------------------------- /binaries/v0.13.0.1/SystemLogParser.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Esri/SystemLogParser/6642f4aef3777837f0acc150294dd0597331d76a/binaries/v0.13.0.1/SystemLogParser.zip -------------------------------------------------------------------------------- /binaries/v0.13.1.0/SystemLogParser.zip: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Esri/SystemLogParser/6642f4aef3777837f0acc150294dd0597331d76a/binaries/v0.13.1.0/SystemLogParser.zip --------------------------------------------------------------------------------