├── MANIFEST.in ├── requirements.txt ├── sophos_central_api_connector ├── docs │ ├── local_sites.md │ ├── inventory.md │ ├── intelix_configuration.md │ ├── misp_configuration.md │ ├── get_roles.md │ ├── get_admins.md │ ├── get_firewalls.md │ ├── get_firewall_groups.md │ ├── sophos_configuration.md │ ├── splunk_configuration.md │ ├── alerts.md │ ├── intelix.md │ └── ioc_hunter.md ├── config │ ├── misp_config.ini │ ├── sophos_config.ini │ ├── splunk_config.ini │ ├── sophos_central_api_config.py │ └── intelix_config.ini ├── sophos_central_api_delete_data.py ├── get_firewalls.py ├── get_admins.py ├── get_firewall_groups.py ├── get_roles.py ├── queries │ ├── xdr_queries │ │ └── xdr_ioc_hunter.sql │ └── live_discover_queries │ │ └── ld_ioc_hunter.sql ├── sophos_central_api_awssecrets.py ├── sophos_central_api_tenants.py ├── sophos_central_api_auth.py ├── sophos_central_api_get_data.py ├── sophos_central_api_output.py ├── sophos_central_hec_splunk.py ├── sophos_central_api_intelix.py ├── ioc_hunter.py ├── sophos_central_api_connector_utils.py ├── sophos_central_api_polling.py └── sophos_central_api_live_discover.py ├── .gitignore ├── setup.py └── README.md /MANIFEST.in: -------------------------------------------------------------------------------- 1 | recursive-include sophos_central_api_connector *.ini *.py *.md *.sqlite -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | boto3>=1.17.105 2 | botocore>=1.20.105 3 | certifi>=2021.5.30 4 | chardet>=4.0.0 5 | configparser>=5.0.2 6 | docutils>=0.17.1 7 | intelix>=0.1.2 8 | jmespath>=0.10.0 9 | python-dateutil>=2.8.1 10 | requests>=2.25.1 11 | s3transfer>=0.4.2 12 | six>=1.16.0 13 | urllib3>=1.26.6 14 | -------------------------------------------------------------------------------- /sophos_central_api_connector/docs/local_sites.md: -------------------------------------------------------------------------------- 1 | ## Local Site Information 2 | 3 | You can review the sites which have been added to Global Settings > Website Management using the API. 4 | 5 | As with previous abilities you can pull the sites for a specific tenant or for all tenants for review. It will automatically reference 6 | the category integer to a human readable category. This can be output to all the available options. 7 | ```python 8 | python sophos_central_main.py --auth --get local-sites --output 9 | ``` -------------------------------------------------------------------------------- /sophos_central_api_connector/config/misp_config.ini: -------------------------------------------------------------------------------- 1 | #This is the config file for accessing your MISP instance 2 | 3 | #The awsSecret section allows you to specify your AWS Secrets Manager 4 | #information to utilise the safe keeping of your API key 5 | #The api_key are the names you use for the 'Secret Key' 6 | #when you created a new secret to store the MISP API key 7 | [aws] 8 | secret_name: 9 | region_name: 10 | api_key: 11 | 12 | #Add the url for your MISP instance 13 | [url] 14 | misp_instance: https:///attributes/restSearch -------------------------------------------------------------------------------- /sophos_central_api_connector/docs/inventory.md: -------------------------------------------------------------------------------- 1 | ## Endpoint Information 2 | 3 | Gathering the inventory information can be done for all of your tenants or one specific tenant. There are various methods on how this data can be presented. The output methods are stdout, json or sending the data to Splunk. More detailed example are covered under the Advanced Usage section. 4 | 5 | There are various products which can utilise data from stdout such as running a script in Splunk and indexing the data dynamically. Also this can be used as a response in automation to provide further details on a systems health etc. 6 | 7 | The syntax to use when requesting to get inventory is the following: 8 | ```python 9 | python sophos_central_main.py --auth --get inventory --output 10 | ``` -------------------------------------------------------------------------------- /sophos_central_api_connector/docs/intelix_configuration.md: -------------------------------------------------------------------------------- 1 | ## SophosLabs Intelix Configuration 2 | > ### **Important!** 3 | > Whilst you are able to set static API credentials in this configuration we strongly advise that this is only done for testing purposes. 4 | > Where possible use AWS Secrets Manager to store your credential id and token 5 | > Please reference the authentication section under advanced usage to use the correct parameter 6 | 7 | The AWS secret credentials follows the same format as the sophos_config.ini. 8 | 9 | There is an additional section in regards to the IP lookup categorisation. To align with the URL riskLevel we have provided the ability to set the IP categories against the risk level. The values already set 10 | and should be reviewed and amended to prevent deleting sites incorrectly from the local sites settings in Central. -------------------------------------------------------------------------------- /sophos_central_api_connector/docs/misp_configuration.md: -------------------------------------------------------------------------------- 1 | ## MISP Configuration 2 | 3 | This configuration file is to provide details for accessing your API Key for MISP. 4 | 5 | Details on obtaining the correct URL and API Key please follow the documentation: 6 | * [MISP URL](https://www.circl.lu/doc/misp/automation/#automation-url) 7 | * [MISP API Key](https://www.circl.lu/doc/misp/automation/#automation-key) 8 | 9 | We are moving away from you having the ability to place your credentials in the configuration in plain text. 10 | 11 | You can access your AWS secrets by configuring your details as below: 12 | - **secret_name:** \ 13 | - **region_name:** \ 14 | - **api_key:** \ 15 | 16 | If no details are entered here, when running the tool you will be prompted for your details using `getpass` 17 | 18 | Additionally set your MISP instance URL for where you will obtain your attributes with `/attributes/restSearch` 19 | -------------------------------------------------------------------------------- /sophos_central_api_connector/docs/get_roles.md: -------------------------------------------------------------------------------- 1 | ## Get Roles (get_roles.py) 2 | 3 | This is based on following Sophos Central documentation page: 4 | https://developer.sophos.com/docs/common-v1/1/routes/roles/get 5 | 6 | To simplify the workflow of gathering various aspects of Central this is a separate script to gather the roles from 7 | your Sophos Central tenants. You do not need to pass parameters to run the script. However, there are some global 8 | variables which can be changed to adjust tenant, output. 9 | 10 | Global variables that can be changed: 11 | - tenant 12 | - output 13 | 14 | You still have control of how the data is saved: 15 | - JSON 16 | - STDOUT 17 | - Splunk 18 | 19 | The default setting is `STDOUT`, if you change the setting to `JSON` the files will be saved under the `output` folder: 20 | ``` 21 | sophos_central_api_connector 22 | |___output 23 | | |___roles_data 24 | | | _.json 25 | | | ... 26 | ``` 27 | 28 | The log level can be adjusted under the main function for the variable: `log_level` to gather more granular feedback when 29 | running the script. -------------------------------------------------------------------------------- /sophos_central_api_connector/config/sophos_config.ini: -------------------------------------------------------------------------------- 1 | #This is the config file for sophos_central_main.py 2 | 3 | #The awsSecret section allows you to specify your AWS Secrets Manager 4 | #information to utilise the safe keeping of your Client ID and Client Secret 5 | #The client_id_key and client_secret_key are the names you use for the 'Secret Key' 6 | #when you created a new secret to store the Sophos Central API Client ID and Client Secret 7 | [aws] 8 | secret_name: 9 | region_name: 10 | client_id_key: 11 | client_secret_key: 12 | 13 | #The static config is for when you want to enter the plain text of the Client ID and Client Secret 14 | #we recommend you only use this on a test environment and destroy the API key afterwards 15 | [static] 16 | client_id: 17 | client_secret: 18 | 19 | #Add in a positive number to the below configs. These stipulate the size of the page requested for the 20 | #inventory and alerts in the Sophos Central API. 21 | #Either use a ':' or '=' followed by your desired page size 22 | [page_size] 23 | inventory_ps: 500 24 | alerts_ps: 100 25 | settings_ps: 100 26 | ld_query: 250 27 | ld_results: 1000 28 | firewall: 100 29 | fw_grps: 100 30 | admin_ps: 100 -------------------------------------------------------------------------------- /sophos_central_api_connector/docs/get_admins.md: -------------------------------------------------------------------------------- 1 | ## Get Admins (get_admins.py) 2 | 3 | This is based on following Sophos Central documentation page: 4 | https://developer.sophos.com/docs/common-v1/1/routes/admins/get 5 | 6 | To simplify the workflow of gathering various aspects of Central this is a separate script to gather the admins from 7 | your Sophos Central tenants. You do not need to pass parameters to run the script. However, there are some global 8 | variables which can be changed to adjust tenant, output and page size. 9 | 10 | Global variables that can be changed: 11 | - page_size 12 | - tenant 13 | - output 14 | 15 | page_size: 16 | The default setting is set to `50` this can be changed in line with the api documentation value limits for `pageSize` 17 | 18 | You still have control of how the data is saved: 19 | - JSON 20 | - STDOUT 21 | - Splunk 22 | 23 | The default setting is `STDOUT`, if you change the setting to `JSON` the files will be saved under the `output` folder: 24 | ``` 25 | sophos_central_api_connector 26 | |___output 27 | | |___admin_data 28 | | | _.json 29 | | | ... 30 | ``` 31 | 32 | The log level can be adjusted under the main function for the variable: `log_level` to gather more granular feedback when 33 | running the script. -------------------------------------------------------------------------------- /sophos_central_api_connector/docs/get_firewalls.md: -------------------------------------------------------------------------------- 1 | ## Get Firewalls (get_firewalls.py) 2 | 3 | This is based on following Sophos Central documentation page: 4 | https://developer.sophos.com/docs/firewall-v1/1/routes/firewalls/get 5 | 6 | To simplify the workflow of gathering various aspects of Central this is a separate script to gather the firewalls from 7 | your Sophos Central tenants. You do not need to pass parameters to run the script. However, there are some global 8 | variables which can be changed to adjust tenant, output and page size. 9 | 10 | Global variables that can be changed: 11 | - page_size 12 | - tenant 13 | - output 14 | 15 | page_size: 16 | The default setting is set to `100` this can be changed in line with the api documentation value limits for `pageSize` 17 | 18 | You still have control of how the data is saved: 19 | - JSON 20 | - STDOUT 21 | - Splunk 22 | 23 | The default setting is `STDOUT`, if you change the setting to `JSON` the files will be saved under the `output` folder: 24 | ``` 25 | sophos_central_api_connector 26 | |___output 27 | | |_firewall_inventory 28 | | | _.json 29 | | | ... 30 | ``` 31 | 32 | The log level can be adjusted under the main function for the variable: `log_level` to gather more granular feedback when 33 | running the script. -------------------------------------------------------------------------------- /sophos_central_api_connector/docs/get_firewall_groups.md: -------------------------------------------------------------------------------- 1 | ## Get Firewall Groups (get_firewall_groups.py) 2 | 3 | This is based on following Sophos Central documentation page: 4 | https://developer.sophos.com/docs/firewall-v1/1/routes/firewall-groups/get 5 | 6 | To simplify the workflow of gathering various aspects of Central this is a separate script to gather the firewall groups 7 | from your Sophos Central tenants. You do not need to pass parameters to run the script. However, there are some global 8 | variables which can be changed to adjust tenant, output and page size. 9 | 10 | Global variables that can be changed: 11 | - page_size 12 | - tenant 13 | - output 14 | 15 | page_size: 16 | The default setting is set to `100` this can be changed in line with the api documentation value limits for `pageSize` 17 | 18 | You still have control of how the data is saved: 19 | - JSON 20 | - STDOUT 21 | - Splunk 22 | 23 | The default setting is `STDOUT`, if you change the setting to `JSON` the files will be saved under the `output` folder: 24 | ``` 25 | sophos_central_api_connector 26 | |___output 27 | | |_firewall_groups 28 | | | _.json 29 | | | ... 30 | ``` 31 | 32 | The log level can be adjusted under the main function for the variable: `log_level` to gather more granular feedback when 33 | running the script. -------------------------------------------------------------------------------- /sophos_central_api_connector/sophos_central_api_delete_data.py: -------------------------------------------------------------------------------- 1 | import logging 2 | import requests 3 | 4 | 5 | def delete_local_site(del_dict, tenant_url_data): 6 | logging.info("Delete selected sites from local site data") 7 | processed_del_dict = dict() 8 | 9 | for ten_id, ten_item in tenant_url_data.items(): 10 | tenant_id = ten_id 11 | logging.info("Deleting local sites for tenant: {0}".format(tenant_id)) 12 | for site_id, site_item in del_dict.items(): 13 | tenant_ref = site_item['tenantId'] 14 | if tenant_ref == tenant_id: 15 | orig_url = ten_item['orig_url'] 16 | headers = ten_item['headers'] 17 | ls_url = "{0}/{1}".format(orig_url, site_id) 18 | del_ls = requests.delete(ls_url, headers=headers) 19 | del_data = del_ls.json() 20 | del_status = del_data['deleted'] 21 | del_date = del_ls.headers['date'] 22 | logging.debug("Local Site ID: {0}, Deletion Status: {1}".format(site_id, del_status)) 23 | processed_del_dict[site_id] = {"tenantId": site_item['tenantId'], "intelixRisk": site_item['intelixRisk'], 24 | "url": site_item['url'], "delStatus": del_status, "date": del_date} 25 | logging.info("Completed deletion of local sites for tenant: {0}".format(tenant_id)) 26 | return processed_del_dict 27 | -------------------------------------------------------------------------------- /sophos_central_api_connector/config/splunk_config.ini: -------------------------------------------------------------------------------- 1 | # This is the awsSecret information for your Splunk HEC setup. 2 | [splunk_aws] 3 | splunk_aws: 4 | secret_name: 5 | region_name: 6 | token_key: 7 | 8 | # This is the Splunk HEC (HTTP Event Collector) token to be used for sending data to Splunk 9 | # We recommend you only use this on a test environment and destroy the token afterwards and 10 | # that is restricted to a test index 11 | [splunk_static] 12 | token: 13 | 14 | # Information on how to configure HEC can be found at the following Splunk URI: 15 | # https://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector 16 | # The splunk host is uri to your splunk instance that runs HEC and the HEC port 17 | # Valid entries for splunk_ack_enabled and verify_ack_result are 0 or 1. 18 | # Further information on what this setting does can be found here: 19 | # https://docs.splunk.com/Documentation/Splunk/7.3.1/Data/AboutHECIDXAck 20 | # The channel requires to be a UUID, follow the above link for more information on this setting 21 | # The ack_batch_size is how many events will be processed before indexer acknowledgements are checked. 22 | [splunk_hec] 23 | splunk_url: https://:8088/services/collector 24 | splunk_ack_enabled: 25 | verify_ack_result: 26 | channel: 27 | ack_batch_size: 28 | 29 | # Using the following configs will override settings for HEC config set. If nothing is set it will use the 30 | # settings on the config in Splunk 31 | [splunk_transform] 32 | host: 33 | source: 34 | sourcetype: -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | *$py.class 5 | 6 | # C extensions 7 | *.so 8 | 9 | # Distribution / packaging 10 | .Python 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | .eggs/ 17 | lib/ 18 | lib64/ 19 | parts/ 20 | sdist/ 21 | var/ 22 | wheels/ 23 | *.egg-info/ 24 | .installed.cfg 25 | *.egg 26 | MANIFEST 27 | 28 | # PyInstaller 29 | # Usually these files are written by a python script from a template 30 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 31 | *.manifest 32 | *.spec 33 | 34 | # Installer logs 35 | pip-log.txt 36 | pip-delete-this-directory.txt 37 | 38 | # Unit test / coverage reports 39 | htmlcov/ 40 | .tox/ 41 | .coverage 42 | .coverage.* 43 | .cache 44 | nosetests.xml 45 | coverage.xml 46 | *.cover 47 | .hypothesis/ 48 | .pytest_cache/ 49 | 50 | # Translations 51 | *.mo 52 | *.pot 53 | 54 | # Django stuff: 55 | *.log 56 | local_settings.py 57 | db.sqlite3 58 | 59 | # Flask stuff: 60 | instance/ 61 | .webassets-cache 62 | 63 | # Scrapy stuff: 64 | .scrapy 65 | 66 | # Sphinx documentation 67 | sophos_central_api_connector/docs/_build/ 68 | 69 | # PyBuilder 70 | target/ 71 | 72 | # Jupyter Notebook 73 | .ipynb_checkpoints 74 | 75 | # pyenv 76 | .python-version 77 | 78 | # celery beat schedule file 79 | celerybeat-schedule 80 | 81 | # SageMath parsed files 82 | *.sage.py 83 | 84 | # Environments 85 | .env 86 | .venv 87 | env/ 88 | venv/ 89 | ENV/ 90 | env.bak/ 91 | venv.bak/ 92 | 93 | # Spyder project settings 94 | .spyderproject 95 | .spyproject 96 | 97 | # Rope project settings 98 | .ropeproject 99 | 100 | # mkdocs documentation 101 | /site 102 | 103 | # mypy 104 | .mypy_cache/ 105 | .DS_Store -------------------------------------------------------------------------------- /sophos_central_api_connector/docs/sophos_configuration.md: -------------------------------------------------------------------------------- 1 | ## Sophos Configuration 2 | The following configuration files can be changed to reflect your environment and also your needs. 3 | 4 | ### sophos_central_api_config.py 5 | Majority of the variables contained in this config file are to remain static in order to maintain the correct functionality of the Sophos Central API Connector. However, there are two variables which can be changed if you would like default behaviour to be different. 6 | 7 | #### DEFAULT_OUTPUT 8 | This variable is set to `stdout` so in the event that no output parameter is passed to the CLI then results will be returned to the console. You can change this to be another valid parameter value if desired. 9 | 10 | #### DEFAULT_DAYS 11 | This variable is set to `1` in the event that no days parameter is passed in certain scenarios. This default is also used for the default number of days passed for polling alert events. 12 | 13 | ### sophos_config.ini 14 | > #### **Important!** 15 | > Whilst you are able to set static API credentials in this configuration we strongly advise that this is only done for testing purposes. 16 | > Where possible use AWS Secrets Manager to store your credential id and token 17 | > Please reference the authentication section under advanced usage to use the correct parameter 18 | 19 | You can access your AWS secrets by configuring your details as below: 20 | - **secret_name:** \ 21 | - **region_name:** \ 22 | - **client_id_key:** \ 23 | - **client_secret_key:** \ 24 | 25 | The page size configuration is the number of events you would like to appear per page when querying the Sophos Central API. There are maximum sizes which are checked during the execution of the connector. If these are left blank they will use the default page sizes as determined by the API -------------------------------------------------------------------------------- /setup.py: -------------------------------------------------------------------------------- 1 | import setuptools 2 | from os import path 3 | 4 | # read the contents of your README file 5 | this_directory = path.abspath(path.dirname(__file__)) 6 | with open(path.join(this_directory, 'README.md'), encoding='utf-8') as f: 7 | long_description = f.read() 8 | 9 | setuptools.setup(name='sophos_central_api_connector', 10 | version='0.1.6', 11 | description='Sophos Central API Connector', 12 | author='Mark Lanczak-Faulds', 13 | author_email='mark.lanczak-faulds@sophos.com', 14 | long_description=long_description, 15 | long_description_content_type='text/markdown', 16 | classifiers=[ 17 | "Programming Language :: Python :: 3", 18 | "License :: OSI Approved :: GNU General Public License v3 (GPLv3)", 19 | "Operating System :: OS Independent", 20 | ], 21 | url="https://github.com/sophos-cybersecurity/sophos-central-api-connector", 22 | setup_requires=["setuptools_scm"], 23 | packages=['sophos_central_api_connector', 'sophos_central_api_connector.config'], 24 | package_data={'sophos_central_api_connector': ['config/*']}, 25 | python_requires='>=3.6', 26 | install_requires=['boto3>=1.17.21', 27 | 'botocore>=1.20.21', 28 | 'certifi>=2020.12.5', 29 | 'chardet>=4.0.0', 30 | 'configparser>=5.0.2', 31 | 'docutils>=0.16', 32 | 'intelix>=0.1.2', 33 | 'jmespath>=0.10.0', 34 | 'python-dateutil>=2.8.1', 35 | 'requests>=2.25.1', 36 | 's3transfer>=0.3.4', 37 | 'six>=1.15.0', 38 | 'urllib3>=1.26.3'], 39 | include_package_data=True) 40 | -------------------------------------------------------------------------------- /sophos_central_api_connector/docs/splunk_configuration.md: -------------------------------------------------------------------------------- 1 | ## Splunk Configuration 2 | This config is solely for users who are sending the events and inventory directly to Splunk. There are options for both static token information or there is also an option to use the AWS Secrets Manager. 3 | 4 | > #### **Important!** 5 | > We would recommend that the static entry is only used for testing purposes and the token is stored and accessed securely. 6 | > Please reference the authentication section under advanced usage to use the correct parameter 7 | 8 | #### splunk_aws 9 | To enable the use of the token from the AWS Secrets Manager set : `1` 10 | 11 | Setting this option to `0` will allow the use of the static token entry. The 'splunk_hec' section determines the correct settings to where to send the events to Splunk 12 | 13 | #### splunk_url 14 | This is the URL to your Splunk instance: `http(s)://:8088/services/collector` 15 | 16 | Information on how to configure HEC (HTTP Event Collector) can be found at the following Splunk URL: https://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector 17 | 18 | #### splunk_ack_enabled 19 | If you have set Splunk Index Acknowledgements when creating your HEC token, you will need to set this value to '1'. If you have not set the indexer acknowledgements to `0`. This will ensure that the correct URL is used when sending events to Splunk. 20 | 21 | #### verify_ack_result 22 | This is not currently in use 23 | 24 | #### channel 25 | If you have the index acknowledgements checked in your HEC token you will need to provide a channel GUID. These can be anything you like as long as they are unique to your HEC token environment. Further information on setting the channel and enabling Index Acknowledgements 26 | can be found on the Splunk website: https://docs.splunk.com/Documentation/Splunk/7.3.1/Data/AboutHECIDXAck 27 | 28 | #### ack_batch_size 29 | This is not currently in use 30 | 31 | #### splunk_transform 32 | This section of the config allows you to override the details for source, host and sourcetype when the events are sent to Splunk. If this setting is not set then it will use the data that is contained in the transforms file on the indexer. This is not able to override the index that the data is sent to. -------------------------------------------------------------------------------- /sophos_central_api_connector/config/sophos_central_api_config.py: -------------------------------------------------------------------------------- 1 | from pathlib import PurePath 2 | 3 | # Static URI Variables 4 | auth_uri = 'https://id.sophos.com/api/v2/oauth2/token' 5 | whoami_uri = 'https://api.central.sophos.com/whoami/v1' 6 | tenants_ptr_uri = 'https://api.central.sophos.com/partner/v1/tenants' 7 | tenants_org_uri = 'https://api.central.sophos.com/organization/v1/tenants' 8 | alerts_uri = '/common/v1/alerts' 9 | endpoints_uri = '/endpoint/v1/endpoints' 10 | directory_uri = '/common/v1/directory' 11 | settings_uri = '/endpoint/v1/settings' 12 | livediscover_uri = '/live-discover/v1/queries' 13 | xdr_uri = '/xdr-query/v1/queries' 14 | firewall_uri = '/firewall/v1/firewalls' 15 | fw_grps_uri = '/firewall/v1/firewall-groups' 16 | admins_uri = '/common/v1/admins' 17 | roles_uri = '/common/v1/roles' 18 | 19 | 20 | # Default Value 21 | max_alerts_page = 100 22 | max_inv_page = 500 23 | max_localsite_page = 100 24 | max_fw_page = 1000 25 | max_fw_grps_page = 100 26 | max_admins_page = 100 27 | 28 | # Regex Variables 29 | uuid_regex = '[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[1-5][0-9a-fA-F]{3}-[89abAB][0-9a-fA-F]{3}-[0-9a-fA-F]{12}' 30 | 31 | # Path locations 32 | sophos_conf_path = PurePath("/config/sophos_config.ini") 33 | splunk_conf_path = PurePath("/config/splunk_config.ini") 34 | intelix_conf_path = PurePath("/config/intelix_config.ini") 35 | misp_conf_path = PurePath("/config/misp_config.ini") 36 | output_inv_path = PurePath("/output/get_inventory") 37 | output_al_path = PurePath("/output/get_alerts") 38 | output_ls_path = PurePath("/output/get_local_sites") 39 | output_intx_path = PurePath("/output/intelix") 40 | output_intx_del_path = PurePath("/output/intelix/delete_local_sites") 41 | output_queries_path = PurePath("/output/query_results") 42 | output_fw_path = PurePath("/output/firewall_inventory") 43 | output_fw_grps_path = PurePath("/output/firewall_groups") 44 | output_admins_path = PurePath("/output/admin_data") 45 | output_roles_path = PurePath("/output/roles_data") 46 | poll_conf_path = PurePath("/polling/poll_config.json") 47 | poll_alerts_path = PurePath("/polling/alert_ids.json") 48 | poll_temp_path = PurePath("/polling/temp_alert_ids.json") 49 | poll_path = PurePath("/polling") 50 | logging_path = PurePath("/logs/failed_events.json") 51 | temp_path = PurePath("/tmp") 52 | -------------------------------------------------------------------------------- /sophos_central_api_connector/config/intelix_config.ini: -------------------------------------------------------------------------------- 1 | # Intelix AWS secret information 2 | 3 | [intelix] 4 | secret_name: 5 | region_name: 6 | client_id_key: 7 | client_secret_key: 8 | 9 | #The static config is for when you want to enter the plain text of the Client ID and Client Secret 10 | #we recommend you only use this on a test environment and destroy the API key afterwards 11 | [static] 12 | client_id: 13 | client_secret: 14 | 15 | # This config file allows you to set the risk against the ip categories returned from SophosLabs intelix to allow 16 | # ease of managing local-site clean-up. This aims to align the 'riskLevel' for the url_lookup for the ip_lookup 17 | 18 | # riskLevel entries for url_lookup. Number is set to get equivalent for ip_lookup 19 | # UNCLASSIFIED: Threat is unknown. - 0 20 | # TRUSTED: Url is a trusted one. No threat appears. - 1 21 | # LOW: Threat is low for a given url. - 2 22 | # MEDIUM: Threat is medium for a given url. - 3 23 | # HIGH: Threat is high for a given url. - 4 24 | 25 | # anything not in the below list will set as unclassified 26 | 27 | # Current categories for IP lookup: 28 | # malware: Known source of malware 29 | # botnet: Known FUR, TFX and RIP bot proxy IP 30 | # spam_mta: Known spam source / network 31 | # psh Known source of phishing 32 | # spammers: Known FUR and RIP spam network 33 | # enduser_network: Known dynamic IP 34 | # bot: Known bot proxy IP 35 | # generic_mta: Known mail server 36 | # clean_mta: Known whitelisted mail server 37 | # free_mta: Known free email service provider 38 | # bulk_mta: Known bulk email service provider 39 | # isp_mta: Known ISP's outbound mail server 40 | # biz_mta: Known corporate email service provider 41 | # bulk_mta_grey: Known grey bulk email service provider 42 | # news_mta: Known newsletter provider 43 | # notifications_mta: Notification alert 44 | # illegal: Suspected criminal source 45 | 46 | [ip_lookup_risk] 47 | malware: 4 48 | botnet: 4 49 | spam_mta: 4 50 | psh: 4 51 | spammers: 4 52 | enduser_network: 2 53 | bot: 4 54 | generic_mta: 2 55 | clean_mta: 1 56 | free_mta: 2 57 | bulk_mta: 3 58 | isp_mta: 2 59 | biz_mta: 2 60 | bulk_mta_grey: 3 61 | news_mta: 2 62 | notifications_mta: 2 63 | illegal: 4 64 | unclassified: 0 -------------------------------------------------------------------------------- /sophos_central_api_connector/docs/alerts.md: -------------------------------------------------------------------------------- 1 | ## Alert/Event Information 2 | 3 | To gather alerts for your tenants you can pass the alerts option when running the --get parameter. Some additional options are available 4 | when gathering alerts above calling the inventory of machines: 5 | 6 | - --days 7 | - --poll_alerts 8 | - --reset 9 | 10 | As with calling the inventory option, you can pull alerts for a specific tenant or all of the tenants. In addition you can specify the number of days of events you would like to pull back by using the days parameter. 11 | 12 | Sophos Central holds event data for 90 days, so when calling the days parameter you can specifiy days as an integer from 1-90. If no days parameter is passed a default of 1 day is set. below is an example of passing the days parameter: 13 | ```python 14 | python sophos_central_main.py --auth --get alerts --days --output 15 | ``` 16 | 17 | ### Polling Alert Information 18 | The polling option is available for alert events. This is so you can gather alerts over a period of time and maintain a list of events in Splunk. 19 | 20 | The correct syntax to poll for alert events is as follows: 21 | ```python 22 | python sophos_central_main.py --auth --get alerts --days --poll_alerts --output 23 | ``` 24 | On the initial run of this syntax the following files will be created: 25 | - poll_config.json 26 | - alert_ids.json 27 | 28 | The poll_config.json maintains the results of the last poll attempt and also from when the next poll should get events from and to. Along with maintaining the state of the current run and logging whether any failures have occurred. 29 | 30 | When running the poll_alerts for a second time a new file will be generated called: 31 | - temp_alert_ids.json 32 | 33 | This file maintains a list of events which have already been successfully sent to Splunk and will be removed from the alerts obtained from the Sophos Central API. 34 | 35 | ### Resetting Polling 36 | If you need to reset the polling and start re-pulling in events you can pass the reset parameter to initiate this. Along with the reset parameter you can also pass the days parameter in order to specify how many days should be pulled using the API. Syntax on how to pass this is as follows: 37 | 38 | ```python 39 | python sophos_central_main.py --auth --get alerts --days --reset --poll_alerts --output 40 | ``` 41 | 42 | Running this syntax will delete the files: 43 | - poll_config.json 44 | - alert_ids.json 45 | - temp_alert_ids.json -------------------------------------------------------------------------------- /sophos_central_api_connector/get_firewalls.py: -------------------------------------------------------------------------------- 1 | import logging 2 | from sophos_central_api_connector import sophos_central_api_auth as api_auth 3 | from sophos_central_api_connector import sophos_central_api_awssecrets as awssecret 4 | from sophos_central_api_connector import sophos_central_api_connector_utils as api_utils 5 | from sophos_central_api_connector import sophos_central_api_get_data as get_api 6 | from sophos_central_api_connector import sophos_central_api_output as api_output 7 | 8 | 9 | # Global variables 10 | page_size = "100" 11 | api = "firewall" # do not change 12 | tenant= None 13 | output = "stdout" 14 | sourcetype_value = None 15 | 16 | 17 | def get_firewalls(tenant_url_data, output, page_size, tenant): 18 | # Validate page size set 19 | page_size = api_utils.validate_page_size(page_size, api) 20 | 21 | # Generate events and send them to output 22 | for ten_id, ten_item in tenant_url_data.items(): 23 | # Pass the ten_url_data and gather the alerts 24 | logging.info("Attempting to get firewall information") 25 | tenant_id = ten_id 26 | # get data information for the tenant in the loop 27 | json_data = get_api.get_data(tenant_url_data, page_size, tenant_id, api) 28 | # process data by the output parameter 29 | events = api_output.process_output(output, json_data, tenant_url_data, tenant_id, api, sourcetype_value) 30 | else: 31 | # Completed gathering data for this tenant 32 | logging.info("Completed processing for Tenant ID: {0}".format(tenant_id)) 33 | 34 | 35 | def main(): 36 | log_level="INFO" 37 | if log_level is None: 38 | logging.disable(True) 39 | else: 40 | logging.disable(False) 41 | log_name = log_level 42 | level = getattr(logging, log_name) 43 | log_fmt = '%(asctime)s: [%(levelname)s]: %(message)s' 44 | logging.basicConfig(level=level, format=log_fmt, datefmt='%d/%m/%Y %I:%M:%S %p') 45 | 46 | logging.info("Start of Logging") 47 | 48 | # Auth and get tenant information 49 | tenant_info = api_auth.ten_auth() 50 | 51 | # Generate tenant url data 52 | tenant_url_data = api_utils.generate_tenant_urls(tenant_info, page_size, api, from_str=None, to_str=None) 53 | 54 | for key, value in tenant_url_data.items(): 55 | # If a tenant has been passed in the CLI arguments it checks whether it exists in the tenants obtained 56 | if tenant == key: 57 | # The tenant passed has been found 58 | logging.info("Tenant ID passed: '{0}'".format(key)) 59 | tenant_url_data = {key: value} 60 | else: 61 | pass 62 | 63 | get_firewalls(tenant_url_data, output, page_size, tenant) 64 | 65 | logging.info("Script complete") 66 | exit(0) 67 | 68 | 69 | if __name__ == "__main__": 70 | main() -------------------------------------------------------------------------------- /sophos_central_api_connector/get_admins.py: -------------------------------------------------------------------------------- 1 | import logging 2 | from sophos_central_api_connector import sophos_central_api_auth as api_auth 3 | from sophos_central_api_connector import sophos_central_api_awssecrets as awssecret 4 | from sophos_central_api_connector import sophos_central_api_connector_utils as api_utils 5 | from sophos_central_api_connector import sophos_central_api_get_data as get_api 6 | from sophos_central_api_connector import sophos_central_api_output as api_output 7 | 8 | 9 | # Global variables 10 | page_size = "50" #change in line with documentation 11 | api = "admins" #do not change 12 | tenant = None 13 | output = "stdout" 14 | sourcetype_value = None 15 | 16 | 17 | 18 | def get_admins(tenant_url_data, output, page_size, tenant): 19 | # Validate page size set 20 | page_size = api_utils.validate_page_size(page_size, api) 21 | 22 | # Generate events and send them to output 23 | for ten_id, ten_item in tenant_url_data.items(): 24 | # Pass the ten_url_data and gather the alerts 25 | logging.info("Attempting to get admin information") 26 | tenant_id = ten_id 27 | # get data information for the tenant in the loop 28 | json_data = get_api.get_data(tenant_url_data, page_size, tenant_id, api) 29 | # process data by the output parameter 30 | events = api_output.process_output(output, json_data, tenant_url_data, tenant_id, api, sourcetype_value) 31 | else: 32 | # Completed gathering data for this tenant 33 | logging.info("Completed processing for Tenant ID: {0}".format(tenant_id)) 34 | 35 | 36 | def main(): 37 | log_level="INFO" 38 | if log_level is None: 39 | logging.disable(True) 40 | else: 41 | logging.disable(False) 42 | log_name = log_level 43 | level = getattr(logging, log_name) 44 | log_fmt = '%(asctime)s: [%(levelname)s]: %(message)s' 45 | logging.basicConfig(level=level, format=log_fmt, datefmt='%d/%m/%Y %I:%M:%S %p') 46 | 47 | logging.info("Start of Logging") 48 | 49 | # Auth and get tenant information 50 | tenant_info = api_auth.ten_auth() 51 | 52 | # Generate tenant url data 53 | tenant_url_data = api_utils.generate_tenant_urls(tenant_info, page_size, api, from_str=None, to_str=None) 54 | 55 | for key, value in tenant_url_data.items(): 56 | # If a tenant has been passed in the CLI arguments it checks whether it exists in the tenants obtained 57 | if tenant == key: 58 | # The tenant passed has been found 59 | logging.info("Tenant ID passed: '{0}'".format(key)) 60 | tenant_url_data = {key: value} 61 | else: 62 | pass 63 | 64 | get_admins(tenant_url_data, output, page_size, tenant) 65 | 66 | logging.info("Script complete") 67 | exit(0) 68 | 69 | 70 | if __name__ == "__main__": 71 | main() -------------------------------------------------------------------------------- /sophos_central_api_connector/get_firewall_groups.py: -------------------------------------------------------------------------------- 1 | import logging 2 | from sophos_central_api_connector import sophos_central_api_auth as api_auth 3 | from sophos_central_api_connector import sophos_central_api_awssecrets as awssecret 4 | from sophos_central_api_connector import sophos_central_api_connector_utils as api_utils 5 | from sophos_central_api_connector import sophos_central_api_get_data as get_api 6 | from sophos_central_api_connector import sophos_central_api_output as api_output 7 | 8 | 9 | # Global variables 10 | page_size = "100" 11 | api = "firewall_groups" # do not change 12 | tenant= None 13 | output = "stdout" 14 | sourcetype_value = None 15 | 16 | 17 | def get_firewall_groups(tenant_url_data, output, page_size, tenant): 18 | # Validate page size set 19 | page_size = api_utils.validate_page_size(page_size, api) 20 | 21 | # Generate events and send them to output 22 | for ten_id, ten_item in tenant_url_data.items(): 23 | # Pass the ten_url_data and gather the alerts 24 | logging.info("Attempting to get firewall group information") 25 | tenant_id = ten_id 26 | # get data information for the tenant in the loop 27 | json_data = get_api.get_data(tenant_url_data, page_size, tenant_id, api) 28 | # process data by the output parameter 29 | events = api_output.process_output(output, json_data, tenant_url_data, tenant_id, api, sourcetype_value) 30 | else: 31 | # Completed gathering data for this tenant 32 | logging.info("Completed processing for Tenant ID: {0}".format(tenant_id)) 33 | 34 | 35 | def main(): 36 | log_level="INFO" 37 | if log_level is None: 38 | logging.disable(True) 39 | else: 40 | logging.disable(False) 41 | log_name = log_level 42 | level = getattr(logging, log_name) 43 | log_fmt = '%(asctime)s: [%(levelname)s]: %(message)s' 44 | logging.basicConfig(level=level, format=log_fmt, datefmt='%d/%m/%Y %I:%M:%S %p') 45 | 46 | logging.info("Start of Logging") 47 | 48 | # Auth and get tenant information 49 | tenant_info = api_auth.ten_auth() 50 | 51 | # Generate tenant url data 52 | tenant_url_data = api_utils.generate_tenant_urls(tenant_info, page_size, api, from_str=None, to_str=None) 53 | 54 | for key, value in tenant_url_data.items(): 55 | # If a tenant has been passed in the CLI arguments it checks whether it exists in the tenants obtained 56 | if tenant == key: 57 | # The tenant passed has been found 58 | logging.info("Tenant ID passed: '{0}'".format(key)) 59 | tenant_url_data = {key: value} 60 | else: 61 | pass 62 | 63 | get_firewall_groups(tenant_url_data, output, page_size, tenant) 64 | 65 | logging.info("Script complete") 66 | exit(0) 67 | 68 | 69 | if __name__ == "__main__": 70 | main() -------------------------------------------------------------------------------- /sophos_central_api_connector/get_roles.py: -------------------------------------------------------------------------------- 1 | import logging 2 | from sophos_central_api_connector import sophos_central_api_auth as api_auth 3 | from sophos_central_api_connector import sophos_central_api_awssecrets as awssecret 4 | from sophos_central_api_connector import sophos_central_api_connector_utils as api_utils 5 | from sophos_central_api_connector import sophos_central_api_get_data as get_api 6 | from sophos_central_api_connector import sophos_central_api_output as api_output 7 | 8 | 9 | # Global variables 10 | page_size = None # do not change 11 | api = "roles" # do not change 12 | tenant= None 13 | output = "stdout" 14 | sourcetype_value = None 15 | 16 | 17 | def get_roles(tenant_url_data, output, page_size, tenant): 18 | # Generate events and send them to output 19 | for ten_id, ten_item in tenant_url_data.items(): 20 | # Pass the ten_url_data and gather the alerts 21 | logging.info("Attempting to get role information") 22 | tenant_id = ten_id 23 | # get data information for the tenant in the loop 24 | json_data = get_api.get_data(tenant_url_data, page_size, tenant_id, api) 25 | # process data by the output parameter 26 | logging.info("Adding Tenant Information") 27 | for json_id, json_item in json_data.items(): 28 | json_item['tenant_id'] = tenant_id 29 | events = api_output.process_output(output, json_data, tenant_url_data, tenant_id, api, sourcetype_value) 30 | else: 31 | # Completed gathering data for this tenant 32 | logging.info("Completed processing for Tenant ID: {0}".format(tenant_id)) 33 | 34 | 35 | def main(): 36 | log_level="INFO" 37 | if log_level is None: 38 | logging.disable(True) 39 | else: 40 | logging.disable(False) 41 | log_name = log_level 42 | level = getattr(logging, log_name) 43 | log_fmt = '%(asctime)s: [%(levelname)s]: %(message)s' 44 | logging.basicConfig(level=level, format=log_fmt, datefmt='%d/%m/%Y %I:%M:%S %p') 45 | 46 | logging.info("Start of Logging") 47 | 48 | # Auth and get tenant information 49 | tenant_info = api_auth.ten_auth() 50 | 51 | # Generate tenant url data 52 | tenant_url_data = api_utils.generate_tenant_urls(tenant_info, page_size, api, from_str=None, to_str=None) 53 | 54 | for key, value in tenant_url_data.items(): 55 | # If a tenant has been passed in the CLI arguments it checks whether it exists in the tenants obtained 56 | if tenant == key: 57 | # The tenant passed has been found 58 | logging.info("Tenant ID passed: '{0}'".format(key)) 59 | tenant_url_data = {key: value} 60 | else: 61 | pass 62 | 63 | get_roles(tenant_url_data, output, page_size, tenant) 64 | 65 | logging.info("Script complete") 66 | exit(0) 67 | 68 | 69 | if __name__ == "__main__": 70 | main() -------------------------------------------------------------------------------- /sophos_central_api_connector/queries/xdr_queries/xdr_ioc_hunter.sql: -------------------------------------------------------------------------------- 1 | WITH RECURSIVE 2 | -- Decompose the JSON structure into a temporary table 3 | Extract_from_JSON(idata, ind_type, ind_data, entry) AS ( 4 | VALUES (lower('$$IOC_JSON$$'), json_extract_scalar('$$IOC_JSON$$', '$.ioc_data[0].indicator_type'), json_extract_scalar('$$IOC_JSON$$', '$.ioc_data[0].data'), 0) 5 | UNION ALL 6 | SELECT 7 | idata, 8 | json_extract_scalar(idata, '$.ioc_data['||CAST(entry+1 AS VARCHAR)||'].indicator_type'), 9 | json_extract_scalar(idata, '$.ioc_data['||CAST(entry+1 AS VARCHAR)||'].data'), entry + 1 10 | FROM Extract_from_JSON 11 | WHERE json_extract_scalar(idata, '$.ioc_data['||CAST(entry+1 AS VARCHAR)||'].indicator_type') > '' 12 | ), 13 | 14 | ioc_list (indicator_type, ioc_data) AS ( 15 | SELECT 16 | LOWER(ind_type) ind_type, 17 | LOWER(REPLACE(ind_data,'\\','\')) ind_data -- Convert path info with \\ to simply one \ 18 | FROM 19 | Extract_from_JSON 20 | ORDER by entry ASC 21 | ), 22 | Detections (timestamps, time, indicator_type, ioc_data, Detection_Type, ep_name, os_name, os_platform, os_version, name, path, sha1, sha256, destination_ip, domain, query_name) AS ( 23 | SELECT 24 | timestamps AS time, 25 | time, 26 | ioc_list.indicator_type, 27 | ioc_list.ioc_data, 28 | CASE indicator_type 29 | WHEN 'ip' THEN (SELECT 'Destination IP MATCH' WHERE LOWER(destination_ip) LIKE ioc_data ) 30 | WHEN 'domain' THEN (SELECT 'DOMAIN MATCH' WHERE LOWER(domain) LIKE ioc_data ) 31 | WHEN 'url' THEN (SELECT 'DOMAIN MATCH' WHERE LOWER(domain) LIKE ioc_data ) 32 | WHEN 'filepath' THEN (SELECT 'FilePath MATCH' WHERE LOWER(path) LIKE ioc_data ) 33 | WHEN 'file_path' THEN (SELECT 'FilePath MATCH' WHERE LOWER(path) LIKE ioc_data ) 34 | WHEN 'file_path_name' THEN (SELECT 'FilePath MATCH' WHERE LOWER(path) LIKE ioc_data ) 35 | WHEN 'pathname' THEN (SELECT 'FilePath MATCH' WHERE LOWER(path) LIKE ioc_data ) 36 | WHEN 'filename' THEN (SELECT 'FileName MATCH' WHERE LOWER(name) LIKE ioc_data ) 37 | WHEN 'sha1' THEN (SELECT 'SHA1 MATCH' WHERE LOWER(sha1) LIKE ioc_data ) 38 | WHEN 'sha256' THEN (SELECT 'SHA256 MATCH' WHERE LOWER(sha256) LIKE ioc_data ) 39 | ELSE 'UNKNOWN MATCH METHOD' 40 | END Detection_Type, 41 | LOWER(meta_hostname) AS ep_name, 42 | meta_os_name AS os_name, 43 | meta_os_platform AS os_platform, 44 | meta_os_version AS os_version, 45 | name, 46 | path, 47 | sha1, 48 | sha256, 49 | destination_ip, 50 | domain, 51 | query_name 52 | FROM 53 | xdr_data 54 | JOIN ioc_list ON 55 | LOWER(domain) LIKE ioc_data OR 56 | LOWER(destination_ip) LIKE ioc_data OR 57 | LOWER(path) LIKE ioc_data OR 58 | LOWER(sha1) LIKE ioc_data OR 59 | LOWER(sha256) LIKE ioc_data OR 60 | LOWER(name) LIKE ioc_data 61 | ) 62 | SELECT 63 | * 64 | FROM 65 | Detections 66 | WHERE 67 | Detection_Type > '' -------------------------------------------------------------------------------- /sophos_central_api_connector/docs/intelix.md: -------------------------------------------------------------------------------- 1 | ## Intelix 2 | 3 | We have incorporated the SophosLabs Intelix API to assist with cleaning up local site information which can be obtained from the global settings mentioned above. 4 | 5 | If you have not used or signed up for the Intelix API further information can be found here: https://api.labs.sophos.com/doc/index.html#registration-howto 6 | 7 | We use the Intelix API package as a base: https://github.com/sophos-cybersecurity/intelix 8 | 9 | There are a number of options available. 10 | 11 | #### Test 12 | You can run a test to ensure that your configuration is working without the need to run the full commands. 13 | ```python 14 | python sophos_central_main.py --auth --intelix test 15 | ``` 16 | 17 | #### Report 18 | 19 | Running the intelix command in report mode will gather the local site information (all or one tenant) and check Intelix API to see if SophosLabs detect the site. 20 | 21 | As part of the process we automatically dedupe the URLs queried with Intelix to reduce the api calls made. 22 | 23 | It will automatically generate JSON files for the reports. One which is combined Intelix and local site data: `_