├── README.md ├── integration.py └── requirements /README.md: -------------------------------------------------------------------------------- 1 | # QRadar MISP Integration 2 | Integrate QRadar with IOC (Attributes) from MISP - Open Source Threat Intelligence Platform 3 | 4 | ### IBM QRadar: 5 | 6 | ![alt text](https://www.threatconnect.com/wp-content/uploads/QRadar-logo-Website.png "IBM QRadar") 7 | 8 | IBM QRadar Security Information and Event Management (SIEM) centrally collects and analyzes log and network flow data throughout even the most highly distributed environments to provide actionable insights into threats. 9 | 10 | Using advanced analytics, the solution automatically sorts through millions to billions of events per day to detect anomalous and malicious activities, identify and group related events, and generate prioritized alerts to only the most critical threats. 11 | 12 | ### MISP: 13 | 14 | ![alt text](https://raw.githubusercontent.com/MISP/MISP/2.4/INSTALL/logos/misp-logo.png "MISP") 15 | 16 | The MISP threat sharing platform is a free and open source software helping information sharing of threat intelligence including cyber security indicators. 17 | 18 | MISP is a threat intelligence platform for gathering, sharing, storing and correlating Indicators of Compromise of targeted attacks, threat intelligence, financial fraud information, vulnerability information or even counter-terrorism information. 19 | 20 | Many organizations use MISP to maintain internal repository of IOCs involved in any security incident of the organization. 21 | 22 | MISP has rich RESTful API support to integrate with any producers or consumers of threat intelligence information. 23 | 24 | ### Requirements: 25 | 26 | - Linux 27 | - Python 3 28 | - apscheduler 29 | 30 | ### Installation: 31 | ```sh 32 | git clone https://github.com/karthikkbala/MISP-QRadar-Integration.git 33 | ``` 34 | 35 | ### Configuration: 36 | 37 | Edit the integration.py file with favourite editor and provide the following information. 38 | 39 | ```sh 40 | misp_auth_key = "mxVt2yZWkS39XemrgtyhbfYts7ZeeheQ50dXKLHO" 41 | qradar_auth_key = "811aacf9-ef79-456h-98d4-5d27b7a94844" 42 | qradar_ref_set = "MISP_Event_IOC" 43 | misp_server = "IP Address of MISP Server" 44 | qradar_server = "IP Address of QRadar Server" 45 | frequency = 60 # In minutes 46 | ``` 47 | 48 | ### Usage: 49 | ```sh 50 | python3 integration.py >> /var/log/misp-integration.log & 51 | ``` 52 | 53 | ### Error Handling - Use Cases 54 | - Validate if the reference set exists 55 | - Identify the Element Type of the reference set 56 | - If the Reference Set - Element Type is IP, only the IPs from the MISP will be imported to Reference Set. 57 | - Socket connection validation for QRadar and MISP 58 | 59 | ### Output - Success 60 | 61 | ``` 62 | 17:05:50 -- Checking HTTPS Connectivity to QRadar 63 | 17:05:50 -- (Success) HTTPS Connectivity to QRadar 64 | 17:05:50 -- Checking HTTPS Connectivity to MISP 65 | 17:05:50 -- (Success) HTTPS Connectivity to MISP 66 | 17:05:50 -- Validating if reference set MISP_Event_IOC exists 67 | 17:05:50 -- Validating reference set MISP_Event_IOC - (Success) 68 | 17:05:50 -- Identifying Reference set MISP_Event_IOC element type 69 | 17:05:50 -- Reference set element type = IP (Success) 70 | 17:05:50 -- The QRadar Reference Set MISP_Event_IOC Element Type = "IP". Only IPs will be imported to QRadar and the other IOC types will be discarded 71 | 17:05:50 -- Initiating, GET data from MISP on 72 | 17:05:51 -- MISP API Query (Success) 73 | 17:05:51 -- 36 IOCs imported 74 | 17:05:51 -- Trying to clean the IOCs to IP address, as MISP_Event_IOC element type = IP 75 | 17:05:51 -- (Success) Extracted 16 IPs from initial import. 76 | 17:05:51 -- Initiating, IOC POST to QRadar 77 | 17:05:51 -- Imported 16 IOCs to QRadar (Success) 78 | ``` 79 | -------------------------------------------------------------------------------- /integration.py: -------------------------------------------------------------------------------- 1 | import requests 2 | import json 3 | import sys 4 | import time 5 | import re 6 | import socket 7 | import datetime 8 | from apscheduler.schedulers.blocking import BlockingScheduler 9 | import urllib3 10 | urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning) 11 | 12 | #------*****------# 13 | 14 | misp_auth_key = "H23CE6eC3BYCGKJGJG67892vArAY7GmqNN2nGI" 15 | qradar_auth_key = "811aacf9-jh68-444h-98f4-5d25b7a94844" 16 | qradar_ref_set = "MISP_Event_IOC" 17 | misp_server = "MISP Server IP" 18 | qradar_server = "QRadar Server IP" 19 | frequency = 60 # In minutes 20 | 21 | #------*****------# 22 | 23 | misp_url = "https://" + misp_server + "/attributes/restSearch/json/null/" 24 | QRadar_POST_url = "https://" + qradar_server + "/api/reference_data/sets/bulk_load/" + qradar_ref_set 25 | 26 | MISP_headers = { 27 | 'authorization': misp_auth_key, 28 | 'cache-control': "no-cache", 29 | } 30 | 31 | QRadar_headers = { 32 | 'sec': qradar_auth_key, 33 | 'content-type': "application/json", 34 | } 35 | 36 | def validate_refSet(): 37 | validate_refSet_url = "https://" + qradar_server + "/api/reference_data/sets/" + qradar_ref_set 38 | validate_response = requests.request("GET", validate_refSet_url, headers=QRadar_headers, verify=False) 39 | print (time.strftime("%H:%M:%S") + " -- " + "Validating if reference set " + qradar_ref_set + " exists") 40 | if validate_response.status_code == 200: 41 | print(time.strftime("%H:%M:%S") + " -- " + "Validating reference set " + qradar_ref_set + " - (Success) ") 42 | validate_response_data = validate_response.json() 43 | refSet_etype = (validate_response_data["element_type"]) 44 | print(time.strftime("%H:%M:%S") + " -- " + "Identifying Reference set " + qradar_ref_set + " element type") 45 | print(time.strftime("%H:%M:%S") + " -- " + "Reference set element type = " + refSet_etype + " (Success) ") 46 | if refSet_etype == "IP": 47 | print (time.strftime("%H:%M:%S") + " -- " + "The QRadar Reference Set " + qradar_ref_set + " Element Type = \"IP\". Only IPs will be imported to QRadar and the other IOC types will be discarded") 48 | get_misp_data(refSet_etype) 49 | else: 50 | get_misp_data(refSet_etype) 51 | else: 52 | print(time.strftime("%H:%M:%S") + " -- " + "QRadar Reference Set does not exist, please verify if reference set exists in QRadar.") 53 | sys.exit() 54 | 55 | def get_misp_data(refSet_etype): 56 | print(time.strftime("%H:%M:%S") + " -- " + "Initiating, GET data from MISP on " + misp_server) 57 | misp_response = requests.request('GET', misp_url, headers=MISP_headers, verify=False) 58 | json_data = misp_response.json() 59 | ioc_list = [] 60 | if misp_response.status_code == 200: 61 | print(time.strftime("%H:%M:%S") + " -- " + "MISP API Query (Success) ") 62 | for data in json_data["response"]["Attribute"]: 63 | iocs = (data['value']) 64 | ioc_list.append(iocs) 65 | import_data = json.dumps(ioc_list) 66 | ioc_count = len(ioc_list) 67 | print(time.strftime("%H:%M:%S") + " -- " + str(ioc_count) + " IOCs imported") 68 | if refSet_etype == "IP": 69 | print(time.strftime("%H:%M:%S") + " -- " + "Trying to clean the IOCs to IP address, as " + qradar_ref_set + " element type = IP") 70 | r = re.compile("^\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}$") 71 | ioc_cleaned = list(filter(r.match, ioc_list)) 72 | ioc_cleaned_data = json.dumps(ioc_cleaned) 73 | ioc_count_cleaned = len(ioc_cleaned) 74 | print(time.strftime("%H:%M:%S") + " -- " + "(Success) Extracted " + str(ioc_count_cleaned) + " IPs from initial import.") 75 | qradar_post_IP(ioc_cleaned_data, ioc_count_cleaned) 76 | else: 77 | qradar_post_all(import_data, ioc_count) 78 | else: 79 | print(time.strftime("%H:%M:%S") + " -- " + "MISP API Query (Failed), Please check the network connectivity") 80 | sys.exit() 81 | 82 | def qradar_post_IP(ioc_cleaned_data, ioc_count_cleaned): 83 | print(time.strftime("%H:%M:%S") + " -- " + "Initiating, IOC POST to QRadar ") 84 | qradar_response = requests.request("POST", QRadar_POST_url, data=ioc_cleaned_data, headers=QRadar_headers, verify=False) 85 | if qradar_response.status_code == 200: 86 | print(time.strftime("%H:%M:%S") + " -- " + "Imported " + str(ioc_count_cleaned) + " IOCs to QRadar (Success)" ) 87 | else: 88 | print(time.strftime("%H:%M:%S") + " -- " + "Could not POST IOCs to QRadar (Failure)") 89 | 90 | def qradar_post_all(import_data, ioc_count): 91 | print(time.strftime("%H:%M:%S") + " -- " + "Initiating, IOC POST to QRadar ") 92 | qradar_response = requests.request("POST", QRadar_POST_url, data=import_data, headers=QRadar_headers, verify=False) 93 | if qradar_response.status_code == 200: 94 | print(time.strftime("%H:%M:%S") + " -- " + " (Finished) Imported " + str(ioc_count) + " IOCs to QRadar (Success)" ) 95 | print(time.strftime("%H:%M:%S") + " -- " + "Waiting to next schedule in " + schedule + "minutes") 96 | else: 97 | print(time.strftime("%H:%M:%S") + " -- " + "Could not POST IOCs to QRadar (Failure)") 98 | 99 | def socket_check_qradar(): 100 | print(time.strftime("%H:%M:%S") + " -- " + "Checking HTTPS Connectivity to QRadar") 101 | sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 102 | result = sock.connect_ex((qradar_server, int(443))) 103 | if result == 0: 104 | print(time.strftime("%H:%M:%S") + " -- " + "(Success) HTTPS Connectivity to QRadar") 105 | socket_check_misp() 106 | else: 107 | print(time.strftime("%H:%M:%S") + " -- " + "Could not establish HTTPS connection to QRadar, Please check connectivity before proceeding.") 108 | 109 | def socket_check_misp(): 110 | print(time.strftime("%H:%M:%S") + " -- " + "Checking HTTPS Connectivity to MISP") 111 | sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) 112 | result = sock.connect_ex((misp_server, int(443))) 113 | if result == 0: 114 | print(time.strftime("%H:%M:%S") + " -- " + "(Success) HTTPS Connectivity to MISP") 115 | validate_refSet() 116 | else: 117 | print(time.strftime("%H:%M:%S") + " -- " + "Could not establish HTTPS connection to MISP Server, Please check connectivity before proceeding.") 118 | 119 | scheduler = BlockingScheduler() 120 | scheduler.add_job(socket_check_qradar, 'interval', minutes=frequency, next_run_time=datetime.datetime.now()) 121 | scheduler.start() 122 | -------------------------------------------------------------------------------- /requirements: -------------------------------------------------------------------------------- 1 | apscheduler 2 | --------------------------------------------------------------------------------