├── LICENSE ├── README.md └── schemas ├── entities ├── alert.yml ├── any.yml ├── audit_policy.yml ├── cloud.yml ├── destination.yml ├── destination_nat.yml ├── device.yml ├── dns.yml ├── etl.yml ├── event.yml ├── file.yml ├── geo.yml ├── group.yml ├── hash.yml ├── http.yml ├── ip.yml ├── kerberos.yml ├── logon.yml ├── mac.yml ├── meta.yml ├── module.yml ├── network.yml ├── pipe.yml ├── port.yml ├── process.yml ├── registry.yml ├── rule.yml ├── service.yml ├── source.yml ├── source_nat.yml ├── target.yml ├── threat.yml ├── tls.yml ├── url.yml ├── user.yml ├── user_agent.yml └── x509_and_certificates.yml └── tables └── network_session.yml /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2022 Open Threat Research Forge 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # OSSEM Common Data Model 2 | 3 | The OSSEM common data model (CDM) facilitates the normalization and consistency of diverse data sources by providing standardized and extensible data schemas. 4 | 5 | **This projects is designed to:** 6 | 7 | * Simplify data management procedures by creating a structure around diverse data sources 8 | * Integrate diverse data sources and provide a standardized way for security analysts to query data 9 | * Extend the standard schemas to accomodate custom data entities and allow the community to contribute back 10 | -------------------------------------------------------------------------------- /schemas/entities/alert.yml: -------------------------------------------------------------------------------- 1 | name: Alert 2 | prefix: 3 | - Alert 4 | id: C433BF57-A975-4950-9E67-4427199D6C5D 5 | description: Alert fields that describe/normalize an indicator from a tool of a possible issue. 6 | attributes: 7 | - name: Id 8 | type: integer 9 | description: Alert identifier defined by the tool or system that triggered the alert. Alert ids might repeat across different data sources 10 | sample_value: '1234' 11 | - name: Signature 12 | type: string 13 | description: The name or title of an alert 14 | sample_value: EvilActor:CnCv2 15 | - name: Message 16 | type: string 17 | description: The message provided by the alert 18 | sample_value: A file exhibiting behavior of the evil/actor command and control framework 2 was detected. 19 | - name: Description 20 | type: string 21 | description: The expanded description of the alert event 22 | sample_value: 'This is event x' 23 | - name: Severity 24 | type: string 25 | description: The severity of an alert 26 | sample_value: Priority 5 27 | - name: Category 28 | type: string 29 | description: The category of an alert 30 | sample_value: Malware 31 | - name: Version 32 | type: string 33 | description: A signature or alert version 34 | sample_value: '1.2' 35 | references: [] 36 | tags: [] 37 | -------------------------------------------------------------------------------- /schemas/entities/any.yml: -------------------------------------------------------------------------------- 1 | name: Any 2 | prefix: 3 | - Any 4 | id: 2BE52B26-EE8E-4552-8C88-B17F1C379607 5 | description: 'Fields used to define metadata for a single field to include data from multiple fields with similar/same values/data. This data is most commonly created from an ETL pipeline. Any fields below that contain a ''*'' indicates those are searches and not actual fields (key/values). This is because certain values are not desirable to copy/duplicate. However, because of a common schema we can still find are values for a specific common type, without duplicating or copying everything to one field!' 6 | attributes: 7 | - name: EventId 8 | type: string 9 | description: Allows searching a single field for all log IDs. All log ID fields copied/duplicated to a single field as an array. 10 | sample_value: '``' 11 | - name: Hash 12 | type: string 13 | description: Allows searching a single field for all hashes. All hash fields copied/duplicated to a single field as an array. 14 | sample_value: '``' 15 | - name: IpAddr 16 | type: ip 17 | description: Allows searching a single field for all IPs. All IP fields copied/duplicated to a single field as an array. 18 | sample_value: '``' 19 | - name: IpGeo.asOrg 20 | type: string 21 | description: Allows searching a single field for all BGP AS Organization Names. All AS name fields copied/duplicated to a single field as an array. 22 | sample_value: '``' 23 | - name: IpGeo.asn 24 | type: integer 25 | description: Allows searching a single field for all BGP AS Numbers. All AS number fields copied/duplicated to a single field as an array. 26 | sample_value: '``' 27 | - name: MacAddr 28 | type: string 29 | description: Allows searching a single field for all MAC addresses. All MAC address fields copied/duplicated to a single field as an array. 30 | sample_value: '``' 31 | - name: User 32 | type: string 33 | description: Allows searching a single field for all users. All user fields copied/duplicated to a single field as an array. 34 | sample_value: '``' 35 | - name: VlanId 36 | type: integer 37 | description: Allows searching a single field for all VLAN IDs. All VLAN ID fields copied/duplicated to a single field as an array. 38 | sample_value: '' 39 | references: [] 40 | tags: [] 41 | -------------------------------------------------------------------------------- /schemas/entities/audit_policy.yml: -------------------------------------------------------------------------------- 1 | name: AuditPolicy 2 | prefix: 3 | - AuditPolicy 4 | - AuditPolicyPrevious 5 | id: 8630A479-E521-43BA-81B9-526049143F71 6 | description: Event fields used to define/normalize metadata about security audit policy. 7 | attributes: 8 | - name: Id 9 | type: HexInt64 10 | description: unique Audit Policy hexadecimal identifier. 11 | sample_value: '0x703e' 12 | - name: Sd 13 | type: UnicodeString 14 | description: Security Descriptor Definition Language (SDDL) value for the configuration 15 | sample_value: 'D:(A;;DCSWRPDTRC;;;BA)(D;;DCSWRPDTRC;;;SY)S:NO\_ACCESS\_CONTROL' 16 | - name: Category 17 | type: string 18 | description: Audit policy category description. i.e For the Windows Security channel, this could be something such as Detailed Tracking. For Zeek conn.log, this would be network-protocols. 19 | sample_value: Detailed tracking 20 | - name: CategoryId 21 | type: string 22 | description: Audit policy category id. i.e For the Windows Security channel, this could be something such as %%8276 for Detailed Tracking. 23 | sample_value: '%%8276' 24 | - name: CategoryGuid 25 | type: string 26 | description: Unique GUID for audit policy category. i.e For the Windows Security channel, this could be something such as {6997984C-797A-11D9-BED3-505054503030} for Detailed Tracking category. 27 | sample_value: {6997984C-797A-11D9-BED3-505054503030} 28 | - name: SubCategory 29 | type: string 30 | description: Audit policy sub-category description. i.e For the Windows Security channel, this could be something such as Process Creation. 31 | sample_value: Process Creation 32 | - name: SubCategoryId 33 | type: string 34 | description: Audit policy sub-category id. i.e For the Windows Security channel, this could be something such as %%13312 for Process Creation. 35 | sample_value: '%%13312' 36 | - name: SubCategoryGuid 37 | type: string 38 | description: Unique GUID for audit policy sub-category. i.e For the Windows Security channel, this could be something such as {0CCE922B-69AE-11D9-BED3-505054503030} for Process Creation subcategory. 39 | sample_value: {0CCE922B-69AE-11D9-BED3-505054503030} 40 | - name: Changes 41 | type: UnicodeString 42 | description: Changes which were made for configuration 43 | sample_value: '%%8448, %%8450' 44 | - name: UsersCount 45 | type: UInt32 46 | description: Number of users for which the configuration was defined 47 | sample_value: '2' 48 | - name: CrashOnAuditFailValue 49 | type: UInt32 50 | description: Contains new value of CrashOnAuditFail flag for configuration. 51 | sample_value: '1' 52 | - name: SidList 53 | type: string 54 | description: The list of special group SIDs, which New Logon\Security ID is a member of. 55 | sample_value: '{S-1-5-21-3457937927-2839227994-823803824-512}' 56 | - name: Source 57 | type: UnicodeString 58 | description: The name of registered security event source. You can see all registered security event source names in this registry path:“HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\EventLog\Security” 59 | sample_value: 'FSRM Audit' 60 | - name: SourceId 61 | type: HexInt64 62 | description: The unique hexadecimal identifier of registered security event source 63 | sample_value: '0x1cc4e' 64 | references: 65 | - https://docs.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/security-policy-settings 66 | - https://docs.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/administer-security-policy- 67 | - https://www.ultimatewindowssecurity.com/securitylog/encyclopedia/event.aspx?eventID=4908 68 | tags: [] 69 | -------------------------------------------------------------------------------- /schemas/entities/cloud.yml: -------------------------------------------------------------------------------- 1 | name: Cloud 2 | prefix: 3 | - Cloud 4 | id: D973F78A-95EF-414B-A770-9F138DFE68C0 5 | description: Event fields used to identify/normalize infrastructure and application in the cloud from different cloud providers. 6 | attributes: 7 | - name: AppId 8 | type: string 9 | description: The ID of the application for an HTTP application as identified by a proxy. This value is usually specific to the proxy used. 10 | sample_value: '124' 11 | - name: AppName 12 | type: string 13 | description: The name of an application provided by a cloud service. 14 | sample_value: AppOne 15 | - name: AppOperation 16 | type: string 17 | description: The operation the user performed in the context of the application for an HTTP application as identified by a proxy. This value is usually specific to the proxy used. 18 | sample_value: DELETE 19 | - name: AppRiskLevel 20 | type: string 21 | description: The risk level associated with an HTTP application as identified by a proxy. This value is usually specific to the proxy used. 22 | sample_value: 3 23 | references: [] 24 | tags: [] 25 | -------------------------------------------------------------------------------- /schemas/entities/destination.yml: -------------------------------------------------------------------------------- 1 | name: Destination 2 | prefix: 3 | - Dst 4 | id: F02F775F-69EB-40D2-94F6-78207C015F46 5 | description: Event fields used to define/normalize the destination (server) in a network connection event. 6 | attributes: 7 | - name: DomainHostname 8 | type: string 9 | description: The destination server, host, hostname, domain, domain name or what people commonly might refer to as a domain or website when someone is browsing the Internet. Some examples, would include the TLS server name, HTTP Host, DNS Query Name, etc. For information on how to define and use this field refer to the documentation [here](https://ossemproject.com/cdm/guidelines/domain_or_hostname_or_fqdn.html) 10 | sample_value: www.google.com 11 | - name: Bytes 12 | type: integer 13 | description: The number of bytes sent from the destination to the source for the connection or session. 14 | sample_value: '100' 15 | - name: MimeType 16 | type: string 17 | description: Destination MIME type as seen in (layer 7) application layer details or as defined by an application scanner such as an anti-virus/EDR. For HTTP this is usually from the server's "Content-Type" header. https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Complete_list_of_MIME_types 18 | sample_value: application/pdf 19 | - name: Packets 20 | type: integer 21 | description: The number of packets sent from the destination to the source for the connection or session (Reply). The meaning of a packet is defined by the reporting device. 22 | sample_value: '5' 23 | - name: VlanId 24 | type: integer 25 | description: The destination VLAN ID if it can be determined. Most commonly if from a firewall/switch/router then it can be determined 26 | sample_value: '1000' 27 | - name: VlanName 28 | type: string 29 | description: The destination VLAN Name. Most commonly if from a firewall/switch/router then it can be determined 30 | sample_value: untrust-dmz 31 | - name: MetaDstHostNameCategory 32 | type: string 33 | description: The defined grouping of a URL (or could be just based on the domain in the URL) related to what it is (ie adult, news, advertising, parked domains, etc) 34 | sample_value: Search Engines 35 | - name: Zone 36 | type: string 37 | description: The network zone of the destination, as defined by the reporting device. 38 | sample_value: 'dmz' 39 | - name: InterfaceGuid 40 | type: string 41 | description: GUID of the network interface which was used for authentication request (if applicable). Most of the time you would use the interface_name field for the uid. 42 | sample_value: 7C202E90-2FBE-4275-AB0E-9BF67E04BEDF 43 | - name: InterfaceName 44 | type: string 45 | description: The network interface used for the connection or session by the destination device. 46 | sample_value: eth02 47 | - name: ResourceId 48 | type: string 49 | description: The resource Id of the destination device in a network connection 50 | sample_value: '/subscriptions/33333333-8888-4444-a115-aaaaaaaaaaaa/resourcegroups/shokobo/providers/microsoft.compute/virtualmachines/sysmachine2' 51 | - name: ResourceGroup 52 | type: string 53 | description: The ID of the group to which the destination device belongs in a network connection. This might be an AWS account, or an Azure subscription or Resource Group 54 | sample_value: 'DatabaseVMs' 55 | - name: OriginalValue 56 | type: string 57 | description: original value of a destination before any ETL modifications. For example, if wanting to cleanup a network share and keep the IP - this field would be used to keep the original value 58 | sample_value: "8.8.8.8" 59 | references: 60 | - text: Examples of MIME types 61 | link: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Complete_list_of_MIME_types 62 | tags: [] 63 | -------------------------------------------------------------------------------- /schemas/entities/destination_nat.yml: -------------------------------------------------------------------------------- 1 | name: DestinationNat 2 | prefix: 3 | - DstNat 4 | id: 7F538EC3-4A7D-45C7-A135-0C4C0513F161 5 | description: Event fields used to define/normalize the destination NAT (network address translation) in a network connection event. 6 | attributes: 7 | - name: OriginalValue 8 | type: string 9 | description: original value of a destination NAT before any modifications. For example, if wanting to cleanup a network share and keep the IP - this field would be used to keep the original value 10 | sample_value: "8.8.8.8" 11 | references: [] 12 | tags: [] 13 | -------------------------------------------------------------------------------- /schemas/entities/device.yml: -------------------------------------------------------------------------------- 1 | name: Device 2 | prefix: 3 | - Dvc 4 | id: 4DA0ECFA-E640-44E7-B374-E31DA9EB8BE4 5 | description: Events used to normalize events for the device or endpoint that generated the event (source or destination). 6 | extends_entities: 7 | - Source 8 | - Destination 9 | attributes: 10 | - name: Action 11 | type: string 12 | description: If reported by an intermediary device such as a firewall, the action taken by device. 13 | sample_value: 'allow' 14 | - name: InboundInterface 15 | type: string 16 | description: If reported by an intermediary device such as a firewall, the network interface used by it for the connection to the source device 17 | sample_value: 'eth0' 18 | - name: OutboundInterface 19 | type: string 20 | description: If reported by an intermediary device such as a firewall, the network interface used by it for the connection to the destination device. 21 | sample_value: 'Ethernet 4' 22 | - name: Hostname 23 | type: string 24 | description: The host name from which the event/log came from. There may be multiple host names in an event (i.e. syslog could have forwarder host name), this field is to be the most true log host name (i.e. NOT the forwarders name). 25 | sample_value: bobs.uncle-pc 26 | - name: Domain 27 | type: string 28 | description: Name of the domain the device is part of. 29 | sample_value: hunt.wardog.com 30 | - name: Fqdn 31 | type: string 32 | description: The fully qualified domain name of the host 33 | sample_value: WKHR001.hunt.wardog.com 34 | - name: InterfaceGuid 35 | type: string 36 | description: GUID of the network interface which was used for authentication request 37 | sample_value: '{2BB33827-6BB6-48DB-8DE6-DB9E0B9F9C9B}' 38 | - name: InterfaceName 39 | type: string 40 | description: the name (description) of the network interface that was used for authentication request. You can get the list of all available network adapters using "ipconfig /all" command 41 | sample_value: Microsoft Hyper-V Network Adapter 42 | - name: Os 43 | type: string 44 | description: The OS of the device 45 | sample_value: 'iOS' 46 | - name: ModelName 47 | type: string 48 | description: The model name of the device 49 | sample_value: 'Samsung Galaxy Note' 50 | - name: ModelNumber 51 | type: string 52 | description: The model number of the device 53 | sample_value: '10' 54 | - name: Type 55 | type: string 56 | description: The type of the device 57 | sample_value: 'mobile' 58 | references: [] 59 | tags: [] 60 | -------------------------------------------------------------------------------- /schemas/entities/dns.yml: -------------------------------------------------------------------------------- 1 | name: Dns 2 | prefix: 3 | - Dns 4 | id: 891B8455-BCDC-4C07-8648-0912D6352532 5 | description: Event fields used to define metadata in DNS events. This commonly includes data in logs that contain DNS queries. Including, but not limited to, Zeek dns.log, Suricata DNS, Sysmon EventID 22, Windows DNS debug/trace logs. In the verbiage below, request is used to denote the client (or forwarded address if applicable) that is making the DNS request. This would commonly be the client/source that is looking up a domain.The response/answer, is used to denote the server that responded with the answer or responded to the request/client. It is important to remember that in DNS logs their are multiple servers that may be involved in the response. This is similar to how packets are forwarded through routers. 6 | attributes: 7 | - name: AdditionalAuthoritativeName 8 | type: string 9 | description: additional authoritative response data from the supplemental information in the "additional" section of the DNS response defined in https://tools.ietf.org/html/rfc2181#section-5.4.1 10 | sample_value: google.com 11 | - name: AdditionalName 12 | type: string 13 | description: additional response data from the supplemental information in the "additional" section of the DNS response defined in https://tools.ietf.org/html/rfc2181#section-5.4.1 14 | sample_value: 10.10.10.1 15 | - name: Flags 16 | type: array_string 17 | description: An array of DNS flags if the data source does not parse them or set as boolean 18 | sample_value: '[ "1", "0" ]' 19 | - name: FlagsAuthenticated 20 | type: boolean 21 | description: The "AD" flag. Indicates in a response that all data included in the answer and authority sections of the response have been authenticated by the server according to the policies of that server. see https://tools.ietf.org/html/rfc3655#section-6.1 for more information. This is related to DNSSEC 22 | sample_value: 'false' 23 | - name: FlagsAuthoritative 24 | type: boolean 25 | description: The "AA" flag. Whether the response (answer) from the server was authoritative 26 | sample_value: 'true' 27 | - name: FlagsCheckingDisabled 28 | type: boolean 29 | description: The "CD" flag. Indicates checking disabled for DNSSEC 30 | sample_value: 'true' 31 | - name: FlagsRecursionAvailable 32 | type: boolean 33 | description: The "RA" flag. Indicates the server supports recursive queries 34 | sample_value: 'false' 35 | - name: FlagsRecursionDesired 36 | type: boolean 37 | description: The "RD" flag. Client requested recursion for the lookup/request 38 | sample_value: 'true' 39 | - name: FlagsTruncated 40 | type: boolean 41 | description: The "TC" flag. Indicating (from the server) that response was more than permitted for the single sessions channel, this is usually 512 bytes. 42 | sample_value: 'true' 43 | - name: FlagsZ 44 | type: integer 45 | description: The "Z" flag. This is a reserved field for older DNS implementations https://tools.ietf.org/html/rfc5395 46 | sample_value: '0' 47 | - name: QueryClass 48 | type: string 49 | description: The class of the dns record requested in decimal format, normally this should be 1. Query class is related to zone information, therefore most clients would be request this type of class 50 | sample_value: '1' 51 | - name: QueryClassName 52 | type: integer 53 | description: The class of the dns record requested as a string, normally this should be C_INTERNET. Query class is related to zone information, therefore most clients would be request this type of class 54 | sample_value: C_INTERNET 55 | - name: QueryName 56 | type: string 57 | description: what was queried 58 | sample_value: google.com 59 | - name: QueryType 60 | type: string 61 | description: The type of dns requested in decimal format 62 | sample_value: '28' 63 | - name: QueryTypeName 64 | type: integer 65 | description: The type of dns requested as a string 66 | sample_value: AAAA 67 | - name: Rejected 68 | type: boolean 69 | description: The server responded to the query but no answers were given. If not in the log source, could also be determined by a successful dns response code and no answers/replies returned 70 | sample_value: 'false' 71 | - name: ResponseCode 72 | type: integer 73 | description: The response code returned from the server for the request in decimal format 74 | sample_value: '0' 75 | - name: ResponseCodeName 76 | type: string 77 | description: The response code returned from the server for the request as a string 78 | sample_value: NOERROR 79 | - name: ResponseName 80 | type: array_string 81 | description: The results returned for the dns query. can contain a mix of IPs or domains 82 | sample_value: 8.8.8.8 83 | - name: ResponseTtl 84 | type: array_float 85 | description: The time to live (TTL) for each response_name 86 | sample_value: '``' 87 | - name: Rtt 88 | type: float 89 | description: round trip time (RTT) of the dns query to answer 90 | sample_value: '0.006946' 91 | - name: TransactionId 92 | type: integer 93 | description: Hexadecimal identifier assigned by the program that generated the DNS query. Is 16-bit. Can be used to match up DNS requests across software/clients 94 | sample_value: 4D11 95 | - name: TransactionIdHex 96 | type: string 97 | description: transaction_id in decimal format 98 | sample_value: '19729' 99 | references: 100 | - text: RFC2181 101 | link: https://tools.ietf.org/html/rfc2181#section-5.4.1 102 | - text: RFC 2535 103 | link: https://tools.ietf.org/html/rfc3655#section-6.1 104 | - text: see RFC5395 105 | link: https://tools.ietf.org/html/rfc5395 106 | tags: [] 107 | -------------------------------------------------------------------------------- /schemas/entities/etl.yml: -------------------------------------------------------------------------------- 1 | name: Etl 2 | prefix: 3 | - Etl 4 | id: D3BAF300-8CA2-4B2D-A908-62294220B8D9 5 | description: Event fields used to define/normalize specific metadata about the event during the processing of an ETL (Extract, Transform, Load) pipeline. 6 | attributes: 7 | - name: FormatApplied 8 | type: string 9 | description: Formatting or encoding applied during the ETL processing. Also referred to as CODEC in some use cases. Can be an array if multiple formats were applied/determined 10 | sample_value: '[ "sylog", "json" ]' 11 | - name: FormatIsCef 12 | type: boolean 13 | description: During ETL processing, event is determined to be CEF (format) 14 | sample_value: 'false' 15 | - name: FormatIsJson 16 | type: boolean 17 | description: During ETL processing, event is determined to be JSON (format) 18 | sample_value: 'true' 19 | - name: FormatIsSyslog 20 | type: boolean 21 | description: 'During ETL processing, event is determined to be Syslog (format). Technically you could send data encoded in different format over syslog (ie: CEF or JSON), therefore an event/log can have this tag/field as well as other format fields' 22 | sample_value: 'true' 23 | - name: FormatIsXml 24 | type: boolean 25 | description: During ETL processing, event is determined to be XML (format) 26 | sample_value: 'true' 27 | - name: HostAgentType 28 | type: string 29 | description: Type of forwarder from the client (i.e. winlogbeat, nxlog, rsyslog, etc) 30 | sample_value: nxlog 31 | - name: HostAgentUid 32 | type: string 33 | description: UID for the host's software/agent a part of the event 34 | sample_value: fe4fb818-088f-4529-a343-b94baf057a53 35 | - name: InfoTags 36 | type: string 37 | description: Use for any additional information about an event/log during ETL/processing pipeline. Commonly, you would use this for things that are rare but happen (i.e. parsing error for non conforming RFC). Use this field to alert or give context to a user/analyst when looking at the data. 38 | sample_value: inferred network_protocol as udp 39 | - name: InputApplicationName 40 | type: string 41 | description: Application name used to receive or gather the log for the very first portion of the ETL processing (i.e. kafka, beats, syslog) 42 | sample_value: kafka 43 | - name: InputApplicationProtocol 44 | type: string 45 | description: 'Application protocol used to receive or gather the log for the very first portion of the ETL processing (ex: syslog, http, sftp)' 46 | sample_value: kafka 47 | - name: InputSrcPort 48 | type: integer 49 | description: The Port (network) the client/source used to send the log for the very first portion of the ETL processing. Mostly used in syslog 50 | sample_value: 48231 51 | - name: InputPort 52 | type: integer 53 | description: Port (network) used to receive or gather the log for the very first portion of the ETL processing 54 | sample_value: '9092' 55 | - name: InputProtocol 56 | type: string 57 | description: 'Protocol (network) used to receive or gather the log for the very first portion of the ETL processing (ie: tcp, udp, icmp)' 58 | sample_value: tcp 59 | - name: KafkaConsumerGroup 60 | type: string 61 | description: Consumer group that the etl was apart of from consuming from a Kafka topic 62 | sample_value: helk_logstash 63 | - name: KafkaKey 64 | type: string 65 | description: Record key, if any 66 | sample_value: '' 67 | - name: KafkaOffset 68 | type: long 69 | description: Kafka partition for the event 70 | sample_value: 204802842 71 | - name: KafkaPartition 72 | type: integer 73 | description: Kafka partition for the event 74 | sample_value: 1 75 | - name: KafkaTime 76 | type: date 77 | description: Depending on your Kafka broker configuration, this can be either when the record was created (default) or when it was received by the broker 78 | sample_value: 4/11/2018 5:49:25 79 | - name: KafkaTopic 80 | type: string 81 | description: Kafka topic name 82 | sample_value: winevent 83 | - name: Pipeline 84 | type: string 85 | description: Used to keep track of tags related to transforms, enrichment, or modifications made in an ETL pipeline 86 | sample_value: all-add_processed_timestamp 87 | - name: ProcessedTime 88 | type: date 89 | description: The first time the event gets processed by the ETL (processing pipeline) 90 | sample_value: 4/11/2018 5:49:25 91 | - name: Version 92 | type: string 93 | description: The schema or transform versioning that is being applied 94 | sample_value: v1.0.1 95 | references: [] 96 | tags: [] 97 | -------------------------------------------------------------------------------- /schemas/entities/event.yml: -------------------------------------------------------------------------------- 1 | name: Event 2 | prefix: 3 | - Event 4 | id: 6483A215-4827-4F06-BC2C-EC9FCF41CA96 5 | description: Event attributes used to define/normalize specific metadata of the event itself. This also includes information about the host where the event was originally generated. In scenarios where an event is forwarded (Windows Event Forwarding, Syslog, etc), the ETL entity must be used. 6 | attributes: 7 | - name: Timestamp 8 | type: datetime 9 | description: 'The most accurate timestamp of the log. Commonly this will be the original reporting timestamp from the log. However, if you believe the log timestamp has been altered or skewed (ie: either due to timezone issues or NTP skew)then replace this field with the most likely timestamp. Always keep the original log timestamp in the field creation_timestamp' 10 | sample_value: '2017-01-21 09:12:34' 11 | - name: Duration 12 | type: float 13 | description: The length/duration of the event in seconds (e.g., 1 min is 60.0) 14 | sample_value: '60' 15 | - name: Error 16 | type: string 17 | description: Information about an error 18 | sample_value: 'an error occurred' 19 | - name: ErrorCode 20 | type: integer 21 | description: Integer that defines a particular error 22 | sample_value: '4564' 23 | - name: Id 24 | type: integer 25 | description: event identifier for specific event logs. Event ids might repeat across different data sources. This is most common in Windows using EventID 26 | sample_value: 4688 27 | - name: StartTime 28 | type: datetime 29 | description: The time in which the event stated 30 | sample_value: '2017-01-21 09:12:34' 31 | - name: EndTime 32 | type: datetime 33 | description: The time in which the event ended 34 | sample_value: '2017-04-12 12:00:00' 35 | - name: CreationTime 36 | type: datetime 37 | description: original time when event/log was created as reported from the log source itself 38 | sample_value: '2017-01-21 09:12:34' 39 | - name: TimeIngested 40 | type: datetime 41 | description: The time the event was ingested to SIEM or data pipeline. 42 | sample_value: 2157-01-21 09:12:34 43 | - name: SchemaVersion 44 | type: string 45 | description: Azure Sentinel Schema Version 46 | sample_value: '0.1' 47 | - name: CategoryType 48 | type: string 49 | description: A description of the event, which can help with categorization. If the vendor defines a category/grouping for its log. i.e. Zeek has a few category types for its many logs (network-protocols, network-observations, etc...). Example. sysmon event id 12 is EventType field is this. 50 | sample_value: network-protocols 51 | - name: OriginalMessage 52 | type: string 53 | description: The (original) log message from the source before any ETL manipulations/modifications 54 | sample_value: 'a long message' 55 | - name: OriginalTime 56 | type: datetime 57 | description: original time when event/log was created as reported from the log source itself. 58 | sample_value: 4/11/2018 5:46:18 59 | - name: RecordedTime 60 | type: datetime 61 | description: The time the log was recorded on disk or data plane or if there is another timestamp with the log (common scenario if there is a a manager of many devices or the log itself tracks log time and log written/recorded time) (e.g., 1 min is 60.0). 62 | sample_value: 4/11/2018 5:46:18 63 | - name: Severity 64 | type: string 65 | description: The severity of the event as defined manually or usually via the original log, commonly this would be syslog severity. The number codes should be converted to their corresponding string value. 66 | sample_value: high 67 | - name: Status 68 | type: string 69 | description: Defines the status of a particular event 70 | sample_value: User logon with expired account 71 | - name: StatusCode 72 | type: integer 73 | description: Integer that defines a particular status 74 | sample_value: '3221225875' 75 | - name: SubStatus 76 | type: string 77 | description: Additional status information 78 | sample_value: Account expired 300 days ago 79 | - name: SubStatusCode 80 | type: integer 81 | description: Integer that defines a particular event_sub_status 82 | sample_value: '0' 83 | - name: Timezone 84 | type: string 85 | description: Timezone of the event if it can be determined. Format such as UTC, UTC+1, UTC-5, etc.. 86 | sample_value: UTC 87 | - name: Vendor 88 | type: string 89 | description: The vendor of the product generating the event 90 | sample_value: 'Microsoft' 91 | - name: VendorId 92 | type: string 93 | description: Global unique identifer used to identify the vendor of the product generating the event 94 | sample_value: '64115818-57D3-4468-B767-D401066C28BE' 95 | - name: Product 96 | type: string 97 | description: The product generating the event. Vendor and product might be the same for some data sources. 98 | sample_value: 'Windows' 99 | - name: ProductVersion 100 | type: string 101 | description: The version of the product generating the event 102 | sample_value: '10' 103 | - name: Type 104 | type: string 105 | description: Type of event being collected. i.e For Windows it would be the Event Provider (Microsoft-Windows-Security-Auditing). I.e. Paloalto, it would be the type of event such as Traffic or Threat. I.e. Zeek Logs, one example could be the conn.log. 106 | sample_value: Microsoft-Windows-Security-Auditing 107 | - name: SubType 108 | type: string 109 | description: If there are subsets of an event log type, this field carries the next level value. i.e For windows, it would be the Security channel. 110 | sample_value: Security 111 | - name: CategoryType 112 | type: string 113 | description: If the event contains a category, then this it. i.e For the Windows Security channel, this could be something such as Audit object access. For Zeek conn.log, this would be network-protocols. 114 | sample_value: Audit Object Access 115 | - name: SubCategoryType 116 | type: string 117 | description: If the event contains a sub-category, then this it. i.e For the Windows Security channel, this could be something such as Audit Registry. 118 | sample_value: Audit Registry 119 | - name: TypeDetailed 120 | type: string 121 | description: Additional description of type if applicable 122 | sample_value: '``' 123 | - name: OriginalUid 124 | type: string 125 | description: Original unique ID specific to the log/event as recorded from the source. 126 | sample_value: CMzY3i4YoNZ3mT5yu5 127 | - name: Uid 128 | type: string 129 | description: Original unique ID specific to the log/event assigned to the event (not original). 130 | sample_value: 516a64e3-8360-4f1e-a67c-d96b3d52df54 131 | - name: Count 132 | type: integer 133 | description: The number of aggregated events, if applicable 134 | sample_value: 10 135 | - name: Message 136 | type: string 137 | description: A general message or description, either included in, or generated from the record 138 | sample_value: 'TCP access denied' 139 | - name: ResourceGroup 140 | type: string 141 | description: The resource group to which the device generating the record belongs. This might be an AWS account, or an Azure subscription or Resource Group 142 | sample_value: 'DBVM' 143 | - name: ResourceId 144 | type: string 145 | description: The resource ID of the device generating the message. 146 | sample_value: '/subscriptions/aaabbbcc-dddd-eeee-1234-1234567890ab/resourcegroups/shokobo/providers/microsoft.compute/virtualmachines/sysmachine' 147 | - name: ReportUrl 148 | type: string 149 | description: url of the full analysis report, if applicable 150 | sample_value: 'https://192.168.1.1/reports/ade-123-afa.log' 151 | - name: Result 152 | type: string 153 | description: The result reported for the activity. Empty value when not applicable 154 | sample_value: success 155 | - name: ResultDetails 156 | type: string 157 | description: Reason or details for the result reported in event_result 158 | sample_value: Wrong Password 159 | references: [] 160 | tags: [] 161 | -------------------------------------------------------------------------------- /schemas/entities/file.yml: -------------------------------------------------------------------------------- 1 | name: File 2 | prefix: 3 | - File 4 | - FilePrevious 5 | id: E94FB7C5-185A-459F-8C27-87600E32E598 6 | description: Event fields used to define/normalize metadata about files either locally or over the wire (Network Traffic). This entity and attributes can extend other entities such as source and destination. 7 | extends_entities: 8 | - Source 9 | - Destination 10 | - Process 11 | - Service 12 | attributes: 13 | - name: Name 14 | type: string 15 | description: name of the file without its full path. This could be a local file or one transmitted over the network. 16 | sample_value: a.exe 17 | - name: Path 18 | type: string 19 | description: full path of a file including the name of the file. This could be a local file or one transmitted over the network. 20 | sample_value: C:\users\wardog\z.exe 21 | - name: Extension 22 | type: string 23 | description: The extension name or type of the file. 24 | sample_value: exe 25 | - name: Company 26 | type: string 27 | description: Company name a file belongs to 28 | sample_value: Microsoft Corporation 29 | - name: Product 30 | type: string 31 | description: The file's product name 32 | sample_value: Microsoft® Windows® Operating System 33 | - name: Description 34 | type: string 35 | description: Description of a file 36 | sample_value: Console Window Host 37 | - name: MimeType 38 | type: string 39 | description: MIME type name specified for the file 40 | sample_value: application/msword 41 | - name: Size 42 | type: integer 43 | description: Size of the file, in bytes. 44 | sample_value: '45' 45 | - name: SystemBlockSize 46 | type: integer 47 | description: Block size of filesystem 48 | sample_value: '' 49 | - name: Directory 50 | type: string 51 | description: Directory of file(s). It does not include the file name 52 | sample_value: C:\users\wardog\ 53 | - name: Inode 54 | type: integer 55 | description: Filesystem inode number 56 | sample_value: '' 57 | - name: HardLinks 58 | type: integer 59 | description: Number of hard links 60 | sample_value: '3' 61 | - name: Symlink 62 | type: integer 63 | description: 1 if the path is a symlink, otherwise 0 64 | sample_value: '0' 65 | - name: SymlinkName 66 | type: string 67 | description: path of the symlink 68 | sample_value: C:\Docs\My.exe 69 | - name: LinkName 70 | type: string 71 | description: path of the hard link 72 | sample_value: C:\Docs\My.exe 73 | - name: AccessedTime 74 | type: date 75 | description: When the file was last accessed . Also known as `atime` 76 | sample_value: 2016-11-25 18:21:47 77 | - name: CreationTime 78 | type: date 79 | description: When the file was created. Also known as `crtime` 80 | sample_value: 2016-11-25 18:21:47 81 | - name: ChangedTime 82 | type: date 83 | description: When the file was last changed. Also known as `ctime` 84 | sample_value: 2016-11-25 18:21:47 85 | - name: ModifiedTime 86 | type: date 87 | description: When the file was last modified. Also known as `mtime` 88 | sample_value: 2016-11-25 18:21:47 89 | - name: SystemType 90 | type: string 91 | description: 'The file system type, ex: fat32, ntfs, vmfs, ext3, ext4, xfs' 92 | sample_value: ntfs 93 | - name: Version 94 | type: string 95 | description: file version. i.e. image loaded version 96 | sample_value: 10.0.16299.15 (WinBuild.160101.0800) 97 | references: [] 98 | tags: [] 99 | -------------------------------------------------------------------------------- /schemas/entities/geo.yml: -------------------------------------------------------------------------------- 1 | name: Geo 2 | prefix: 3 | - Geo 4 | id: F5C320EA-B930-4AEA-BE49-AC424D39760E 5 | description: Event fields used to define/normalize metadata about a geographical location. 6 | extends_entities: 7 | - Source 8 | - Destination 9 | attributes: 10 | - name: Latitude 11 | type: string 12 | description: The latitude of the geographical coordinate associated with the IP address in the network session. 13 | sample_value: "38.8951" 14 | - name: Longitude 15 | type: string 16 | description: The longitude of the geographical coordinate associated with the IP address in the network session. 17 | sample_value: "-77.0364" 18 | - name: Continent 19 | type: string 20 | description: The continent associated with the IP address in the network session. 21 | sample_value: South America 22 | - name: Country 23 | type: string 24 | description: The country associated with the IP address in the network session. 25 | sample_value: Peru 26 | - name: CountryCapital 27 | type: string 28 | description: The capital of the country associated with the IP address in the network session. 29 | sample_value: Lima 30 | - name: CountryCode 31 | type: string 32 | description: 51 33 | sample_value: Country code 34 | - name: City 35 | type: string 36 | description: The city associated to the IP address in the network session. 37 | sample_value: San Miguel 38 | - name: Region 39 | type: string 40 | description: The region within a country associated with the IP address in the network session. 41 | sample_value: East US 42 | references: [] 43 | tags: [] 44 | -------------------------------------------------------------------------------- /schemas/entities/group.yml: -------------------------------------------------------------------------------- 1 | name: Group 2 | prefix: 3 | - Group 4 | id: 1DB9799D-8463-4962-92D3-93F0D6AD65F1 5 | description: Event fields used to define/normalize metadata about a security group, or distribution group that is created changed or deleted. 6 | attributes: 7 | - name: Name 8 | type: string 9 | description: the name of a security group, or a distribution group that is created,changed, or deleted 10 | sample_value: AccountOperators 11 | - name: Domain 12 | type: string 13 | description: domain or computer name of the group 14 | sample_value: CONTOSO 15 | - name: Sid 16 | type: string 17 | description: SID of a group 18 | sample_value: S-1-5-21-3457937927-2839227994-823803824-6605 19 | - name: SamName 20 | type: string 21 | description: 'this is a name of the group used to support clients and servers from previous versions of Windows (pre-Windows 2000 logon name). The value of sAMAccountName attribute of new group object. For example: ServiceDesk. For local groups it is simply a name of new group' 22 | sample_value: AccountOperators 23 | - name: SidHistory 24 | type: string 25 | description: contains previous SIDs used for the object if the object was moved from another domain. Whenever an object is moved from one domain to another, a new SID is created and becomes the objectSID. The previous SID is added to the sIDHistory property. This parameter contains the value of sIDHistory attribute of a group object. This parameter might not be captured in the event, and in that case appears as "-". For local groups it is not applicable and always has "-" value. 26 | sample_value: '-' 27 | references: [] 28 | tags: [] 29 | -------------------------------------------------------------------------------- /schemas/entities/hash.yml: -------------------------------------------------------------------------------- 1 | name: Hash 2 | prefix: 3 | - Hash 4 | id: 42C1A34E-D474-468D-8EFB-09454CA8BFC2 5 | description: Event fields used to define/normalize data related to hashes of an image/binary/file. 6 | extends_entities: 7 | - File 8 | - Certificate 9 | - Process 10 | attributes: 11 | - name: Md5 12 | type: string 13 | description: MD5 hash of the image/binary/file 14 | sample_value: 6A255BEBF3DBCD13585538ED47DBAFD7 15 | - name: Sha1 16 | type: string 17 | description: SHA1 hash of the image/binary/file 18 | sample_value: B0BF5AC2E81BBF597FAD5F349FEEB32CAC449FA2 19 | - name: Sha256 20 | type: string 21 | description: SHA256 hash of the image/binary/file 22 | sample_value: 4668BB2223FFB983A5F1273B9E3D9FA2C5CE4A0F1FB18CA5C1B285762020073C 23 | - name: Sha512 24 | type: string 25 | description: SHA512 hash of the image/binary/file 26 | sample_value: 1AD1D79F85D8F6A50EA282F63898D652661DAA0C1FD361C22647CABC98A70E8CBCE83200D579D10DD0A3D46BE9496DCDFDDF28B0C5E9709343B032A8796FBECB 27 | - name: Imphash 28 | type: string 29 | description: IMPHASH hash of the image/binary/file 30 | sample_value: 2505BD03D7BD285E50CE89CEC02B333B 31 | references: [] 32 | tags: [] 33 | -------------------------------------------------------------------------------- /schemas/entities/http.yml: -------------------------------------------------------------------------------- 1 | name: Http 2 | prefix: 3 | - Http 4 | id: 841A8C9B-EFE2-40E6-9917-DB6784944D75 5 | description: Event fields used to define/normalize metadata about HTTP (Hypertext Transfer Protocol) information. This is based on information in the layer 7 (HTTP) application, however can also include HTTP information from an endpoint/server. IIS, Apache, NGINX, proxy logs, and other variances of logs that have HTTP information would go in here. Also, if the HTTP connection is from a decrypted/MITM HTTPS/TLS session then portions of that information, where applicable, would go in here. 6 | attributes: 7 | - name: CookieVariables 8 | type: string 9 | description: The values of (HTTP) cookies 10 | sample_value: T1NTRU0K 11 | - name: InformationalCode 12 | type: integer 13 | description: integer response code of 100-199 14 | sample_value: '101' 15 | - name: InformationalMessage 16 | type: string 17 | description: message/text of the integer response code that was 100-199 18 | sample_value: Switching Protocols 19 | - name: ProxiedHeaders 20 | type: string 21 | description: All of the headers that may indicate if the request was proxied. i.e. FORWARDED;X-FORWARDED-FOR;X-FORWARDED-FROM;CLIENT-IP;VIA;XROXY-CONNECTION;PROXY-CONNECTION 22 | sample_value: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36 23 | - name: ReferrerOriginal 24 | type: string 25 | description: HTTP header "Referer". The HTTP referer header for HTTP/HTTPS network sessions. 26 | sample_value: 'https://sub.domain.tld/path/a/b/JavaScript' 27 | - name: RequestBodyBytes 28 | type: integer 29 | description: Amount of bytes that the source/client sent 30 | sample_value: '2' 31 | - name: RequestHeaderHost 32 | type: string 33 | description: Value of the HOST header from the client. This should be copied to dst_host_name 34 | sample_value: www.activewebsoftwares.com 35 | - name: RequestHeaderNames 36 | type: string 37 | description: List of any additional (or all) HTTP headers. Because a client can use any HTTP header they want and there are already hundreds of https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers it would be impossible to define a specific field for each one. 38 | sample_value: X-Forwarded-For 39 | - name: RequestHeaderOrigin 40 | type: string 41 | description: Value of the Origin header from the client 42 | sample_value: origin 43 | - name: RequestHeaderValues 44 | type: string 45 | description: Values for the request_header_names parameters 46 | sample_value: 10.1.1.1 47 | - name: RequestMethod 48 | type: string 49 | description: Type of HTTP request that was made. Other examples could be (anything) PUT, POST, HEAD, DELETE 50 | sample_value: GET 51 | - name: ResponseBodyBytes 52 | type: integer 53 | description: Amount of bytes that the destination/server returned 54 | sample_value: '87' 55 | - name: ResponseBodyOriginal 56 | type: string 57 | description: The raw HTTP (response) body 58 | sample_value:
This is title
Hello world 59 | - name: ResponseHeaderNames 60 | type: string 61 | description: List of any additional (or all) HTTP headers. Because a server can use any HTTP header they want and there are already hundreds of https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers it would be impossible to define a specific field for each one. 62 | sample_value: X-Forwarded-For 63 | - name: ResponseHeaderValues 64 | type: string 65 | description: Values for the response_header_names parameters 66 | sample_value: 10.1.1.1 67 | - name: StatusCode 68 | type: integer 69 | description: HTTP Server reply code 70 | sample_value: '200' 71 | - name: StatusMessage 72 | type: string 73 | description: HTTP server reply message 74 | sample_value: OK 75 | - name: Version 76 | type: string 77 | description: HTTP request version 78 | sample_value: '1.1' 79 | - name: RequestTime 80 | type: integer 81 | description: The amount of time in milliseconds it took to send the request to the server, if applicable. 82 | sample_value: 700 83 | - name: ResponseTime 84 | type: integer 85 | description: The amount of time in milliseconds it took to receive a response in the server, if applicable. 86 | sample_value: 800 87 | - name: ContentType 88 | type: string 89 | description: The HTTP Response content type header for HTTP/HTTPS network sessions. 90 | sample_value: '' 91 | - name: RequestXff 92 | type: string 93 | description: The HTTP X-Forwarded-For header for HTTP/HTTPS network sessions. 94 | sample_value: '203.0.113.195' 95 | references: 96 | - text: common HTTP headers 97 | link: https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers 98 | tags: [] 99 | -------------------------------------------------------------------------------- /schemas/entities/ip.yml: -------------------------------------------------------------------------------- 1 | name: Ip 2 | prefix: 3 | - Ip 4 | id: 3D673B51-773A-433E-AE6B-EA4CD6B5F775 5 | description: Event fields used to define/normalize metadata about IP addresses in a network. It follows the standard from the Destination, Source and device categories. 6 | extends_entities: 7 | - Destination 8 | - DestinationNat 9 | - Source 10 | - SourceNat 11 | - Any 12 | - Device 13 | attributes: 14 | - name: Addr 15 | type: ip 16 | description: IP address assigned to the device generating the event and/or the IP address in the network packet. This could be used in the context of source, destination, device and even NAT when it is provided by an intermediary NAT device such as a firewall. 17 | sample_value: 192.168.1.2 18 | - name: IsIpv6 19 | type: boolean 20 | description: If IP address is IP version 6 21 | sample_value: 'false' 22 | - name: DhcpAssignedIpAddr 23 | type: ip 24 | description: IP address assigned by the DHCP server. 25 | sample_value: 192.168.1.2 26 | references: [] 27 | tags: [] 28 | -------------------------------------------------------------------------------- /schemas/entities/kerberos.yml: -------------------------------------------------------------------------------- 1 | name: Kerberos 2 | prefix: 3 | - Krb 4 | id: 76E77A91-9DBB-4B5F-BFC8-B788023DC824 5 | description: 'Event fields used to define/normalize Kerberos Ticket Granting Service and Kerberos Ticket Granting Tickets. For certificate information within Kerberos see the ./x509_and_certificates.md' 6 | attributes: 7 | - name: ServiceName 8 | type: string 9 | description: the name of the account or computer for which the TGS ticket was requested 10 | sample_value: WIN2008R2$ 11 | - name: TicketOptions 12 | type: string 13 | description: this is a set of different ticket flags in hexadecimal format. 14 | sample_value: '0x40810010' 15 | - name: TicketStatus 16 | type: string 17 | description: hexadecimal result code of TGS issue operation. 18 | sample_value: '0x0' 19 | - name: TicketEncryptionType 20 | type: string 21 | description: the cryptographic suite that was used for issued TGS. 22 | sample_value: '0x12' 23 | - name: TicketRequestType 24 | type: string 25 | description: Request type - Authentication Service ("AS") or Ticket Granting Service ("TGS") 26 | sample_value: TGS 27 | - name: TicketPreAuthType 28 | type: integer 29 | description: the code number of pre-Authentication type which was used in TGT request. 30 | sample_value: '15' 31 | references: [] 32 | tags: [] 33 | -------------------------------------------------------------------------------- /schemas/entities/logon.yml: -------------------------------------------------------------------------------- 1 | name: Logon 2 | prefix: 3 | - Logon 4 | id: E25FD5C0-5553-4A8D-888F-12500C2FCB31 5 | extends_entities: 6 | - User 7 | description: Event fields used to define/normalize metadata about logon events. 8 | attributes: 9 | - name: Id 10 | type: integer 11 | description: hexadecimal value that can help you correlate this event with recent events that might contain the same Logon ID 12 | sample_value: '0x8dcdc' 13 | - name: Guid 14 | type: string 15 | description: 'a GUID that can help you correlate this event with another event that can contain the same Logon GUID, "4769(S, F): A Kerberos service ticket was requested event on a domain controller. It also can be used for correlation between a 4624 event and several other events (on the same computer) that can contain the same Logon GUID, "4648(S): A logon was attempted using explicit credentials" and "4964(S): Special groups have been assigned to a new logon."' 16 | sample_value: '{00000000-0000-0000-0000-000000000000}' 17 | - name: Type 18 | type: integer 19 | description: the type of logon which was performed 20 | sample_value: '2' 21 | - name: ProcessName 22 | type: string 23 | description: 'The name of the trusted logon process that was used for the logon. See event "4611: A trusted logon process has been registered with the Local Security Authority" description for more information.' 24 | sample_value: User32 25 | - name: AuthenticationPackageName 26 | type: string 27 | description: 'The name of the authentication package which was used for the logon authentication process. Default packages loaded on LSA startup are located in "HKLM\SYSTEM\CurrentControlSet\Control\Lsa\OSConfig" registry key. Other packages can be loaded at runtime. When a new package is loaded a "4610: An authentication package has been loaded by the Local Security Authority" (typically for NTLM) or "4622: A security package has been loaded by the Local Security Authority" (typically for Kerberos) event is logged to indicate that a new package has been loaded along with the package name.' 28 | sample_value: Negotiate 29 | - name: TransmittedServices 30 | type: string 31 | description: the list of transmitted services. Transmitted services are populated if the logon was a result of a S4U (Service For User) logon process. S4U is a Microsoft extension to the Kerberos Protocol to allow an application service to obtain a Kerberos service ticket on behalf of a user - most commonly done by a front-end website to access an internal resource on behalf of a user. 32 | sample_value: '-' 33 | - name: AuthenticationLanPackageName 34 | type: string 35 | description: 'The name of the LAN Manager sub-package (NTLM-family protocol name) that was used during logon. Possible values are: NTLM V1, NTLM V2, LM. Only populated if Authentication Package = NTLM.' 36 | sample_value: '-' 37 | - name: KeyLength 38 | type: integer 39 | description: the length of NTLM Session Security key. Typically it has 128 bit or 56 bit length. This parameter is always 0 if "Authentication Package" = "Kerberos", because it is not applicable for Kerberos protocol. This field will also have "0" value if Kerberos was negotiated using Negotiate authentication package. 40 | sample_value: '0' 41 | - name: ImpersonationLevel 42 | type: string 43 | description: Impersonation level 44 | sample_value: '%%1833' 45 | - name: RestrictedAdminMode 46 | type: string 47 | description: Only populated for RemoteInteractive logon type sessions. This is a Yes/No flag indicating if the credentials provided were passed using Restricted Admin mode. Restricted Admin mode was added in Win8.1/2012R2 but this flag was added to the event in Win10. If not a RemoteInteractive logon, then this will be "-" string. 48 | sample_value: '-' 49 | - name: VirtualAccount 50 | type: string 51 | description: a "Yes" or "No" flag, which indicates if the account is a virtual account (e.g., "Managed Service Account"), which was introduced in Windows 7 and Windows Server 2008 R2 to provide the ability to identify the account that a given Service uses, instead of just using "NetworkService". 52 | sample_value: '%%1843' 53 | - name: UserLinkedId 54 | type: integer 55 | description: A hexadecimal value of the paired logon session. If there is no other logon session associated with this logon session, then the value is "0x0". 56 | sample_value: '0x0' 57 | - name: ElevatedToken 58 | type: string 59 | description: a "Yes" or "No" flag. If "Yes" then the session this event represents is elevated and has administrator privileges. 60 | sample_value: '%%1842' 61 | - name: UserClaims 62 | type: string 63 | description: list of user claims for new logon session. This field contains user claims if user account was logged in and device claims if computer account was logged in 64 | sample_value: 'ad://ext/cn:88d2b96fdb2b4c49 <%%1818> : "dadmin" ad://ext/Department:88d16a8edaa8c66b <%%1818> : "IT"' 65 | - name: DeviceClaims 66 | type: string 67 | description: list of device claims for new logon session 68 | sample_value: '-' 69 | references: [] 70 | tags: [] 71 | -------------------------------------------------------------------------------- /schemas/entities/mac.yml: -------------------------------------------------------------------------------- 1 | name: Mac 2 | prefix: 3 | - Mac 4 | id: 616ABC0D-E91C-4B99-B292-239B54566A34 5 | description: Event fields used to define/normalize metadata about MAC addresses in a network. 6 | extends_entities: 7 | - Source 8 | - Destination 9 | - Device 10 | attributes: 11 | - name: Addr 12 | type: string 13 | description: MAC address of the device where the event was generated or network interface where a connection starts or ends. 14 | sample_value: 00:11:22:33:44:55 15 | references: [] 16 | tags: [] 17 | -------------------------------------------------------------------------------- /schemas/entities/meta.yml: -------------------------------------------------------------------------------- 1 | name: Meta 2 | prefix: 3 | - Meta 4 | id: A87500A5-FD4D-4B64-8DC1-13E645AB20A6 5 | description: 'For example: url_category would be copied to meta_url_category' 6 | attributes: 7 | - name: Category 8 | type: string 9 | description: 'Description to define a grouping of a value. Commonly used for URL/domain category (ie: Adult, Abuse, Parked, RFC-1918, etc)' 10 | sample_value: '``' 11 | - name: Ttp 12 | type: string 13 | description: Tactic, technique, and procedure 14 | sample_value: '``' 15 | - name: Alert 16 | type: string 17 | description: Data describing an alert 18 | sample_value: '``' 19 | - name: GeoLocation 20 | type: geo_point 21 | description: Geo longitude and latitude point of a field 22 | sample_value: '``' 23 | - name: AsNumber 24 | type: integer 25 | description: Autonomous System (AS) number (BGP AS Number) 26 | sample_value: '``' 27 | - name: AsOrg 28 | type: string 29 | description: Autonomous System (AS) organization (BGP AS Name) 30 | sample_value: '``' 31 | references: [] 32 | tags: [] 33 | -------------------------------------------------------------------------------- /schemas/entities/module.yml: -------------------------------------------------------------------------------- 1 | name: Module 2 | prefix: 3 | - Module 4 | id: 10744F70-5EF6-4BA6-85D7-8BA86C435334 5 | description: Event fields used to define/normalize metadata about modules loaded into a process. A process module represents a .dll or .exe file that is loaded into a particular process. 6 | attributes: 7 | - name: Name 8 | type: string 9 | description: name of the modules loaded into a process without the full path 10 | sample_value: msvcrt.dll 11 | - name: Path 12 | type: string 13 | description: full path of a module loaded into a process 14 | sample_value: C:\Windows\System32\msvcrt.dll 15 | - name: IsSigned 16 | type: boolean 17 | description: is the module loaded signed? 18 | sample_value: 'TRUE' 19 | - name: Signature 20 | type: string 21 | description: The signer 22 | sample_value: Microsoft Corporation 23 | - name: SignatureStatus 24 | type: string 25 | description: status of the signature 26 | sample_value: Valid 27 | references: 28 | - https://docs.microsoft.com/en-us/dotnet/api/system.diagnostics.process.modules?view=netcore-3.1 29 | tags: [] 30 | -------------------------------------------------------------------------------- /schemas/entities/network.yml: -------------------------------------------------------------------------------- 1 | name: Network 2 | prefix: 3 | - Network 4 | id: D495B0FC-EECA-48D9-8B3C-6BA971CFAB1E 5 | description: Event fields used to define metadata about network information seen in a typical OSI layer. This includes data both from an endpoint as well as a network monitoring device/application (NSM, Firewall, IPS, IDS, etc...). This differentiates from data that is specific to Source and Destination specific information such as Source or Destination bytes, packets, IP address, mac address, TCP flags. 6 | attributes: 7 | - name: FingerprintNetworkCommunityId 8 | type: string 9 | description: Network community ID as outlined by the standard from https://github.com/corelight/community-id-spec. Standardized hashing of network tuple. The combination, most commonly, of Source IP, Source Port, Destination IP, Destination Port, and IP Protocol allows pivoting between multiple log types 10 | sample_value: 1:EeVyZ07VGj1n0rld+xCLFdM+u8M= 11 | - name: ApplicationName 12 | type: string 13 | description: Layer 7 (application) name specific to service/name/software as provided by a device or user 14 | sample_value: google-drive 15 | - name: ApplicationProtocol 16 | type: string 17 | description: 'Layer 7 (application) in the OSI model. Ex: HTTP,SMB,FTP,SSH, etc.' 18 | sample_value: HTTP 19 | - name: Bytes 20 | type: integer 21 | description: Total bytes for the session. If this field does not exist in the log source, then its possible in your ETL pipeline to combine the source and destination bytes. 22 | sample_value: '102034' 23 | - name: ConnectionHistory 24 | type: string 25 | description: TCP Flags and other potential IP header info 26 | sample_value: '' 27 | - name: ConnectionHistoryDetailed 28 | type: string 29 | description: Detailed description of the information in connection_history 30 | sample_value: '' 31 | - name: ConnectionState 32 | type: string 33 | description: The end state of the session/connection as defined in short abbreviation 34 | sample_value: '' 35 | - name: ConnectionStateDetailed 36 | type: string 37 | description: Detailed description of the information in network_connection_state 38 | sample_value: '' 39 | - name: Direction 40 | type: string 41 | description: User/Device defined name of the direction of the connection or session (Inbound or Outbound). 42 | sample_value: outbound 43 | - name: Initiated 44 | type: boolean 45 | description: 'Whether the session was initiated or received. Most commonly used in relation to an endpoint/device. False = the endpoint did not initiate the session (ie: was scanned or RDP connection made to it)' 46 | sample_value: 'TRUE' 47 | - name: InnerVlanId 48 | type: integer 49 | description: Normally the VLAN can not be determined as source/destination and VLANs are stacked/wrapped. This is the VLAN "inside" 50 | sample_value: '150' 51 | - name: IpBytes 52 | type: long 53 | description: Total IP bytes, according to ip headers, for the session. If this field does not exist in the log source, then its possible in your ETL pipeline to combine the source and destination bytes 54 | sample_value: '14564' 55 | - name: MissedBytes 56 | type: long 57 | description: bytes that a network sensor or other system/application may have missed 58 | sample_value: '5' 59 | - name: OuterVlanId 60 | type: integer 61 | description: Normally the VLAN can not be determined as source/destination and VLANs are stacked/wrapped. This is the VLAN on the "outside" 62 | sample_value: '160' 63 | - name: Packets 64 | type: long 65 | description: Total packets for the session. If this field does not exist in the log source, then its possible in your ETL pipeline to combine the source and destination packets 66 | sample_value: '143' 67 | - name: Protocol 68 | type: string 69 | description: 'Transport layer in the OSI model. Also known as, IP Protocol. Ex: TCP,UDP,ICMP,ICMP-v6, etc. Convert to lowercase' 70 | sample_value: tcp 71 | - name: SessionId 72 | type: string 73 | description: The session identifier as reported by the network sensor device. Typically, not available for connections. 74 | sample_value: 'S198_13_1_27_12321_D205_13_1_27_443_0012' 75 | - name: Duration 76 | type: integer 77 | description: The amount of time, in millisecond, for the completion of the network session or connection. 78 | sample_value: 1500 79 | - name: IcmpCode 80 | type: integer 81 | description: For an ICMP message, ICMP message type numeric value (RFC 2780 or RFC 4443). 82 | sample_value: 34 83 | - name: IcmpType 84 | type: string 85 | description: For an ICMP message, ICMP message type text representation (RFC 2780 or RFC 4443) 86 | sample_value: 'Destination Unreachable' 87 | - name: RuleName 88 | type: string 89 | description: The name or ID of the rule by which DeviceAction was decided upon 90 | sample_value: AnyAnyDrop 91 | - name: RuleNumber 92 | type: integer 93 | description: Matched rule number 94 | sample_value: 23 95 | references: [] 96 | tags: [] 97 | -------------------------------------------------------------------------------- /schemas/entities/pipe.yml: -------------------------------------------------------------------------------- 1 | name: Pipe 2 | prefix: 3 | - Pipe 4 | id: 7192E569-54E3-4979-89F8-12342AC4B38E 5 | description: Event fields used to define/normalize metadata about pipes being created or connected for inter-process communication locally or remotely. 6 | attributes: 7 | - name: Name 8 | type: string 9 | description: name of pipe created or connected 10 | sample_value: \srvsvc 11 | - name: Instances 12 | type: integer 13 | description: Number of instances of the named pipe 14 | sample_value: '' 15 | - name: MaxInstances 16 | type: integer 17 | description: The maximum number of instances creatable for this pipe 18 | sample_value: '' 19 | - name: Flags 20 | type: string 21 | description: The flags indicating whether this pipe connection is a server or client end, and if the pipe for sending messages or bytes 22 | sample_value: '' 23 | references: [] 24 | tags: [] 25 | -------------------------------------------------------------------------------- /schemas/entities/port.yml: -------------------------------------------------------------------------------- 1 | name: Port 2 | prefix: 3 | - Port 4 | id: 1BF5FD61-BDC1-4E13-84B8-D973BAA3A7E8 5 | description: Event fields used to define/normalize metadata about ports in a network connection. 6 | extends_entities: 7 | - Source 8 | - SourceNat 9 | - Destination 10 | - DestinationNat 11 | attributes: 12 | - name: Number 13 | type: integer 14 | description: Port number used in a network connection. This could be used in the context of source, destination and even NAT when it is provided by an intermediary NAT device such as a firewall. 15 | sample_value: '138' 16 | - name: Name 17 | type: string 18 | description: Name of the port used in a network connection. This is usually determined by IANA common port assignment. Therefore, this means its a guess and NOT actually what the application/ is what the actually. 19 | sample_value: netbios-dgm 20 | references: [] 21 | tags: [] 22 | -------------------------------------------------------------------------------- /schemas/entities/process.yml: -------------------------------------------------------------------------------- 1 | name: Process 2 | prefix: 3 | - Process 4 | - ProcessParent 5 | id: C9573023-9A39-4C94-88BD-B911E3C800A6 6 | description: Event fields used to define metadata about processes in an system. Isolated memory address space that is used to run a program. Inside a processes' address space the system can load code modules, but must have at latest one thread running to do so. 7 | extends_entities: 8 | - Source 9 | - Target 10 | attributes: 11 | - name: Id 12 | type: integer 13 | description: Process unique identifier used by the current operating system to identify a process. 14 | sample_value: 4756 15 | - name: Guid 16 | type: string 17 | description: Process global unique identifer used to identify a process across other operating systems. This can be created by group hashing values such as Process Name, Process Id, Process Start Time, Process Path and even Computer Name. Datasets such as Sysmon call this the ProcessGuid. This is similar to the output from the UUIDGEN command. 18 | sample_value: A98268C1-9C2E-5ACD-0000-0010396CAB00 19 | - name: Name 20 | type: string 21 | description: Name of the process derived from the Image file or executable file used to define the initial code and data mapped into the process' virtual address space. This does not contain the full patth of the Image file. 22 | sample_value: conhost.exe 23 | - name: CurrentDirectory 24 | type: string 25 | description: The full path to the current directory for the process. The string can also specify a UNC path. 26 | sample_value: C:\Users\Panda\Test 27 | - name: CommandLine 28 | type: string 29 | description: Command arguments that were were executed by the process in the endpoint. 30 | sample_value: C:\WINDOWS\system32\conhost.exe 0xffffffff -ForceV1 31 | - name: IntegrityLevel 32 | type: string 33 | description: Integrity label assigned to a process 34 | sample_value: Medium 35 | - name: InjectedAddress 36 | type: string 37 | description: The memory address where the subprocess is injected 38 | sample_value: '0xFFFFBC6422DD9C20' 39 | - name: GrantedAccess 40 | type: string 41 | description: granted access code requested/used to open a target process 42 | sample_value: '0x1000' 43 | - name: CallTrace 44 | type: string 45 | description: Stack trace of where open process is called 46 | sample_value: C:\WINDOWS\SYSTEM32\ntdll.dll+a0344 | C:\WINDOWS\System32\KERNELBASE.dll+64794| c:\windows\system32\lsm.dll+10e93| c:\windows\system32\lsm.dll+f9ea| C:\WINDOWS\System32\RPCRT4.dll+76d23| C:\WINDOWS\System32\RPCRT4.dll+d9390| C:\WINDOWS\System32\RPCRT4.dll+a81c| C:\WINDOWS\System32\RPCRT4.dll+273b4| C:\WINDOWS\System32\RPCRT4.dll+2654e| C:\WINDOWS\System32\RPCRT4.dll+26cfb| C:\WINDOWS\System32\RPCRT4.dll+3083f| C:\WINDOWS\System32\RPCRT4.dll+313a6| C:\WINDOWS\System32\RPCRT4.dll+2d12e| C:\WINDOWS\System32\RPCRT4.dll+2e853| C:\WINDOWS\System32\RPCRT4.dll+5cc68| C:\WINDOWS\SYSTEM32\ntdll.dll+365ce| C:\WINDOWS\SYSTEM32\ntdll.dll+34b46| C:\WINDOWS\System32\KERNEL32.DLL+11fe4| C:\WINDOWS\SYSTEM32\ntdll.dll+6efc1 47 | - name: IsHidden 48 | type: boolean 49 | description: Describes if the process is hidden. 50 | sample_value: "True" 51 | references: 52 | - https://www.hackers-arise.com/post/2017/07/04/reverse-engineering-malware-part-4-windows-internals 53 | - https://docs.microsoft.com/en-us/dotnet/api/system.diagnostics.processstartinfo.filename?view=netcore-3.1 54 | - https://docs.microsoft.com/en-us/windows/win32/api/processthreadsapi/nf-processthreadsapi-createprocessa 55 | tags: [] 56 | -------------------------------------------------------------------------------- /schemas/entities/registry.yml: -------------------------------------------------------------------------------- 1 | name: Registry 2 | prefix: 3 | - Registry 4 | - RegistryPrevious 5 | id: 46295005-AC42-41CA-A45E-4CB669E39030 6 | description: Event fields used to define metadata about Windows registry entries in a system. The registry is a hierarchical database that contains data that is critical for the operation of Windows and the applications and services that run on Windows. The data is structured in a tree format. Each node in the tree is called a key. Each key can contain both subkeys and data entries called values. Sometimes, the presence of a key is all the data that an application requires; other times, an application opens a key and uses the values associated with the key. A key can have any number of values, and the values can be in any form. 7 | attributes: 8 | - name: RootKey 9 | type: string 10 | description: Root-Keys are the root, or primary divisions, of the registry. They do not contain configuration data; they contain the keys, subkeys, and values in which the data is stored. There are six root keys (HKCU, HKU, HKCR, HKLM, HKCC and HKPD) that store information related to currently looged on users, local accounts, performance, and even the current hardware profile. Root-key names represent Windows handles (H) to Keys (K). 11 | sample_value: HKLM or HKEY_LOCAL_MAHINE 12 | - name: HivePath 13 | type: string 14 | description: A hive is a logical group of keys, subkeys, and values in the registry that has a set of supporting files loaded into memory when the operating system is started or a user logs in. 15 | sample_value: HKEY_LOCAL_MACHINE\SAM 16 | - name: KeyPath 17 | type: string 18 | description: Next-level down from registry root-keys. This field contains the full path of a registry key. This is a combination of the root key, hive, key, sub-key, and value. A key is a folder in the registry that contain other sub-keys. 19 | sample_value: HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Run\WardogPersistence 20 | - name: KeyName 21 | type: string 22 | description: This field contains the key name without the full path. Take in consideration the name of the key value in the registry key path. 23 | sample_value: Run 24 | - name: KeyHandleId 25 | type: string 26 | description: This field contains the hexadecimal value of the handle requested to the registry key. 27 | sample_value: '0xa40' 28 | - name: ValueName 29 | type: string 30 | description: Registry values are the lowest-level element in the registry. They appear in the right pane of the registry editor window. Each entry consists of the value name, its Data Types in the Registry (which defines the length and format of data that the entry can store), and a field known as the data of the registry value. These are also known as registry entries. This field contains the key value name without the full registry key path. 31 | sample_value: WardogPersistence 32 | - name: ValueType 33 | type: string 34 | description: values store different kinds of data such as REG_NONE (No value type), REG_SZ (Fixed-length Unicode string), REG_EXPAND_SZ (Variable-length Unicode string that can have embedded environment variables), etc. 35 | sample_value: REG_EXPAND_SZ 36 | - name: ValueData 37 | type: string 38 | description: Each registry key value consists of a value name and its associated data. Registry key value data store the actual configuration data for the operating system and the programs that run on the system. As such, they are different from subtrees, keys, and subkeys, which are merely containers. 39 | sample_value: 'C:\Path\malware' 40 | - name: KeyAccessRights 41 | type: string 42 | description: The Windows security model enables you to control access to registry keys. The valid access rights for registry keys include the DELETE, READ_CONTROL, WRITE_DAC, and WRITE_OWNER standard access rights. Registry keys do not support the SYNCHRONIZE standard access right. 43 | sample_value: KEY_ALL_ACCESS (0xF003F) 44 | - name: EventType 45 | type: string 46 | description: Describes the activity around registry keys and values such as creation, deletion and modification. 47 | sample_value: CreateKey 48 | references: 49 | - https://support.microsoft.com/en-us/help/256986/windows-registry-information-for-advanced-users 50 | - https://docs.microsoft.com/en-us/windows/win32/sysinfo/structure-of-the-registry 51 | - https://docs.microsoft.com/en-us/windows/win32/sysinfo/registry-hives 52 | - https://docs.microsoft.com/en-us/windows/win32/sysinfo/registry-key-security-and-access-rights 53 | - https://docs.microsoft.com/en-us/previous-versions/windows/it-pro/windows-server-2003/cc781906(v=ws.10)?redirectedfrom=MSDN 54 | tags: [] 55 | -------------------------------------------------------------------------------- /schemas/entities/rule.yml: -------------------------------------------------------------------------------- 1 | name: Rule 2 | prefix: 3 | - Rule 4 | id: 37BBC207-8AD2-4DAC-BFE9-B5B77A84CF5E 5 | description: Event fields used to define/normalize metadata about rules. 6 | attributes: 7 | - name: Name 8 | type: string 9 | description: The name or ID of the rule by which DeviceAction was decided upon 10 | sample_value: 'Any Any Drop' 11 | - name: Number 12 | type: string 13 | description: Matched rule number 14 | sample_value: 7 15 | references: [] 16 | tags: [] 17 | -------------------------------------------------------------------------------- /schemas/entities/service.yml: -------------------------------------------------------------------------------- 1 | name: Service 2 | prefix: 3 | - Service 4 | id: 3229C925-3CED-46AA-8B42-EEEDFD9F81A4 5 | description: Event fields used to define/normalize metadata about services or deamons in a network environment. 6 | attributes: 7 | - name: Name 8 | type: string 9 | description: Name of the service 10 | sample_value: EventLog 11 | - name: DisplayName 12 | type: string 13 | description: The display name to be used by user interface programs to identify the service. 14 | sample_value: Windows Event Log 15 | - name: ImagePath 16 | type: string 17 | description: This is the fully rooted path to the file that will execute to start the service. The path can also include arguments for an auto-start service. For example, "d:\myshare\myservice.exe arg1 arg2". These arguments are passed to the service entry point (typically the main function). 18 | sample_value: '%windir%\system32\svchost.exe -k apphost' 19 | - name: Type 20 | type: string 21 | description: Indicates the type of service. 22 | sample_value: '0x20' 23 | - name: StartType 24 | type: string 25 | description: The service start type. 26 | sample_value: '2' 27 | - name: AccountName 28 | type: UnicodeString 29 | description: The security context that the service will run as when started. 30 | sample_value: LocalSystem 31 | - name: HandleId 32 | type: string 33 | description: This field contains the hexadecimal value of the handle requested to the service object. 34 | sample_value: '0x16fa36af2a0' 35 | references: 36 | - https://docs.microsoft.com/en-us/windows/win32/services/about-services 37 | - https://docs.fedoraproject.org/en-US/fedora/latest/system-administrators-guide/infrastructure-services/Services_and_Daemons/ 38 | - https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/deployment_guide/ch-services_and_daemons 39 | - https://docs.nvidia.com/networking-ethernet-software/cumulus-linux-37/System-Configuration/Services-and-Daemons-in-Cumulus-Linux/ 40 | - https://docs.microsoft.com/en-us/windows/win32/api/winsvc/nf-winsvc-createservicea 41 | tags: [] 42 | -------------------------------------------------------------------------------- /schemas/entities/source.yml: -------------------------------------------------------------------------------- 1 | name: Source 2 | prefix: 3 | - Src 4 | id: A72A1C38-7DEF-4E9F-8686-9372C16C76AE 5 | description: Event fields used to define/normalize the source (client) in a network connection event. 6 | attributes: 7 | - name: DomainHostname 8 | type: string 9 | description: The source server, host, hostname, domain, or domain name. Some examples, would include the TLS server name, HTTP Host, DNS Query Name, etc. For information on how to define and use this field refer to the documentation [here](https://ossemproject.com/cdm/guidelines/domain_or_hostname_or_fqdn.html) 10 | sample_value: www.google.com 11 | - name: Bytes 12 | type: integer 13 | description: The number of bytes sent from the source to the destination for the connection or session. 14 | sample_value: '100' 15 | - name: MimeType 16 | type: string 17 | description: Source MIME type as seen in (layer 7) application layer details or as defined by an application scanner such as an anti-virus/EDR. For HTTP this is usually from the server's "Content-Type" header. https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Complete_list_of_MIME_types 18 | sample_value: application/pdf 19 | - name: Packets 20 | type: integer 21 | description: The number of packets sent from the source to the destination for the connection or session. The meaning of a packet is defined by the reporting device. 22 | sample_value: '5' 23 | - name: VlanId 24 | type: integer 25 | description: The Source VLAN ID if it can be determined. Most commonly if from a firewall/switch/router then it can be determined 26 | sample_value: '100' 27 | - name: VlanName 28 | type: string 29 | description: The Source VLAN Name. Most commonly if from a firewall/switch/router then it can be determined 30 | sample_value: management 31 | - name: Zone 32 | type: string 33 | description: The network zone of the source, as defined by the reporting device. 34 | sample_value: 'dmz' 35 | - name: InterfaceGuid 36 | type: string 37 | description: GUID of the network interface which was used for authentication request (if applicable). Most of the time you would use the interface_name field for the uid. 38 | sample_value: 7C202E90-2FBE-4275-AB0E-9BF67E04BEDF 39 | - name: InterfaceName 40 | type: string 41 | description: The network interface used for the connection or session by the source device. 42 | sample_value: eth02 43 | - name: ResourceId 44 | type: string 45 | description: The resource Id of the source device in a network connection 46 | sample_value: '/subscriptions/33333333-8888-4444-a115-aaaaaaaaaaaa/resourcegroups/shokobo/providers/microsoft.compute/virtualmachines/sysmachine2' 47 | - name: ResourceGroup 48 | type: string 49 | description: The ID of the group to which the source device belongs in a network connection. This might be an AWS account, or an Azure subscription or Resource Group 50 | sample_value: 'DatabaseVMs' 51 | references: 52 | - text: Examples of MIME types 53 | link: https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Complete_list_of_MIME_types 54 | tags: [] 55 | -------------------------------------------------------------------------------- /schemas/entities/source_nat.yml: -------------------------------------------------------------------------------- 1 | name: SourceNat 2 | prefix: 3 | - SrcNat 4 | id: 034C3864-6220-475D-98DD-DDE36A88E7CA 5 | description: Event fields used to define the destination NAT (network address translation) in a network connection event. 6 | attributes: [] 7 | references: [] 8 | tags: [] 9 | -------------------------------------------------------------------------------- /schemas/entities/target.yml: -------------------------------------------------------------------------------- 1 | name: Target 2 | prefix: 3 | - Target 4 | id: EDC82376-035A-45CD-9E59-3D53AB5B32FF 5 | description: Event fields used to define entities being targeted by other entities locally in a system. This is different from a network connection event. It is more related to events that involve relationships defined locally by entities such as files, processes,users, etc. 6 | attributes: 7 | - name: ServerName 8 | type: string 9 | description: the name of the server on which the new process was run. Has "localhost" value if the process was run locally. 10 | sample_value: localhost 11 | references: [] 12 | tags: [] 13 | -------------------------------------------------------------------------------- /schemas/entities/threat.yml: -------------------------------------------------------------------------------- 1 | name: Threat 2 | prefix: 3 | - Threat 4 | id: D43449A2-E178-4B04-BAF4-A06B263F93BA 5 | description: Event fields used to define/normalize metadata about threats in an network. 6 | attributes: 7 | - name: Name 8 | type: string 9 | description: The name of the threat or malware identified 10 | sample_value: EICAR Test File 11 | - name: Id 12 | type: string 13 | description: The ID of a threat identified by a security system such as Web Security Gateway of an IPS and is associated with this network session. 14 | sample_value: Tr.124 15 | - name: Category 16 | type: string 17 | description: Trojan The category of a threat identified by a security system such as Web Security Gateway of an IPS and is associated with this network session. 18 | sample_value: Trojan 19 | references: [] 20 | tags: [] 21 | -------------------------------------------------------------------------------- /schemas/entities/tls.yml: -------------------------------------------------------------------------------- 1 | name: Tls 2 | prefix: 3 | - Tls 4 | id: 134FAD61-D073-4D9E-934D-EF7BDFFB1B74 5 | description: 'Event fields used to define/normalize metadata about a TLS(SSL) record. This document is a work in progress, but is a foundational start there is included. Specifically the main foundations of TLS info is already in here' 6 | attributes: 7 | - name: Cipher 8 | type: string 9 | description: The cipher (encryption) parameters used to make the TLS connection 10 | sample_value: TLS_RSA_WITH_AES_128_CBC_SHA 11 | - name: Curve 12 | type: string 13 | description: Elliptic curve the server chose when using ECDH/ECDHE 14 | sample_value: TLS_RSA_WITH_AES_128_CBC_SHA 15 | - name: Established 16 | type: boolean 17 | description: Indicates if the session has been established successfully, or if it was aborted during the handshake 18 | sample_value: 'true' 19 | - name: NextProtocol 20 | type: string 21 | description: Next protocol the server chose using the application layer next protocol extension, if present. 22 | sample_value: spdy/3.1 23 | - name: ServerName 24 | type: string 25 | description: The name of the requested server/destination, this should be copied to dst_host_name 26 | sample_value: www.google.com 27 | - name: Resumed 28 | type: boolean 29 | description: If the session was resumed from previous established connection 30 | sample_value: 'false' 31 | - name: Version 32 | type: string 33 | description: 'Version of TLS/SSL used (ie: SSLv3.0, TLSv1.1, TLSv1.2, TLSv1.3' 34 | sample_value: TLSv10 35 | - name: VersionNumber 36 | type: integer 37 | description: Numeric value of the tls_version 38 | sample_value: '``' 39 | references: [] 40 | tags: [] 41 | -------------------------------------------------------------------------------- /schemas/entities/url.yml: -------------------------------------------------------------------------------- 1 | name: Url 2 | prefix: 3 | - Url 4 | id: 46A0FFC5-91BC-46D9-947B-DF2A10114526 5 | description: 'Event fields used to define/normalize metadata about a URL/URI. There is a lot of ambiguity from the community on the difference URL vs URI. Granted, URL would normally include the domain, port (if applicable), user, password, query, fragment, and URI. 6 | 7 | However, there are many scenarios from log sources where one could not distinguish whether it was the full URL or just the URI.
8 | 9 | URL data can be seen in various log sources as defined in http.md as well as other applications such as SIP. URLs, especially in HTTP, have a best practice implementation however it is not necessary to adhere for connections/data to be established.' 10 | attributes: 11 | - name: Category 12 | type: string 13 | description: 'The defined grouping of a URL (or could be just based on the domain in the URL) related to what it is (ie: adult, news, advertising, parked domains, etc)' 14 | sample_value: Search Engines 15 | - name: Extension 16 | type: string 17 | description: The extension (.dll, .php, zip, .msi, .txt, etc) without the "." 18 | sample_value: php 19 | - name: Fragment 20 | type: string 21 | description: The portion of the URL after the last "#", this is defined in https://tools.ietf.org/html/rfc3986#section-3.5. This is also referred to as the "hash" in some implementations. This value does not always exist 22 | sample_value: '``' 23 | - name: Hostname 24 | type: string 25 | description: The domain/host/hostname of the URL. This could be an IP address or any variation of a value but is more than likely a domain/hostname 26 | sample_value: google.com 27 | - name: Original 28 | type: string 29 | description: The entirety of the URL combined together and or the URL in the truest form from the log source. Some log sources will already parse out portions of the URL into their respective fields. Other logs will even parse out the portions of the URL into their respective field but also include the "original" URL. Always try to include this field, because HTTP/URLs never truly have to conform to any RFC/implementation and thus any parsing/logging implementation could have any number of assumptions/mistakes - therefore it is best to keep a original value 30 | sample_value: ftp://BigwheelPassword:BigwheelBobUser@google.com:8088/common/Current/client/search/greatsearch.php?hash=215696fc36392ca70f89228b98060afb%20processname=example.exe#gid=l1k4h 31 | - name: Path 32 | type: string 33 | description: Everything beginning with and after the first "/". This portion should usually exist in the log source / URL.. Even if the path is just "/". Also, even if the query or fragment have not been parsed yet then you still include them in this value 34 | sample_value: '``' 35 | - name: Port 36 | type: integer 37 | description: The port in the URL. This is not to be confused with destination.md. In your ETL pipeline you should check if the value derived from the URL is actually an integer (unless properly verified in the data source)... because as mentioned throughout, URLs can be manipulated/mis-implemented in many different ways 38 | sample_value: '8088' 39 | - name: QueryNames 40 | type: string 41 | description: 'The keys/fields derived from the query. Due to the limitless variations of implementations of a URL, providing a nested object of key/values is not recommened. Whether an attacker is injecting data into a URL or an incorrect implementaiton or malicious implementation - it''s possible you could have keys/fields of values of anything you can imagine (ie: "%*%)%*#Nf..$2f>hr..n fa.fa s\\\\\"\\jhrwq": "somevalue"' 42 | sample_value: '[ "hash" , "processname" ]' 43 | - name: QueryValues 44 | type: string 45 | description: 'The values derived from the query. Due to the limitless variations of implementations of a URL, providing a nested object of key/values is not recommened. Whether an attacker is injecting data into a URL or an incorrect implementaiton or malicious implementation - it''s possible you could have keys/fields of values of anything you can imagine (ie: "%*%)%*#Nf..$2f>hr..n fa.fa s\\\\\"\\jhrwq": "somevalue"' 46 | sample_value: '[ "215696fc36392ca70f89228b98060afb", "example.exe" ]' 47 | - name: Scheme 48 | type: string 49 | description: 'Defines the network location (ie: smtp, ftp, smb, ldap, etc). This portion may not exist in many log sources. The is usually the value that comes before the first "://". This is also referred to as URN/origin' 50 | sample_value: ftp 51 | - name: UserName 52 | type: string 53 | description: The username defined in the URL. This is meant to be distinguished from something such as the value in the Authorization header in an HTTP request (or even the Proxy Authentication HTTP header). This value should be copied to any.md 54 | sample_value: BigwheelBobUser 55 | - name: UserPassword 56 | type: string 57 | description: The password defined in the URL. This is meant to be distinguished from something such as the value in the Authorization header in an HTTP request (or even the Proxy Authentication HTTP header) 58 | sample_value: BigwheelPassword 59 | references: 60 | - text: HTTP 61 | link: http.md 62 | - text: RFC-3986 63 | link: https://tools.ietf.org/html/rfc3986#section-3.5 64 | tags: [] 65 | -------------------------------------------------------------------------------- /schemas/entities/user.yml: -------------------------------------------------------------------------------- 1 | name: User 2 | prefix: 3 | - User 4 | id: CA1218A4-D188-4B95-8793-882A58AA3F89 5 | description: Event fields used to define/normalize metadata about users in a network environment. 6 | extends_entities: 7 | - Source 8 | - Destination 9 | - Target 10 | attributes: 11 | - name: Name 12 | type: string 13 | description: Name of the user associated with the main event (i.e. Network session). There could be a sense of direction depending how it is used together with other entities (i.e. src_user_name or dst_user_name) 14 | sample_value: wardog 15 | - name: Domain 16 | type: string 17 | description: The domain or computer name associated to the user in a session. In active directory, this would be the name of the domain the user belongs to. 18 | sample_value: CONTOSO 19 | - name: SessionId 20 | type: integer 21 | description: ID of the session the user belongs to. 22 | sample_value: '1' 23 | - name: ReporterSid 24 | type: string 25 | description: SID of account that reported information about the main event 26 | sample_value: S-1-5-18 27 | - name: ReporterName 28 | type: string 29 | description: the name of the account that reported information about the main event 30 | sample_value: WIN-GG82ULGC9GO$ 31 | - name: ReporterDomain 32 | type: string 33 | description: domain name of the user that reported the main event 34 | sample_value: WORKGROUP 35 | - name: ReporterId 36 | type: integer 37 | description: unique identifier of the user that reported the main event 38 | sample_value: '0x3e7' 39 | - name: Sid 40 | type: string 41 | description: Security identifier of the user. Typically, the identity used to authenticate a server. 42 | sample_value: S-1-5-21-1377283216-344919071-3415362939-500 43 | - name: NetworkAccountName 44 | type: string 45 | description: User name used for outbound (network) connections 46 | sample_value: '-' 47 | - name: NetworkAccountDomain 48 | type: string 49 | description: Domain for the user that will be used for outbound (network) connections. 50 | sample_value: '-' 51 | - name: LinkedLogonId 52 | type: integer 53 | description: A hexadecimal value of the paired logon session. 54 | sample_value: '0x0' 55 | - name: SidList 56 | type: string 57 | description: the list of special group SIDs, which New Logon\Security ID is a member of. 58 | sample_value: '{S-1-5-21-3457937927-2839227994-823803824-512}' 59 | - name: SecurityPackage 60 | type: string 61 | description: the name of Security Package used during an authentication event. 62 | sample_value: CREDSSP 63 | - name: Upn 64 | type: string 65 | description: In Active Directory, the User Principal Name (UPN) attribute is a user identifier for logging in, separate from a Windows domain login. 66 | sample_value: dadmin@contoso 67 | - name: CredType 68 | type: string 69 | description: types of credentials which were presented for delegation 70 | sample_value: '%%8098' 71 | - name: Identity 72 | type: string 73 | description: User Principal Name (UPN) or another type of account identifier for which 802.1x authentication request was made. 74 | sample_value: host/XXXXXXXX.redmond.corp.microsoft.com 75 | - name: Password 76 | type: string 77 | description: User password if seen in the request. Commonly seen in network logs and authentication proxy/logs. 78 | sample_value: bobspassword 79 | - name: Aadid 80 | type: string 81 | description: The User Azure AD ID of the identity associated with a cloud network session. It applies to source and destination entities. 82 | sample_value: '5e8b0f4d-2cd4-4e17-9467-b0f6a5c0c4d0' 83 | - name: AccessList 84 | type: UnicodeString 85 | description: List of access rights which were requested or used by the user when interacting with objects such as registry keys and services. These access rights depend on the object type. 86 | sample_value: '%%1538 %%1541 %%4416 %%4417 %%4418 %%4419 %%4420 %%4423 %%4424' 87 | references: [] 88 | tags: [] 89 | -------------------------------------------------------------------------------- /schemas/entities/user_agent.yml: -------------------------------------------------------------------------------- 1 | name: UserAgent 2 | prefix: 3 | - UserAgent 4 | id: 219C796B-99A5-4CE3-B381-6B74EDDCF5F9 5 | description: Event fields used to define/normalize metadata related to user agents in http requests. 6 | extends_entities: 7 | - Http 8 | attributes: 9 | - name: Original 10 | type: string 11 | description: The User agent seen in the HTTP request. 12 | sample_value: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/74.0.3729.169 Safari/537.36 13 | references: [] 14 | tags: [] 15 | -------------------------------------------------------------------------------- /schemas/entities/x509_and_certificates.yml: -------------------------------------------------------------------------------- 1 | name: Certificate 2 | prefix: 3 | - Certificate 4 | id: 0CBE0892-F642-4F26-A31D-636386749B05 5 | description: This document is a work in progress, but is a foundational. Specifically the main foundations of certificate information is already in here. 6 | extends_entities: 7 | - Destination 8 | attributes: 9 | - name: Issuer 10 | type: string 11 | description: Information about the CA that issued the certificate 12 | sample_value: CN=neu5ron.local,OU=Admin 13 | - name: Subject 14 | type: string 15 | description: Information about the CA that issued the certificate 16 | sample_value: CN=natetoken,OU=Admin,DC=neu5ron,DC=local 17 | - name: SerialNumber 18 | type: string 19 | description: Serial number, this is chosen by the CA (certificate authority) which issued the certificate. Therefore this can relatively be arbritary if the CA does not follow a standard or is malicious. 20 | sample_value: '5157550' 21 | references: [] 22 | tags: [] 23 | -------------------------------------------------------------------------------- /schemas/tables/network_session.yml: -------------------------------------------------------------------------------- 1 | name: network_session 2 | id: 189BC2EE-44BF-4A8A-A257-5521C67D457B 3 | version: 1.0.0 4 | description: Event fields used to normalize data related to network sessions. 5 | entities: 6 | - cloud 7 | - name: device 8 | prefix: 9 | - dvc 10 | attributes: 11 | - action 12 | - inbound_interface 13 | - outbound_interface 14 | - hostname 15 | - ip_addr 16 | - mac_addr 17 | - domain 18 | - fqdn 19 | - interface_guid 20 | - interface_name 21 | - name: destination 22 | prefix: 23 | - dst 24 | attributes: 25 | - bytes 26 | - dvc_hostname 27 | - dvc_fqdn 28 | - dvc_ip_addr 29 | - dvc_mac_addr 30 | - dvc_domain 31 | - interface_name 32 | - interface_guid 33 | - ip_addr 34 | - geo_city 35 | - geo_country 36 | - geo_region 37 | - geo_latitude 38 | - geo_longitude 39 | - mac_addr 40 | - domain_hostname 41 | - port_number 42 | - packets 43 | - resource_id 44 | - user_aadid 45 | - user_sid 46 | - user_name 47 | - user_upn 48 | - user_domain 49 | - zone 50 | - name: destination_nat 51 | prefix: 52 | - dst_nat 53 | attributes: 54 | - ip_addr 55 | - port_number 56 | - name: source 57 | prefix: 58 | - src 59 | attributes: 60 | - bytes 61 | - dvc_hostname 62 | - dvc_fqdn 63 | - dvc_ip_addr 64 | - dvc_mac_addr 65 | - dvc_domain 66 | - dvc_os 67 | - dvc_model_name 68 | - dvc_model_number 69 | - dvc_type 70 | - interface_name 71 | - interface_guid 72 | - ip_addr 73 | - geo_city 74 | - geo_country 75 | - geo_region 76 | - geo_latitude 77 | - geo_longitude 78 | - mac_addr 79 | - port_number 80 | - packets 81 | - resource_id 82 | - user_aadid 83 | - user_sid 84 | - user_name 85 | - user_upn 86 | - user_domain 87 | - zone 88 | - name: source_nat 89 | prefix: 90 | - src_nat 91 | attributes: 92 | - ip_addr 93 | - port_number 94 | - name: http 95 | prefix: 96 | - http 97 | attributes: 98 | - version 99 | - request_time 100 | - response_time 101 | - request_method 102 | - status_code 103 | - content_type 104 | - referrer_original 105 | - request_xff 106 | - user_agent_original 107 | - name: file 108 | prefix: 109 | - file 110 | attributes: 111 | - name 112 | - path 113 | - extension 114 | - mime_type 115 | - size 116 | - hash_sha1 117 | - hash_md5 118 | - hash_sha256 119 | - hash_sha512 120 | - name: url 121 | prefix: 122 | - url 123 | attributes: 124 | - category 125 | - original 126 | - hostname 127 | - name: network 128 | prefix: 129 | - network 130 | attributes: 131 | - application_protocol 132 | - bytes 133 | - direction 134 | - duration 135 | - packets 136 | - session_id 137 | - protocol 138 | - icmp_code 139 | - icmp_type 140 | - rule_name 141 | - rule_number 142 | - name: event 143 | prefix: 144 | - event 145 | attributes: 146 | - count 147 | - message 148 | - start_time 149 | - end_time 150 | - severity 151 | - original_uid 152 | - time_ingested 153 | - uid 154 | - vendor 155 | - product 156 | - product_version 157 | - resource_id 158 | - report_url 159 | - result 160 | - result_details 161 | - schema_version 162 | - type 163 | - sub_type 164 | - name: threat 165 | prefix: 166 | - threat 167 | attributes: 168 | - category 169 | - id 170 | - name --------------------------------------------------------------------------------