Empty fields in OAT event
This issue may be caused by the following:
- OAT Event coming from a variety of endpoints
- Syslog parsing raw_log field in OAT event following a CEF mapping schema
You may find the latest CEF mapping schema on Online Help article, CEF Observed Attack Techniques Logs.
The CEF mapping schema is globally applied to all endpoint events. However, events from different endpoints may not fulfill all the CEF fields in raw_log field.
- Endpoint events not being able to fulfill the complete set of CEF mapping fields
To illustrate an example of an OAT event with a complete set of mapping fields, refer to the following OAT event reported by Deep Security:
- raw_log.pname: "Trend Micro Deep Security"
- raw_log.productCode: "sds"
In this event, the following fields will be mapped to CEF fields in Syslog.
Event JSON Path CEF Field CEF Field Description raw_log.src src Source IP raw_log.spt spt Port of "src" raw_log.dst dst Destination IP raw_log.dpt dpt Port of "dst" { "source": "detection", "id": "eyJzdGFydCI6ICIyMDIzLTAxLTE3VDE1OjM5OjQzWiIsICJlbmQiOiAiMjAyMy0wMS0yNFQxNTozOTo0M1oiLCAic291cmNlX3R5cGUiOiAiZGV0ZWN0aW9uIiwgInV1aWQiOiAiYjEwMmE1NGMtNGI4NS00OGQ4LTkzMWMtYTM4YTc4ZTM0YWYwIn0=", "detection_time": "2023-01-20T07:41:39Z", "level": "high", "name": "PHP File Uploaded To Web Server", "description": "Detect attempts to upload a PHP file remotely", "tactics": [ "TA0011" ], "techniques": [ "T1105" ], "filters": [ // ... ], "endpoint": { "name": "win-jbsbdm3vs8o.client.tw.trendnet.org", "guid": "e0f49a6c-c5ff-f480-e57b-313dacd5030c", "ips": [ "10.1.145.165" ] }, "entity": { "type": "endpoint", "displayName": "win-jbsbdm3vs8o.client.tw.trendnet.org(10.1.145.165)", "details": { "endpointHostName": "win-jbsbdm3vs8o.client.tw.trendnet.org", "endpointGUID": "e0f49a6c-c5ff-f480-e57b-313dacd5030c", "endpointIp": [ "10.1.145.165" ] } }, "raw_log": { "endpointHostName": "win-jbsbdm3vs8o.client.tw.trendnet.org", "endpointIp": [ "10.1.145.165" ], "src": [ "10.1.145.165" ], "spt": 50008, "dst": [ "204.79.197.200" ], "dpt": 80, "tags": [ "XSAE.SDS-100115-1005434", "XSAE.F2246", "mitre.t1105", "mitrev8.t1105", "MITRE.T1105", "MITREV9.T1105", "mitrev9.t1105" ], "act": [ "Detect Only: Reset" ], "eventId": "100115", "eventName": "DEEP_PACKET_INSPECTION_EVENT", "eventTime": "2023-01-20T07:41:39Z", "eventTimeDT": "2023-01-20T07:41:39Z", "filterRiskLevel": "high", "instanceId": "i-5566952700", "interestedIp": [ "10.1.145.165" ], "pname": "Trend Micro Deep Security", "productCode": "sds", "proto": "TCP", "protoFlag": "ACK PSH DF=1", "pver": "12.0.0.360", "remarks": "{\"note\":\"\"IE\"\"}", "rt": "2023-01-20T07:41:39+00:00", "rtDate": "2023-01-20", "rtHour": 7, "rtWeekDay": "Friday", "rt_utc": "2023-01-20T07:41:39Z", "ruleId": 1005434, "senderGUID": "D112B291-5846-4522-B0F7-BFB1459FF0FE", "severity": 4 } }
To illustrate an example of an OAT event with an incomplete set of mapping fields, refer to the following OAT event reported by Apex One:- raw_log.pname: "Apex One"
- raw_log.productCode": "sao"
In this event, below fields are missing, therefore this CEF fields will be EMPTY in Syslog.
- raw_log.src
- raw_log.spt
- raw_log.dst
- raw_log.dpt
{ "source": "detection", "id": "eyJzdGFydCI6ICIyMDIzLTAxLTE3VDE1OjM5OjQzWiIsICJlbmQiOiAiMjAyMy0wMS0yNFQxNTozOTo0M1oiLCAic291cmNlX3R5cGUiOiAiZGV0ZWN0aW9uIiwgInV1aWQiOiAiYjI5ZTY1MDUtYWRlOC00Y2NlLWJjMmMtNGVmY2U4MmU0NDBkIn0=", "detection_time": "2023-01-20T02:30:04Z", "level": "high", "name": "File Detections in Windows Directory - Blocked", "description": "A file under windows directory is detected, which can significantly cause security risk on the endpoint, and Trend Micro product successfully taken action.", "tactics": [], "techniques": [], "filters": [ // ... ], "endpoint": { "name": "LAB-LUWAK-1047", "guid": "feb01d39-5125-43da-b30d-8d2cee8ebbb2", "ips": [ "10.209.14.34" ] }, "entity": { "type": "endpoint", "displayName": "LAB-LUWAK-1047(10.209.14.34)", "details": { "endpointHostName": "LAB-LUWAK-1047", "endpointGUID": "feb01d39-5125-43da-b30d-8d2cee8ebbb2", "endpointIp": [ "10.209.14.34" ] } }, "raw_log": { "endpointHostName": "LAB-LUWAK-1047", "endpointIp": [ "10.209.14.34" ], "channel": "Local file or network drive", "tags": [ "XSAE.F2714", "THREAT.MALWARE", "XSAE.F3388", "XSAE.F3066" ], "eventSubName": "Virus", "fileName": [ "eicar" ], "fileHash": "90663ffcc77adaa0626dec76ffd75669bc562144", "fullPath": "C:\\Windows\\System32\\eicar", "malName": "Eicar_test_1", "actResult": [ "File quarantined" ], "scanType": "Real-time Scan", "deviceGUID": "fcabcd65-ceeb-45e3-8ca1-cc4678fd81aa", "domainName": "Workgroup", "dvchost": "CU-6195-2", "endpointGUID": "feb01d39-5125-43da-b30d-8d2cee8ebbb2", "endpointMacAddress": "00-50-56-89-36-6D", "engType": "Virus Scan NT Kernel Engine", "engVer": "22.580.1004", "eventId": "100100", "eventName": "MALWARE_DETECTION", "eventTime": "2023-01-20T02:30:04Z", "eventTimeDT": "2023-01-20T02:30:04Z", "filePath": "C:\\Windows\\System32\\", "filterRiskLevel": "high", "firstAct": "Clean", "firstActResult": "Unable to clean file", "interestedIp": [ "10.209.14.34" ], "logKey": "002248497494-63865CDC-04DE-CD41-014E_20", "mDevice": [ "10.0.0.4", "fe80::64e5:abc6:2816:28db" ], "mDeviceGUID": "00224849-7494-6386-5CDC-04DECD41014E", "malDst": "LAB-LUWAK-1047", "malSubType": "Unknown", "malType": "Virus/Malware", "mpname": "Apex Central", "mpver": "2019.6288", "pComp": "Scan Module", "patVer": "1814100", "pname": "Apex One", "productCode": "sao", "pver": "14.0", "rt": "2023-01-20T02:30:04+00:00", "rtDate": "2023-01-20", "rtHour": 2, "rtWeekDay": "Friday", "rt_utc": "2023-01-20T02:30:04Z", "ruleName": "Virus found in file", "secondAct": "Move", "secondActResult": "File quarantined", "senderGUID": "00224849-7494-6386-5CDC-054AE1B9B466", "senderIp": [ "10.0.0.4", "fe80::64e5:abc6:2816:28db" ], "severity": 4, "uuid": "b29e6505-ade8-4cce-bc2c-4efce82e440d" } }
Duplicate entries for the same Event ID
You may see multiple OAT event entries with the same ID in Syslog. This is because an OAT event may contain multiple objects in field filters. Syslog will split one OAT event into multiple entries for each object in filters.
As an example below, the OAT event has 2 objects in filters. This event will be split into 2 entries in Syslog, each with same id but different unique_id.
{ "source": "detection", "id": "eyJzdGFydCI6ICIyMDIzLTAxLTE3VDE1OjM5OjQzWiIsICJlbmQiOiAiMjAyMy0wMS0yNFQxNTozOTo0M1oiLCAic291cmNlX3R5cGUiOiAiZGV0ZWN0aW9uIiwgInV1aWQiOiAiYjEwMmE1NGMtNGI4NS00OGQ4LTkzMWMtYTM4YTc4ZTM0YWYwIn0=", "detection_time": "2023-01-20T07:41:39Z", "level": "high", "name": "PHP File Uploaded To Web Server", "description": "Detect attempts to upload a PHP file remotely", "tactics": [ "TA0011" ], "techniques": [ "T1105" ], "filters": [ { "id": "F2246", "unique_id": "c9a3a697-1559-4726-abdf-852a2ec015d6", "level": "high", "name": "PHP File Uploaded To Web Server", "description": "Detect attempts to upload a PHP file remotely", "tactics": [ "TA0011" ], "techniques": [ "T1105" ], "highlightedObjects": [ { "field": "src", "master": true, "type": "ip", "value": [ "10.1.145.165" ] }, { "field": "dst", "type": "ip", "value": [ "204.79.197.200" ] }, { "field": "act", "type": "text", "value": [ "Detect Only: Reset" ] } ] }, { "id": "SDS-100115-1005434", "unique_id": "2b0fda06-abde-5427-a687-004124bb15f1", "level": "low", "name": "Disallow Upload Of A PHP File", "description": "Detected a PHP file being uploaded.", "tactics": [ "TA0011" ], "techniques": [ "T1105" ], "highlightedObjects": [ { "field": "interestedHost", "master": true, "type": "host", "value": "win-jbsbdm3vs8o.client.tw.trendnet.org" }, { "field": "src", "type": "ip", "value": [ "10.1.145.165" ] }, { "field": "dst", "type": "ip", "value": [ "204.79.197.200" ] }, { "field": "ruleName", "type": "detection_name", "value": "Disallow Upload Of A PHP File (ATT&CK T1105)" }, { "field": "processName", "type": "text", "value": "C:\\Windows\\System32\\svchost.exe" }, { "field": "act", "type": "text", "value": [ "Detect Only: Reset" ] }, { "field": "spt", "type": "port", "value": 50008 }, { "field": "dpt", "type": "port", "value": 80 } ] } ], "endpoint": { "name": "win-jbsbdm3vs8o.client.tw.trendnet.org", "guid": "e0f49a6c-c5ff-f480-e57b-313dacd5030c", "ips": [ "10.1.145.165" ] }, "entity": { "type": "endpoint", "displayName": "win-jbsbdm3vs8o.client.tw.trendnet.org(10.1.145.165)", "details": { "endpointHostName": "win-jbsbdm3vs8o.client.tw.trendnet.org", "endpointGUID": "e0f49a6c-c5ff-f480-e57b-313dacd5030c", "endpointIp": [ "10.1.145.165" ] } } } From