Use case:
Detect if the parsed logs contain single or multiple warning messages then add a field stating both cases.
Logstash configuration & testing:
Suppose that we have the following log files representing both cases described above:
[root@localhost logstashfilter]# cat multivaluewarn.json
{"waf": {"ver": "2.0","warnRules": "3000030;3000057;950001;950109;959073;973335;981173;981244;981318","denyMsg": "Anomaly Score Exceeded for SQL Injection","denyActions": "3","warnMsg": "Basic SQL Authentication Bypass Attempts 3/3;Cross-site Scripting (XSS) common keywords;SQL Injection Attack;Multiple URL Encoding Detected;SQL Injection Attack;IE XSS Filters - Attack Detected;Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded;Basic SQL Authentication Bypass Attempts 1/3;SQL Injection Attack: Common Injection Testing Detected"}}
[root@localhost logstashfilter]# cat singlevaluewarn.json
{"waf": {"ver": "2.0","warnRules": "681984","policy": "api_89894","warnMsg": "Alert rq without DEVICEID header","warnTags": "DEVICEID_Detection","warnActions": "2"}}Reading the logs we can see that the field [waf][warnMsg] separates the warning messages using a semi-colon ; in the case of multiple warnings.
Translating the gathered information into a Logstash pipeline would result in:
input {
stdin { codec => json }
}
filter {
if ";" in [waf][warnMsg]{
mutate {
add_field => [ "wafWarningMSG", "multi warnings" ]
}
}
else {
mutate {
add_field => [ "wafWarningMSG", "single" ]
}
}
}Add the pipeline to a conf file (called it warningPipeline.conf ), then test and view the result using the Logastash binary with /usr/share/logstash/bin/logstash -f warningPipeline.config < multivaluewarn.json
The output shows a new field called wafWarningMSG with value multi warnings :
{
"@timestamp" => 2022-09-20T13:46:13.355Z,
"@version" => "1",
"host" => "localhost.localdomain",
"waf" => {
"denyActions" => "3",
"ver" => "2.0",
"warnMsg" => "Basic SQL Authentication Bypass Attempts 3/3;Cross-site Scripting (XSS) common keywords;SQL Injection Attack;Multiple URL Encoding Detected;SQL Injection Attack;IE XSS Filters - Attack Detected;Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded;Basic SQL Authentication Bypass Attempts 1/3;SQL Injection Attack: Common Injection Testing Detected",
"warnRules" => "3000030;3000057;950001;950109;959073;973335;981173;981244;981318",
"denyMsg" => "Anomaly Score Exceeded for SQL Injection"
},
"wafWarningMSG" => "multi warnings"
}Hope you find it useful and feel free to reach out for any questions 😀