Confidence metadata tag and its impact & meaning

Taken from a great feedback interaction from one of our users, I thought I’d share @bmurphy’s response on the Confidence metadata tag we’ve recently introduced. The user asked after whether the tag contents meant whether it meant a low-FP rate (the rule is firing on what it’s meant to fire on) or whether it was a measure in confidence in the trigger’s maliciousness. Brandon’s response:

I would say that both (low FPs and high quality) are true depending on what the rule is designed to alert on.

Consider the “High” confidence of the following rule:

> 2044374 - ET HUNTING Likely Hex Encoded Executable as String - Double Quote Separated
I applied “High” confidence to this rule to indicate that I am very sure, if it alerts, there is a Double Quote Separated PE file being delivered. However, I lack context to determine if this traffic is malicious or benign. Hence the rule is in “HUNTING” category.

Now consider “High” of the following rules:

> 2044379 - ET MALWARE ReverseRat 3.0 CnC Checkin M1

> 2044380 - ET MALWARE ReverseRat 3.0 CnC Checkin M2

Here I applied “High” because I’m very confident it’s firing on ReverseRAT traffic. These rules are based on the observed RC4 encryption keys and known plain text. Here we have high quality intel and low FPs.

Ok, last one example, same malware family of rules but with a medium confidence.

> 2853606 - ETPRO MALWARE ReverseRAT Activity (POST) - Generic

This rule is designed to more generically detect ReverseRAT and does not depend on observed RC4 keys and the known plain text. Instead this matches a rather terse POST request with HTTP headers in a specific order. This pattern persisted over multiple versions of ReverseRAT. However, I would call this detection logic a bit “loose” and it is very possible that it will FP. So I reduced my confidence down to medium. Today I checked in on this rule and sure enough, it had some FPs. A tune will be going out today to eliminate them. If after some time I feel like this rule is not producing FPs and the “Signal to Noise Ratio” is in a good place, I might bump the confidence up to High.

Hopefully these real world examples help understand how we apply and use the confidence tag.

Also, if you see FPs or disable a rule because it’s too noisy, etc. please let us know! I beg you for more feedback. Sometimes we’ll say it’s a local tune or disabling it might be a good choice for your environment, other times we can tune the rule and make it better for everyone. Either way it’s insight that we wouldn’t otherwise have and will and does have guide how we create and manage the ruleset.

That last is particularly important! You all are extra sets of eyes (and IDS sensors!) out there for us - every bit of feedback we receive can help us put out the best rulesets we can!

1 Like