Translating Suricata http.header_names content to Snort

[bingohotdog]

Emerging Threats aims to provide detection for Suricata and Snort users with its rulesets. Today, the rulesets supports Suricata (4.0.5 and 5.0.0) and Snort (2.9.17).

For myself, I can say that I write rules in Suricata and then translate them to Snort. In this post, I cover how you might tackle translating Suricata http.header_names content to Snort http_header/http_raw_header content.

What are HTTP Headers?
HTTP Headers are used to send additional data during an HTTP request or response. Google Chrome’s Developer blog post shows how you could see the HTTP headers that appear as you use your browser:

Notice how in this image, an HTTP Response has completed and that additional HTTP Headers appear after the General section.

Why are HTTP Headers important for detection?
Adding HTTP Headers and their values can sharpen your signatures! Here are two situations to consider:

• Suppose a malware variant consistently sends GET Requests along with Content-Type: 42069. You could create a signature that targets this content and confidently identify this malware variant going forward!

• Suppose a malware variant has a particular pattern with this HTTP Header e.g. it has an odd, but expected pattern like Referer is never present and that only Host and User-Agent headers appear. You could target this pattern.

Why use Suricata’s http.header_names?
In Suricata 5+, http.header_names was implemented. It helps with creating detection for pattern scenario above. Nice Job, Suricata!

As far as I know, Snort does not have this. I’ll review the pain related to translating this later in the post…

More on http.header_names

The documentation states that “[it] inspect[s] a buffer only containing the names of the HTTP headers. Useful for making sure a header is not present or testing for a certain order of headers. Buffer starts with a \r\n and ends with an extra \r\n.”

Remember, a buffer is like another word of data. And so, Suricata will inspect data that contains or does not contain certain HTTP Header strings. In this data, the buffer starts with a \r\n and ends with an extra \r\n. (Another way to reference \r\n is to say CRLF (Carriage Return Line Feed) which is frequently used to denote the end of a line.)

And so, if you give Suricata the sample HTTP Response seen in the Google Chrome’s Developer blog post, Suricata would parse the buffer/data as…

\r\naccept-ranges: bytes\r\ncache-control: public, max-age=0\r\n...etag: ...\r\n\r\n

#Notice how the content is in one line!

Let’s look at the example provided in the Suricata docs:

Example Buffer

\\r\\nHost\\r\\n\\r\\n

Example Alert
alert http any any -> any any (http.header_names; content:"|0d 0a|Host|0d 0a|"; sid:1;)

#Notice how \r\n has been replaced with their hex equivalent 0d 0a!

[bingohotdog]

Translation Notes between Suricata Rules to Snort Rules
Here are some initial consideration notes when translating Suricata rules using http.header_names.

• Does Snort understand http.header_names? No, Snort will reject the rule.

• What Snort keyword should I use instead? Use http_header or http_raw_header AND PCRE. Without a PCRE, neither of these buffers can mimic similar as Suricata’s http.header_names.

  • http_header (extracted NORMALIZED Header, 3.5 Payload Detection Rule Options
  • http_raw_header (extracted UNNORMALIZED Header, 3.5 Payload Detection Rule Options

It should be noted that Suricata implements its own version of http_headers. It varies from Snort’s implementation somewhat, see: 6.36. Differences From Snort — Suricata 6.0.1 documentation.

Please keep in mind for the scope of this post, we are comparing Suricata’s http.header_names vs Snort’s http_header/http_raw_header + PCRE only.

[bingohotdog]

Let’s proceed with creating a Suricata rule that uses http.header_names and then translate that rule to Snort.

Here is the sample PCAP for this exercise: discourse_example.pcap (1.2 KB). I must admit, this sample isn’t the most exciting, but it will get the point across in this lesson.

Sample Analysis
The sample contains a malicious HTTP POST request and headers.

POST /Home/0a09c8844ba8f0936c20bd791130d6b6/actions/auth.php HTTP/1.1
Accept-Encoding: identity
Content-Type: application/x-www-form-urlencoded
Referer: https://netflixacc-logon.anondns.net/Home/
Content-Length: 23
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:82.0) Gecko/20100101 Firefox/82.0
Host: netflixacc-logon.anondns.net
Cookie: PHPSESSID=972cd8ffbcc9acb9bc6726083ca63734
Connection: close

login_user=&login_pass=

Creating a Suricata Rule
With our eyes, we see the sample has 8 header names present….

Accept-Encoding
Content-Type
Referer
Content-Length
User-Agent
Host
Cookie
Connection

If you remember in a previous post, Suricata will parse the HTTP buffer for HTTP Header names. It will append \r\n to the start and \r\n\rn to the end of the the buffer.

With our Suricata hat on, we know there are \r\n present! With a quick blink you should know see…

\r\nAccept-Encoding
Content-Type\r\n
Referer\r\n
Content-Length\r\n
User-Agent\r\n
Host\r\n
Cookie\r\n
Connection\r\n\r\n #Note the extra \r\n to signal the end of buffer!

Ok, technically, all these headers are in one line, but I just wanted to visually help people understand what’s going on.

Let’s tidy these headers by replacing \r\n with their hex equivalent |0d 0a| and putting these headers in one line.

content:”|0d 0a|Accept-Encoding|0d 0a|Content-Type|0d 0a|Referer|0d 0a|Content-Length|0d 0a|User-Agent|0d 0a|Host|0d 0a|Cookie|0d 0a|Connection|0d 0a 0d 0a|”;

Here’s our rule.

alert http $HOME_NET any → $EXTERNAL_NET any (msg:“Rule with header_names in one content”; flow:established,to_server; http.method; content:“POST”; http.header_names; content:“|0d 0a|Accept-Encoding|0d 0a|Content-Type|0d 0a|Referer|0d 0a|Content-Length|0d 0a|User-Agent|0d 0a|Host|0d 0a|Cookie|0d 0a|Connection|0d 0a 0d 0a|”; classtype:bad-unknown; sid:1; rev:1;)

BTW, we could have split these headers into individual contents like and could have created a rule like

alert http $HOME_NET any → $EXTERNAL_NET any (msg:“Rule where header_names is split to multiple contents”; flow:established,to_server; http.method; content:“POST”; http.header_names; content:“Accept-Encoding”; content:“Content-Type”; distance:0; content:“Referer”; distance:0; content:“Content-Length”; distance:0; content:“User-Agent”; distance:0; content:“Host”; distance:0; content:“Cookie”; distance:0; content:“Connection”; distance:0; classtype:bad-unknown; sid:2; rev:1;)

This rule does match on a buffer that contains these headers…but it doesn’t match the same intention as the other rule. We wanted an EXACT match of all the headers.

Instead, this rule’s will match any buffer that contains these headers.

http.header_names; content:“Accept-Encoding”;

This could match on buffer’s with “Accept-Encoding” and “X-Accept-Encoding”. Did you want this loose match or did you want an exact one?

(In my experience, this ^ variation performs poorly in comparison to header_names values in one content. So, if you’re able to do an exact match, opt for that.)

[bingohotdog]

Suricata to Snort Translation of http.header_names.

Now, let’s translate sid:1 to Snort.

Consider sid:1, without translating the http.header_names content, we have the following template…

alert tcp $HOME_NET any → $EXTERNAL_NET $HTTP_PORTS (flow:established,to_server; content:“POST”; http_method; classtype:bad-unknown; sid:1; rev:1;)

Remember, Snort doesn’t have http.header_names, but has http_header and http_raw_header.

In the Snort docs, they provide an example as to how http_header is expected to be used:

http_header; 
content:"User-Agent: abcip",fast_pattern,nocase;
content:"Accept-Language: en-us",nocase,distance 0;

Snort’s http_header/http_raw_header expects the HTTP Header name and value!

Suricata’s http.header_names does not expect any values! Only the header names.

Also, there is also no documented usage of \r\n, we will need to review Snort Hexdumps to undertand where they are.

To emphasize the above point, this rule, which wants to do an exact match on header names, will not translate 1:1.

alert tcp $HOME_NET any → $EXTERNAL_NET $HTTP_PORTS (flow:established,to_server; content:“POST”; http_method; content:“|0d 0a|Accept-Encoding|0d 0a|Content-Type|0d 0a|Referer|0d 0a|Content-Length|0d 0a|User-Agent|0d 0a|Host|0d 0a|Cookie|0d 0a|Connection|0d 0a 0d 0a|”; http_header; classtype:bad-unknown; sid:1; rev:1;)

The buffer parsed by http_header has headers and values. You can not ignore them with http_header/http_raw_header without the usage of PCRE.

Tackling Translation Quirks

The goal is to create a Snort rule that looks for these header names and skips the values. This can be done with PCRE.

Snort PCREs
Here’s a refresher on Snort PCRE usage:

http_uri;
content:"/vulnerable_endpoint.php",fast_pattern,nocase;
# pcre gets evaluated against data in the specified sticky buffer
pcre:"/[?&]interface=[\x60\x3b]/i";

What this means is that the content acts as an anchor. After Snort finds this anchor, it will then continue to apply the PCRE. We also see the use of modifiers like /i. Whenever possible we should take advantage of these. Here are modifiers relevant for our goal:

• i case insensitive
• m By default, the string is treated as one big line of characters. ^ and $ match at the beginning and ending of the string. When m is set, ^ and $ match immediately following or immediately before any newline in the buffer, as well as the very start and very end of the buffer.
• R Match relative to the end of the last pattern match. (Similar to distance:0;)
• H Match normalized HTTP request or HTTP response header (Similar to http_header). This modifier is not allowed with the unnormalized HTTP request or HTTP response header modifier(D) for the same content.

And so, we have a tiny template to consider:

content:"?" #anchor!
http_header;
pcre:"/?/Hi"

What should our drafted PCRE pattern be? It should be something like…

• Parse new header name
• Ignore header value
• \r\n
• Repeat

Let’s put that all together…

content:“Accept-Encoding”; depth:15; # Look for the content at the beginning of the line
http_header;
pcre:“/\x3a\x20[^\r\n]+\r\nContent-Type\x3a\x20[^\r\n]+\r\nReferer\x3a\x20[^\r\n]+\r\n\Content-Length\x3a\x20[^\r\n]+\r\nUser-Agent\x3a\x20[^\r\n]+\r\nHost\x3a\x20[^\r\n]+\r\nCookie\x3a\x20\r\nConnection\x3a\x20[^\r\n]+[\r\n]+$/HRi”;

OR, we could just do everything in a PCRE like…
pcre:“/^Accept-Encoding\x3a\x20[^\r\n]+\r\nContent-Type\x3a\x20[^\r\n]+\r\nReferer\x3a\x20[^\r\n]+\r\n\Content-Length\x3a\x20[^\r\n]+\r\nUser-Agent\x3a\x20[^\r\n]+\r\nHost\x3a\x20[^\r\n]+\r\nCookie\x3a\x20\r\nConnection\x3a\x20[^\r\n]+[\r\n]+$/Hi”;

Notice how we dropped the anchor and http_header.The /H modifier is used to denote

Whoa!!! Why did I add \x3a\x20?

Why does the Cookie header PCRE pattern ( Cookie\x3a\x20\r\nConnection) seem to skip the value? It looks different from the rest of the PCRE.

The WHY motivations are found in Snort Hexdumps.

** Quick Note: The Cookie header is parsed differently in Suricata and Snort. And it shows! This is worth another blog post – stay tuned.

Hidden Quirks when Snort parses HTTP Headers
Translating from Suriata to Snort is not easy. We rely on the hexdumps to help us build our rules.

For one thing, you’ll notice that Snort does not start the HTTP Header Names buffer with \r\n!

Also because Snort doesn’t parse the Header Names only, you’ll need to make sure your regex works as expected. The big takeaway here is to compare Snort and Suricata hex dumps when you need to troubleshoot your Snort rule translations. You’ll find the hiccups eventually!

Here’s how Snort parsed the sample as a hex dump

00000000  41 63 63 65 70 74 2D 45  6E 63 6F 64 69 6E 67 3A  |Accept-Encoding:|
00000010  20 69 64 65 6E 74 69 74  79 0D 0A 43 6F 6E 74 65  | identity..Conte|
00000020  6E 74 2D 54 79 70 65 3A  20 61 70 70 6C 69 63 61  |nt-Type: applica|
00000030  74 69 6F 6E 2F 78 2D 77  77 77 2D 66 6F 72 6D 2D  |tion/x-www-form-|
00000040  75 72 6C 65 6E 63 6F 64  65 64 0D 0A 52 65 66 65  |urlencoded..Refe|
00000050  72 65 72 3A 20 68 74 74  70 73 3A 2F 2F 6E 65 74  |rer: https://net|
00000060  66 6C 69 78 61 63 63 2D  6C 6F 67 6F 6E 2E 61 6E  |flixacc-logon.an|
00000070  6F 6E 64 6E 73 2E 6E 65  74 2F 48 6F 6D 65 2F 0D  |ondns.net/Home/.|
00000080  0A 43 6F 6E 74 65 6E 74  2D 4C 65 6E 67 74 68 3A  |.Content-Length:|
00000090  20 32 33 0D 0A 55 73 65  72 2D 41 67 65 6E 74 3A  | 23..User-Agent:|
000000a0  20 4D 6F 7A 69 6C 6C 61  2F 35 2E 30 20 28 57 69  | Mozilla/5.0 (Wi|
000000b0  6E 64 6F 77 73 20 4E 54  20 31 30 2E 30 3B 20 57  |ndows NT 10.0; W|
000000c0  69 6E 36 34 3B 20 78 36  34 3B 20 72 76 3A 38 32  |in64; x64; rv:82|
000000d0  2E 30 29 20 47 65 63 6B  6F 2F 32 30 31 30 30 31  |.0) Gecko/201001|
000000e0  30 31 20 46 69 72 65 66  6F 78 2F 38 32 2E 30 0D  |01 Firefox/82.0.|
000000f0  0A 48 6F 73 74 3A 20 6E  65 74 66 6C 69 78 61 63  |.Host: netflixac|
00000100  63 2D 6C 6F 67 6F 6E 2E  61 6E 6F 6E 64 6E 73 2E  |c-logon.anondns.|
00000110  6E 65 74 0D 0A 43 6F 6F  6B 69 65 3A 20 0D 0A 43  |net..Cookie: ..C|
00000120  6F 6E 6E 65 63 74 69 6F  6E 3A 20 63 6C 6F 73 65  |onnection: close|
00000130  0D 0A 0D 0A         

And here’s the ASSUMED Suricata hexdump from the sample.

00000040                       41  63 63 65 70 74 2d 45 6e   A ccept-En
00000050  63 6f 64 69 6e 67 3a 20  69 64 65 6e 74 69 74 79   coding:  identity
00000060  0d 0a 43 6f 6e 74 65 6e  74 2d 54 79 70 65 3a 20   ..Conten t-Type:
00000070  61 70 70 6c 69 63 61 74  69 6f 6e 2f 78 2d 77 77   applicat ion/x-ww
00000080  77 2d 66 6f 72 6d 2d 75  72 6c 65 6e 63 6f 64 65   w-form-u rlencode
00000090  64 0d 0a 52 65 66 65 72  65 72 3a 20 68 74 74 70   d..Refer er: http
000000A0  73 3a 2f 2f 6e 65 74 66  6c 69 78 61 63 63 2d 6c   s://netf lixacc-l
000000B0  6f 67 6f 6e 2e 61 6e 6f  6e 64 6e 73 2e 6e 65 74   ogon.ano ndns.net
000000C0  2f 48 6f 6d 65 2f 0d 0a  43 6f 6e 74 65 6e 74 2d   /Home/.. Content-
000000D0  4c 65 6e 67 74 68 3a 20  32 33 0d 0a 55 73 65 72   Length:  23..User
000000E0  2d 41 67 65 6e 74 3a 20  4d 6f 7a 69 6c 6c 61 2f   -Agent:  Mozilla/
000000F0  35 2e 30 20 28 57 69 6e  64 6f 77 73 20 4e 54 20   5.0 (Win dows NT
00000100  31 30 2e 30 3b 20 57 69  6e 36 34 3b 20 78 36 34   10.0; Wi n64; x64
00000110  3b 20 72 76 3a 38 32 2e  30 29 20 47 65 63 6b 6f   ; rv:82. 0) Gecko
00000120  2f 32 30 31 30 30 31 30  31 20 46 69 72 65 66 6f   /2010010 1 Firefo
00000130  78 2f 38 32 2e 30 0d 0a  48 6f 73 74 3a 20 6e 65   x/82.0.. Host: ne
00000140  74 66 6c 69 78 61 63 63  2d 6c 6f 67 6f 6e 2e 61   tflixacc -logon.a
00000150  6e 6f 6e 64 6e 73 2e 6e  65 74 0d 0a 43 6f 6f 6b   nondns.n et..Cook
00000160  69 65 3a 20 50 48 50 53  45 53 53 49 44 3d 39 37   ie: PHPS ESSID=97
00000170  32 63 64 38 66 66 62 63  63 39 61 63 62 39 62 63   2cd8ffbc c9acb9bc
00000180  36 37 32 36 30 38 33 63  61 36 33 37 33 34 0d 0a   6726083c a63734..
00000190  43 6f 6e 6e 65 63 74 69  6f 6e 3a 20 63 6c 6f 73   Connecti on: clos
000001A0  65 0d 0a 0d 0a                                 	e....

Snort Rule
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Rule with anchor content flow:established,to_server; content:"Accept-Encoding"; http_header; pcre:"/\x3a\x20[^\r\n]+\r\nContent-Type\x3a\x20[^\r\n]+\r\nReferer\x3a\x20[^\r\n]+\r\n\Content-Length\x3a\x20[^\r\n]+\r\nUser-Agent\x3a\x20[^\r\n]+\r\nHost\x3a\x20[^\r\n]+\r\nCookie\x3a\x20\r\nConnection\x3a\x20[^\r\n]+[\r\n]+$$/Hi"; classtype:bad-unknown; sid:3; rev:1;)

or

alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:"Rule with PCRE only"; flow:established,to_server; pcre:“/^Accept-Encoding\x3a\x20[^\r\n]+\r\nContent-Type\x3a\x20[^\r\n]+\r\nReferer\x3a\x20[^\r\n]+\r\n\Content-Length\x3a\x20[^\r\n]+\r\nUser-Agent\x3a\x20[^\r\n]+\r\nHost\x3a\x20[^\r\n]+\r\nCookie\x3a\x20\r\nConnection\x3a\x20[^\r\n]+[\r\n]+$/Hi”; classtype:bad-unknown; sid:4; rev:1;)