Why ET rules used hex bytes instead of the original characters?

[Sam_Freeman]

Hello everyone!

While studying the rules from Emerging Threats, I noticed one peculiarity. In the rules, instead of the original symbols (for example, “.”) their hexadecimal byte values are used.

Example Rules:

1. alert http $HOME_NET any → $EXTERNAL_NET 1024: (msg:“ET DELETED Banker Infostealer/PRG POST on High Port”; flow:to_server,established; content:“POST”; nocase; http_method; content:“|2E|php|3F|2=”; nocase; content:“|26|n=”; nocase; content:“|26|v=”; nocase; content:“|26|i=”; nocase; content:“|26|sp=”; nocase; content:“|26|lcp=”; nocase; reference:url,www.securescience.net/FILES/securescience/10378/pubMalwareCaseStudy.pdf; classtype:trojan-activity; sid:2008326; rev:8; metadata:created_at 2010_07_30, signature_severity Unknown, updated_at 2019_07_26;)
2. alert http $EXTERNAL_NET any → $HOME_NET any (msg:“ET WEB_SERVER Possible Barracuda IM Firewall smtp_test.cgi Cross-Site Scripting Attempt”; flow:established,to_server; http.uri; content:“|2F|cgi|2D|mod|2F|smtp|5F|test|2E|cgi”; nocase; content:“email|3D|”; nocase; content:“hostname|3D|”; nocase; content:“default|5F|domain|3D|”; nocase; pcre:“/(script|img|src|alert|onmouse|onkey|onload|ondragdrop|onblur|onfocus|onclick)/i”; reference:url, PSIRT.COM — Product Security Incident Response Team ; classtype:web-application-attack; sid:2010462; rev:6; metadata:created_at 2010_07_30, confidence Medium, signature_severity Major, updated_at 2020_09_14;)

Why can’t we write “.php?2” instead of “|2E|php|3F|2=” in rule number 1? In theory, Suricata should recognize the dot as a dot. Or in rule number 2, instead of “|2F|cgi|2D|mod|2F|smtp|5F|test|2E|cgi”, can we write “/cgi-mod/smtp_test.cgi”? The seconds variants look more readable for people who will be analyzing the rule detections.

The purpose of replacing symbols with hex bytes is to improve the performance of the Suricata engine, or are there other reasons?

[rgonzalez]

Greetings! Per our styleguide, we use the syntax that is less open to interpretation and/or typos - also we have seen cases where there is inappropriate or odd language and we made the decision that it’s easier to use hex to reference in these cases because it eliminates errors/misinterpretation.

Thanks for asking!

[Sam_Freeman]

Thank you very much for your answer!
Could you provide some examples? I understand cases related to the use of characters from different languages. But dot is dot, for example. “=” is “=”. There are symbols that have only one way of being written. In the examples from my question, it’s precisely the symbols that, in my opinion, have only one way of being represented.

And could you provide a styleguide of your rules?

[ishaughnessy]

Hey @Sam_Freeman

One consideration is that Wireshark will display non-ascii bytes as a . – In this screenshot, none of these are a period |2e|. These are all different characters that can be confused by the human eye · / . / '̣ / ٜ / . Another common example is dashes. - / — / ⸺.

Ultimately the goal is to reduce ambiguity and the potential of human error which will cause false negatives.

Our style guide can be found here Style Guide

One trick you can use for reviewing hex in our rules is the CyberChef formula “From Hex Content”. Generally speaking you can paste in the entire rule and it will clean it up for you but you may need to insert the content matches to avoid unintended changes.

Happy to help if you have followup questions

Thanks,
Isaac

[Sam_Freeman]

Thanks for your answer!

Have a nice day)))