Summary
Thanks to some teamwork, the Emerging Threats Snort 2.9 ruleset is 99% compatible with Snort3. ETOPEN consumers, and/or ETPRO customers who do not use the scada or scada_special ruleset should not experience any problems. The notable exceptions are rules from the following categories/files:
- deleted.rules
- scada.rules
- scada_special.rules
Problems in these rulesets should not affect the majority of customers.
By default, the problematic rules in the scada.rules and scada_special.rules files are not enabled. Additionally, the rules that are causing some issues from both the scada and scada_special rulesets are ETPRO only. Per Emerging Threats policy, any rules in deleted.rules are no longer supported by Proofpoint or the Emerging Threats team, and are always marked as disabled when moved to the deleted rules category. They are moved for a number of reasons mostly pertaining to relevance, and are kept for historical purposes.
What this Achievement means for consumers of the Emerging Threats Snort 2.9 Ruleset:
Currently the Snort2.9 ruleset passes the following checks successfully:
- when passed to snort2lua via the
-cand-roptions, snort2lua will successfully create a .rules file, plus a correspondingsnort.luafile that contains a list of thresholds that were present in any rules processed, since inline thresholding (e.g. limits/thresholds defined within the rule body itself) are no longer a feature of snort3. In a round about way, there isdetection_filter, but itâs not exactly the same. - Emerging Threats rules processed by
snort2luaand included in the userâs lua configuration files (usuallysnort.lua) or command line arguments (--rule-path /path/to/rules/dir/or-R /path/to/your.ruleswill successfully pass Snort-T(configuration testing) with zero warnings or errors.
Compatibility does not Imply Optimization
Be aware that compatibility with Snort3 does not imply that these rules are optimized or that they will not require further analysis to make use of Snort3âs features and/or enhance their performance. Let me provide an example of what Iâm talking about. Letâs look at a recent infostealer rule to detect the Pennywise Infostealer. Here is the snort2.9 version of the rule:
alert tcp $HOME_NET any -> $EXTERNAL_NET 5001 (msg:"ET MALWARE PennyWise Stealer Exfil"; flow:established,to_server; content:"|2a|PennyWise|20|v1|2e|"; fast_pattern; content:"Worker|3a 20|"; content:"IP|3a 20|"; content:"Username|3a 20|"; content:"PC|3a 20|"; content:"System|3a 20|"; content:"|2a|Browsers|3a 2a|"; reference:url,twitter.com/crep1x/status/1638596449226170370; classtype:trojan-activity; sid:2044748; rev:2; metadata:affected_product Windows_XP_Vista_7_8_10_Server_32_64_Bit, attack_target Client_Endpoint, created_at 2023_03_22, deployment Perimeter, former_category MALWARE, malware_family PennyWise, performance_impact Low, confidence High, signature_severity Major, updated_at 2023_03_30;)
and here is the Suricata 5 version:
alert http $HOME_NET any -> $EXTERNAL_NET 5001 (msg:"ET MALWARE PennyWise Stealer Exfil"; flow:established,to_server; http.request_line; content:"POST|20|/uploadfile|20|"; http.request_body; content:"|2a|PennyWise|20|v1|2e|"; fast_pattern; content:"Worker|3a 20|"; content:"IP|3a 20|"; content:"Username|3a 20|"; content:"PC|3a 20|"; content:"System|3a 20|"; reference:url,twitter.com/crep1x/status/1638596449226170370; classtype:trojan-activity; sid:2044748; rev:2; metadata:affected_product Windows_XP_Vista_7_8_10_Server_32_64_Bit, attack_target Client_Endpoint, created_at 2023_03_22, deployment Perimeter, former_category MALWARE, malware_family PennyWise, performance_impact Low, confidence High, signature_severity Major, updated_at 2023_03_30;)
If you compare the two of them, youâll notice that the Suricata rule is protocol aware, and is able to use HTTP sticky buffers. Suricata doesnât care what port http traffic is on. If it detects it as HTTP traffic, you can use the http application layer protocol rule header, and you can use http sticky buffers and modifiers. Snort on the other handâŚ
The use of application layer rule headers, via automatic service inspection (e.g. The âWizardâ system) is a relatively new feature for Snort3. Snort3 is able to automatically detect Application layer protocol traffic based on a set of criteria now, instead of using a portvar, such as HTTP_PORTS. Traditionally, Snort relied on customers knowing what ports to expect what traffic from - through the use of portvar variables in snort.conf.
Seeing as how TCP/IP ports are entirely arbitrary, that means bad guys could choose any TCP port they desire to host a web server. Snort 2.x wouldnât do a terribly great job of analyzing this traffic if it wasnât defined in the HTTP_PORTS port variable, in the TCP stream reassembly ports of the stream5 preprocessor, and in the inspection ports of the http_inspect preprocessor. This would cause a cascading chain of failures, making network detection more difficult.
If a snort rule used $HTTP_PORTS in the rule header, the malicious traffic would never be compared to that rule. Because the port isnât in the stream5 preprocessor, The traffic wouldnât have its TCP stream reassembled. Then as a direct result the http_inspect preprocessor, which requires stream reassembly to work, snort would not recognize the malicious traffic as HTTP traffic.
That means modifiers like http_method, http_uri, urilen, http_header, etc. would not work. So we would be required to write rules for this traffic without HTTP content modifiers or http_inspect features. That also means that if threat actors were especially cheeky and decided to throw in random HTTP Expect: 100-continue headers to divide http header traffic, and http client body traffic across multiple TCP packets, that we would have to somehow account for this, since we have zero stream reassembly for this malicious traffic. This is why the Pennywise infostealer rule above for snort 2.9 is written the way it is.
So any rules we wrote in the past for snort 2.9 that accounted for these gaps, will look roughly the same for snort3 â a syntactically correct rule, but entirely un-optimized.
How do I use snort2lua?
Snort2lua is included with the snort3.x source files, and is automatically compiled while compiling snort3. To my knowledge, there is no standalone snort2lua project or binary available at this time. Also to my knowledge, most Linux distributions do not provide packages for snort3, so manual compilation seems to be the only way to acquire it at this time. For those looking for guidance on how to install snort3 on their distro of choice, I would recommend visiting snort.orgâs documentation page and review the Snort3 Setup Guides section. Alternatively, in the not-so-distant past, I used a (now hidden) Setup guide for snort3 on Ubuntu 18 and 20, and created a script to automatically install snort3. I can confirm that this script works on Ubuntu 18.04, 20.04, and 22.04. Use whatever method you prefer, but from here on out Iâm going to assume that you have both snort3, and snort2lua installed. This can be confirmed by running:
which snort2lua snort
snort -V
by default they get installed to:
/usr/local/bin/snort2lua
/usr/local/bin/snort
If snort3 was installed and ran with the -V option, your output should look something like this:
,,_ -*> Snort++ <*-
o" )~ Version 3.1.57.0
'''' By Martin Roesch & The Snort Team
http://snort.org/contact#team
Copyright (C) 2014-2023 Cisco and/or its affiliates. All rights reserved.
Copyright (C) 1998-2013 Sourcefire, Inc., et al.
Using DAQ version 3.0.11
Using LuaJIT version 2.1.0-beta3
Using OpenSSL 3.0.2 15 Mar 2022
Using libpcap version 1.10.1 (with TPACKET_V3)
Using PCRE version 8.45 2021-06-15
Using ZLIB version 1.2.11
Using Hyperscan version 5.4.0 2021-01-26
Using LZMA version 5.2.5
Before we get started with snort2lua, youâll need a copy of the ETOPEN or ETPRO ruleset for snort2.9. You may download the ruleset using wget or whatever rule manage you use to process snort2.9 rules. However, I highly recommend that your rule manager be configured to dump all of your active snort rules into a single unified rules file. Snort2lua does not have an option to batch process all rules file in a directory, but can only process one rule file at a time. For those of you who do not use a rule management tool, here is a quick an dirty list of commands I recommend running to download the ET/ETPRO rules to the ~/snort2lua directory, moving deleted.rules, scada.rules, and scada_special.rules to ~/snort2lua/rules/disabled, concatenating the remaining rule files into a single rule file located in ~/snort2lua/conversion, then running snort2lua on the concatenated file:
mkdir ~/snort2lua
cd ~/snort2lua
ETOPEN users will run:
wget http://rules.emergingthreats.net/open/snort-2.9.0/emerging.rules.tar.gz
tar -xzvf emerging.rules.tar.gz
ETPRO customers will run:
wget https://rules.emergingthreatspro.com/<your-etpro-pro-oinkcode-here>/snort-2.9.0/etpro.rules.tar.gz
tar -xzvf etpro.rules.tar.gz
From here, both etpro and etopen users will run the following commands:
mkdir ~/snort2lua/rules/disabled
mkdir ~/snort2lua/conversion
ETPRO customers will need to run:
mv ~/snort2lua/rules/*scada*.rules ~/snort2lua/rules/disabled
Then both ETPRO and ETOPEN users will run:
mv ~/snort2lua/rules/*deleted.rules ~/snort2lua/rules/disabled
cat ~/snort2lua/rules/*.rules >> ~/snort2lua/conversion/et_all.rules
cd ~/snort2lua/conversion
snort2lua -c et_all.rules -r et_snort3_all.rules
Snort2lua has a variety of different configuration options and switches, but the two options we are concerned with are -c and -r. These options take the snort 2.9 rules (-c) and convert them to snort3 (-r). After running snort2lua, the conversions directory should have the files et_snort3_all.rules file, and a snort.lua file. This file contains a list of thresholds (suppressions, limits, etc.) that were defined inline in a snort2.9 rule body. Unfortunately, snort3 no longer supports the threshold rule option. There is detection_filter, but its not exactly the same.
Turning snort2luaâs snort.lua file into et_thresholds.lua
In addition to inline thresholds, this snort.lua file will dump every single line comment (this includes all commented out/disabled rules) and place them into a lua-style comment block at the end of the file, causing the size of this file to be somewhat large for whats just a config file. Iâve devised a sed command that will remove the massive comment block and save the changes to a new file titled et_thresholds.tmp.lua:
sed '/^--\[\[/,/\]\]/d;s/--\[\[.*$//' snort.lua >> et_thresholds.tmp.lua
du -h *.lua
336K et_thresholds.tmp.lua
5.4M snort.lua
that looks a lot better now, doesnât it?
The next thing we need to do, is remove these lines from the start of the file:
---------------------------------------------------------------------------
-- Snort++ prototype configuration
---------------------------------------------------------------------------
---------------------------------------------------------------------------
-- setup environment
---------------------------------------------------------------------------
-- given:
-- export DIR=/install/path
-- configure --prefix=$DIR
-- make install
--
-- then:
-- export SNORT_LUA_PATH=$DIR/conf/
---------------------------------------------------------------------------
dir = os.getenv('SNORT_LUA_PATH')
if ( not dir ) then
dir = '.'
end
dofile(dir .. '/snort_defaults.lua')
wizard = default_wizard
and these lines from the new end of the file:
binder =
{
{ use = { type = 'wizard', }, },
}
If you have a preferred text editor, use whatever you are most comfortable with to remove these lines manually, or you can use these commands to do it for you:
sed -i 1,26d et_thresholds.tmp.lua
head -n -6 et_thresholds.tmp.lua >> et_thresholds.lua
Implementing and Testing Your Newly Converted Rules
With all these tasks completed, all that remains is to:
- place the
et_snort3_all.rulesfile into your snort3 rule directory either defined in your snort.lua, or via the--rule-directoryor the-Roption. - place the
et_thresholds.luainto the directory with your other snort3 lua configuration files, and append an include statement telling snort3 to read your new lua file on startup.
In my case, my snort3 rule directory is:
/usr/local/etc/rules
and my snort3 lua config file directory is:
/usr/local/etc/snort
So here is a quick and dirty set of commands to achieve our objectives. These commands will copy our rules file and thresholds file to the proper directories, make a backup of /usr/local/etc/snort/snort.lua (just in case something goes wrong), then append our include statements to /usr/local/etc/snort/snort.lua. Change the directory paths and names as necessary for your snort3 deployment:
cp ~/snort2lua/conversion/et_snort3_all.rules /usr/local/etc/rules
cp ~/snort2lua/conversion/et_thresholds.lua /usr/local/etc/snort
cp /usr/local/etc/snort/snort.lua /usr/local/etc/snort/snort_lua.bak
echo "-- this include statement is necessary for emerging threats rule thresholds to operate correctly" >> /usr/local/etc/snort/snort.lua
echo "include 'et_thresholds.lua'" >> /usr/local/etc/snort/snort.lua
The quickest way to make sure youâve done all of this correctly is to run snort3 with the -T flag to confirm there are no warnings or errors. Try this, replacing the directory path for snort, config files, rules, and plugins as necessary for your deployment:
/usr/local/bin/snort -c /usr/local/etc/snort/snort.lua -u snort -g snort --plugin-path=/usr/local/lib/snort --rule-path /usr/local/etc/rules -T
If everything ran successfully, the last lines should read:
Snort successfully validated the configuration (with 0 warnings).
o")~ Snort exiting
Congratulations! Youâre good to go. We have a lot of work to do in better supporting Snort3, but we hope youâll agree that this is a great first step. Stay tuned for more information.