# Dalton "Iron Chef" Edition **URL:** \ **Category:** Show and Tell\ **Created:** [November 20, 2024, 7:26pm UTC](https://community.emergingthreats.net/t/dalton-iron-chef-edition/2150 "2024-11-20T19:26:42Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![trobinson667](https://sea2.discourse-cdn.com/flex016/user_avatar/community.emergingthreats.net/trobinson667/32/544_2.png) [@trobinson667](https://community.emergingthreats.net/u/trobinson667)\ **Post date:** [November 20, 2024, 7:26pm UTC](https://community.emergingthreats.net/t/dalton-iron-chef-edition/2150/1 "2024-11-20T19:26:42Z")
Hey folks. How many of you are familiar with the [Dalton](https://github.com/secureworks/dalton) project? If I had to describe it, its a testing harness for IDS platforms. Dalton creates a host of Docker containers for various versions of Suricata, Snort, and Zeek IDS platforms specified by the user through a docker-compose.yml file. Using the web interface, users can then submit pcaps and compare them against a standard ruleset (by default, uses the ETOPEN ruleset), or with custom rules via an input box that allows users to enter their own custom rules for testing. This platform can be used to test out your new IDS rules, and/or confirm whether or not existing IDS rules trigger against specific threats. Additionally, the project includes a web interface and step-by-step wizard for flowsynth – a tool that can be used to create your own packet captures. If you’ve ever had a malware report, CVE report, or other data in which some network details are provided, but for one reason or another you couldn’t obtain a pcap, then this is the tool for you. You can use the data you’ve obtained from the report to create your own packet capture that simulates the threat in question. From there, that packet capture can be fed back into dalton and used to create or test IDS signatures. Its a great project, and I recommend it for anyone involved in the analysis of IDS alerts and/or administration of IDS sensors. Over the past couple of days I made some improvements to dalton that are in the process of being merged or otherwise integrated into the project. Here is what I have been up to: ## Update dalton agent docker containers to Ubuntu 24.04 (from 18.04) There was an [issue](https://github.com/secureworks/dalton/issues/199) opened for this in the official github repo, so I decided to try it out myself and see what difficulties would come from updating the container’s operating system for Suricata, Snort, and Zeek. It turns out for Suricata and Zeek, that the changes needed were very minimal. Just needed to update the python packages installed, change the FROM version in the dockerfile to 24.04 and everything worked effortlessly. Snort on the other hand was an entirely different story. I updated the existing snort Dockerfile (`Dockerfile_snort`). Like Suricata (`Dockerfile_suricata`) and zeek (`Dockerfile_zeek`), I updated the FROM directive to 24.04, and changed the python packages being downloaded and use to run the dalton agent, but I also had to: - Install `libtool` and `libtirpc-dev` apt-get packages - Needed to move the `libtirpc-dev` headers from `/usr/include/libtirpc/*` to `/usr/include`, and `/usr/include/rpc` I couldn’t get the includedir option to work for pointing snort to the new location for the RPC headers, so I just moved them to where snort expected to find them - Needed to run `autoreconf -f -i` prior to running `./configure && make && make install` for compiling the snort DAQ libraries With all of these changes, I can compile any version of snort 2.9.16.x or newer. 2.9.15.x, and 2.9.11.x and older required another fix I had to add, that made me create a new Docker file, `Dockerfile_snort_MD_CFLAGS_FIX`. This _new_ dockerfile implemented all of the changes I made to `Dockerfile_snort`, but in addition to that, the `./configure` command for snort has the option `CFLAGS="-fcommon"` appended to the end of that command. So the file command looks something like: `./configure --enable-sourcefire --enable-debug --enable-buffer-dump CFLAGS="-fcommon"` This is necessary to fix an issue in which the linker asserts that functions are being defined multiple times all over the code base. I’m not skilled in C programming, but I [discovered](https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fcommon) that this CFLAG can address the problem. For users who want to run Snort 2.9.15.x or 2.9.11.x or older versions of snort, this dockerfile with the changes I’ve made was necessary to compile snort without multiple definitions errors exploding across my console. I later tested many versions of snort I compiled with these changes, and in my experience, it has no effect on the ability to analyze pcaps or alert on malicious traffic as expected. Notice how I didn’t mention snort versions 2.9.14.x through 2.9.12.x? Its because they required yet another fix, and their own custom Dockerfile to integrate that fix. `Dockerfile_snort_gettid_FIX` Cumulatively integrates all of the fixes in `Dockerfile_snort_MD_CFLAGS_FIX`, and also fixes a problem related to attempting to call gettid() (get thread id) without having included syscall.h in code. The fix required the create of a custom `util.h` file that replaces the one that ships with those versions of snort. This new `util.h` just includes an IFDEF near the end of the file that includes the syscall.h header in order for snort to be able to use gettid(). After I made these changes, and reconfigured the included `docker-compose.yml` to use the custom Dockerfiles I had to create for snort, I submitted a pull request here: > > > Hello! > > I saw #199 sitting in the issues queue and took it upon myself to try …and figure out how to do this. This pull request is the result of my work. In this pull request, I've made the following changes: > > \## Existing Dockerfile changes > > \### Dockerfile\_suricata: > - FROM updated to Ubuntu 24.04 > - Python and pip installed via the \`python3\` and \`python3-pip\` \`apt-get\` packages > - The final line that runs the \`dalton-agent.py\` has been changed from \`python3.8\` specifically to just \`python3\` > - These changes have been tested against Suricata 7.x through suricata 4.x > > \### Dockerfile\_zeek: > - FROM updated to Ubuntu 24.04 > - literally nothing else needed to be done here > > \### Dockerfile\_snort: > - FROM updated to Ubuntu 24.04 > - Python and pip install via the \`python3\` and \`python3-pip\` \`apt-get\` packages > - Other new packages installed: \`libtool\` and \`libtirpc-dev\` > - \`libtirpc-dev\` installs the RPC headers in \`/usr/include/libtirpc/\*\`. Snort doesn't like this, and even with --include-dirs, it was failing to find headers and failing to compile, so I moved the RPC headers to \`/usr/include/rpc\`, and \`/usr/include/tirpc/netconfig.h\` to \`/usr/include\`, the directories where snort expects to find these headers > - DAQ libraries would not compile correctly until I ran \`autoreconf -f -i\` against them, so I've added that to the command chain used to compile the DAQ libraries (e.g. \`autoreconf -f -i && ./configure && make && make install\`) > - The final line that runs the \`dalton-agent.py\` has been changed from \`python3.8\` specifically to just \`python3\` > - This modified dockerfile will successfully compile Snort 2.9.16.x and newer on Ubuntu 24.04 based containers. > - \*\*There are new Dockerfiles necessary to compile older versions of snort\*\* > > \## New Snort Dockerfiles > > \### Dockerfile\_snort\_MD\_CFLAGS\_FIX: > - All of the modifications above in \`Dockerfile\_snort\` have been applied to this dockerfile > - \`python3\`, \`python3-pip\`, \`libtool\` and \`libtirpc-dev\` \`apt-get\` packages > - moving the \`libtirpc-dev\` headers to where snort expects to find them > - \`autoreconf -f -i\` for DAQ libraries > - calling \`dalton-agent.py\` with \`python3\` and not \`python3.8\` > - Snort's \`./configure\` invocation now appends \`CFLAGS="-fcommon"\` to the end of the command. (e.g. \`./configure --enable-sourcefire --enable-debug --enable-buffer-dump CFLAGS="-fcommon"\` > - I'm not skilled enough to figure out how to patch the source code for snort to fix this, but this is what the documentation says the \`-fcommon\` CFLAG does: https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html#index-fcommon > - In my experience, this does not stop snort from analyzing pcaps correctly or triggering expected alerts. > - \*\*The changes in this file are necessary to compile all versions of Snort 2.9.15.x and older on Ubuntu 24.04 based containers\*\* > - There is another special Dockerfile for Snort versions between 2.9.14.x, and 2.9.12.x that we'll cover below. > > \### Dockerfile\_snort\_gettid\_FIX > - Cumulatively added \*\*ALL\*\* of the modifications from \`Dockerfile\_snort\_MD\_CFLAGS\_FIX\` > - \`python3\`, \`python3-pip\`, \`libtool\` and \`libtirpc-dev\` \`apt-get\` packages > - moving the \`libtirpc-dev\` headers to where snort expects to find them > - \`autoreconf -f -i\` for DAQ libraries > - adding \`CFLAGS="-fcommon"\` to snort's \`./configure\` command > - calling \`dalton-agent.py\` with \`python3\` and not \`python3.8\` > - Compiling Snort 2.9.14.x through 2.9.12.x Requires the use of a special \`util.h\` file that I have placed in \`dalton/dalton-agent/Dockerfiles/util.h\` > - Required to fix a bug that involves trying to use \`gettid()\` (get thread id), without \`syscall.h\` not being included during the compile, and the version of GCC/GLIBC being used to compile Snort > - The special \`util.h\` just defines an IFDEF near the end of the \`util.h\` file that includes \`syscall.h\` if it needs to be included to use \`gettid\` > - This \`util.h\` file is copied to the container's \`/src\` directory, and is then copied over to \`/src/\*snort\_version\*/src/util.h\` > - See also: > - https://github.com/ncbi/ncbi-vdb/issues/21 > - https://github.com/yuntongzhang/saver-artifact/commit/d7e53b4261d2416101eb265b098f06e12af5c132 > - https://bbs.archlinux.org/viewtopic.php?id=249904 > - \*\*The Changes in this file are necessary to compile Snort 2.9.14.x through 2.9.12.x on Ubuntu 24.04 based containers\*\* > > \## \`dalton/docker-compose.yml\` Changes > \- For users wishing to utilize older versions of snort, (version 2.9.15.x and older): > - Comment lines have been included to inform users that Snort version 2.9.15.x, as well as 2.9.11.x and older require the use of the \`Dockerfile\_MD\_CFLAGS\_FIX\` dockerfile in order to compile snort on Ubuntu 24.04 containers > - In the default \`docker-compose.yml\`, the Snort 2.9.15.1 2.9.9.0, 2.9.8.3, and 2.9.9.1 dalton agents are all commented out. > - I have taken the liberty of modifying the \`dockerfile:\` directive for the versions of snort mentioned above to point to the new \`Dockerfile\_MD\_CFLAGS\_FIX\` for these older versions of snort to compile correctly (e.g. \`dockerfile: Dockerfiles/Dockerfile\_snort\_MD\_CFLAGS\_FIX\`) > - For users wishing to utilize older versions of snort between version 2.9.14.x and 2.9.12.x: > - In the default \`docker-compose.yml\`, Snort 2.9.14.1 is commented out. This is the Only version of snort in the default configuration file that is affected by the \`gettid()\` bug. > - I have included commented out NOTE lines indicating that \`Dockerfile\_snort\_gettid\_FIX\` is necessary to compile Snort 2.9.14.x through 2.9.12.x > - I have taken the liberty of modifying the \`dockerfile:\` directive for snort 2.9.14.1 to point to the new \`Dockerfile\_snort\_gettid\_FIX\` (e.g. \`dockerfile: Dockerfiles/Dockerfile\_snort\_gettid\_FIX\`) > > \## Testing data > Here are the versions of Suricata I have tested: > > - 7.0.7 (agent-suricata-current) > - 6.0.20 > - 5.0.7 > - 4.1.10 > > Here are the versions of snort I have tested: > > \- 2.9.20 > \- 2.9.19 > \- 2.9.18.1 > \- 2.9.17.1 > \- 2.9.16.1 > \- 2.9.15.1 > \- 2.9.14.1 > \- 2.9.13 > \- 2.9.12 > \- 2.9.11 > \- 2.9.10 > \- 2.9.9.0 > \- 2.9.8.3 > \- 2.9.7.5 > \- 2.9.3.1 > \- 2.9.1.1 > > Here are the versions of zeek I have tested: > - 7.0.1 > - 6.0.6 > > Testing Criteria: > - Does the IDS software compile on Ubuntu 24.04? > - Does the version of the IDS software I compiled populate the agents list on the dalton web interface? > - confirms \`dalton-agent.py\` is operating correctly > - If I feed a pcap to the IDS software I compiled, do I get results I am expecting? > - Special note here: This is how I found issue #227 > > > Thanks for open-sourcing Dalton, I hope this contribution helps the project. I also have a fork with these changes integrated, if you want to use Dalton with my changes, until secureworks either merges my pull, or figures out how they’re going to proceed with updating their containers over here: [GitHub - da667/dalton: Suricata and Snort IDS rule and pcap testing system](http://github.com/da667/dalton). You can download the code like you would for most git projects: `git clone https://github.com/da667/dalton` ## Add cyberchef integration As it is Dalton is an incredibly useful platform for testing various IDS platforms, and making your own pcaps for testing. I also like to use cyberchef as a part of my workflow. If you’re not familiar with [cyberchef](https://github.com/gchq/cyberchef), its considered the “Cyber Swiss Army Knife”. Users can input data and use a variety of recipes to transform, encrypt, decrypt, encode, and decode it as necessary. The applications for this, especially for creating custom pcaps are numerous. It turns out the changes needed to this weren’t terribly invasive. It required three changes to implement: - `dalton/nginx-conf/conf.d/dalton.conf` This is an nginx site config. All I needed to was add a location directive `/cyberchef/` to the configuration file: ```auto location /cyberchef/ { resolver 127.0.0.11; proxy_pass http://cyberchef_current/; } ``` - `dalton/docker-compose.yml` All that was required here is that we had to append a directive to go download the official gchq cyberchef image and include it with the other docker containers that compose dalton: ```auto ########################### ## Cyberchef Integration ## ########################### # Official cyberchef docker image, from github container repo (ghcr) cyberchef: image: ghcr.io/gchq/cyberchef:latest container_name: cyberchef_current hostname: cyberchef_current restart: always ``` - `dalton/app/templates/dalton/layout.html` The last change we make here is just an additional `
  • ` element with a link to the new cyberchef instance: ```auto {% if (request.url_rule|string).endswith("/queue") %}
  • Queue 
    • {% else %}
  • Queue 
    • {% endif %} {% if (request.url_rule|string).endswith("/sensor") %}
  • Sensors 
    • {% else %}
  • Sensors 
    • {% endif %}
  • Flowsynth
  • Cyberchef
    • {% if (request.url_rule|string).endswith("/about") %}
  • About
    • {% else %}
  • About
    • {% endif %} ``` These changes enable us to have a clickable button on the navigation bar: ![image](https://us1.discourse-cdn.com/flex016/uploads/emergingthreats/optimized/2X/2/255b16700372bfc4b98203a8d8ceea7f8c22a35b_2_690x73.png) And when its clicked, instant access to the cyber chef instance: ![image](https://us1.discourse-cdn.com/flex016/uploads/emergingthreats/original/2X/8/8f5d59cd07fc6b19ebfc0b06333df039ada77e30.png) I submitted an issue, and it looks like one of the maintainers re-made the ticket [here](https://github.com/secureworks/dalton/issues/229). If you’re interested in using this cyberchef integration in your lab, run the following command: `git clone --branch with-cyberchef-support https://github.com/da667/dalton` The branches and forks that I’ve made operate identically to the regular secureworks version of dalton. You just need to modify the docker-compose.yml to suit your needs for the versions of Snort, Suricata and Zeek you would like, then just run the `./start-dalton.sh` Good luck, and happy hunting. -Tony R. ---
    **Author:** ![trobinson667](https://sea2.discourse-cdn.com/flex016/user_avatar/community.emergingthreats.net/trobinson667/32/544_2.png) [@trobinson667](https://community.emergingthreats.net/u/trobinson667)\ **Post date:** [November 25, 2024, 11:01pm UTC](https://community.emergingthreats.net/t/dalton-iron-chef-edition/2150/2 "2024-11-25T23:01:25Z")
    Hey everyone, I wanted to provide a brief update on the progress of some of these changes. The official Dalton repo has undergone NUMEROUS changes since I had been tinkering with the codebase. What’s probably the most noticeable update is switching over to pre-compiled Zeek IDS containers. What use to take an eternity to compile (at least with four cores…), is now much faster for acquiring Zeek IDS. As always, check out [GitHub - secureworks/dalton: Suricata, Snort and Zeek IDS rule and pcap testing system](https://github.com/secureworks/dalton) for more info on other changes that have been merged in, and issues that have been resolved. A lot of software dependencies have been updated. With all of these changes, and my inexperience with git, I was advised to resubmit my changes against their newly updated project, and submit individual changes that I want to contribute to dalton as seperate branches, from a fork, with separate pull requests. So, here is the result of that: - Pull request for Ubuntu 24.04 Containers for Snort and Suricata: [Ubuntu 24.04 containers for Suricata (4.x and newer) and Snort (2.9.16.x and newer) by da667 · Pull Request #234 · secureworks/dalton · GitHub](https://github.com/secureworks/dalton/pull/234) - Pull request for Cyberchef integration: [Add cyberchef container, nginx location, and link in layout.html by da667 · Pull Request #235 · secureworks/dalton · GitHub](https://github.com/secureworks/dalton/pull/235) As of right now, neither of these PRs include the changes I found necessary to compile versions of snort 2.9.15.x and older. But, for those who want it all: - The updated secureworks release - Ubuntu 24.04 containers for Snort and Suricata - The necessary fixes to run older versions of Snort - Cyberchef integration I created yet another fork here: [GitHub - da667/dalton at 2404-containers-all-snort-versions-cyberchef](https://github.com/da667/dalton/tree/2404-containers-all-snort-versions-cyberchef) and if you want to use this branch for setting up dalton on your workstation or lab environment, use the following `git` cli client command: `git clone --branch 2404-containers-all-snort-versions-cyberchef https://github.com/da667/dalton` As always, I’ll keep you updated as things change. -Tony