├── .dockerignore ├── .github └── workflows │ └── image.yaml ├── Dockerfile ├── README.md ├── docs ├── Corelight-Bro-Cheatsheets-2.5.pdf ├── README.md ├── Useful-Command-Line-Utilities.pdf ├── bro-cheat-sheet.pdf └── useful_cli_utilities.txt ├── exercise1 ├── exercise1.pcap ├── questions.txt └── solutions.txt ├── exercise2 ├── exercise2.pcap ├── questions.txt └── solutions.txt ├── exercise3 ├── exercise3.pcap ├── questions.txt └── solutions.txt ├── launch_docker.sh ├── presentation └── bro_workshop.pdf └── test ├── check_zeek.sh └── test.pcap /.dockerignore: -------------------------------------------------------------------------------- 1 | launch_docker.sh 2 | presentation 3 | Dockerfile 4 | .dockerignore 5 | .git 6 | .github -------------------------------------------------------------------------------- /.github/workflows/image.yaml: -------------------------------------------------------------------------------- 1 | name: build a multiarch image 2 | 3 | on: 4 | push: 5 | branches: master 6 | 7 | jobs: 8 | build: 9 | runs-on: ubuntu-latest 10 | steps: 11 | - name: checkout code 12 | uses: actions/checkout@v4 13 | # https://github.com/docker/setup-qemu-action 14 | - name: Set up QEMU 15 | uses: docker/setup-qemu-action@v3 16 | # https://github.com/docker/setup-buildx-action 17 | - name: Set up Docker Buildx 18 | uses: docker/setup-buildx-action@v3 19 | - name: login to DockerHub 20 | run: echo "${{ secrets.DOCKER_PASSWORD }}" | docker login -u "${{ secrets.DOCKER_USERNAME }}" --password-stdin 21 | - name: build the image 22 | run: | 23 | docker buildx build --push \ 24 | --provenance=true --sbom=true \ 25 | --tag bearda/broworkshop:latest \ 26 | --platform linux/amd64,linux/arm64 . 27 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:22.04 2 | 3 | LABEL maintainer="Andrew Beard " 4 | 5 | ARG DEBIAN_FRONTEND=noninteractive 6 | RUN apt-get update && \ 7 | apt-get install -y --no-install-recommends ca-certificates curl gpg gpg-agent && \ 8 | apt-get clean && \ 9 | rm -rf /var/lib/apt/lists/* 10 | 11 | # Add the official Zeek package repository and install Zeek 12 | RUN curl -fsSL 'https://download.opensuse.org/repositories/security:zeek/xUbuntu_22.04/Release.key' | gpg --dearmor | tee /etc/apt/trusted.gpg.d/security_zeek.gpg > /dev/null && \ 13 | echo 'deb http://download.opensuse.org/repositories/security:/zeek/xUbuntu_22.04/ /' | tee /etc/apt/sources.list.d/security:zeek.list && \ 14 | apt-get update && \ 15 | apt-get upgrade -y && \ 16 | apt-get install -y --no-install-recommends git less nano zeek-core zeekctl zeek-zkg && \ 17 | apt-get clean && \ 18 | rm -rf /var/lib/apt/lists/* 19 | 20 | ENV ZEEK_HOME=/opt/zeek 21 | ENV PATH="$ZEEK_HOME/bin/:$PATH" 22 | 23 | # Set up the Zeek package manager 24 | RUN zkg autoconfig && \ 25 | echo "@load packages" >> /opt/zeek/share/zeek/site/local.bro 26 | 27 | RUN mkdir -p /root/workshop 28 | COPY . /root/workshop/ 29 | WORKDIR /root/workshop 30 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Bro Workshop 2 | These are the materials from a workshop on the [Bro IDS](https://www.bro.org/) originally presented at [BSides NoVA](http://www.bsidesnova.org/) 2018 and later at [BSides Charleston](http://www.bsidescharleston.com/) 2018. 3 | 4 | ## The Virtual Machine 5 | Most of the workshop involves interacting with a Ubuntu virtual machine. This VM is downloadable from [Release](https://github.com/andrewbeard/broworkshop/releases) section in VMware OVA format. If you don't have VMware Player/Workstation/Fusion a 30-day trial is available. At least one attendee said they were able to import the VM successfully using VirtualBox, but I have no idea how well it actually runs. As noted in the presentation the default login for the VM is bro:broUser. 6 | 7 | ## Docker 8 | My Docker-fu is weak, but I've tried to put together a Dockerfile that can be used to build an image to run the workshop. On the plus side there's no VMware needed and setup should be a breeze (no virtual networking to configure, etc). On the minus side building the image does take a bit of bandwidth, so it's not great for a conference workshop unless everyone gets set up before hand. After the fact it's a lot smaller than downloading the VM, though. From in the repo run: 9 | 10 | ``` 11 | docker build -t broworkshop . 12 | docker run -it --mount type=bind,source=$(pwd),target=/root/workshop broworkshop 13 | ``` 14 | 15 | On the other hand if you don't really care how the sausage is made and just want a working environment you can pull the prebuilt image from the Docker Hub (faster and probably less bandwith than building it locally). You'll still want to run from within the repo so the bind uses the latest workshop files, though: 16 | 17 | ``` 18 | docker run -it --mount type=bind,source=$(pwd),target=/root/workshop bearda/broworkshop 19 | ``` 20 | 21 | ## Presentation 22 | The slides for the presentation are available in PDF format. Some of the slides regarding the shared lab system and WiFi network are no longer relevant, however. 23 | 24 | ## Exercises 25 | Each exercise is in a named subdirectory and includes a pcap file, a text file with questions (identical to the ones in the presentation), and a text file with my suggested solution or solutions. Spoiler-free hints are available by contacting me per the feedback section below. 26 | 27 | ## Feedback 28 | Comments, feedback, and suggestions can be sent to abeard at arbor.net or [@bearda24](https://twitter.com/bearda24). I really appreciate hearing what does and doesn't work for people, so please don't hesitate to drop me a line. 29 | 30 | All nonconstructive criticism and flames may be sent to: 31 | ``` 32 | 1701 JFK Boulevard 33 | Philadelphia, PA 19103 34 | ``` 35 | 36 | ## Licensing 37 | All exercises, pcaps, questions, solutions, and Bro scripts are available under the [Creative Commons Attribution-NonCommercial-ShareAlike 4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/) license. All materials subject to copyright under the docs directory are subject to the original licenses of the copyright holders as stated there. Want to use something under a license that allows commercial use? No problem, just get in touch with me. 38 | 39 | The presentation is available under the [Creative Commons Attribution-NoDerivatives 4.0](https://creativecommons.org/licenses/by-nd/4.0/) license. Why the no derivatives license difference? Long story short, I don't feel comfortable having people modify a presentation and redistributing it with my employer's logo on it. If you'd like to modify the presentation just get in touch with me and I can send you a PowerPoint file with the template removed under a more permissive license. 40 | -------------------------------------------------------------------------------- /docs/Corelight-Bro-Cheatsheets-2.5.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/andrewbeard/broworkshop/c7b141a1099d4c3cc4a1de856a0abc7061f8983e/docs/Corelight-Bro-Cheatsheets-2.5.pdf -------------------------------------------------------------------------------- /docs/README.md: -------------------------------------------------------------------------------- 1 | # Documents 2 | 3 | Included are two quick-reference cards developed by the Bro community. To be clear I've developed neither of these, and claim absolutely no ownership. They're included in the repo only so each student has a copy available. 4 | 5 | ## Corelight-Bro-Cheatsheets-2.5.pdf 6 | A really nice reference for the available Bro logs and what each field means for Bro 2.5.X 7 | 8 | Copied from [corelight/bro-cheatsheets](https://github.com/corelight/bro-cheatsheets) and redistributed via the [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License](https://creativecommons.org/licenses/by-nc-sa/4.0/) 9 | 10 | Copyright [Corelight, Inc](https://www.corelight.com/) 11 | 12 | ## Useful-Command-Line-Utilities.pdf and useful_cli_utilities.txt 13 | Descriptions of a couple command-line test processing utilities which are useful for looking at Bro logs. This info is taken directly from the workshop slides, but included here as it may be useful as a references for some of the exercises. 14 | 15 | ## bro-cheat-sheet.pdf 16 | This one deals almost entirely with Bro command line arguments and scripting. It may come in handy for Exercise 3 in particular when you start modifying and writing your own scripts. 17 | 18 | Copied from [bro/cheatsheet](https://github.com/bro/cheat-sheet) and redistributed via [Creative Commons Attribution-NonCommercial-ShareAlike 3.0 Unported ](https://creativecommons.org/licenses/by-nc-sa/3.0/) license 19 | 20 | -------------------------------------------------------------------------------- /docs/Useful-Command-Line-Utilities.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/andrewbeard/broworkshop/c7b141a1099d4c3cc4a1de856a0abc7061f8983e/docs/Useful-Command-Line-Utilities.pdf -------------------------------------------------------------------------------- /docs/bro-cheat-sheet.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/andrewbeard/broworkshop/c7b141a1099d4c3cc4a1de856a0abc7061f8983e/docs/bro-cheat-sheet.pdf -------------------------------------------------------------------------------- /docs/useful_cli_utilities.txt: -------------------------------------------------------------------------------- 1 | The Pipe Operator 2 | * Output of one command is fed into the input of the next 3 | * Can thinking of it as a filtering and aggregation pipeline 4 | * In most cases here the first command will be reading from a Bro log file 5 | 6 | sudo dmesg | less 7 | 8 | grep 9 | * Search for a string in the input 10 | * -v inverts the search, printing things that don't contain the string 11 | 12 | cat conn.log | grep dns 13 | 14 | bro-cut 15 | * Specify a subset of a bro log 16 | * Can reorder fields 17 | * -d converts timestamps to human-readable (but timestamp field must be included) 18 | 19 | cat conn.log | bro-cut uid missed_bytes 20 | 21 | sort 22 | * Sort the rows of the input 23 | * -r for reverse, -n for numbers 24 | 25 | cat conn.log | bro-cut missed_bytes uid | sort -n 26 | 27 | uniq 28 | * Remove adjacent duplicated lines 29 | * -c counts the number of occurrences 30 | 31 | cat dns.log | bro-cut query | sort | uniq -c 32 | -------------------------------------------------------------------------------- /exercise1/exercise1.pcap: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/andrewbeard/broworkshop/c7b141a1099d4c3cc4a1de856a0abc7061f8983e/exercise1/exercise1.pcap -------------------------------------------------------------------------------- /exercise1/questions.txt: -------------------------------------------------------------------------------- 1 | What's the network's main DNS server? Which host is not using this DNS server? 2 | 3 | What IP address is hosting www.qrz.com? What about aa9pw.com? What’s the difference between these two sites? 4 | 5 | What two external IP addresses are communicated with the most? 6 | 7 | What is the most common application protocol present in the packet capture? What percentage of the connections does this protocol represent? 8 | -------------------------------------------------------------------------------- /exercise1/solutions.txt: -------------------------------------------------------------------------------- 1 | Q: What's the network's main DNS server? Which host is not using this DNS server? 2 | A: Take a look at dns.log. The server should be the responder, and assuming one connection is one DNS query we can get a count of DNS queries by server by running the following: 3 | 4 | cat dns.log | bro-cut id.resp_h | sort | uniq -c | sort -r -n 5 | 6 | The top result with 54 DNS queries is 192.168.1.14, our DNS server. 7 | 8 | The second part involves a little more domain knowledge. The next two IP addresses are ff02::fb (multicast DNS for IPv6) and 224.0.0.251 (multicast DNS for IPv4). These queries don't involve a server, and are pretty common internal to a network (especially with Apple devices). Two IP addresses stand out, though: 8.8.8.8 and 8.8.4.4. These are Google's public DNS servers. To find the client run: 9 | 10 | grep 8.8.8.8 dns.log 11 | 12 | All three requests come from the same host, 192.168.1.143. 13 | 14 | 15 | Q: What IP address is hosting www.qrz.com? What about aa9pw.com? What’s the difference between these two sites? 16 | A: The easy way to do this one is to just grep for the hostnames in all the log files, like so: 17 | 18 | grep www.qrz.com *.log 19 | 20 | The dns and ssl logs both indicate the host is at 23.23.229.197 21 | 22 | grep aa9pw.com *.log 23 | 24 | The http log indicates the host is at 67.43.1.28. 25 | 26 | The biggest difference between the two? This is a little debatable, but these are both good answers: 27 | - There was no DNS request before visiting aa9pw.com, even though the host field showed the domain name. That probably means the DNS request was cached on the client, and that the site had been visited recently (unlike www.qrz.com) 28 | - One shows up in the http log, and the other the ssl log. If you think about this for a bit (or look at the port numbers) it means that www.qrz.com is actually an encrypted SSK.TLS site, while aa9pw.com is a normal clear-text http server. 29 | 30 | Q: What two external IP addresses are communicated with the most? 31 | A: If you look at conn.log you can see the external address is almost always the responder (makes sense, the firewall is generally blocking inbound connections). Count up all the responders 32 | 33 | cat conn.log | bro-cut id.resp_h | sort | uniq -c | sort -r -n 34 | 35 | The top result is our dns server from the first question. It's internal, so we can throw it out. The second is 255.255.255.255, a local-link broadcast address. It's not an external host either. ff02::fb and 224.0.0.251 are both local multicast dns addresses, so we can throw them out too. That leaves only two hosts, 204.79.197.213 and 151.101.200.68 with 6 connections each. 36 | 37 | Q: What is the most common application protocol present in the packet capture? What percentage of the connections does this protocol represent? 38 | A: There are a couple ways to do this, and depending on how you do it you can come up with one of two different answers. That doesn't mean either of them are wrong. 39 | 40 | The first way is to trust Bro's service field to identify the application protocol. It's easy enough to just count all the occurrences of different protocols in that field. 41 | 42 | cat conn.log | bro-cut service | sort | uniq -c | sort -n -r 43 | 44 | The top result is - (or unidentified), which is a find answer in and of itself. The first identified protocol is dns, which is also a fine answer. 45 | 46 | The second way to go about it is to identify the protocol via the server port. 47 | 48 | cat conn.log | bro-cut id.resp_p | sort -n | uniq -c | sort -n -r 49 | 50 | Here port 443, or SSL/TLS comes way out on top. That's also a reasonable answer. The difference becomes clearer when we look at both the port and service together 51 | 52 | cat conn.log | bro-cut id.resp_p service | sort | uniq -c | sort -r -n 53 | 54 | The top hit here is port 443 with an unidentified service. This question is really here to illustrate a point: Bro doesn't always identify the protocol correctly in a number of cases. If you dig pretty deep into this one, looking at the conn_state field in conn.log for these misidentified connections you find a lot of them were "midstream" connections, meaning they were in process when the packet capture started. This is, by the way, what happens when your wife is already watching Netflix when you're trying to come up with examples for your upcoming Bro workshop. 55 | 56 | 57 | 58 | -------------------------------------------------------------------------------- /exercise2/exercise2.pcap: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/andrewbeard/broworkshop/c7b141a1099d4c3cc4a1de856a0abc7061f8983e/exercise2/exercise2.pcap -------------------------------------------------------------------------------- /exercise2/questions.txt: -------------------------------------------------------------------------------- 1 | A client downloaded a PDF file from an external server. The file is no longer available from the remote system, but you have the content in the pcap file. Extract the file. 2 | 3 | Which internal IPv4 addresses correspond to local-link IPv6 addresses? 4 | -------------------------------------------------------------------------------- /exercise2/solutions.txt: -------------------------------------------------------------------------------- 1 | Q: A client downloaded a PDF file from an external server. The file is no longer available from the remote system, but you have the content in the pcap file. Extract the file. 2 | A: Start by running bro with the extract-all-files script 3 | 4 | bro -r exercise2.pcap policy/frameworks/files/extract-all-files 5 | 6 | All of the files end up in the "extract_files" subdirectory, but they're named via the timestamp, protocol, and Bro file id 7 | 8 | You can cheat here and just use the "file *" command to identify the types of all the files in the directory (there's only one PDF). The Bro way is to take a look at the files.log file, though, specifically the mime_type column. You can just grep for "pdf" in the log file, and the mime type for the right row will match (application/pdf). When file extraction is turned on the 3rd to last column will give you the filename of the extracted file. One step shell command: 9 | 10 | cat files.log | bro-cut extracted mime_type | grep pdf 11 | 12 | Q: Which internal IPv4 addresses correspond to local-link IPv6 addresses? 13 | A: Packets from the same host have the same MAC address regardless of the protocol stack being used. You can use this to link the two address types. Start by running bro with the mac-logging script loaded 14 | 15 | bro -r exercise2.pcap policy/protocols/conn/mac-logging 16 | 17 | Create a list of originator MAC and IP addresses 18 | 19 | cat conn.log | bro-cut orig_l2_addr id.orig_h | sort | uniq 20 | 21 | Anywhere you see two references to the same MAC address with an IPv4 and IPv6 address you know they match up: 22 | 23 | 192.168.1.23 -> fe80::46d9:e7ff:fef9:69d6 24 | 192.168.1.130 -> fe80::103a:e22c:a1d3:fd0f 25 | 192.168.1.108 -> fe80::1855:580f:a885:37fe 26 | 192.168.1.113 -> fe80::4c2:38e5:ccd6:248a 27 | 192.168.1.129 -> fe80::47d:bfe8:90dc:8ced 28 | 192.168.1.126 -> fe80::18:2b3b:c958:962e 29 | -------------------------------------------------------------------------------- /exercise3/exercise3.pcap: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/andrewbeard/broworkshop/c7b141a1099d4c3cc4a1de856a0abc7061f8983e/exercise3/exercise3.pcap -------------------------------------------------------------------------------- /exercise3/questions.txt: -------------------------------------------------------------------------------- 1 | A script is being loaded that adds the MD5 and SHA1 hashes for all observed files to files.log. Add SHA256 hashes as well. 2 | 3 | Write a Bro script to print the number of connections when finished processing a capture file. 4 | -------------------------------------------------------------------------------- /exercise3/solutions.txt: -------------------------------------------------------------------------------- 1 | Q: A script is being loaded that adds the MD5 and SHA1 hashes for all observed files to files.log. Add SHA256 hashes as well. 2 | A: This is kind of a monkey-see, monkey-do question. The question refers to an existing script, policy/frameworks/files/hash-all-files. You can take a look at that script (/opt/bro/share/bro/policy/frameworks/files/hash-all-files.bro) and see that it's REALLY simple. There are only 4 significant lines. It's also not obvious, but SHA256 file hash is already implemented in Bro, it's just not enabled by default. The super-simple way to do this is to just add another line to that file, so it ends up looking like this: 3 | 4 | @load base/files/hash 5 | 6 | event file_new(f: fa_file) 7 | { 8 | Files::add_analyzer(f, Files::ANALYZER_MD5); 9 | Files::add_analyzer(f, Files::ANALYZER_SHA1); 10 | Files::add_analyzer(f, Files::ANALYZER_SHA256); 11 | } 12 | 13 | If you're using the shared system you don't have permission to edit that file, though. The better way to do it is to copy that file, and add your own event hook that invokes the SHA256 analyzer like so: 14 | 15 | @load base/files/hash 16 | 17 | event file_new(f: fa_file) 18 | { 19 | Files::add_analyzer(f, Files::ANALYZER_SHA256); 20 | } 21 | 22 | This can be entered in a new bro file (hash256.bro or whatever you want to call it) and loaded via the command line. The hash will show up in files.log along with the md5 and sha1. 23 | 24 | Q: Write a Bro script to print the number of connections when finished processing a capture file. 25 | A: The key here is figuring out the right events you need to hook and the associated syntax. To figure out the events I strongily recommend running the policy/misc/dump-events script against a pcap (any pcap). The two things are to notice are that there's a new_connection event and the last event is called bro_done. You'll probably want to reference the Bro documentation here to get the arguments for each of the events (even if you ignore them): 26 | https://www.bro.org/sphinx/scripts/base/bif/event.bif.bro.html 27 | 28 | From there you end up with a Bro script like this: 29 | 30 | @load base/protocols/conn 31 | global num_connections = 0; 32 | 33 | event new_connection(c: connection) { 34 | num_connections += 1; 35 | } 36 | 37 | event bro_done() { 38 | print(num_connections); 39 | } 40 | 41 | -------------------------------------------------------------------------------- /launch_docker.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | docker run -it --mount type=bind,source=$(pwd),target=/root/workshop bearda/broworkshop 3 | -------------------------------------------------------------------------------- /presentation/bro_workshop.pdf: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/andrewbeard/broworkshop/c7b141a1099d4c3cc4a1de856a0abc7061f8983e/presentation/bro_workshop.pdf -------------------------------------------------------------------------------- /test/check_zeek.sh: -------------------------------------------------------------------------------- 1 | #/bin/bash 2 | 3 | $ZEEK_HOME/bin/zeek -C -r test.pcap 4 | if [ $? -ne 0 ]; then 5 | echo "Error running Zeek!" 6 | exit 1 7 | fi 8 | NUM_LINES=`wc -l conn.log | awk '{print $1}'` 9 | EXP=17 10 | 11 | if [ -e conn.log -a "$NUM_LINES" -ne "$EXP" ]; then 12 | echo "FAILED!" 13 | exit 1 14 | fi 15 | 16 | # Cleanup junk files 17 | rm -rf *.log .state 18 | echo "Passed!" 19 | -------------------------------------------------------------------------------- /test/test.pcap: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/andrewbeard/broworkshop/c7b141a1099d4c3cc4a1de856a0abc7061f8983e/test/test.pcap --------------------------------------------------------------------------------