├── .gitignore ├── LICENSE ├── LICENSE.US_GOV ├── README.md ├── TODO ├── VERSION ├── contrib ├── muttrc.vmail └── topgen.spec ├── etc ├── delegations.dns ├── scrape_sites.txt └── vmail.cfg ├── install.sh ├── man ├── topgen-mkdns.sh.8 ├── topgen-scrape.sh.8 ├── topgen-tor.sh.8 └── topgen-vmail.sh.8 ├── sbin ├── topgen-mkdns.sh ├── topgen-scrape.sh ├── topgen-tor.sh └── topgen-vmail.sh └── systemd ├── topgen-dovecot.service.d └── topgen-dovecot.conf ├── topgen-loopback.service ├── topgen-named-config.service ├── topgen-named.service.d └── topgen-named.conf ├── topgen-nginx.service.d └── topgen-nginx.conf ├── topgen-postfix.service.d └── topgen-postfix.conf └── topgen-tor.service.d └── topgen-tor.conf /.gitignore: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/cmu-sei/topgen/e0ab5441a860a98fce4a49c249fe9b61a0e56747/.gitignore -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright 2016, Carnegie Mellon University 2 | All Rights Reserved. 3 | 4 | TopGen: Virtualized Application Service Simulator 5 | Gabriel L. Somlo 6 | CERT 7 | Software Engineering Institute (SEI) 8 | Carnegie Mellon University 9 | 10 | Permission is hereby granted, free of charge, to any person obtaining a copy 11 | of this software (TopGen) and associated documentation files (the "Software"), 12 | to deal in the Software without restriction, including without limitation the 13 | rights to use, copy, modify, merge, publish, distribute, sublicense, and/or 14 | sell copies of the Software, and to permit persons to whom the Software is 15 | furnished to do so, subject to the following conditions: 16 | 17 | Redistributions of source code must retain the above copyright notice, this 18 | list of conditions and the following disclaimers. 19 | 20 | Redistributions in binary form must reproduce the above copyright notice, this 21 | list of conditions and the following disclaimers in the documentation and/or 22 | other materials provided with the distribution. 23 | 24 | Neither the names of Carnegie Mellon University, CERT, SEI, nor the names of 25 | its contributors may be used to endorse or promote products derived from this 26 | Software without specific prior written permission. THE SOFTWARE IS PROVIDED 27 | "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT 28 | LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE 29 | AND NONINFRINGEMENT. IN NO EVENT SHALL THE CONTIBUTORS OR COPYRIGHT HOLDERS BE 30 | LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF 31 | CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE 32 | SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. 33 | -------------------------------------------------------------------------------- /LICENSE.US_GOV: -------------------------------------------------------------------------------- 1 | NOTICE: THIS SOFTWARE (TopGen) IS PROVIDED PURSUANT TO A GOVERNMENT PURPOSE 2 | RIGHTS LICENSE (see below) AND THE BSD LICENSE (see LICENSE file) 3 | 4 | --------------------------------------------------------------------------- 5 | GOVERNMENT PURPOSE RIGHTS 6 | 7 | CONTRACT NO. 8 | 9 | FA 8721-05-C-0003 10 | 11 | CONTRACTOR NAME 12 | Carnegie Mellon University 13 | 14 | CONTRACTOR ADDRESS 15 | 4500 Fifth Avenue 16 | Pittsburgh, PA 15213 17 | 18 | EXPIRATION DATE 19 | Five years from date of delivery of code and its accompanying documentation. 20 | 21 | PROJECT OWNER 22 | Gabriel L. Somlo 23 | 24 | The Government's rights to use, modify, reproduce, release, perform, display, 25 | or disclose these technical data are restricted by paragraph (b)(2) of the 26 | Rights in Technical Data-Noncommercial Items clause contained in the above 27 | identified contract. No restrictions apply after the expiration date shown 28 | above. Any reproduction of technical data or portions thereof marked with 29 | this legend must also reproduce the markings. 30 | 31 | --------------------------------------------------------------------------- 32 | Copyright 2016 Carnegie Mellon University 33 | 34 | This material is based upon work funded and supported by the Department of 35 | Defense under Contract No. FA8721-05-C-0003 with Carnegie Mellon University 36 | for the operation of the Software Engineering Institute, a federally funded 37 | research and development center. 38 | 39 | Any opinions, findings and conclusions or recommendations expressed in this 40 | material are those of the author(s) and do not necessarily reflect the views 41 | of the United States Department of Defense. 42 | 43 | NO WARRANTY. THIS CARNEGIE MELLON UNIVERSITY AND SOFTWARE ENGINEERING INSTITUTE 44 | MATERIAL IS FURNISHED ON AN "AS_IS" BASIS. CARNEGIE MELLON UNIVERSITY MAKES NO 45 | WARRANTIES OF ANY KIND, EITHER EXPRESSED OR IMPLIED, AS TO ANY MATTER INCLUDING, 46 | BUT NOT LIMITED TO, WARRANTY OF FITNESS FOR PURPOSE OR MERCHANTABILITY, 47 | EXCLUSIVITY, OR RESULTS OBTAINED FROM USE OF THE MATERIAL. CARNEGIE MELLON 48 | UNIVERSITY DOES NOT MAKE ANY WARRANTY OF ANY KIND WITH RESPECT TO FREEDOM FROM 49 | PATENT, TRADEMARK, OR COPYRIGHT INFRINGEMENT. 50 | 51 | [Distribution Statement A] This material has been approved for public release 52 | and unlimited distribution. Please see Copyright notice for non-US Government 53 | use and distribution. 54 | 55 | DM-0003656 56 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # TopGen Overview # 2 | TopGen is a virtualized application service simulator for offline exercise 3 | and training networks. It allows a single host (physical, VM, or container) 4 | to serve multiple co-hosted virtual services (such as multiple HTTP vhosts, 5 | DNS views, and/or SMTP/IMAP virtual mail domains). 6 | 7 | ## Installation ## 8 | TopGen depends on the following software packages: 9 | 10 | wget (1.17.1 or later) 11 | nginx 12 | bind (9) 13 | postfix, dovecot 14 | 15 | Running the './install.sh' script will copy all components of TopGen 16 | to the appropriate locations on the filesystem. Also, see 17 | './contrib/topgen.spec' for instructions on how to build a TopGen RPM 18 | package. 19 | 20 | An example of pre-installed GreyBox and TopGen on a Fedora 38 VM 21 | (user = `admin`, password = `tartans`) can be downloaded from: 22 | 23 | - [Virtualbox OVA](http://mirror.ini.cmu.edu/netfor/NetFor38-x86_64.vbx.ova) 24 | - [Apple/ARM UTM](http://mirror.ini.cmu.edu/netfor/NetFor38-aarch64.utm.tgz) 25 | 26 | *NOTE:* Ensure that your nginx config file ('/etc/nginx/nginx.conf') 27 | includes (a symlink to) '/var/lib/topgen/etc/nginx.conf'. Binary rpm/deb 28 | packages will drop a symlink to the latter file in '/etc/nginx/conf.d/' 29 | and assume that '/etc/nginx/nginx.conf' is preconfigured by default to 30 | include all files from that directory. Also, it is recommended that you 31 | enable "multi_accept on;" in the "events {...}" section of your nginx 32 | config file; this is a recommended best practice which enables nginx to 33 | be more responsive under high load. 34 | 35 | ## Design ## 36 | 37 | ### Application Services ### 38 | TopGen consists of several application services capable of offering virtual 39 | multi-hosting at the application level: 40 | 41 | HTTPD: nginx is hosting multiple virtual websites (vhosts) 42 | 43 | DNS: bind9 (named) implements multiple views, selected based on 44 | the destination IP address used by the client 45 | 46 | SMTP+IMAP: postfix and dovecot are used to implement multiple 47 | virtual mail domains 48 | 49 | ### Network Addressing and Connectivity ### 50 | A large number of host (/32) IP addresses, corresponding to each virtual 51 | application server (each website, nameserver, and mail gateway we simulate) 52 | are then added to the TopGen host's loopback interface. This ensures that 53 | client traffic is delivered to the appropriate application server daemon, 54 | and that replies will originate from the correct source IP address. Routing 55 | infrastructure external to the TopGen host is responsible for directing 56 | all client traffic to TopGen, typically by announcing static default routes 57 | with TopGen's management IP address as the next hop. 58 | 59 | ## Getting Started ## 60 | After installing the TopGen package and its dependencies, follow this 61 | basic process to set up a TopGen server in standalone (physical or VM) 62 | mode: 63 | 64 | ### Add Web Content ### 65 | Run 66 | 67 | topgen-scrape.sh 68 | 69 | to recursively download content from all websites listed in 70 | '/etc/topgen/scrape_sites.txt', along with all dependencies required 71 | to render them correctly in a browser. At least one site must be listed 72 | (or uncommented) in '/etc/topgen/scrape_sites.txt'. This step requires 73 | network connectivity to the Internet, and, depending on the size and 74 | number of websites to be scraped, might take several *weeks* to complete. 75 | For a relatively quick test, scraping just the content from 76 | 'www.wikipedia.org' should only take approximately 15 minutes. 77 | 78 | The collected data, along with some generated content (a self-signed 79 | in-game certificate authority, signed certificates for all virtual web 80 | hosts, and an nginx config file snippet) will be located under the 81 | following set of folders: 82 | 83 | /var/lib/topgen/vhosts/* 84 | /var/lib/topgen/certs/* 85 | /var/lib/topgen/etc/hosts.nginx 86 | /var/lib/topgen/etc/hosts.nginx 87 | /var/lib/topgen/etc/nginx.conf 88 | 89 | Alternatively, manually unpack pre-existing content to populate the 90 | above-mentioned destinations. 91 | 92 | ### Configure Client DNS ### 93 | Ensure the TopGen host uses 8.8.8.8 (and/or 8.8.4.4) as its configured 94 | nameserver(s) in '/etc/resolv.conf'. As long as none of the TopGen 95 | specific services are running and the host has Internet access, it will 96 | use the real public caching nameservers made available by Google. Once 97 | the application service simulator daemons are running, 8.8.8.8 and 8.8.4.4 98 | will be available locally via the loopback interface, and resolve DNS 99 | from the perspective of "fake" Internet as simulated by TopGen. This step 100 | is necessary only for testing by running a client (i.e. web browser) 101 | directly on the TopGen host itself, without the need to involve external 102 | routing infrastructure and dedicated client machines. 103 | 104 | ### Configure Mail Domains ### 105 | Edit '/etc/topgen/vmail.cfg' and list the name (FQDN), IP address, and 106 | list of accounts to be generated for each virtual email domain. Run 107 | 108 | topgen-vmail.sh 109 | 110 | to have the corresponding configuration data placed in 111 | '/var/lib/topgen/etc/[postfix/]' and '/var/lib/topgen/vmail'. 112 | 113 | ### Configure DNS Service ### 114 | Optionally, edit '/etc/topgen/delegations.dns' to specify second-level 115 | domains and /24 IP networks for which DNS service is to be delegated to 116 | another server in the exercise or simulation, other than TopGen. 117 | 118 | Run 119 | 120 | topgen-mkdns.sh 121 | 122 | to prepare views for authoritative (root & top-level) and public caching 123 | nameservers. The latter will apply to destination IPs such as 8.8.8.8 and 124 | 8.8.4.4, impersonating Google's publicly available caching resolvers. 125 | 126 | ### Start Services ### 127 | TopGen ships with systemd unit files for each application service type. 128 | To start web and DNS services, run: 129 | 130 | systemctl start topgen-nginx topgen-named 131 | 132 | Both of the above have a dependency on 'topgen-loopback', which ensures 133 | that all virtual service IP addresses are added to the TopGen host's 134 | loopback interface. 135 | 136 | At this point (provided client DNS was appropriately configured as 137 | specified earlier), open a browser directly on the TopGen host, and 138 | navigate to 'http://topgen.info', which will contain a set of helpful 139 | links for further experimentation. Download and install the CA 140 | certificate, which then allows browsing the available content via HTTPS. 141 | 142 | Optionally, to start the virtual mail service, run: 143 | 144 | systemctl start topgen-postfix topgen-dovecot 145 | 146 | then use a mail client to send and receive email between the various 147 | available mail domains and accounts. An example '.muttrc' file is 148 | provided in './contrib/muttrc.vmail'. 149 | 150 | ### Shutdown ### 151 | To shut down all TopGen application services, run: 152 | 153 | systemctl stop topgen-nginx topgen-named \ 154 | topgen-postfix topgen-dovecot \ 155 | topgen-loopback 156 | 157 | This will stop all service daemons, and remove all secondary IP addresses 158 | from the loopback interface, returning the TopGen host to its default 159 | networking state. 160 | 161 |
162 | For ***best effort*** help, post your question in the `#greybox` (libera.chat) 163 | IRC channel. 164 | -------------------------------------------------------------------------------- /TODO: -------------------------------------------------------------------------------- 1 | - move topgen.info from 1.1.1.1 to e.g. 0.0.0.1 (or something legal but 2 | *not* cloudflare's dns server -- might need to use that address in-game!!! 3 | 4 | - split out topgen-scrape.sh (or implement sub-commands); these are 5 | all potentially very long running operations (e.g., scraping may 6 | take weeks), so unexpected things may happen, and resuming manually 7 | should be possible without too much overhead: 8 | - scraping 9 | - certificate generation 10 | - vhost IP address resolution 11 | 12 | - have scraper (or curator, or whatever generates topgen.info) also generate 13 | http://www.msftncsi.com/ncsi.txt containing "Microsoft NCSI". 14 | 15 | - add debian/ubuntu package spec 16 | - for now, systemd units on debian/ubuntu go under /lib, which is 17 | not (yet) merged with /usr/lib; hold off on addressing this issue 18 | in our unit files, since a merge is underway (for details, see 19 | https://lwn.net/Articles/670071). 20 | 21 | - can we configure topgen-loopback service to shut down when last service 22 | depending on it goes away ? 23 | 24 | - SELinux policy for sbin scripts (right now calling chcon explicitly) 25 | 26 | - augment Web and DNS data with (simulated) WHOIS functionality 27 | -------------------------------------------------------------------------------- /VERSION: -------------------------------------------------------------------------------- 1 | 0.0.97 2 | -------------------------------------------------------------------------------- /contrib/muttrc.vmail: -------------------------------------------------------------------------------- 1 | ############################################################################# 2 | # NOTE: 3 | # 1. Before running mutt in a container, run: 4 | # export TERM=vt100 HOME=$(pwd) 5 | # 2. For a different user/password/mailserver/domain, use: 6 | # sed -i 's/user1/user2/g; s/tartans1/tar2tans/g' .muttrc 7 | # sed -i 's/mx/mail/g; s/foo.org/bar.com/g' .muttrc 8 | ############################################################################# 9 | 10 | # systemwide configuration for default settings: 11 | source /etc/Muttrc 12 | 13 | # override defaults for personalized settings: 14 | set header_cache="~/.mutthdr" 15 | set sort=mailbox-order 16 | set folder_format="%2C %3N %f" 17 | set use_envelope_from=yes 18 | set fast_reply=yes 19 | set edit_headers=yes 20 | set editor="/usr/bin/vi -c 'set textwidth=70'" 21 | set ssl_force_tls=yes 22 | set certificate_file="~/.muttcrt" 23 | 24 | # default folder to open on startup: 25 | set folder=imaps://user1@mx.foo.org 26 | set spoolfile=+INBOX 27 | 28 | # account settings: 29 | folder-hook imaps://user1@mx.foo.org 'set \ 30 | imap_pass=tartans1 \ 31 | folder=imaps://user1@mx.foo.org \ 32 | from="user1 " \ 33 | postponed=+INBOX.Drafts \ 34 | record="+INBOX.Sent Mail" \ 35 | smtp_url=smtp://user1:tartans1@mx.foo.org:587' 36 | -------------------------------------------------------------------------------- /contrib/topgen.spec: -------------------------------------------------------------------------------- 1 | Name: topgen 2 | Version: 0.0.98 3 | Release: 1%{?dist} 4 | Summary: TopGen: Virtualized Application Service Simulator 5 | License: BSD 6 | Url: https://github.com/cmu-sei/%{name} 7 | Source0: http://github.com/cmu-sei/%{name}/archive/master/%{name}-master.tar.xz 8 | Requires(post): systemd-units 9 | Requires(preun): systemd-units 10 | Requires(postun): systemd-units 11 | Requires: coreutils, gawk, grep, sed, openssl, iproute 12 | Requires: bind, nginx, wget 13 | Requires: dovecot, postfix 14 | Requires: tor 15 | BuildRequires: systemd-units 16 | BuildArch: noarch 17 | 18 | %description 19 | TopGen provides a simulation of various Internet application services 20 | (Web, DNS, etc.) for sandboxed cybersecurity exercise environments. 21 | 22 | %prep 23 | %setup -q -n %{name}-master 24 | 25 | %build 26 | echo "nothing to build" 27 | 28 | %install 29 | NAME=%{name} BUILDROOT=%{buildroot} UNITDIR=%{_unitdir} \ 30 | SYSCONFDIR=%{_sysconfdir} LOCALSTATEDIR=%{_localstatedir} \ 31 | SBINDIR=%{_sbindir} MANDIR=%{_mandir} \ 32 | ./install.sh 33 | 34 | %post 35 | %systemd_post topgen-named.service topgen-nginx.service topgen-postfix.service topgen-dovecot.service 36 | 37 | %preun 38 | %systemd_preun topgen-named.service topgen-nginx.service topgen-postfix.service topgen-dovecot.service 39 | 40 | %postun 41 | %systemd_postun_with_restart topgen-named.service topgen-nginx.service topgen-postfix.service topgen-dovecot.service 42 | 43 | %files 44 | %defattr(-,root,root,-) 45 | # miscellaneous doc files and samples: 46 | %doc README.md LICENSE* TODO contrib 47 | # systemd unit files: 48 | %{_unitdir}/topgen* 49 | # nginx symlink to topgen-specific configuration: 50 | %{_sysconfdir}/nginx/conf.d/topgen.conf 51 | # /etc/topgen directory and config files: 52 | %dir %{_sysconfdir}/%{name} 53 | %config(noreplace) %{_sysconfdir}/%{name}/scrape_sites.txt 54 | %config(noreplace) %{_sysconfdir}/%{name}/delegations.dns 55 | %config(noreplace) %{_sysconfdir}/%{name}/vmail.cfg 56 | # executables: 57 | %{_sbindir}/topgen* 58 | # manpages: 59 | %{_mandir}/man*/* 60 | # (initially empty) directory structure for storing topgen data: 61 | %dir %{_localstatedir}/lib/%{name} 62 | %dir %{_localstatedir}/lib/%{name}/etc 63 | %dir %{_localstatedir}/lib/%{name}/etc/postfix 64 | %dir %{_localstatedir}/lib/%{name}/vhosts 65 | %dir %{_localstatedir}/lib/%{name}/certs 66 | %dir %{_localstatedir}/lib/%{name}/named 67 | %dir %attr (0700, dovenull, dovenull) %{_localstatedir}/lib/%{name}/vmail 68 | 69 | %changelog 70 | * Tue Jul 12 2022 Gabriel Somlo 0.0.98-1 71 | - initial fedora package 72 | -------------------------------------------------------------------------------- /etc/delegations.dns: -------------------------------------------------------------------------------- 1 | ##------------------------------------------------------------------------- 2 | ## DELEGATIONS: 3 | ## For each delegated 2nd-level domain (forward) or /24 subnet (reverse), 4 | ## we list one or more name server fqdn(s), which will handle lookups 5 | ## for that respective domain or subnet. 6 | ## We then separately list each name server fqdn with its ip address. 7 | ## NOTE: each listed delegation name server MUST have a subsequent entry 8 | ## listing its IP address, but we do not enforce that programmatically. 9 | ## Failure to provide an IP address for any of the delegation name servers 10 | ## results in undefined behavior at this time. 11 | ## FIXME: we can either read in the following three hashes from a config 12 | ## file programmatically (and add the checks there), or we can add a step 13 | ## to check for consistency across the three hashes, if it turns out that 14 | ## mistakes are simply TOO easy to make :) 15 | ##------------------------------------------------------------------------- 16 | 17 | # Map delegated forward domains to their designated name servers: 18 | # (these will end up as NS records in the corresponding TLD forward zone) 19 | # NOTE: 2nd level (e.g., foo.tld) domains only!!! 20 | declare -A DELEGATIONS_FWD=( 21 | #['evl.edu']='ns.evl.edu' 22 | # ['eisenhower.mil']='ns.eisenhower.mil ns.eisenhower.com' 23 | #['eisenhower.mil']='ns1.eisenhower.mil' 24 | #['mail.mil']='ns1.mail.mil' 25 | #['exercise-control.com']='mail.exercise-control.com' 26 | #['gmail.com']='mail.gmail.com' 27 | #['freenode.org']='mail.freenode.org' 28 | #['twitter.com']='mail.twitter.com' 29 | ) 30 | 31 | # Map delegated reverse domains to their designated name servers: 32 | # (these will end up as NS records in the corresponding /8 TLD reverse zone) 33 | # NOTE: Class-C (/24) networks only!!! 34 | declare -A DELEGATIONS_REV=( 35 | #['81.22.162']='ns.evl.edu' 36 | # ['155.6.4']='ns.eisenhower.mil ns.eisenhower.com' 37 | #['155.6.4']='ns1.eisenhower.mil' 38 | #['142.59.1']='ns1.mail.mil' 39 | #['199.59.150']='mail.exercise-control.com mail.gmail.com mail.freenode.org mail.twitter.com' 40 | ) 41 | 42 | # Specify IP addresses for each designated name server listed above: 43 | # (these will end up as A (glue) records in their respective TLD zones) 44 | declare -A DELEGATIONS_NS=( 45 | #['ns.evl.edu']='81.22.162.10' 46 | # ['ns.eisenhower.mil']='155.6.4.10' 47 | # ['ns.eisenhower.com']='155.6.4.11' 48 | #['ns1.eisenhower.mil']='155.6.4.10' 49 | #['ns1.mail.mil']='142.59.1.10' 50 | #['mail.exercise-control.com']='199.59.150.11' 51 | #['mail.gmail.com']='199.59.150.11' 52 | #['mail.freenode.org']='199.59.150.11' 53 | #['mail.twitter.com']='199.59.150.11' 54 | ) 55 | -------------------------------------------------------------------------------- /etc/scrape_sites.txt: -------------------------------------------------------------------------------- 1 | #www.cert.org 2 | #www.wikipedia.org 3 | -------------------------------------------------------------------------------- /etc/vmail.cfg: -------------------------------------------------------------------------------- 1 | # hostname. : : ... 2 | 3 | mx.foo.org 111.0.10.10 user1:tartans1 user2:tartans1 4 | 5 | mail.bar.com 111.0.20.20 user1:foo1bar user2:tar2tans user3:l33t 6 | -------------------------------------------------------------------------------- /install.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # install TopGen 4 | # (glsomlo@cert.org, May 2016) 5 | 6 | /bin/echo "Installing using the following filesystem locations:" 7 | /bin/echo "NAME =${NAME:=topgen};" 8 | /bin/echo "BUILDROOT =${BUILDROOT:=};" 9 | /bin/echo "UNITDIR =${UNITDIR:=/usr/lib/systemd/system};" 10 | /bin/echo "SYSCONFDIR =${SYSCONFDIR:=/etc};" 11 | /bin/echo "LOCALSTATEDIR=${LOCALSTATEDIR:=/var};" 12 | /bin/echo "SBINDIR =${SBINDIR:=/sbin};" 13 | /bin/echo "MANDIR =${MANDIR:=/usr/share/man};" 14 | 15 | /usr/bin/install -d $BUILDROOT/$UNITDIR 16 | /usr/bin/install -d $BUILDROOT/$SBINDIR 17 | /usr/bin/install -d $BUILDROOT/$MANDIR/man8 18 | /usr/bin/install -d $BUILDROOT/$SYSCONFDIR/nginx/conf.d 19 | /usr/bin/install -d $BUILDROOT/$SYSCONFDIR/$NAME 20 | /usr/bin/install -d $BUILDROOT/$LOCALSTATEDIR/lib/$NAME/etc/postfix 21 | /usr/bin/install -d $BUILDROOT/$LOCALSTATEDIR/lib/$NAME/vhosts 22 | /usr/bin/install -d $BUILDROOT/$LOCALSTATEDIR/lib/$NAME/certs 23 | /usr/bin/install -d $BUILDROOT/$LOCALSTATEDIR/lib/$NAME/named 24 | /usr/bin/install -d $BUILDROOT/$LOCALSTATEDIR/lib/$NAME/vmail 25 | /bin/ln -s $LOCALSTATEDIR/lib/$NAME/etc/nginx.conf \ 26 | $BUILDROOT/$SYSCONFDIR/nginx/conf.d/topgen.conf 27 | # symlink to standard services, then amend via drop-in override configurations: 28 | for i in systemd/topgen-*.service.d; do 29 | tmp=${i##*topgen-} 30 | svc=${tmp%%.d} 31 | /bin/ln -s $svc $BUILDROOT/$UNITDIR/topgen-$svc 32 | done 33 | cp -r systemd/* $BUILDROOT/$UNITDIR 34 | /usr/bin/install -m 0755 -t $BUILDROOT/$SBINDIR sbin/* 35 | /usr/bin/install -m 0644 -t $BUILDROOT/$MANDIR/man8 man/* 36 | /usr/bin/install -m 0644 -t $BUILDROOT/$SYSCONFDIR/$NAME etc/* 37 | -------------------------------------------------------------------------------- /man/topgen-mkdns.sh.8: -------------------------------------------------------------------------------- 1 | .TH topgen-mkdns.sh 8 "MAY 2016" "TopGen Simulator" "TopGen Manuals" 2 | .SH NAME 3 | topgen-mkdns.sh \- generate multi-view bind9 configuration for TopGen. 4 | .SH SYNOPSIS 5 | .B topgen-mkdns.sh 6 | [ 7 | .B \-fqh 8 | ] [ 9 | .B \-w 10 | .I web-hosts 11 | ] [ 12 | .B \-x 13 | .I xtra-hosts 14 | ] [ 15 | .B \-m 16 | .I mail-hosts 17 | ] [ 18 | .B \-d 19 | .I delegations 20 | ] [ 21 | .B \-n 22 | .I dns-hosts 23 | ] [ 24 | .B \-c 25 | .I named-conf 26 | ] [ 27 | .B \-z 28 | .I zone-folder 29 | ] 30 | .SH DESCRIPTION 31 | .B topgen-mkdns.sh 32 | generates bind9 configuration based on a given set of hosts files 33 | (containing pairs), and a list of nameservers acting 34 | as 2nd-level domain delegation targets. 35 | .SH OPTIONS 36 | Options available for the 37 | .B topgen-mkdns.sh 38 | command: 39 | .TP 40 | \fB\-w\fR \fIweb-hosts\fR 41 | Specifies an alternative hosts file for httpd vhosts. 42 | .br 43 | This option defaults to \fB\fI/var/lib/topgen/etc/hosts.nginx\fR. 44 | .TP 45 | \fB\-x\fR \fIxtra-hosts\fR 46 | Optionally specifies additional hosts file(s) to be 47 | added to the simulated DNS database (may be used multiple times). 48 | .br 49 | This option is useful when the IP addresses in question should \fBnot\fR 50 | be added to the TopGen host's loopback address: in this case, TopGen's 51 | DNS service will resolve the records, but no virtual application server 52 | (httpd or mail) will be offered by TopGen itself under that address or 53 | host name. 54 | .TP 55 | \fB\-m\fR \fImail-hosts\fR 56 | Specifies an alternative hosts file for mail servers, 57 | for which MX records are also generated besides the usual A and PTR. 58 | .br 59 | This option defaults to \fB\fI/var/lib/topgen/etc/hosts.vmail\fR. 60 | .TP 61 | \fB\-d\fR \fIdelegations\fR 62 | Specifies an alternative source for 2nd-level (forward) and /24 (reverse) 63 | delegation targets. 64 | .br 65 | This option defaults to \fB\fI/etc/topgen/delegations.dns\fR. 66 | .TP 67 | \fB\-n\fR \fIdns-hosts\fR 68 | Specifies an alternative destination for the hosts file 69 | containnig all virtual DNS servers (views) generated by this script. 70 | .br 71 | This option defaults to \fB\fI/var/lib/topgen/etc/hosts.named\fR. 72 | .TP 73 | \fB\-c\fR \fInamed-conf\fR 74 | Specifies an alternative destination for the bind9 named.conf file. 75 | .br 76 | This option defaults to \fB\fI/var/lib/topgen/etc/named.conf\fR. 77 | .TP 78 | \fB\-z\fR \fIzone-folder\fR 79 | Specifies an alternative destination for the location of each zone 80 | file added to named.conf. 81 | .br 82 | This option defaults to \fB\fI/var/lib/topgen/named\fR. 83 | .TP 84 | \fB\-f\fR 85 | Force re-creation of TopGen bind9 configuration. 86 | .br 87 | CAUTION: any pre-existing configuration will be lost! 88 | .TP 89 | \fB\-q\fR 90 | Suppress warnings and success notifications (output will still be 91 | generated if errors are encountered). 92 | .SH "SEE ALSO" 93 | .BR topgen-scrape.sh (8), 94 | .BR topgen-vmail.sh (8) 95 | .SH BUGS 96 | No error checking is performed on the hashes read in from delegations.dns. 97 | SELinux policy for the TopGen package should be sorted out, so we 98 | don't have to manually set context on the various bits and pieces 99 | of generated configuration. 100 | .SH AUTHORS 101 | Gabriel Somlo . 102 | -------------------------------------------------------------------------------- /man/topgen-scrape.sh.8: -------------------------------------------------------------------------------- 1 | .TH topgen-scrape.sh 8 "MAY 2016" "TopGen Simulator" "TopGen Manuals" 2 | .SH NAME 3 | topgen-scrape.sh \- collect and curate a corpus of web sites for TopGen. 4 | .SH SYNOPSIS 5 | .B topgen-scrape.sh 6 | [ 7 | .B \-vh 8 | ] [ 9 | .B \-s 10 | .I site-list 11 | ] [ 12 | .B \-t 13 | .I target-dir 14 | ] 15 | .SH DESCRIPTION 16 | .B topgen-scrape.sh 17 | will recursively scrape, clean, and curate a given list of Web sites. 18 | Additionally, it will issue certificates signed with a self-signed 19 | TopGen CA (which it will also generate, if needed). Finally, the script 20 | will generate a drop-in configuration file for the nginx HTTP server, 21 | and a hosts file containing pairs for each scraped 22 | virtual web site (vhost). 23 | .SH OPTIONS 24 | Options available for the 25 | .B topgen-scrape.sh 26 | command: 27 | .TP 28 | \fB\-s\fR \fIsite-list\fR 29 | Specifies an alternative list of websites to be scraped. Sites should 30 | be listed one per line, followed by an optional scrape depth (default 1). 31 | Lines beginning with '#' are ignored. 32 | .br 33 | This option defaults to \fB\fI/etc/topgen/scrape_sites.txt\fR. 34 | .TP 35 | \fB\-t\fR \fItarget-dir\fR 36 | Specifies an alternative directory where all results (scraped content, 37 | list of vhosts, certificates, config files, etc. are stored. 38 | .br 39 | This option defaults to \fB\fI/var/lib/topgen\fR. 40 | .TP 41 | \fB\-v\fR 42 | Increase verbosity level of output (may be used multiple times). 43 | .SH "SEE ALSO" 44 | .BR topgen-vmail.sh (8), 45 | .BR topgen-mkdns.sh (8) 46 | .SH BUGS 47 | Content collection should probably be separated from certificate 48 | generation and signing. 49 | .SH AUTHORS 50 | Gabriel Somlo . 51 | -------------------------------------------------------------------------------- /man/topgen-tor.sh.8: -------------------------------------------------------------------------------- 1 | .TH topgen-tor.sh 8 "MAY 2018" "TopGen Simulator" "TopGen Manuals" 2 | .SH NAME 3 | topgen-tor.sh \- configure simulated TOR Directory Authority for TopGen. 4 | .SH SYNOPSIS 5 | .B topgen-tor.sh 6 | [ 7 | .B \-fh 8 | ] [ 9 | .B \-t 10 | .I target-dir 11 | ] [ 12 | .B \-n 13 | .I da-hostname 14 | ] 15 | .SH DESCRIPTION 16 | .B topgen-tor.sh 17 | Generates TOR Directory Authority keys and a corresponding sample config file. 18 | A link to the config file will also be inserted into the virual Web host 19 | representing the simulator's front page. 20 | .SH OPTIONS 21 | Options available for the 22 | .B topgen-tor.sh 23 | command: 24 | .TP 25 | \fB\-t\fR \fItarget-dir\fR 26 | Specifies an alternative base directory relative to which all other 27 | configuration is located. 28 | .br 29 | This option defaults to \fB\fI/var/lib/topgen\fR. 30 | .TP 31 | \fB\-n\fR \fIda-hostname\fR 32 | Fully qualified host name of the TOR Directory Authority, also shared 33 | with the virtual Web host representing the simulator's front page. 34 | .br 35 | This option defaults to \fB\fItopgen.info\fR. 36 | .TP 37 | \fB\-f\fR 38 | Forcibly overwrite and re-create any pre-existing configuration elements. 39 | .br 40 | CAUTION: any pre-existing configuration will be lost! 41 | .SH "SEE ALSO" 42 | .BR topgen-scrape.sh (8), 43 | .BR topgen-mkdns.sh (8) 44 | .SH BUGS 45 | None that we know of, probably lots we don't know about! 46 | .SH AUTHORS 47 | Gabriel Somlo . 48 | -------------------------------------------------------------------------------- /man/topgen-vmail.sh.8: -------------------------------------------------------------------------------- 1 | .TH topgen-vmail.sh 8 "MAY 2016" "TopGen Simulator" "TopGen Manuals" 2 | .SH NAME 3 | topgen-vmail.sh \- configure virtual mail domains for TopGen. 4 | .SH SYNOPSIS 5 | .B topgen-vmail.sh 6 | [ 7 | .B \-fvh 8 | ] [ 9 | .B \-c 10 | .I vmail-config 11 | ] [ 12 | .B \-t 13 | .I target-dir 14 | ] 15 | .SH DESCRIPTION 16 | .B topgen-vmail.sh 17 | generates virtual mail server configuration for the (domain, server, users) 18 | sets provided in a configuration file. 19 | .SH OPTIONS 20 | Options available for the 21 | .B topgen-vmail.sh 22 | command: 23 | .TP 24 | \fB\-c\fR \fIvmail-config\fR 25 | Specifies an alternative virtual domain configuration file. Entries 26 | are provided one per line, using the following format: 27 | .br 28 | . : : ... 29 | .br 30 | This option defaults to \fB\fI/etc/topgen/vmail.cfg\fR. 31 | .TP 32 | \fB\-t\fR \fItarget-dir\fR 33 | Specifies an alternative directory where all results (scraped content, 34 | list of vhosts, certificates, config files, etc. are stored. 35 | .br 36 | This option defaults to \fB\fI/var/lib/topgen\fR. 37 | .TP 38 | \fB\-f\fR 39 | Force re-creation of virtual mail domain configuration. 40 | .br 41 | CAUTION: any pre-existing configuration will be lost! 42 | .TP 43 | \fB\-v\fR 44 | Increase verbosity level of output (may be used multiple times). 45 | .SH "SEE ALSO" 46 | .BR topgen-scrape.sh (8), 47 | .BR topgen-mkdns.sh (8) 48 | .SH BUGS 49 | The same certificate could be used for both IMAP and SMTP, if not for 50 | SELinux labeling preventing it. SELinux policy for the TopGen package 51 | should be sorted out, so we don't have to manually set context on the 52 | various bits and pieces of generated configuration. 53 | .SH AUTHORS 54 | Gabriel Somlo . 55 | -------------------------------------------------------------------------------- /sbin/topgen-mkdns.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Generate full bind9 configuration from a (TopGen) hosts file 4 | # (glsomlo@cert.org, June 2015) 5 | # 6 | # We assume a well-behaved hosts file consisting of valid lines 7 | # of the form (a file auto-generated by the TopGen 8 | # scraper should work unequivocally; for now, though, we skip any 9 | # additional error checking and validation). 10 | # 11 | # NOTE on DELEGATIONS: 12 | # For each delegated 2nd-level domain (forward) or /24 subnet (reverse), 13 | # we list one or more name server fqdn(s), which will handle lookups 14 | # for that respective domain or subnet. 15 | # We then separately list each name server fqdn with its ip address. 16 | # Each listed delegation name server MUST have a subsequent entry 17 | # containing its IP address, but, for now, we do not enforce that. 18 | # (Failure to provide an IP address for any of the delegation name 19 | # servers results in undefined behavior at this time. 20 | # FIXME: we can either read in the following three hashes from a config 21 | # file programmatically (and add the checks there), or we can add a step 22 | # to check for consistency across the three hashes, if it turns out that 23 | # mistakes are simply TOO easy to make :) 24 | 25 | # input hosts file ( for all virtual Web hosts): 26 | SRC_WHOSTS='/var/lib/topgen/etc/hosts.nginx' 27 | 28 | # additional input hosts files (): 29 | SRC_XHOSTS='' 30 | 31 | # input hosts file ( for all virtual mail servers): 32 | SRC_MHOSTS='/var/lib/topgen/etc/hosts.vmail' 33 | 34 | # input delegations file (hashes to be sourced into our namespace): 35 | SRC_DELEG='/etc/topgen/delegations.dns' 36 | 37 | # output hosts file ( for all virtual DNS servers impersonated): 38 | NAMED_HOSTS='/var/lib/topgen/etc/hosts.named' 39 | 40 | # output named.conf 41 | NAMED_CONF='/var/lib/topgen/etc/named.conf' 42 | 43 | # output folder for zone files: 44 | NAMED_ZD='/var/lib/topgen/named' 45 | 46 | # if "yes", force/overwrite any prior existing configuration 47 | FORCE_GEN='no' 48 | 49 | # if "yes", do not print warnings and a success notification 50 | QUIET_GEN='no' 51 | 52 | ########################################################################### 53 | #### NO FURTHER USER-SERVICEABLE PARTS BEYOND THIS POINT !!!!!!! #### 54 | ########################################################################### 55 | 56 | # Caching servers (for use in view match, and to ensure existence of A records): 57 | declare -A CACHING_NS=( 58 | ['b.resolvers.level3.net']='4.2.2.2' 59 | ['google-public-dns-a.google.com']='8.8.8.8' 60 | ['google-public-dns-b.google.com']='8.8.4.4' 61 | ) 62 | 63 | # TLD servers (for use in view match, and to ensure existence of A records): 64 | declare -A TOPLEVEL_NS=( 65 | ['ns.level3.net']='4.4.4.8' 66 | ['ns.att.net']='12.12.12.24' 67 | ['ns.verisign.com']='69.58.181.181' 68 | ) 69 | 70 | # Root servers (for use in view match, and to ensure existence of A records): 71 | # NOTE: these are "well-known", i.e. hardcoded on various other software 72 | # packages (e.g., bind9), so don't change them unless you REALLY know 73 | # what you're doing !!! 74 | declare -A ROOT_NS=( 75 | ['a.root-servers.net']='198.41.0.4' 76 | ['b.root-servers.net']='192.228.79.201' 77 | ['c.root-servers.net']='192.33.4.12' 78 | ['d.root-servers.net']='199.7.91.13' 79 | ['e.root-servers.net']='192.203.230.10' 80 | ['f.root-servers.net']='192.5.5.241' 81 | ['g.root-servers.net']='192.112.36.4' 82 | ['h.root-servers.net']='128.63.2.53' 83 | ['i.root-servers.net']='192.36.148.17' 84 | ['j.root-servers.net']='192.58.128.30' 85 | ['k.root-servers.net']='193.0.14.129' 86 | ['l.root-servers.net']='199.7.83.42' 87 | ['m.root-servers.net']='202.12.27.33' 88 | ) 89 | 90 | # usage blurb (to be printed with error messages): 91 | USAGE_BLURB=" 92 | Usage: $0 [-w ] [-m ] [-d ] \\ 93 | [-n ] [-c ] [-z ] 94 | 95 | The optional command line arguments are: 96 | 97 | -w name of the input hosts list containing 98 | pairs of all virtual Web TopGen hosts to be hosted by 99 | the web server; 100 | (default: $SRC_WHOSTS). 101 | 102 | -x name of additional hosts list(s) containing 103 | mappings to be resolved by the DNS infrastructure; 104 | (default: ). 105 | 106 | -m name of the input hosts list containing 107 | pairs of all of TopGen's virtual mail servers; 108 | (default: $SRC_MHOSTS). 109 | 110 | -d name of the input file containing hashes associating 111 | delegated forward 2nd-level domains and reverse /24 112 | networks with their respective authoritative name 113 | servers, and also providing an IP address for each 114 | such name server; 115 | (default: $SRC_DELEG). 116 | 117 | -n name of the output hosts list containing 118 | pairs for all virtual DNS TopGen servers supported by 119 | the configuration being generated; 120 | (default: $NAMED_HOSTS). 121 | 122 | -c name of the generated bind9 named.conf file; 123 | (default: $NAMED_CONF). 124 | 125 | -z name of the folder where all zone files referenced 126 | in named.conf will be generated; 127 | (default: $NAMED_ZD). 128 | 129 | -f do not stop if pre-existing configuration is encountered; 130 | instead, forcibly remove and re-create the configuration. 131 | CAUTION: pre-existing configuration will be lost ! 132 | 133 | -q do not print warnings or a success notification 134 | (output will still be generated if exiting with an error). 135 | 136 | Generate bind9 configuration (named.conf, zone files, and a list 137 | of DNS virtual hosts) based on a given hosts file (containing a 138 | list of pairs) and a list of 2nd-level domain 139 | delegation target name servers. 140 | " # end usage blurb 141 | 142 | # process command line options (overriding any defaults, or '-h'): 143 | OPTIND=1 144 | while getopts "w:x:m:d:n:c:z:fqh?" OPT; do 145 | case "$OPT" in 146 | w) 147 | SRC_WHOSTS=$OPTARG 148 | ;; 149 | x) 150 | SRC_XHOSTS="$SRC_XHOSTS $OPTARG" 151 | ;; 152 | m) 153 | SRC_MHOSTS=$OPTARG 154 | ;; 155 | d) 156 | SRC_DELEG=$OPTARG 157 | ;; 158 | n) 159 | NAMED_HOSTS=$OPTARG 160 | ;; 161 | c) 162 | NAMED_CONF=$OPTARG 163 | ;; 164 | z) 165 | NAMED_ZD=$OPTARG 166 | ;; 167 | f) 168 | FORCE_GEN='yes' 169 | ;; 170 | q) 171 | QUIET_GEN='yes' 172 | ;; 173 | *) 174 | echo "$USAGE_BLURB" 175 | exit 0 176 | ;; 177 | esac 178 | done 179 | 180 | # we should be left with NO FURTHER arguments on the command line: 181 | shift $((OPTIND-1)) 182 | [ -z "$@" ] || { 183 | echo " 184 | ERROR: invalid argument: $@ 185 | 186 | $USAGE_BLURB 187 | " 188 | exit 1 189 | } 190 | 191 | # assert existence of $SRC_WHOSTS and $SRC_DELEG files, and $NAMED_ZD folder: 192 | [ -s "$SRC_WHOSTS" -a -s "$SRC_DELEG" -a -d "$NAMED_ZD" ] || { 193 | echo " 194 | ERROR: files \"$SRC_WHOSTS\", \"$SRC_DELEG\", and 195 | folder \"$NAMED_ZD\" MUST exist 196 | before running this command! 197 | 198 | $USAGE_BLURB 199 | " 200 | exit 1 201 | } 202 | 203 | # view-specific zone files go in folders underneath $NAMED_ZD: 204 | ROOT_ZD="$NAMED_ZD/rootsrv" 205 | TLD_ZD="$NAMED_ZD/tldsrv" 206 | 207 | # assert non-existence of $NAMED_HOSTS, $NAMED_CONF, and per-view zone folders: 208 | [ "$FORCE_GEN" == "yes" ] && rm -rf $NAMED_HOSTS $NAMED_CONF $ROOT_ZD $TLD_ZD 209 | [ -s "$NAMED_HOSTS" -o -s "$NAMED_CONF" -o -d "$ROOT_ZD" -o -d "$TLD_ZD" ] && { 210 | echo " 211 | ERROR: files \"$NAMED_HOSTS\", \"$NAMED_CONF\", or 212 | folders \"$ROOT_ZD\", \"$TLD_ZD\" must NOT exist! 213 | Please remove them manually before running this command again! 214 | 215 | $USAGE_BLURB 216 | " 217 | exit 1 218 | } 219 | 220 | # create $ROOT_ZD and $TLD_ZD folders at this point: 221 | mkdir $ROOT_ZD 222 | mkdir $TLD_ZD 223 | 224 | 225 | # source delegations (imports DELEGATIONS_FWD, DELEGATIONS_REV, DELEGATIONS_NS): 226 | . $SRC_DELEG 227 | 228 | 229 | # associative array (hash) with key=fqdn and value=ip_addr for each host 230 | declare -A HOSTS_LIST 231 | 232 | # assoc. array (hash) with key=domain and value=mx_hostname for each domain 233 | declare -A DOMAIN_MX 234 | 235 | # check that a host is not delegated, and insert it into HOSTS_LIST hash 236 | # also insert into DOMAIN_MX array if $IS_MX is "true" 237 | function hosts_list_add { 238 | IPADDR=$1 239 | FQDN=$2 240 | IS_MX=$3 241 | 242 | # parse TLD (last dot-separated segment of FQDN): 243 | TLD=${FQDN##*.} 244 | 245 | # parse second-level domain: 246 | # FIXME: can we parse into array, then get TLD=[last] and SLD=[last-1] ? 247 | TMP=${FQDN%.*} # everything EXCEPT tld 248 | SLD=${TMP##*.} # last dot-separated segment of "everything EXCEPT tld" 249 | 250 | # skip host if FQDN falls within one of the delegated forward domains: 251 | for DOM in "${!DELEGATIONS_FWD[@]}"; do 252 | [ "$SLD.$TLD" = "$DOM" ] && { 253 | [ "$QUIET_GEN" == "no" ] && echo " 254 | WARNING: skipping host $FQDN in delegated domain $DOM" 255 | return 256 | } 257 | done 258 | 259 | # skip host if IP address falls within one of the delegated IP networks: 260 | for NET in "${!DELEGATIONS_REV[@]}"; do 261 | [ "${IPADDR%.*}" = "$NET" ] && { 262 | [ "$QUIET_GEN" == "no" ] && echo " 263 | WARNING: skipping host $FQDN: ip $IPADDR in delegated network $NET" 264 | return 265 | } 266 | done 267 | 268 | # add host to HOSTS_LIST hash: 269 | HOSTS_LIST[$FQDN]=$IPADDR 270 | 271 | # is this an MX server? 272 | [ "$IS_MX" == "true" ] && DOMAIN_MX[$SLD.$TLD]+=" $FQDN" 273 | } 274 | 275 | 276 | # grab hosts in $SRC_[W|X]HOSTS, skip & warn on collision with delegations: 277 | for HF in $SRC_WHOSTS $SRC_XHOSTS; do 278 | while read IPADDR FQDN; do 279 | hosts_list_add $IPADDR $FQDN false 280 | done < "$HF" 281 | done 282 | 283 | # grab hosts in $SRC_MHOSTS, skipping & warning on collision with delegations: 284 | # NOTE: also make sure these get added as MX records for their domain 285 | [ -s "$SRC_MHOSTS" ] && while read IPADDR FQDN; do 286 | hosts_list_add $IPADDR $FQDN true 287 | done < $SRC_MHOSTS 288 | 289 | 290 | # unconditionally add all delegations' designated name servers: 291 | for NS in "${!DELEGATIONS_NS[@]}"; do 292 | [ -n "${HOSTS_LIST[$NS]}" ] && { 293 | [ "$QUIET_GEN" == "no" ] && echo " 294 | WARNING: Delegation name server $NS already in $SRC_WHOSTS! 295 | (old address ${HOSTS_LIST[$NS]}, using ${DELEGATIONS_NS[$NS]})" 296 | } 297 | HOSTS_LIST[$NS]=${DELEGATIONS_NS[$NS]} 298 | done 299 | 300 | # unconditionally add all public caching name servers: 301 | for NS in "${!CACHING_NS[@]}"; do 302 | [ -n "${HOSTS_LIST[$NS]}" ] && { 303 | [ "$QUIET_GEN" == "no" ] && echo " 304 | WARNING: Top level name server $NS already exists! 305 | (old address ${HOSTS_LIST[$NS]}, using ${CACHING_NS[$NS]})" 306 | } 307 | HOSTS_LIST[$NS]=${CACHING_NS[$NS]} 308 | done 309 | 310 | # unconditionally add all toplevel name servers: 311 | for NS in "${!TOPLEVEL_NS[@]}"; do 312 | [ -n "${HOSTS_LIST[$NS]}" ] && { 313 | [ "$QUIET_GEN" == "no" ] && echo " 314 | WARNING: Top level name server $NS already exists! 315 | (old address ${HOSTS_LIST[$NS]}, using ${TOPLEVEL_NS[$NS]})" 316 | } 317 | HOSTS_LIST[$NS]=${TOPLEVEL_NS[$NS]} 318 | done 319 | 320 | # unconditionally add all root name servers: 321 | for NS in "${!ROOT_NS[@]}"; do 322 | [ -n "${HOSTS_LIST[$NS]}" ] && { 323 | [ "$QUIET_GEN" == "no" ] && echo " 324 | WARNING: Top level name server $NS already exists! 325 | (old address ${HOSTS_LIST[$NS]}, using ${ROOT_NS[$NS]})" 326 | } 327 | HOSTS_LIST[$NS]=${ROOT_NS[$NS]} 328 | done 329 | 330 | 331 | # generate dns hosts file: 332 | { 333 | for NS in "${!CACHING_NS[@]}"; do 334 | echo "${CACHING_NS[$NS]} $NS" 335 | done 336 | for NS in "${!TOPLEVEL_NS[@]}"; do 337 | echo "${TOPLEVEL_NS[$NS]} $NS" 338 | done 339 | for NS in "${!ROOT_NS[@]}"; do 340 | echo "${ROOT_NS[$NS]} $NS" 341 | done 342 | } > $NAMED_HOSTS 343 | 344 | 345 | # Initialize $NAMED_CONF, starting with ACLs covering each server class: 346 | { 347 | echo -e "acl \"cache_addrs\" {" 348 | for NS in "${!CACHING_NS[@]}"; do 349 | echo -e "\t${CACHING_NS[$NS]}/32; /* $NS */" 350 | done 351 | echo -e "};\n\nacl \"root_addrs\" {" 352 | for NS in "${!ROOT_NS[@]}"; do 353 | echo -e "\t${ROOT_NS[$NS]}/32; /* $NS */" 354 | done 355 | echo -e "};\n\nacl \"tld_addrs\" {" 356 | for NS in "${!TOPLEVEL_NS[@]}"; do 357 | echo -e "\t${TOPLEVEL_NS[$NS]}/32; /* $NS */" 358 | done 359 | } > $NAMED_CONF 360 | 361 | # full path+name of root zone file: 362 | ROOT_ZONE="$ROOT_ZD/root.zone" 363 | 364 | # continue generating $NAMED_CONF (boilerplate options and view configuration) 365 | cat >> $NAMED_CONF << EOT 366 | }; 367 | 368 | options { 369 | listen-on port 53 { "cache_addrs"; "root_addrs"; "tld_addrs"; }; 370 | allow-query { any; }; 371 | recursion no; 372 | check-names master ignore; 373 | directory "/var/named"; 374 | dump-file "/var/named/data/cache_dump.db"; 375 | statistics-file "/var/named/data/named_stats.txt"; 376 | memstatistics-file "/var/named/data/named_mem_stats.txt"; 377 | pid-file "/run/named/named.pid"; 378 | session-keyfile "/run/named/session.key"; 379 | }; 380 | 381 | logging { 382 | channel default_debug { 383 | file "data/named.run"; 384 | severity dynamic; 385 | print-time yes; 386 | }; 387 | }; 388 | 389 | view "caching" { 390 | match-destinations { "cache_addrs"; }; 391 | 392 | recursion yes; 393 | dnssec-validation no; 394 | 395 | zone "." IN { 396 | type hint; 397 | file "named.ca"; 398 | }; 399 | }; 400 | 401 | view "rootsrv" { 402 | match-destinations { "root_addrs"; }; 403 | 404 | zone "." IN { 405 | type master; 406 | file "$ROOT_ZONE"; 407 | allow-update { none; }; 408 | }; 409 | }; 410 | 411 | view "tldsrv" { 412 | match-destinations { "tld_addrs"; }; 413 | 414 | EOT 415 | 416 | 417 | # Initialize $ROOT_ZONE file: 418 | cat > $ROOT_ZONE << "EOT" 419 | $TTL 300 420 | $ORIGIN @ 421 | @ SOA a.root-servers.net. admin.step-fwd.net. (15061601 600 300 800 300) 422 | EOT 423 | 424 | # list root servers (as NS records) programmatically: 425 | for NS in "${!ROOT_NS[@]}"; do 426 | echo -e "\t\t\tNS\t$NS." 427 | done >> $ROOT_ZONE 428 | 429 | # list root servers (as A glue records) programmatically: 430 | for NS in "${!ROOT_NS[@]}"; do 431 | echo -e "$NS.\tA\t${ROOT_NS[$NS]}" 432 | done >> $ROOT_ZONE 433 | 434 | # list TLD servers (as NS delegations for in-addr.arpa) programmatically: 435 | cat >> $ROOT_ZONE << "EOT" 436 | ; 437 | ; in-game reverse DNS is also handled by the tld servers: 438 | ; 439 | EOT 440 | for NS in "${!TOPLEVEL_NS[@]}"; do 441 | echo -e "in-addr.arpa.\t\tNS\t$NS." 442 | done >> $ROOT_ZONE 443 | 444 | # list TLD servers (as A glue records) programmatically: 445 | for NS in "${!TOPLEVEL_NS[@]}"; do 446 | echo -e "$NS.\tA\t${TOPLEVEL_NS[$NS]}" 447 | done >> $ROOT_ZONE 448 | 449 | cat >> $ROOT_ZONE << "EOT" 450 | ; 451 | ; begin root zone data here: 452 | ; 453 | EOT 454 | 455 | 456 | # header for regular (tld) zone files 457 | TLD_ZONE_HDR=$( 458 | cat <<- "EOT" 459 | $TTL 300 460 | $ORIGIN @ 461 | @ SOA ns.level3.net. admin.step-fwd.net. (15061601 600 300 800 300) 462 | EOT 463 | for NS in "${!TOPLEVEL_NS[@]}"; do 464 | echo -e "\t\t\tNS\t$NS." 465 | done 466 | for NS in "${!TOPLEVEL_NS[@]}"; do 467 | echo -e "$NS.\tA\t${TOPLEVEL_NS[$NS]}" 468 | done 469 | cat <<- "EOT" 470 | ; 471 | ; begin zone data here: 472 | ; 473 | EOT 474 | ) # end regular zone header 475 | 476 | 477 | # verify existence of forward zone, creating it if necessary: 478 | function ensure_exists_forward 479 | if [ ! -s "$TLD_ZD/$1.zone" ]; then 480 | # create forward zone file, paste generic header: 481 | echo "$TLD_ZONE_HDR" > $TLD_ZD/$1.zone 482 | 483 | # add zone entry in named.conf: 484 | cat >> $NAMED_CONF << EOT 485 | zone "$1" IN { 486 | type master; 487 | file "$TLD_ZD/$1.zone"; 488 | allow-update { none; }; 489 | }; 490 | EOT 491 | 492 | # add NS records pointing at our TLD servers to root.zone: 493 | for NS in "${!TOPLEVEL_NS[@]}"; do 494 | echo -e "$1.\tNS\t$NS" >> $ROOT_ZONE 495 | done 496 | fi 497 | 498 | 499 | # verify existence of reverse zone, creating it if necessary: 500 | function ensure_exists_reverse 501 | if [ ! -s "$TLD_ZD/$1.zone" ]; then 502 | # create reverse zone file, paste generic header: 503 | echo "$TLD_ZONE_HDR" > $TLD_ZD/$1.zone 504 | 505 | # add zone entry in named.conf: 506 | cat >> $NAMED_CONF << EOT 507 | zone "$1.in-addr.arpa." IN { 508 | type master; 509 | file "$TLD_ZD/$1.zone"; 510 | allow-update { none; }; 511 | }; 512 | EOT 513 | fi 514 | 515 | 516 | # add A and PTR records for all hosts in $HOSTS_LIST, creating zones if needed: 517 | for FQDN in "${!HOSTS_LIST[@]}"; do 518 | 519 | IPADDR=${HOSTS_LIST[$FQDN]} 520 | 521 | # parse IP address octets (no sanity checking): 522 | # FIXME: parse into array, then use [0], [1], [2], and [3] instead!!! 523 | IFS=. read IP1 IP2 IP3 IP4 <<< "$IPADDR" 524 | 525 | # parse TLD (last dot-separated segment of FQDN): 526 | TLD=${FQDN##*.} 527 | 528 | # add A record for this host (stripping off $TLD from $FQDN): 529 | ensure_exists_forward $TLD 530 | echo -e "${FQDN%.*}\tA\t$IPADDR" >> $TLD_ZD/$TLD.zone 531 | 532 | # add PTR record to reverse zone file 533 | ensure_exists_reverse $IP1 534 | echo -e "$IP4.$IP3.$IP2\tPTR\t$FQDN." >> $TLD_ZD/$IP1.zone 535 | 536 | done # end for FQDN in "${!HOSTS_LIST[@]}" 537 | 538 | 539 | # add NS records for all forward delegations: 540 | for DOM in "${!DELEGATIONS_FWD[@]}"; do 541 | 542 | # parse TLD (last dot-separated segment of DOM): 543 | TLD=${DOM##*.} 544 | 545 | # add NS record for each of DOM's designated name servers: 546 | ensure_exists_forward $TLD 547 | for NS in ${DELEGATIONS_FWD[$DOM]}; do 548 | echo -e "${DOM%.*}\tNS\t$NS." >> $TLD_ZD/$TLD.zone 549 | done 550 | 551 | done # for DOM in "${!DELEGATIONS_FWD[@]}" 552 | 553 | 554 | # add NS records for all reverse delegations: 555 | for NET in "${!DELEGATIONS_REV[@]}"; do 556 | 557 | # parse IP network octets (no sanity checking): 558 | # FIXME: parse into array... (see above) 559 | IFS=. read IP1 IP2 IP3 <<< "$NET" 560 | 561 | # add NS record for each of DOM's designated name servers: 562 | ensure_exists_reverse $IP1 563 | for NS in ${DELEGATIONS_REV[$NET]}; do 564 | echo -e "$IP3.$IP2\tNS\t$NS." >> $TLD_ZD/$IP1.zone 565 | done 566 | 567 | done # for NET in "${!DELEGATIONS_REV[@]}" 568 | 569 | 570 | # add MX records for all vmail domains: 571 | for DOM in "${!DOMAIN_MX[@]}"; do 572 | 573 | # parse TLD (last dot-separated segment of DOM): 574 | TLD=${DOM##*.} 575 | 576 | # add MX record for each of DOM's designated name servers: 577 | ensure_exists_forward $TLD 578 | for MX in ${DOMAIN_MX[$DOM]}; do 579 | echo -e "${DOM%.*}\tMX\t10\t$MX." >> $TLD_ZD/$TLD.zone 580 | done 581 | 582 | done # for DOM in "${!DELEGATIONS_FWD[@]}" 583 | 584 | 585 | # close out named.conf: 586 | echo '};' >> $NAMED_CONF 587 | 588 | 589 | # we're done! 590 | SUCCESS_BLURB=" 591 | SUCCESS: bind9 configuration generated based on the following inputs: 592 | 593 | Web hosts: $SRC_WHOSTS 594 | Xtra hosts: $SRC_XHOSTS 595 | Mail srvrs: $([ -s "$SRC_MHOSTS" ] && echo $SRC_MHOSTS) 596 | Delegations: $SRC_DELEG 597 | 598 | Output was written to the following locations: 599 | 600 | DNS hosts: $NAMED_HOSTS 601 | named.conf: $NAMED_CONF 602 | zone folder: $NAMED_ZD 603 | 604 | Required next steps may include: 605 | 606 | - update loopback interface to contain virtual DNS host IP addresses 607 | 608 | - (re-)start topgen-named service 609 | " 610 | 611 | [ "$QUIET_GEN" == "no" ] && echo "$SUCCESS_BLURB" 612 | 613 | 614 | # FIXME: Sort out SELinux policy associated with topgen package !!! 615 | # But, for now, let's label the relevant files for use by named: 616 | chcon -t named_conf_t $NAMED_CONF 617 | chcon -R -t named_zone_t $NAMED_ZD/* 618 | -------------------------------------------------------------------------------- /sbin/topgen-scrape.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Scrape and curate a corpus of Web sites for TopGen 4 | # (glsomlo@cert.org, December 2015) 5 | 6 | # input: original list of sites to scrape: 7 | TOPGEN_ORIG="/etc/topgen/scrape_sites.txt" 8 | 9 | # topgen corpus directory structure: 10 | TOPGEN_VARLIB='/var/lib/topgen' 11 | 12 | # if "yes", do not print out warnings: 13 | let VERBOSE=0 14 | 15 | # default recursive scrape depth: 16 | let DFL_DEPTH=1 17 | 18 | ########################################################################### 19 | #### NO FURTHER USER-SERVICEABLE PARTS BEYOND THIS POINT !!!!!!! #### 20 | ########################################################################### 21 | 22 | USAGE_BLURB=" 23 | Usage: $0 [-s ] [-t ] 24 | 25 | The optional command line arguments are: 26 | 27 | -s file containing space or newline separated sites 28 | to be scraped for static content; lines beginning 29 | with '#' are ignored; 30 | (default: $TOPGEN_ORIG). 31 | 32 | -t directory where all results (scraped content, list 33 | of vhosts, certificates, configuration files, etc. 34 | are stored; 35 | (default: $TOPGEN_VARLIB). 36 | 37 | -v increase verbosity level (may be used multiple times); 38 | (default: $VERBOSE). 39 | 40 | Recursively scrape, clean, curate a given list of Web sites. Additionally, 41 | issue certificates signed with a self-signed TopGen CA (which is in turn 42 | also generated, if necessary). Generate a drop-in config file for the nginx 43 | HTTP server, and a hosts file containing entries for each 44 | scraped vhost. 45 | " # end usage blurb 46 | 47 | # process command line options: 48 | OPTIND=1 49 | while getopts "s:t:vh?" OPT; do 50 | case "$OPT" in 51 | s) 52 | TOPGEN_ORIG=$OPTARG 53 | ;; 54 | t) 55 | TOPGEN_VARLIB=$OPTARG 56 | ;; 57 | v) 58 | let VERBOSE++ 59 | ;; 60 | *) 61 | echo "$USAGE_BLURB" 62 | exit 0 63 | ;; 64 | esac 65 | done 66 | 67 | # we should be left with NO FURTHER arguments on the command line: 68 | shift $((OPTIND-1)) 69 | [ -z "$@" ] || { 70 | echo " 71 | ERROR: invalid argument: $@ 72 | 73 | $USAGE_BLURB 74 | " 75 | exit 1 76 | } 77 | 78 | # once $TOPGEN_VARLIB is set, these are its relevant subdirectories: 79 | TOPGEN_VARETC="$TOPGEN_VARLIB/etc" 80 | TOPGEN_CERTS="$TOPGEN_VARLIB/certs" 81 | TOPGEN_VHOSTS="$TOPGEN_VARLIB/vhosts" 82 | # topgen.info vhost directory: 83 | TOPGEN_SITE="$TOPGEN_VHOSTS/topgen.info" 84 | 85 | # assert existence of required input files and target folders: 86 | [ -s "$TOPGEN_ORIG" -a \ 87 | -d "$TOPGEN_VHOSTS" -a -d "$TOPGEN_VARETC" -a -d "$TOPGEN_CERTS" ] || { 88 | echo " 89 | ERROR: file \"$TOPGEN_ORIG\" and 90 | folders \"$TOPGEN_VHOSTS\", 91 | \"$TOPGEN_VARETC\", and 92 | \"$TOPGEN_CERTS\" MUST exist 93 | before running this command! 94 | 95 | $USAGE_BLURB 96 | " 97 | exit 1 98 | } 99 | 100 | # assert target folder is empty: 101 | [ -n "$(ls -A $TOPGEN_VHOSTS)" ] && { 102 | echo " 103 | ERROR: folder \"$TOPGEN_VHOSTS\" MUST be empty 104 | before running this command! 105 | 106 | $USAGE_BLURB 107 | " 108 | exit 1 109 | } 110 | 111 | # default wget options: 112 | WGET_OPTS='-prEHN --convert-file-only --no-check-certificate -e robots=off --random-wait -t 2' 113 | # make wget quiet, unless verbosity >= 2: 114 | ((VERBOSE>=2)) || WGET_OPTS="$WGET_OPTS -q" 115 | # make wget print debug output if verbosity >= 3 116 | ((VERBOSE>=3)) && WGET_OPTS="$WGET_OPTS -d" 117 | 118 | # scrape sites listed in the $TOPGEN_ORIG file: 119 | grep -v '^#' "$TOPGEN_ORIG" | while read URL DEPTH; do 120 | wget $WGET_OPTS -U "Mozilla/5.0 (X11)" -P $TOPGEN_VHOSTS \ 121 | -l ${DEPTH:-$DFL_DEPTH} $URL 122 | done 123 | 124 | # cleanup; remove IP-only vhosts, and vhosts ending with a ":": 125 | shopt -s extglob 126 | rm -rf $TOPGEN_VHOSTS/+([[:digit:].]) $TOPGEN_VHOSTS/*:* 127 | 128 | # "curate" remaining vhosts; handle [www.]example.org/index.html issue: 129 | for VH in $TOPGEN_VHOSTS/*; do 130 | VB=${VH##*/} # basename 131 | NUM=$(ls $VH | wc -l) 132 | # if example.org has only one file named "index.html", AND if 133 | # www.example.org exists and DOESN'T contain "index.html", apply the fix: 134 | if [ $NUM -eq 1 -a -f "$VH/index.html" -a -d "$TOPGEN_VHOSTS/www.$VB" \ 135 | -a ! -f "$TOPGEN_VHOSTS/www.$VB/index.html" ]; then 136 | cp "$VH/index.html" "$TOPGEN_VHOSTS/www.$VB/" 137 | ((VERBOSE)) && echo "curate: $VH/index.html -> $TOPGEN_VHOSTS/www.$VB/" 138 | fi 139 | done 140 | 141 | # topgen.info is reserved, be sure to create it completely from scratch: 142 | rm -rf "$TOPGEN_SITE" 143 | mkdir "$TOPGEN_SITE" 144 | 145 | # intersection of scraped vhosts in $TOPGEN_VHOSTS against $TOPGEN_ORIG 146 | # (NOTE: sed strips " DEPTH", "http://", and "/path/..." from each line) 147 | comm -12 <(ls -1 $TOPGEN_VHOSTS) \ 148 | <(grep -v '^#' "$TOPGEN_ORIG" | \ 149 | sed -e 's|[[:space:]].*||; s|^.*//||; s|/.*$||' | \ 150 | sort -u) \ 151 | > "$TOPGEN_SITE/orig_vhosts.txt" 152 | 153 | # generate topgen.info landing page (index.html): 154 | { 155 | cat <<- "EOT" 156 | 157 | 158 | Welcome to TopGen.info ! 159 | 160 |
161 |

Welcome to TopGen.info !

162 | This is a simulation of the World Wide Web. View this site in either 163 | 173 | Below is a list of Web sites mirrored for this simulation: 174 |
    175 | EOT 176 | for VH in $(< "$TOPGEN_SITE/orig_vhosts.txt"); do 177 | echo "
  • $VH" 178 | done 179 | cat <<- "EOT" 180 |
181 | 182 | 183 | EOT 184 | } > "$TOPGEN_SITE/index.html" 185 | 186 | # NOTE: hand-crafted vhosts should manually be added to $TOPGEN_VHOSTS here! 187 | 188 | # generate TopGen CA (unless already provided): 189 | [ -s "$TOPGEN_VARETC/topgen_ca.key" -a -s "$TOPGEN_VARETC/topgen_ca.cer" ] || 190 | openssl req -newkey rsa:2048 -nodes -keyout "$TOPGEN_VARETC/topgen_ca.key" \ 191 | -days 7300 -x509 -out "$TOPGEN_VARETC/topgen_ca.cer" \ 192 | -subj '/C=US/ST=PA/L=Pgh/O=CMU/OU=CERT/CN=topgen_ca' \ 193 | 2>/dev/null 194 | # ... and place a copy in $TOPGEN_SITE: 195 | cp "$TOPGEN_VARETC/topgen_ca.cer" "$TOPGEN_SITE" 196 | 197 | # generate TopGen vhost key (unless already provided): 198 | [ -s "$TOPGEN_VARETC/topgen_vh.key" ] || 199 | openssl genrsa -out "$TOPGEN_VARETC/topgen_vh.key" 2048 2>/dev/null 200 | 201 | # create CA configuration (file and /tmp directory structure): 202 | TMP_CA_DIR=$(mktemp -d /tmp/TopGenCA.XXXXXXXXXXXXXX) 203 | echo "000a" > "$TMP_CA_DIR/serial" # seed the serial file 204 | touch "$TMP_CA_DIR/index" # empty (but required) index file 205 | 206 | # CA configuration file: 207 | TMP_CA_CONF="[ ca ] 208 | default_ca = topgen_ca 209 | 210 | [ crl_ext ] 211 | authorityKeyIdentifier=keyid:always 212 | 213 | [ topgen_ca ] 214 | private_key = $TOPGEN_VARETC/topgen_ca.key 215 | certificate = $TOPGEN_VARETC/topgen_ca.cer 216 | new_certs_dir = $TMP_CA_DIR 217 | database = $TMP_CA_DIR/index 218 | serial = $TMP_CA_DIR/serial 219 | default_days = 3650 220 | default_md = sha512 221 | copy_extensions = copy 222 | unique_subject = no 223 | policy = topgen_ca_policy 224 | x509_extensions = topgen_ca_ext 225 | 226 | [ topgen_ca_policy ] 227 | countryName = supplied 228 | stateOrProvinceName = supplied 229 | localityName = supplied 230 | organizationName = supplied 231 | organizationalUnitName = supplied 232 | commonName = supplied 233 | emailAddress = optional 234 | 235 | [ topgen_ca_ext ] 236 | basicConstraints = CA:false 237 | nsCertType = server 238 | nsComment = \"TopGen CA Generated Certificate\" 239 | subjectKeyIdentifier = hash 240 | authorityKeyIdentifier = keyid,issuer:always 241 | " 242 | 243 | # per-vhost CSR configuration template: 244 | # (NOTE: using [alt_names] to work around CN length limit for long hostnames) 245 | TMP_VH_CONF=$(sed -e '/req_extensions/s/^# //; 246 | /^\[ v3_req \]/a subjectAltName = @alt_names' \ 247 | /etc/pki/tls/openssl.cnf) 248 | 249 | # start nginx.conf here (vhost entries to be appended from loop below) 250 | cat > "$TOPGEN_VARETC/nginx.conf" <<- EOT 251 | # use a common key for all certificates: 252 | ssl_certificate_key $TOPGEN_VARETC/topgen_vh.key; 253 | 254 | # ensure enumerated https server blocks fit into nginx hash table: 255 | server_names_hash_bucket_size 256; 256 | server_names_hash_max_size 131070; 257 | EOT 258 | 259 | # nuke hosts.nginx (vhost entries to be appended below): 260 | rm -f "$TOPGEN_VARETC/hosts.nginx" 261 | 262 | # process each vhost: 263 | for VH in $TOPGEN_VHOSTS/*; do 264 | VB=${VH##*/} # basename 265 | 266 | # issue certificate (based on csr generated on-the-fly): 267 | openssl ca -batch -notext \ 268 | -config <(echo "$TMP_CA_CONF") -out "$TOPGEN_CERTS/$VB.cer" \ 269 | -in <(openssl req -new -key "$TOPGEN_VARETC/topgen_vh.key" \ 270 | -subj '/C=US/ST=PA/L=Pgh/O=CMU/OU=CERT/CN=topgen_vh' \ 271 | -config <(echo "$TMP_VH_CONF" 272 | echo -e "[alt_names]\nDNS.1 = $VB") \ 273 | 2>/dev/null) \ 274 | 2>/dev/null 275 | 276 | # append vhost https block to nginx.conf file: 277 | cat >> "$TOPGEN_VARETC/nginx.conf" <<- EOT 278 | 279 | server { 280 | listen 80; 281 | listen 443 ssl; 282 | ssl_certificate $TOPGEN_CERTS/$VB.cer; 283 | server_name $VB; 284 | root $VH; 285 | } 286 | EOT 287 | 288 | # resolve vhost DNS IP address, write to hosts.nginx 289 | VHIP=$(getent ahostsv4 $VB | head -1 | cut -d' ' -f1) 290 | # use made-up IP '1.1.1.1' for unresolvable vhosts: 291 | echo "${VHIP:-1.1.1.1} $VB" >> "$TOPGEN_VARETC/hosts.nginx" 292 | done 293 | 294 | # done with CA directory: 295 | rm -rf "$TMP_CA_DIR" 296 | -------------------------------------------------------------------------------- /sbin/topgen-tor.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Generate TOR DA keys and config file for TopGen 4 | # (glsomlo@cert.org, May 2018) 5 | 6 | # TopGen home directory: 7 | TG_DIR='/var/lib/topgen' 8 | 9 | # TOR DA fqdn: 10 | TDA_HOST='topgen.info' 11 | 12 | ########################################################################### 13 | #### NO FURTHER USER-SERVICEABLE PARTS BEYOND THIS POINT !!!!!!! #### 14 | ########################################################################### 15 | 16 | PATH='/usr/bin:/usr/sbin' 17 | 18 | # if "yes", force/overwrite prior/existing configuration 19 | FORCE_GEN='no' 20 | 21 | # usage blurb (to be printed with error messages): 22 | USAGE_BLURB=" 23 | Usage: $0 [-t ] [-n ] 24 | 25 | The optional command line arguments are: 26 | 27 | -t base directory relative to which all 28 | other configuration is located; 29 | (default: $TG_DIR). 30 | 31 | -n fully qualified host name of the TOR 32 | Directory Authority, also shared with 33 | the virtual Web host representing the 34 | simulator's front page; 35 | (default: $TDA_HOST). 36 | 37 | -f forcibly overwrite and re-create any 38 | pre-existing configuration elements. 39 | 40 | Generate TOR Directory Authority keys and a corresponding sample 41 | config file. A link to the config file will also be inserted into 42 | the virtual Web host representing the simulator's front page. 43 | " # end usage blurb 44 | 45 | # process command line options: 46 | OPTIND=1 47 | while getopts "fh?" OPT; do 48 | case "$OPT" in 49 | t) 50 | TG_DIR=$OPTARG 51 | ;; 52 | n) 53 | TDA_HOST=$OPTARG 54 | ;; 55 | f) 56 | FORCE_GEN='yes' 57 | ;; 58 | *) 59 | echo "$USAGE_BLURB" 60 | exit 0 61 | ;; 62 | esac 63 | done 64 | 65 | # we should be left with NO FURTHER arguments on the command line: 66 | shift $((OPTIND-1)) 67 | [ -z "$@" ] || { 68 | echo " 69 | ERROR: invalid argument: $@ 70 | 71 | $USAGE_BLURB 72 | " 73 | exit 1 74 | } 75 | 76 | # string matching mention of the Tor DA in various files: 77 | TDA_STR='TOR Directory Authority' 78 | 79 | # once $TG_CFG is set, these are the relevant elements underneath: 80 | 81 | # TopGen config directory: 82 | TG_CFG="$TG_DIR/etc" 83 | 84 | # TopGen Web hosts file: 85 | TG_WHF="$TG_CFG/hosts.nginx" 86 | 87 | # TOR config directory: 88 | TOR_CFG="$TG_CFG/tor" 89 | 90 | # TopGen vhost folder and index.html file: 91 | TG_VHOST="$TG_DIR/vhosts/$TDA_HOST" 92 | TG_INDEX="$TG_VHOST/index.html" 93 | 94 | # assert existence of required input files and target folders: 95 | [ -s "$TG_WHF" -a -s "$TG_INDEX" ] || { 96 | echo " 97 | ERROR: files \"$TG_WHF\" and 98 | \"$TG_INDEX\" MUST exist 99 | before running this command! 100 | " 101 | exit 1 102 | } 103 | 104 | # parse DA ip-addr from TopGen Web hosts file: 105 | TDA_ADDR=$(grep " $TDA_HOST$" "$TG_WHF" | cut -d' ' -f1) 106 | [ -n "$TDA_ADDR" ] || { 107 | echo " 108 | ERROR: Could not parse IP address of DA host \"$TDA_HOST\" 109 | from file \"$TG_WHF\"! 110 | " 111 | exit 1 112 | } 113 | 114 | # remove previously existing settings if "-f" flag was invoked: 115 | [ "$FORCE_GEN" == "yes" ] && { 116 | rm -rf "$TOR_CFG" 117 | sed -i "/$TDA_STR/d" "$TG_INDEX" 118 | } 119 | 120 | # at this point there should not be any TOR DA configuration present: 121 | [ -d "$TOR_CFG" ] || grep -q "$TDA_STR" "$TG_INDEX" && { 122 | echo " 123 | ERROR: directory \"$TOR_CFG\" MUST NOT exist, and file 124 | \"$TG_INDEX\" MUST NOT reference 125 | a \"$TDA_STR\" before running this command! 126 | 127 | $USAGE_BLURB 128 | " 129 | exit 1 130 | } 131 | 132 | # generate router keys for the DA: 133 | tor --datadirectory $TOR_CFG --list-fingerprint --orport 1 \ 134 | --dirauthority "127.0.0.1:1 ffffffffffffffffffffffffffffffffffffffff" | \ 135 | grep '\[err\]' 136 | 137 | [ -s "$TOR_CFG/fingerprint" -a -d "$TOR_CFG/keys" ] || { 138 | echo " 139 | ERROR: Command to create DA router keys failed to create file: 140 | \"$TOR_CFG/fingerprint\" 141 | and directory: 142 | \"$TOR_CFG/keys\"! 143 | " 144 | exit 1 145 | } 146 | 147 | # generate DA certificate: 148 | ( 149 | cd "$TOR_CFG/keys" 150 | tor-gencert --create-identity-key -m 24 -a $TDA_ADDR:7000 \ 151 | --passphrase-fd 0 << "$TOR_CFG/torrc" <<- EOT 167 | # common settings 168 | RunAsDaemon 1 169 | TestingTorNetwork 1 170 | AssumeReachable 1 171 | TestingConsensusMaxDownloadTries 2 172 | UseDefaultFallbackDirs 0 173 | DataDirectory /var/lib/tor 174 | PidFile /var/lib/tor/pid 175 | #Log notice file /var/lib/tor/notice.log 176 | Log info file /var/lib/tor/info.log 177 | SafeLogging 0 178 | 179 | # generated directory authority: 180 | DirAuthority orport=5000 v3ident=$TDA_V3ID $TDA_ADDR:7000 $TDA_FP 181 | 182 | # instance-specific settings (pass in via cmdline as needed): 183 | 184 | # routers (incl. dir-auth): 185 | #ORPort 5000 186 | 187 | # directory server(s) only: 188 | #DirPort 7000 189 | 190 | # directory authority only: 191 | #Address $TDA_ADDR 192 | #OutboundBindAddress $TDA_ADDR 193 | #AuthoritativeDirectory 1 194 | #V3AuthoritativeDirectory 1 195 | #V3AuthVotingInterval 10 196 | #V3AuthVoteDelay 2 197 | #V3AuthDistDelay 2 198 | 199 | # hidden services (e.g., ssh and http): 200 | # NOTE: make sure daemons listen *only* on 127.0.0.1 (not "any")!!! 201 | #HiddenServiceDir /var/lib/tor/my_hidden_svcs 202 | #HiddenServicePort 80 127.0.0.1:80 203 | #HiddenServicePort 22 127.0.0.1:22 204 | EOT 205 | 206 | # this will be referenced by the systemd 'topgen-tor-da.service' unit file: 207 | echo "TDA_ADDR='$TDA_ADDR'" > "$TOR_CFG/tda_addr" 208 | 209 | # clean up lock file 210 | rm -f "$TOR_CFG/lock" 211 | 212 | # add reference to TOR DA into Web vhost front page: 213 | cp -f "$TOR_CFG/torrc" "$TG_VHOST" 214 | TDA_PARA="This is also the simulated $TDA_STR: use this configfile for all in-game TOR nodes!

" 215 | sed -i "/Below is a list of Web sites/i $TDA_PARA" "$TG_INDEX" 216 | 217 | # FIXME: Sort out SELinux policy associated with topgen package !!! 218 | # But, for now, let's label the relevant files and folders manually: 219 | chcon -R -t tor_etc_t "$TOR_CFG" 220 | # also, allow DA binding to ports 5000 and 7000: 221 | setsebool -P tor_bind_all_unreserved_ports 1 222 | 223 | echo "Success!!!" 224 | -------------------------------------------------------------------------------- /sbin/topgen-vmail.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Configure virtual mail service for TopGen 4 | # (glsomlo@cert.org, February 2016) 5 | 6 | # input: topgen virtual mail config file 7 | VMAIL_CFG='/etc/topgen/vmail.cfg' 8 | 9 | # topgen directory structure: 10 | TOPGEN_VARLIB='/var/lib/topgen' 11 | 12 | # if "yes", do not print out warnings: 13 | let VERBOSE=0 14 | 15 | # if "yes", force/overwrite any prior existing configuration 16 | FORCE_GEN='no' 17 | 18 | ########################################################################### 19 | #### NO FURTHER USER-SERVICEABLE PARTS BEYOND THIS POINT !!!!!!! #### 20 | ########################################################################### 21 | 22 | USAGE_BLURB=" 23 | Usage: $0 [-c ] [-t ] 24 | 25 | The optional command line arguments are: 26 | 27 | -c file containing virtual mail domain configuration 28 | entries, one per line. The format of each line is: 29 | . : :... 30 | (default: $VMAIL_CFG). 31 | 32 | -t directory where all resulting configuration files 33 | are stored; 34 | (default: $TOPGEN_VARLIB). 35 | 36 | -f don't stop if encountering pre-existing configuration; 37 | instead, forcibly remove & re-create configuration. 38 | CAUTION: pre-existing configuration will be lost! 39 | 40 | -v increase verbosity level (may be used multiple times); 41 | (default: $VERBOSE). 42 | 43 | Generate virtual mail server configuration for the (domain, server, users) 44 | sets given in a configuration file. Place configuration data under 45 | /etc/postfix/, and ensure /vmail/ exists 46 | and is owned by dovenull:dovenull. 47 | " # end usage blurb 48 | 49 | # process command line options: 50 | OPTIND=1 51 | while getopts "c:t:fvh?" OPT; do 52 | case "$OPT" in 53 | s) 54 | VMAIL_CFG=$OPTARG 55 | ;; 56 | t) 57 | TOPGEN_VARLIB=$OPTARG 58 | ;; 59 | f) 60 | FORCE_GEN='yes' 61 | ;; 62 | v) 63 | let VERBOSE++ 64 | ;; 65 | *) 66 | echo "$USAGE_BLURB" 67 | exit 0 68 | ;; 69 | esac 70 | done 71 | 72 | # we should be left with NO FURTHER arguments on the command line: 73 | shift $((OPTIND-1)) 74 | [ -z "$@" ] || { 75 | echo " 76 | ERROR: invalid argument: $@ 77 | 78 | $USAGE_BLURB 79 | " 80 | exit 1 81 | } 82 | 83 | # once $TOPGEN_VARLIB is set, these are its relevant subdirectories: 84 | VMAIL_CFGDIR="$TOPGEN_VARLIB/etc/postfix" 85 | VMAIL_MBOXDIR="$TOPGEN_VARLIB/vmail" 86 | VMAIL_HOSTFILE="$TOPGEN_VARLIB/etc/hosts.vmail" 87 | 88 | # assert existence of required input files and target folders: 89 | [ -s "$VMAIL_CFG" -a \ 90 | -d "$VMAIL_CFGDIR" -a -d "$VMAIL_MBOXDIR" ] || { 91 | echo " 92 | ERROR: file \"$VMAIL_CFG\" and 93 | folders \"$VMAIL_CFGDIR\", and 94 | \"$VMAIL_MBOXDIR\" MUST exist 95 | before running this command! 96 | 97 | $USAGE_BLURB 98 | " 99 | exit 1 100 | } 101 | 102 | # assert vmail config folder is empty: 103 | [ "$FORCE_GEN" == "yes" ] && rm -rf $VMAIL_CFGDIR/* 104 | [ -n "$(ls -A $VMAIL_CFGDIR)" ] && { 105 | echo " 106 | ERROR: folder \"$VMAIL_CFGDIR\" MUST be empty 107 | before running this command! 108 | 109 | $USAGE_BLURB 110 | " 111 | exit 1 112 | } 113 | 114 | # assert non-existence of vmail hosts file 115 | [ "$FORCE_GEN" == "yes" ] && rm -rf $VMAIL_HOSTFILE 116 | [ -s "$VMAIL_HOSTFILE" ] && { 117 | echo " 118 | ERROR: file \"$VMAIL_HOSTFILE\" must NOT exist! 119 | Please remove it manually before re-running this command! 120 | 121 | $USAGE_BLURB 122 | " 123 | exit 1 124 | } 125 | 126 | # assert mailbox data folder is owned by dovenull: 127 | [ "$(stat -c %U $VMAIL_MBOXDIR)" != "dovenull" ] && { 128 | echo " 129 | ERROR: folder \"$VMAIL_MBOXDIR\" MUST be owned by \"dovenull\"! 130 | 131 | $USAGE_BLURB 132 | " 133 | exit 1 134 | } 135 | 136 | 137 | # Start by generating domain-specific configuration entries 138 | # from the parsed content of $VMAIL_CFG: 139 | # 140 | cat $VMAIL_CFG | grep -v ^# | while read HOST ADDR USERS; do 141 | 142 | # skip if no users: 143 | [ -n "$USERS" ] || continue 144 | 145 | DOMAIN=${HOST#*.} 146 | 147 | # add ipaddr-fqdn pair to vmail specific hosts file: 148 | echo "$ADDR $HOST" >> $VMAIL_HOSTFILE 149 | 150 | # add domain to pf_virt_dom (for postfix:master.cf:$virtual_mailbox_domains) 151 | echo $DOMAIN >> $VMAIL_CFGDIR/pf_virt_dom 152 | 153 | # add domain-specific outbound sender to postfix:master.cf and pf_dom_send: 154 | cat >> $VMAIL_CFGDIR/master.cf <<- EOT 155 | mx_$DOMAIN 156 | unix - - n - - smtp 157 | -o smtp_helo_name=$DOMAIN 158 | -o smtp_bind_address=$ADDR 159 | EOT 160 | # (for postfix:master.cf:$sender_dependent_default_transport_maps) 161 | echo "$DOMAIN mx_$DOMAIN" >> $VMAIL_CFGDIR/pf_dom_send 162 | 163 | # add domain imap server ip to dc_addr_dom (for dovecot:dovecot.conf) 164 | echo "$ADDR:::::::domain=$DOMAIN" >> $VMAIL_CFGDIR/dc_addr_dom 165 | 166 | # create domain users (pf_usr_mbox for mail dirs, dc_usr_pass for auth): 167 | for U in $USERS; do 168 | USER=${U%:*} 169 | PASS=${U#*:} 170 | echo "$USER@$DOMAIN $DOMAIN/$USER/" >> $VMAIL_CFGDIR/pf_usr_mbox 171 | echo "$USER@$DOMAIN:{PLAIN}$PASS" >> $VMAIL_CFGDIR/dc_usr_pass 172 | done 173 | 174 | done 175 | 176 | # If, for some reason, no domain-specific config entries were generated 177 | # (e.g., empty $VMAIL_CFG file), skip generating meaningless global configs: 178 | [ -s "$VMAIL_HOSTFILE" -a -s "$VMAIL_CFGDIR/pf_virt_dom" -a \ 179 | -s "$VMAIL_CFGDIR/master.cf" -a -s "$VMAIL_CFGDIR/pf_dom_send" -a \ 180 | -s "$VMAIL_CFGDIR/dc_addr_dom" -a -s "$VMAIL_CFGDIR/pf_usr_mbox" -a \ 181 | -s "$VMAIL_CFGDIR/dc_usr_pass" ] || exit 0 182 | 183 | # generate fixed (components of) postfix & dovecot config files: 184 | # 185 | 186 | DOVENULL_ID=$(getent passwd dovenull | cut -d: -f3,4) 187 | 188 | cat > $VMAIL_CFGDIR/main.cf <<- EOT 189 | # Local mail: when people email @topgen.info, local aliases get 190 | # mail delivered inside the topgen container, in /var/spool/mail/ 191 | # 192 | mydomain = topgen.info 193 | myhostname = greybox.topgen.info 194 | mynetworks = 127.0.0.0/8 195 | mydestination = localhost, \$myhostname, localhost.\$mydomain, \$mydomain 196 | # if e.g. root tries to read its @topgen.info mail via cmdline, 197 | # we need these entries to be local to our container, 198 | # and match expected defaults: 199 | data_directory = /var/lib/postfix 200 | queue_directory = /var/spool/postfix 201 | mail_spool_directory = /var/spool/mail 202 | 203 | # virtual multi-domain mailserver setup: 204 | # 205 | virtual_mailbox_domains = $VMAIL_CFGDIR/pf_virt_dom 206 | virtual_mailbox_base = $VMAIL_MBOXDIR 207 | virtual_mailbox_maps = 208 | texthash:$VMAIL_CFGDIR/pf_usr_mbox 209 | static:DFLT 210 | # use already existing dovenull account numerical uid/gid: 211 | virtual_uid_maps = static:${DOVENULL_ID%:*} 212 | virtual_gid_maps = static:${DOVENULL_ID#*:} 213 | # if recipient domain not local, use correct sender "personality": 214 | # (NOTE: map sender domains to transport/service entries in master.cf) 215 | sender_dependent_default_transport_maps = 216 | texthash:$VMAIL_CFGDIR/pf_dom_send 217 | 218 | # force remote clients to authenticate: 219 | # (NOTE: see also "submission" entry in master.cf; 220 | # "smtps", identically configured, would listen 221 | # on deprecated port 465) 222 | # 223 | smtpd_sasl_auth_enable = yes 224 | smtpd_sasl_type = dovecot 225 | # auth service socket from dovecot (relative to \$queue_directory): 226 | smtpd_sasl_path = private/auth 227 | smtpd_client_restrictions = 228 | permit_mynetworks, 229 | permit_sasl_authenticated, 230 | reject 231 | # enable compatibility with outlook/exchange smtp auth: 232 | broken_sasl_auth_clients = yes 233 | 234 | # force remote clients to use encryption: 235 | # 236 | smtpd_tls_security_level = encrypt 237 | smtpd_tls_cert_file = $VMAIL_CFGDIR/pf_tls.cer 238 | smtpd_tls_key_file = $VMAIL_CFGDIR/pf_tls.key 239 | EOT 240 | 241 | cat > $VMAIL_CFGDIR/dovecot.conf <<- EOT 242 | log_path = /var/log/dovecot.log 243 | mbox_write_locks = fcntl 244 | 245 | ssl = required 246 | ssl_cert = <$VMAIL_CFGDIR/dc_tls.cer 247 | ssl_key = <$VMAIL_CFGDIR/dc_tls.key 248 | 249 | auth_mechanisms = plain login cram-md5 250 | 251 | # set domain based on local IP used by client: 252 | passdb { 253 | driver = passwd-file 254 | args = username_format=%l $VMAIL_CFGDIR/dc_addr_dom 255 | result_success = continue 256 | } 257 | 258 | passdb { 259 | driver = passwd-file 260 | args = $VMAIL_CFGDIR/dc_usr_pass 261 | } 262 | 263 | # match postfix location: 264 | mail_location = maildir:$VMAIL_MBOXDIR/%d/%n 265 | 266 | userdb { 267 | driver = static 268 | args = uid=dovenull gid=dovenull home=$VMAIL_MBOXDIR/%d/%n 269 | } 270 | 271 | # auth service available to postfix 272 | # (as postfix:main.cf:\$smtpd_sasl_path): 273 | service auth { 274 | # "/var/spool/postfix" is postfix:main.cf:\$queue_directory; 275 | unix_listener /var/spool/postfix/private/auth { 276 | mode = 0660 277 | user = postfix 278 | group = postfix 279 | } 280 | } 281 | EOT 282 | 283 | # this must be *prepended* to what's currently in master.cf: 284 | sed -i '1 { 285 | h 286 | r /dev/stdin 287 | g 288 | N 289 | }' $VMAIL_CFGDIR/master.cf <<- "EOT" 290 | # service type private unpriv chroot wakeup maxproc cmd+args 291 | # name (yes) (yes) (no) (never) (100) 292 | # ====================================================================== 293 | smtp inet n - n - - smtpd 294 | submission inet n - n - - smtpd 295 | -o smtpd_sasl_auth_enable=yes 296 | -o smtpd_recipient_restrictions=permit_sasl_authenticated,reject 297 | -o milter_macro_daemon_name=ORIGINATING 298 | pickup unix n - n 60 1 pickup 299 | cleanup unix n - n - 0 cleanup 300 | qmgr unix n - n 300 1 qmgr 301 | tlsmgr unix - - n 1000? 1 tlsmgr 302 | rewrite unix - - n - - trivial-rewrite 303 | bounce unix - - n - 0 bounce 304 | defer unix - - n - 0 bounce 305 | trace unix - - n - 0 bounce 306 | verify unix - - n - 1 verify 307 | flush unix n - n 1000? 0 flush 308 | proxymap unix - - n - - proxymap 309 | proxywrite unix - - n - 1 proxymap 310 | smtp unix - - n - - smtp 311 | relay unix - - n - - smtp 312 | showq unix n - n - - showq 313 | error unix - - n - - error 314 | retry unix - - n - - error 315 | discard unix - - n - - discard 316 | local unix - n n - - local 317 | virtual unix - n n - - virtual 318 | lmtp unix - - n - - lmtp 319 | anvil unix - - n - 1 anvil 320 | scache unix - - n - 1 scache 321 | EOT 322 | 323 | #FIXME: maybe this could be the same cert+key, but then we'll HAVE to figure 324 | # out SELinux labeling (since postfix wants postfix_etc_t and dovecot 325 | # wants dovecot_etc_t, and right now one can't read files labeled for 326 | # the other !!! 327 | openssl req -subj '/C=US/ST=PA/L=Pgh/O=CMU/OU=CERT/CN=smtp.topgen.info' \ 328 | -newkey rsa:2048 -nodes -keyout $VMAIL_CFGDIR/pf_tls.key \ 329 | -days 7300 -x509 -out $VMAIL_CFGDIR/pf_tls.cer 2>/dev/null 330 | openssl req -subj '/C=US/ST=PA/L=Pgh/O=CMU/OU=CERT/CN=imap.topgen.info' \ 331 | -newkey rsa:2048 -nodes -keyout $VMAIL_CFGDIR/dc_tls.key \ 332 | -days 7300 -x509 -out $VMAIL_CFGDIR/dc_tls.cer 2>/dev/null 333 | 334 | # FIXME: Sort out SELinux policy associated with topgen package !!! 335 | # But, for now, let's label the relevant files and folders manually: 336 | chcon -t dovecot_etc_t $VMAIL_CFGDIR/dovecot.conf $VMAIL_CFGDIR/dc_* 337 | chcon -t postfix_etc_t $VMAIL_CFGDIR/*.cf $VMAIL_CFGDIR/pf_* 338 | chcon -R -t mail_spool_t $VMAIL_MBOXDIR 339 | -------------------------------------------------------------------------------- /systemd/topgen-dovecot.service.d/topgen-dovecot.conf: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=TopGen (Dovecot-based) IMAP server 3 | ConditionPathExists=/var/lib/topgen/etc/postfix/dovecot.conf 4 | Wants=topgen-loopback.service 5 | 6 | [Service] 7 | ExecStart= 8 | ExecStart=/usr/sbin/dovecot -F -c /var/lib/topgen/etc/postfix/dovecot.conf 9 | ExecReload= 10 | ExecReload=/usr/sbin/dovecot -c /var/lib/topgen/etc/postfix/dovecot.conf reload 11 | -------------------------------------------------------------------------------- /systemd/topgen-loopback.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=TopGen Loopback Interface Secondary IP Configuration Service 3 | After=network.target 4 | 5 | [Service] 6 | Type=oneshot 7 | RemainAfterExit=yes 8 | # systemd doesn't support compound commands, so we cheat a little: 9 | ExecStart=/usr/bin/sh -c "for i in $(cat /var/lib/topgen/etc/hosts.* | awk '{print $1}' | sort -u); do ip addr add $i scope global dev lo; done" 10 | ExecStop=/usr/sbin/ip addr flush scope global dev lo 11 | 12 | [Install] 13 | WantedBy=multi-user.target 14 | -------------------------------------------------------------------------------- /systemd/topgen-named-config.service: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=TopGen DNS Configuration Service 3 | # we absolutely need nginx hosts and delegation info: 4 | ConditionPathExists=/var/lib/topgen/etc/hosts.nginx 5 | ConditionPathExists=/etc/topgen/delegations.dns 6 | # start if either of the following is missing: 7 | ConditionPathExists=|!/var/lib/topgen/etc/hosts.named 8 | ConditionPathExists=|!/var/lib/topgen/named/rootsrv 9 | ConditionPathExists=|!/var/lib/topgen/named/tldsrv 10 | PartOf=topgen-named.service 11 | 12 | [Service] 13 | Type=oneshot 14 | RemainAfterExit=yes 15 | ExecStart=/usr/sbin/topgen-mkdns.sh -fq 16 | # just added a bunch of hosts (hosts.named), must condrestart loopback service: 17 | ExecStart=/usr/bin/systemctl try-restart topgen-loopback.service 18 | -------------------------------------------------------------------------------- /systemd/topgen-named.service.d/topgen-named.conf: -------------------------------------------------------------------------------- 1 | [Unit] 2 | # replace original named service (can't coexist with it): 3 | Description=TopGen DNS Service 4 | # configuration must be present and loopbacks must be set up: 5 | After=topgen-named-config.service topgen-loopback.service 6 | Wants=topgen-named-config.service 7 | # ensure we're restarted when loopbacks change: 8 | # (named hates having its listening IPs yanked out, so we can't use Wants here) 9 | Requires=topgen-loopback.service 10 | 11 | [Service] 12 | # we replace (not append to) ExecStartPre and ExecStart: 13 | ExecStartPre= 14 | ExecStartPre=/usr/sbin/named-checkconf -z /var/lib/topgen/etc/named.conf 15 | ExecStart= 16 | ExecStart=/usr/sbin/named -u named -c /var/lib/topgen/etc/named.conf 17 | -------------------------------------------------------------------------------- /systemd/topgen-nginx.service.d/topgen-nginx.conf: -------------------------------------------------------------------------------- 1 | [Unit] 2 | # replace original nginx service (can't coexist with it): 3 | Description=TopGen HTTP (nginx) Service 4 | # we need nginx hosts and config file: 5 | ConditionPathExists=/var/lib/topgen/etc/hosts.nginx 6 | ConditionPathExists=/var/lib/topgen/etc/nginx.conf 7 | # loopbacks must be set up (eventually, no need to specify 'After'): 8 | Wants=topgen-loopback.service 9 | 10 | [Service] 11 | # don't time out if nginx takes a while to load: 12 | TimeoutSec=0 13 | -------------------------------------------------------------------------------- /systemd/topgen-postfix.service.d/topgen-postfix.conf: -------------------------------------------------------------------------------- 1 | [Unit] 2 | Description=TopGen (Postfix-based) smtp server 3 | ConditionPathExists=/var/lib/topgen/etc/postfix/main.cf 4 | ConditionPathExists=/var/lib/topgen/etc/postfix/master.cf 5 | Wants=topgen-loopback.service 6 | 7 | [Service] 8 | Environment="MAIL_CONFIG=/var/lib/topgen/etc/postfix" 9 | -------------------------------------------------------------------------------- /systemd/topgen-tor.service.d/topgen-tor.conf: -------------------------------------------------------------------------------- 1 | [Unit] 2 | # replace original tor service: 3 | Description=TopGen TOR Directory Authority 4 | # make sure the config file exists: 5 | ConditionPathExists=/var/lib/topgen/etc/tor/torrc 6 | # we need DNS resolution and loopbacks to be up: 7 | After=topgen-named.service topgen-loopback.service 8 | Wants=topgen-named.service 9 | 10 | [Service] 11 | Environment=TDA_ADDR='127.0.0.1' 12 | EnvironmentFile=-/var/lib/topgen/etc/tor/tda_addr 13 | ExecStartPre= 14 | ExecStartPre=/usr/bin/cp -r /var/lib/topgen/etc/tor/keys /var/lib/tor/ 15 | ExecStartPre=/usr/bin/chown -R toranon /var/lib/tor/keys 16 | ExecStartPre=/usr/bin/tor --runasdaemon 0 --defaults-torrc /usr/share/tor/defaults-torrc -f /var/lib/topgen/etc/tor/torrc --verify-config 17 | ExecStart= 18 | ExecStart=/usr/bin/tor --runasdaemon 0 --defaults-torrc /usr/share/tor/defaults-torrc -f /var/lib/topgen/etc/tor/torrc --orport 5000 --dirport 7000 --nickname topgen --address ${TDA_ADDR} --outboundbindaddress ${TDA_ADDR} --authoritativedirectory 1 --v3authoritativedirectory 1 --v3authvotinginterval 10 --v3authvotedelay 2 --v3authdistdelay 2 --exitpolicy 'reject *:*' 19 | --------------------------------------------------------------------------------