├── bin └── .gitkeep ├── cids └── .gitkeep ├── shared └── .gitkeep ├── containers └── .gitkeep ├── image ├── base │ ├── VERSION │ ├── cron │ ├── runit-1.d-anacron │ ├── runit-2 │ ├── runit-3 │ ├── runit-1 │ ├── runit-1.d-cleanup-pids │ ├── rsyslog │ ├── cron.d_anacron │ ├── install-pngquant │ ├── runit-1.d-00-fix-var-logs │ ├── install-gifsicle │ ├── boot │ ├── install-pngcrush │ ├── install-redis │ ├── thpoff.c │ ├── install-nginx │ ├── install-imagemagick │ └── Dockerfile ├── discourse_dev │ ├── sudoers.discourse │ ├── ensure-database │ ├── Dockerfile │ └── postgres_dev.template.yml ├── monitor │ ├── Dockerfile │ └── src │ │ └── monitor.rb ├── discourse_bench │ ├── run_bench.sh │ └── Dockerfile ├── discourse_fast_switch │ ├── create_switch.rb │ └── Dockerfile ├── auto_build.rb ├── discourse_test │ └── Dockerfile └── README.md ├── templates ├── sshd.template.yml ├── syslog.template.yml ├── README.md ├── cron.template.yml ├── fastly.template.yml ├── web.china.template.yml ├── import │ ├── chrome-dep.template.yml │ ├── mysql-dep.template.yml │ ├── mssql-dep.template.yml │ ├── mbox.template.yml │ └── phpbb3.template.yml ├── cache-dns.template.yml ├── web.socketed.template.yml ├── cloudflare.template.yml ├── web.ratelimited.template.yml ├── web.onion.template.yml ├── web.ssl.template.yml ├── redis.template.yml ├── web.letsencrypt.ssl.template.yml ├── postgres.12.template.yml ├── postgres.9.5.template.yml ├── postgres.10.template.yml ├── postgres.template.yml └── web.template.yml ├── samples ├── redis.yml ├── data.yml ├── mail-receiver.yml ├── web_only.yml └── standalone.yml ├── .gitignore ├── LICENSE ├── scripts └── mk_swapfile ├── README.md ├── discourse-doctor ├── discourse-setup └── launcher /bin/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /cids/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /shared/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /containers/.gitkeep: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /image/base/VERSION: -------------------------------------------------------------------------------- 1 | 1.4.0 2 | -------------------------------------------------------------------------------- /image/base/cron: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | exec 2>&1 3 | cd / 4 | exec cron -f 5 | -------------------------------------------------------------------------------- /image/base/runit-1.d-anacron: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | /usr/sbin/anacron -s 3 | -------------------------------------------------------------------------------- /image/discourse_dev/sudoers.discourse: -------------------------------------------------------------------------------- 1 | discourse ALL = NOPASSWD: ALL 2 | -------------------------------------------------------------------------------- /image/base/runit-2: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | exec /usr/bin/runsvdir -P /etc/service 3 | -------------------------------------------------------------------------------- /image/base/runit-3: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | /bin/run-parts --verbose /etc/runit/3.d 4 | -------------------------------------------------------------------------------- /templates/sshd.template.yml: -------------------------------------------------------------------------------- 1 | # This file is deprecated; you can remove it from your app.yml 2 | run: 3 | -------------------------------------------------------------------------------- /image/base/runit-1: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | /bin/run-parts --verbose --exit-on-error /etc/runit/1.d || exit 100 4 | -------------------------------------------------------------------------------- /templates/syslog.template.yml: -------------------------------------------------------------------------------- 1 | run: 2 | - exec: echo rsyslog template is included in base image, remove 3 | 4 | -------------------------------------------------------------------------------- /image/base/runit-1.d-cleanup-pids: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | /bin/echo "Cleaning stale PID files" 3 | /bin/rm -f /var/run/*.pid 4 | -------------------------------------------------------------------------------- /templates/README.md: -------------------------------------------------------------------------------- 1 | [pups](https://github.com/samsaffron/pups)-managed templates you may use to bootstrap your environment. 2 | -------------------------------------------------------------------------------- /templates/cron.template.yml: -------------------------------------------------------------------------------- 1 | run: 2 | - exec: 3 | hook: cron 4 | cmd: echo cron is now included in base image, remove from templates 5 | -------------------------------------------------------------------------------- /image/base/rsyslog: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | exec 2>&1 3 | cd / 4 | chgrp syslog /var/log 5 | chmod g+w /var/log 6 | rm -f /var/run/rsyslogd.pid 7 | exec rsyslogd -n 8 | -------------------------------------------------------------------------------- /templates/fastly.template.yml: -------------------------------------------------------------------------------- 1 | run: 2 | - replace: 3 | filename: "/etc/nginx/conf.d/discourse.conf" 4 | from: "$proxy_add_x_forwarded_for" 5 | to: "$http_fastly_client_ip" 6 | global: true 7 | -------------------------------------------------------------------------------- /image/base/cron.d_anacron: -------------------------------------------------------------------------------- 1 | # this avoids using invoke-rc.d which is bust in image 2 | 3 | SHELL=/bin/sh 4 | PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin 5 | 6 | 30 7 * * * root /usr/sbin/anacron -s >/dev/null 7 | -------------------------------------------------------------------------------- /image/base/install-pngquant: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | # version check: https://pngquant.org/ 5 | VERSION=2.12.5 6 | 7 | cd /tmp 8 | git clone -b $VERSION --single-branch https://github.com/pornel/pngquant 9 | cd pngquant 10 | make && make install 11 | cd / 12 | rm -fr /tmp/pngq* 13 | -------------------------------------------------------------------------------- /templates/web.china.template.yml: -------------------------------------------------------------------------------- 1 | hooks: 2 | before_web: 3 | - exec: 4 | cmd: 5 | - gem sources --add https://gems.ruby-china.com/ --remove https://rubygems.org/ 6 | 7 | before_bundle_exec: 8 | - exec: 9 | cmd: 10 | - su discourse -c 'bundle config mirror.https://rubygems.org https://gems.ruby-china.com/' 11 | -------------------------------------------------------------------------------- /image/base/runit-1.d-00-fix-var-logs: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | mkdir -p /var/log/nginx 3 | chown -R www-data:www-data /var/log/nginx 4 | chmod -R 644 /var/log/nginx 5 | chmod 755 /var/log/nginx 6 | touch /var/log/syslog && chown -f root:adm /var/log/syslog* 7 | touch /var/log/auth.log && chown -f root:adm /var/log/auth.log* 8 | touch /var/log/kern.log && chown -f root:adm /var/log/kern.log* 9 | -------------------------------------------------------------------------------- /samples/redis.yml: -------------------------------------------------------------------------------- 1 | templates: 2 | - "templates/redis.template.yml" 3 | 4 | env: 5 | LANG: en_US.UTF-8 6 | 7 | # any extra arguments for Docker? 8 | # docker_args: 9 | 10 | volumes: 11 | - volume: 12 | host: /var/discourse/shared/redis 13 | guest: /shared 14 | - volume: 15 | host: /var/discourse/shared/redis/log/var-log 16 | guest: /var/log 17 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | shared/* 2 | *.swp 3 | *~ 4 | \#*\# 5 | containers/* 6 | cids/* 7 | bin/* 8 | image/.build.out 9 | image/.build.hash 10 | image/.docker_temp_* 11 | image/img.tar 12 | image/squash.tar 13 | image/nsenter/nsenter 14 | image/docker-squash 15 | image/docker-squash.tar.gz 16 | image/discourse_dev/postgres.template.yml 17 | image/discourse_dev/redis.template.yml 18 | .gc-state/* 19 | .vagrant/ 20 | -------------------------------------------------------------------------------- /image/discourse_dev/ensure-database: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | if [ ! -f /shared/postgres_data/PG_VERSION ]; then 4 | mkdir -p /shared/postgres_data 5 | chown -R postgres:postgres /shared/postgres_data 6 | chmod 700 /shared/postgres_data 7 | cp -R /shared/postgres_data_orig/* /shared/postgres_data 8 | chown -R postgres:postgres /shared/postgres_data 9 | chmod 700 /shared/postgres_data 10 | fi 11 | -------------------------------------------------------------------------------- /image/monitor/Dockerfile: -------------------------------------------------------------------------------- 1 | # Used to gather information about CPU and memory 2 | # reporting it back to statsd 3 | 4 | # samsaffron/discourse_monitor 5 | # version 0.0.2 6 | 7 | FROM samsaffron/discourse_base:1.0.7 8 | #LABEL maintainer="Sam Saffron \"https://twitter.com/samsaffron\"" 9 | 10 | ADD src/monitor.rb src/monitor.rb 11 | RUN gem install statsd-ruby docker-api 12 | 13 | CMD ruby src/monitor.rb 14 | -------------------------------------------------------------------------------- /image/discourse_bench/run_bench.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # start Redis-Server 4 | redis-server /etc/redis/redis.conf 5 | 6 | # start PostgreSQL 7 | /etc/init.d/postgresql start 8 | 9 | # get latest source 10 | git pull 11 | 12 | # install needed gems 13 | sudo -E -u discourse bundle install 14 | 15 | # start mailcatcher 16 | mailcatcher --http-ip 0.0.0.0 17 | 18 | # run the benchmark 19 | sudo -E -u discourse ruby script/bench.rb 20 | -------------------------------------------------------------------------------- /templates/import/chrome-dep.template.yml: -------------------------------------------------------------------------------- 1 | # This template installs Chrome for imports 2 | 3 | hooks: 4 | after_bundle_exec: 5 | - exec: 6 | cd: /tmp 7 | cmd: 8 | - curl -sS -o - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add 9 | - echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google-chrome.list 10 | - apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y google-chrome-stable -------------------------------------------------------------------------------- /image/base/install-gifsicle: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | # version check: https://www.lcdf.org/gifsicle/ 5 | VERSION=1.92 6 | HASH="5ab556c01d65fddf980749e3ccf50b7fd40de738b6df679999294cc5fabfce65" 7 | 8 | cd /tmp 9 | curl -O http://www.lcdf.org/gifsicle/gifsicle-$VERSION.tar.gz 10 | sha256sum gifsicle-$VERSION.tar.gz 11 | echo "$HASH gifsicle-$VERSION.tar.gz" | sha256sum -c 12 | 13 | tar zxf gifsicle-$VERSION.tar.gz 14 | cd gifsicle-$VERSION 15 | ./configure 16 | make install 17 | cd / 18 | rm -fr /tmp/gifsicle* 19 | -------------------------------------------------------------------------------- /templates/import/mysql-dep.template.yml: -------------------------------------------------------------------------------- 1 | # This template adds the 'mysql2' gem for import scripts depending on it 2 | 3 | params: 4 | home: /var/www/discourse 5 | 6 | hooks: 7 | after_bundle_exec: 8 | - exec: 9 | cd: $home 10 | cmd: 11 | - echo "gem 'mysql2'" >> Gemfile 12 | - apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y libmariadb-dev 13 | - su discourse -c 'bundle config unset deployment' 14 | - su discourse -c 'bundle install --no-deployment --path vendor/bundle --jobs 4 --without test development' 15 | -------------------------------------------------------------------------------- /templates/cache-dns.template.yml: -------------------------------------------------------------------------------- 1 | # You can use this template to cache DNS lookups for 2 | # 3 | # DISCOURSE_DB_HOST 4 | # DISCOURSE_DB_BACKUP_HOST 5 | # DISCOURSE_REDIS_HOST 6 | # DISCOURSE_REDIS_SLAVE_HOST 7 | # 8 | # If you are using a hostname vs IP for these entries and have 9 | # an internal DNS outage Discourse will stop working 10 | # 11 | # This template allows you to cache entry in the hosts file 12 | # 13 | # Note, for this to work correctly you must also install the 14 | # discourse-prometheus plugin to collect stats 15 | 16 | hooks: 17 | after_bundle_exec: 18 | - exec: "mkdir -p /etc/service/cache_critical_dns/" 19 | - exec: "cp /var/www/discourse/script/cache_critical_dns /etc/service/cache_critical_dns/run" 20 | -------------------------------------------------------------------------------- /image/discourse_fast_switch/create_switch.rb: -------------------------------------------------------------------------------- 1 | require 'fileutils' 2 | 3 | puts "-"*100,"creating switch","-"*100 4 | 5 | system("cd /var/www/discourse && git pull") 6 | 7 | ['24', '25'].each do |v| 8 | bin = "/usr/local/bin/use_#{v}" 9 | 10 | File.write(bin, < /dev/null 17 | exit 18 | } 19 | 20 | /etc/runit/1 || exit $? 21 | /etc/runit/2& 22 | RUNSVDIR=$! 23 | echo "Started runsvdir, PID is $RUNSVDIR" 24 | trap shutdown SIGTERM SIGHUP 25 | wait $RUNSVDIR 26 | 27 | shutdown 28 | -------------------------------------------------------------------------------- /image/base/install-pngcrush: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | # version check: https://sourceforge.net/projects/pmt/files/pngcrush/ 5 | PNGCRUSH_VERSION=1.8.13 6 | PNGCRUSH_HASH="bac37d4b2be88d7e88aadcde9661beb3a513a90e7d26784f906c1e2da8ba332e" 7 | 8 | cd /tmp 9 | wget https://sourceforge.net/projects/pmt/files/pngcrush/$PNGCRUSH_VERSION/pngcrush-$PNGCRUSH_VERSION.tar.gz/download -O pngcrush-$PNGCRUSH_VERSION.tar.gz 10 | sha256sum pngcrush-$PNGCRUSH_VERSION.tar.gz 11 | echo "$PNGCRUSH_HASH pngcrush-$PNGCRUSH_VERSION.tar.gz" | sha256sum -c 12 | 13 | tar zxf pngcrush-$PNGCRUSH_VERSION.tar.gz 14 | cd pngcrush-$PNGCRUSH_VERSION 15 | make && cp -f pngcrush /usr/local/bin 16 | cd / 17 | rm -fr /tmp/pngcrush-$PNGCRUSH_VERSION 18 | rm /tmp/pngcrush-$PNGCRUSH_VERSION.tar.gz 19 | -------------------------------------------------------------------------------- /templates/web.socketed.template.yml: -------------------------------------------------------------------------------- 1 | run: 2 | - file: 3 | path: /etc/runit/1.d/remove-old-socket 4 | chmod: "+x" 5 | contents: | 6 | #!/bin/bash 7 | rm -f /shared/nginx.http*.sock 8 | - file: 9 | path: /etc/runit/3.d/remove-old-socket 10 | chmod: "+x" 11 | contents: | 12 | #!/bin/bash 13 | rm -rf /shared/nginx.http*.sock 14 | - replace: 15 | filename: "/etc/nginx/conf.d/discourse.conf" 16 | from: /listen 80;/ 17 | to: | 18 | listen unix:/shared/nginx.http.sock; 19 | set_real_ip_from unix:; 20 | - replace: 21 | filename: "/etc/nginx/conf.d/discourse.conf" 22 | from: /listen 443 ssl http2;/ 23 | to: | 24 | listen unix:/shared/nginx.https.sock ssl http2; 25 | set_real_ip_from unix:; 26 | -------------------------------------------------------------------------------- /templates/import/mssql-dep.template.yml: -------------------------------------------------------------------------------- 1 | # This template adds the 'tiny_tds' gem for import scripts depending on it 2 | 3 | params: 4 | home: /var/www/discourse 5 | 6 | hooks: 7 | after_web_config: 8 | - exec: 9 | cd: /tmp 10 | cmd: 11 | - wget -O freetds.tar.gz http://www.freetds.org/files/stable/freetds-1.00.91.tar.gz 12 | - tar -xzf freetds.tar.gz 13 | - exec: 14 | cd: /tmp/freetds-* 15 | cmd: 16 | - ./configure --prefix=/usr/local --with-tdsver=7.3 17 | - make && make install 18 | 19 | after_bundle_exec: 20 | - exec: 21 | cd: $home 22 | cmd: 23 | - echo "gem 'tiny_tds'" >> Gemfile 24 | - su discourse -c 'bundle config unset deployment' 25 | - su discourse -c 'bundle install --no-deployment --path vendor/bundle --jobs 4 --without test development' 26 | -------------------------------------------------------------------------------- /image/discourse_fast_switch/Dockerfile: -------------------------------------------------------------------------------- 1 | # NAME: discourse/discourse_fast_switch 2 | # VERSION: 1.5.0 3 | 4 | # Allow to easily switch Ruby version in images that derive off this 5 | FROM discourse/base:2.0.20180608 6 | 7 | #LABEL maintainer="Sam Saffron \"https://twitter.com/samsaffron\"" 8 | 9 | RUN apt -y install ruby bison autoconf &&\ 10 | cd /src && git clone https://github.com/sstephenson/ruby-build.git &&\ 11 | /src/ruby-build/install.sh &&\ 12 | sudo ruby-build 2.4.4 /usr/ruby_24 &&\ 13 | cp -R /usr/ruby_24/bin/* /usr/local/bin/ &&\ 14 | cp -R /usr/ruby_24/lib/* /usr/local/lib/ &&\ 15 | cp -R /usr/ruby_24/share/* /usr/local/share/ &&\ 16 | cp -R /usr/ruby_24/include/* /usr/local/include/ &&\ 17 | apt -y remove ruby 18 | 19 | RUN cd / && ruby-build 2.5.1 /usr/ruby_25 20 | 21 | ADD create_switch.rb /src/create_switch.rb 22 | 23 | RUN ruby /src/create_switch.rb 24 | -------------------------------------------------------------------------------- /templates/cloudflare.template.yml: -------------------------------------------------------------------------------- 1 | run: 2 | - file: 3 | path: /tmp/add-cloudflare-ips 4 | chmod: +x 5 | contents: | 6 | #!/bin/bash -e 7 | # Download list of CloudFlare ips 8 | wget https://www.cloudflare.com/ips-v4/ -O - > /tmp/cloudflare-ips 9 | wget https://www.cloudflare.com/ips-v6/ -O - >> /tmp/cloudflare-ips 10 | # Make into nginx commands and escape for inclusion into sed append command 11 | CONTENTS=$( 6 | #include 7 | #include 8 | #include 9 | 10 | int main( int argc, char **argv) { 11 | if (argc < 2) { 12 | fprintf(stderr, "ERROR: expecting at least 1 argument!\n"); 13 | return -1; 14 | } 15 | prctl(PR_SET_THP_DISABLE, 1, 0, 0, 0); 16 | 17 | char* newargv[argc]; 18 | int i; 19 | 20 | newargv[argc-1] = NULL; 21 | for (i=1; i 0) { 34 | fprintf(stderr, "ERROR: %i errno while attempting to run file\n", errno); 35 | return -1; 36 | } 37 | 38 | return 0; 39 | } 40 | -------------------------------------------------------------------------------- /image/discourse_bench/Dockerfile: -------------------------------------------------------------------------------- 1 | # NAME: discourse/discourse_bench 2 | # VERSION: 1.4.0 3 | FROM discourse/discourse_test:1.4.0 4 | ENV RAILS_ENV profile 5 | 6 | #LABEL maintainer="Gerhard Schlager " 7 | 8 | # Install ApacheBench 9 | RUN apt-get install -y apache2-utils libsqlite3-dev 10 | 11 | # configure Git to suppress warnings 12 | RUN git config --global user.email "you@example.com" &&\ 13 | git config --global user.name "Your Name" 14 | 15 | RUN git pull &&\ 16 | sudo -u discourse bundle install --standalone 17 | 18 | RUN gem install facter &&\ 19 | gem install mailcatcher 20 | 21 | # reconfigure PostgreSQL template databases to use UTF-8 22 | # otherwise rake db:create fails 23 | RUN pg_dropcluster --stop 9.5 main &&\ 24 | pg_createcluster --start -e UTF-8 9.5 main 25 | 26 | # create role "discourse" 27 | RUN /etc/init.d/postgresql start &&\ 28 | sleep 5 &&\ 29 | sudo -u postgres createuser --superuser discourse 30 | 31 | ADD run_bench.sh /tmp/run_bench.sh 32 | RUN chmod +x /tmp/run_bench.sh 33 | 34 | ENTRYPOINT ["/tmp/run_bench.sh"] 35 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2018 Civilized Discourse Construction Kit, Inc. 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /samples/data.yml: -------------------------------------------------------------------------------- 1 | # A container for all things Data, be sure to set a secret password for 2 | # discourse account, SOME_SECRET is just an example 3 | # 4 | 5 | templates: 6 | - "templates/postgres.template.yml" 7 | - "templates/redis.template.yml" 8 | 9 | # any extra arguments for Docker? 10 | # docker_args: 11 | 12 | params: 13 | db_default_text_search_config: "pg_catalog.english" 14 | 15 | ## Set db_shared_buffers to a max of 25% of the total memory. 16 | ## will be set automatically by bootstrap based on detected RAM, or you can override 17 | #db_shared_buffers: "256MB" 18 | 19 | ## can improve sorting performance, but adds memory usage per-connection 20 | #db_work_mem: "40MB" 21 | 22 | env: 23 | # ensure locale exists in container, you may need to install it 24 | LANG: en_US.UTF-8 25 | 26 | volumes: 27 | - volume: 28 | host: /var/discourse/shared/data 29 | guest: /shared 30 | - volume: 31 | host: /var/discourse/shared/data/log/var-log 32 | guest: /var/log 33 | 34 | # TODO: SOME_SECRET to a password for the discourse user 35 | hooks: 36 | after_postgres: 37 | - exec: 38 | stdin: | 39 | alter user discourse with password 'SOME_SECRET'; 40 | cmd: su - postgres -c 'psql discourse' 41 | 42 | raise_on_fail: false 43 | -------------------------------------------------------------------------------- /templates/import/mbox.template.yml: -------------------------------------------------------------------------------- 1 | # This template configures the container for importing emails and mbox files. 2 | 3 | params: 4 | home: /var/www/discourse 5 | 6 | hooks: 7 | after_web_config: 8 | - file: 9 | path: /usr/local/bin/import_mbox.sh 10 | chmod: "+x" 11 | contents: | 12 | #!/bin/bash 13 | set -e 14 | 15 | chown discourse /shared/import/settings.yml 16 | chown discourse -R /shared/import/data 17 | 18 | cd $home 19 | echo "The mbox import is starting..." 20 | echo 21 | su discourse -c 'bundle exec ruby script/import_scripts/mbox.rb /shared/import/settings.yml' 22 | 23 | - exec: 24 | cd: $home 25 | cmd: 26 | - mkdir -p /shared/import/data 27 | - chown discourse -R /shared/import 28 | - cp -n script/import_scripts/mbox/settings.yml /shared/import/settings.yml 29 | 30 | after_bundle_exec: 31 | - exec: 32 | cd: $home 33 | cmd: 34 | - apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y libsqlite3-dev 35 | - echo "gem 'sqlite3'" >> Gemfile 36 | - su discourse -c 'bundle config unset deployment' 37 | - su discourse -c 'bundle install --no-deployment --path vendor/bundle --jobs 4 --without test development' 38 | -------------------------------------------------------------------------------- /scripts/mk_swapfile: -------------------------------------------------------------------------------- 1 | #!/bin/bash -e 2 | # This script adds a 1GB swapfile to the system 3 | 4 | function do_err() { 5 | code=$? 6 | echo "Command failed with code $code: $BASH_COMMAND" 7 | exit $code 8 | 9 | } 10 | trap do_err ERR 11 | 12 | 13 | function set_swappiness() { 14 | if ! grep -q '^vm.swappiness' /etc/sysctl.conf; then 15 | echo -n 'Setting ' 16 | sysctl -w vm.swappiness=10 17 | echo vm.swappiness = 10 >> /etc/sysctl.conf 18 | fi 19 | } 20 | 21 | function get_new_swapfile() { 22 | for i in `seq 0 99`; do 23 | if [ ! -e /swapfile.$i ]; then 24 | echo /swapfile.$i 25 | return 26 | fi 27 | done 28 | # Seriously? 100 swapfiles already exist? 29 | echo "too many swapfiles" 30 | exit 1 31 | } 32 | 33 | [ `id -u` -eq 0 ] || { echo "You must be root to run this script"; exit 1; } 34 | 35 | # how big? default 1GB 36 | declare -i num_gb 37 | num_gb="${1-1}" 38 | [ $num_gb -lt 1 ] && { echo "Please specify an integer >= 1"; exit 1; } 39 | echo "Creating a ${num_gb}GB swapfile..." 40 | 41 | set_swappiness 42 | 43 | SWAPFILE=$(get_new_swapfile) 44 | 45 | umask 077 46 | dd if=/dev/zero of=$SWAPFILE bs=1k count=$(($num_gb * 1024 * 1024)) conv=excl 47 | mkswap $SWAPFILE 48 | swapon $SWAPFILE 49 | echo "$SWAPFILE swap swap auto 0 0" >> /etc/fstab 50 | 51 | echo 1GiB swapfile successfully added 52 | -------------------------------------------------------------------------------- /templates/web.onion.template.yml: -------------------------------------------------------------------------------- 1 | # Adds another server on port 80 for hidden service hosting 2 | 3 | run: 4 | - exec: 5 | cmd: 6 | # Check DISCOURSE_ONION variable has been configured 7 | - if [ -z "$DISCOURSE_ONION" ]; then echo "DISCOURSE_ONION ENV variable is required and has not been set."; exit 1; fi 8 | 9 | - exec: 10 | cmd: 11 | # Copy default nginx file 12 | - "cp $home/config/nginx.sample.conf /etc/nginx/conf.d/onion.conf" 13 | 14 | # Remove duplicate entries that would crash the server 15 | - replace: 16 | filename: "/etc/nginx/conf.d/onion.conf" 17 | from: /upstream[^\}]+\}/m 18 | to: "" 19 | 20 | - replace: 21 | filename: "/etc/nginx/conf.d/onion.conf" 22 | from: /map[^\}]+\}/m 23 | to: "" 24 | 25 | - replace: 26 | filename: "/etc/nginx/conf.d/onion.conf" 27 | from: /types[^\}]+\}/m 28 | to: "" 29 | 30 | - replace: 31 | filename: "/etc/nginx/conf.d/onion.conf" 32 | from: /proxy_cache_path.*$/ 33 | to: "" 34 | 35 | - replace: 36 | filename: "/etc/nginx/conf.d/onion.conf" 37 | from: /log_format.*$/ 38 | to: "" 39 | 40 | - replace: 41 | filename: "/etc/nginx/conf.d/onion.conf" 42 | from: /server_name.+$/ 43 | to: server_name $$ENV_DISCOURSE_ONION; 44 | 45 | # Apply the same replacements done on web.template.yml to the nginx file 46 | - replace: 47 | filename: "/etc/nginx/conf.d/onion.conf" 48 | from: /client_max_body_size.+$/ 49 | to: client_max_body_size $upload_size ; 50 | -------------------------------------------------------------------------------- /image/auto_build.rb: -------------------------------------------------------------------------------- 1 | # simple build file to be used locally by Sam 2 | # 3 | require 'pty' 4 | require 'optparse' 5 | 6 | images = { 7 | base: { name: 'base', tag: "discourse/base:build", squash: true }, 8 | discourse_test_build: { name: 'discourse_test', tag: "discourse/discourse_test:build", squash: false}, 9 | discourse_test_public: { name: 'discourse_test', tag: "discourse/discourse_test:release", squash: true, extra_args: ' --build-arg tag=release '}, 10 | discourse_dev: { name: 'discourse_dev', tag: "discourse/discourse_dev:build", squash: false }, 11 | } 12 | 13 | def run(command) 14 | lines = [] 15 | PTY.spawn(command) do |stdin, stdout, pid| 16 | begin 17 | stdin.each do |line| 18 | lines << line 19 | puts line 20 | end 21 | rescue Errno::EIO 22 | # we are done 23 | end 24 | end 25 | 26 | lines 27 | end 28 | 29 | def build(image) 30 | lines = run("cd #{image[:name]} && docker build . --no-cache --tag #{image[:tag]} #{image[:squash] ? '--squash' : ''} #{image[:extra_args] ? image[:extra_args] : ''}") 31 | raise "Error building the image for #{image[:name]}: #{lines[-1]}" if lines[-1] =~ /successfully built/ 32 | end 33 | 34 | def dev_deps() 35 | run("sed -e 's/\(db_name: discourse\)/\1_development/' ../templates/postgres.template.yml > discourse_dev/postgres.template.yml") 36 | run("cp ../templates/redis.template.yml discourse_dev/redis.template.yml") 37 | end 38 | 39 | image = ARGV[0].intern 40 | raise 'Image not found' unless images.include?(image) 41 | 42 | puts "Building #{images[image]}" 43 | dev_deps() if image == :discourse_dev 44 | 45 | build(images[image]) 46 | -------------------------------------------------------------------------------- /image/discourse_test/Dockerfile: -------------------------------------------------------------------------------- 1 | # NAME: discourse/discourse_test 2 | # VERSION: release 3 | ARG tag=build 4 | FROM discourse/base:$tag 5 | ENV RAILS_ENV test 6 | 7 | MAINTAINER Sam Saffron "https://twitter.com/samsaffron" 8 | 9 | # configure Git to suppress warnings 10 | RUN sudo -E -u discourse -H git config --global user.email "you@example.com" &&\ 11 | sudo -E -u discourse -H git config --global user.name "Your Name" 12 | 13 | RUN gem update bundler --force &&\ 14 | cd /var/www/discourse &&\ 15 | chown -R discourse . &&\ 16 | rm -fr .bundle &&\ 17 | sudo -u discourse git pull &&\ 18 | sudo -u discourse bundle install --standalone --jobs=4 19 | 20 | RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - &&\ 21 | echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list &&\ 22 | curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - &&\ 23 | echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list &&\ 24 | apt update &&\ 25 | apt install -y libgconf-2-4 google-chrome-stable yarn nodejs &&\ 26 | npm install -g eslint babel-eslint &&\ 27 | cd /var/www/discourse && sudo -E -u discourse -H yarn install 28 | 29 | RUN cd /var/www/discourse && sudo -E -u discourse -H bundle exec rake plugin:install_all_official &&\ 30 | sudo -E -u discourse -H bundle exec rake plugin:install_all_gems &&\ 31 | chown -R discourse /var/run/postgresql 32 | 33 | WORKDIR /var/www/discourse 34 | ENV LANG en_US.UTF-8 35 | ENTRYPOINT sudo -E -u discourse -H ruby script/docker_test.rb 36 | -------------------------------------------------------------------------------- /templates/web.ssl.template.yml: -------------------------------------------------------------------------------- 1 | run: 2 | - exec: 3 | cmd: 4 | - "mkdir -p /shared/ssl/" 5 | - replace: 6 | filename: "/etc/nginx/conf.d/discourse.conf" 7 | from: /server.+{/ 8 | to: | 9 | server { 10 | listen 80; 11 | return 301 https://$$ENV_DISCOURSE_HOSTNAME$request_uri; 12 | } 13 | server { 14 | - replace: 15 | hook: ssl 16 | filename: "/etc/nginx/conf.d/discourse.conf" 17 | from: /listen 80;\s+gzip on;/m 18 | to: | 19 | listen 443 ssl http2; 20 | ssl_protocols TLSv1.2 TLSv1.3; 21 | ssl_ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384; 22 | ssl_prefer_server_ciphers off; 23 | 24 | ssl_certificate /shared/ssl/ssl.crt; 25 | ssl_certificate_key /shared/ssl/ssl.key; 26 | 27 | ssl_session_tickets off; 28 | ssl_session_timeout 1d; 29 | ssl_session_cache shared:SSL:1m; 30 | 31 | gzip on; 32 | 33 | add_header Strict-Transport-Security 'max-age=31536000'; # remember the certificate for a year and automatically connect to HTTPS for this domain 34 | 35 | if ($http_host != $$ENV_DISCOURSE_HOSTNAME) { 36 | rewrite (.*) https://$$ENV_DISCOURSE_HOSTNAME$1 permanent; 37 | } 38 | - replace: 39 | filename: "/etc/nginx/conf.d/discourse.conf" 40 | from: "location @discourse {" 41 | to: | 42 | location @discourse { 43 | add_header Strict-Transport-Security 'max-age=31536000'; # remember the certificate for a year and automatically connect to HTTPS for this domain 44 | -------------------------------------------------------------------------------- /image/discourse_dev/Dockerfile: -------------------------------------------------------------------------------- 1 | # NAME: discourse/discourse_dev 2 | # VERSION: release 3 | FROM discourse/base:release 4 | ENV RAILS_ENV development 5 | 6 | #LABEL maintainer="Sam Saffron \"https://twitter.com/samsaffron\"" 7 | 8 | # Install for mailcatcher gem 9 | RUN apt update && apt install -y libsqlite3-dev \ 10 | && gem install mailcatcher && rm -rf /var/lib/apt/lists/* 11 | 12 | # Remove the code added on base image 13 | RUN rm -rf /var/www/* 14 | 15 | # Give discourse user no-passwd sudo permissions (for bundle install) 16 | ADD sudoers.discourse /etc/sudoers.d/discourse 17 | 18 | # get redis going 19 | ADD redis.template.yml /pups/redis.yml 20 | RUN /pups/bin/pups /pups/redis.yml 21 | 22 | RUN locale-gen en_US.UTF-8 23 | ENV LANG en_US.UTF-8 24 | ENV LANGUAGE en_US:en 25 | ENV LC_ALL en_US.UTF-8 26 | 27 | # get postgres going 28 | ADD postgres.template.yml /pups/postgres.yml 29 | RUN LANG=en_US.UTF-8 /pups/bin/pups /pups/postgres.yml 30 | 31 | # add dev databases 32 | ADD postgres_dev.template.yml /pups/postgres_dev.yml 33 | RUN /pups/bin/pups /pups/postgres_dev.yml 34 | 35 | # move default postgres_data out of the way 36 | RUN mv /shared/postgres_data /shared/postgres_data_orig 37 | 38 | # re-instantiate data on boot if needed (this will allow it to persist across 39 | # invocations when used with a mounted volume) 40 | ADD ensure-database /etc/runit/1.d/ensure-database 41 | 42 | RUN wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - &&\ 43 | echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list &&\ 44 | curl -sS https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add - &&\ 45 | echo "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list &&\ 46 | apt update &&\ 47 | apt install -y google-chrome-stable yarn nodejs &&\ 48 | npm install -g eslint babel-eslint 49 | -------------------------------------------------------------------------------- /samples/mail-receiver.yml: -------------------------------------------------------------------------------- 1 | ## this is the incoming mail receiver container template 2 | ## 3 | ## After making changes to this file, you MUST rebuild 4 | ## /var/discourse/launcher rebuild mail-receiver 5 | ## 6 | ## BE *VERY* CAREFUL WHEN EDITING! 7 | ## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT! 8 | ## visit http://www.yamllint.com/ to validate this file as needed 9 | 10 | base_image: discourse/mail-receiver:release 11 | update_pups: false 12 | 13 | expose: 14 | - "25:25" # SMTP 15 | 16 | env: 17 | LANG: en_US.UTF-8 18 | 19 | ## Where e-mail to your forum should be sent. In general, it's perfectly fine 20 | ## to use the same domain as the forum itself here. 21 | MAIL_DOMAIN: discourse.example.com 22 | # uncomment these (and the volume below!) to support TLS 23 | # POSTCONF_smtpd_tls_key_file: /letsencrypt/discourse.example.com/prop.ltcmp.net.key 24 | # POSTCONF_smtpd_tls_cert_file: /letsencrypt/discourse.example.com/fullchain.cer 25 | # POSTCONF_smtpd_tls_security_level: may 26 | 27 | 28 | ## The URL of the mail processing endpoint of your Discourse forum. 29 | ## This is simply your forum's base URL, with `/admin/email/handle_mail` 30 | ## appended. Be careful if you're running a subfolder setup -- in that case, 31 | ## the URL needs to have the subfolder included! 32 | DISCOURSE_MAIL_ENDPOINT: 'http://discourse.example.com/admin/email/handle_mail' 33 | 34 | ## The master API key of your Discourse forum. You can get this from 35 | ## the "API" tab of your admin panel. 36 | DISCOURSE_API_KEY: abcdefghijklmnop 37 | 38 | ## The username to use for processing incoming e-mail. Unless you have 39 | ## renamed the `system` user, you should leave this as-is. 40 | DISCOURSE_API_USERNAME: system 41 | 42 | volumes: 43 | - volume: 44 | host: /var/discourse/shared/mail-receiver/postfix-spool 45 | guest: /var/spool/postfix 46 | # uncomment to support TLS 47 | # - volume: 48 | # host: /var/discourse/shared/standalone/letsencrypt 49 | # guest: /letsencrypt 50 | 51 | 52 | -------------------------------------------------------------------------------- /templates/redis.template.yml: -------------------------------------------------------------------------------- 1 | run: 2 | - file: 3 | path: /etc/service/redis/run 4 | chmod: "+x" 5 | contents: | 6 | #!/bin/sh 7 | exec 2>&1 8 | exec thpoff chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf 9 | - file: 10 | path: /etc/service/redis/log/run 11 | chmod: "+x" 12 | contents: | 13 | #!/bin/sh 14 | mkdir -p /var/log/redis 15 | exec svlogd /var/log/redis 16 | - file: 17 | path: /etc/runit/3.d/10-redis 18 | chmod: "+x" 19 | contents: | 20 | #!/bin/bash 21 | sv stop redis 22 | 23 | - replace: 24 | filename: "/etc/redis/redis.conf" 25 | from: "daemonize yes" 26 | to: "" 27 | - replace: 28 | filename: "/etc/redis/redis.conf" 29 | from: /^pidfile.*$/ 30 | to: "" 31 | 32 | - exec: 33 | cmd: 34 | - install -d -m 0755 -o redis -g redis /shared/redis_data 35 | 36 | - replace: 37 | filename: "/etc/redis/redis.conf" 38 | from: /^logfile.*$/ 39 | to: "logfile \"\"" 40 | 41 | - replace: 42 | filename: "/etc/redis/redis.conf" 43 | from: /^bind .*$/ 44 | to: "" 45 | 46 | - replace: 47 | filename: "/etc/redis/redis.conf" 48 | from: /^dir .*$/ 49 | to: "dir /shared/redis_data" 50 | 51 | - replace: 52 | filename: "/etc/redis/redis.conf" 53 | from: /^protected-mode yes/ 54 | to: "protected-mode no" 55 | 56 | - exec: 57 | cmd: echo redis installed 58 | hook: redis 59 | - exec: cat /etc/redis/redis.conf | grep logfile 60 | 61 | - exec: 62 | background: true 63 | cmd: exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf 64 | 65 | - exec: sleep 10 66 | 67 | # we can not migrate without redis, launch it if needed 68 | hooks: 69 | before_code: 70 | - exec: 71 | background: true 72 | cmd: exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf 73 | after_code: 74 | - replace: 75 | filename: /etc/service/unicorn/run 76 | from: "# redis" 77 | to: sv start redis || exit 1 78 | -------------------------------------------------------------------------------- /image/README.md: -------------------------------------------------------------------------------- 1 | # Docker images 2 | 3 | ## Building new images 4 | 5 | To build a new image, just run `ruby auto_build.rb image-name`. The build process will build a local image with a predefined tag. 6 | 7 | Images and tag names are defined [here](https://github.com/discourse/discourse_docker/blob/master/image/auto_build.rb#L6-L11). 8 | 9 | > **A note about --squash**: By default we squash the images we serve on Docker Hub. You will need to [enable experimental features](https://github.com/docker/docker-ce/blob/master/components/cli/experimental/README.md) on your Docker daemon for that. 10 | 11 | 12 | ## More about the images 13 | 14 | See both `auto_build.rb` and the respective `Dockerfile`s for details on _how_ all of this happens. 15 | 16 | 17 | ### base ([discourse/base](https://hub.docker.com/r/discourse/base/)) 18 | 19 | All of the dependencies for running Discourse. This includes runit, postgres, nginx, ruby, imagemagick, etc. It also includes the creation of the "discourse" user and `/var/www` directory. 20 | 21 | 22 | ### discourse_dev ([discourse/discourse_dev](https://hub.docker.com/r/discourse/discourse_dev/)) 23 | 24 | Adds redis and postgres just like the "standalone" template for Discourse in order to have an all-in-one container for development. Note that you are expected to mount your local discourse source directory to `/src`. See [the README in GitHub's discourse/bin/docker](https://github.com/discourse/discourse/tree/master/bin/docker/) for utilities that help with this. 25 | 26 | Note that the discourse user is granted "sudo" permission without asking for a password in the discourse_dev image. This is to facilitate the command-line Docker tools in discourse proper that run commands as the discourse user. 27 | 28 | 29 | ### discourse_test ([discourse/discourse_test](https://hub.docker.com/r/discourse/discourse_test/)) 30 | 31 | Builds on the discourse image and adds testing tools and a default testing entrypoint. 32 | 33 | 34 | ### discourse_bench ([discourse/discourse_bench](https://hub.docker.com/r/discourse/discourse_bench/)) 35 | 36 | Builds on the discourse_test image and adds benchmark testing. 37 | 38 | 39 | ### discourse_fast_switch ([discourse/discourse_fast_switch](https://hub.docker.com/r/discourse/discourse_fast_switch/)) 40 | 41 | Builds on the discourse image and adds the ability to easily switch versions of Ruby. 42 | -------------------------------------------------------------------------------- /image/base/install-nginx: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | # version check: https://nginx.org/en/download.html 5 | VERSION=1.17.9 6 | HASH="7dd65d405c753c41b7fdab9415cfb4bdbaf093ec6d9f7432072d52cb7bcbb689" 7 | 8 | apt install -y autoconf 9 | 10 | cd /tmp 11 | git clone https://github.com/bagder/libbrotli 12 | cd libbrotli 13 | ./autogen.sh 14 | ./configure 15 | make install 16 | 17 | cd /tmp 18 | curl -O https://nginx.org/download/nginx-$VERSION.tar.gz 19 | sha256sum nginx-$VERSION.tar.gz 20 | echo "$HASH nginx-$VERSION.tar.gz" | sha256sum -c 21 | tar zxf nginx-$VERSION.tar.gz 22 | cd nginx-$VERSION 23 | 24 | # nginx-common for boilerplate files etc. 25 | apt install -y nginx-common libpcre3 libpcre3-dev zlib1g zlib1g-dev 26 | 27 | cd /tmp 28 | # this is the reason we are compiling by hand... 29 | git clone https://github.com/google/ngx_brotli.git 30 | # now ngx_brotli has brotli as a submodule 31 | cd /tmp/ngx_brotli 32 | git submodule update --init 33 | 34 | cd /tmp/nginx-$VERSION 35 | # ignoring depracations with -Wno-deprecated-declarations while we wait for this https://github.com/google/ngx_brotli/issues/39#issuecomment-254093378 36 | ./configure --with-cc-opt='-g -O2 -fPIE -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -Wno-deprecated-declarations' --with-ld-opt='-Wl,-Bsymbolic-functions -fPIE -pie -Wl,-z,relro -Wl,-z,now' --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --http-log-path=/var/log/nginx/access.log --error-log-path=/var/log/nginx/error.log --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --with-debug --with-pcre-jit --with-ipv6 --with-http_ssl_module --with-http_stub_status_module --with-http_realip_module --with-http_auth_request_module --with-http_addition_module --with-http_dav_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_v2_module --with-http_sub_module --with-stream --with-stream_ssl_module --with-mail --with-mail_ssl_module --with-threads --add-module=/tmp/ngx_brotli 37 | 38 | make install 39 | 40 | mv /usr/share/nginx/sbin/nginx /usr/sbin 41 | 42 | cd / 43 | rm -fr /tmp/nginx 44 | rm -fr /tmp/libbrotli 45 | rm -fr /tmp/ngx_brotli 46 | rm -fr /etc/nginx/modules-enabled/* 47 | -------------------------------------------------------------------------------- /image/monitor/src/monitor.rb: -------------------------------------------------------------------------------- 1 | require 'statsd-ruby' 2 | require 'docker' 3 | 4 | $statsd = Statsd.new '10.0.0.1', 8125 5 | 6 | module Docker 7 | class CloseConnectionError < StandardError; end 8 | class Container 9 | def name 10 | info["Names"].first[1..-1] 11 | end 12 | 13 | def stats 14 | path = path_for(:stats) 15 | 16 | result = nil 17 | 18 | streamer = lambda do |chunk, remaining, total| 19 | result ||= chunk 20 | raise CloseConnectionError if result 21 | end 22 | options = { response_block: streamer }.merge(connection.options) 23 | 24 | Excon.get(connection.url + path[1..-1], options) rescue CloseConnectionError 25 | 26 | Docker::Util.parse_json(result) 27 | end 28 | end 29 | end 30 | 31 | def median(array) 32 | sorted = array.sort 33 | len = sorted.length 34 | return ((sorted[(len - 1) / 2] + sorted[len / 2]) / 2.0).to_i 35 | end 36 | 37 | 38 | def analyze_container(container) 39 | 40 | data = container.exec(["ps", "-eo", "rss,args"])[0].join("\n").split("\n") 41 | unicorns = data.grep(/unicorn/).map(&:to_i) 42 | sidekiqs = data.grep(/sidekiq/).map(&:to_i) 43 | 44 | result = {} 45 | 46 | if unicorns.length > 0 47 | result["unicorn.max_rss"] = unicorns.max 48 | result["unicorn.median_rss"] = median(unicorns) 49 | end 50 | 51 | if sidekiqs.length > 0 52 | result["sidekiq.max_rss"] = sidekiqs.max 53 | result["sidekiq.median_rss"] = median(sidekiqs) 54 | end 55 | result["total_mem_usage"] = container.stats["memory_stats"]["usage"] 56 | 57 | @prev_stats ||= {} 58 | prev_stats = @prev_stats[container.name] 59 | @prev_stats[container.name] = stats = container.stats 60 | 61 | if prev_stats 62 | cpu_delta = stats["cpu_stats"]["system_cpu_usage"] - prev_stats["cpu_stats"]["system_cpu_usage"] 63 | app_cpu_delta = stats["cpu_stats"]["cpu_usage"]["total_usage"] - prev_stats["cpu_stats"]["cpu_usage"]["total_usage"] 64 | 65 | result["cpu_usage"] = (app_cpu_delta.to_f / cpu_delta.to_f) * stats["cpu_stats"]["cpu_usage"]["percpu_usage"].length * 100.0 66 | end 67 | 68 | result 69 | 70 | end 71 | 72 | def containers 73 | Docker::Container.all 74 | end 75 | 76 | hostname = Docker.info["Name"] 77 | 78 | STDERR.puts "#{Time.now} Starting Monitor" 79 | 80 | while true 81 | 82 | begin 83 | containers.each do |c| 84 | 85 | analyze_container(c).each do |k, v| 86 | if v && v > 0 87 | $statsd.gauge "#{hostname}.#{c.name}.#{k}", v 88 | end 89 | end 90 | end 91 | rescue => e 92 | STDERR.puts e 93 | STDERR.puts e.backtrace 94 | end 95 | 96 | sleep 60 97 | end 98 | 99 | -------------------------------------------------------------------------------- /image/discourse_dev/postgres_dev.template.yml: -------------------------------------------------------------------------------- 1 | run: 2 | 3 | - replace: 4 | filename: "/etc/postgresql/12/main/postgresql.conf" 5 | from: /#?fullpage_writes *=.*/ 6 | to: "fullpage_writes = off" 7 | 8 | - replace: 9 | filename: "/etc/postgresql/12/main/postgresql.conf" 10 | from: /#?fsync *=.*/ 11 | to: "fsync = off" 12 | 13 | - exec: 14 | background: true 15 | # use fast shutdown for pg 16 | stop_signal: INT 17 | cmd: HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/12/bin/postmaster -D /etc/postgresql/12/main 18 | 19 | - exec: 20 | background: true 21 | cmd: exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf 22 | 23 | # give db a few secs to start up 24 | - exec: "sleep 5" 25 | 26 | - exec: su postgres -c 'createdb discourse_development' || true 27 | - exec: su postgres -c 'psql discourse_development -c "grant all privileges on database discourse_development to discourse;"' || true 28 | - exec: su postgres -c 'psql discourse_development -c "alter schema public owner to discourse;"' 29 | - exec: su postgres -c 'psql discourse_development -c "create extension if not exists hstore;"' 30 | - exec: su postgres -c 'psql discourse_development -c "create extension if not exists pg_trgm;"' 31 | 32 | - exec: su postgres -c 'createdb discourse_test' || true 33 | - exec: su postgres -c 'psql discourse_test -c "grant all privileges on database discourse_test to discourse;"' || true 34 | - exec: su postgres -c 'psql discourse_test -c "alter schema public owner to discourse;"' 35 | - exec: su postgres -c 'psql discourse_test -c "create extension if not exists hstore;"' 36 | - exec: su postgres -c 'psql discourse_test -c "create extension if not exists pg_trgm;"' 37 | 38 | - exec: su postgres -c 'createdb discourse_test_multisite' || true 39 | - exec: su postgres -c 'psql discourse_test_multisite -c "grant all privileges on database discourse_test_multisite to discourse;"' || true 40 | - exec: su postgres -c 'psql discourse_test_multisite -c "alter schema public owner to discourse;"' 41 | - exec: su postgres -c 'psql discourse_test_multisite -c "create extension if not exists hstore;"' 42 | - exec: su postgres -c 'psql discourse_test_multisite -c "create extension if not exists pg_trgm;"' 43 | 44 | - exec: cd tmp && git clone https://github.com/discourse/discourse.git --depth=1 && cd /tmp/discourse && sudo -u discourse bundle install 45 | - exec: chown -R discourse /tmp/discourse 46 | - exec: cd /tmp/discourse && sudo -u discourse bundle exec rake db:migrate 47 | - exec: cd /tmp/discourse && sudo -u discourse RAILS_ENV=test bundle exec rake db:migrate 48 | - exec: rm -fr /tmp/discourse 49 | -------------------------------------------------------------------------------- /image/base/install-imagemagick: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | # version check: https://github.com/ImageMagick/ImageMagick/releases 5 | IMAGE_MAGICK_VERSION="7.0.10-6" 6 | IMAGE_MAGICK_HASH="37d36f4d736eb16e0dd43c50302e1d01d1bb1125165333df8273508a22f8a64d" 7 | 8 | # version check: https://libpng.sourceforge.io/index.html 9 | LIBPNG_VERSION="1.6.37" 10 | LIBPNG_HASH="daeb2620d829575513e35fecc83f0d3791a620b9b93d800b763542ece9390fb4" 11 | 12 | # version check: https://github.com/strukturag/libheif/releases 13 | LIBHEIF_VERSION="1.7.0" 14 | LIBHEIF_HASH="11645cf2536f779be82ba9c25854fb7211b0ac30458f4764f1f7de88763deb21" 15 | 16 | PREFIX=/usr/local 17 | WDIR=/tmp/imagemagick 18 | 19 | # Install build deps 20 | apt -y -q remove imagemagick 21 | apt -y -q install ghostscript gsfonts pkg-config autoconf libbz2-dev libjpeg-dev libtiff-dev libfreetype6-dev libde265-dev 22 | 23 | mkdir -p $WDIR 24 | cd $WDIR 25 | 26 | # Build and install libpng 27 | wget -O $WDIR/libpng.tar.gz https://prdownloads.sourceforge.net/libpng/libpng-$LIBPNG_VERSION.tar.gz?download 28 | sha256sum $WDIR/libpng.tar.gz 29 | echo "$LIBPNG_HASH $WDIR/libpng.tar.gz" | sha256sum -c 30 | 31 | tar -xzvf $WDIR/libpng.tar.gz 32 | cd $WDIR/libpng-$LIBPNG_VERSION 33 | 34 | ./configure --prefix=$PREFIX 35 | make all && make install 36 | 37 | # Build and install libheif 38 | cd $WDIR 39 | wget -O $WDIR/libheif.tar.gz "https://github.com/strukturag/libheif/archive/v$LIBHEIF_VERSION.tar.gz" 40 | sha256sum $WDIR/libheif.tar.gz 41 | echo "$LIBHEIF_HASH $WDIR/libheif.tar.gz" | sha256sum -c 42 | tar -xzvf $WDIR/libheif.tar.gz 43 | cd libheif-$LIBHEIF_VERSION 44 | ./autogen.sh 45 | ./configure 46 | make && make install 47 | 48 | # Build and install ImageMagick 49 | wget -O $WDIR/ImageMagick.tar.gz "https://github.com/ImageMagick/ImageMagick/archive/$IMAGE_MAGICK_VERSION.tar.gz" 50 | sha256sum $WDIR/ImageMagick.tar.gz 51 | echo "$IMAGE_MAGICK_HASH $WDIR/ImageMagick.tar.gz" | sha256sum -c 52 | IMDIR=$WDIR/$(tar tzf $WDIR/ImageMagick.tar.gz --wildcards "ImageMagick-*/configure" |cut -d/ -f1) 53 | tar zxf $WDIR/ImageMagick.tar.gz -C $WDIR 54 | cd $IMDIR 55 | PKG_CONF_LIBDIR=$PREFIX/lib LDFLAGS=-L$PREFIX/lib CFLAGS=-I$PREFIX/include ./configure \ 56 | --prefix=$PREFIX \ 57 | --enable-static \ 58 | --enable-bounds-checking \ 59 | --enable-hdri \ 60 | --enable-hugepages \ 61 | --with-threads \ 62 | --with-modules \ 63 | --with-quantum-depth=16 \ 64 | --without-magick-plus-plus \ 65 | --with-bzlib \ 66 | --with-zlib \ 67 | --without-autotrace \ 68 | --with-freetype \ 69 | --with-jpeg \ 70 | --without-lcms \ 71 | --with-lzma \ 72 | --with-png \ 73 | --with-tiff \ 74 | --with-heic 75 | make all && make install 76 | 77 | cd $HOME 78 | rm -rf $WDIR 79 | ldconfig /usr/local/lib 80 | -------------------------------------------------------------------------------- /samples/web_only.yml: -------------------------------------------------------------------------------- 1 | # IMPORTANT: SET A SECRET PASSWORD in Postgres for the Discourse User 2 | # TODO: change SOME_SECRET in this template 3 | 4 | templates: 5 | - "templates/web.template.yml" 6 | - "templates/web.ratelimited.template.yml" 7 | ## Uncomment these two lines if you wish to add Lets Encrypt (https) 8 | #- "templates/web.ssl.template.yml" 9 | #- "templates/web.letsencrypt.ssl.template.yml" 10 | 11 | ## which TCP/IP ports should this container expose? 12 | ## If you want Discourse to share a port with another webserver like Apache or nginx, 13 | ## see https://meta.discourse.org/t/17247 for details 14 | expose: 15 | - "80:80" # http 16 | - "443:443" # https 17 | 18 | # Use 'links' key to link containers together, aka use Docker --link flag. 19 | links: 20 | - link: 21 | name: data 22 | alias: data 23 | 24 | # any extra arguments for Docker? 25 | # docker_args: 26 | 27 | params: 28 | ## Which Git revision should this container use? (default: tests-passed) 29 | #version: tests-passed 30 | 31 | env: 32 | LANG: en_US.UTF-8 33 | # DISCOURSE_DEFAULT_LOCALE: en 34 | 35 | ## How many concurrent web requests are supported? Depends on memory and CPU cores. 36 | ## will be set automatically by bootstrap based on detected CPUs, or you can override 37 | #UNICORN_WORKERS: 3 38 | 39 | ## TODO: The domain name this Discourse instance will respond to 40 | DISCOURSE_HOSTNAME: 'discourse.example.com' 41 | 42 | ## Uncomment if you want the container to be started with the same 43 | ## hostname (-h option) as specified above (default "$hostname-$config") 44 | #DOCKER_USE_HOSTNAME: true 45 | 46 | ## TODO: List of comma delimited emails that will be made admin and developer 47 | ## on initial signup example 'user1@example.com,user2@example.com' 48 | DISCOURSE_DEVELOPER_EMAILS: 'me@example.com,you@example.com' 49 | 50 | ## TODO: The SMTP mail server used to validate new accounts and send notifications 51 | # SMTP ADDRESS, username, and password are required 52 | # WARNING the char '#' in SMTP password can cause problems! 53 | DISCOURSE_SMTP_ADDRESS: smtp.example.com 54 | #DISCOURSE_SMTP_PORT: 587 55 | DISCOURSE_SMTP_USER_NAME: user@example.com 56 | DISCOURSE_SMTP_PASSWORD: pa$$word 57 | #DISCOURSE_SMTP_ENABLE_START_TLS: true # (optional, default true) 58 | 59 | ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate 60 | #LETSENCRYPT_ACCOUNT_EMAIL: me@example.com 61 | 62 | ## TODO: configure connectivity to the databases 63 | DISCOURSE_DB_SOCKET: '' 64 | #DISCOURSE_DB_USERNAME: discourse 65 | DISCOURSE_DB_PASSWORD: SOME_SECRET 66 | DISCOURSE_DB_HOST: data 67 | DISCOURSE_REDIS_HOST: data 68 | 69 | ## The http or https CDN address for this Discourse instance (configured to pull) 70 | ## see https://meta.discourse.org/t/14857 for details 71 | #DISCOURSE_CDN_URL: https://discourse-cdn.example.com 72 | 73 | volumes: 74 | - volume: 75 | host: /var/discourse/shared/web-only 76 | guest: /shared 77 | - volume: 78 | host: /var/discourse/shared/web-only/log/var-log 79 | guest: /var/log 80 | 81 | ## Plugins go here 82 | ## see https://meta.discourse.org/t/19157 for details 83 | hooks: 84 | after_code: 85 | - exec: 86 | cd: $home/plugins 87 | cmd: 88 | - git clone https://github.com/discourse/docker_manager.git 89 | 90 | ## Remember, this is YAML syntax - you can only have one block with a name 91 | run: 92 | - exec: echo "Beginning of custom commands" 93 | 94 | ## If you want to configure password login for root, uncomment and change: 95 | ## Use only one of the following lines: 96 | #- exec: /usr/sbin/usermod -p 'PASSWORD_HASH' root 97 | #- exec: /usr/sbin/usermod -p "$(mkpasswd -m sha-256 'RAW_PASSWORD')" root 98 | 99 | ## If you want to authorized additional users, uncomment and change: 100 | #- exec: ssh-import-id username 101 | #- exec: ssh-import-id anotherusername 102 | 103 | - exec: echo "End of custom commands" 104 | - exec: awk -F\# '{print $1;}' ~/.ssh/authorized_keys | awk 'BEGIN { print "Authorized SSH keys for this container:"; } NF>=2 {print $NF;}' 105 | -------------------------------------------------------------------------------- /samples/standalone.yml: -------------------------------------------------------------------------------- 1 | ## this is the all-in-one, standalone Discourse Docker container template 2 | ## 3 | ## After making changes to this file, you MUST rebuild 4 | ## /var/discourse/launcher rebuild app 5 | ## 6 | ## BE *VERY* CAREFUL WHEN EDITING! 7 | ## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT! 8 | ## visit http://www.yamllint.com/ to validate this file as needed 9 | 10 | templates: 11 | - "templates/postgres.template.yml" 12 | - "templates/redis.template.yml" 13 | - "templates/web.template.yml" 14 | - "templates/web.ratelimited.template.yml" 15 | ## Uncomment these two lines if you wish to add Lets Encrypt (https) 16 | #- "templates/web.ssl.template.yml" 17 | #- "templates/web.letsencrypt.ssl.template.yml" 18 | 19 | ## which TCP/IP ports should this container expose? 20 | ## If you want Discourse to share a port with another webserver like Apache or nginx, 21 | ## see https://meta.discourse.org/t/17247 for details 22 | expose: 23 | - "80:80" # http 24 | - "443:443" # https 25 | 26 | params: 27 | db_default_text_search_config: "pg_catalog.english" 28 | 29 | ## Set db_shared_buffers to a max of 25% of the total memory. 30 | ## will be set automatically by bootstrap based on detected RAM, or you can override 31 | #db_shared_buffers: "256MB" 32 | 33 | ## can improve sorting performance, but adds memory usage per-connection 34 | #db_work_mem: "40MB" 35 | 36 | ## Which Git revision should this container use? (default: tests-passed) 37 | #version: tests-passed 38 | 39 | env: 40 | LANG: en_US.UTF-8 41 | # DISCOURSE_DEFAULT_LOCALE: en 42 | 43 | ## How many concurrent web requests are supported? Depends on memory and CPU cores. 44 | ## will be set automatically by bootstrap based on detected CPUs, or you can override 45 | #UNICORN_WORKERS: 3 46 | 47 | ## TODO: The domain name this Discourse instance will respond to 48 | ## Required. Discourse will not work with a bare IP number. 49 | DISCOURSE_HOSTNAME: 'discourse.example.com' 50 | 51 | ## Uncomment if you want the container to be started with the same 52 | ## hostname (-h option) as specified above (default "$hostname-$config") 53 | #DOCKER_USE_HOSTNAME: true 54 | 55 | ## TODO: List of comma delimited emails that will be made admin and developer 56 | ## on initial signup example 'user1@example.com,user2@example.com' 57 | DISCOURSE_DEVELOPER_EMAILS: 'me@example.com,you@example.com' 58 | 59 | ## TODO: The SMTP mail server used to validate new accounts and send notifications 60 | # SMTP ADDRESS, username, and password are required 61 | # WARNING the char '#' in SMTP password can cause problems! 62 | DISCOURSE_SMTP_ADDRESS: smtp.example.com 63 | #DISCOURSE_SMTP_PORT: 587 64 | DISCOURSE_SMTP_USER_NAME: user@example.com 65 | DISCOURSE_SMTP_PASSWORD: pa$$word 66 | #DISCOURSE_SMTP_ENABLE_START_TLS: true # (optional, default true) 67 | 68 | ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate 69 | #LETSENCRYPT_ACCOUNT_EMAIL: me@example.com 70 | 71 | ## The http or https CDN address for this Discourse instance (configured to pull) 72 | ## see https://meta.discourse.org/t/14857 for details 73 | #DISCOURSE_CDN_URL: https://discourse-cdn.example.com 74 | 75 | ## The Docker container is stateless; all data is stored in /shared 76 | volumes: 77 | - volume: 78 | host: /var/discourse/shared/standalone 79 | guest: /shared 80 | - volume: 81 | host: /var/discourse/shared/standalone/log/var-log 82 | guest: /var/log 83 | 84 | ## Plugins go here 85 | ## see https://meta.discourse.org/t/19157 for details 86 | hooks: 87 | after_code: 88 | - exec: 89 | cd: $home/plugins 90 | cmd: 91 | - git clone https://github.com/discourse/docker_manager.git 92 | 93 | ## Any custom commands to run after building 94 | run: 95 | - exec: echo "Beginning of custom commands" 96 | ## If you want to set the 'From' email address for your first registration, uncomment and change: 97 | ## After getting the first signup email, re-comment the line. It only needs to run once. 98 | #- exec: rails r "SiteSetting.notification_email='info@unconfigured.discourse.org'" 99 | - exec: echo "End of custom commands" 100 | -------------------------------------------------------------------------------- /templates/import/phpbb3.template.yml: -------------------------------------------------------------------------------- 1 | # This template installs MariaDB and all dependencies needed for importing from phpBB3. 2 | 3 | params: 4 | home: /var/www/discourse 5 | 6 | hooks: 7 | after_web_config: 8 | - exec: 9 | cd: /etc/service 10 | cmd: 11 | - rm -R unicorn 12 | - rm -R nginx 13 | - rm -R cron 14 | 15 | - exec: 16 | cd: /etc/runit/3.d 17 | cmd: 18 | - rm 01-nginx 19 | - rm 02-unicorn 20 | 21 | - file: 22 | path: /etc/mysql/conf.d/import.cnf 23 | contents: | 24 | [mysqld] 25 | # disable InnoDB since it is extremly slow in Docker container 26 | default-storage-engine=MyISAM 27 | default-tmp-storage-engine=MyISAM 28 | innodb=OFF 29 | sql_mode=NO_AUTO_CREATE_USER 30 | 31 | datadir=/shared/import/mysql/data 32 | 33 | skip-host-cache 34 | skip-name-resolve 35 | 36 | - exec: 37 | cmd: 38 | - mkdir -p /shared/import/mysql/data 39 | - apt-get update && DEBIAN_FRONTEND=noninteractive apt-get install -y nano libmariadbclient-dev mariadb-server 40 | - sed -Ei 's/^log/#&/' /etc/mysql/my.cnf 41 | 42 | - file: 43 | path: /etc/service/mysql/run 44 | chmod: "+x" 45 | contents: | 46 | #!/bin/bash 47 | cd / 48 | umask 077 49 | 50 | # Make sure the datadir exists, is accessible and contains all system tables 51 | mkdir -p /shared/import/mysql/data 52 | chown mysql -R /shared/import/mysql/data 53 | /usr/bin/mysql_install_db --user=mysql 54 | 55 | # Shamelessly copied from http://smarden.org/runit1/runscripts.html#mysql 56 | MYSQLADMIN='/usr/bin/mysqladmin --defaults-extra-file=/etc/mysql/debian.cnf' 57 | trap "$MYSQLADMIN shutdown" 0 58 | trap 'exit 2' 1 2 3 15 59 | /usr/bin/mysqld_safe & wait 60 | 61 | - file: 62 | path: /etc/runit/3.d/99-mysql 63 | chmod: "+x" 64 | contents: | 65 | #!/bin/bash 66 | sv stop mysql 67 | 68 | - file: 69 | path: /usr/local/bin/import_phpbb3.sh 70 | chmod: "+x" 71 | contents: | 72 | #!/bin/bash 73 | set -e 74 | 75 | chown discourse /shared/import/settings.yml 76 | chown discourse -R /shared/import/data 77 | 78 | # Allow connection as root user without password 79 | mysql -uroot -e "UPDATE mysql.user SET plugin = 'mysql_native_password' WHERE user = 'root' AND plugin = 'unix_socket'" 80 | mysql -uroot -e "FLUSH PRIVILEGES" 81 | 82 | if [ -f "/shared/import/data/phpbb_mysql.sql" ]; then 83 | if [ -f "/shared/import/mysql/imported" ] && ! sha256sum --check /shared/import/mysql/imported &>/dev/null ; then 84 | echo "Checksum of database dump changed..." 85 | rm /shared/import/mysql/imported 86 | fi 87 | 88 | if [ ! -f "/shared/import/mysql/imported" ]; then 89 | echo "Loading database dump into MySQL..." 90 | mysql -uroot -e "DROP DATABASE IF EXISTS phpbb" 91 | mysql -uroot -e "CREATE DATABASE phpbb" 92 | mysql -uroot --default-character-set=utf8 --database=phpbb < /shared/import/data/phpbb_mysql.sql 93 | sha256sum /shared/import/data/phpbb_mysql.sql > /shared/import/mysql/imported 94 | fi 95 | else 96 | sv stop mysql 97 | fi 98 | 99 | cd $home 100 | echo "The phpBB3 import is starting..." 101 | echo 102 | su discourse -c 'bundle exec ruby script/import_scripts/phpbb3.rb /shared/import/settings.yml' 103 | 104 | - exec: 105 | cd: $home 106 | cmd: 107 | - mkdir -p /shared/import/data 108 | - chown discourse -R /shared/import 109 | - cp -n script/import_scripts/phpbb3/settings.yml /shared/import/settings.yml 110 | 111 | after_bundle_exec: 112 | - exec: 113 | cd: $home 114 | cmd: 115 | - echo "gem 'mysql2'" >> Gemfile 116 | - echo "gem 'ruby-bbcode-to-md', :github => 'nlalonde/ruby-bbcode-to-md'" >> Gemfile 117 | - su discourse -c 'bundle config unset deployment' 118 | - su discourse -c 'bundle install --no-deployment --path vendor/bundle --jobs 4 --without test development' 119 | -------------------------------------------------------------------------------- /templates/web.letsencrypt.ssl.template.yml: -------------------------------------------------------------------------------- 1 | env: 2 | LETSENCRYPT_DIR: "/shared/letsencrypt" 3 | 4 | hooks: 5 | after_ssl: 6 | - exec: 7 | cmd: 8 | - if [ -z "$LETSENCRYPT_ACCOUNT_EMAIL" ]; then echo "LETSENCRYPT_ACCOUNT_EMAIL ENV variable is required and has not been set."; exit 1; fi 9 | - /bin/bash -c "if [[ ! \"$LETSENCRYPT_ACCOUNT_EMAIL\" =~ ([^@]+)@([^\.]+) ]]; then echo \"LETSENCRYPT_ACCOUNT_EMAIL is not a valid email address\"; exit 1; fi" 10 | 11 | - exec: 12 | cmd: 13 | - cd /root && git clone --branch 2.8.2 --depth 1 https://github.com/Neilpang/acme.sh.git && cd /root/acme.sh 14 | - touch /var/spool/cron/crontabs/root 15 | - install -d -m 0755 -g root -o root $LETSENCRYPT_DIR 16 | - cd /root/acme.sh && LE_WORKING_DIR="${LETSENCRYPT_DIR}" ./acme.sh --install --log "${LETSENCRYPT_DIR}/acme.sh.log" 17 | - cd /root/acme.sh && LE_WORKING_DIR="${LETSENCRYPT_DIR}" ./acme.sh --upgrade --auto-upgrade 18 | 19 | - file: 20 | path: "/etc/nginx/letsencrypt.conf" 21 | contents: | 22 | user www-data; 23 | worker_processes auto; 24 | daemon on; 25 | 26 | events { 27 | worker_connections 768; 28 | # multi_accept on; 29 | } 30 | 31 | http { 32 | sendfile on; 33 | tcp_nopush on; 34 | tcp_nodelay on; 35 | keepalive_timeout 65; 36 | types_hash_max_size 2048; 37 | 38 | access_log /var/log/nginx/access.letsencrypt.log; 39 | error_log /var/log/nginx/error.letsencrypt.log; 40 | 41 | server { 42 | listen 80; 43 | listen [::]:80; 44 | 45 | location ~ /.well-known { 46 | root /var/www/discourse/public; 47 | allow all; 48 | } 49 | } 50 | } 51 | 52 | - file: 53 | path: /etc/runit/1.d/letsencrypt 54 | chmod: "+x" 55 | contents: | 56 | #!/bin/bash 57 | /usr/sbin/nginx -c /etc/nginx/letsencrypt.conf 58 | 59 | issue_cert() { 60 | LE_WORKING_DIR="${LETSENCRYPT_DIR}" $$ENV_LETSENCRYPT_DIR/acme.sh --issue $2 -d $$ENV_DISCOURSE_HOSTNAME --keylength $1 -w /var/www/discourse/public 61 | } 62 | 63 | cert_exists() { 64 | [[ "$(cd $$ENV_LETSENCRYPT_DIR/$$ENV_DISCOURSE_HOSTNAME$1 && openssl verify -CAfile ca.cer fullchain.cer | grep "OK")" ]] 65 | } 66 | 67 | ######################################################## 68 | # RSA cert 69 | ######################################################## 70 | issue_cert "4096" 71 | 72 | if ! cert_exists ""; then 73 | # Try to issue the cert again if something goes wrong 74 | issue_cert "4096" "--force" 75 | fi 76 | 77 | LE_WORKING_DIR="${LETSENCRYPT_DIR}" $$ENV_LETSENCRYPT_DIR/acme.sh \ 78 | --installcert \ 79 | -d $$ENV_DISCOURSE_HOSTNAME \ 80 | --fullchainpath /shared/ssl/$$ENV_DISCOURSE_HOSTNAME.cer \ 81 | --keypath /shared/ssl/$$ENV_DISCOURSE_HOSTNAME.key \ 82 | --reloadcmd "sv reload nginx" 83 | 84 | ######################################################## 85 | # ECDSA cert 86 | ######################################################## 87 | issue_cert "ec-256" 88 | 89 | if ! cert_exists "_ecc"; then 90 | # Try to issue the cert again if something goes wrong 91 | issue_cert "ec-256" "--force" 92 | fi 93 | 94 | LE_WORKING_DIR="${LETSENCRYPT_DIR}" $$ENV_LETSENCRYPT_DIR/acme.sh \ 95 | --installcert --ecc \ 96 | -d $$ENV_DISCOURSE_HOSTNAME \ 97 | --fullchainpath /shared/ssl/$$ENV_DISCOURSE_HOSTNAME_ecc.cer \ 98 | --keypath /shared/ssl/$$ENV_DISCOURSE_HOSTNAME_ecc.key \ 99 | --reloadcmd "sv reload nginx" 100 | 101 | if cert_exists "" || cert_exists "_ecc"; then 102 | grep -q 'force_https' "/var/www/discourse/config/discourse.conf" || echo "force_https = 'true'" >> "/var/www/discourse/config/discourse.conf" 103 | fi 104 | 105 | /usr/sbin/nginx -c /etc/nginx/letsencrypt.conf -s stop 106 | 107 | - replace: 108 | filename: "/etc/nginx/conf.d/discourse.conf" 109 | from: /ssl_certificate.+/ 110 | to: | 111 | ssl_certificate /shared/ssl/$$ENV_DISCOURSE_HOSTNAME.cer; 112 | ssl_certificate /shared/ssl/$$ENV_DISCOURSE_HOSTNAME_ecc.cer; 113 | 114 | - replace: 115 | filename: /shared/letsencrypt/account.conf 116 | from: /#?ACCOUNT_EMAIL=.+/ 117 | to: | 118 | ACCOUNT_EMAIL=$$ENV_LETSENCRYPT_ACCOUNT_EMAIL 119 | 120 | - replace: 121 | filename: "/etc/nginx/conf.d/discourse.conf" 122 | from: /ssl_certificate_key.+/ 123 | to: | 124 | ssl_certificate_key /shared/ssl/$$ENV_DISCOURSE_HOSTNAME.key; 125 | ssl_certificate_key /shared/ssl/$$ENV_DISCOURSE_HOSTNAME_ecc.key; 126 | 127 | - replace: 128 | filename: "/etc/nginx/conf.d/discourse.conf" 129 | from: /add_header.+/ 130 | to: | 131 | add_header Strict-Transport-Security 'max-age=63072000'; 132 | -------------------------------------------------------------------------------- /image/base/Dockerfile: -------------------------------------------------------------------------------- 1 | # NAME: discourse/base 2 | # VERSION: release 3 | FROM debian:buster-slim 4 | 5 | ENV PG_MAJOR 12 6 | ENV RUBY_ALLOCATOR /usr/lib/libjemalloc.so.1 7 | ENV RAILS_ENV production 8 | 9 | #LABEL maintainer="Sam Saffron \"https://twitter.com/samsaffron\"" 10 | 11 | RUN echo 2.0.`date +%Y%m%d` > /VERSION 12 | 13 | RUN apt update && apt install -y gnupg sudo curl 14 | RUN echo "debconf debconf/frontend select Teletype" | debconf-set-selections 15 | RUN apt update && apt -y install fping 16 | RUN sh -c "fping proxy && echo 'Acquire { Retries \"0\"; HTTP { Proxy \"http://proxy:3128\";}; };' > /etc/apt/apt.conf.d/40proxy && apt update || true" 17 | RUN apt -y install software-properties-common 18 | RUN apt-mark hold initscripts 19 | RUN apt -y upgrade 20 | 21 | RUN apt install -y locales locales-all 22 | ENV LC_ALL en_US.UTF-8 23 | ENV LANG en_US.UTF-8 24 | ENV LANGUAGE en_US.UTF-8 25 | 26 | RUN curl https://apt.postgresql.org/pub/repos/apt/ACCC4CF8.asc | apt-key add - 27 | RUN echo "deb http://apt.postgresql.org/pub/repos/apt/ buster-pgdg main" | \ 28 | tee /etc/apt/sources.list.d/postgres.list 29 | RUN curl --silent --location https://deb.nodesource.com/setup_10.x | sudo bash - 30 | RUN apt -y update 31 | # install these without recommends to avoid pulling in e.g. 32 | # X11 libraries, mailutils 33 | RUN apt -y install --no-install-recommends git rsyslog logrotate cron ssh-client less 34 | RUN apt -y install build-essential rsync \ 35 | libxslt-dev libcurl4-openssl-dev \ 36 | libssl-dev libyaml-dev libtool \ 37 | libxml2-dev gawk parallel \ 38 | postgresql-${PG_MAJOR} postgresql-client-${PG_MAJOR} \ 39 | postgresql-contrib-${PG_MAJOR} libpq-dev libreadline-dev \ 40 | anacron wget \ 41 | psmisc vim whois brotli libunwind-dev \ 42 | libtcmalloc-minimal4 cmake 43 | RUN sed -i -e 's/start -q anacron/anacron -s/' /etc/cron.d/anacron 44 | RUN sed -i.bak 's/$ModLoad imklog/#$ModLoad imklog/' /etc/rsyslog.conf 45 | RUN dpkg-divert --local --rename --add /sbin/initctl 46 | RUN sh -c "test -f /sbin/initctl || ln -s /bin/true /sbin/initctl" 47 | RUN cd / &&\ 48 | apt -y install runit socat &&\ 49 | mkdir -p /etc/runit/1.d &&\ 50 | apt clean &&\ 51 | rm -f /etc/apt/apt.conf.d/40proxy &&\ 52 | locale-gen en_US &&\ 53 | apt install -y nodejs &&\ 54 | npm install -g uglify-js &&\ 55 | npm install -g svgo 56 | 57 | ADD install-nginx /tmp/install-nginx 58 | RUN /tmp/install-nginx 59 | 60 | RUN apt -y install advancecomp jhead jpegoptim libjpeg-turbo-progs optipng 61 | 62 | RUN mkdir /jemalloc-stable && cd /jemalloc-stable &&\ 63 | wget https://github.com/jemalloc/jemalloc/releases/download/3.6.0/jemalloc-3.6.0.tar.bz2 &&\ 64 | tar -xjf jemalloc-3.6.0.tar.bz2 && cd jemalloc-3.6.0 && ./configure --prefix=/usr && make && make install &&\ 65 | cd / && rm -rf /jemalloc-stable 66 | 67 | RUN mkdir /jemalloc-new && cd /jemalloc-new &&\ 68 | wget https://github.com/jemalloc/jemalloc/releases/download/5.2.1/jemalloc-5.2.1.tar.bz2 &&\ 69 | tar -xjf jemalloc-5.2.1.tar.bz2 && cd jemalloc-5.2.1 && ./configure --prefix=/usr --with-install-suffix=5.2.1 && make build_lib && make install_lib &&\ 70 | cd / && rm -rf /jemalloc-new 71 | 72 | RUN echo 'gem: --no-document' >> /usr/local/etc/gemrc &&\ 73 | mkdir /src && cd /src && git clone https://github.com/sstephenson/ruby-build.git &&\ 74 | cd /src/ruby-build && ./install.sh &&\ 75 | cd / && rm -rf /src/ruby-build && (ruby-build 2.6.6 /usr/local) 76 | 77 | RUN gem update --system 78 | 79 | RUN gem install bundler --force &&\ 80 | rm -rf /usr/local/share/ri/2.6.6/system &&\ 81 | cd / && git clone https://github.com/discourse/pups.git 82 | 83 | ADD install-redis /tmp/install-redis 84 | RUN /tmp/install-redis 85 | 86 | ADD install-imagemagick /tmp/install-imagemagick 87 | RUN /tmp/install-imagemagick 88 | 89 | # Validate install 90 | RUN ruby -Eutf-8 -e "v = \`convert -version\`; %w{png tiff jpeg freetype heic}.each { |f| unless v.include?(f); STDERR.puts('no ' + f + ' support in imagemagick'); exit(-1); end }" 91 | 92 | ADD install-pngcrush /tmp/install-pngcrush 93 | RUN /tmp/install-pngcrush 94 | 95 | ADD install-gifsicle /tmp/install-gifsicle 96 | RUN /tmp/install-gifsicle 97 | 98 | ADD install-pngquant /tmp/install-pngquant 99 | RUN /tmp/install-pngquant 100 | 101 | # This tool allows us to disable huge page support for our current process 102 | # since the flag is preserved through forks and execs it can be used on any 103 | # process 104 | ADD thpoff.c /src/thpoff.c 105 | RUN gcc -o /usr/local/sbin/thpoff /src/thpoff.c && rm /src/thpoff.c 106 | 107 | # clean up for docker squash 108 | RUN rm -fr /usr/share/man &&\ 109 | rm -fr /usr/share/doc &&\ 110 | rm -fr /usr/share/vim/vim74/tutor &&\ 111 | rm -fr /usr/share/vim/vim74/doc &&\ 112 | rm -fr /usr/share/vim/vim74/lang &&\ 113 | rm -fr /usr/local/share/doc &&\ 114 | rm -fr /usr/local/share/ruby-build &&\ 115 | rm -fr /root/.gem &&\ 116 | rm -fr /root/.npm &&\ 117 | rm -fr /tmp/* &&\ 118 | rm -fr /usr/share/vim/vim74/spell/en* 119 | 120 | 121 | # this can probably be done, but I worry that people changing PG locales will have issues 122 | # cd /usr/share/locale && rm -fr `ls -d */ | grep -v en` 123 | 124 | RUN mkdir -p /etc/runit/3.d 125 | 126 | ADD runit-1 /etc/runit/1 127 | ADD runit-1.d-cleanup-pids /etc/runit/1.d/cleanup-pids 128 | ADD runit-1.d-anacron /etc/runit/1.d/anacron 129 | ADD runit-1.d-00-fix-var-logs /etc/runit/1.d/00-fix-var-logs 130 | ADD runit-2 /etc/runit/2 131 | ADD runit-3 /etc/runit/3 132 | ADD boot /sbin/boot 133 | 134 | ADD cron /etc/service/cron/run 135 | ADD rsyslog /etc/service/rsyslog/run 136 | ADD cron.d_anacron /etc/cron.d/anacron 137 | 138 | # Discourse specific bits 139 | RUN useradd discourse -s /bin/bash -m -U &&\ 140 | mkdir -p /var/www &&\ 141 | cd /var/www &&\ 142 | git clone https://github.com/discourse/discourse.git &&\ 143 | cd discourse &&\ 144 | git remote set-branches --add origin tests-passed &&\ 145 | chown -R discourse:discourse /var/www/discourse &&\ 146 | cd /var/www/discourse &&\ 147 | sudo -u discourse bundle install --deployment --jobs 4 --without test development &&\ 148 | bundle exec rake maxminddb:get &&\ 149 | find /var/www/discourse/vendor/bundle -name tmp -type d -exec rm -rf {} + 150 | -------------------------------------------------------------------------------- /templates/postgres.12.template.yml: -------------------------------------------------------------------------------- 1 | params: 2 | db_synchronous_commit: "off" 3 | db_shared_buffers: "256MB" 4 | db_work_mem: "10MB" 5 | db_default_text_search_config: "pg_catalog.english" 6 | db_name: discourse 7 | db_user: discourse 8 | db_checkpoint_segments: 6 9 | db_logging_collector: off 10 | db_log_min_duration_statement: 100 11 | 12 | hooks: 13 | before_code: 14 | - replace: 15 | filename: /etc/service/unicorn/run 16 | from: "# postgres" 17 | to: sv start postgres || exit 1 18 | 19 | run: 20 | - exec: mkdir -p /shared/postgres_run 21 | - exec: chown postgres:postgres /shared/postgres_run 22 | - exec: chmod 775 /shared/postgres_run 23 | - exec: rm -fr /var/run/postgresql 24 | - exec: ln -s /shared/postgres_run /var/run/postgresql 25 | - exec: socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1 26 | - exec: rm -fr /shared/postgres_run/.s* 27 | - exec: rm -fr /shared/postgres_run/*.pid 28 | - exec: mkdir -p /shared/postgres_run/12-main.pg_stat_tmp 29 | - exec: chown postgres:postgres /shared/postgres_run/12-main.pg_stat_tmp 30 | - file: 31 | path: /etc/service/postgres/run 32 | chmod: "+x" 33 | contents: | 34 | #!/bin/sh 35 | exec 2>&1 36 | HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/12/bin/postmaster -D /etc/postgresql/12/main 37 | 38 | - file: 39 | path: /etc/runit/3.d/99-postgres 40 | chmod: "+x" 41 | contents: | 42 | #!/bin/bash 43 | sv stop postgres 44 | 45 | - exec: 46 | cmd: 47 | - chown -R root /var/lib/postgresql/12/main 48 | - "[ ! -e /shared/postgres_data ] && install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u postgres /usr/lib/postgresql/12/bin/initdb -D /shared/postgres_data || exit 0" 49 | - chown -R postgres:postgres /shared/postgres_data 50 | - chown -R postgres:postgres /var/run/postgresql 51 | 52 | - replace: 53 | filename: "/etc/postgresql/12/main/postgresql.conf" 54 | from: "data_directory = '/var/lib/postgresql/12/main'" 55 | to: "data_directory = '/shared/postgres_data'" 56 | 57 | # listen on all interfaces 58 | - replace: 59 | filename: "/etc/postgresql/12/main/postgresql.conf" 60 | from: /#?listen_addresses *=.*/ 61 | to: "listen_addresses = '*'" 62 | 63 | # sync commit off is faster and less spiky, also marginally less safe 64 | - replace: 65 | filename: "/etc/postgresql/12/main/postgresql.conf" 66 | from: /#?synchronous_commit *=.*/ 67 | to: "synchronous_commit = $db_synchronous_commit" 68 | 69 | # default is 128MB which is way too small 70 | - replace: 71 | filename: "/etc/postgresql/12/main/postgresql.conf" 72 | from: /#?shared_buffers *=.*/ 73 | to: "shared_buffers = $db_shared_buffers" 74 | 75 | # default is 1MB which is too small 76 | - replace: 77 | filename: "/etc/postgresql/12/main/postgresql.conf" 78 | from: /#?work_mem *=.*/ 79 | to: "work_mem = $db_work_mem" 80 | 81 | # allow for other 82 | - replace: 83 | filename: "/etc/postgresql/12/main/postgresql.conf" 84 | from: /#?default_text_search_config *=.*/ 85 | to: "default_text_search_config = '$db_default_text_search_config'" 86 | 87 | # Necessary to enable backups 88 | - exec: 89 | cmd: 90 | - install -d -m 0755 -o postgres -g postgres /shared/postgres_backup 91 | 92 | - replace: 93 | filename: "/etc/postgresql/12/main/postgresql.conf" 94 | from: /#?checkpoint_segments *=.*/ 95 | to: "checkpoint_segments = $db_checkpoint_segments" 96 | 97 | - replace: 98 | filename: "/etc/postgresql/12/main/postgresql.conf" 99 | from: /#?logging_collector *=.*/ 100 | to: "logging_collector = $db_logging_collector" 101 | 102 | - replace: 103 | filename: "/etc/postgresql/12/main/postgresql.conf" 104 | from: /#?log_min_duration_statement *=.*/ 105 | to: "log_min_duration_statement = $db_log_min_duration_statement" 106 | 107 | - replace: 108 | filename: "/etc/postgresql/12/main/pg_hba.conf" 109 | from: /^#local +replication +postgres +peer$/ 110 | to: "local replication postgres peer" 111 | 112 | # allow all to connect in with md5 auth 113 | - replace: 114 | filename: "/etc/postgresql/12/main/pg_hba.conf" 115 | from: /^host.*all.*all.*127.*$/ 116 | to: "host all all 0.0.0.0/0 md5" 117 | 118 | - exec: 119 | background: true 120 | # use fast shutdown for pg 121 | stop_signal: INT 122 | cmd: HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/12/bin/postmaster -D /etc/postgresql/12/main 123 | 124 | # give db a few secs to start up 125 | - exec: "sleep 5" 126 | 127 | - exec: su postgres -c 'createdb $db_name' || true 128 | - exec: su postgres -c 'psql $db_name -c "create user $db_user;"' || true 129 | - exec: su postgres -c 'psql $db_name -c "grant all privileges on database $db_name to $db_user;"' || true 130 | - exec: su postgres -c 'psql $db_name -c "alter schema public owner to $db_user;"' 131 | - exec: su postgres -c 'psql template1 -c "create extension if not exists hstore;"' 132 | - exec: su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"' 133 | - exec: su postgres -c 'psql $db_name -c "create extension if not exists hstore;"' 134 | - exec: su postgres -c 'psql $db_name -c "create extension if not exists pg_trgm;"' 135 | - exec: 136 | stdin: | 137 | update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = '$db_name' AND encoding = pg_char_to_encoding('SQL_ASCII'); 138 | cmd: sudo -u postgres psql $db_name 139 | raise_on_fail: false 140 | 141 | - file: 142 | path: /var/lib/postgresql/take-database-backup 143 | chown: postgres:postgres 144 | chmod: "+x" 145 | contents: | 146 | #!/bin/bash 147 | ID=db-$(date +%F_%T) 148 | FILENAME=/shared/postgres_backup/$ID.tar.gz 149 | pg_basebackup --format=tar --pgdata=- --xlog --gzip --label=$ID > $FILENAME 150 | echo $FILENAME 151 | 152 | - file: 153 | path: /var/spool/cron/crontabs/postgres 154 | contents: | 155 | # m h dom mon dow command 156 | #MAILTO=? 157 | #0 */4 * * * /var/lib/postgresql/take-database-backup 158 | 159 | - exec: 160 | hook: postgres 161 | cmd: "echo postgres installed!" 162 | -------------------------------------------------------------------------------- /templates/postgres.9.5.template.yml: -------------------------------------------------------------------------------- 1 | params: 2 | db_synchronous_commit: "off" 3 | db_shared_buffers: "256MB" 4 | db_work_mem: "10MB" 5 | db_default_text_search_config: "pg_catalog.english" 6 | db_name: discourse 7 | db_user: discourse 8 | db_checkpoint_segments: 6 9 | db_logging_collector: off 10 | db_log_min_duration_statement: 100 11 | 12 | hooks: 13 | before_code: 14 | - replace: 15 | filename: /etc/service/unicorn/run 16 | from: "# postgres" 17 | to: sv start postgres || exit 1 18 | 19 | run: 20 | - exec: apt-get remove -y postgresql-10 postgresql-client-10 postgresql-contrib-10 21 | - exec: apt-get update && apt-get install -y postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5 22 | - exec: mkdir -p /shared/postgres_run 23 | - exec: chown postgres:postgres /shared/postgres_run 24 | - exec: chmod 775 /shared/postgres_run 25 | - exec: rm -fr /var/run/postgresql 26 | - exec: ln -s /shared/postgres_run /var/run/postgresql 27 | - exec: socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1 28 | - exec: rm -fr /shared/postgres_run/.s* 29 | - exec: rm -fr /shared/postgres_run/*.pid 30 | - exec: mkdir -p /shared/postgres_run/9.5-main.pg_stat_tmp 31 | - exec: chown postgres:postgres /shared/postgres_run/9.5-main.pg_stat_tmp 32 | - file: 33 | path: /etc/service/postgres/run 34 | chmod: "+x" 35 | contents: | 36 | #!/bin/sh 37 | exec 2>&1 38 | echo -1000 >/proc/self/oom_score_adj 39 | HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/9.5/bin/postmaster -D /etc/postgresql/9.5/main 40 | 41 | - file: 42 | path: /etc/runit/3.d/99-postgres 43 | chmod: "+x" 44 | contents: | 45 | #!/bin/bash 46 | sv stop postgres 47 | 48 | - exec: 49 | cmd: 50 | - chown -R root /var/lib/postgresql/9.5/main 51 | - "[ ! -e /shared/postgres_data ] && install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u postgres /usr/lib/postgresql/9.5/bin/initdb -D /shared/postgres_data || exit 0" 52 | - chown -R postgres:postgres /shared/postgres_data 53 | - chown -R postgres:postgres /var/run/postgresql 54 | 55 | - replace: 56 | filename: "/etc/postgresql/9.5/main/postgresql.conf" 57 | from: "data_directory = '/var/lib/postgresql/9.5/main'" 58 | to: "data_directory = '/shared/postgres_data'" 59 | 60 | # listen on all interfaces 61 | - replace: 62 | filename: "/etc/postgresql/9.5/main/postgresql.conf" 63 | from: /#?listen_addresses *=.*/ 64 | to: "listen_addresses = '*'" 65 | 66 | # sync commit off is faster and less spiky, also marginally less safe 67 | - replace: 68 | filename: "/etc/postgresql/9.5/main/postgresql.conf" 69 | from: /#?synchronous_commit *=.*/ 70 | to: "synchronous_commit = $db_synchronous_commit" 71 | 72 | # default is 128MB which is way too small 73 | - replace: 74 | filename: "/etc/postgresql/9.5/main/postgresql.conf" 75 | from: /#?shared_buffers *=.*/ 76 | to: "shared_buffers = $db_shared_buffers" 77 | 78 | # default is 1MB which is too small 79 | - replace: 80 | filename: "/etc/postgresql/9.5/main/postgresql.conf" 81 | from: /#?work_mem *=.*/ 82 | to: "work_mem = $db_work_mem" 83 | 84 | # allow for other 85 | - replace: 86 | filename: "/etc/postgresql/9.5/main/postgresql.conf" 87 | from: /#?default_text_search_config *=.*/ 88 | to: "default_text_search_config = '$db_default_text_search_config'" 89 | 90 | # Necessary to enable backups 91 | - exec: 92 | cmd: 93 | - install -d -m 0755 -o postgres -g postgres /shared/postgres_backup 94 | 95 | - replace: 96 | filename: "/etc/postgresql/9.5/main/postgresql.conf" 97 | from: /#?checkpoint_segments *=.*/ 98 | to: "checkpoint_segments = $db_checkpoint_segments" 99 | 100 | - replace: 101 | filename: "/etc/postgresql/9.5/main/postgresql.conf" 102 | from: /#?logging_collector *=.*/ 103 | to: "logging_collector = $db_logging_collector" 104 | 105 | - replace: 106 | filename: "/etc/postgresql/9.5/main/postgresql.conf" 107 | from: /#?log_min_duration_statement *=.*/ 108 | to: "log_min_duration_statement = $db_log_min_duration_statement" 109 | 110 | - replace: 111 | filename: "/etc/postgresql/9.5/main/pg_hba.conf" 112 | from: /^#local +replication +postgres +peer$/ 113 | to: "local replication postgres peer" 114 | 115 | # allow all to connect in with md5 auth 116 | - replace: 117 | filename: "/etc/postgresql/9.5/main/pg_hba.conf" 118 | from: /^host.*all.*all.*127.*$/ 119 | to: "host all all 0.0.0.0/0 md5" 120 | 121 | - exec: 122 | background: true 123 | # use fast shutdown for pg 124 | stop_signal: INT 125 | cmd: HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/9.5/bin/postmaster -D /etc/postgresql/9.5/main 126 | 127 | # give db a few secs to start up 128 | - exec: "sleep 5" 129 | 130 | - exec: su postgres -c 'createdb $db_name' || true 131 | - exec: su postgres -c 'psql $db_name -c "create user $db_user;"' || true 132 | - exec: su postgres -c 'psql $db_name -c "grant all privileges on database $db_name to $db_user;"' || true 133 | - exec: su postgres -c 'psql $db_name -c "alter schema public owner to $db_user;"' 134 | - exec: su postgres -c 'psql template1 -c "create extension if not exists hstore;"' 135 | - exec: su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"' 136 | - exec: su postgres -c 'psql $db_name -c "create extension if not exists hstore;"' 137 | - exec: su postgres -c 'psql $db_name -c "create extension if not exists pg_trgm;"' 138 | - exec: 139 | stdin: | 140 | update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = '$db_name' AND encoding = pg_char_to_encoding('SQL_ASCII'); 141 | cmd: sudo -u postgres psql $db_name 142 | raise_on_fail: false 143 | 144 | - file: 145 | path: /var/lib/postgresql/take-database-backup 146 | chown: postgres:postgres 147 | chmod: "+x" 148 | contents: | 149 | #!/bin/bash 150 | ID=db-$(date +%F_%T) 151 | FILENAME=/shared/postgres_backup/$ID.tar.gz 152 | pg_basebackup --format=tar --pgdata=- --xlog --gzip --label=$ID > $FILENAME 153 | echo $FILENAME 154 | 155 | - file: 156 | path: /var/spool/cron/crontabs/postgres 157 | contents: | 158 | # m h dom mon dow command 159 | #MAILTO=? 160 | #0 */4 * * * /var/lib/postgresql/take-database-backup 161 | 162 | - exec: 163 | hook: postgres 164 | cmd: "echo postgres installed!" 165 | -------------------------------------------------------------------------------- /templates/postgres.10.template.yml: -------------------------------------------------------------------------------- 1 | params: 2 | db_synchronous_commit: "off" 3 | db_shared_buffers: "256MB" 4 | db_work_mem: "10MB" 5 | db_default_text_search_config: "pg_catalog.english" 6 | db_name: discourse 7 | db_user: discourse 8 | db_checkpoint_segments: 6 9 | db_logging_collector: off 10 | db_log_min_duration_statement: 100 11 | 12 | hooks: 13 | before_code: 14 | - replace: 15 | filename: /etc/service/unicorn/run 16 | from: "# postgres" 17 | to: sv start postgres || exit 1 18 | 19 | run: 20 | - exec: DEBIAN_FRONTEND=noninteractive apt-get purge -y postgresql-12 postgresql-client-12 postgresql-contrib-12 21 | - exec: apt-get update && apt-get install -y postgresql-10 postgresql-client-10 postgresql-contrib-10 22 | - exec: mkdir -p /shared/postgres_run 23 | - exec: chown postgres:postgres /shared/postgres_run 24 | - exec: chmod 775 /shared/postgres_run 25 | - exec: rm -fr /var/run/postgresql 26 | - exec: ln -s /shared/postgres_run /var/run/postgresql 27 | - exec: socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1 28 | - exec: rm -fr /shared/postgres_run/.s* 29 | - exec: rm -fr /shared/postgres_run/*.pid 30 | - exec: mkdir -p /shared/postgres_run/10-main.pg_stat_tmp 31 | - exec: chown postgres:postgres /shared/postgres_run/10-main.pg_stat_tmp 32 | - file: 33 | path: /etc/service/postgres/run 34 | chmod: "+x" 35 | contents: | 36 | #!/bin/sh 37 | exec 2>&1 38 | echo -1000 >/proc/self/oom_score_adj 39 | HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/10/bin/postmaster -D /etc/postgresql/10/main 40 | 41 | - file: 42 | path: /etc/runit/3.d/99-postgres 43 | chmod: "+x" 44 | contents: | 45 | #!/bin/bash 46 | sv stop postgres 47 | 48 | - exec: 49 | cmd: 50 | - chown -R root /var/lib/postgresql/10/main 51 | - "[ ! -e /shared/postgres_data ] && install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u postgres /usr/lib/postgresql/10/bin/initdb -D /shared/postgres_data || exit 0" 52 | - chown -R postgres:postgres /shared/postgres_data 53 | - chown -R postgres:postgres /var/run/postgresql 54 | 55 | - replace: 56 | filename: "/etc/postgresql/10/main/postgresql.conf" 57 | from: "data_directory = '/var/lib/postgresql/10/main'" 58 | to: "data_directory = '/shared/postgres_data'" 59 | 60 | # listen on all interfaces 61 | - replace: 62 | filename: "/etc/postgresql/10/main/postgresql.conf" 63 | from: /#?listen_addresses *=.*/ 64 | to: "listen_addresses = '*'" 65 | 66 | # sync commit off is faster and less spiky, also marginally less safe 67 | - replace: 68 | filename: "/etc/postgresql/10/main/postgresql.conf" 69 | from: /#?synchronous_commit *=.*/ 70 | to: "synchronous_commit = $db_synchronous_commit" 71 | 72 | # default is 128MB which is way too small 73 | - replace: 74 | filename: "/etc/postgresql/10/main/postgresql.conf" 75 | from: /#?shared_buffers *=.*/ 76 | to: "shared_buffers = $db_shared_buffers" 77 | 78 | # default is 1MB which is too small 79 | - replace: 80 | filename: "/etc/postgresql/10/main/postgresql.conf" 81 | from: /#?work_mem *=.*/ 82 | to: "work_mem = $db_work_mem" 83 | 84 | # allow for other 85 | - replace: 86 | filename: "/etc/postgresql/10/main/postgresql.conf" 87 | from: /#?default_text_search_config *=.*/ 88 | to: "default_text_search_config = '$db_default_text_search_config'" 89 | 90 | # Necessary to enable backups 91 | - exec: 92 | cmd: 93 | - install -d -m 0755 -o postgres -g postgres /shared/postgres_backup 94 | 95 | - replace: 96 | filename: "/etc/postgresql/10/main/postgresql.conf" 97 | from: /#?checkpoint_segments *=.*/ 98 | to: "checkpoint_segments = $db_checkpoint_segments" 99 | 100 | - replace: 101 | filename: "/etc/postgresql/10/main/postgresql.conf" 102 | from: /#?logging_collector *=.*/ 103 | to: "logging_collector = $db_logging_collector" 104 | 105 | - replace: 106 | filename: "/etc/postgresql/10/main/postgresql.conf" 107 | from: /#?log_min_duration_statement *=.*/ 108 | to: "log_min_duration_statement = $db_log_min_duration_statement" 109 | 110 | - replace: 111 | filename: "/etc/postgresql/10/main/pg_hba.conf" 112 | from: /^#local +replication +postgres +peer$/ 113 | to: "local replication postgres peer" 114 | 115 | # allow all to connect in with md5 auth 116 | - replace: 117 | filename: "/etc/postgresql/10/main/pg_hba.conf" 118 | from: /^host.*all.*all.*127.*$/ 119 | to: "host all all 0.0.0.0/0 md5" 120 | 121 | - exec: 122 | background: true 123 | # use fast shutdown for pg 124 | stop_signal: INT 125 | cmd: HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/10/bin/postmaster -D /etc/postgresql/10/main 126 | 127 | # give db a few secs to start up 128 | - exec: "sleep 5" 129 | 130 | - exec: su postgres -c 'createdb $db_name' || true 131 | - exec: su postgres -c 'psql $db_name -c "create user $db_user;"' || true 132 | - exec: su postgres -c 'psql $db_name -c "grant all privileges on database $db_name to $db_user;"' || true 133 | - exec: su postgres -c 'psql $db_name -c "alter schema public owner to $db_user;"' 134 | - exec: su postgres -c 'psql template1 -c "create extension if not exists hstore;"' 135 | - exec: su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"' 136 | - exec: su postgres -c 'psql $db_name -c "create extension if not exists hstore;"' 137 | - exec: su postgres -c 'psql $db_name -c "create extension if not exists pg_trgm;"' 138 | - exec: 139 | stdin: | 140 | update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = '$db_name' AND encoding = pg_char_to_encoding('SQL_ASCII'); 141 | cmd: sudo -u postgres psql $db_name 142 | raise_on_fail: false 143 | 144 | - file: 145 | path: /var/lib/postgresql/take-database-backup 146 | chown: postgres:postgres 147 | chmod: "+x" 148 | contents: | 149 | #!/bin/bash 150 | ID=db-$(date +%F_%T) 151 | FILENAME=/shared/postgres_backup/$ID.tar.gz 152 | pg_basebackup --format=tar --pgdata=- --xlog --gzip --label=$ID > $FILENAME 153 | echo $FILENAME 154 | 155 | - file: 156 | path: /var/spool/cron/crontabs/postgres 157 | contents: | 158 | # m h dom mon dow command 159 | #MAILTO=? 160 | #0 */4 * * * /var/lib/postgresql/take-database-backup 161 | 162 | - exec: 163 | hook: postgres 164 | cmd: "echo postgres installed!" 165 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ### About 2 | 3 | - [Docker](https://docker.com/) is an open source project to pack, ship and run any Linux application in a lighter weight, faster container than a traditional virtual machine. 4 | 5 | - Docker makes it much easier to deploy [a Discourse forum](https://github.com/discourse/discourse) on your servers and keep it updated. For background, see [Sam's blog post](http://samsaffron.com/archive/2013/11/07/discourse-in-a-docker-container). 6 | 7 | - The templates and base image configure Discourse with the Discourse team's recommended optimal defaults. 8 | 9 | ### Getting Started 10 | 11 | The simplest way to get started is via the **standalone** template, which can be installed in 30 minutes or less. For detailed install instructions, see 12 | 13 | https://github.com/discourse/discourse/blob/master/docs/INSTALL-cloud.md 14 | 15 | ### Directory Structure 16 | 17 | #### `/cids` 18 | 19 | Contains container ids for currently running Docker containers. cids are Docker's "equivalent" of pids. Each container will have a unique git like hash. 20 | 21 | #### `/containers` 22 | 23 | This directory is for container definitions for your various Discourse containers. You are in charge of this directory, it ships empty. 24 | 25 | #### `/samples` 26 | 27 | Sample container definitions you may use to bootstrap your environment. You can copy templates from here into the containers directory. 28 | 29 | #### `/shared` 30 | 31 | Placeholder spot for shared volumes with various Discourse containers. You may elect to store certain persistent information outside of a container, in our case we keep various logfiles and upload directory outside. This allows you to rebuild containers easily without losing important information. Keeping uploads outside of the container allows you to share them between multiple web instances. 32 | 33 | #### `/templates` 34 | 35 | [pups](https://github.com/samsaffron/pups)-managed templates you may use to bootstrap your environment. 36 | 37 | #### `/image` 38 | 39 | Dockerfiles for Discourse; see [the README](image/README.md) for further details. 40 | 41 | The Docker repository will always contain the latest built version at: https://hub.docker.com/r/discourse/base/, you should not need to build the base image. 42 | 43 | ### Launcher 44 | 45 | The base directory contains a single bash script which is used to manage containers. You can use it to "bootstrap" a new container, enter, start, stop and destroy a container. 46 | 47 | ``` 48 | Usage: launcher COMMAND CONFIG [--skip-prereqs] 49 | Commands: 50 | start: Start/initialize a container 51 | stop: Stop a running container 52 | restart: Restart a container 53 | destroy: Stop and remove a container 54 | enter: Use docker exec to enter a container 55 | logs: Docker logs for container 56 | memconfig: Configure sane defaults for available RAM 57 | bootstrap: Bootstrap a container for the config based on a template 58 | rebuild: Rebuild a container (destroy old, bootstrap, start new) 59 | ``` 60 | 61 | If the environment variable "SUPERVISED" is set to true, the container won't be detached, allowing a process monitoring tool to manage the restart behaviour of the container. 62 | 63 | ### Container Configuration 64 | 65 | The beginning of the container definition can contain the following "special" sections: 66 | 67 | #### templates: 68 | 69 | ``` 70 | templates: 71 | - "templates/cron.template.yml" 72 | - "templates/postgres.template.yml" 73 | ``` 74 | 75 | This template is "composed" out of all these child templates, this allows for a very flexible configuration structure. Furthermore you may add specific hooks that extend the templates you reference. 76 | 77 | #### expose: 78 | 79 | ``` 80 | expose: 81 | - "2222:22" 82 | - "127.0.0.1:20080:80" 83 | ``` 84 | 85 | Publish port 22 inside the container on port 2222 on ALL local host interfaces. In order to bind to only one interface, you may specify the host's IP address as `([:[host_port]])|():[/udp]` as defined in the [docker port binding documentation](http://docs.docker.com/userguide/dockerlinks/). To expose a port without publishing it, specify only the port number (e.g., `80`). 86 | 87 | 88 | #### volumes: 89 | 90 | ``` 91 | volumes: 92 | - volume: 93 | host: /var/discourse/shared 94 | guest: /shared 95 | 96 | ``` 97 | 98 | Expose a directory inside the host to the container. 99 | 100 | #### links: 101 | ``` 102 | links: 103 | - link: 104 | name: postgres 105 | alias: postgres 106 | ``` 107 | 108 | Links another container to the current container. This will add `--link postgres:postgres` 109 | to the options when running the container. 110 | 111 | #### environment variables: 112 | 113 | Setting environment variables to the current container. 114 | 115 | ``` 116 | # app.yml 117 | 118 | env: 119 | DISCOURSE_DB_HOST: some-host 120 | DISCOURSE_DB_NAME: {{config}}_discourse 121 | ``` 122 | 123 | The above will add `-e DISCOURSE_DB_HOST=some-host -e DISCOURSE_DB_NAME=app_discourse` to the options when running the container. 124 | 125 | #### labels: 126 | ``` 127 | # app.yml 128 | 129 | labels: 130 | monitor: 'true' 131 | app_name: {{config}}_discourse 132 | ``` 133 | 134 | Add labels to the current container. The above will add `--l monitor=true -l app_name=dev_discourse` to the options 135 | when running the container 136 | 137 | ### Upgrading Discourse 138 | 139 | The Docker setup gives you multiple upgrade options: 140 | 141 | 1. Use the front end at http://yoursite.com/admin/upgrade to upgrade an already running image. 142 | 143 | 2. Create a new base image manually by running: 144 | - `./launcher rebuild my_image` 145 | 146 | ### Single Container vs. Multiple Container 147 | 148 | The samples directory contains a standalone template. This template bundles all of the software required to run Discourse into a single container. The advantage is that it is easy. 149 | 150 | The multiple container configuration setup is far more flexible and robust, however it is also more complicated to set up. A multiple container setup allows you to: 151 | 152 | - Minimize downtime when upgrading to new versions of Discourse. You can bootstrap new web processes while your site is running and only after it is built, switch the new image in. 153 | - Scale your forum to multiple servers. 154 | - Add servers for redundancy. 155 | - Have some required services (e.g. the database) run on beefier hardware. 156 | 157 | If you want a multiple container setup, see the `data.yml` and `web_only.yml` templates in the samples directory. To ease this process, `launcher` will inject an env var called `DISCOURSE_HOST_IP` which will be available inside the image. 158 | 159 | WARNING: In a multiple container configuration, *make sure* you setup iptables or some other firewall to protect various ports (for postgres/redis). 160 | On Ubuntu, install the `ufw` or `iptables-persistent` package to manage firewall rules. 161 | 162 | ### Email 163 | 164 | For a Discourse instance to function properly Email must be set up. Use the `SMTP_URL` env var to set your SMTP address, see sample templates for an example. The Docker image does not contain postfix, exim or another MTA, it was omitted because it is very tricky to set up correctly. 165 | 166 | ### Troubleshooting 167 | 168 | View the container logs: `./launcher logs my_container` 169 | 170 | Spawn a shell inside your container using `./launcher enter my_container`. This is the most foolproof method if you have host root access. 171 | 172 | If you see network errors trying to retrieve code from `github.com` or `rubygems.org` try again - sometimes there are temporary interruptions and a retry is all it takes. 173 | 174 | Behind a proxy network with no direct access to the Internet? Add proxy information to the container environment by adding to the existing `env` block in the `container.yml` file: 175 | 176 | ```yaml 177 | env: 178 | …existing entries… 179 | HTTP_PROXY: http://proxyserver:port/ 180 | http_proxy: http://proxyserver:port/ 181 | HTTPS_PROXY: http://proxyserver:port/ 182 | https_proxy: http://proxyserver:port/ 183 | ``` 184 | 185 | ### Security 186 | 187 | Directory permissions in Linux are UID/GID based, if your numeric IDs on the 188 | host do not match the IDs in the guest, permissions will mismatch. On clean 189 | installs you can ensure they are in sync by looking at `/etc/passwd` and 190 | `/etc/group`, the Discourse account will have UID 1000. 191 | 192 | 193 | ### Advanced topics 194 | 195 | - [Setting up SSL with Discourse Docker](https://meta.discourse.org/t/allowing-ssl-for-your-discourse-docker-setup/13847) 196 | - [Multisite configuration with Docker](https://meta.discourse.org/t/multisite-configuration-with-docker/14084) 197 | - [Linking containers for a multiple container setup](https://meta.discourse.org/t/linking-containers-for-a-multiple-container-setup/20867) 198 | - [Using Rubygems mirror to improve connection problem in China](https://meta.discourse.org/t/replace-rubygems-org-with-taobao-mirror-to-resolve-network-error-in-china/21988/1) 199 | 200 | License 201 | === 202 | MIT 203 | -------------------------------------------------------------------------------- /templates/postgres.template.yml: -------------------------------------------------------------------------------- 1 | params: 2 | db_synchronous_commit: "off" 3 | db_shared_buffers: "256MB" 4 | db_work_mem: "10MB" 5 | db_default_text_search_config: "pg_catalog.english" 6 | db_name: discourse 7 | db_user: discourse 8 | db_checkpoint_segments: 6 9 | db_logging_collector: off 10 | db_log_min_duration_statement: 100 11 | 12 | hooks: 13 | before_code: 14 | - replace: 15 | filename: /etc/service/unicorn/run 16 | from: "# postgres" 17 | to: sv start postgres || exit 1 18 | 19 | run: 20 | - exec: locale-gen $LANG && update-locale 21 | - exec: mkdir -p /shared/postgres_run 22 | - exec: chown postgres:postgres /shared/postgres_run 23 | - exec: chmod 775 /shared/postgres_run 24 | - exec: rm -fr /var/run/postgresql 25 | - exec: ln -s /shared/postgres_run /var/run/postgresql 26 | - exec: socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 && echo postgres already running stop container ; exit 1 27 | - exec: rm -fr /shared/postgres_run/.s* 28 | - exec: rm -fr /shared/postgres_run/*.pid 29 | - exec: mkdir -p /shared/postgres_run/12-main.pg_stat_tmp 30 | - exec: chown postgres:postgres /shared/postgres_run/12-main.pg_stat_tmp 31 | - file: 32 | path: /etc/service/postgres/run 33 | chmod: "+x" 34 | contents: | 35 | #!/bin/sh 36 | exec 2>&1 37 | HOME=/var/lib/postgresql USER=postgres exec thpoff chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/12/bin/postmaster -D /etc/postgresql/12/main 38 | 39 | - file: 40 | path: /etc/service/postgres/log/run 41 | chmod: "+x" 42 | contents: | 43 | #!/bin/sh 44 | mkdir -p /var/log/postgres 45 | exec svlogd /var/log/postgres 46 | 47 | - file: 48 | path: /etc/runit/3.d/99-postgres 49 | chmod: "+x" 50 | contents: | 51 | #!/bin/bash 52 | sv stop postgres 53 | 54 | - file: 55 | path: /root/upgrade_postgres 56 | chmod: "+x" 57 | contents: | 58 | #!/bin/bash 59 | PG_MAJOR_OLD=`cat /shared/postgres_data/PG_VERSION` 60 | 61 | if [ ! "12" = "$PG_MAJOR_OLD" ]; then 62 | echo Upgrading PostgreSQL from version ${PG_MAJOR_OLD} to 12 63 | free_disk=$(df -P -B1 /shared | tail -n 1 | awk '{print $4}') 64 | required=$(($(du -sb /shared/postgres_data | awk '{print $1}') * 2)) 65 | 66 | if [ "$free_disk" -lt "$required" ]; then 67 | echo "WARNING: Upgrading PostgresSQL would require an addtional $(numfmt --to=si $(($required - $free_disk))) of disk space" 68 | echo "Please free up some space, or expand your disk, before continuing." 69 | echo '' 70 | echo 'To avoid upgrading change "templates/postgres.template.yml" TO "templates/postgres.10.template.yml" in containers/app.yml' 71 | exit 1 72 | fi 73 | 74 | if [ -d /shared/postgres_data_old ]; then 75 | mv /shared/postgres_data_old /shared/postgres_data_older 76 | fi 77 | 78 | rm -fr /shared/postgres_data_new 79 | install -d -m 0755 -o postgres -g postgres /shared/postgres_data_new && sudo -u postgres /usr/lib/postgresql/12/bin/initdb -D /shared/postgres_data_new || exit 0 80 | apt-get update 81 | apt-get install -y postgresql-${PG_MAJOR_OLD} 82 | chown -R postgres:postgres /var/lib/postgresql/12 83 | /etc/init.d/postgresql stop 84 | rm -fr /shared/postgres_data/postmaster.pid 85 | cd ~postgres 86 | cp -pr /etc/postgresql/${PG_MAJOR_OLD}/main/* /shared/postgres_data 87 | echo >> /shared/postgres_data/postgresql.conf 88 | echo "data_directory = '/shared/postgres_data'" >> /shared/postgres_data/postgresql.conf 89 | SUCCESS=true 90 | sudo -u postgres /usr/lib/postgresql/12/bin/pg_upgrade -d /shared/postgres_data -D /shared/postgres_data_new -b /usr/lib/postgresql/${PG_MAJOR_OLD}/bin -B /usr/lib/postgresql/12/bin || SUCCESS=false 91 | 92 | if [[ "$SUCCESS" == 'false' ]]; then 93 | echo ------------------------------------------------------------------------------------- 94 | echo UPGRADE OF POSTGRES FAILED 95 | echo 96 | echo Please visit https://meta.discourse.org/t/postgresql-12-update/151236 for support 97 | echo 98 | echo You can run "./launcher start app" to restart your app in the meanwhile 99 | echo 100 | exit 1 101 | fi 102 | 103 | mv /shared/postgres_data /shared/postgres_data_old 104 | mv /shared/postgres_data_new /shared/postgres_data 105 | 106 | echo ------------------------------------------------------------------------------------- 107 | echo UPGRADE OF POSTGRES COMPLETE 108 | echo 109 | echo Old ${PG_MAJOR_OLD} database is stored at /shared/postgres_data_old 110 | echo 111 | echo To complete the upgrade, rebuild again using: 112 | echo 113 | echo ./launcher rebuild app 114 | echo ------------------------------------------------------------------------------------- 115 | # Magic exit status to denote no failure 116 | exit 77 117 | fi 118 | 119 | - exec: 120 | cmd: 121 | - chown -R root /var/lib/postgresql/12/main 122 | - "[ ! -e /shared/postgres_data ] && install -d -m 0755 -o postgres -g postgres /shared/postgres_data && sudo -E -u postgres /usr/lib/postgresql/12/bin/initdb -D /shared/postgres_data || exit 0" 123 | - chown -R postgres:postgres /shared/postgres_data 124 | - chown -R postgres:postgres /var/run/postgresql 125 | 126 | - exec: /root/upgrade_postgres 127 | - exec: rm /root/upgrade_postgres 128 | 129 | - replace: 130 | filename: "/etc/postgresql/12/main/postgresql.conf" 131 | from: "data_directory = '/var/lib/postgresql/12/main'" 132 | to: "data_directory = '/shared/postgres_data'" 133 | 134 | # listen on all interfaces 135 | - replace: 136 | filename: "/etc/postgresql/12/main/postgresql.conf" 137 | from: /#?listen_addresses *=.*/ 138 | to: "listen_addresses = '*'" 139 | 140 | # sync commit off is faster and less spiky, also marginally less safe 141 | - replace: 142 | filename: "/etc/postgresql/12/main/postgresql.conf" 143 | from: /#?synchronous_commit *=.*/ 144 | to: "synchronous_commit = $db_synchronous_commit" 145 | 146 | # default is 128MB which is way too small 147 | - replace: 148 | filename: "/etc/postgresql/12/main/postgresql.conf" 149 | from: /#?shared_buffers *=.*/ 150 | to: "shared_buffers = $db_shared_buffers" 151 | 152 | # default is 1MB which is too small 153 | - replace: 154 | filename: "/etc/postgresql/12/main/postgresql.conf" 155 | from: /#?work_mem *=.*/ 156 | to: "work_mem = $db_work_mem" 157 | 158 | # allow for other 159 | - replace: 160 | filename: "/etc/postgresql/12/main/postgresql.conf" 161 | from: /#?default_text_search_config *=.*/ 162 | to: "default_text_search_config = '$db_default_text_search_config'" 163 | 164 | # Necessary to enable backups 165 | - exec: 166 | cmd: 167 | - install -d -m 0755 -o postgres -g postgres /shared/postgres_backup 168 | 169 | - replace: 170 | filename: "/etc/postgresql/12/main/postgresql.conf" 171 | from: /#?checkpoint_segments *=.*/ 172 | to: "checkpoint_segments = $db_checkpoint_segments" 173 | 174 | - replace: 175 | filename: "/etc/postgresql/12/main/postgresql.conf" 176 | from: /#?logging_collector *=.*/ 177 | to: "logging_collector = $db_logging_collector" 178 | 179 | - replace: 180 | filename: "/etc/postgresql/12/main/postgresql.conf" 181 | from: /#?log_min_duration_statement *=.*/ 182 | to: "log_min_duration_statement = $db_log_min_duration_statement" 183 | 184 | - replace: 185 | filename: "/etc/postgresql/12/main/pg_hba.conf" 186 | from: /^#local +replication +postgres +peer$/ 187 | to: "local replication postgres peer" 188 | 189 | # allow all to connect in with md5 auth 190 | - replace: 191 | filename: "/etc/postgresql/12/main/pg_hba.conf" 192 | from: /^host.*all.*all.*127.*$/ 193 | to: "host all all 0.0.0.0/0 md5" 194 | 195 | - exec: 196 | background: true 197 | # use fast shutdown for pg 198 | stop_signal: INT 199 | cmd: HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/12/bin/postmaster -D /etc/postgresql/12/main 200 | 201 | # give db a few secs to start up 202 | - exec: "sleep 5" 203 | 204 | - exec: su postgres -c 'createdb $db_name' || true 205 | - exec: su postgres -c 'psql $db_name -c "create user $db_user;"' || true 206 | - exec: su postgres -c 'psql $db_name -c "grant all privileges on database $db_name to $db_user;"' || true 207 | - exec: su postgres -c 'psql $db_name -c "alter schema public owner to $db_user;"' 208 | - exec: su postgres -c 'psql template1 -c "create extension if not exists hstore;"' 209 | - exec: su postgres -c 'psql template1 -c "create extension if not exists pg_trgm;"' 210 | - exec: su postgres -c 'psql $db_name -c "create extension if not exists hstore;"' 211 | - exec: su postgres -c 'psql $db_name -c "create extension if not exists pg_trgm;"' 212 | - exec: 213 | stdin: | 214 | update pg_database set encoding = pg_char_to_encoding('UTF8') where datname = '$db_name' AND encoding = pg_char_to_encoding('SQL_ASCII'); 215 | cmd: sudo -u postgres psql $db_name 216 | raise_on_fail: false 217 | 218 | - file: 219 | path: /var/lib/postgresql/take-database-backup 220 | chown: postgres:postgres 221 | chmod: "+x" 222 | contents: | 223 | #!/bin/bash 224 | ID=db-$(date +%F_%T) 225 | FILENAME=/shared/postgres_backup/$ID.tar.gz 226 | pg_basebackup --format=tar --pgdata=- --xlog --gzip --label=$ID > $FILENAME 227 | echo $FILENAME 228 | 229 | - file: 230 | path: /var/spool/cron/crontabs/postgres 231 | contents: | 232 | # m h dom mon dow command 233 | #MAILTO=? 234 | #0 */4 * * * /var/lib/postgresql/take-database-backup 235 | 236 | - exec: 237 | hook: postgres 238 | cmd: "echo postgres installed!" 239 | -------------------------------------------------------------------------------- /templates/web.template.yml: -------------------------------------------------------------------------------- 1 | env: 2 | # You can have redis on a different box 3 | RAILS_ENV: 'production' 4 | UNICORN_WORKERS: 3 5 | UNICORN_SIDEKIQS: 1 6 | # this gives us very good cache coverage, 96 -> 99 7 | # in practice it is 1-2% perf improvement 8 | RUBY_GLOBAL_METHOD_CACHE_SIZE: 131072 9 | # stop heap doubling in size so aggressively, this conserves memory 10 | RUBY_GC_HEAP_GROWTH_MAX_SLOTS: 40000 11 | RUBY_GC_HEAP_INIT_SLOTS: 400000 12 | RUBY_GC_HEAP_OLDOBJECT_LIMIT_FACTOR: 1.5 13 | 14 | DISCOURSE_DB_SOCKET: /var/run/postgresql 15 | DISCOURSE_DB_HOST: 16 | DISCOURSE_DB_PORT: 17 | 18 | 19 | params: 20 | version: tests-passed 21 | 22 | home: /var/www/discourse 23 | upload_size: 10m 24 | 25 | run: 26 | - exec: thpoff echo "thpoff is installed!" 27 | - exec: /usr/local/bin/ruby -e 'if ENV["DISCOURSE_SMTP_ADDRESS"] == "smtp.example.com"; puts "Aborting! Mail is not configured!"; exit 1; end' 28 | - exec: /usr/local/bin/ruby -e 'if ENV["DISCOURSE_HOSTNAME"] == "discourse.example.com"; puts "Aborting! Domain is not configured!"; exit 1; end' 29 | - exec: /usr/local/bin/ruby -e 'if (ENV["DISCOURSE_CDN_URL"] || "")[0..1] == "//"; puts "Aborting! CDN must have a protocol specified. Once fixed you should rebake your posts now to correct all posts."; exit 1; end' 30 | - exec: chown -R discourse /home/discourse 31 | # TODO: move to base image (anacron can not be fired up using rc.d) 32 | - exec: rm -f /etc/cron.d/anacron 33 | - file: 34 | path: /etc/cron.d/anacron 35 | contents: | 36 | SHELL=/bin/sh 37 | PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin 38 | 39 | 30 7 * * * root /usr/sbin/anacron -s >/dev/null 40 | - file: 41 | path: /etc/runit/1.d/copy-env 42 | chmod: "+x" 43 | contents: | 44 | #!/bin/bash 45 | env > ~/boot_env 46 | conf=/var/www/discourse/config/discourse.conf 47 | 48 | # find DISCOURSE_ env vars, strip the leader, lowercase the key 49 | /usr/local/bin/ruby -e 'ENV.each{|k,v| puts "#{$1.downcase} = '\''#{v}'\''" if k =~ /^DISCOURSE_(.*)/}' > $conf 50 | 51 | - file: 52 | path: /etc/service/unicorn/run 53 | chmod: "+x" 54 | contents: | 55 | #!/bin/bash 56 | exec 2>&1 57 | # redis 58 | # postgres 59 | cd $home 60 | chown -R discourse:www-data /shared/log/rails 61 | LD_PRELOAD=$RUBY_ALLOCATOR HOME=/home/discourse USER=discourse exec thpoff chpst -u discourse:www-data -U discourse:www-data bundle exec config/unicorn_launcher -E production -c config/unicorn.conf.rb 62 | 63 | - file: 64 | path: /etc/service/nginx/run 65 | chmod: "+x" 66 | contents: | 67 | #!/bin/sh 68 | exec 2>&1 69 | exec /usr/sbin/nginx 70 | 71 | - file: 72 | path: /etc/runit/3.d/01-nginx 73 | chmod: "+x" 74 | contents: | 75 | #!/bin/bash 76 | sv stop nginx 77 | 78 | - file: 79 | path: /etc/runit/3.d/02-unicorn 80 | chmod: "+x" 81 | contents: | 82 | #!/bin/bash 83 | sv stop unicorn 84 | 85 | - exec: 86 | cd: $home 87 | hook: code 88 | cmd: 89 | - git reset --hard 90 | - git clean -f 91 | - git remote set-branches --add origin master 92 | - git pull 93 | - git fetch origin $version 94 | - git checkout $version 95 | - mkdir -p tmp 96 | - chown discourse:www-data tmp 97 | - mkdir -p tmp/pids 98 | - mkdir -p tmp/sockets 99 | - touch tmp/.gitkeep 100 | - mkdir -p /shared/log/rails 101 | - bash -c "touch -a /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log" 102 | - bash -c "ln -s /shared/log/rails/{production,production_errors,unicorn.stdout,unicorn.stderr,sidekiq}.log $home/log" 103 | - bash -c "mkdir -p /shared/{uploads,backups}" 104 | - bash -c "ln -s /shared/{uploads,backups} $home/public" 105 | - bash -c "mkdir -p /shared/tmp/{backups,restores}" 106 | - bash -c "ln -s /shared/tmp/{backups,restores} $home/tmp" 107 | - chown -R discourse:www-data /shared/log/rails /shared/uploads /shared/backups /shared/tmp 108 | # scrub broken symlinks from plugins that have been removed 109 | - find public/plugins/ -maxdepth 1 -xtype l -delete 110 | 111 | - exec: 112 | cmd: 113 | - "cp $home/config/nginx.sample.conf /etc/nginx/conf.d/discourse.conf" 114 | - "rm /etc/nginx/sites-enabled/default" 115 | - "mkdir -p /var/nginx/cache" 116 | 117 | - replace: 118 | filename: /etc/nginx/nginx.conf 119 | from: pid /run/nginx.pid; 120 | to: daemon off; 121 | 122 | - replace: 123 | filename: "/etc/nginx/conf.d/discourse.conf" 124 | from: /upstream[^\}]+\}/m 125 | to: "upstream discourse { 126 | server 127.0.0.1:3000; 127 | }" 128 | 129 | - replace: 130 | filename: "/etc/nginx/conf.d/discourse.conf" 131 | from: /server_name.+$/ 132 | to: server_name _ ; 133 | 134 | - replace: 135 | filename: "/etc/nginx/conf.d/discourse.conf" 136 | from: /client_max_body_size.+$/ 137 | to: client_max_body_size $upload_size ; 138 | 139 | - exec: 140 | cmd: echo "done configuring web" 141 | hook: web_config 142 | 143 | - exec: 144 | cd: $home 145 | hook: web 146 | cmd: 147 | # ensure we are on latest bundler 148 | - gem update bundler 149 | - find $home ! -user discourse -exec chown discourse {} \+ 150 | 151 | - exec: 152 | cd: $home 153 | hook: bundle_exec 154 | cmd: 155 | - su discourse -c 'bundle install --deployment --retry 3 --jobs 4 --verbose --without test development' 156 | 157 | - exec: 158 | cd: $home 159 | cmd: 160 | - su discourse -c 'bundle exec rake plugin:pull_compatible_all' 161 | raise_on_fail: false 162 | 163 | - exec: 164 | cd: $home 165 | hook: db_migrate 166 | cmd: 167 | - su discourse -c 'bundle exec rake db:migrate' 168 | - exec: 169 | cd: $home 170 | hook: assets_precompile 171 | cmd: 172 | - su discourse -c 'bundle exec rake assets:precompile' 173 | - file: 174 | path: /usr/local/bin/discourse 175 | chmod: +x 176 | contents: | 177 | #!/bin/bash 178 | (cd /var/www/discourse && RAILS_ENV=production sudo -H -E -u discourse bundle exec script/discourse "$@") 179 | 180 | - file: 181 | path: /usr/local/bin/rails 182 | chmod: +x 183 | contents: | 184 | #!/bin/bash 185 | # If they requested a console, load pry instead 186 | if [ "$*" == "c" -o "$*" == "console" ] 187 | then 188 | (cd /var/www/discourse && RAILS_ENV=production sudo -H -E -u discourse bundle exec pry -r ./config/environment) 189 | else 190 | (cd /var/www/discourse && RAILS_ENV=production sudo -H -E -u discourse bundle exec script/rails "$@") 191 | fi 192 | 193 | - file: 194 | path: /usr/local/bin/rake 195 | chmod: +x 196 | contents: | 197 | #!/bin/bash 198 | (cd /var/www/discourse && RAILS_ENV=production sudo -H -E -u discourse bundle exec bin/rake "$@") 199 | 200 | - file: 201 | path: /usr/local/bin/rbtrace 202 | chmod: +x 203 | contents: | 204 | #!/bin/bash 205 | (cd /var/www/discourse && RAILS_ENV=production sudo -H -E -u discourse bundle exec rbtrace "$@") 206 | 207 | - file: 208 | path: /usr/local/bin/stackprof 209 | chmod: +x 210 | contents: | 211 | #!/bin/bash 212 | (cd /var/www/discourse && RAILS_ENV=production sudo -H -E -u discourse bundle exec stackprof "$@") 213 | 214 | - file: 215 | path: /etc/update-motd.d/10-web 216 | chmod: +x 217 | contents: | 218 | #!/bin/bash 219 | echo 220 | echo Use: rails, rake or discourse to execute commands in production 221 | echo 222 | 223 | - file: 224 | path: /etc/logrotate.d/rails 225 | contents: | 226 | /shared/log/rails/*.log 227 | { 228 | rotate 7 229 | dateext 230 | daily 231 | missingok 232 | delaycompress 233 | compress 234 | postrotate 235 | sv 1 unicorn 236 | endscript 237 | } 238 | 239 | - file: 240 | path: /etc/logrotate.d/nginx 241 | contents: | 242 | /var/log/nginx/*.log { 243 | daily 244 | missingok 245 | rotate 7 246 | compress 247 | delaycompress 248 | create 0644 www-data www-data 249 | sharedscripts 250 | postrotate 251 | sv 1 nginx 252 | endscript 253 | } 254 | 255 | # move state out of the container this fancy is done to support rapid rebuilds of containers, 256 | # we store anacron and logrotate state outside the container to ensure its maintained across builds 257 | # later move this snipped into an intialization script 258 | # we also ensure all the symlinks we need to /shared are in place in the correct structure 259 | # this allows us to bootstrap on one machine and then run on another 260 | - file: 261 | path: /etc/runit/1.d/00-ensure-links 262 | chmod: +x 263 | contents: | 264 | #!/bin/bash 265 | if [[ ! -L /var/lib/logrotate ]]; then 266 | rm -fr /var/lib/logrotate 267 | mkdir -p /shared/state/logrotate 268 | ln -s /shared/state/logrotate /var/lib/logrotate 269 | fi 270 | if [[ ! -L /var/spool/anacron ]]; then 271 | rm -fr /var/spool/anacron 272 | mkdir -p /shared/state/anacron-spool 273 | ln -s /shared/state/anacron-spool /var/spool/anacron 274 | fi 275 | if [[ ! -d /shared/log/rails ]]; then 276 | mkdir -p /shared/log/rails 277 | chown -R discourse:www-data /shared/log/rails 278 | fi 279 | if [[ ! -d /shared/uploads ]]; then 280 | mkdir -p /shared/uploads 281 | chown -R discourse:www-data /shared/uploads 282 | fi 283 | if [[ ! -d /shared/backups ]]; then 284 | mkdir -p /shared/backups 285 | chown -R discourse:www-data /shared/backups 286 | fi 287 | 288 | rm -rf /shared/tmp/{backups,restores} 289 | mkdir -p /shared/tmp/{backups,restores} 290 | chown -R discourse:www-data /shared/tmp/{backups,restores} 291 | 292 | # change login directory to Discourse home 293 | - file: 294 | path: /root/.bash_profile 295 | chmod: 644 296 | contents: | 297 | cd $home 298 | -------------------------------------------------------------------------------- /discourse-doctor: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | LOG_FILE="/tmp/discourse-debug.txt" 3 | WORKING_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 4 | 5 | log() { 6 | if [ "$1" == "-e" ] 7 | then 8 | shift 9 | echo -e "$*" | tee -a "$LOG_FILE" 10 | else 11 | echo "$*" | tee -a "$LOG_FILE" 12 | fi 13 | } 14 | 15 | check_root() { 16 | if [[ $EUID -ne 0 ]]; then 17 | log "This script must be run as root. Please sudo or log in as root first." 1>&2 18 | exit 1 19 | fi 20 | } 21 | 22 | ## 23 | ## Check whether a connection to HOSTNAME ($1) on PORT ($2) is possible 24 | ## 25 | connect_to_port() { 26 | HOST="$1" 27 | PORT="$2" 28 | VERIFY=$(date +%s | sha256sum | base64 | head -c 20) 29 | echo -e "HTTP/1.1 200 OK\n\n $VERIFY" | nc -w 4 -l -p $PORT >/dev/null 2>&1 & 30 | if curl --proto =http -s $HOST:$PORT --connect-timeout 3 | grep $VERIFY >/dev/null 2>&1 & 31 | then 32 | return 0 33 | else 34 | return 1 35 | fi 36 | } 37 | 38 | check_ip_match() { 39 | HOST="$1" 40 | log 41 | log Checking your domain name . . . 42 | if connect_to_port $HOST 443 43 | then 44 | log 45 | log "Connection to $HOST succeeded." 46 | else 47 | log WARNING:: This server does not appear to be accessible at $HOST:443. 48 | log 49 | if connect_to_port $HOST 80 50 | then 51 | log A connection to port 80 succeeds, however. 52 | log This suggests that your DNS settings are correct, 53 | log but something is keeping traffic to port 443 from getting to your server. 54 | log Check your networking configuration to see that connections to port 443 are allowed. 55 | else 56 | log "A connection to http://$HOST (port 80) also fails." 57 | log 58 | log This suggests that $HOST resolves to the wrong IP address 59 | log or that traffic is not being routed to your server. 60 | fi 61 | log 62 | log Google: \"open ports YOUR CLOUD SERVICE\" for information for resolving this problem. 63 | log 64 | log This test might not work for all situations, 65 | log so if you can access Discourse at http://$HOST, this might not indicate a problem. 66 | sleep 3 67 | fi 68 | } 69 | 70 | check_docker_is_installed() { 71 | log -e "\n==================== DOCKER INFO ====================" 72 | docker_path="$(which docker.io || which docker)" 73 | if [ -z $docker_path ]; then 74 | log "Docker is not installed. Have you installed Discourse at all?" 75 | log "Perhaps you're looking for ./discourse-setup ." 76 | log "There is no point in continuing." 77 | exit 78 | else 79 | log -e "DOCKER VERSION: $(docker --version)" 80 | log -e "\nDOCKER PROCESSES (docker ps -a)\n\n$(sudo docker ps -a)\n" 81 | fi 82 | } 83 | 84 | get_OS() { 85 | log -e "OS: $(uname -s)" 86 | } 87 | 88 | check_disk_and_memory() { 89 | log -e "\n\n==================== MEMORY INFORMATION ====================" 90 | os_type=$(get_OS) 91 | if [ "$os_type" == "Darwin" ]; then 92 | log -e "RAM: $( free -m | awk '/Mem:/ {print $2}' ) \n" 93 | else 94 | log -e "RAM (MB): $( free -m --si | awk ' /Mem:/ {print $2} ')\n" 95 | fi 96 | log "$(free -m)" 97 | 98 | log -e "\n==================== DISK SPACE CHECK ====================" 99 | log "---------- OS Disk Space ----------" 100 | log "$(df -h / /var/discourse /var/lib/docker /var/lib/docker/* | uniq)" 101 | 102 | if [ "$version" != "NOT FOUND" ] 103 | then 104 | log 105 | log "---------- Container Disk Space ----------" 106 | log "$(sudo docker exec -w /var/www/discourse -i $app_name df -h / /shared/ /shared/postgres_data /shared/redis_data /shared/backups /var/log | uniq)" 107 | fi 108 | 109 | log -e "\n==================== DISK INFORMATION ====================" 110 | log "$( fdisk -l )" 111 | log -e "\n==================== END DISK INFORMATION ====================" 112 | 113 | free_disk="$(df /var | tail -n 1 | awk '{print $4}')" 114 | # Arguably ./launcher is doing this so discourse-doctor does not need to . . . 115 | if [ "$free_disk" -lt 5000 ]; then 116 | log "\n\n==================== DISK SPACE PROBLEM ====================" 117 | log "WARNING: you appear to have very low disk space." 118 | log "This could be the cause of problems running your site." 119 | log "Please free up some space, or expand your disk, before continuing." 120 | log 121 | log "Run \'apt-get autoremove && apt-get autoclean\' to clean up unused" 122 | log "packages and \'./launcher cleanup\' to remove stale Docker containers." 123 | exit 1 124 | fi 125 | } 126 | 127 | get_discourse_version() { 128 | version="" 129 | version=$(wget -q --timeout=3 https://$VERSION_HOSTNAME/privacy -O -|grep generator|head -1 |cut -d "=" -f 3|cut -d '-' -f 1 |cut -d '"' -f 2) &> /dev/null 130 | if ! echo $version | grep Discourse 131 | then 132 | version=$(wget -q --timeout=3 http://$VERSION_HOSTNAME/privacy -O -|grep generator|head -1 |cut -d "=" -f 3|cut -d '-' -f 1 |cut -d '"' -f 2) &> /dev/null 133 | fi 134 | if [ -z "$version" ] 135 | then 136 | version="NOT FOUND" 137 | fi 138 | log "Discourse version at $VERSION_HOSTNAME: $version" 139 | } 140 | 141 | check_if_hostname_resolves_here() { 142 | log "========================================" 143 | VERSION_HOSTNAME=$DISCOURSE_HOSTNAME 144 | get_discourse_version 145 | DISCOURSE_VERSION="$version" 146 | VERSION_HOSTNAME=localhost 147 | get_discourse_version 148 | LOCALHOST_VERSION="$version" 149 | if [ "$DISCOURSE_VERSION" != "$LOCALHOST_VERSION" ] 150 | then 151 | log "==================== DNS PROBLEM ====================" 152 | log "This server reports $LOCALHOST_VERSION, but $DISCOURSE_HOSTNAME reports $DISCOURSE_VERSION." 153 | log "This suggests that you have a DNS problem or that an intermediate proxy is to blame." 154 | log "If you are using Cloudflare, or a CDN, it may be improperly configured." 155 | fi 156 | } 157 | 158 | ## 159 | ## get discourse configuration values from YML file 160 | ## 161 | get_discourse_config() { 162 | log -e "\n==================== YML SETTINGS ====================" 163 | read_config "DISCOURSE_HOSTNAME" 164 | DISCOURSE_HOSTNAME=$read_config_result 165 | log DISCOURSE_HOSTNAME=$DISCOURSE_HOSTNAME 166 | read_config "DISCOURSE_SMTP_ADDRESS" 167 | SMTP_ADDRESS=$read_config_result 168 | log SMTP_ADDRESS=$SMTP_ADDRESS 169 | read_config "DISCOURSE_DEVELOPER_EMAILS" 170 | DEVELOPER_EMAILS=$read_config_result 171 | log DEVELOPER_EMAILS=$DEVELOPER_EMAILS 172 | read_config "DISCOURSE_SMTP_PASSWORD" 173 | SMTP_PASSWORD=$read_config_result 174 | log SMTP_PASSWORD=$read_config_result 175 | read_config "DISCOURSE_SMTP_PORT" 176 | SMTP_PORT=$read_config_result 177 | log SMTP_PORT=$read_config_result 178 | read_config "DISCOURSE_SMTP_USER_NAME" 179 | SMTP_USER_NAME=$read_config_result 180 | log SMTP_USER_NAME=$read_config_result 181 | read_config "LETSENCRYPT_ACCOUNT_EMAIL" 182 | letsencrypt_account_email=$read_config_result 183 | log "LETSENCRYPT_ACCOUNT_EMAIL=$letsencrypt_account_email" 184 | } 185 | 186 | check_plugins() { 187 | log -e "\n\n==================== PLUGINS ====================" 188 | log -e "$(grep git containers/$app_name.yml)" 189 | grep git containers/$app_name.yml > /tmp/$PPID.grep 190 | 191 | if grep -cv "github.com/discourse" /tmp/$PPID.grep > /dev/null 192 | then 193 | log -e "\nWARNING:" 194 | log You have what appear to be non-official plugins. 195 | log "If you are having trouble, you should disable them and try rebuilding again." 196 | else 197 | log -e "\nNo non-official plugins detected." 198 | fi 199 | log -e "\nSee https://github.com/discourse/discourse/blob/master/lib/plugin/metadata.rb for the official list.\n" 200 | } 201 | 202 | dump_yaml() { 203 | log -e "\n\n==================== YML DUMP ====================" 204 | log Dumping $app_name.yml 205 | log -e "\n\n" 206 | } 207 | 208 | ## 209 | ## read a variable from the config file and stick it in read_config_result 210 | ## 211 | read_config() { 212 | config_line=$(egrep "^ #?$1:" $web_file) 213 | read_config_result=$(echo $config_line | awk --field-separator=":" '{print $2}') 214 | read_config_result=$(echo $read_config_result | sed "s/^\([\"']\)\(.*\)\1\$/\2/g") 215 | } 216 | 217 | ## 218 | ## call rake emails:test inside the container 219 | ## 220 | check_email() { 221 | log -e "\n==================== MAIL TEST ====================" 222 | log "For a robust test, get an address from http://www.mail-tester.com/" 223 | echo "Or just send a test message to yourself." 224 | EMAIL=$(echo $DEVELOPER_EMAILS |cut -d , -f 1) 225 | read -p "Email address for mail test? ('n' to skip) [$EMAIL]: " new_value 226 | if [ ! -z "$new_value" ] 227 | then 228 | EMAIL="$new_value" 229 | fi 230 | if [ "$new_value" != "n" ] && [ "$new_value" != "N" ] 231 | then 232 | log "Sending mail to $EMAIL. . . " 233 | log "$(sudo docker exec -w /var/www/discourse -i $app_name rake emails:test[$EMAIL])" 234 | else 235 | log "Mail test skipped." 236 | fi 237 | } 238 | 239 | get_yml_file() { 240 | app_name="" 241 | if [ -f containers/app.yml ] 242 | then 243 | app_name="app" 244 | web_file=containers/$app_name.yml 245 | log "Found $web_file" 246 | elif [ -f containers/web_only.yml ] 247 | then 248 | log "YML=web_only.yml" 249 | app_name="web_only" 250 | web_file=containers/$app_name.yml 251 | log "Found $web_file" 252 | else 253 | log "Can't find app.yml or web_only.yml." 254 | log "Giving up." 255 | exit 256 | fi 257 | } 258 | 259 | check_docker() { 260 | docker ps | tail -n +2 > /tmp/$UUID-docker.txt 261 | 262 | if grep $app_name /tmp/$UUID-docker.txt 263 | then 264 | log -e "\nDiscourse container $app_name is running" 265 | else 266 | log "==================== SERIOUS PROBLEM!!!! ====================" 267 | log "$app_name not running!" 268 | log "Attempting to rebuild" 269 | log "==================== REBUILD LOG ====================" 270 | # too hard to pass STDERR of ./launcher to log() 271 | ./launcher rebuild $app_name 2>&1 | tee -a $LOG_FILE 272 | log "==================== END REBUILD LOG ====================" 273 | docker ps | tail -n +2 > /tmp/$UUID-docker.txt 274 | if grep $app_name /tmp/$UUID-docker.txt 275 | then 276 | log -e "\nDiscourse container $app_name is now running." 277 | log ". . . waiting 30 seconds for container to crank up. . . " 278 | sleep 30 279 | else 280 | log "Failed to rebuild $app_name." 281 | # check_ip_match checks if curl to $DISCOURSE_HOSTNAME gets to this server 282 | # It works only if ports 80 and 443 are free 283 | check_ip_match $DISCOURSE_HOSTNAME 284 | log "You should probably remove any non-standard plugins and rebuild." 285 | NO_CONTAINER='y' 286 | log "Attempting to restart existing container. . . " 287 | ./launcher start $app_name 2>&1 | tee -a $LOG_FILE 288 | docker ps | tail -n +2 > /tmp/$UUID-docker.txt 289 | if grep $app_name /tmp/$UUID-docker.txt 290 | then 291 | log "Restarted the container." 292 | NO_CONTAINER='n' 293 | else 294 | log "Failed to restart the container." 295 | fi 296 | fi 297 | fi 298 | } 299 | 300 | ## 301 | ## redact passwords and email addresses from log file 302 | ## 303 | clean_up_log_file() { 304 | for VAR 305 | in SMTP_PASSWORD LETSENCRYPT_ACCOUNT_EMAIL DEVELOPER_EMAILS DISCOURSE_DB_PASSWORD 'Sending mail to' 306 | do 307 | echo "Replacing: $VAR" 308 | sed -i -e 's/'"$VAR"'\([=: ]\)\S*/'"$VAR"'\1REDACTED /g' $LOG_FILE 309 | done 310 | } 311 | 312 | print_done() { 313 | log 314 | log "==================== DONE! ====================" 315 | DOCTOR_FILE=$(date +%s | sha256sum | base64 | head -c 20).txt 316 | 317 | if [ $app_name == 'app' ] && [ "$NO_CONTAINER" != 'y' ] 318 | then 319 | read -p "Would you like to serve a publicly available version of this file? (Y/n)" serve 320 | if [ $serve == 'Y' ] 321 | then 322 | cp $LOG_FILE shared/standalone/log/var-log/$DOCTOR_FILE 323 | sudo docker exec -w /var/www/discourse -i $app_name cp /var/log/$DOCTOR_FILE public 324 | log "The output of this program may be available at http://$DISCOURSE_HOSTNAME/$DOCTOR_FILE" 325 | log "You should inspect that file carefully before sharing the URL." 326 | else 327 | log "Publicly available log not generated." 328 | fi 329 | fi 330 | # The following is not in the web log file since it was copied above, which seems corect 331 | log 332 | log "You can examine the output of this script with " 333 | log "LESS=-Ri less $LOG_FILE" 334 | log 335 | log "BUT FIRST, make sure that you know the first three commands below!!!" 336 | log 337 | log "Commands to know when viewing the file with the above command (called 'less'): " 338 | log "q -- quit" 339 | log "/error -- search for the word 'error'" 340 | log "n -- search for the next occurrence" 341 | log "g -- go to the beginning of the file" 342 | log "f -- go forward a page" 343 | log "b -- go back a page" 344 | log "G -- go to the end of the file" 345 | } 346 | 347 | initialize_log_file() { 348 | rm -f $LOG_FILE 349 | touch $LOG_FILE 350 | log DISCOURSE DOCTOR $(date) 351 | log -e "OS: $(uname -a)\n\n" 352 | } 353 | 354 | ## 355 | ## END FUNCTION DECLARATION 356 | ## 357 | 358 | check_root 359 | cd $WORKING_DIR || exit 360 | initialize_log_file 361 | get_yml_file 362 | get_discourse_config 363 | check_docker_is_installed 364 | check_docker 365 | check_plugins 366 | check_if_hostname_resolves_here 367 | check_disk_and_memory 368 | check_email 369 | clean_up_log_file 370 | print_done 371 | -------------------------------------------------------------------------------- /discourse-setup: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" 3 | cd $DIR 4 | 5 | ## 6 | ## Make sure only root can run our script 7 | ## 8 | check_root() { 9 | if [[ $EUID -ne 0 ]]; then 10 | echo "This script must be run as root. Please sudo or log in as root first." 1>&2 11 | exit 1 12 | fi 13 | } 14 | 15 | ## 16 | ## Check whether a connection to HOSTNAME ($1) on PORT ($2) is possible 17 | ## 18 | connect_to_port () { 19 | HOST="$1" 20 | PORT="$2" 21 | VERIFY=$(date +%s | sha256sum | base64 | head -c 20) 22 | if ! [ -x "$(command -v nc)" ]; then 23 | echo "In order to check the connection to $HOST:$PORT we need to open a socket using netcat." 24 | echo However netcat is not installed on your system. You can continue without this check 25 | echo or abort the setup, install netcat and try again. 26 | while true; do 27 | read -p "Would you like to continue without this check? [yn] " yn 28 | case $yn in 29 | [Yy]*) return 2 ;; 30 | [Nn]*) exit ;; 31 | *) echo "Please answer y or n." ;; 32 | esac 33 | done 34 | else 35 | echo -e "HTTP/1.1 200 OK\n\n $VERIFY" | nc -w 4 -l -p $PORT >/dev/null 2>&1 & 36 | if curl --proto =http -s $HOST:$PORT --connect-timeout 3 | grep $VERIFY >/dev/null 2>&1; then 37 | return 0 38 | else 39 | curl --proto =http -s localhost:$PORT >/dev/null 2>&1 40 | return 1 41 | fi 42 | fi 43 | } 44 | 45 | check_IP_match() { 46 | HOST="$1" 47 | echo 48 | echo Checking your domain name . . . 49 | connect_to_port $HOST 443; ec=$? 50 | case $ec in 51 | 0) 52 | echo "Connection to $HOST succeeded." 53 | ;; 54 | 1) 55 | echo "WARNING: Port 443 of computer does not appear to be accessible using hostname: $HOST." 56 | if connect_to_port $HOST 80; then 57 | echo 58 | echo SUCCESS: A connection to port 80 succeeds! 59 | echo This suggests that your DNS settings are correct, 60 | echo but something is keeping traffic to port 443 from getting to your server. 61 | echo Check your networking configuration to see that connections to port 443 are allowed. 62 | else 63 | echo "WARNING: Connection to http://$HOST (port 80) also fails." 64 | echo 65 | echo "This suggests that $HOST resolves to some IP address that does not reach this " 66 | echo machine where you are installing discourse. 67 | fi 68 | echo 69 | echo "The first thing to do is confirm that $HOST resolves to the IP address of this server." 70 | echo You usually do this at the same place you purchased the domain. 71 | echo 72 | echo If you are sure that the IP address resolves correctly, it could be a firewall issue. 73 | echo A web search for \"open ports YOUR CLOUD SERVICE\" might help. 74 | echo 75 | echo This tool is designed only for the most standard installations. If you cannot resolve 76 | echo the issue above, you will need to edit containers/app.yml yourself and then type 77 | echo 78 | echo ./launcher rebuild app 79 | echo 80 | exit 1 81 | ;; 82 | 2) 83 | echo "Continuing without port check." 84 | ;; 85 | esac 86 | } 87 | 88 | ## 89 | ## Do we have docker? 90 | ## 91 | check_and_install_docker () { 92 | docker_path=`which docker.io || which docker` 93 | if [ -z $docker_path ]; then 94 | read -p "Docker not installed. Enter to install from https://get.docker.com/ or Ctrl+C to exit" 95 | curl https://get.docker.com/ | sh 96 | fi 97 | docker_path=`which docker.io || which docker` 98 | if [ -z $docker_path ]; then 99 | echo Docker install failed. Quitting. 100 | exit 101 | fi 102 | } 103 | 104 | ## 105 | ## What are we running on 106 | ## 107 | check_OS() { 108 | echo `uname -s` 109 | } 110 | 111 | ## 112 | ## OS X available memory 113 | ## 114 | check_osx_memory() { 115 | echo `free -m | awk '/Mem:/ {print $2}'` 116 | } 117 | 118 | ## 119 | ## Linux available memory 120 | ## 121 | check_linux_memory() { 122 | ## some VMs report just under 1GB of RAM, so 123 | ## make an exception and allow those with more 124 | ## than 989MB 125 | mem=`free -m --si | awk ' /Mem:/ {print $2}'` 126 | if [ "$mem" -ge 990 -a "$mem" -lt 1000 ]; then 127 | echo 1 128 | else 129 | echo `free -g --si | awk ' /Mem:/ {print $2} '` 130 | fi 131 | } 132 | 133 | ## 134 | ## Do we have enough memory and disk space for Discourse? 135 | ## 136 | check_disk_and_memory() { 137 | 138 | os_type=$(check_OS) 139 | avail_mem=0 140 | if [ "$os_type" == "Darwin" ]; then 141 | avail_mem=$(check_osx_memory) 142 | else 143 | avail_mem=$(check_linux_memory) 144 | fi 145 | 146 | if [ "$avail_mem" -lt 1 ]; then 147 | echo "WARNING: Discourse requires 1GB RAM to run. This system does not appear" 148 | echo "to have sufficient memory." 149 | echo 150 | echo "Your site may not work properly, or future upgrades of Discourse may not" 151 | echo "complete successfully." 152 | exit 1 153 | fi 154 | 155 | if [ "$avail_mem" -le 2 ]; then 156 | total_swap=`free -g --si | awk ' /Swap:/ {print $2} '` 157 | 158 | if [ "$total_swap" -lt 2 ]; then 159 | echo "WARNING: Discourse requires at least 2GB of swap when running with 2GB of RAM" 160 | echo "or less. This system does not appear to have sufficient swap space." 161 | echo 162 | echo "Without sufficient swap space, your site may not work properly, and future" 163 | echo "upgrades of Discourse may not complete successfully." 164 | echo 165 | echo "Ctrl+C to exit or wait 5 seconds to have a 2GB swapfile created." 166 | sleep 5 167 | 168 | ## 169 | ## derived from https://meta.discourse.org/t/13880 170 | ## 171 | install -o root -g root -m 0600 /dev/null /swapfile 172 | fallocate -l 2G /swapfile 173 | mkswap /swapfile 174 | swapon /swapfile 175 | echo "/swapfile swap swap auto 0 0" | tee -a /etc/fstab 176 | sysctl -w vm.swappiness=10 177 | echo 'vm.swappiness = 10' > /etc/sysctl.d/30-discourse-swap.conf 178 | 179 | total_swap=`free -g --si | awk ' /Swap:/ {print $2} '` 180 | if [ "$total_swap" -lt 2 ]; then 181 | echo "Failed to create swap: are you root? Are you running on real hardware, or a fully virtualized server?" 182 | exit 1 183 | fi 184 | 185 | fi 186 | fi 187 | 188 | 189 | free_disk="$(df /var | tail -n 1 | awk '{print $4}')" 190 | if [ "$free_disk" -lt 5000 ]; then 191 | echo "WARNING: Discourse requires at least 5GB free disk space. This system" 192 | echo "does not appear to have sufficient disk space." 193 | echo 194 | echo "Insufficient disk space may result in problems running your site, and" 195 | echo "may not even allow Discourse installation to complete successfully." 196 | echo 197 | echo "Please free up some space, or expand your disk, before continuing." 198 | echo 199 | echo "Run \`apt-get autoremove && apt-get autoclean\` to clean up unused" 200 | echo "packages and \`./launcher cleanup\` to remove stale Docker containers." 201 | exit 1 202 | fi 203 | 204 | } 205 | 206 | 207 | ## 208 | ## If we have lots of RAM or lots of CPUs, bump up the defaults to scale better 209 | ## 210 | scale_ram_and_cpu() { 211 | 212 | local changelog=/tmp/changelog.$PPID 213 | # grab info about total system ram and physical (NOT LOGICAL!) CPU cores 214 | avail_gb=0 215 | avail_cores=0 216 | os_type=$(check_OS) 217 | if [ "$os_type" == "Darwin" ]; then 218 | avail_gb=$(check_osx_memory) 219 | avail_cores=`sysctl hw.ncpu | awk '/hw.ncpu:/ {print $2}'` 220 | else 221 | avail_gb=$(check_linux_memory) 222 | avail_cores=$((`awk '/cpu cores/ {print $4;exit}' /proc/cpuinfo`*`sort /proc/cpuinfo | uniq | grep -c "physical id"`)) 223 | fi 224 | echo "Found ${avail_gb}GB of memory and $avail_cores physical CPU cores" 225 | 226 | # db_shared_buffers: 128MB for 1GB, 256MB for 2GB, or 256MB * GB, max 4096MB 227 | if [ "$avail_gb" -eq "1" ] 228 | then 229 | db_shared_buffers=128 230 | else 231 | if [ "$avail_gb" -eq "2" ] 232 | then 233 | db_shared_buffers=256 234 | else 235 | db_shared_buffers=$(( 256 * $avail_gb )) 236 | fi 237 | fi 238 | db_shared_buffers=$(( db_shared_buffers < 4096 ? db_shared_buffers : 4096 )) 239 | 240 | sed -i -e "s/^ #\?db_shared_buffers:.*/ db_shared_buffers: \"${db_shared_buffers}MB\"/w $changelog" $data_file 241 | if [ -s $changelog ] 242 | then 243 | echo "setting db_shared_buffers = ${db_shared_buffers}MB" 244 | rm $changelog 245 | fi 246 | 247 | # UNICORN_WORKERS: 2 * GB for 2GB or less, or 2 * CPU, max 8 248 | if [ "$avail_gb" -le "2" ] 249 | then 250 | unicorn_workers=$(( 2 * $avail_gb )) 251 | else 252 | unicorn_workers=$(( 2 * $avail_cores )) 253 | fi 254 | unicorn_workers=$(( unicorn_workers < 8 ? unicorn_workers : 8 )) 255 | 256 | sed -i -e "s/^ #\?UNICORN_WORKERS:.*/ UNICORN_WORKERS: ${unicorn_workers}/w $changelog" $web_file 257 | if [ -s $changelog ] 258 | then 259 | echo "setting UNICORN_WORKERS = ${unicorn_workers}" 260 | rm $changelog 261 | fi 262 | 263 | echo $data_file memory parameters updated. 264 | } 265 | 266 | 267 | ## 268 | ## standard http / https ports must not be occupied 269 | ## 270 | check_ports() { 271 | check_port "80" 272 | check_port "443" 273 | echo "Ports 80 and 443 are free for use" 274 | } 275 | 276 | 277 | ## 278 | ## check a port to see if it is already in use 279 | ## 280 | check_port() { 281 | 282 | local valid=$(netstat -tln | awk '{print $4}' | grep ":${1}\$") 283 | 284 | if [ -n "$valid" ]; then 285 | echo "Port ${1} appears to already be in use." 286 | echo 287 | echo "This will show you what command is using port ${1}" 288 | lsof -i tcp:${1} -s tcp:listen 289 | echo 290 | echo "If you are trying to run Discourse simultaneously with another web" 291 | echo "server like Apache or nginx, you will need to bind to a different port" 292 | echo 293 | echo "See https://meta.discourse.org/t/17247" 294 | echo 295 | echo "If you are reconfiguring an already-configured Discourse, use " 296 | echo 297 | echo "./launcher stop app" 298 | echo 299 | echo "to stop Discourse before you reconfigure it and try again." 300 | exit 1 301 | fi 302 | } 303 | 304 | ## 305 | ## read a variable from the config file 306 | ## 307 | read_config() { 308 | config_line=`egrep "^ #?$1:" $web_file` 309 | read_config_result=`echo $config_line | awk -F":" '{print $2}'` 310 | read_config_result=`echo $read_config_result | sed "s/^\([\"']\)\(.*\)\1\$/\2/g"` 311 | } 312 | 313 | read_default() { 314 | config_line=`egrep "^ #?$1:" samples/standalone.yml` 315 | read_default_result=`echo $config_line | awk -F":" '{print $2}'` 316 | read_default_result=`echo $read_config_result | sed "s/^\([\"']\)\(.*\)\1\$/\2/g"` 317 | } 318 | 319 | ## 320 | ## prompt user for typical Discourse config file values 321 | ## 322 | ask_user_for_config() { 323 | 324 | # NOTE: Defaults now come from standalone.yml 325 | 326 | local changelog=/tmp/changelog.$PPID 327 | read_config "DISCOURSE_SMTP_ADDRESS" 328 | local smtp_address=$read_config_result 329 | # NOTE: if there are spaces between emails, this breaks, but a human should be paying attention 330 | read_config "DISCOURSE_DEVELOPER_EMAILS" 331 | local developer_emails=$read_config_result 332 | read_config "DISCOURSE_SMTP_PASSWORD" 333 | local smtp_password=$read_config_result 334 | read_config "DISCOURSE_SMTP_PORT" 335 | local smtp_port=$read_config_result 336 | read_config "DISCOURSE_SMTP_USER_NAME" 337 | local smtp_user_name=$read_config_result 338 | if [ "$smtp_password" = "pa$$word" ] 339 | then 340 | smtp_password = "" 341 | fi 342 | read_config "LETSENCRYPT_ACCOUNT_EMAIL" 343 | local letsencrypt_account_email=$read_config_result 344 | if [ -z $letsencrypt_account_email ] 345 | then 346 | letsencrypt_account_email="me@example.com" 347 | fi 348 | if [ "$letsencrypt_account_email" = "me@example.com" ] 349 | then 350 | local letsencrypt_status="ENTER to skip" 351 | else 352 | local letsencrypt_status="Enter 'OFF' to disable." 353 | fi 354 | 355 | read_config "DISCOURSE_HOSTNAME" 356 | hostname=$read_config_result 357 | 358 | local new_value="" 359 | local config_ok="n" 360 | local update_ok="y" 361 | 362 | echo "" 363 | 364 | while [[ "$config_ok" == "n" ]] 365 | do 366 | if [ ! -z "$hostname" ] 367 | then 368 | read -p "Hostname for your Discourse? [$hostname]: " new_value 369 | if [ ! -z "$new_value" ] 370 | then 371 | hostname="$new_value" 372 | fi 373 | if [[ $hostname =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]] 374 | then 375 | echo 376 | echo "Discourse requires a DNS hostname. IP addresses are unsupported and will not work." 377 | echo 378 | hostname="discourse.example.com" 379 | fi 380 | fi 381 | 382 | check_IP_match $hostname 383 | 384 | if [ ! -z "$developer_emails" ] 385 | then 386 | read -p "Email address for admin account(s)? [$developer_emails]: " new_value 387 | if [ ! -z "$new_value" ] 388 | then 389 | developer_emails="$new_value" 390 | fi 391 | fi 392 | 393 | if [ ! -z "$smtp_address" ] 394 | then 395 | read -p "SMTP server address? [$smtp_address]: " new_value 396 | if [ ! -z "$new_value" ] 397 | then 398 | smtp_address="$new_value" 399 | fi 400 | fi 401 | 402 | if [ ! -z "$smtp_port" ] 403 | then 404 | read -p "SMTP port? [$smtp_port]: " new_value 405 | if [ ! -z "$new_value" ] 406 | then 407 | smtp_port="$new_value" 408 | fi 409 | fi 410 | 411 | ## 412 | ## automatically set correct user name based on common mail providers unless it's been set 413 | ## 414 | if [ "$smtp_user_name" == "user@example.com" ] 415 | then 416 | if [ "$smtp_address" == "smtp.sparkpostmail.com" ] 417 | then 418 | smtp_user_name="SMTP_Injection" 419 | fi 420 | if [ "$smtp_address" == "smtp.sendgrid.net" ] 421 | then 422 | smtp_user_name="apikey" 423 | fi 424 | if [ "$smtp_address" == "smtp.mailgun.org" ] 425 | then 426 | smtp_user_name="postmaster@$hostname" 427 | fi 428 | fi 429 | 430 | if [ ! -z "$smtp_user_name" ] 431 | then 432 | read -p "SMTP user name? [$smtp_user_name]: " new_value 433 | if [ ! -z "$new_value" ] 434 | then 435 | smtp_user_name="$new_value" 436 | fi 437 | fi 438 | 439 | read -p "SMTP password? [$smtp_password]: " new_value 440 | if [ ! -z "$new_value" ] 441 | then 442 | smtp_password="$new_value" 443 | fi 444 | 445 | if [ ! -z $letsencrypt_account_email ] 446 | then 447 | read -p "Optional email address for Let's Encrypt warnings? ($letsencrypt_status) [$letsencrypt_account_email]: " new_value 448 | if [ ! -z "$new_value" ] 449 | then 450 | letsencrypt_account_email="$new_value" 451 | if [ "${new_value,,}" = "off" ] 452 | then 453 | letsencrypt_status="ENTER to skip" 454 | else 455 | letsencrypt_status="Enter 'OFF' to disable." 456 | fi 457 | fi 458 | fi 459 | 460 | echo -e "\nDoes this look right?\n" 461 | echo "Hostname : $hostname" 462 | echo "Email : $developer_emails" 463 | echo "SMTP address : $smtp_address" 464 | echo "SMTP port : $smtp_port" 465 | echo "SMTP username : $smtp_user_name" 466 | echo "SMTP password : $smtp_password" 467 | 468 | if [ "$letsencrypt_status" == "Enter 'OFF' to disable." ] 469 | then 470 | echo "Let's Encrypt : $letsencrypt_account_email" 471 | fi 472 | 473 | 474 | echo "" 475 | read -p "ENTER to continue, 'n' to try again, Ctrl+C to exit: " config_ok 476 | done 477 | 478 | sed -i -e "s/^ DISCOURSE_HOSTNAME:.*/ DISCOURSE_HOSTNAME: $hostname/w $changelog" $web_file 479 | if [ -s $changelog ] 480 | then 481 | rm $changelog 482 | else 483 | echo "DISCOURSE_HOSTNAME change failed." 484 | update_ok="n" 485 | fi 486 | 487 | sed -i -e "s/^ DISCOURSE_DEVELOPER_EMAILS:.*/ DISCOURSE_DEVELOPER_EMAILS: \'$developer_emails\'/w $changelog" $web_file 488 | if [ -s $changelog ] 489 | then 490 | rm $changelog 491 | else 492 | echo "DISCOURSE_DEVELOPER_EMAILS change failed." 493 | update_ok="n" 494 | fi 495 | 496 | sed -i -e "s/^ DISCOURSE_SMTP_ADDRESS:.*/ DISCOURSE_SMTP_ADDRESS: $smtp_address/w $changelog" $web_file 497 | if [ -s $changelog ] 498 | then 499 | rm $changelog 500 | else 501 | echo "DISCOURSE_SMTP_ADDRESS change failed." 502 | update_ok="n" 503 | fi 504 | 505 | sed -i -e "s/^ #\?DISCOURSE_SMTP_PORT:.*/ DISCOURSE_SMTP_PORT: $smtp_port/w $changelog" $web_file 506 | if [ -s $changelog ] 507 | then 508 | rm $changelog 509 | else 510 | echo "DISCOURSE_SMTP_PORT change failed." 511 | update_ok="n" 512 | fi 513 | 514 | sed -i -e "s/^ #\?DISCOURSE_SMTP_USER_NAME:.*/ DISCOURSE_SMTP_USER_NAME: $smtp_user_name/w $changelog" $web_file 515 | if [ -s $changelog ] 516 | then 517 | rm $changelog 518 | else 519 | echo "DISCOURSE_SMTP_USER_NAME change failed." 520 | update_ok="n" 521 | fi 522 | 523 | if [[ "$smtp_password" == *"\""* ]] 524 | then 525 | SLASH="BROKEN" 526 | echo "========================================" 527 | echo "WARNING!!!" 528 | echo "Your password contains a quote (\")" 529 | echo "Your SMTP Password will not be set. You will need to edit app.yml to enter it." 530 | echo "========================================" 531 | update_ok="n" 532 | else 533 | SLASH="|" 534 | if [[ "$smtp_password" == *"$SLASH"* ]] 535 | then SLASH="+" 536 | if [[ "$smtp_password" == *"$SLASH"* ]] 537 | then 538 | SLASH="Q" 539 | if [[ "$smtp_password" == *"$SLASH"* ]] 540 | then 541 | SLASH="BROKEN" 542 | echo "========================================" 543 | echo "WARNING!!!" 544 | echo "Your password contains all available delimiters (+, |, and Q). " 545 | echo "Your SMTP Password will not be set. You will need to edit app.yml to enter it." 546 | echo "========================================" 547 | update_ok="n" 548 | fi 549 | fi 550 | fi 551 | fi 552 | if [[ "$SLASH" != "BROKEN" ]] 553 | then 554 | sed -i -e "s${SLASH}^ #\?DISCOURSE_SMTP_PASSWORD:.*${SLASH} DISCOURSE_SMTP_PASSWORD: \"${smtp_password}\"${SLASH}w $changelog" $web_file 555 | 556 | if [ -s $changelog ] 557 | then 558 | rm $changelog 559 | else 560 | echo "DISCOURSE_SMTP_PASSWORD change failed." 561 | update_ok="n" 562 | fi 563 | fi 564 | 565 | echo "Enabling Let's Encrypt" 566 | sed -i -e "s/^ #\?LETSENCRYPT_ACCOUNT_EMAIL:.*/ LETSENCRYPT_ACCOUNT_EMAIL: $letsencrypt_account_email/w $changelog" $web_file 567 | if [ -s $changelog ] 568 | then 569 | rm $changelog 570 | else 571 | echo "LETSENCRYPT_ACCOUNT_EMAIL change failed." 572 | update_ok="n" 573 | fi 574 | local src='^ #\?- "templates\/web.ssl.template.yml"' 575 | local dst=' \- "templates\/web.ssl.template.yml"' 576 | sed -i -e "s/$src/$dst/w $changelog" $web_file 577 | if [ -s $changelog ] 578 | then 579 | echo "web.ssl.template.yml enabled" 580 | else 581 | update_ok="n" 582 | echo "web.ssl.template.yml NOT ENABLED--was it on already?" 583 | fi 584 | local src='^ #\?- "templates\/web.letsencrypt.ssl.template.yml"' 585 | local dst=' - "templates\/web.letsencrypt.ssl.template.yml"' 586 | 587 | sed -i -e "s/$src/$dst/w $changelog" $web_file 588 | if [ -s $changelog ] 589 | then 590 | echo "letsencrypt.ssl.template.yml enabled" 591 | else 592 | update_ok="n" 593 | echo "letsencrypt.ssl.template.yml NOT ENABLED -- was it on already?" 594 | fi 595 | 596 | if [ "$update_ok" == "y" ] 597 | then 598 | echo -e "\nConfiguration file at $config_file updated successfully!\n" 599 | else 600 | echo -e "\nUnfortunately, there was an error changing $config_file\n" 601 | echo -d "This may happen if you have made unexpected changes." 602 | exit 1 603 | fi 604 | } 605 | 606 | ## 607 | ## is our config file valid? Does it have the required fields set? 608 | ## 609 | validate_config() { 610 | 611 | valid_config="y" 612 | 613 | for x in DISCOURSE_SMTP_ADDRESS DISCOURSE_SMTP_USER_NAME DISCOURSE_SMTP_PASSWORD \ 614 | DISCOURSE_DEVELOPER_EMAILS DISCOURSE_HOSTNAME 615 | do 616 | read_config $x 617 | local result=$read_config_result 618 | read_default $x 619 | local default=$read_default_result 620 | 621 | if [ ! -z "$result" ] 622 | then 623 | if [[ "$config_line" = *"$default"* ]] 624 | then 625 | echo "$x left at incorrect default of $default" 626 | valid_config="n" 627 | fi 628 | config_val=`echo $config_line | awk '{print $2}'` 629 | if [ -z $config_val ] 630 | then 631 | echo "$x was not configured" 632 | valid_config="n" 633 | fi 634 | else 635 | echo "$x not present" 636 | valid_config="n" 637 | fi 638 | done 639 | 640 | if [ "$valid_config" != "y" ]; then 641 | echo -e "\nSorry, these $web_file settings aren't valid -- can't continue!" 642 | echo "If you have unusual requirements, edit $web_file and then: " 643 | echo "./launcher bootstrap $app_name" 644 | exit 1 645 | fi 646 | } 647 | 648 | 649 | ## 650 | ## template file names 651 | ## 652 | 653 | if [ "$1" == "2container" ] 654 | then 655 | app_name=web_only 656 | data_name=data 657 | web_template=samples/web_only.yml 658 | data_template=samples/data.yml 659 | web_file=containers/$app_name.yml 660 | data_file=containers/$data_name.yml 661 | else 662 | app_name=app 663 | data_name=app 664 | web_template=samples/standalone.yml 665 | data_template="" 666 | web_file=containers/$app_name.yml 667 | data_file=containers/$app_name.yml 668 | fi 669 | changelog=/tmp/changelog 670 | 671 | ## 672 | ## Check requirements before creating a copy of a config file we won't edit 673 | ## 674 | check_root 675 | check_and_install_docker 676 | check_disk_and_memory 677 | 678 | if [ -a "$web_file" ] 679 | then 680 | echo "The configuration file $web_file already exists!" 681 | echo 682 | echo ". . . reconfiguring . . ." 683 | echo 684 | echo 685 | DATE=`date +"%Y-%m-%d-%H%M%S"` 686 | BACKUP=$app_name.yml.$DATE.bak 687 | echo Saving old file as $BACKUP 688 | cp $web_file containers/$BACKUP 689 | echo "Stopping existing container in 5 seconds or Control-C to cancel." 690 | sleep 5 691 | ./launcher stop app 692 | echo 693 | else 694 | check_ports 695 | cp -v $web_template $web_file 696 | if [ "$data_name" == "data" ] 697 | then 698 | echo "--------------------------------------------------" 699 | echo "This two container setup is currently unsupported. Use at your own risk!" 700 | echo "--------------------------------------------------" 701 | DISCOURSE_DB_PASSWORD=`date +%s | sha256sum | base64 | head -c 20` 702 | 703 | sed -i -e "s/DISCOURSE_DB_PASSWORD: SOME_SECRET/DISCOURSE_DB_PASSWORD: $DISCOURSE_DB_PASSWORD/w $changelog" $web_file 704 | if [ -s $changelog ] 705 | then 706 | rm $changelog 707 | else 708 | echo "Problem changing DISCOURSE_DB_PASSWORD" in $web_file 709 | fi 710 | 711 | cp -v $data_template $data_file 712 | quote=\' 713 | sed -i -e "s/password ${quote}SOME_SECRET${quote}/password '$DISCOURSE_DB_PASSWORD'/w $changelog" $data_file 714 | if [ -s $changelog ] 715 | then 716 | rm $changelog 717 | else 718 | echo "Problem changing DISCOURSE_DB_PASSWORD" in $data_file 719 | fi 720 | fi 721 | fi 722 | 723 | scale_ram_and_cpu 724 | ask_user_for_config 725 | validate_config 726 | 727 | ## 728 | ## if we reach this point without exiting, OK to proceed 729 | ## rebuild won't fail if there's nothing to rebuild and does the restart 730 | ## 731 | echo "Updates successful. Rebuilding in 5 seconds." 732 | sleep 5 # Just a chance to ^C in case they were too fast on the draw 733 | if [ "$data_name" == "$app_name" ] 734 | then 735 | echo Building $app_name 736 | ./launcher rebuild $app_name 737 | else 738 | echo Building $data_name now . . . 739 | ./launcher rebuild $data_name 740 | echo Building $app_name now . . . 741 | ./launcher rebuild $app_name 742 | fi 743 | -------------------------------------------------------------------------------- /launcher: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | usage () { 4 | echo "Usage: launcher COMMAND CONFIG [--skip-prereqs] [--docker-args STRING]" 5 | echo "Commands:" 6 | echo " start: Start/initialize a container" 7 | echo " stop: Stop a running container" 8 | echo " restart: Restart a container" 9 | echo " destroy: Stop and remove a container" 10 | echo " enter: Open a shell to run commands inside the container" 11 | echo " logs: View the Docker logs for a container" 12 | echo " bootstrap: Bootstrap a container for the config based on a template" 13 | echo " run: Run the given command with the config in the context of the last bootstrapped image" 14 | echo " rebuild: Rebuild a container (destroy old, bootstrap, start new)" 15 | echo " cleanup: Remove all containers that have stopped for > 24 hours" 16 | echo " start-cmd: Generate docker command used to start container" 17 | echo 18 | echo "Options:" 19 | echo " --skip-prereqs Don't check launcher prerequisites" 20 | echo " --docker-args Extra arguments to pass when running docker" 21 | echo " --skip-mac-address Don't assign a mac address" 22 | echo " --run-image Override the image used for running the container" 23 | exit 1 24 | } 25 | 26 | # for potential re-exec later 27 | SAVED_ARGV=("$@") 28 | 29 | command=$1 30 | config=$2 31 | 32 | # user_args_argv is assigned once when the argument vector is parsed. 33 | user_args_argv="" 34 | # user_args is mutable: its value may change when templates are parsed. 35 | # Superset of user_args_argv. 36 | user_args="" 37 | 38 | user_run_image="" 39 | 40 | if [[ $command == "run" ]]; then 41 | run_command=$3 42 | fi 43 | 44 | while [ ${#} -gt 0 ]; do 45 | case "${1}" in 46 | --debug) 47 | DEBUG="1" 48 | ;; 49 | --skip-prereqs) 50 | SKIP_PREREQS="1" 51 | ;; 52 | --skip-mac-address) 53 | SKIP_MAC_ADDRESS="1" 54 | ;; 55 | --docker-args) 56 | user_args_argv="$2" 57 | user_args="$user_args_argv" 58 | shift 59 | ;; 60 | --run-image) 61 | user_run_image="$2" 62 | shift 63 | ;; 64 | esac 65 | 66 | shift 1 67 | done 68 | 69 | if [ -z "$command" -o -z "$config" -a "$command" != "cleanup" ]; then 70 | usage 71 | exit 1 72 | fi 73 | 74 | # Docker doesn't like uppercase characters, spaces or special characters, catch it now before we build everything out and then find out 75 | re='[[:upper:]/ !@#$%^&*()+~`=]' 76 | if [[ $config =~ $re ]]; 77 | then 78 | echo 79 | echo "ERROR: Config name '$config' must not contain upper case characters, spaces or special characters. Correct config name and rerun $0." 80 | echo 81 | exit 1 82 | fi 83 | 84 | cd "$(dirname "$0")" 85 | 86 | docker_min_version='17.03.1' 87 | docker_rec_version='17.06.2' 88 | git_min_version='1.8.0' 89 | git_rec_version='1.8.0' 90 | 91 | config_file=containers/"$config".yml 92 | cidbootstrap=cids/"$config"_bootstrap.cid 93 | local_discourse=local_discourse 94 | image="discourse/base:2.0.20200512-1735" 95 | docker_path=`which docker.io 2> /dev/null || which docker` 96 | git_path=`which git` 97 | 98 | if [ "${SUPERVISED}" = "true" ]; then 99 | restart_policy="--restart=no" 100 | attach_on_start="-a" 101 | attach_on_run="-a stdout -a stderr" 102 | else 103 | attach_on_run="-d" 104 | fi 105 | 106 | if [ -n "$DOCKER_HOST" ]; then 107 | docker_ip=`sed -e 's/^tcp:\/\/\(.*\):.*$/\1/' <<< "$DOCKER_HOST"` 108 | elif [ -x "$(which ip 2>/dev/null)" ]; then 109 | docker_ip=`ip addr show docker0 | \ 110 | grep 'inet ' | \ 111 | awk '{ split($2,a,"/"); print a[1] }';` 112 | else 113 | docker_ip=`ifconfig | \ 114 | grep -B1 "inet addr" | \ 115 | awk '{ if ( $1 == "inet" ) { print $2 } else if ( $2 == "Link" ) { printf "%s:" ,$1 } }' | \ 116 | grep docker0 | \ 117 | awk -F: '{ print $3 }';` 118 | fi 119 | 120 | # From https://stackoverflow.com/a/44660519/702738 121 | compare_version() { 122 | if [[ $1 == $2 ]]; then 123 | return 1 124 | fi 125 | local IFS=. 126 | local i a=(${1%%[^0-9.]*}) b=(${2%%[^0-9.]*}) 127 | local arem=${1#${1%%[^0-9.]*}} brem=${2#${2%%[^0-9.]*}} 128 | for ((i=0; i<${#a[@]} || i<${#b[@]}; i++)); do 129 | if ((10#${a[i]:-0} < 10#${b[i]:-0})); then 130 | return 1 131 | elif ((10#${a[i]:-0} > 10#${b[i]:-0})); then 132 | return 0 133 | fi 134 | done 135 | if [ "$arem" '<' "$brem" ]; then 136 | return 1 137 | elif [ "$arem" '>' "$brem" ]; then 138 | return 0 139 | fi 140 | return 1 141 | } 142 | 143 | 144 | install_docker() { 145 | echo "Docker is not installed, you will need to install Docker in order to run Launcher" 146 | echo "See https://docs.docker.com/installation/" 147 | exit 1 148 | } 149 | 150 | pull_image() { 151 | # Add a single retry to work around dockerhub TLS errors 152 | $docker_path pull $image || $docker_path pull $image 153 | } 154 | 155 | check_prereqs() { 156 | 157 | if [ -z $docker_path ]; then 158 | install_docker 159 | fi 160 | 161 | # 1. docker daemon running? 162 | # we send stderr to /dev/null cause we don't care about warnings, 163 | # it usually complains about swap which does not matter 164 | test=`$docker_path info 2> /dev/null` 165 | if [[ $? -ne 0 ]] ; then 166 | echo "Cannot connect to the docker daemon - verify it is running and you have access" 167 | exit 1 168 | fi 169 | 170 | # 2. running an approved storage driver? 171 | if ! $docker_path info 2> /dev/null | egrep -q 'Storage Driver: (aufs|zfs|overlay2)$'; then 172 | echo "Your Docker installation is not using a supported storage driver. If we were to proceed you may have a broken install." 173 | echo "aufs is the recommended storage driver, although zfs and overlay2 may work as well." 174 | echo "Other storage drivers are known to be problematic." 175 | echo "You can tell what filesystem you are using by running \"docker info\" and looking at the 'Storage Driver' line." 176 | echo 177 | echo "If you wish to continue anyway using your existing unsupported storage driver," 178 | echo "read the source code of launcher and figure out how to bypass this check." 179 | exit 1 180 | fi 181 | 182 | # 3. running recommended docker version 183 | test=($($docker_path --version)) # Get docker version string 184 | test=${test[2]//,/} # Get version alone and strip comma if exists 185 | 186 | # At least minimum docker version 187 | if compare_version "${docker_min_version}" "${test}"; then 188 | echo "ERROR: Docker version ${test} not supported, please upgrade to at least ${docker_min_version}, or recommended ${docker_rec_version}" 189 | exit 1 190 | fi 191 | 192 | # Recommend newer docker version 193 | if compare_version "${docker_rec_version}" "${test}"; then 194 | echo "WARNING: Docker version ${test} deprecated, recommend upgrade to ${docker_rec_version} or newer." 195 | fi 196 | 197 | # 4. discourse docker image is downloaded 198 | test=`$docker_path images | awk '{print $1 ":" $2 }' | grep "$image"` 199 | 200 | if [ -z "$test" ]; then 201 | echo 202 | echo "WARNING: We are about to start downloading the Discourse base image" 203 | echo "This process may take anywhere between a few minutes to an hour, depending on your network speed" 204 | echo 205 | echo "Please be patient" 206 | echo 207 | 208 | pull_image 209 | fi 210 | 211 | # 5. running recommended git version 212 | test=($($git_path --version)) # Get git version string 213 | test=${test[2]//,/} # Get version alone and strip comma if exists 214 | 215 | # At least minimum version 216 | if compare_version "${git_min_version}" "${test}"; then 217 | echo "ERROR: Git version ${test} not supported, please upgrade to at least ${git_min_version}, or recommended ${git_rec_version}" 218 | exit 1 219 | fi 220 | 221 | # Recommend best version 222 | if compare_version "${git_rec_version}" "${test}"; then 223 | echo "WARNING: Git version ${test} deprecated, recommend upgrade to ${git_rec_version} or newer." 224 | fi 225 | 226 | # 6. able to attach stderr / out / tty 227 | test=`$docker_path run $user_args -i --rm -a stdout -a stderr $image echo working` 228 | if [[ "$test" =~ "working" ]] ; then : ; else 229 | echo "Your Docker installation is not working correctly" 230 | echo 231 | echo "See: https://meta.discourse.org/t/docker-error-on-bootstrap/13657/18?u=sam" 232 | exit 1 233 | fi 234 | 235 | # 7. enough space for the bootstrap on docker folder 236 | folder=`$docker_path info --format '{{.DockerRootDir}}'` 237 | safe_folder=${folder:-/var/lib/docker} 238 | test=$(($(stat -f --format="%a*%S" $safe_folder)/1024**3 < 5)) 239 | if [[ $test -ne 0 ]] ; then 240 | echo "You have less than 5GB of free space on the disk where $safe_folder is located. You will need more space to continue" 241 | df -h $safe_folder 242 | echo 243 | if tty >/dev/null; then 244 | read -p "Would you like to attempt to recover space by cleaning docker images and containers in the system?(y/N)" -n 1 -r 245 | echo 246 | if [[ $REPLY =~ ^[Yy]$ ]] 247 | then 248 | $docker_path container prune --force --filter until=1h >/dev/null 249 | $docker_path image prune --all --force --filter until=1h >/dev/null 250 | echo "If the cleanup was successful, you may try again now" 251 | fi 252 | fi 253 | exit 1 254 | fi 255 | } 256 | 257 | 258 | if [ -z "$SKIP_PREREQS" ] && [ "$command" != "cleanup" ]; then 259 | check_prereqs 260 | fi 261 | 262 | host_run() { 263 | read -r -d '' env_ruby << 'RUBY' 264 | require 'yaml' 265 | 266 | input = STDIN.readlines.join 267 | yaml = YAML.load(input) 268 | 269 | if host_run = yaml['host_run'] 270 | params = yaml['params'] || {} 271 | host_run.each do |run| 272 | params.each do |k,v| 273 | run = run.gsub("$#{k}", v) 274 | end 275 | STDOUT.write "#{run}--SEP--" 276 | end 277 | end 278 | RUBY 279 | 280 | host_run=`cat $config_file | $docker_path run $user_args --rm -i -a stdout -a stdin $image ruby -e "$env_ruby"` 281 | 282 | while [ "$host_run" ] ; do 283 | iter=${host_run%%--SEP--*} 284 | echo 285 | echo "Host run: $iter" 286 | $iter || exit 1 287 | echo 288 | host_run="${host_run#*--SEP--}" 289 | done 290 | } 291 | 292 | 293 | set_volumes() { 294 | volumes=`cat $config_file | $docker_path run $user_args --rm -i -a stdout -a stdin $image ruby -e \ 295 | "require 'yaml'; puts YAML.load(STDIN.readlines.join)['volumes'].map{|v| '-v ' << v['volume']['host'] << ':' << v['volume']['guest'] << ' '}.join"` 296 | } 297 | 298 | set_links() { 299 | links=`cat $config_file | $docker_path run $user_args --rm -i -a stdout -a stdin $image ruby -e \ 300 | "require 'yaml'; puts YAML.load(STDIN.readlines.join)['links'].map{|l| '--link ' << l['link']['name'] << ':' << l['link']['alias'] << ' '}.join"` 301 | } 302 | 303 | find_templates() { 304 | local templates=`cat $1 | $docker_path run $user_args --rm -i -a stdin -a stdout $image ruby -e \ 305 | "require 'yaml'; puts YAML.load(STDIN.readlines.join)['templates']"` 306 | 307 | local arrTemplates=${templates// / } 308 | 309 | if [ ! -z "$templates" ]; then 310 | for template in "${arrTemplates[@]}" 311 | do 312 | local nested_templates=$(find_templates $template) 313 | 314 | if [ ! -z "$nested_templates" ]; then 315 | templates="$templates $nested_templates" 316 | fi 317 | done 318 | 319 | echo $templates 320 | else 321 | echo "" 322 | fi 323 | } 324 | 325 | set_template_info() { 326 | templates=$(find_templates $config_file) 327 | 328 | arrTemplates=(${templates// / }) 329 | config_data=$(cat $config_file) 330 | 331 | input="hack: true" 332 | 333 | for template in "${arrTemplates[@]}" 334 | do 335 | [ ! -z $template ] && { 336 | input="$input _FILE_SEPERATOR_ $(cat $template)" 337 | } 338 | done 339 | 340 | # we always want our config file last so it takes priority 341 | input="$input _FILE_SEPERATOR_ $config_data" 342 | 343 | read -r -d '' env_ruby << 'RUBY' 344 | require 'yaml' 345 | 346 | input=STDIN.readlines.join 347 | # default to UTF-8 for the dbs sake 348 | env = {'LANG' => 'en_US.UTF-8'} 349 | input.split('_FILE_SEPERATOR_').each do |yml| 350 | yml.strip! 351 | begin 352 | env.merge!(YAML.load(yml)['env'] || {}) 353 | rescue Psych::SyntaxError => e 354 | puts e 355 | puts "*ERROR." 356 | rescue => e 357 | puts yml 358 | p e 359 | end 360 | end 361 | puts env.map{|k,v| "-e\n#{k}=#{v}" }.join("\n") 362 | RUBY 363 | 364 | tmp_input_file=$(mktemp) 365 | 366 | echo "$input" > "$tmp_input_file" 367 | raw=`exec cat "$tmp_input_file" | $docker_path run $user_args --rm -i -a stdin -a stdout $image ruby -e "$env_ruby"` 368 | 369 | rm -f "$tmp_input_file" 370 | 371 | env=() 372 | ok=1 373 | while read i; do 374 | if [ "$i" == "*ERROR." ]; then 375 | ok=0 376 | elif [ -n "$i" ]; then 377 | env[${#env[@]}]="${i//\{\{config\}\}/${config}}" 378 | fi 379 | done <<< "$raw" 380 | 381 | if [ "$ok" -ne 1 ]; then 382 | echo "${env[@]}" 383 | echo "YAML syntax error. Please check your containers/*.yml config files." 384 | exit 1 385 | fi 386 | 387 | # labels 388 | read -r -d '' labels_ruby << 'RUBY' 389 | require 'yaml' 390 | 391 | input=STDIN.readlines.join 392 | labels = {} 393 | input.split('_FILE_SEPERATOR_').each do |yml| 394 | yml.strip! 395 | begin 396 | labels.merge!(YAML.load(yml)['labels'] || {}) 397 | rescue Psych::SyntaxError => e 398 | puts e 399 | puts "*ERROR." 400 | rescue => e 401 | puts yml 402 | p e 403 | end 404 | end 405 | puts labels.map{|k,v| "-l\n#{k}=#{v}" }.join("\n") 406 | RUBY 407 | 408 | tmp_input_file=$(mktemp) 409 | 410 | echo "$input" > "$tmp_input_file" 411 | raw=`exec cat "$tmp_input_file" | $docker_path run $user_args --rm -i -a stdin -a stdout $image ruby -e "$labels_ruby"` 412 | 413 | rm -f "$tmp_input_file" 414 | 415 | labels=() 416 | ok=1 417 | while read i; do 418 | if [ "$i" == "*ERROR." ]; then 419 | ok=0 420 | elif [ -n "$i" ]; then 421 | labels[${#labels[@]}]=$(echo $i | sed s/{{config}}/${config}/g) 422 | fi 423 | done <<< "$raw" 424 | 425 | if [ "$ok" -ne 1 ]; then 426 | echo "${labels[@]}" 427 | echo "YAML syntax error. Please check your containers/*.yml config files." 428 | exit 1 429 | fi 430 | 431 | # expose 432 | read -r -d '' ports_ruby << 'RUBY' 433 | require 'yaml' 434 | 435 | input=STDIN.readlines.join 436 | ports = [] 437 | input.split('_FILE_SEPERATOR_').each do |yml| 438 | yml.strip! 439 | begin 440 | ports += (YAML.load(yml)['expose'] || []) 441 | rescue Psych::SyntaxError => e 442 | puts e 443 | puts "*ERROR." 444 | rescue => e 445 | puts yml 446 | p e 447 | end 448 | end 449 | puts ports.map { |p| p.to_s.include?(':') ? "-p\n#{p}" : "--expose\n#{p}" }.join("\n") 450 | RUBY 451 | 452 | tmp_input_file=$(mktemp) 453 | 454 | echo "$input" > "$tmp_input_file" 455 | raw=`exec cat "$tmp_input_file" | $docker_path run $user_args --rm -i -a stdin -a stdout $image ruby -e "$ports_ruby"` 456 | 457 | rm -f "$tmp_input_file" 458 | 459 | ports=() 460 | ok=1 461 | while read i; do 462 | if [ "$i" == "*ERROR." ]; then 463 | ok=0 464 | elif [ -n "$i" ]; then 465 | ports[${#ports[@]}]=$i 466 | fi 467 | done <<< "$raw" 468 | 469 | if [ "$ok" -ne 1 ]; then 470 | echo "${ports[@]}" 471 | echo "YAML syntax error. Please check your containers/*.yml config files." 472 | exit 1 473 | fi 474 | 475 | merge_user_args 476 | } 477 | 478 | if [ -z $docker_path ]; then 479 | install_docker 480 | fi 481 | 482 | [ "$command" == "cleanup" ] && { 483 | $docker_path container prune --filter until=1h 484 | $docker_path image prune --all --filter until=1h 485 | 486 | if [ -d /var/discourse/shared/standalone/postgres_data_old ]; then 487 | echo 488 | echo "Old PostgreSQL backup data cluster detected taking up $(du -hs /var/discourse/shared/standalone/postgres_data_old | awk '{print $1}') detected" 489 | read -p "Would you like to remove it? (Y/n): " -n 1 -r && echo 490 | 491 | if [[ $REPLY =~ ^[Yy]$ ]]; then 492 | echo "removing old PostgreSQL data cluster at /var/discourse/shared/standalone/postgres_data_old..." 493 | rm -rf /var/discourse/shared/standalone/postgres_data_old* 494 | else 495 | exit 1 496 | fi 497 | fi 498 | 499 | exit 0 500 | } 501 | 502 | if [ ! "$command" == "setup" ]; then 503 | if [[ ! -e $config_file ]]; then 504 | echo "Config file was not found, ensure $config_file exists" 505 | echo 506 | echo "Available configs ( `cd containers && ls -dm *.yml | tr -s '\n' ' ' | awk '{ gsub(/\.yml/, ""); print }'`)" 507 | exit 1 508 | fi 509 | fi 510 | 511 | docker_version=($($docker_path --version)) 512 | docker_version=${test[2]//,/} 513 | restart_policy=${restart_policy:---restart=always} 514 | 515 | set_existing_container(){ 516 | existing=`$docker_path ps -a | awk '{ print $1, $(NF) }' | grep " $config$" | awk '{ print $1 }'` 517 | } 518 | 519 | run_stop() { 520 | 521 | set_existing_container 522 | 523 | if [ ! -z $existing ] 524 | then 525 | ( 526 | set -x 527 | $docker_path stop -t 10 $config 528 | ) 529 | else 530 | echo "$config was not started !" 531 | echo "./discourse-doctor may help diagnose the problem." 532 | exit 1 533 | fi 534 | } 535 | 536 | set_run_image() { 537 | run_image=`cat $config_file | $docker_path run $user_args --rm -i -a stdin -a stdout $image ruby -e \ 538 | "require 'yaml'; puts YAML.load(STDIN.readlines.join)['run_image']"` 539 | 540 | if [ -n "$user_run_image" ]; then 541 | run_image=$user_run_image 542 | elif [ -z "$run_image" ]; then 543 | run_image="$local_discourse/$config" 544 | fi 545 | } 546 | 547 | set_boot_command() { 548 | boot_command=`cat $config_file | $docker_path run $user_args --rm -i -a stdin -a stdout $image ruby -e \ 549 | "require 'yaml'; puts YAML.load(STDIN.readlines.join)['boot_command']"` 550 | 551 | if [ -z "$boot_command" ]; then 552 | 553 | no_boot_command=`cat $config_file | $docker_path run $user_args --rm -i -a stdin -a stdout $image ruby -e \ 554 | "require 'yaml'; puts YAML.load(STDIN.readlines.join)['no_boot_command']"` 555 | 556 | if [ -z "$no_boot_command" ]; then 557 | boot_command="/sbin/boot" 558 | fi 559 | fi 560 | } 561 | 562 | merge_user_args() { 563 | local docker_args 564 | 565 | docker_args=`cat $config_file | $docker_path run $user_args --rm -i -a stdout -a stdin $image ruby -e \ 566 | "require 'yaml'; puts YAML.load(STDIN.readlines.join)['docker_args']"` 567 | 568 | if [[ -n "$docker_args" ]]; then 569 | user_args="$user_args_argv $docker_args" 570 | fi 571 | } 572 | 573 | run_start() { 574 | 575 | if [ -z "$START_CMD_ONLY" ] 576 | then 577 | existing=`$docker_path ps | awk '{ print $1, $(NF) }' | grep " $config$" | awk '{ print $1 }'` 578 | echo $existing 579 | if [ ! -z $existing ] 580 | then 581 | echo "Nothing to do, your container has already started!" 582 | exit 0 583 | fi 584 | 585 | existing=`$docker_path ps -a | awk '{ print $1, $(NF) }' | grep " $config$" | awk '{ print $1 }'` 586 | if [ ! -z $existing ] 587 | then 588 | echo "starting up existing container" 589 | ( 590 | set -x 591 | $docker_path start $config 592 | ) 593 | exit 0 594 | fi 595 | fi 596 | 597 | host_run 598 | 599 | set_template_info 600 | set_volumes 601 | set_links 602 | set_run_image 603 | set_boot_command 604 | 605 | # get hostname and settings from container configuration 606 | for envar in "${env[@]}" 607 | do 608 | if [[ $envar == DOCKER_USE_HOSTNAME* ]] || [[ $envar == DISCOURSE_HOSTNAME* ]] 609 | then 610 | # use as environment variable 611 | eval $envar 612 | fi 613 | done 614 | 615 | ( 616 | hostname=`hostname -s` 617 | # overwrite hostname 618 | if [ "$DOCKER_USE_HOSTNAME" = "true" ] 619 | then 620 | hostname=$DISCOURSE_HOSTNAME 621 | else 622 | hostname=$hostname-$config 623 | fi 624 | 625 | # we got to normalize so we only have allowed strings, this is more comprehensive but lets see how bash does first 626 | # hostname=`$docker_path run $user_args --rm $image ruby -e 'print ARGV[0].gsub(/[^a-zA-Z-]/, "-")' $hostname` 627 | # docker added more hostname rules 628 | hostname=${hostname//_/-} 629 | 630 | 631 | if [ -z "$SKIP_MAC_ADDRESS" ] ; then 632 | mac_address="--mac-address $($docker_path run $user_args -i --rm -a stdout -a stderr $image /bin/sh -c "echo $hostname | md5sum | sed 's/^\(..\)\(..\)\(..\)\(..\)\(..\).*$/02:\1:\2:\3:\4:\5/'")" 633 | fi 634 | 635 | if [ ! -z "$START_CMD_ONLY" ] ; then 636 | docker_path="true" 637 | fi 638 | 639 | set -x 640 | 641 | $docker_path run --shm-size=512m $links $attach_on_run $restart_policy "${env[@]}" "${labels[@]}" -h "$hostname" \ 642 | -e DOCKER_HOST_IP="$docker_ip" --name $config -t "${ports[@]}" $volumes $mac_address $user_args \ 643 | $run_image $boot_command 644 | 645 | ) 646 | exit 0 647 | 648 | } 649 | 650 | run_run() { 651 | set_template_info 652 | set_volumes 653 | set_links 654 | set_run_image 655 | 656 | unset ERR 657 | (exec $docker_path run --rm --shm-size=512m $user_args $links "${env[@]}" -e DOCKER_HOST_IP="$docker_ip" -i -a stdin -a stdout -a stderr $volumes $run_image \ 658 | /bin/bash -c "$run_command") || ERR=$? 659 | 660 | if [[ $ERR > 0 ]]; then 661 | exit 1 662 | fi 663 | } 664 | 665 | run_bootstrap() { 666 | host_run 667 | 668 | # Is the image available? 669 | # If not, pull it here so the user is aware what's happening. 670 | $docker_path history $image >/dev/null 2>&1 || pull_image 671 | 672 | set_template_info 673 | 674 | base_image=`cat $config_file | $docker_path run $user_args --rm -i -a stdin -a stdout $image ruby -e \ 675 | "require 'yaml'; puts YAML.load(STDIN.readlines.join)['base_image']"` 676 | 677 | update_pups=`cat $config_file | $docker_path run $user_args --rm -i -a stdin -a stdout $image ruby -e \ 678 | "require 'yaml'; puts YAML.load(STDIN.readlines.join)['update_pups']"` 679 | 680 | if [[ ! X"" = X"$base_image" ]]; then 681 | image=$base_image 682 | fi 683 | 684 | set_volumes 685 | set_links 686 | 687 | rm -f $cidbootstrap 688 | 689 | run_command="cd /pups &&" 690 | if [[ ! "false" = $update_pups ]]; then 691 | run_command="$run_command git pull &&" 692 | fi 693 | run_command="$run_command /pups/bin/pups --stdin" 694 | 695 | echo $run_command 696 | 697 | unset ERR 698 | 699 | tmp_input_file=$(mktemp) 700 | 701 | echo "$input" > "$tmp_input_file" 702 | (exec cat "$tmp_input_file" | $docker_path run --shm-size=512m $user_args $links "${env[@]}" -e DOCKER_HOST_IP="$docker_ip" --cidfile $cidbootstrap -i -a stdin -a stdout -a stderr $volumes $image \ 703 | /bin/bash -c "$run_command") || ERR=$? 704 | 705 | rm -f "$tmp_input_file" 706 | 707 | unset FAILED 708 | # magic exit code that indicates a retry 709 | if [[ "$ERR" == 77 ]]; then 710 | $docker_path rm `cat $cidbootstrap` 711 | rm $cidbootstrap 712 | exit 77 713 | elif [[ "$ERR" > 0 ]]; then 714 | FAILED=TRUE 715 | fi 716 | 717 | if [[ $FAILED = "TRUE" ]]; then 718 | if [[ ! -z "$DEBUG" ]]; then 719 | $docker_path commit `cat $cidbootstrap` $local_discourse/$config-debug || echo 'FAILED TO COMMIT' 720 | echo "** DEBUG ** Maintaining image for diagnostics $local_discourse/$config-debug" 721 | fi 722 | 723 | $docker_path rm `cat $cidbootstrap` 724 | rm $cidbootstrap 725 | echo "** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one." 726 | echo "./discourse-doctor may help diagnose the problem." 727 | exit 1 728 | fi 729 | 730 | sleep 5 731 | 732 | $docker_path commit `cat $cidbootstrap` $local_discourse/$config || echo 'FAILED TO COMMIT' 733 | $docker_path rm `cat $cidbootstrap` && rm $cidbootstrap 734 | } 735 | 736 | case "$command" in 737 | bootstrap) 738 | run_bootstrap 739 | echo "Successfully bootstrapped, to startup use ./launcher start $config" 740 | exit 0 741 | ;; 742 | 743 | run) 744 | run_run 745 | exit 0 746 | ;; 747 | 748 | enter) 749 | exec $docker_path exec -it $config /bin/bash --login 750 | ;; 751 | 752 | stop) 753 | run_stop 754 | exit 0 755 | ;; 756 | 757 | logs) 758 | 759 | $docker_path logs $config 760 | exit 0 761 | ;; 762 | 763 | restart) 764 | run_stop 765 | run_start 766 | exit 0 767 | ;; 768 | 769 | start-cmd) 770 | START_CMD_ONLY="1" 771 | run_start 772 | exit 0; 773 | ;; 774 | 775 | start) 776 | run_start 777 | exit 0 778 | ;; 779 | 780 | rebuild) 781 | if [ "$(git symbolic-ref --short HEAD)" == "master" ]; then 782 | echo "Ensuring launcher is up to date" 783 | 784 | git remote update 785 | 786 | LOCAL=$(git rev-parse HEAD) 787 | REMOTE=$(git rev-parse @{u}) 788 | BASE=$(git merge-base HEAD @{u}) 789 | 790 | if [ $LOCAL = $REMOTE ]; then 791 | echo "Launcher is up-to-date" 792 | 793 | elif [ $LOCAL = $BASE ]; then 794 | echo "Updating Launcher..." 795 | git pull || (echo 'failed to update' && exit 1) 796 | 797 | echo "Launcher updated, restarting..." 798 | exec "$0" "${SAVED_ARGV[@]}" 799 | 800 | elif [ $REMOTE = $BASE ]; then 801 | echo "Your version of Launcher is ahead of origin" 802 | 803 | else 804 | echo "Launcher has diverged source, this is only expected in Dev mode" 805 | fi 806 | 807 | fi 808 | 809 | set_existing_container 810 | 811 | if [ ! -z $existing ] 812 | then 813 | echo "Stopping old container" 814 | ( 815 | set -x 816 | $docker_path stop -t 60 $config 817 | ) 818 | fi 819 | 820 | run_bootstrap 821 | 822 | if [ ! -z $existing ] 823 | then 824 | echo "Removing old container" 825 | ( 826 | set -x 827 | $docker_path rm $config 828 | ) 829 | fi 830 | 831 | run_start 832 | exit 0 833 | ;; 834 | 835 | 836 | destroy) 837 | (set -x; $docker_path stop -t 10 $config && $docker_path rm $config) || (echo "$config was not found" && exit 0) 838 | exit 0 839 | ;; 840 | esac 841 | 842 | usage 843 | --------------------------------------------------------------------------------