├── .gitignore ├── README.md ├── Vagrantfile └── provisioning ├── files ├── apt_periodic ├── graphite.conf ├── local_settings.py ├── sample_cron.sh ├── selenium-start.sh ├── selenium.conf └── sitespeed.conf ├── grafana.sh ├── graphite.sh ├── security.sh └── sitespeed.sh /.gitignore: -------------------------------------------------------------------------------- 1 | .vagrant 2 | .idea 3 | 4 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | Devstack Performance 2 | ==================== 3 | 4 | `devstack-performance` is a Vagrant provisioning script that sets up a full performance monitoring stack on your local environment. 5 | 6 | Installation 7 | ------------ 8 | Here's the steps and results you'll immediately see. It's possible the instructions below become outdated as we continue to update the project. The most accurate documentation will always live on GitHub. 9 | 10 | ### 1) Install Vagrant 11 | While this can generally be a very quick process, each local environment is different and the best docs are on the vagrant site: http://docs.vagrantup.com/v2/installation/. 12 | 13 | ### 2) Install Vagrant Host Manager 14 | We use this simply because it adds entries for graphite, grafana and sitespeed to point to your VM's IP address automatically when you provision the VM. 15 | https://github.com/smdahlen/vagrant-hostmanager 16 | 17 | ### 3) Clone Devstack Performance Repository 18 | This contains everything needed to spin up an instance on your local machine, it currently requires 2 Gb of memory and takes about 7-10 minutes to provision the VM the first time. 19 | ``` 20 | $ git clone https://github.com/ndevrinc/devstack-performance 21 | $ cd devstack-performance 22 | $ vagrant up 23 | ``` 24 | 25 | ***And now you wait...*** Once the provisioning scripts complete fully it will begin to record metrics for http://www.sitespeed.io every 5 minutes. If you'd like feel free to edit the provision/files/sample_cron.sh file before you vagrant up, or simply edit it on the box afterwards. Assuming your hosts entries were updated properly you should be able to pull up these pages in a browser: 26 | 27 | #### Sitespeed Results 28 | http://sitespeed (simply shows browse-able doc structure to display the sitespeed.io HTML files) 29 | ![Browse Sitespeed.io directories](https://ndevr.io/wp-content/uploads/2015/08/sitespeed-directory-browse-e1439302456917.png) 30 | 31 | #### Graphite Results 32 | http://graphite (gives you access to your Graphite data) 33 | ![Sample Graphite Graph of Ndevr Performance Data](https://ndevr.io/wp-content/uploads/2015/08/graphite-sample-graph.png) 34 | 35 | #### Graphana Results 36 | http://grafana:3000 (gives you access to your Grafana dashboard admin) 37 | ![Comparing Automattic, Acquia and Laravel sites](https://ndevr.io/wp-content/uploads/2015/08/graphana-opensource.png) 38 | Comparing Automattic, Acquia and Laravel sites 39 | 40 | ![Our pages are quite a bit faster...](https://ndevr.io/wp-content/uploads/2015/08/graphana-ndevr.png) 41 | Our pages are quite a bit faster... 42 | 43 | ### 4) Edit the sample_cron.sh 44 | We've provided a few extra commented out lines in the script for use as examples, but here are the documented sitespeed.io configuration options. We use this file to point sitespeed.io to our local environments while sending the Graphite data to our hosted instance so we can share/compare our data better. 45 | ### 5) Oops 46 | That's fine, that's why we are using Vagrant. If you find yourself making to mistakes to back out from simply destroy the VM and start again. You've likely made some customizations to the sample_cron.sh file, so keep those handy when you start over. 47 | -------------------------------------------------------------------------------- /Vagrantfile: -------------------------------------------------------------------------------- 1 | # encoding: utf-8 2 | # -*- mode: ruby -*- 3 | # vi: set ft=ruby : 4 | 5 | # Vagrantfile API/syntax version. Don't touch unless you know what you're doing! 6 | VAGRANTFILE_API_VERSION = "2" 7 | 8 | 9 | # PROJECT VARIABLES 10 | project_name = "performance" 11 | ip_address = "192.168.100.108" 12 | 13 | Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| 14 | config.vm.box = "ubuntu/trusty64" 15 | 16 | # Private networking, automatically adds entry to host's /etc/hosts 17 | config.hostmanager.enabled = true 18 | config.hostmanager.manage_host = true 19 | config.vm.define project_name do |node| 20 | node.vm.hostname = project_name 21 | node.vm.network :private_network, ip: ip_address 22 | node.hostmanager.aliases = %w(graphite sitespeed grafana) 23 | end 24 | config.vm.provision :hostmanager 25 | 26 | # Map this entire repo to /var/www/[project_name]/ 27 | # Uses NFS so it modifies your /etc/exports 28 | config.vm.synced_folder "./provisioning/files", "/home/vagrant/provisioning-files/", type: "nfs" 29 | 30 | 31 | config.vm.provider "virtualbox" do |vb| 32 | vb.customize ["modifyvm", :id, "--memory", "2048"] 33 | end 34 | 35 | # This allows the git commands to work using host server keys 36 | config.ssh.forward_agent = true 37 | 38 | # To avoid stdin/tty issues 39 | config.vm.provision "fix-no-tty", type: "shell" do |s| 40 | s.privileged = false 41 | s.inline = "sudo sed -i '/tty/!s/mesg n/tty -s \\&\\& mesg n/' /root/.profile" 42 | end 43 | 44 | config.vm.provision "shell" do |s| 45 | s.path = "provisioning/sitespeed.sh" 46 | s.keep_color = true 47 | end 48 | config.vm.provision "shell" do |s| 49 | s.path = "provisioning/graphite.sh" 50 | s.keep_color = true 51 | end 52 | config.vm.provision "shell" do |s| 53 | s.path = "provisioning/grafana.sh" 54 | s.keep_color = true 55 | end 56 | config.vm.provision "shell" do |s| 57 | s.path = "provisioning/security.sh" 58 | s.keep_color = true 59 | end 60 | 61 | if Vagrant.has_plugin?("vagrant-cachier") 62 | # Configure cached packages to be shared between instances of the same base box. 63 | # More info on http://fgrehm.viewdocs.io/vagrant-cachier/usage 64 | config.cache.scope = :box 65 | end 66 | end 67 | -------------------------------------------------------------------------------- /provisioning/files/apt_periodic: -------------------------------------------------------------------------------- 1 | APT::Periodic::Update-Package-Lists "1"; 2 | APT::Periodic::Download-Upgradeable-Packages "1"; 3 | APT::Periodic::AutocleanInterval "7"; 4 | APT::Periodic::Unattended-Upgrade "1"; -------------------------------------------------------------------------------- /provisioning/files/graphite.conf: -------------------------------------------------------------------------------- 1 | # This needs to be in your server's config somewhere, probably 2 | # the main httpd.conf 3 | # NameVirtualHost *:80 4 | 5 | 6 | WSGISocketPrefix /var/run/apache2/wsgi 7 | 8 | 9 | ServerName graphite 10 | DocumentRoot "/opt/graphite/webapp" 11 | ErrorLog /var/log/apache2/error.log 12 | CustomLog /var/log/apache2/access.log common 13 | 14 | # I've found that an equal number of processes & threads tends 15 | # to show the best performance for Graphite (ymmv). 16 | WSGIDaemonProcess graphite processes=5 threads=5 display-name='%{GROUP}' inactivity-timeout=120 17 | WSGIProcessGroup graphite 18 | WSGIApplicationGroup %{GLOBAL} 19 | WSGIImportScript /opt/graphite/conf/graphite.wsgi process-group=graphite application-group=%{GLOBAL} 20 | 21 | # file in this directory that you can safely use, just copy it to graphite.wgsi 22 | WSGIScriptAlias / /opt/graphite/conf/graphite.wsgi 23 | 24 | Alias /static "/opt/graphite/webapp/content" 25 | 26 | Order deny,allow 27 | Allow from all 28 | Require all granted 29 | 30 | 31 | Alias /content/ /opt/graphite/webapp/content/ 32 | 33 | SetHandler None 34 | 35 | 36 | # XXX In order for the django admin site media to work you 37 | # must change @DJANGO_ROOT@ to be the path to your django 38 | # installation, which is probably something like: 39 | # /usr/lib/python2.6/site-packages/django 40 | Alias /media/ "/usr/local/lib/python2.7/dist-packages/django/contrib/admin/media/" 41 | 42 | SetHandler None 43 | 44 | 45 | # The graphite.wsgi file has to be accessible by apache. It won't 46 | # be visible to clients because of the DocumentRoot though. 47 | 48 | Order deny,allow 49 | Allow from all 50 | Require all granted 51 | 52 | 53 | 54 | 55 | -------------------------------------------------------------------------------- /provisioning/files/local_settings.py: -------------------------------------------------------------------------------- 1 | """Copyright 2008 Orbitz WorldWide 2 | 3 | Licensed under the Apache License, Version 2.0 (the "License"); 4 | you may not use this file except in compliance with the License. 5 | You may obtain a copy of the License at 6 | 7 | http://www.apache.org/licenses/LICENSE-2.0 8 | 9 | Unless required by applicable law or agreed to in writing, software 10 | distributed under the License is distributed on an "AS IS" BASIS, 11 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 12 | See the License for the specific language governing permissions and 13 | limitations under the License.""" 14 | # Django settings for graphite project. 15 | # DO NOT MODIFY THIS FILE DIRECTLY - use local_settings.py instead 16 | import os 17 | import sys 18 | from os.path import abspath, dirname, join 19 | from warnings import warn 20 | 21 | from django.core.urlresolvers import reverse_lazy 22 | 23 | 24 | GRAPHITE_WEB_APP_SETTINGS_LOADED = False 25 | WEBAPP_VERSION = '0.10.0-alpha' 26 | DEBUG = False 27 | JAVASCRIPT_DEBUG = False 28 | 29 | # Filesystem layout 30 | WEB_DIR = dirname( abspath(__file__) ) 31 | WEBAPP_DIR = dirname(WEB_DIR) 32 | GRAPHITE_ROOT = dirname(WEBAPP_DIR) 33 | # Initialize additional path variables 34 | # Defaults for these are set after local_settings is imported 35 | STATIC_ROOT = '' 36 | STATIC_URL = '/static/' 37 | URL_PREFIX = '' 38 | CONF_DIR = '' 39 | DASHBOARD_CONF = '' 40 | GRAPHTEMPLATES_CONF = '' 41 | STORAGE_DIR = '' 42 | WHITELIST_FILE = '' 43 | INDEX_FILE = '' 44 | LOG_DIR = '' 45 | CERES_DIR = '' 46 | WHISPER_DIR = '' 47 | RRD_DIR = '' 48 | STANDARD_DIRS = [] 49 | 50 | CLUSTER_SERVERS = [] 51 | 52 | # Cluster settings 53 | CLUSTER_SERVERS = [] 54 | REMOTE_FIND_TIMEOUT = 3.0 55 | REMOTE_FETCH_TIMEOUT = 6.0 56 | REMOTE_RETRY_DELAY = 60.0 57 | REMOTE_EXCLUDE_LOCAL = False 58 | REMOTE_READER_CACHE_SIZE_LIMIT = 1000 59 | CARBON_METRIC_PREFIX='carbon' 60 | CARBONLINK_HOSTS = ["127.0.0.1:7002"] 61 | CARBONLINK_TIMEOUT = 1.0 62 | CARBONLINK_HASHING_KEYFUNC = None 63 | CARBONLINK_RETRY_DELAY = 15 64 | REPLICATION_FACTOR = 1 65 | MEMCACHE_HOSTS = [] 66 | MEMCACHE_KEY_PREFIX = '' 67 | FIND_CACHE_DURATION = 300 68 | FIND_TOLERANCE = 2 * FIND_CACHE_DURATION 69 | DEFAULT_CACHE_DURATION = 60 #metric data and graphs are cached for one minute by default 70 | LOG_CACHE_PERFORMANCE = False 71 | LOG_ROTATE = True 72 | MAX_FETCH_RETRIES = 2 73 | 74 | #Remote rendering settings 75 | REMOTE_RENDERING = False #if True, rendering is delegated to RENDERING_HOSTS 76 | RENDERING_HOSTS = [] 77 | REMOTE_RENDER_CONNECT_TIMEOUT = 1.0 78 | LOG_RENDERING_PERFORMANCE = False 79 | 80 | #Miscellaneous settings 81 | SMTP_SERVER = "localhost" 82 | DOCUMENTATION_URL = "http://graphite.readthedocs.org/" 83 | ALLOW_ANONYMOUS_CLI = True 84 | LOG_METRIC_ACCESS = False 85 | LEGEND_MAX_ITEMS = 10 86 | RRD_CF = 'AVERAGE' 87 | STORAGE_FINDERS = ( 88 | 'graphite.finders.standard.StandardFinder', 89 | ) 90 | 91 | #Authentication settings 92 | USE_LDAP_AUTH = False 93 | LDAP_SERVER = "" # "ldapserver.mydomain.com" 94 | LDAP_PORT = 389 95 | LDAP_USE_TLS = False 96 | LDAP_SEARCH_BASE = "" # "OU=users,DC=mydomain,DC=com" 97 | LDAP_BASE_USER = "" # "CN=some_readonly_account,DC=mydomain,DC=com" 98 | LDAP_BASE_PASS = "" # "my_password" 99 | LDAP_USER_QUERY = "" # "(username=%s)" For Active Directory use "(sAMAccountName=%s)" 100 | LDAP_URI = None 101 | 102 | #Set this to True to delegate authentication to the web server 103 | USE_REMOTE_USER_AUTHENTICATION = False 104 | REMOTE_USER_BACKEND = "" # Provide an alternate or subclassed backend 105 | 106 | # Django 1.5 requires this so we set a default but warn the user 107 | SECRET_KEY = 'BARELY_SAFE_NONDEFAULT' 108 | 109 | # Django 1.5 requires this to be set. Here we default to prior behavior and allow all 110 | ALLOWED_HOSTS = [ '*' ] 111 | 112 | # Override to link a different URL for login (e.g. for django_openid_auth) 113 | LOGIN_URL = reverse_lazy('account_login') 114 | 115 | # Set the default timezone to UTC 116 | TIME_ZONE = 'UTC' 117 | 118 | # Set to True to require authentication to save or delete dashboards 119 | DASHBOARD_REQUIRE_AUTHENTICATION = False 120 | # Require Django change/delete permissions to save or delete dashboards. 121 | # NOTE: Requires DASHBOARD_REQUIRE_AUTHENTICATION to be set 122 | DASHBOARD_REQUIRE_PERMISSIONS = False 123 | # Name of a group to which the user must belong to save or delete dashboards. Alternative to 124 | # DASHBOARD_REQUIRE_PERMISSIONS, particularly useful when using only LDAP (without Admin app) 125 | # NOTE: Requires DASHBOARD_REQUIRE_AUTHENTICATION to be set 126 | DASHBOARD_REQUIRE_EDIT_GROUP = None 127 | 128 | DATABASES = None 129 | 130 | # If using rrdcached, set to the address or socket of the daemon 131 | FLUSHRRDCACHED = '' 132 | 133 | ## Load our local_settings 134 | try: 135 | from graphite.local_settings import * # noqa 136 | except ImportError: 137 | print >> sys.stderr, "Could not import graphite.local_settings, using defaults!" 138 | 139 | ## Load Django settings if they werent picked up in local_settings 140 | if not GRAPHITE_WEB_APP_SETTINGS_LOADED: 141 | from graphite.app_settings import * # noqa 142 | 143 | STATICFILES_DIRS = ( 144 | join(WEBAPP_DIR, 'content'), 145 | ) 146 | 147 | ## Set config dependent on flags set in local_settings 148 | # Path configuration 149 | if not STATIC_ROOT: 150 | STATIC_ROOT = join(GRAPHITE_ROOT, 'static') 151 | 152 | if not CONF_DIR: 153 | CONF_DIR = os.environ.get('GRAPHITE_CONF_DIR', join(GRAPHITE_ROOT, 'conf')) 154 | if not DASHBOARD_CONF: 155 | DASHBOARD_CONF = join(CONF_DIR, 'dashboard.conf') 156 | if not GRAPHTEMPLATES_CONF: 157 | GRAPHTEMPLATES_CONF = join(CONF_DIR, 'graphTemplates.conf') 158 | 159 | if not STORAGE_DIR: 160 | STORAGE_DIR = os.environ.get('GRAPHITE_STORAGE_DIR', join(GRAPHITE_ROOT, 'storage')) 161 | if not WHITELIST_FILE: 162 | WHITELIST_FILE = join(STORAGE_DIR, 'lists', 'whitelist') 163 | if not INDEX_FILE: 164 | INDEX_FILE = join(STORAGE_DIR, 'index') 165 | if not LOG_DIR: 166 | LOG_DIR = join(STORAGE_DIR, 'log', 'webapp') 167 | if not WHISPER_DIR: 168 | WHISPER_DIR = join(STORAGE_DIR, 'whisper/') 169 | if not CERES_DIR: 170 | CERES_DIR = join(STORAGE_DIR, 'ceres/') 171 | if not RRD_DIR: 172 | RRD_DIR = join(STORAGE_DIR, 'rrd/') 173 | if not STANDARD_DIRS: 174 | try: 175 | import whisper # noqa 176 | if os.path.exists(WHISPER_DIR): 177 | STANDARD_DIRS.append(WHISPER_DIR) 178 | except ImportError: 179 | print >> sys.stderr, "WARNING: whisper module could not be loaded, whisper support disabled" 180 | try: 181 | import rrdtool # noqa 182 | if os.path.exists(RRD_DIR): 183 | STANDARD_DIRS.append(RRD_DIR) 184 | except ImportError: 185 | pass 186 | 187 | if DATABASES is None: 188 | DATABASES = { 189 | 'default': { 190 | 'NAME': join(STORAGE_DIR, 'graphite.db'), 191 | 'ENGINE': 'django.db.backends.sqlite3', 192 | 'USER': '', 193 | 'PASSWORD': '', 194 | 'HOST': '', 195 | 'PORT': '', 196 | }, 197 | } 198 | 199 | # Handle URL prefix in static files handling 200 | if URL_PREFIX and not STATIC_URL.startswith(URL_PREFIX): 201 | STATIC_URL = '/{0}{1}'.format(URL_PREFIX.strip('/'), STATIC_URL) 202 | 203 | # Default sqlite db file 204 | # This is set here so that a user-set STORAGE_DIR is available 205 | if 'sqlite3' in DATABASES.get('default',{}).get('ENGINE','') \ 206 | and not DATABASES.get('default',{}).get('NAME'): 207 | DATABASES['default']['NAME'] = join(STORAGE_DIR, 'graphite.db') 208 | 209 | # Caching shortcuts 210 | if MEMCACHE_HOSTS: 211 | CACHES['default'] = { 212 | 'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache', 213 | 'LOCATION': MEMCACHE_HOSTS, 214 | 'TIMEOUT': DEFAULT_CACHE_DURATION, 215 | 'KEY_PREFIX': MEMCACHE_KEY_PREFIX, 216 | } 217 | 218 | # Authentication shortcuts 219 | if USE_LDAP_AUTH and LDAP_URI is None: 220 | LDAP_URI = "ldap://%s:%d/" % (LDAP_SERVER, LDAP_PORT) 221 | 222 | if USE_REMOTE_USER_AUTHENTICATION or REMOTE_USER_BACKEND: 223 | MIDDLEWARE_CLASSES += ('django.contrib.auth.middleware.RemoteUserMiddleware',) 224 | if REMOTE_USER_BACKEND: 225 | AUTHENTICATION_BACKENDS.insert(0,REMOTE_USER_BACKEND) 226 | else: 227 | AUTHENTICATION_BACKENDS.insert(0,'django.contrib.auth.backends.RemoteUserBackend') 228 | 229 | if USE_LDAP_AUTH: 230 | AUTHENTICATION_BACKENDS.insert(0,'graphite.account.ldapBackend.LDAPBackend') 231 | 232 | if SECRET_KEY == 'UNSAFE_DEFAULT': 233 | warn('SECRET_KEY is set to an unsafe default. This should be set in local_settings.py for better security') 234 | 235 | USE_TZ = True 236 | -------------------------------------------------------------------------------- /provisioning/files/sample_cron.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | cd /var/www/sitespeedio 5 | 6 | sitespeed.io -u http://www.sitespeed.io -b firefox,chrome -n 1 -d 0 --graphiteHost localhost --graphiteNamespace local --seleniumServer http://127.0.0.1:4444/wd/hub >> /tmp/sitespeed-run.txt 7 | -------------------------------------------------------------------------------- /provisioning/files/selenium-start.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | date 4 | # print versions 5 | chromedriver --version 6 | google-chrome-stable --version 7 | 8 | echo 'Starting Xvfb ...' 9 | export DISPLAY=:99 10 | 2>/dev/null 1>&2 Xvfb :99 -shmem -screen 0 1366x768x16 & 11 | echo 'Starting Selenium server ... access it at http://127.0.0.1:4444/wd/hub' 12 | 2>/dev/null 1>&2 nohup java -jar /bin/selenium-server-standalone-2.46.0.jar -Djava.security.egd=file:/dev/./urandom > /tmp/selenium.log & 13 | exec "$@" 14 | -------------------------------------------------------------------------------- /provisioning/files/selenium.conf: -------------------------------------------------------------------------------- 1 | # Starts up selenium for browser testing 2 | description "regular background program processing daemon" 3 | 4 | start on runlevel [2345] 5 | stop on runlevel [!2345] 6 | 7 | expect fork 8 | respawn 9 | 10 | exec /bin/selenium-start.sh -------------------------------------------------------------------------------- /provisioning/files/sitespeed.conf: -------------------------------------------------------------------------------- 1 | # This needs to be in your server's config somewhere, probably 2 | # the main httpd.conf 3 | # NameVirtualHost *:80 4 | 5 | # This line also needs to be in your server's config. 6 | # LoadModule wsgi_module modules/mod_wsgi.so 7 | 8 | # You need to manually edit this file to fit your needs. 9 | # This configuration assumes the default installation prefix 10 | # of /opt/graphite/, if you installed graphite somewhere else 11 | # you will need to change all the occurances of /opt/graphite/ 12 | # in this file to your chosen install location. 13 | 14 | 15 | ServerName sitespeed 16 | DocumentRoot "/var/www/sitespeedio/sitespeed-result" 17 | ErrorLog /var/log/apache2/error.log 18 | CustomLog /var/log/apache2/access.log common 19 | 20 | 21 | Options +Indexes 22 | Order deny,allow 23 | Allow from all 24 | Require all granted 25 | 26 | 27 | 28 | 29 | -------------------------------------------------------------------------------- /provisioning/grafana.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | if [ ! -f /etc/graphana_provisioned_at ] 5 | then 6 | 7 | #Installing Graphana 8 | wget https://grafanarel.s3.amazonaws.com/builds/grafana_2.1.0_amd64.deb 9 | apt-get install -y adduser libfontconfig 10 | dpkg -i grafana_2.1.0_amd64.deb 11 | echo "deb https://packagecloud.io/grafana/stable/debian/ wheezy main" >> /etc/apt/sources.list 12 | curl https://packagecloud.io/gpg.key | apt-key add - 13 | apt-get update 14 | apt-get install grafana 15 | 16 | echo 'Graphana Installation finished' 17 | 18 | date > /etc/graphana_provisioned_at 19 | fi 20 | exit -------------------------------------------------------------------------------- /provisioning/graphite.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | if [ ! -f /etc/graphite_provisioned_at ] 5 | then 6 | 7 | # Install Python Dev Headers 8 | apt-get install -y python-dev 9 | 10 | # Download and install pip 11 | wget -N https://bootstrap.pypa.io/get-pip.py 12 | chmod +x get-pip.py 13 | python get-pip.py 14 | #clean up get-pip.py file 15 | rm get-pip.py 16 | 17 | # Install Django 18 | pip install django 19 | pip install django-tagging 20 | pip install pytz 21 | pip install pyparsing 22 | apt-get install -y libffi-dev 23 | pip install cairocffi 24 | 25 | # Install Graphite 26 | pip install https://github.com/graphite-project/ceres/tarball/master 27 | pip install whisper 28 | pip install https://github.com/graphite-project/carbon/tarball/master --install-option="--prefix=/opt/graphite" --install-option="--install-lib=/opt/graphite/lib" 29 | pip install https://github.com/graphite-project/graphite-web/tarball/master --install-option="--prefix=/opt/graphite" --install-option="--install-lib=/opt/graphite/webapp" 30 | 31 | # Configure and start carbon 32 | pushd /opt/graphite/conf 33 | cp carbon.conf.example carbon.conf 34 | cp storage-schemas.conf.example storage-schemas.conf 35 | cp /opt/graphite/conf/graphite.wsgi.example /opt/graphite/conf/graphite.wsgi 36 | PYTHONPATH=/opt/graphite/webapp django-admin.py migrate --run-syncdb --noinput --settings=graphite.settings 37 | 38 | # Need to start carbon for graphite to be able to collect data 39 | /opt/graphite/bin/carbon-cache.py start 40 | 41 | #Install Apache 42 | apt-get install -y apache2 apache2-mpm-prefork apache2-utils libexpat1 ssl-cert 43 | cp /home/vagrant/provisioning-files/graphite.conf /etc/apache2/sites-available/ 44 | ln -s /etc/apache2/sites-available/graphite.conf /etc/apache2/sites-enabled/graphite.conf 45 | cp /home/vagrant/provisioning-files/sitespeed.conf /etc/apache2/sites-available/ 46 | ln -s /etc/apache2/sites-available/sitespeed.conf /etc/apache2/sites-enabled/sitespeed.conf 47 | 48 | chown -R www-data:www-data /opt/graphite/storage 49 | 50 | # Create docroot directory for sitespeed.io generated files to be browseable 51 | mkdir /var/www/sitespeedio 52 | mkdir /var/www/sitespeedio/sitespeed-result 53 | chown -R www-data:www-data /var/www/sitespeedio 54 | 55 | apt-get install -y libapache2-mod-wsgi 56 | 57 | # Add a sample bash script to run on cron 58 | cp /home/vagrant/provisioning-files/sample_cron.sh /home/vagrant/sample_cron.sh 59 | chmod +x /home/vagrant/sample_cron.sh 60 | chown vagrant:vagrant /home/vagrant/sample_cron.sh 61 | echo "*/5 * * * * www-data /home/vagrant/sample_cron.sh" >> /etc/crontab 62 | crontab /etc/crontab 63 | 64 | echo 'Graphite Installation finished' 65 | 66 | date > /etc/graphite_provisioned_at 67 | fi 68 | exit -------------------------------------------------------------------------------- /provisioning/security.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | echo -n "Do you want to configure secure server (y/n)? " 5 | read answer 6 | if echo "$answer" | grep -iq "^y" ;then 7 | 8 | if [ ! -f /etc/security_provisioned_at ] 9 | then 10 | 11 | echo "Installing (ufw fail2ban unattended-upgrades)" 12 | apt-get install -y ufw fail2ban unattended-upgrades 13 | 14 | echo "Adjust APT update intervals" 15 | cp /home/vagrant/provisioning-files/apt_periodic /etc/apt/apt.conf.d/10periodic 16 | 17 | echo -n "What IP Address is safe? All others are blocked " 18 | read ip 19 | sudo ufw default deny 20 | sudo ufw allow from 127.0.0.1 21 | sudo ufw allow from $ip 22 | sudo ufw enable 23 | 24 | echo "Disable Password Authentication for SSH" 25 | sudo sed -i 's/^PasswordAuthentication.*/PasswordAuthentication no/' /etc/ssh/sshd_config 26 | echo "Disallow root SSH access" 27 | sudo sed -i 's/^PermitRootLogin.*/PermitRootLogin no/' /etc/ssh/sshd_config 28 | sudo service ssh restart 29 | 30 | echo 'Security Installation finished' 31 | 32 | date > /etc/security_provisioned_at 33 | fi 34 | fi 35 | exit -------------------------------------------------------------------------------- /provisioning/sitespeed.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | if [ ! -f /etc/sitespeed_provisioned_at ] 5 | then 6 | # Add Google public key to apt, needed to get Chrome 7 | wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | sudo apt-key add - 8 | 9 | # Add Google to the apt-get source list, needed to get Chrome 10 | sh -c 'echo "deb http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list' 11 | 12 | # Update 13 | apt-get update 14 | 15 | # Install git, curl, java, firefox, chrome, unzip, xvfb 16 | apt-get -y install git curl default-jre-headless firefox google-chrome-stable unzip xvfb 17 | 18 | # Extras for xvfb 19 | apt-get install -y libgl1-mesa-dri xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic 20 | 21 | # Get node and npm 22 | apt-get install -y nodejs npm 23 | 24 | # Setup node (is there a better way to do this?) 25 | if [ ! -f /usr/local/bin/node ] 26 | then 27 | ln -s /usr/bin/nodejs /usr/local/bin/node 28 | fi 29 | 30 | # install sitespeed.io 31 | npm install -g sitespeed.io 32 | 33 | # Now fetch the chrome driver 34 | wget -N http://chromedriver.storage.googleapis.com/2.13/chromedriver_linux64.zip 35 | unzip chromedriver_linux64.zip 36 | rm chromedriver_linux64.zip 37 | chmod +x chromedriver 38 | mv -f chromedriver /usr/bin/chromedriver 39 | 40 | # Set locale 41 | echo "export LC_ALL='en_US.utf8'" >> ~/.bashrc 42 | 43 | # Turn off chrome auto update 44 | if [ -f /etc/apt/sources.list.d/google-chrome.list ] 45 | then 46 | mv /etc/apt/sources.list.d/google-chrome.list /etc/apt/sources.list.d/google-chrome.list.save 47 | fi 48 | 49 | # Install Selenium 50 | npm install selenium-standalone@latest -g 51 | selenium-standalone install 52 | cp /home/vagrant/provisioning-files/selenium-start.sh /bin/ 53 | cd /bin 54 | wget https://selenium-release.storage.googleapis.com/2.46/selenium-server-standalone-2.46.0.jar 55 | chmod +x /bin/selenium-start.sh 56 | /bin/selenium-start.sh 57 | cp /home/vagrant/provisioning-files/selenium.conf /etc/init/ 58 | 59 | echo 'Installation finished' 60 | 61 | date > /etc/sitespeed_provisioned_at 62 | fi 63 | exit --------------------------------------------------------------------------------