Table of Contents
148 |-
150 |
- 1. Code samples for the Practical Devops book
151 |
-
152 |
- 1.1. ch3 How Architecture affects DevOps
153 |
-
154 |
- 1.1.1. Liquibase hello-world 155 |
- 1.1.2. Manual installation 156 |
158 | - 1.2. ch4 Everything is code
159 |
-
160 |
- 1.2.1. Docker intermission 161 |
- 1.2.2. setting up a basic git server 162 |
- 1.2.3. Gerrit 163 |
- 1.2.4. Gitlab 164 |
166 | - 1.3. ch5 Build the code
167 |
-
168 |
- 1.3.1. Cheating with FPM 169 |
- 1.3.2. Build servers, Jenkins in particular 170 |
172 | - 1.4. ch6 Test the code
173 |
-
174 |
- 1.4.1. A Junit example 175 |
- 1.4.2. Arquilian 176 |
- 1.4.3. Automated acceptance testing 177 |
- 1.4.4. A complete test automation scenario 178 |
180 | - 1.5. ch7 Deploying the code
181 |
-
182 |
- 1.5.1. Executing code on the client 183 |
- 1.5.2. Puppet master, Puppet agent 184 |
- 1.5.3. Ansible 185 |
- 1.5.4. Deploying with Chef 186 |
- 1.5.5. Deploying with Saltstack 187 |
- 1.5.6. Vagrant 188 |
190 | - 1.6. ch8 Monitoring the code
191 |
-
192 |
- 1.6.1. Nagios 193 |
- 1.6.2. Munin 194 |
- 1.6.3. Ganglia 195 |
- 1.6.4. Graphite 196 |
- 1.6.5. Log handling 197 |
199 | - 1.7. ch9 Issue Tracking
200 |
-
201 |
- 1.7.1. Bugzilla 202 |
- 1.7.2. Trac 203 |
- 1.7.3. Redmine 204 |
- 1.7.4. The Gitlab issue tracker 205 |
- 1.7.5. Jira 206 |
208 | - 1.8. ch10 The Internet of Things and DevOps
209 |
-
210 |
- 1.8.1. NodeMCU 211 |
213 |
215 | - 1.1. ch3 How Architecture affects DevOps
153 |
1 Code samples for the Practical Devops book
220 |222 | These are the code samples for the book "Practical Devops" by Joakim 223 | Verona published by Packt Publishing. 224 |
225 | 226 |227 | The books home page: 228 | https://www.packtpub.com/networking-and-servers/practical-devops 229 |
230 | 231 |232 | In most cases you will need the book to make the most out of the exercises. 233 |
234 | 235 |236 | Please ensure that you have the latest version of these exercises 237 | before continuing. 238 |
239 | 240 |241 | To get the code samples from Github do: 242 |
243 |git clone https://github.com/jave/practicaldevops.git 246 |247 |
250 | And to keep the samples updated, 251 |
252 |cd practicaldevops
255 | git pull https://github.com/jave/practicaldevops.git
256 |
257 | 1.1 ch3 How Architecture affects DevOps
262 |1.1.1 Liquibase hello-world
265 |cd ch3/liquibase-helloworld
269 | mvn liquibase:update
270 |
271 | 1.1.2 Manual installation
277 |dnf install postgresql
281 | dnf install nginx
282 | cd ch3/crm1
283 | lein build
284 | lein run
285 |
286 | 1.2 ch4 Everything is code
292 |1.2.1 Docker intermission
295 |297 | These instructions are for Fedora, but they are similar for other 298 | distributions such as Ubuntu. 299 |
300 | 301 |302 | To make sure Docker is working properly, 303 | see the following documentation for Fedora. 304 |
305 | 306 |307 | https://docs.docker.com/v1.5/installation/fedora/ 308 |
309 | 310 |-
311 |
- For fedora 21 and later do: 312 |
dnf -y install docker 316 |317 |
-
320 |
- docker-io was renamed to docker from Fedora 21, so use "docker-io" on older red hat 321 | derivates, "docker" on newer 322 | 323 |
- Use a sudo capable user to run docker commands, or the root user 324 | 325 |
- You can also add a docker group with rights to use the docker socket 326 | needed to communicate with the docker daemon. 327 |
330 | This approach is described here 331 | https://docs.docker.com/v1.5/installation/fedora/ 332 |
333 | 334 |335 | In summary: 336 |
337 |$ sudo groupadd docker
340 | $ sudo chown root:docker /var/run/docker.sock
341 | $ sudo usermod -a -G docker $USERNAME
342 |
343 | -
346 |
- You might need "setenforce 0" to start docker. The comand will 347 | disable selinux, which has security implications. Use this only on a 348 | test machine. 349 | 350 |
- To start and enable docker on reboot: 351 |
sudo systemctl start docker 355 | sudo systemctl enable docker 356 |357 |
360 | To verify that docker works: 361 |
362 |sudo docker run -i -t fedora /bin/bash 365 |366 |
369 | For some exercises you need to have docker-compose installed first. 370 |
371 | 372 |373 | On Fedora 23 you can do: 374 |
375 |dnf install docker-compose 378 |379 |
382 | In earlier versions you needed to download docker-compose manually. 383 |
384 |1.2.2 setting up a basic git server
388 |390 | bare repo: 391 |
392 |cd /opt/git 395 | mkdir project.git 396 | cd project.git 397 | git init --bare 398 |399 |
-
402 |
- Now try cloning, making changes, and pushing to the server 403 |
1.2.3 Gerrit
408 |410 | Run a Gerrit container: 411 |
412 |docker run -d -p 8080:8080 -p 29418:29418 openfrontier/gerrit 415 |416 |
419 | On the host machine you can now install the supporting git-review 420 | package: 421 |
422 |sudo dnf install git-review 425 |426 |
429 | Rebase your commits on top of the commits in the remote repository: 430 |
431 |git pull --rebase origin master 434 |435 |
438 | Interactively edit the history, possibly squashing commits together to 439 | make a more readable history: 440 |
441 |git rebase -i origin/master 444 |445 |
1.2.4 Gitlab
452 |454 | Now create a directory for gitlab, and fetch the compose file: 455 |
456 |mkdir gitlab
459 | cd gitlab
460 | wget https://raw.githubusercontent.com/sameersbn/docker-gitlab/master/docker-compose.yml
461 |
462 | 465 | Now start the gitlab stack. 466 |
467 |docker-compose up 470 |471 |
474 | When the containers are up and running, access the web ui: 475 |
476 | 477 |478 | http://loaclhost:10080 479 |
480 | 481 |482 | and enter the following credentials: 483 |
484 | 485 |-
486 |
- username: root 487 |
- password: 5iveL!fe 488 |
1.3 ch5 Build the code
497 |499 | Create a "freestyle" class job in Jenkins that runs the "fortune" 500 | command. 501 |
502 | 503 |504 | First install Jenkins. 505 |
506 |dnf install jenkins 509 |510 |
513 | Then follow the instruction in the book to configure the job. 514 |
515 |1.3.1 Cheating with FPM
518 |520 | To install fpm: 521 |
522 |yum install rubygems 525 | yum install ruby 526 | yum install ruby-devel 527 | gem install fpm 528 |529 |
532 | Package this shell script: 533 |
534 |#!/bin/sh 537 | echo 'Hello World!' 538 | 539 | chmod a+x usr/local/bin/hello.sh 540 | fpm -s dir -t rpm -n hello-world -v 1 -C installdir usr 541 | 542 | rpm -qivp hello-world.rpm 543 | rpm -ivh hello-world.rpm 544 |545 |
1.3.2 Build servers, Jenkins in particular
550 |dnf install jenkins 554 |555 |
systemctl start jenkins 560 |561 |
1.4 ch6 Test the code
568 |1.4.1 A Junit example
571 |cd ch6/hello-junit
575 | mvn install
576 |
577 | 1.4.2 Arquilian
582 |584 | There is an arquillian hello-world in the Arquillian documentation. 585 |
586 |git clone https://github.com/aslakknutsen/arquillian-example-helloworld.git 589 | cd arquillian-example-helloworld 590 | mvn install 591 |592 |
1.4.3 Automated acceptance testing
598 |600 | There are two implementations, one with annotations, and one with 601 | Lambda notation. 602 |
603 | 604 |605 | While the lambda notation is easier to read than the annotation 606 | syntax, cucumbers lambda notation is fairly new and can be problematic 607 | to get to work depending on your Java implementation. 608 |
609 | 610 |611 | To run the annotation based example: 612 |
613 |cd ch6/hello-cucumber6 616 | mvn clean test 617 |618 |
621 | To run the lambda based example: 622 |
623 |cd ch6/hello-cucumber8 626 | mvn clean test 627 |628 |
1.4.4 A complete test automation scenario
636 |- hello-selenium-world638 |
639 | Hello selenium world is a minimal selenium example that should 640 | open a firefox browser window and ask google 'hello world'. 641 | You should see a list of search matches for 'hello world'. 642 |
643 | 644 |645 | It is useful to check that this example runs before testing other examples. 646 | To run it: 647 |
648 |649 | 650 |654 |cd ch6/hello-selenium 651 | mvn test 652 |
653 |
655 | - Running the usermanager example manually656 |
657 | You will need Leiningen, http://leiningen.org/ 658 | https://raw.githubusercontent.com/technomancy/leiningen/stable/bin/lein 659 |
660 |661 | 662 |666 |ch6/usermanager 663 | lein run 664 |
665 |
667 | - Running the automated test668 |669 | 670 |673 |
autotest_v1/bin/autotest.sh 671 |
672 |
674 | - Handling the tricky dependencies with Docker675 |676 | 677 |681 |
docker run -d -p 4444:4444 --name selenium-hub selenium/hub 678 | docker run -d --link selenium-hub:hub selenium/node-firefox 679 |
680 |
1.5 ch7 Deploying the code
686 |1.5.1 Executing code on the client
689 |salt -E '.*' cmd.run 'ls -l' 693 |694 |
1.5.2 Puppet master, Puppet agent
700 |702 | rfkrocktk/puppet is a convenient docker image for exploring puppet. 703 |
704 | 705 |-
706 |
- https://hub.docker.com/r/rfkrocktk/puppet/ this is the agent 707 |
- https://hub.docker.com/r/rfkrocktk/puppetmaster/ this is the master 708 |
docker --name dockerduck --hostname dockerduck -e PUPPETMASTER_TCP_HOST=ultramaster.example.com \ 713 | -v /var/lib/docker/dockercontainer/puppet/ssl:/var/lib/puppet/ssl rfkrocktk/puppet 714 |715 |
1.5.3 Ansible
721 |FROM williamyeh/ansible:centos7 725 |726 |
docker run -v `pwd`/ansible:/ansible -it <hash> bash 731 | cd /ansible 732 | ansible-playbook -i inventory playbook.yml --connection=local --sudo 733 |734 |
737 | A docker container which supports systemd: 738 |
739 |FROM fedora 742 | RUN yum -y update; yum clean all 743 | RUN yum install ansible sudo 744 | RUN systemctl mask systemd-remount-fs.service dev-hugepages.mount \ 745 | sys-fs-fuse-connections.mount \ 746 | systemd-logind.service getty.target console-getty.service 747 | RUN cp /usr/lib/systemd/system/dbus.service /etc/systemd/system/;\ 748 | sed -i 's/OOMScoreAdjust=-900//' /etc/systemd/system/dbus.service 749 | 750 | VOLUME ["/sys/fs/cgroup", "/run", "/tmp"] 751 | ENV container=docker 752 | 753 | CMD ["/usr/sbin/init"] 754 |755 |
758 | To run the new container: 759 |
760 |docker run -it --rm -v /sys/fs/cgroup:/sys/fs/cgroup:ro -v `pwd`/ansible:/ansible <hash>
763 |
764 | 767 | Connect to the container: 768 |
769 |docker exec -it <hash> bash 772 |773 |
776 | A slightly more advanced exercise: 777 |
778 |--- 781 | - hosts: localhost 782 | vars: 783 | http_port: 80 784 | max_clients: 200 785 | remote_user: root 786 | tasks: 787 | - name: ensure apache is at the latest version 788 | yum: name=httpd state=latest 789 | - name: write the apache config file 790 | template: src=/srv/httpd.j2 dest=/etc/httpd.conf 791 | notify: 792 | - restart apache 793 | - name: ensure apache is running (and enable it at boot) 794 | service: name=httpd state=started enabled=yes 795 | handlers: 796 | - name: restart apache 797 | service: name=httpd state=restarted 798 |799 |
1.5.4 Deploying with Chef
806 |808 | Start a clean container for the exercise: 809 |
810 |docker run -it ubuntu bash 813 |814 |
817 | Install Chef in the container: 818 |
819 |curl -L https://www.opscode.com/chef/install.sh | bash 822 |823 |
826 | Verify the chef-solo was installed: 827 |
828 |chef-solo -v 831 |832 |
835 | Fetch and unpack a pre-rolled chef configuration: 836 |
837 |curl -L http://github.com/opscode/chef-repo/tarball/master -o master.tgz 840 | tar -zxf master.tgz 841 | mv chef-repo* chef-repo 842 | rm master.tgz 843 |844 |
847 | Create a configuration file for chef: 848 |
849 |mkdir .chef 852 | echo "cookbook_path [ '/root/chef-repo/cookbooks' ]" > .chef/knife.rb 853 |854 |
857 | Now create a template: 858 |
859 |knife cookbook create phpapp 862 |863 |
1.5.5 Deploying with Saltstack
869 |871 | Start a Saltstack container: 872 |
873 |docker run -i -t --name=saltdocker_master_1 -h master -p 4505 -p 4506 \ 876 | -p 8080 -p 8081 -e SALT_NAME=master -e SALT_USE=master \ 877 | -v `pwd`/srv/salt:/srv/salt:rw jacksoncage/salt 878 |879 |
882 | Start a shell inside the Saltstack container: 883 |
884 |docker exec -i -t saltdocker_master_1 bash 887 |888 |
891 | Salt state to install httpd: 892 |
893 |top.sls: 896 | base: 897 | '*': 898 | - webserver 899 | 900 | webserver.sls: 901 | apache2: # ID declaration 902 | pkg: # state declaration 903 | - installed # function declaration 904 |905 |
908 | Run this command to ensure the desired state: 909 |
910 |salt-call --local state.highstate -l debug 913 |914 |
1.5.6 Vagrant
921 |yum install 'vagrant*'
925 |
926 | 929 | To use Vagrants Virtualbox driver, you need to set up Virtualbox 930 | according to your distribution. 931 |
932 | 933 |934 | Create a virtual machine with Vagrant from a recipy: 935 |
936 |vagrant init hashicorp/precise32 939 |940 |
943 | Try starting the machine: 944 |
945 |vagrant up 948 |949 |
952 | You can now ssh to the machine: 953 |
954 |vagrant ssh 957 |958 |
961 | Add this to the Vagrant file: 962 |
963 |Vagrant.configure("2") do |config| 966 | config.vm.box = "hashicorp/precise32" 967 | config.vm.provision :shell, path: "bootstrap.sh" 968 | end 969 |970 |
973 | And create a bootstrap.sh file that will install Apache httpd: 974 |
975 |#!/usr/bin/env bash 978 | apt-get update 979 | apt-get install -y apache2 980 |981 |
1.6 ch8 Monitoring the code
987 |1.6.1 Nagios
990 |992 | Start a Nagios container: 993 |
994 |docker run -e NAGIOSADMIN_USER=nagiosadmin -e NAGIOSAMDIN_PASS=nagios -p 80:30000 cpuguy83/nagios 997 |998 |
1001 | Start a second container to monitor: 1002 |
1003 |docker run -p 30001:80 nginx 1006 |1007 |
1010 | A docker compose file for the scenario: 1011 |
1012 |nagios: 1015 | image: mt-nagios 1016 | build: 1017 | - mt-nagios 1018 | ports: 1019 | - 80:30000 1020 | environment: 1021 | - NAGIOSADMIN_USER=nagiosadmin 1022 | - NAGIOSAMDIN_PASS=nagios 1023 | volumes: 1024 | ./nagios:/etc/nagios 1025 | nginx: 1026 | image: nginx 1027 |1028 |
1031 | Configuration files for the Nagios example: 1032 |
1033 |define host { 1036 | name regular-host 1037 | use linux-server 1038 | register 0 1039 | max_check_attempts 5 1040 | } 1041 | 1042 | define host{ 1043 | use regular-host 1044 | host_name client1 1045 | address 192.168.200.15 1046 | contact_groups admins 1047 | notes test client1 1048 | } 1049 |1050 |
1052 | hostgroups.cfg 1053 |
1054 | 1055 |define hostgroup { 1058 | hostgroup_name test-group 1059 | alias Test Servers 1060 | members client1 1061 | } 1062 | 1063 | services.cfg 1064 | #+BEGIN_SRC sh 1065 | define service { 1066 | use generic-service 1067 | hostgroup_name test-group 1068 | service_description PING 1069 | check_command check_ping!200.0,20%!600.0,60% 1070 | } 1071 |1072 |
1075 | An example mail configuration: 1076 |
1077 |define contact{ 1080 | contact_name matangle-admin 1081 | use generic-contact 1082 | alias Nagios Admin 1083 | email pd-admin@matangle.com 1084 | } 1085 | 1086 | define contactgroup{ 1087 | contactgroup_name admins 1088 | alias Nagios Administrators 1089 | members matange-admin 1090 | } 1091 |1092 |
1.6.2 Munin
1098 |docker run -p 30005:80 lrivallain/munin:latest 1102 |1103 |
1106 | Running commands in the munin container: 1107 |
1108 |docker exec -it <hash> bash 1111 | su - munin --shell=/bin/bash 1112 | /usr/share/munin/munin-update 1113 |1114 |
1117 | If you are having trouble running munin-update, try: 1118 |
1119 |chown munin.munin /var/log/munin/munin-update.log 1122 |1123 |
1126 | It may still take some time for the graphs to display. 1127 |
1128 | 1129 |1130 | This is the code for the munin plugin: 1131 |
1132 |graph_title Load average 1135 | graph_vlabel load 1136 | load.label load 1137 |1138 |
1141 | To emit data you simply print it to stdout. 1142 |
1143 | 1144 | 1145 |printf "load.value " 1148 | cut -d' ' -f2 /proc/loadavg 1149 |1150 |
1153 | Here is an example script. 1154 |
1155 | 1156 | 1157 |#!/bin/sh 1160 | 1161 | case $1 in 1162 | config) 1163 | cat <<'EOM' 1164 | graph_title Load average 1165 | graph_vlabel load 1166 | load.label load 1167 | EOM 1168 | exit 0;; 1169 | esac 1170 | 1171 | printf "load.value " 1172 | cut -d' ' -f2 /proc/loadavg 1173 |1174 |
1.6.3 Ganglia
1180 |1182 | To get help with the container: 1183 |
1184 |docker run wookietreiber/ganglia --help 1187 |1188 |
1191 | To run the Ganglia container: 1192 |
1193 |docker run -p 30010:80 wookietreiber/ganglia 1196 |1197 |
1.6.4 Graphite
1204 |1206 | Start Graphite: 1207 |
1208 |docker run -it -p 30020:80 -p 2003:2003 sitespeedio/graphite 1211 |1212 |
1215 | Try the following url: http://localhost:30020/ 1216 |
1217 |1.6.5 Log handling
1221 |1223 | Start Kibana and Elasticsearch: 1224 |
1225 |docker run -d elasticsearch && 1228 | docker run --link some-elasticsearch:elasticsearch -d kibana 1229 |1230 |
1.7 ch9 Issue Tracking
1237 |1.7.1 Bugzilla
1240 |docker run -p 6050:80 dklawren/docker-bugzilla 1244 |1245 |
1.7.2 Trac
1250 |docker run -d -p 6051:8080 barogi/trac:1.0.2 1254 |1255 |
1.7.3 Redmine
1260 |docker run -d -p 6052:3000 redmine 1264 |1265 |
1.7.4 The Gitlab issue tracker
1270 |1272 | Trying the Gitlab CLI: 1273 |
1274 |GITLAB_API_PRIVATE_TOKEN=<token from your project> 1277 | GITLAB_API_ENDPOINT=http://gitlab.matangle.com:50003/api/v3 1278 | gitlab help Issues 1279 |1280 |
1.7.5 Jira
1285 |docker run -p 6053:8080 cptactionhank/atlassian-jira:latest 1289 |1290 |
1.8 ch10 The Internet of Things and DevOps
1297 |1.8.1 NodeMCU
1300 |1302 | To get a newer firmware(please change the version to the latest 1303 | available first): 1304 |
1305 |wget https://github.com/nodemcu/nodemcu-firmware/releases/download/0.9.6-dev_20150704/nodemcu_integer_0.9.6-dev_20150704.bin 1308 |1309 |
1312 | Get esptool: 1313 |
1314 |git clone https://github.com/themadinventor/esptool.git 1317 |1318 |
1321 | Install pyserial: 1322 |
1323 |sudo dnf install pyserial 1326 |1327 |
1330 | Burn the firmware: 1331 |
1332 |sudo python ./esptool.py --port /dev/ttyUSB0 write_flash 0x00000 nodemcu_integer_0.9.6-dev_20150704.bin 1335 |1336 |
1339 | You might need additional arguments: 1340 |
1341 |sudo esptool.py --port=/dev/ttyUSB0 write_flash 0x0 nodemcu_integer_0.9.6-dev_20150704.bin -fs 32m -fm dio -ff 40m 1344 |1345 |
1348 | Do some tests to see that the connection is working: 1349 |
1350 |sudo ./esptool.py read_mac 1353 | Connecting... 1354 | MAC: 18:fe:34:00:d7:21 1355 | 1356 | sudo ./esptool.py flash_id 1357 | Connecting... 1358 | Manufacturer: e0 1359 | Device: 4016 1360 |1361 |
1364 | Try the LED: 1365 |
1366 |gpio.write(0, gpio.LOW) -- turn led on 1369 |1370 |
gpio.write(0, gpio.HIGH) -- turn led off 1375 |1376 |
1379 | Blink the LED in a loop: 1380 |
1381 |while 1 do -- loop forever 1384 | gpio.write(0, gpio.HIGH) -- turn led off 1385 | tmr.delay(1000000) -- wait one second 1386 | gpio.write(0, gpio.LOW) -- turn led on 1387 | tmr.delay(1000000) -- wait one second 1388 | end 1389 |1390 |
1392 | To connect to a wireless network. 1393 |
1394 | 1395 |wifi.setmode(wifi.STATION) 1398 | wifi.sta.config("SSID","password") 1399 |1400 |
1403 | To see the IP we got: 1404 |
1405 |print(wifi.sta.getip())
1408 |
1409 | 1412 | Connecting to a web server: 1413 |
1414 |conn=net.createConnection(net.TCP, false) 1417 | conn:on("receive", function(conn, pl) print(pl) end) 1418 | conn:connect(80,"121.41.33.127") 1419 | conn:send("GET / HTTP/1.1\r\nHost: www.nodemcu.com\r\n" 1420 | .."Connection: keep-alive\r\nAccept: */*\r\n\r\n") 1421 |1422 |
1425 | Timer: 1426 |
1427 |tmr.alarm(1, 1000, 1, function() 1430 | print("hello world") 1431 | end ) 1432 |1433 |
1436 | Stop the timer: 1437 |
1438 |tmr.stop(1) 1441 |1442 |