├── README.md ├── docker-overlay.png ├── images ├── README.md └── vm.svg ├── image.md ├── linux.md ├── subcommands.md ├── documentation.md ├── commit.md ├── network.md ├── history.md ├── registry.md ├── volume.md ├── install.md ├── entrypoint.md ├── orchestration.md ├── docker-compose.yaml ├── mirantis.md ├── networking.md ├── docker.md ├── swarm.md ├── caltech ├── 2022-06-11.md ├── 2022-05-28.md ├── 2022-06-05.md ├── 2022-05-29.md ├── 2022-06-12.md └── 2022-06-04.md ├── docker-stack.md ├── caltech_2022-11 ├── 2022-11-05.md └── 2022-11-06.md ├── 2022-05-01.md ├── caltech_2022-09 ├── 2022-09-17.md ├── 2022-09-18.md ├── 2022-09-24.md ├── 2022-09-25.md ├── 2022-10-01.md └── 2022-10-02.md ├── caltech_2022-08 ├── 2022-08-06.md ├── 2022-08-13.md ├── 2022-08-14.md ├── 2022-08-20.md ├── 2022-08-21.md └── 2022-08-07.md ├── 2022-05-07.md ├── 2022-04-30.md ├── swarm-example.md ├── 2022-04-24.md ├── 2022-04-23.md ├── caltech_2022-10 ├── 2022-10-30.md ├── 2022-10-29.md ├── 2022-10-08.md ├── 2022-10-16.md ├── 2022-10-15.md └── 2022-10-09.md ├── docker-example.md ├── 2022-05-08.md └── networking2.md /README.md: -------------------------------------------------------------------------------- 1 | # dca -------------------------------------------------------------------------------- /docker-overlay.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/sebastian-colomar/dca/HEAD/docker-overlay.png -------------------------------------------------------------------------------- /images/README.md: -------------------------------------------------------------------------------- 1 | In this folder you will find several pictures to help comprehend the subject. 2 | -------------------------------------------------------------------------------- /image.md: -------------------------------------------------------------------------------- 1 | ``` 2 | sudo docker container run -d --name web --rm nginx:alpine 3 | sudo docker image history nginx:alpine 4 | sudo docker container export web -o nginx.tar 5 | sudo docker image import nginx.tar nginx:flat 6 | sudo docker image history nginx:flat 7 | ``` 8 | -------------------------------------------------------------------------------- /linux.md: -------------------------------------------------------------------------------- 1 | ``` 2 | ss --listening --numeric --tcp 3 | sudo ss --listening --numeric --tcp --processes 4 | 5 | sudo apt install --yes nginx 6 | sudo ss --listening --numeric --processes | grep :80\ 7 | 8 | sudo apt purge nginx --yes 9 | sudo apt autoremove --yes 10 | -------------------------------------------------------------------------------- /subcommands.md: -------------------------------------------------------------------------------- 1 | ``` 2 | docker container --help 3 | docker image --help 4 | docker volume --help 5 | docker network --help 6 | ``` 7 | ``` 8 | docker swarm --help 9 | docker node --help 10 | docker stack --help 11 | docker service --help 12 | ``` 13 | ``` 14 | kubectl ... 15 | ``` 16 | -------------------------------------------------------------------------------- /documentation.md: -------------------------------------------------------------------------------- 1 | https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html-single/resource_management_guide/index 2 | 3 | https://docs.docker.com/ 4 | 5 | https://kubernetes.io/docs/home/ 6 | 7 | https://blog.usejournal.com/how-to-enable-docker-remote-api-on-docker-host-7b73bd3278c6 8 | -------------------------------------------------------------------------------- /commit.md: -------------------------------------------------------------------------------- 1 | ``` 2 | docker container run --detach --entrypoint /bin/sh --name test --rm --tty library/alpine:latest 3 | docker container exec test /usr/binwhich php 4 | docker container exec test /sbin/apk add php 5 | docker container exec test /usr/bin/which php 6 | docker container commit test alpine:php 7 | ``` 8 | -------------------------------------------------------------------------------- /network.md: -------------------------------------------------------------------------------- 1 | ``` 2 | docker network ls 3 | docker network create sebastian 4 | docker network inspect sebastian 5 | docker network prune --force 6 | ``` 7 | ``` 8 | docker container run -d --name web --network host nginx:alpine 9 | docker container exec web ip route 10 | ip route 11 | docker container exec web netstat -lnt 12 | netstat -lnt 13 | docker container exec web curl localhost 14 | curl localhost 15 | ``` 16 | -------------------------------------------------------------------------------- /history.md: -------------------------------------------------------------------------------- 1 | 2002 - LINUX NAMESPACES -> PARTITION KERNEL RESOURCES 2 | 3 | 2007 - CONTROL GROUPS -> CONTAINERIZE KERNEL RESOURCES 4 | 5 | 2008 - LXC -> MANAGE LINUX CONTAINERS 6 | 7 | 2013 - DOCKER -> MANAGE LINUX CONTAINERS 8 | 9 | 2014 - KUBERNETES -> ORCHESTRATE LINUX CONTAINERS 10 | 11 | 2016 - SWARM -> ORCHESTRATE CONTAINERS 12 | 13 | 2017 - CONTAINERD -> MANAGE LINUX CONTAINERS 14 | 15 | 2020 - CRI-O -> MANAGE LINUX CONTAINERS 16 | -------------------------------------------------------------------------------- /registry.md: -------------------------------------------------------------------------------- 1 | ``` 2 | docker run --detach --name registry --publish 5000:5000 --restart always --volume registry:/var/lib/registry:rw docker.io/library/registry:2 3 | docker pull docker.io/library/busybox:latest 4 | docker tag docker.io/library/busybox:latest localhost:5000/my_library/my_busybox:1.0 5 | docker push localhost:5000/my_library/my_busybox:1.0 6 | docker pull localhost:5000/my_library/my_busybox:1.0 7 | docker volume inspect registry 8 | sudo find /var/lib/docker/volumes/registry/_data/ 9 | ``` 10 | -------------------------------------------------------------------------------- /volume.md: -------------------------------------------------------------------------------- 1 | ```bash 2 | sudo docker volume create sebastian 3 | sudo ls /var/lib/docker/volumes/sebastian/_data -l 4 | sudo docker container run --rm --name test -d -v sebastian:/opt alpine ping localhost 5 | sudo docker container exec test ls /opt -l 6 | sudo docker container exec test touch /opt/sebastian 7 | sudo docker container exec test ls /opt -l 8 | sudo docker volume inspect test 9 | sudo ls /var/lib/docker/volumes/sebastian/_data -l 10 | ``` 11 | ``` 12 | mkdir example 13 | sudo docker container run --rm --name test2 -d -v $PWD/example:/opt alpine ping localhost 14 | sudo docker container exec test2 ls -l /opt 15 | ls example -l 16 | sudo docker container exec test2 touch /opt/helloworld 17 | sudo docker container exec test2 ls -l /opt 18 | ls example -l 19 | ``` 20 | -------------------------------------------------------------------------------- /install.md: -------------------------------------------------------------------------------- 1 | ``` 2 | sudo apt-get remove docker docker-engine docker.io containerd runc 3 | sudo apt-get update 4 | sudo apt-get install \ 5 | apt-transport-https \ 6 | ca-certificates \ 7 | curl \ 8 | gnupg \ 9 | lsb-release 10 | curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg 11 | echo \ 12 | "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \ 13 | $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null 14 | sudo apt-get update 15 | sudo apt-get install docker-ce docker-ce-cli containerd.io 16 | 17 | sudo groupadd docker 18 | sudo usermod -aG docker $USER 19 | newgrp docker 20 | 21 | docker run hello-world 22 | -------------------------------------------------------------------------------- /entrypoint.md: -------------------------------------------------------------------------------- 1 | ``` 2 | docker container run --entrypoint ENTRYPOINT_BINARY_EXECUTABLE --rm IMAGE_FILESYSTEM:RELEASE ARGUMENTS_FOR_THE_ENTRYPOINT 3 | ``` 4 | ``` 5 | docker container run --entrypoint echo --rm alpine:latest "HELLO WORLD" 6 | docker container run --entrypoint /bin/echo --rm alpine:latest "HELLO WORLD" 7 | docker container run --entrypoint /sbin/ip --rm alpine:latest route 8 | docker container run --entrypoint /bin/ls --rm alpine:latest /bin 9 | docker container run --entrypoint /usr/bin/which --rm alpine:latest netstat 10 | docker container run --entrypoint /bin/netstat --rm alpine:latest --help 11 | ``` 12 | ``` 13 | cat /proc/1/cgroups 14 | docker container run --detach --entrypoint /bin/ping --name pinger --rm alpine:latest localhost 15 | docker container logs pinger 16 | docker container ls 17 | docker container top pinger 18 | cat /proc/27226/cgroup 19 | docker container stats pinger --no-stream 20 | docker container kill pinger 21 | docker container run --cpus 0.05 --detach --entrypoint /bin/ping --name pinger --rm alpine:latest localhost 22 | docker container kill pinger 23 | docker container run --cpus 0.05 --detach --entrypoint /bin/ping --memory 10M --name pinger --rm alpine:latest localhost 24 | docker container stats pinger --no-stream 25 | ``` 26 | ``` 27 | docker container ls 28 | docker ps 29 | ``` 30 | -------------------------------------------------------------------------------- /orchestration.md: -------------------------------------------------------------------------------- 1 | # Introduction to Orchestration 2 | Reasons to use orchestration: High availability 3 | * https://raw.githubusercontent.com/sebastian-colomar/dca/main/images/ha-failover.svg 4 | * https://raw.githubusercontent.com/sebastian-colomar/dca/main/images/high_availability.svg 5 | * https://raw.githubusercontent.com/sebastian-colomar/dca/main/images/rob_cam.svg 6 | 7 | There are two well known orchestrators: 8 | * Swarm 9 | * Kubernetes 10 | 11 | # Docker Compose vs Docker Swarm vs Kubernetes 12 | 1. Docker Compose is a Python script that works as wrapper of Docker commands: 13 | ``` 14 | which docker-compose 15 | 16 | file /usr/bin/docker-compose 17 | 18 | head /usr/bin/docker-compose 19 | ``` 20 | 1. Docker Compose does not provide high availability because it is only available as a standalone host machine. If we need high availability then we need to use Docker Swarm or Kubernetes. Docker Swarm is a feature of the Docker engine that is enabled with the following commands: 21 | 22 | * https://labs.play-with-docker.com/ 23 | 24 | ``` 25 | docker swarm init --advertise-addr $( hostname -i ) 26 | ``` 27 | 1. Check the status of the cluster with the command: 28 | 29 | ``` 30 | docker node ls 31 | ``` 32 | 1. Architecture of managers and workers: 33 | 34 | * https://d33wubrfki0l68.cloudfront.net/2475489eaf20163ec0f54ddc1d92aa8d4c87c96b/e7c81/images/docs/components-of-kubernetes.svg 35 | -------------------------------------------------------------------------------- /docker-compose.yaml: -------------------------------------------------------------------------------- 1 | # docker-compose.yaml 2 | # THIS IS THE LIST OF DOCKER CONFIGS WE WANT TO APPLY 3 | configs: 4 | # THIS IS THE DOCKER CONFIG WE CREATED IN A PREVIOUS STEP 5 | index.php: 6 | # THIS WILL INVOKE THE PREVIOUSLY CREATED DOCKER CONFIG 7 | external: true 8 | # THIS IS THE LIST OF SERVICES WE WANT TO CREATE 9 | services: 10 | # THIS IS THE NAME OF THE SERVICE 11 | phpinfo: 12 | # THESE ARE THE ARGUMENTS OF THE ENTRYPOINT 13 | command: 14 | - -f 15 | - index.php 16 | - -S 17 | - 0.0.0.0:8080 18 | # EQUIVALENT TO: --config source=index.php,target=/app/index.php,mode=0400,uid=65534 19 | configs: 20 | - 21 | mode: 0400 22 | source: index.php 23 | target: /app/index.php 24 | uid: '65534' 25 | # EQUIVALENT TO: --mode replicated --replicas 2 --restart-condition any 26 | deploy: 27 | mode: replicated 28 | replicas: 2 29 | restart_policy: 30 | condition: any 31 | # EQUIVALENT TO: --entrypoint php 32 | entrypoint: php 33 | # THIS IS THE NAME OF THE DOCKER IMAGE 34 | image: php 35 | # EQUIVALENT TO: --publish 8080 36 | ports: 37 | - 8080 38 | # EQUIVALENT TO: --read-only 39 | read_only: true 40 | # EQUIVALENT TO: --user nobody 41 | user: nobody 42 | # EQUIVALENT TO: --workdir /app/ 43 | working_dir: /app/ 44 | # THIS IS THE DOCKER API VERSION THAT WE WANT TO USE FOR THIS DEPLOYMENT 45 | version: "3.8" 46 | -------------------------------------------------------------------------------- /mirantis.md: -------------------------------------------------------------------------------- 1 | ## INSTALL MIRANTIS CONTAINER RUNTIME 2 | https://docs.mirantis.com/mcr/20.10/install/mcr-linux/ubuntu.html 3 | ``` 4 | sudo apt-get --yes remove docker docker-engine docker-ce docker-ce-cli docker.io 5 | sudo apt-get --yes update 6 | sudo apt-get --yes install apt-transport-https ca-certificates curl software-properties-common 7 | DOCKER_EE_URL="http://repos.mirantis.com" 8 | DOCKER_EE_VERSION=20.10 9 | curl -fsSL "${DOCKER_EE_URL}/ubuntu/gpg" | sudo apt-key add - 10 | sudo apt-key fingerprint 6D085F96 11 | sudo add-apt-repository "deb [arch=$(dpkg --print-architecture)] $DOCKER_EE_URL/ubuntu $(lsb_release -cs) stable-$DOCKER_EE_VERSION" 12 | sudo apt-get --yes update 13 | sudo apt-get --yes install docker-ee docker-ee-cli containerd.io 14 | ``` 15 | ## ADD DOCKER GROUP 16 | https://docs.docker.com/engine/install/linux-postinstall/ 17 | ``` 18 | sudo groupadd docker 19 | sudo usermod -aG docker $USER 20 | newgrp docker 21 | docker run hello-world 22 | ``` 23 | ## INSTALL MIRANTIS KUBERNETES ENGINE 24 | https://docs.mirantis.com/mke/3.4/install/install-mke-image.html 25 | ``` 26 | docker container run --rm --interactive --tty --name ucp --volume /var/run/docker.sock:/var/run/docker.sock mirantis/ucp:3.4.9 install --host-address $( ip route | grep dev.eth0.proto.kernel | awk '{ print $9 }' ) --interactive --force-minimums 27 | ``` 28 | ## UNINSTALL MIRANTIS KUBERNETES ENGINE 29 | ``` 30 | docker container run --rm -it -v /var/run/docker.sock:/var/run/docker.sock --name ucp mirantis/ucp:3.4.9 uninstall-ucp --interactive 31 | ``` 32 | -------------------------------------------------------------------------------- /networking.md: -------------------------------------------------------------------------------- 1 | ``` 2 | docker container run --detach --name c0 --network none --tty library/alpine:latest 3 | 4 | docker network create net1 5 | docker container run --detach --name c1 --network net1 --tty library/alpine:latest 6 | 7 | docker network create net2 8 | docker container run --detach --name c2 --network net2 --tty library/alpine:latest 9 | ``` 10 | ``` 11 | docker container exec c0 ifconfig 12 | docker container exec c1 ifconfig 13 | docker container exec c2 ifconfig 14 | ``` 15 | ``` 16 | ip route 17 | 18 | docker container exec c0 ip route 19 | docker container exec c1 ip route 20 | docker container exec c2 ip route 21 | ``` 22 | ``` 23 | docker container exec c0 ping -c1 localhost 24 | docker container exec c1 ping -c1 localhost 25 | docker container exec c2 ping -c1 localhost 26 | ``` 27 | ``` 28 | docker container exec c0 ping -c1 8.8.8.8 29 | docker container exec c1 ping -c1 8.8.8.8 30 | docker container exec c2 ping -c1 8.8.8.8 31 | ``` 32 | ``` 33 | docker container inspect c0 | grep IPAddress 34 | docker container inspect c1 | grep IPAddress 35 | docker container inspect c2 | grep IPAddress 36 | ``` 37 | ``` 38 | docker container exec c1 ping c2 39 | docker container exec c2 ping c1 40 | ``` 41 | ``` 42 | docker network connect net1 c2 43 | docker container inspect c2 | grep IPAddress 44 | docker network inspect net1 45 | 46 | docker container exec c1 ping -c1 c2 47 | docker container exec c2 ping -c1 c1 48 | ``` 49 | ``` 50 | docker network disconnect net1 c2 51 | docker container inspect c2 | grep IPAddress 52 | docker network inspect net1 53 | 54 | docker container exec c1 ping -c1 c2 55 | docker container exec c2 ping -c1 c1 56 | ``` 57 | 58 | -------------------------------------------------------------------------------- /docker.md: -------------------------------------------------------------------------------- 1 | ```bash 2 | 1998 sudo docker run --name test -d alpine 3 | 1999 sudo docker ps 4 | 2000 sudo docker ps -a 5 | 2001 sudo ls /var/lib/docker/container 6 | 2002 sudo ls /var/lib/docker/containers 7 | 2003 sudo docker ps -a --no-trunc 8 | 2004 sudo docker start f01ec86941fdd6384d948914cd3a2cefa8c2fdc87a5df575ef8e57835bfdcf7f 9 | 2005 sudo docker ps 10 | 2006 sudo docker kill f01ec86941fdd6384d948914cd3a2cefa8c2fdc87a5df575ef8e57835bfdcf7f 11 | 2007 sudo docker ps 12 | 2008 sudo docker ps -a --no-trunc 13 | 2009 sudo docker run --name test -d -t alpine 14 | 2010 sudo docker ps -a 15 | 2011 sudo docker rm test nginx 16 | 2012 sudo docker run --name test -d -t alpine 17 | 2013 sudo docker ps 18 | 2014 df -h 19 | 2015 df 20 | 2016 sudo docker ps 21 | 2017 sudo docker exec test echo HELLO WORLD 22 | 2018 sudo docker exec test date 23 | 2019 sudo docker exec test sleep 3 24 | 2020 sudo docker exec test bash 25 | 2021 sudo docker exec test find / 26 | 2022 sudo docker exec test which bash 27 | 2023 which bash 28 | 2025 sudo docker exec test ls -l /bin/bash 29 | 2026 sudo docker exec test find / | grep bash 30 | 2027 sudo docker exec test find / | grep sh$ 31 | 2028 sudo docker exec -i -t test sh 32 | 2029 sudo docker exec test free 33 | 2030 sudo docker exec test df 34 | 2031 sudo docker exec test ss 35 | 2032 sudo docker exec test netstat 36 | 2033 sudo docker exec test netstat -ltn 37 | 2034 sudo docker exec test id 38 | 2035 sudo docker exec test curl 39 | 2036 sudo docker exec test apk add curl 40 | 2037 sudo docker exec test curl www.google.com -I 41 | 2038 sudo docker exec test which tcpdump 42 | 2039 sudo docker exec test apk add tcpdump 43 | 2040 sudo docker exec test which tcpdump 44 | ``` 45 | ``` 46 | docker run --detach --name NAME_OF_CONTAINER --rm NAME_OF_IMAGE:RELEASE_NAME 47 | docker exec NAME_OF_CONTAINER COMMAND ARGUMENT_OF_COMMAND 48 | -------------------------------------------------------------------------------- /swarm.md: -------------------------------------------------------------------------------- 1 | ```bash 2 | sudo docker swarm init 3 | sudo docker node ls 4 | ``` 5 | ``` 6 | git clone https://github.com/academiaonline/dca-phpinfo 7 | cd dca-phpinfo 8 | sudo docker stack deploy -c etc/swarm/manifests/dca-phpinfo.yaml dca-phpinfo 9 | ``` 10 | ``` 11 | sudo docker service ls 12 | sudo docker service logs dca-phpinfo_dca-phpinfo 13 | curl localhost -I 14 | ``` 15 | ``` 16 | sudo docker stack rm dca-phpinfo 17 | cd 18 | rm -rf dca-phpinfo 19 | git clone https://github.com/academiaonline/dca-phpinfo 20 | cd dca-phpinfo 21 | sudo docker stack deploy -c etc/swarm/manifests/dca-phpinfo.yaml dca-phpinfo 22 | sudo docker service ls 23 | sudo docker service logs dca-phpinfo_dca-phpinfo 24 | curl localhost -I 25 | ``` 26 | ``` 27 | sudo docker container --help 28 | sudo docker container exec 29 | sudo docker container kill 30 | sudo docker container ls 31 | sudo docker container port 32 | sudo docker container prune 33 | sudo docker container rm 34 | sudo docker container run 35 | sudo docker image --help 36 | sudo docker image build 37 | sudo docker image prune 38 | sudo docker image pull 39 | sudo docker image push 40 | sudo docker volume --help 41 | sudo docker network --help 42 | ``` 43 | ``` 44 | sudo docker node --help 45 | sudo docker node ls 46 | sudo docker service --help 47 | sudo docker service create 48 | sudo docker service ls 49 | sudo docker service logs 50 | sudo docker service ps 51 | sudo docker service scale 52 | sudo docker stack --help 53 | sudo docker stack deploy 54 | sudo docker stack ls 55 | sudo docker stack rm 56 | sudo docker swarm --help 57 | sudo docker swarm init 58 | sudo docker swarm join 59 | sudo docker swarm leave 60 | ``` 61 | ``` 62 | sudo docker service create --name dca-phpinfo -p 8000:8080 academiaonline/dca-phpinfo:latest 63 | sudo docker service ls 64 | sudo docker service ps dca-phpinfo 65 | sudo docker service scale dca-phpinfo=3 66 | sudo docker service ps dca-phpinfo 67 | sudo docker service rm dca-phpinfo 68 | ``` 69 | ```bash 70 | sudo iptables -S -t nat 71 | ``` 72 | -------------------------------------------------------------------------------- /caltech/2022-06-11.md: -------------------------------------------------------------------------------- 1 | # Install Mirantis Container Runtime: (Docker Engine) 2 | 1. https://docs.mirantis.com/mcr/20.10/qs-ubuntu/install-ubuntu.html 3 | 4 | ``` 5 | sudo apt-get --yes remove docker docker-engine docker-ce docker-ce-cli docker.io 6 | sudo apt-get update 7 | sudo apt-get --yes install apt-transport-https ca-certificates curl software-properties-common 8 | DOCKER_EE_URL="https://repos.mirantis.com" 9 | DOCKER_EE_VERSION=20.10 10 | curl -fsSL "${DOCKER_EE_URL}/ubuntu/gpg" | sudo apt-key add - 11 | sudo apt-key fingerprint 6D085F96 12 | sudo add-apt-repository "deb [arch=$(dpkg --print-architecture)] $DOCKER_EE_URL/ubuntu $(lsb_release -cs) stable-$DOCKER_EE_VERSION" 13 | sudo apt-get update 14 | sudo apt-get --yes install docker-ee docker-ee-cli containerd.io 15 | ``` 16 | # Install Mirantis Kubernetes Engine: (Docker and Kubernetes Orchestrator) 17 | 1. https://docs.mirantis.com/mke/3.5/install/install-mke-image.html 18 | ``` 19 | sudo docker run --interactive --name ucp --rm --tty --volume /var/run/docker.sock:/var/run/docker.sock mirantis/ucp:3.5.3 install --host-address $( hostname --ip-address ) --interactive --force-minimums 20 | ``` 21 | # Uninstall Mirantis Kubernetes Engine: 22 | ``` 23 | sudo docker container run --rm -it -v /var/run/docker.sock:/var/run/docker.sock --name ucp mirantis/ucp:3.5.3 uninstall-ucp --interactive 24 | ``` 25 | # Install Mirantis Secure Registry: (Docker Trusted Registry) 26 | 1. https://docs.mirantis.com/msr/2.9/install/install-online.html 27 | ``` 28 | sudo docker run -it --rm mirantis/dtr:2.9.7 install --ucp-insecure-tls --ucp-url $( hostname --ip-address ):443 29 | ``` 30 | # Documentation: 31 | 1. https://docs.mirantis.com/mke/3.5/ops.html 32 | 2. https://docs.mirantis.com/msr/2.9/ops.html 33 | 34 | # Security: 35 | ``` 36 | sudo docker pull mysql:8.0.29 37 | sudo docker pull mysql@sha256:0c0beeac7ca1937d60f54e1fb0c4a5c0b0ffee2aae37488fbc9f5ea301425551 38 | sudo docker inspect tomcat:alpine | grep RepoDigests -A2 39 | sudo docker inspect calico/cni:v3.17.6 | grep RepoDigests -A2 40 | sudo docker run --read-only nginx 41 | sudo docker run --read-only --volume /var/cache/nginx/ --volume /var/run/ nginx 42 | sudo docker run --read-only --volume vol1:/var/cache/nginx/:rw --volume vol2:/var/run/:rw nginx 43 | sudo docker run --read-only --volume vol1:/var/cache/nginx/:ro --volume vol2:/var/run/:ro nginx 44 | ``` 45 | -------------------------------------------------------------------------------- /docker-stack.md: -------------------------------------------------------------------------------- 1 | # How to deploy a Docker stack using a Docker compose file 2 | 3 | * https://docs.docker.com/compose/compose-file/compose-file-v3/ 4 | 5 | ``` 6 | tee ${PWD}/phpinfo/docker-compose.yaml 0<&1 | tee --append ${log} 17 | ``` 18 | Once the initialization has completed continue with the following commands still on the first master node: 19 | ``` 20 | sudo kubectl apply --filename ${calico} --kubeconfig ${kubeconfig} 2>& 1 | tee --append ${log} 21 | mkdir -p ${HOME}/.kube/ 22 | sudo cp /etc/kubernetes/admin.conf ${HOME}/.kube/config 23 | sudo chown -R $( id -u ):$( id -g ) ${HOME}/.kube/ 24 | echo 'source <(kubectl completion bash)' | tee --append ${HOME}/.bashrc 25 | source ${HOME}/.bashrc 26 | ``` 27 | Check that the Leader master is ready before continuing: 28 | ``` 29 | kubectl get no | grep Ready 30 | ``` 31 | Once the Leader master is ready you can continue with these commands: 32 | ``` 33 | sudo sed --in-place /${kube}/d /etc/hosts 34 | sudo sed --in-place /127.0.0.1.*localhost/s/$/' '${kube}/ /etc/hosts 35 | grep ^kubeadm.join ${log} -A1 36 | ``` 37 | Now you are ready to join the worker nodes to the Kubernetes cluster. Run the following commands on the worker nodes: 38 | ``` 39 | ip_master1=172.X.X.X # HERE SUBSTITUTE THE VALUE OF THE IP ADDRESS FOR THE LEADER MASTER ${ip_leader} 40 | # FOR EXAMPLE: ip_master1=172.31.1.138 41 | kube=kube-apiserver 42 | echo ${ip_master1} ${kube} | sudo tee --append /etc/hosts 43 | ``` 44 | In order to join the worker node to the cluster you need to run the output of the previous command from the Leader master like explained in the following paragraph: 45 | ``` 46 | sudo XXXXXXXXXXX # HERE YOU SUBSTITUTE THE OUTPUT OF THE FOLLOWING COMMAND RUN ON THE LEADER MASTER: grep ^kubeadm.join ${log} -A1 47 | # FOR EXAMPLE: 48 | # sudo kubeadm join kube-apiserver:6443 --token xxx.xxx --discovery-token-ca-cert-hash sha256:xxx 49 | ``` 50 | Check on the Leader master that the worker nodes have correctly joined the cluster: 51 | ``` 52 | kubectl get no 53 | ``` 54 | Other useful commands: 55 | ``` 56 | kubectl api-resources 57 | kubectl create ns ns1 58 | kubectl create ns ns2 59 | kubectl api-resources | grep netpol 60 | 61 | ``` 62 | 63 | -------------------------------------------------------------------------------- /caltech/2022-05-28.md: -------------------------------------------------------------------------------- 1 | ``` 2 | ltrace sleep 10 3 | ``` 4 | 1. https://docs.docker.com/ 5 | 2. https://docs.docker.com/get-docker/ 6 | 3. https://docs.docker.com/desktop/linux/install/ 7 | 4. https://docs.docker.com/desktop/linux/install/ubuntu/ 8 | 5. https://docs.docker.com/engine/install/ubuntu/ 9 | ``` 10 | sudo docker pull nginx 11 | sudo docker run nginx 12 | sudo docker run --publish 80:80 nginx 13 | ``` 14 | ``` 15 | sudo ls /var/lib/docker/overlay2/ 16 | sudo docker images 17 | sudo docker inspect nginx 18 | sudo docker history nginx 19 | sudo docker inspect nginx | grep Layers -A7 20 | sudo docker history nginx | grep -v 0B 21 | ``` 22 | ``` 23 | sudo docker network create nginx-network 24 | sudo docker run --network nginx-network --publish 80:80 nginx 25 | ``` 26 | ``` 27 | sudo docker ps --all 28 | sudo docker container prune --force 29 | sudo docker rmi nginx 30 | sudo docker image prune --force 31 | ``` 32 | ``` 33 | sudo docker run --publish 80:80 nginx 34 | sudo docker run --detach --publish 80:80 nginx 35 | sudo docker ps 36 | sudo docker rm --force exciting_maxwell 37 | sudo docker run --detach --name nginx-container --publish 80:80 nginx 38 | sudo docker logs nginx-container 39 | sudo ls /var/lib/docker/containers/1a2ea7dd41be352b261dab15c7f1020f87874ef23e3baedd9444ab35b6131cbe/ 40 | ``` 41 | 1. https://docs.docker.com/config/containers/logging/configure/ 42 | ``` 43 | sudo docker swarm init 44 | sudo docker node ls 45 | sudo docker swarm join --token SWMTKN-1-xxx-yyy 172.31.12.103:2377 46 | sudo docker node ls 47 | sudo docker service create --mode replicated --publish 80:80 --replicas 6 nginx 48 | hostname --ip-address 49 | sudo docker service ls 50 | sudo docker service logs youthful_archimedes 51 | sudo docker inspect youthful_archimedes 52 | sudo docker service rm youthful_archimedes 53 | sudo docker service create --mode global --publish 80:80 nginx 54 | sudo docker service ls 55 | sudo docker service rm ecstatic_einstein 56 | ``` 57 | ``` 58 | sudo docker network ls 59 | sudo docker network create nginx-network --driver overlay 60 | sudo docker service create --mode global --network nginx-network --publish 80:80 nginx 61 | sudo docker service ls 62 | sudo docker service rm relaxed_euclid 63 | sudo docker container prune --force 64 | sudo docker image prune --force 65 | sudo docker network prune --force 66 | ``` 67 | ``` 68 | mkdir phpinfo 69 | tee phpinfo/index.php 0< 71 | EOF 72 | sudo docker run --publish 80:80 --volume ${PWD}/phpinfo/index.php:/index.php php -f index.php -S 0.0.0.0:80 73 | ``` 74 | ``` 75 | tee phpinfo/Dockerfile 0< 24 | 25 | EOF 26 | ``` 27 | 28 | ``` 29 | sudo apt update 30 | 31 | sudo apt install php --assume-yes 32 | ``` 33 | 34 | ``` 35 | php -f phpinfo/index.php -S localhost:8080 36 | ``` 37 | Go to this location to see the resulting web page: 38 | 1. http://localhost:8080/phpinfo/ 39 | 40 | # Introduction to Docker 41 | 42 | 1. https://hub.docker.com/ 43 | 44 | ``` 45 | sudo docker version 46 | ``` 47 | ``` 48 | service docker status 49 | ``` 50 | # Example of containerization of our PHP sample application 51 | 52 | Let us first create a network for our container: 53 | ``` 54 | sudo docker network create phpinfo-network --driver bridge 55 | ``` 56 | Let us download the Docker image from Docker Hub: 57 | ``` 58 | sudo docker pull index.docker.io/library/php:alpine@sha256:ab23b416d86aec450ee7b75727f6bbec272edc2764a1b6fad13bc2823c59bb6b 59 | 60 | sudo docker images 61 | ``` 62 | Let us see all the options for docker run command: 63 | ``` 64 | sudo docker run --help 65 | ``` 66 | Let us create the container: 67 | ``` 68 | sudo docker run --cpus 0.100 --detach --entrypoint php --env AUTHOR=Sebastian --expose 8080 --memory 100M --memory-reservation 100M --name phpinfo --network phpinfo-network --read-only --restart always --user nobody:nogroup --volume ${PWD}/phpinfo/:/var/data/:ro --workdir /var/data/ index.docker.io/library/php:alpine@sha256:ab23b416d86aec450ee7b75727f6bbec272edc2764a1b6fad13bc2823c59bb6b -f index.php -S 0.0.0.0:8080 69 | ``` 70 | ## Troubleshooting the created container: 71 | 72 | View the table of processes running inside you container 73 | ``` 74 | sudo docker top phpinfo 75 | ``` 76 | View the logs of your container: 77 | ``` 78 | sudo docker logs phpinfo 79 | ``` 80 | Show the resources consumption statistics of your container: 81 | ``` 82 | sudo docker stats phpinfo --no-stream 83 | ``` 84 | Show the content of the working directory: 85 | ``` 86 | sudo docker exec phpinfo ls -l 87 | ``` 88 | Test the connection to the webserver: 89 | ``` 90 | sudo docker exec phpinfo curl localhost:8080 -I -s 91 | ``` 92 | In order to remove the container (you will need to remove the container before creating another container with the same name): 93 | ``` 94 | sudo docker rm phpinfo --force 95 | ``` 96 | -------------------------------------------------------------------------------- /caltech_2022-08/2022-08-06.md: -------------------------------------------------------------------------------- 1 | 1. https://en.wikipedia.org/wiki/Linux_namespaces 2 | 2. https://en.wikipedia.org/wiki/Cgroups 3 | 3. https://en.wikipedia.org/wiki/Docker_(software) 4 | ``` 5 | cat /etc/os-release 6 | strings /proc/1/cmdline 7 | strings /proc/1/cgroup 8 | ls /proc/ 9 | top 10 | man proc 11 | strings /proc/2239/cmdline 12 | strings /proc/2239/cgroup 13 | ``` 14 | ``` 15 | docker run --detach --entrypoint sleep --name test docker.io/library/busybox:latest infinity 16 | docker top test 17 | docker exec test strings /proc/1/cmdline 18 | docker exec test strings /proc/1/cgroup 19 | docker ps --no-trunc 20 | ``` 21 | ``` 22 | docker run --detach --entrypoint sleep --name test2 docker.io/library/busybox:latest infinity 23 | docker top test2 24 | docker exec test2 strings /proc/1/cmdline 25 | docker exec test2 strings /proc/1/cgroup 26 | docker ps --no-trunc 27 | ``` 28 | ``` 29 | service docker status 30 | docker ps 31 | ``` 32 | 1. https://docs.docker.com/engine/install/ 33 | 2. https://docs.docker.com/engine/install/ubuntu/ 34 | 3. https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user 35 | ``` 36 | sudo groupadd docker 37 | sudo usermod -aG docker $USER 38 | newgrp docker 39 | ``` 40 | ``` 41 | docker diff test 42 | docker exec test touch sebastian 43 | docker diff test 44 | sudo find /var/lib/docker/ | grep sebastian 45 | ``` 46 | ``` 47 | touch SEBASTIAN 48 | docker exec test touch sebastian 49 | sudo find / 2> /dev/null | grep -E "SEBASTIAN|sebastian" 50 | docker exec test find / | grep -E "SEBASTIAN|sebastian" 51 | ``` 52 | Check in the output of the following commands that both files have the same inode so they are stored only once: 53 | ``` 54 | sudo stat /var/lib/docker/overlay2/f0fa42db9b2a7dd85250fcc718ba31c4784a605aacbe99b325ff89dc9408ce6f/diff/sebastian 55 | sudo stat /var/lib/docker/overlay2/f0fa42db9b2a7dd85250fcc718ba31c4784a605aacbe99b325ff89dc9408ce6f/merged/sebastian 56 | ``` 57 | ``` 58 | which sleep 59 | stat /bin/sleep 60 | ``` 61 | Check in the output of the following commands that both binaries have the same inode so that both containers are sharing the same object: 62 | ``` 63 | docker exec test which sleep 64 | docker exec test stat /bin/sleep 65 | docker exec test2 which sleep 66 | docker exec test2 stat /bin/sleep 67 | ``` 68 | 1. https://camo.githubusercontent.com/dacf0edbdc66abe64ebca0769a154d62fddbfa3a/68747470733a2f2f74686570726163746963616c6465762e73332e616d617a6f6e6177732e636f6d2f692f6f62356a6963776a376b6c716179346176656b682e706e67 69 | 2. https://www.oreilly.com/library/view/getting-started-with/9781838645700/assets/dfc9cf05-7ad2-4f58-87a4-4702cd72dbbc.jpg 70 | ``` 71 | sudo ls /var/lib/docker/overlay2/ 72 | ``` 73 | Examples of Dockerfiles: 74 | 1. https://github.com/sebastian-colomar/spring-petclinic/blob/main/Dockerfile 75 | 2. https://github.com/sebastian-colomar/phpinfo/blob/2022-01/Dockerfile 76 | 3. https://github.com/sebastian-colomar/dockercoins/blob/main/hasher/Dockerfile 77 | 4. https://github.com/sebastian-colomar/dockercoins/blob/main/rng/Dockerfile 78 | 5. https://github.com/sebastian-colomar/dockercoins/blob/main/webui/Dockerfile 79 | 6. https://github.com/sebastian-colomar/dockercoins/blob/main/worker/Dockerfile 80 | 7. https://github.com/nginxinc/docker-nginx/blob/master/mainline/debian/Dockerfile 81 | -------------------------------------------------------------------------------- /caltech_2022-08/2022-08-13.md: -------------------------------------------------------------------------------- 1 | 1. https://en.wikipedia.org/wiki/Cgroups 2 | ``` 3 | strings /proc/1/cmdline 4 | strings /proc/1/cgroup 5 | strings /proc/125/cgroup 6 | strings /proc/126/cgroup 7 | top 8 | ``` 9 | ``` 10 | docker ps 11 | docker top registry 12 | strings /proc/$( docker top registry | tail -1 | awk '{ print $2 }' )/cgroup 13 | ``` 14 | 1. https://en.wikipedia.org/wiki/Linux_namespaces 15 | ``` 16 | sudo find /var/lib/docker/overlay2/ 17 | docker exec registry touch sebastian.txt 18 | sudo find /var/lib/docker/overlay2/ | grep sebastian.txt 19 | ``` 20 | ``` 21 | docker top registry 22 | ps -f -p $( docker top registry | tail -1 | awk '{ print $2 }' ) 23 | docker exec registry ps -f 24 | ``` 25 | ``` 26 | ls /proc/ | more 27 | docker exec registry ls /proc/ | more 28 | ``` 29 | ``` 30 | ifconfig 31 | docker exec registry ifconfig 32 | ``` 33 | ``` 34 | docker images 35 | docker inspect registry:2 36 | docker history registry:2 37 | sudo find /var/lib/docker/overlay2/ 38 | ``` 39 | 1. https://github.com/sebastian-colomar/Circabc-OSS/blob/docker/Dockerfile.tomcat-full 40 | 1. https://en.wikipedia.org/wiki/LAPDm 41 | 2. https://en.wikipedia.org/wiki/Bonding_protocol 42 | 3. https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html-single/networking_guide/index#ch-Configure_Network_Bonding 43 | ``` 44 | docker network ls 45 | ``` 46 | ``` 47 | docker run --detach --name test-none --network none --tty docker.io/library/busybox:latest 48 | docker exec test-none ifconfig 49 | docker exec test-none ping localhost -c3 50 | docker exec test-none ping 8.8.8.8 51 | ``` 52 | ``` 53 | docker run --detach --name test-bridge --network bridge --tty docker.io/library/busybox:latest 54 | docker exec test-bridge ifconfig 55 | docker exec test-bridge ping localhost -c1 56 | docker exec test-bridge ping 8.8.8.8 -c1 57 | docker exec test-bridge ping google.com -c1 58 | docker network disconnect --force bridge test-bridge 59 | docker exec test-bridge ifconfig 60 | docker exec test-bridge ping localhost -c1 61 | docker exec test-bridge ping 8.8.8.8 -c1 62 | docker network connect bridge test-bridge 63 | docker exec test-bridge ping 8.8.8.8 -c1 64 | ``` 65 | ``` 66 | docker run --detach --name test-host --network host --tty docker.io/library/busybox:latest 67 | docker exec test-host ifconfig 68 | ``` 69 | ``` 70 | docker inspect bridge 71 | docker exec registry ping 172.17.0.4 -c1 72 | ``` 73 | ``` 74 | docker network create --driver bridge custom-bridge 75 | docker network ls 76 | docker run --detach --name test-bridge-custom-1 --network custom-bridge --tty docker.io/library/busybox:latest 77 | docker run --detach --name test-bridge-custom-2 --network custom-bridge --tty docker.io/library/busybox:latest 78 | docker inspect custom-bridge 79 | docker exec test-bridge-custom-1 ping 172.18.0.3 -c1 80 | docker exec test-bridge-custom-1 ping 172.17.0.3 -c1 81 | docker exec test-bridge ping 172.17.0.3 -c1 82 | docker exec test-bridge-custom-1 ping test-bridge-custom-2 -c1 83 | docker exec test-bridge ping registry -c1 84 | docker network connect bridge test-bridge-custom-1 85 | docker exec test-bridge ping 172.17.0.3 -c1 86 | docker exec test-bridge-custom-1 ifconfig 87 | docker network disconnect bridge test-bridge-custom-1 88 | docker exec test-bridge-custom-1 ifconfig 89 | docker exec test-bridge-custom-1 ping 172.17.0.3 -c1 90 | ``` 91 | -------------------------------------------------------------------------------- /caltech_2022-11/2022-11-06.md: -------------------------------------------------------------------------------- 1 | # REGARDING ROOT USER AND CONTAINER OWNERSHIP 2 | ``` 3 | sudo docker container run --tty library/php:alpine id 4 | 5 | sudo docker container run --tty --user root:root library/php:alpine id 6 | 7 | sudo docker container run --tty --user nobody:nogroup library/php:alpine id 8 | ``` 9 | * https://docs.docker.com/engine/install/linux-postinstall/ 10 | 11 | # DOCKERFILE VS DOCKER RUN 12 | 13 | ``` 14 | sudo docker container run --env AUTHOR=Sebastian --entrypoint php --expose 8080 --health-cmd 'php -v' --label OWNER=me --stop-signal SIGTERM --user nobody:nogroup --volume ${PWD}:/data/ --workdir /data/ library/php:alpine -v 15 | 16 | sudo docker container run --env AUTHOR=Sebastian --entrypoint printenv --expose 8080 --health-cmd 'php -v' --label OWNER=me --stop-signal SIGTERM --user nobody:nogroup --volume ${PWD}:/data/ --workdir /data/ library/php:alpine | grep AUTHOR 17 | 18 | sudo docker container run --env AUTHOR=Sebastian --entrypoint whoami --expose 8080 --health-cmd 'php -v' --label OWNER=me --stop-signal SIGTERM --user nobody:nogroup --volume ${PWD}:/data/ --workdir /data/ library/php:alpine 19 | 20 | sudo docker container run --env AUTHOR=Sebastian --entrypoint pwd --expose 8080 --health-cmd 'php -v' --label OWNER=me --stop-signal SIGTERM --user nobody:nogroup --volume ${PWD}:/data/ --workdir /data/ library/php:alpine 21 | 22 | sudo docker container run --env AUTHOR=Sebastian --entrypoint ping --expose 8080 --health-cmd 'php -v' --label OWNER=me --stop-signal SIGTERM --user nobody:nogroup --volume ${PWD}:/data/ --workdir /data/ library/php:alpine localhost 23 | 24 | sudo docker container run --env AUTHOR=Sebastian --expose 8080 --health-cmd 'php -v' --label OWNER=me --stop-signal SIGTERM --user nobody:nogroup --volume ${PWD}:/data/ --workdir /data/ library/php:alpine ping localhost 25 | 26 | sudo docker container run --env AUTHOR=Sebastian --expose 8080 --health-cmd 'php -v' --label OWNER=me --stop-signal SIGTERM --user nobody:nogroup --volume ${PWD}:/data/ --workdir /data/ library/php:alpine php -v 27 | ``` 28 | 29 | # DOCKERFILE VS IMAGE LAYERS 30 | ``` 31 | mkdir --parents layers-test/ 32 | 33 | tee layers-test/Dockerfile 0<' | sudo docker secret create index.php - 45 | sudo docker secret ls 46 | sudo docker secret inspect index.php 47 | sudo docker service create --entrypoint php --name phpinfo-secret --publish 8080 --read-only --replicas 2 --restart-condition any --secret source=index.php,target=/app/index.php,mode=0400,uid=65534 --user nobody --workdir /app/ php -f index.php -S 0.0.0.0:8080 48 | ``` 49 | ``` 50 | echo '' | sudo docker config create index.php - 51 | sudo docker config ls 52 | sudo docker config inspect index.php 53 | sudo docker service create --entrypoint php --mode replicated --name phpinfo-config --publish 8080 --read-only --replicas 2 --restart-condition any --config source=index.php,target=/app/index.php,mode=0400,uid=65534 --user nobody --workdir /app/ php -f index.php -S 0.0.0.0:8080 54 | ``` 55 | ``` 56 | # docker-compose.yaml 57 | # php -f index.php -S 0.0.0.0:8080 58 | # sudo docker run --detach --entrypoint php --name phpinfo --publish 8080 docker.io/library/php:latest -f index.php -S 0.0.0.0:8080 59 | # sudo docker service create --entrypoint php --mode replicated --name phpinfo --publish 8080 --read-only --replicas 2 --restart-condition any --secret source=index.php,target=/app/index.php,mode=0400,uid=65534 --user nobody --workdir /app/ php -f index.php -S 0.0.0.0:8080 60 | # sudo docker stack deploy --compose-file docker-compose.yaml PHPINFO 61 | secrets: 62 | index.php: 63 | external: true 64 | services: 65 | phpinfo: 66 | deploy: 67 | mode: replicated 68 | replicas: 2 69 | restart_policy: 70 | condition: any 71 | command: 72 | - -f 73 | - index.php 74 | - -S 75 | - 0.0.0.0:8080 76 | entrypoint: 77 | - php 78 | image: php 79 | ports: 80 | - 8080 81 | read_only: true 82 | secrets: 83 | - source: index.php 84 | target: /app/index.php 85 | mode: 0400 86 | uid: '65534' 87 | user: nobody 88 | working_dir: /app/ 89 | version: '3.8' 90 | ``` 91 | ``` 92 | sudo docker stack ls 93 | sudo docker stack services PHPINFO 94 | sudo docker stack ps PHPINFO 95 | sudo docker service logs PHPINFO_phpinfo 96 | ``` 97 | -------------------------------------------------------------------------------- /caltech/2022-05-29.md: -------------------------------------------------------------------------------- 1 | 1. https://hub.docker.com/ 2 | ``` 3 | sudo docker tag php:phpinfo academiaonline/php:phpinfo 4 | sudo docker push academiaonline/php:phpinfo 5 | ``` 6 | ``` 7 | sudo docker service ls 8 | sudo docker service rm confident_sutherland 9 | sudo docker service create --mode global --publish 80:80 academiaonline/php:phpinfo -f index.php -S 0.0.0.0:80 10 | ``` 11 | ``` 12 | sudo docker run --detach --publish 5000:5000 --restart always registry:2 13 | sudo docker tag php:phpinfo localhost:5000/php:phpinfo 14 | hostname --ip-address 15 | sudo tee /etc/docker/daemon.json 0< 30 | ``` 31 | ``` 32 | sudo apt-get update 33 | sudo apt-get install php -y 34 | mkdir php/ 35 | tee php/index.php 0< 37 | EOF 38 | php -f php/index.php -S 0.0.0.0:8080 39 | # -f Parse and execute 40 | # -S : Run with built-in web server 41 | ``` 42 | ``` 43 | tee php/Dockerfile 0< 77 | EOF 78 | docker config create php-config php/index.php 79 | docker service create --config source=php-config,target=index.php --entrypoint php --name php-svc-3 --publish 8080 docker.io/library/php:alpine -f index.php -S 0.0.0.0:8080 80 | docker secret create php-secret php/index.php 81 | docker service create --secret source=php-secret,target=index.php --entrypoint php --name php-svc-4 --publish 8080 docker.io/library/php:alpine -f index.php -S 0.0.0.0:8080 82 | docker service create --secret source=php-secret,target=/index.php --entrypoint php --name php-svc-5 --publish 8080 docker.io/library/php:alpine -f index.php -S 0.0.0.0:8080 83 | ``` 84 | -------------------------------------------------------------------------------- /2022-04-30.md: -------------------------------------------------------------------------------- 1 | ``` 2 | sudo docker inspect bridge 3 | sudo docker run --detach --entrypoint ping --name ping docker.io/library/busybox:latest localhost 4 | sudo docker logs ping 5 | sudo docker top ping 6 | sudo docker stats ping --no-stream 7 | sudo docker ps 8 | sudo docker exec ping ls /var/ 9 | sudo docker inspect bridge 10 | sudo docker network create your_bridge 11 | sudo docker network ls 12 | sudo docker inspect your_bridge 13 | sudo docker run --detach --entrypoint ping --name your_ping --network your_bridge docker.io/library/busybox:latest localhost 14 | sudo docker inspect your_bridge 15 | 16 | ``` 17 | ``` 18 | sudo docker run --entrypoint ls --rm busybox:latest /var/ 19 | sudo docker run --detach --entrypoint ls --rm busybox:latest /var/ 20 | ``` 21 | ``` 22 | sudo docker service create --entrypoint python --name webserver --publish 8000 docker.io/library/python:alpine -m http.server 23 | sudo docker service scale webserver=6 24 | sudo docker service ls 25 | sudo docker service logs webserver 26 | sudo docker service create --entrypoint ping --name ping docker.io/library/busybox:latest localhost 27 | sudo docker service scale ping=6 28 | sudo docker service logs ping 29 | sudo docker service ps ping 30 | sudo docker ps 31 | ``` 32 | ``` 33 | sudo docker container inspect ping 34 | sudo docker network ls 35 | sudo docker inspect bridge 36 | sudo docker inspect your_bridge 37 | sudo docker network connect your_bridge ping 38 | sudo docker inspect your_bridge 39 | sudo docker container inspect ping 40 | sudo docker network disconnect bridge ping 41 | sudo docker container inspect ping 42 | 43 | ``` 44 | ``` 45 | sudo docker ps | grep webserver ; sudo docker service ps webserver 46 | sudo docker rm --force webserver.4.un5shfao1660zz7ad3t1cfpgm ; sudo docker ps | grep webserver ; sudo docker service ps webserver 47 | ``` 48 | ``` 49 | sudo docker service create --entrypoint ping --env OWNER=Sebastian --name ping-with-env --replicas 6 docker.io/library/busybox:latest localhost 50 | sudo docker exec ping-with-env.6.ptaiyyleyn9jzkygfnfy4sp03 printenv | grep ^OWNER 51 | 52 | ``` 53 | ``` 54 | sudo docker network ls 55 | sudo docker run --detach --entrypoint ping --name no-net --network none --rm busybox localhost 56 | sudo docker inspect no-net 57 | sudo docker exec no-net ip address 58 | sudo docker run --detach --entrypoint ping --name host-net --network host --rm busybox localhost 59 | sudo docker inspect host-net 60 | sudo docker exec host-net ip address 61 | sudo docker exec host-net ip route 62 | sudo docker network prune --force 63 | ``` 64 | ``` 65 | sudo docker stats 66 | ``` 67 | ``` 68 | sudo docker service create --entrypoint ping --mode global --name ping-global busybox localhost 69 | sudo docker service ps ping-global 70 | ``` 71 | ``` 72 | sudo docker node ls 73 | sudo docker node demote ip-172-31-1-138 74 | sudo docker node promote ip-172-31-1-138 75 | sudo docker node demote ip-172-31-12-249 76 | sudo docker node demote ip-172-31-2-239 77 | ``` 78 | 1. https://techcommunity.microsoft.com/t5/image/serverpage/image-id/100302iA949F8A130209F6E/image-size/large?v=v2&px=999 79 | 2. https://docs.docker.com/compose/compose-file/compose-file-v3/ 80 | ``` 81 | # sudo docker service create --entrypoint python --name webserver --publish 8000 docker.io/library/python:alpine -m http.server 82 | # sudo docker stack deploy --compose-file docker-compose.yaml WEBSERVER 83 | version: '3.8' 84 | services: 85 | webserver: 86 | deploy: 87 | replicas: 6 88 | entrypoint: 89 | - python 90 | ports: 91 | - 8000 92 | image: docker.io/library/python:alpine 93 | command: 94 | - -m 95 | - http.server 96 | ``` 97 | ``` 98 | sudo docker stack ls 99 | sudo docker stack ps WEBSERVER 100 | sudo docker stack services WEBSERVER 101 | sudo docker service logs WEBSERVER_webserver 102 | 103 | ``` 104 | ``` 105 | # sudo docker service create --entrypoint ping --mode global --name ping-global busybox localhost 106 | # sudo docker stack deploy --compose-file ping-global.yaml PING_GLOBAL 107 | version: '3.8' 108 | services: 109 | ping-global: 110 | entrypoint: 111 | - ping 112 | image: busybox 113 | command: 114 | - localhost 115 | deploy: 116 | mode: global 117 | ``` 118 | -------------------------------------------------------------------------------- /swarm-example.md: -------------------------------------------------------------------------------- 1 | 1. https://labs.play-with-docker.com/ 2 | ``` 3 | docker swarm init --advertise-addr $( hostname -i ) 4 | docker swarm join-token worker 5 | docker swarm join-token manager 6 | docker node ls 7 | ``` 8 | ``` 9 | mkdir php/ 10 | tee php/index.php 0< 12 | EOF 13 | docker config create php-config php/index.php 14 | docker service create --config source=php-config,target=index.php --entrypoint php --name php-svc-3 --publish 8080 docker.io/library/php:alpine -f index.php -S 0.0.0.0:8080 15 | docker secret create php-secret php/index.php 16 | docker service create --secret source=php-secret,target=index.php --entrypoint php --name php-svc-4 --publish 8080 docker.io/library/php:alpine -f index.php -S 0.0.0.0:8080 17 | docker service create --secret source=php-secret,target=/index.php --entrypoint php --name php-svc-5 --publish 8080 docker.io/library/php:alpine -f index.php -S 0.0.0.0:8080 18 | ``` 19 | ``` 20 | docker node ls 21 | docker service ls 22 | docker service ps php-svc-4 23 | docker service logs php-svc-4 24 | docker ps 25 | docker exec php-svc-4.1.lx3fu84r3mble8snpzbz3fx7k df 26 | ``` 27 | ``` 28 | docker service update --secret-rm php-secret php-svc-4 29 | docker service ps php-svc-4 30 | docker ps 31 | docker exec php-svc-4.1.vus6clyku7qfsocjffefiwiwr df 32 | docker service update --secret-add source=php-secret,target=/index.php php-svc-4 33 | docker service ps php-svc-4 34 | docker ps 35 | docker exec php-svc-4.1.253x45roucx7v7utczpl14n2s df 36 | ``` 37 | ``` 38 | docker service scale php-svc-3=5 39 | docker service ps php-svc-3 40 | docker service ls 41 | docker ps 42 | docker rm --force php-svc-3.1.boqyufqmrk4rc78ga9w176qa7 ; docker ps 43 | docker service ps php-svc-3 44 | docker restart php-svc-3.1.iujff6vlnhphgzga2dsu4mexe 45 | docker service ps php-svc-3 46 | docker node update --availability drain node4 47 | docker service ps php-svc-3 48 | docker node update --availability active node4 49 | docker service ps php-svc-3 50 | ``` 51 | 1. https://docs.docker.com/compose/compose-file/compose-file-v3/ 52 | ``` 53 | mkdir -p php/ 54 | tee php/index.php 0< 56 | EOF 57 | ``` 58 | ``` 59 | tee php/docker-compose.yaml 0< 12 | EOF 13 | docker config create php-config php/index.php 14 | docker service create --config source=php-config,target=index.php --entrypoint php --name php-svc-3 --publish 8080 docker.io/library/php:alpine -f index.php -S 0.0.0.0:8080 15 | docker secret create php-secret php/index.php 16 | docker service create --secret source=php-secret,target=index.php --entrypoint php --name php-svc-4 --publish 8080 docker.io/library/php:alpine -f index.php -S 0.0.0.0:8080 17 | docker service create --secret source=php-secret,target=/index.php --entrypoint php --name php-svc-5 --publish 8080 docker.io/library/php:alpine -f index.php -S 0.0.0.0:8080 18 | ``` 19 | ``` 20 | docker node ls 21 | docker service ls 22 | docker service ps php-svc-4 23 | docker service logs php-svc-4 24 | docker ps 25 | docker exec php-svc-4.1.lx3fu84r3mble8snpzbz3fx7k df 26 | ``` 27 | ``` 28 | docker service update --secret-rm php-secret php-svc-4 29 | docker service ps php-svc-4 30 | docker ps 31 | docker exec php-svc-4.1.vus6clyku7qfsocjffefiwiwr df 32 | docker service update --secret-add source=php-secret,target=/index.php php-svc-4 33 | docker service ps php-svc-4 34 | docker ps 35 | docker exec php-svc-4.1.253x45roucx7v7utczpl14n2s df 36 | ``` 37 | ``` 38 | docker service scale php-svc-3=5 39 | docker service ps php-svc-3 40 | docker service ls 41 | docker ps 42 | docker rm --force php-svc-3.1.boqyufqmrk4rc78ga9w176qa7 ; docker ps 43 | docker service ps php-svc-3 44 | docker restart php-svc-3.1.iujff6vlnhphgzga2dsu4mexe 45 | docker service ps php-svc-3 46 | docker node update --availability drain node4 47 | docker service ps php-svc-3 48 | docker node update --availability active node4 49 | docker service ps php-svc-3 50 | ``` 51 | 1. https://docs.docker.com/compose/compose-file/compose-file-v3/ 52 | ``` 53 | mkdir -p php/ 54 | tee php/index.php 0< 56 | EOF 57 | ``` 58 | ``` 59 | tee php/docker-compose.yaml 0< 5 | EOF 6 | sudo docker run --publish 80:80 --read-only --user nobody:nogroup --volume ${PWD}/phpinfo/index.php:/data/index.php:ro --workdir /data/ php -f index.php -S 0.0.0.0:80 7 | ``` 8 | ``` 9 | tee phpinfo/Dockerfile 0<' | tee index.php 24 | php -f index.php -S 0.0.0.0:9000 25 | sudo docker run --detach --entrypoint php --name phpinfo --publish 9000 docker.io/library/php:alpine -f index.php -S 0.0.0.0:9000 26 | sudo docker ps 27 | sudo docker rm --force phpinfo 28 | sudo docker run --detach --entrypoint php --name phpinfo --publish 9000 --volume ${PWD}/index.php:/index.php:ro docker.io/library/php:alpine -f index.php -S 0.0.0.0:9000 29 | sudo docker run --detach --entrypoint php --name phpinfo-data --publish 9000 --volume ${PWD}/index.php:/data/index.php:ro --workdir /data/ docker.io/library/php:alpine -f index.php -S 0.0.0.0:9000 30 | ``` 31 | ``` 32 | docker network ls 33 | docker node ls 34 | docker service ls 35 | docker stack ls 36 | docker swarm init 37 | docker swarm leave 38 | docker volume ls 39 | ``` 40 | ``` 41 | docker build 42 | docker cp 43 | docker diff 44 | docker exec 45 | docker history 46 | docker images 47 | docker inspect 48 | docker kill 49 | docker login 50 | docker logout 51 | docker logs 52 | docker ps 53 | docker pull 54 | docker push 55 | docker rename 56 | docker restart 57 | docker rm 58 | docker rmi 59 | docker run 60 | docker stats 61 | docker tag 62 | docker top 63 | docker version 64 | ``` 65 | 1. https://docs.docker.com/ 66 | 2. https://docs.docker.com/get-docker/ 67 | 3. https://docs.docker.com/engine/install/ 68 | ``` 69 | sudo apt-get remove docker docker-engine docker.io containerd runc 70 | sudo apt-get update 71 | sudo apt-get install ca-certificates curl gnupg lsb-release -y 72 | curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg 73 | echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null 74 | sudo apt-get update 75 | sudo apt-get install docker-ce docker-ce-cli containerd.io -y 76 | ``` 77 | ``` 78 | sudo docker run --entrypoint which --rm docker.io/library/busybox:latest php 79 | sudo docker run --entrypoint which --rm docker.io/library/php:alpine php 80 | ``` 81 | 1. https://hub.docker.com/ 82 | ``` 83 | sudo docker run --entrypoint which --rm docker.io/library/alpine:latest php 84 | sudo docker run --entrypoint which --rm docker.io/library/php:alpine php 85 | mkdir Builder 86 | tee ./Builder/Dockerfile 0<&1 | tee --append ${log} 77 | sudo kubectl apply --filename ${calico} --kubeconfig ${kubeconfig} 2>&1 | tee --append ${log} 78 | mkdir -p ${HOME}/.kube 79 | sudo cp /etc/kubernetes/admin.conf ${HOME}/.kube/config 80 | sudo chown -R $( id -u ):$( id -g ) ${HOME}/.kube/ 81 | echo 'source <(kubectl completion bash)' | tee --append ${HOME}/.bashrc 82 | source ${HOME}/.bashrc 83 | while true ; do kubectl get no | grep Ready | grep --invert-match NotReady && break ; sleep ${sleep} ; done 84 | sudo sed --in-place /${kube}/d /etc/hosts ; sudo sed --in-place /127.0.0.1.*localhost/s/$/' '${kube}/ /etc/hosts 85 | grep ^kubeadm.join /tmp/install-leader.log -A1 86 | master=$( kubectl get node | grep master | awk '{ print $1 }' ) 87 | kubectl taint node ${master} node-role.kubernetes.io/master:NoSchedule- 88 | ``` 89 | ``` 90 | python3 -m http.server 91 | curl localhost:8000 92 | ``` 93 | ``` 94 | sudo docker run --publish 8000:8000 python python3 -m http.server 95 | curl localhost:8000 96 | ``` 97 | ``` 98 | sudo docker service create --name webserver --publish 8000:8000 python python3 -m http.server 99 | curl $( hostname --ip-address ):8000 100 | sudo docker service rm webserver 101 | ``` 102 | ``` 103 | tee docker-compose.yaml 0<& 1 | tee kubeadm-init.log 54 | 55 | mkdir -p ${HOME}/.kube 56 | 57 | sudo cp /etc/kubernetes/admin.conf ${HOME}/.kube/config 58 | 59 | sudo chown -R $( id -u ):$( id -g ) ${HOME}/.kube/ 60 | 61 | echo 'source <(kubectl completion bash)' | tee --append ${HOME}/.bashrc 62 | 63 | source ${HOME}/.bashrc 64 | 65 | kubectl apply --filename https://docs.projectcalico.org/${calico_version}/manifests/calico.yaml 66 | 67 | watch kubectl --namespace kube-system get all 68 | ``` 69 | # UNDERSTANDING KUBERNETES NETWORKING 70 | ``` 71 | labsuser@ip-172-31-13-145:~$ kubectl --namespace phpinfo get svc 72 | NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE 73 | phpinfo-svc NodePort 10.101.240.199 80:31575/TCP 25m 74 | labsuser@ip-172-31-13-145:~$ kubectl --namespace phpinfo get ep 75 | NAME ENDPOINTS AGE 76 | phpinfo-svc 10.5.1.2:8080,10.5.2.2:8080 25m 77 | ``` 78 | # Install Mirantis Container Runtime for Ubuntu 79 | The following instructions have been extracted from this documentation: 80 | * https://docs.mirantis.com/mcr/20.10/install/mcr-linux/ubuntu.html 81 | 1. Uninstall old versions 82 | 83 | ``` 84 | sudo apt-get --yes remove docker docker-engine docker-ce docker-ce-cli docker.io 85 | ``` 86 | 1. Update the apt package index 87 | 88 | ``` 89 | sudo apt-get update 90 | ``` 91 | 1. Install packages to allow apt to use a repository over HTTPS 92 | 93 | ``` 94 | sudo apt-get install apt-transport-https ca-certificates curl software-properties-common -y 95 | ``` 96 | 1. Temporarily store https://repos.mirantis.com in an environment variable. This variable assignment does not persist when the session ends 97 | 98 | ``` 99 | DOCKER_EE_URL="https://repos.mirantis.com" 100 | ``` 101 | 1. Temporarily add a $DOCKER_EE_VERSION variable into your environment 102 | 103 | ``` 104 | DOCKER_EE_VERSION=20.10 105 | ``` 106 | 1. Add Docker’s official GPG key using your customer Mirantis Container Runtime repository URL 107 | 108 | ``` 109 | curl -fsSL "${DOCKER_EE_URL}/ubuntu/gpg" | sudo apt-key add - 110 | ``` 111 | 1. Set up the stable repository, using the following command as-is (which works due to the variable set up earlier in the process) 112 | 113 | ``` 114 | sudo add-apt-repository "deb [arch=$(dpkg --print-architecture)] $DOCKER_EE_URL/ubuntu $(lsb_release -cs) stable-$DOCKER_EE_VERSION" 115 | ``` 116 | 1. Update the apt package index 117 | ``` 118 | sudo apt-get update 119 | ``` 120 | 1. Install the latest version of Mirantis Container Runtime and containerd. Any existing installation of MCR is replaced 121 | 122 | ``` 123 | sudo apt-get --yes install docker-ee docker-ee-cli containerd.io 124 | ``` 125 | # Install the Mirantis Kubernetes Engine image 126 | The following instructions have been extracted from this documentation: 127 | * https://docs.mirantis.com/mke/3.5/install/install-mke-image.html 128 | 1. Install MKE 129 | 130 | ``` 131 | sudo docker run --interactive --name ucp --rm --tty --volume /var/run/docker.sock:/var/run/docker.sock mirantis/ucp:3.5.5 install --host-address $( hostname --ip-address ) --interactive --force-minimums 132 | ``` 133 | 134 | -------------------------------------------------------------------------------- /caltech_2022-09/2022-09-18.md: -------------------------------------------------------------------------------- 1 | # How to install the Docker engine 2 | 1. https://docs.docker.com/engine/install/ 3 | 4 | ## Install Docker Engine on Ubuntu 5 | 1. https://docs.docker.com/engine/install/ubuntu/ 6 | 7 | ### Procedure 8 | 1. First become root: 9 | 10 | ``` 11 | sudo su --login root 12 | ``` 13 | 3. Check installed OS: 14 | 15 | ``` 16 | cat /etc/os-release 17 | ``` 18 | 19 | 1. Check the architecture: 20 | 21 | ``` 22 | lscpu 23 | ``` 24 | 25 | 1. Uninstall old versions: 26 | 27 | ``` 28 | apt-get remove docker docker-engine docker.io containerd runc --assume-yes 29 | ``` 30 | 31 | 1. Update the apt package index and install packages to allow apt to use a repository over HTTPS: 32 | 33 | ``` 34 | apt-get update 35 | 36 | apt-get install ca-certificates curl gnupg lsb-release --assume-yes 37 | ``` 38 | 39 | 1. Add Docker’s official GPG key: 40 | 41 | ``` 42 | mkdir -p /etc/apt/keyrings 43 | 44 | curl -fsSL https://download.docker.com/linux/ubuntu/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg 45 | ``` 46 | 1. Use the following command to set up the repository: 47 | 48 | ``` 49 | echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null 50 | ``` 51 | 1. Update the apt package index, and install the latest version of Docker Engine, containerd, and Docker Compose: 52 | 53 | ``` 54 | apt-get update 55 | 56 | apt-get install docker-ce docker-ce-cli containerd.io docker-compose-plugin --assume-yes 57 | ``` 58 | 1. Verify the installation: 59 | 60 | ``` 61 | docker version 62 | 63 | service docker status 64 | ``` 65 | 66 | # How to build a new Docker image for our PHP sample application 67 | 68 | 1. Create the Dockerfile: 69 | 70 | ``` 71 | tee ${HOME}/phpinfo/Dockerfile 0< 132 | --- 133 | apiVersion: v1 134 | kind: Secret 135 | metadata: 136 | name: php-secret 137 | stringData: 138 | secret.php: 139 | --- 140 | apiVersion: v1 141 | kind: Service 142 | metadata: 143 | name: phpinfo-svc 144 | spec: 145 | ports: 146 | - port: 80 147 | protocol: TCP 148 | targetPort: 8080 149 | selector: 150 | app: phpinfo 151 | type: NodePort 152 | EOF 153 | ``` 154 | ``` 155 | kubectl delete --filename php/kube-compose.yaml 156 | kubectl apply --filename php/kube-compose.yaml 157 | watch kubectl get all 158 | ``` 159 | 1. https://docs.mirantis.com/mcr/20.10/install/mcr-linux/ubuntu.html 160 | 161 | ``` 162 | sudo apt-get --yes remove docker docker-engine docker-ce docker-ce-cli docker.io 163 | sudo apt-get update 164 | sudo apt-get --yes install apt-transport-https ca-certificates curl software-properties-common 165 | DOCKER_EE_URL="https://repos.mirantis.com" 166 | DOCKER_EE_VERSION=20.10 167 | curl -fsSL "${DOCKER_EE_URL}/ubuntu/gpg" | sudo apt-key add - 168 | sudo apt-key fingerprint 6D085F96 169 | sudo add-apt-repository "deb [arch=$(dpkg --print-architecture)] $DOCKER_EE_URL/ubuntu $(lsb_release -cs) stable-$DOCKER_EE_VERSION" 170 | sudo apt-get update 171 | sudo apt-get --yes install docker-ee docker-ee-cli containerd.io 172 | ``` 173 | 1. https://docs.mirantis.com/mke/3.5/install/install-mke-image.html 174 | 175 | ``` 176 | docker run --interactive --name ucp --rm --tty --volume /var/run/docker.sock:/var/run/docker.sock mirantis/ucp:3.5.5 install --host-address $( hostname --ip-address ) --interactive --force-minimums 177 | ``` 178 | -------------------------------------------------------------------------------- /docker-example.md: -------------------------------------------------------------------------------- 1 | # Introduction to Docker 2 | 3 | 1. https://en.wikipedia.org/wiki/Linux_namespaces 4 | 1. https://en.wikipedia.org/wiki/Cgroups 5 | 6 | # Shell environment 7 | - https://shell.cloud.google.com/ 8 | 9 | # PHP sample application 10 | 11 | 1. https://www.php.net/docs.php 12 | 2. https://www.php.net/manual/en/function.phpinfo 13 | 14 | Create a new project folder in your home directory: 15 | ``` 16 | mkdir --parents ${PWD}/phpinfo/ 17 | ``` 18 | Create a PHP file in the project folder: 19 | ``` 20 | tee ${PWD}/phpinfo/index.php 0< 29 | 30 | EOF 31 | ``` 32 | Execute the PHP application parsing the PHP file and launching a PHP embedded webserver: 33 | ``` 34 | php -f ${PWD}/phpinfo/index.php -S localhost:9000 & 35 | ``` 36 | Check the web server created: 37 | ``` 38 | curl localhost:9000/phpinfo/index.php -I -s 39 | ``` 40 | Find the PID of the running process: 41 | ``` 42 | ps aux|grep phpinfo --max-count 1 43 | ``` 44 | Or alternatively: 45 | ``` 46 | pidof php 47 | ``` 48 | Save this PID in an environment variable: 49 | ``` 50 | pid=$(pidof php) 51 | ``` 52 | Now you can inspect the running process: 53 | ``` 54 | ls /proc/${pid}/ 55 | ``` 56 | ``` 57 | strings /proc/${pid}/cmdline 58 | ``` 59 | ``` 60 | strings /proc/${pid}/net/fib_trie 61 | ``` 62 | 63 | # Introduction to Docker 64 | 65 | 1. https://docs.docker.com/ 66 | 2. https://hub.docker.com/ 67 | 68 | Check the version of the Docker client: 69 | ``` 70 | docker version 71 | ``` 72 | Check the status of the Docker service: 73 | ``` 74 | service docker status 75 | ``` 76 | # Example of containerization of our PHP sample application 77 | 78 | Let us first create a network for our container: 79 | ``` 80 | docker network create phpinfo --driver bridge 81 | ``` 82 | Let us download the Docker image from Docker Hub: 83 | ``` 84 | docker pull index.docker.io/library/php:alpine 85 | 86 | docker images 87 | ``` 88 | Let us see the Docker registry and the Dockerfile: 89 | - https://hub.docker.com/_/php 90 | - https://github.com/docker-library/php/blob/master/8.3/alpine3.19/cli/Dockerfile 91 | 92 | Let us see all the options for docker run command: 93 | ``` 94 | docker run --help 95 | ``` 96 | Let us create the container: 97 | ``` 98 | docker run --cpus 0.01 --detach --env AUTHOR=Sebastian --memory 20M --memory-reservation 10M --name phpinfo --network phpinfo --publish 60000:9000 --read-only --restart always --user nobody:nogroup --volume ${HOME}/phpinfo/index.php:/var/data/index.php:ro --workdir /var/data/ index.docker.io/library/php:alpine php -f index.php -S 0.0.0.0:9000 99 | ``` 100 | # Troubleshooting the Docker container: 101 | 102 | View the table of processes running inside you container 103 | ``` 104 | docker top phpinfo 105 | ``` 106 | View the logs of your container: 107 | ``` 108 | docker logs phpinfo 109 | ``` 110 | Show the resources consumption statistics of your container: 111 | ``` 112 | docker stats phpinfo --no-stream 113 | ``` 114 | Show the content of the working directory: 115 | ``` 116 | docker exec phpinfo ls -l 117 | ``` 118 | Test the connection to the webserver from inside the container: 119 | ``` 120 | docker exec phpinfo curl localhost:9000/index.php -I -s 121 | ``` 122 | Test the connection to the webserver from outside the container: 123 | ``` 124 | curl localhost:60000/index.php -I -s 125 | ``` 126 | In order to remove the container: 127 | ``` 128 | docker rm phpinfo --force 129 | ``` 130 | 131 | # How to deploy a Docker stack using a Docker compose file 132 | 133 | ``` 134 | tee ${PWD}/phpinfo/docker-compose.yaml 0< 9c6f07244728 7 | Step 2/3 : RUN apk add php 8 | ---> Using cache 9 | ---> 235724fa6651 10 | Step 3/3 : RUN apk add curl 11 | ---> Running in 35cae1b285ca 12 | fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/main/x86_64/APKINDEX.tar.gz 13 | fetch https://dl-cdn.alpinelinux.org/alpine/v3.16/community/x86_64/APKINDEX.tar.gz 14 | (1/5) Installing ca-certificates (20220614-r0) 15 | (2/5) Installing brotli-libs (1.0.9-r6) 16 | (3/5) Installing nghttp2-libs (1.47.0-r0) 17 | (4/5) Installing libcurl (7.83.1-r4) 18 | (5/5) Installing curl (7.83.1-r4) 19 | Executing busybox-1.35.0-r17.trigger 20 | Executing ca-certificates-20220614-r0.trigger 21 | OK: 19 MiB in 28 packages 22 | Removing intermediate container 35cae1b285ca 23 | ---> d457adf0b420 24 | Successfully built d457adf0b420 25 | Successfully tagged example:latest 26 | ``` 27 | # HOW TO FLATTEN A DOCKER IMAGE FROM A DOCKERFILE 28 | ``` 29 | tee ${HOME}/phpinfo/Dockerfile 0< 143 | --- 144 | apiVersion: v1 145 | kind: Secret 146 | metadata: 147 | name: phpinfo-secret 148 | stringData: 149 | secret.php: 150 | --- 151 | apiVersion: v1 152 | kind: Service 153 | metadata: 154 | name: phpinfo-svc 155 | spec: 156 | ports: 157 | - port: 80 158 | protocol: TCP 159 | targetPort: 8080 160 | selector: 161 | app: phpinfo 162 | type: NodePort 163 | 164 | EOF 165 | ``` 166 | 1. Delete any previous deployment of the same file: 167 | ``` 168 | kubectl create ns phpinfo 169 | 170 | kubectl --namespace phpinfo delete --filename kube-compose.yaml 171 | ``` 172 | 1. Deploy the compose file: 173 | 174 | ``` 175 | kubectl --namespace phpinfo apply --filename kube-compose.yaml 176 | 177 | watch kubectl --namespace phpinfo get all 178 | ``` 179 | -------------------------------------------------------------------------------- /caltech_2022-10/2022-10-08.md: -------------------------------------------------------------------------------- 1 | # How to clean your Docker environment 2 | 1. Remove all Docker containers: 3 | ``` 4 | sudo docker rm --force --volumes $( sudo docker ps --all --quiet ) 5 | ``` 6 | 1. Force the removal of all Docker images: 7 | ``` 8 | sudo docker rmi --force $( sudo docker images --all --quiet ) 9 | ``` 10 | 1. Remove the Docker volumes: 11 | 12 | ``` 13 | sudo docker volume rm --force $( sudo docker volume ls --quiet ) 14 | ``` 15 | 1. Remove all Docker networks (except the defaults): 16 | 17 | ``` 18 | sudo docker swarm leave --force 19 | 20 | sudo docker network rm $( sudo docker network ls --quiet ) 21 | ``` 22 | # Introduction to the Course 23 | 1. Containers are partitions of the kernel resources of the host machine. We mainly isolate the network, the filesystems and the processes. The technologies behind this partitioning are called Linux namespaces and control groups: 24 | * https://en.wikipedia.org/wiki/Linux_namespaces 25 | * https://en.wikipedia.org/wiki/Cgroups 26 | 27 | 3. https://docs.openshift.com/ 28 | 4. https://kubernetes.io/docs/home/ 29 | 5. https://docs.docker.com/ 30 | 31 | 32 | # PHP sample application 33 | 1. https://www.php.net/docs.php 34 | 2. https://www.php.net/manual/en/function.phpinfo 35 | 36 | ``` 37 | mkdir --parents phpinfo 38 | 39 | tee phpinfo/index.php 0< 44 | 45 | EOF 46 | ``` 47 | 48 | ``` 49 | sudo apt update 50 | 51 | sudo apt install php --assume-yes 52 | ``` 53 | 54 | ``` 55 | php -f phpinfo/index.php -S localhost:8080 56 | ``` 57 | Go to this location to see the resulting web page: 58 | 1. http://localhost:8080/phpinfo/ 59 | 60 | Open another terminal to inspect the deployment: 61 | ``` 62 | ps -a -f 63 | 64 | pidof php 65 | 66 | cat /proc/$( pidof php )/cgroup 67 | 68 | cat /proc/1/cgroup 69 | ``` 70 | Now create a simple Docker container: 71 | ``` 72 | sudo docker run --detach --name test --tty busybox 73 | ``` 74 | Check again the control group for this container: 75 | ``` 76 | sudo docker top test 77 | 78 | pidof sh 79 | 80 | cat /proc/$( pidof sh )/cgroup 81 | 82 | cat /proc/1/cgroup 83 | ``` 84 | To summarize the results: 85 | ``` 86 | cat /proc/11/cgroup 87 | 88 | cat /proc/$( pidof php )/cgroup 89 | 90 | cat /proc/$( pidof sh )/cgroup 91 | ``` 92 | # Introduction to Docker 93 | 94 | 1. https://hub.docker.com/ 95 | 96 | ``` 97 | sudo docker version 98 | ``` 99 | ``` 100 | service docker status 101 | ``` 102 | 1. Docker container: 103 | * isolated environment created using Linux namespaces and control groups (isolated partitions of the kernel resources of the host machine) 104 | 4. Docker image: 105 | * external root filesystem containing a basic operating system and the necessary libraries and dependencies 106 | 6. Docker volume: 107 | * external filesystem containing the custom data and configuration for our Docker container 108 | 8. Docker network: 109 | * network that will provide a valid IP address to our Docker container 110 | 111 | # Introduction to Docker images 112 | 1. Dockerfile of a basic Docker image containing Ubuntu operating system: 113 | 114 | * https://git.launchpad.net/cloud-images/+oci/ubuntu-base/plain/Dockerfile?h=refs/tags/dist-jammy-amd64-20221003&id=89da42508a832d893e03b5d4b606da47c3a9bc70 115 | 1. Dockerfile of Python Docker image: 116 | 117 | * https://raw.githubusercontent.com/docker-library/python/master/3.11/bullseye/Dockerfile 118 | 1. Dockerfile of basic PHP Docker image: 119 | 120 | * https://raw.githubusercontent.com/sebastian-colomar/phpinfo/main/docker/Dockerfile 121 | # Example of containerization of our PHP sample application 122 | 123 | Let us first create a Docker network for our Docker container: 124 | ``` 125 | sudo docker network ls 126 | 127 | sudo docker network create phpinfo-network --driver bridge 128 | 129 | sudo docker network ls | grep phpinfo-network 130 | ``` 131 | Let us download the Docker image from Docker Hub: 132 | ``` 133 | sudo docker images 134 | 135 | sudo docker pull index.docker.io/library/php:alpine 136 | 137 | sudo docker images | grep php.*alpine 138 | ``` 139 | This is the Dockerfile for the previously downloaded Docker image: 140 | * https://github.com/docker-library/php/blob/master/8.1/alpine3.16/cli/Dockerfile 141 | 142 | The history of the Docker image will also provide me the Dockerfile instructions in reverse order: 143 | ``` 144 | sudo docker history index.docker.io/library/php:alpine --no-trunc 145 | ``` 146 | There is another way to see the content of the Docker image through the `inspect` command: 147 | ``` 148 | sudo docker inspect index.docker.io/library/php:alpine 149 | ``` 150 | The Graph Driver section of the previous output shows the actual location inside the host filesystem of the image layers. 151 | The Lower Directory contains the superposition of the immutable lower layers in reverse order. 152 | 153 | The last folder in the Lower Directory sections contains the first layer of the Docker image corresponding to the first line of the Docker file (that is the Alpine operating system base image). That first layer does not contain the following binaries: curl, xz, openssl. 154 | 155 | These packages were installed by the second line of the Dockerfile and they are therefore contained inside the second layer (in reverse order) of the Lower Directory as can be checked with the following commands: 156 | ``` 157 | $ sudo find /var/lib/docker/overlay2/f7a83dc729b7d554ce1c1e2ff567407720377f177f49dbb704a47c6343a0cc2f/diff | grep -E "bin/curl$|bin/xz$|bin/openssl$" 158 | ``` 159 | ``` 160 | # RUN apk add curl xz openssl 161 | $ sudo find /var/lib/docker/overlay2/fe353517eea371162ce8cb8b25dbb3966b77043cca4c761268be28fbc1fb1795/diff | grep -E "bin/curl$|bin/xz$|bin/openssl$" 162 | /var/lib/docker/overlay2/fe353517eea371162ce8cb8b25dbb3966b77043cca4c761268be28fbc1fb1795/diff/usr/bin/xz 163 | /var/lib/docker/overlay2/fe353517eea371162ce8cb8b25dbb3966b77043cca4c761268be28fbc1fb1795/diff/usr/bin/curl 164 | /var/lib/docker/overlay2/fe353517eea371162ce8cb8b25dbb3966b77043cca4c761268be28fbc1fb1795/diff/usr/bin/openssl 165 | ``` 166 | -------------------------------------------------------------------------------- /caltech_2022-08/2022-08-07.md: -------------------------------------------------------------------------------- 1 | ``` 2 | docker ps 3 | docker ps --all 4 | docker start test 5 | ``` 6 | ``` 7 | docker top test 8 | docker exec --interactive --tty test top 9 | docker inspect test 10 | ``` 11 | ``` 12 | mkdir test/ 13 | tee test/Dockerfile 0< 64 | EOF 65 | php -f php/index.php -S 0.0.0.0:8080 66 | # -f Parse and execute 67 | # -S : Run with built-in web server 68 | ``` 69 | ``` 70 | tee php/Dockerfile 0<& 1 | tee kubeadm-init.log 47 | ``` 48 | 1. Check the static pods that have been created for the Control Plane: 49 | ``` 50 | ls /etc/kubernetes/manifests/ 51 | ls -l /etc/kubernetes/admin.conf 52 | ``` 53 | 1. Execute the following commands to avoid the need of running Kubernetes commands as root: 54 | ``` 55 | mkdir -p ${HOME}/.kube 56 | sudo cp /etc/kubernetes/admin.conf ${HOME}/.kube/config 57 | sudo chown -R $( id -u ):$( id -g ) ${HOME}/.kube/ 58 | echo 'source <(kubectl completion bash)' | tee --append ${HOME}/.bashrc 59 | source ${HOME}/.bashrc 60 | ``` 61 | 1. Deploy a network: 62 | ``` 63 | kubectl apply --filename https://docs.projectcalico.org/v3.21/manifests/calico.yaml 64 | watch kubectl get no 65 | ``` 66 | 1. Create a token to add more workers to the cluster: 67 | ``` 68 | sudo kubeadm token create --print-join-command 69 | ``` 70 | 1. Test the installation: 71 | ``` 72 | watch kubectl get no 73 | kubectl run test --image docker.io/library/nginx:alpine 74 | kubectl get po --output wide 75 | kubectl describe po 76 | kubectl get po --output yaml 77 | kubectl logs test 78 | kubectl get po/test --output yaml | tee po.yaml 79 | ``` 80 | # Kubernetes Playground 81 | 1. Login to the Kubernetes playground with your Docker account and start a new session: 82 | 83 | * https://labs.play-with-k8s.com/ 84 | 1. Add a new instance and initialize the kubernetes cluster master node: 85 | 86 | ``` 87 | kubeadm init --apiserver-advertise-address $(hostname -i) --pod-network-cidr 10.5.0.0/16 88 | ``` 89 | 1. Initialize the cluster networking: 90 | 91 | ``` 92 | kubectl apply -f https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/kubeadm-kuberouter.yaml 93 | ``` 94 | 1. Deploy a sample application: 95 | 96 | ``` 97 | kubectl apply -f https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/application/nginx-app.yaml 98 | ``` 99 | 1. Check the status of the deployment: 100 | 101 | ``` 102 | kubectl get all 103 | ``` 104 | 105 | # Docker Swarm 106 | 107 | 1. Create a Docker Compose file for the cluster using Docker Swarm (Docker Compose version 3): 108 | 109 | * https://docs.docker.com/compose/compose-file/compose-file-v3/ 110 | 111 | ``` 112 | tee ${PWD}/docker-swarm.yaml 0<' --namespace phpinfo 4 | kubectl get secret index.php --namespace phpinfo --output yaml 5 | echo PD9waHAgcGhwaW5mbygpOz8+ | base64 --decode 6 | ``` 7 | 1. https://kubernetes.io/docs/home/ 8 | 2. https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.24/ 9 | 3. https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/ 10 | 4. https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#secret-v1-core 11 | ``` 12 | # secret.yaml 13 | apiVersion: v1 14 | kind: Secret 15 | metadata: 16 | name: phpinfo-secret 17 | namespace: phpinfo 18 | stringData: 19 | index.php: 20 | type: Opaque 21 | # kubectl apply --filename secret.yaml --namespace phpinfo 22 | # kubectl get secret phpinfo-secret --namespace phpinfo --output yaml 23 | ``` 24 | 1. https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#configmap-v1-core 25 | ``` 26 | # cm.yaml 27 | apiVersion: v1 28 | data: 29 | index.php: 30 | kind: ConfigMap 31 | metadata: 32 | name: phpinfo-cm 33 | namespace: phpinfo 34 | # kubectl apply --filename cm.yaml --namespace phpinfo 35 | # kubectl get cm phpinfo-cm --namespace phpinfo --output yaml 36 | ``` 37 | 1. https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#service-v1-core 38 | ``` 39 | # svc.yaml 40 | apiVersion: v1 41 | kind: Service 42 | metadata: 43 | name: phpinfo 44 | namespace: phpinfo 45 | spec: 46 | ports: 47 | - port: 80 48 | protocol: TCP 49 | targetPort: 8080 50 | selector: 51 | app: phpinfo 52 | type: NodePort 53 | # kubectl apply --filename svc.yaml --namespace phpinfo 54 | ``` 55 | ``` 56 | kubectl api-resources | grep -E "DaemonSet|CronJob|Job|pods.*po.*true.*Pod|StatefulSet|ReplicationController|ReplicaSet|Deployment" 57 | ``` 58 | 1. https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#pod-v1-core 59 | ``` 60 | # po.yaml 61 | apiVersion: v1 62 | kind: Pod 63 | metadata: 64 | labels: 65 | app: phpinfo 66 | name: phpinfo-po 67 | namespace: phpinfo 68 | spec: 69 | containers: 70 | - args: 71 | - -f 72 | - index.php 73 | - -S 74 | - 0.0.0.0:8080 75 | command: 76 | - php 77 | image: php:alpine 78 | name: phpinfo-container 79 | ports: 80 | - containerPort: 8080 81 | protocol: TCP 82 | volumeMounts: 83 | - mountPath: /app/index.php 84 | name: phpinfo-volume 85 | readOnly: true 86 | subPath: index.php 87 | workingDir: /app/ 88 | restartPolicy: Always 89 | securityContext: 90 | runAsUser: 65534 91 | volumes: 92 | - name: phpinfo-volume 93 | secret: 94 | defaultMode: 0444 95 | items: 96 | - key: index.php 97 | mode: 0444 98 | path: index.php 99 | secretName: phpinfo-secret 100 | # kubectl apply --filename po.yaml --namespace phpinfo 101 | # kubectl get po --namespace phpinfo 102 | # kubectl logs phpinfo-po --namespace phpinfo 103 | # kubectl exec phpinfo-po --namespace phpinfo -- ls -l 104 | # kubectl exec phpinfo-po --namespace phpinfo -- curl localhost:8080 -I -s 105 | # kubectl describe po phpinfo-po --namespace phpinfo 106 | # kubectl describe svc phpinfo --namespace phpinfo 107 | ``` 108 | 1. https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.24/#replicationcontroller-v1-core 109 | ``` 110 | # rc.yaml 111 | apiVersion: v1 112 | kind: ReplicationController 113 | metadata: 114 | name: phpinfo-rc 115 | namespace: phpinfo 116 | spec: 117 | replicas: 2 118 | selector: 119 | app: phpinfo 120 | template: 121 | metadata: 122 | labels: 123 | app: phpinfo 124 | spec: 125 | containers: 126 | - args: 127 | - -f 128 | - index.php 129 | - -S 130 | - 0.0.0.0:8080 131 | command: 132 | - php 133 | image: php:alpine 134 | name: phpinfo-container 135 | ports: 136 | - containerPort: 8080 137 | protocol: TCP 138 | volumeMounts: 139 | - mountPath: /app/index.php 140 | name: phpinfo-volume 141 | readOnly: true 142 | subPath: index.php 143 | workingDir: /app/ 144 | restartPolicy: Always 145 | securityContext: 146 | runAsUser: 65534 147 | volumes: 148 | - name: phpinfo-volume 149 | secret: 150 | defaultMode: 0444 151 | items: 152 | - key: index.php 153 | mode: 0444 154 | path: index.php 155 | secretName: phpinfo-secret 156 | # kubectl apply --filename rc.yaml --namespace phpinfo 157 | # kubectl exec phpinfo-rc-kk8z5 --namespace phpinfo -- curl localhost:8080 -sI 158 | # kubectl exec phpinfo-rc-kk8z5 --namespace phpinfo -- curl 10.106.46.99 -sI 159 | ``` 160 | 1. https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.24/#replicasetspec-v1-apps 161 | ``` 162 | # rs.yaml 163 | apiVersion: apps/v1 164 | kind: ReplicaSet 165 | metadata: 166 | name: phpinfo-rs 167 | namespace: phpinfo 168 | spec: 169 | replicas: 2 170 | selector: 171 | matchLabels: 172 | app: phpinfo 173 | template: 174 | metadata: 175 | labels: 176 | app: phpinfo 177 | spec: 178 | containers: 179 | - args: 180 | - -f 181 | - index.php 182 | - -S 183 | - 0.0.0.0:8080 184 | command: 185 | - php 186 | image: php:alpine 187 | name: phpinfo-container 188 | ports: 189 | - containerPort: 8080 190 | protocol: TCP 191 | volumeMounts: 192 | - mountPath: /app/index.php 193 | name: phpinfo-volume 194 | readOnly: true 195 | subPath: index.php 196 | workingDir: /app/ 197 | restartPolicy: Always 198 | securityContext: 199 | runAsUser: 65534 200 | volumes: 201 | - name: phpinfo-volume 202 | secret: 203 | defaultMode: 0444 204 | items: 205 | - key: index.php 206 | mode: 0444 207 | path: index.php 208 | secretName: phpinfo-secret 209 | # kubectl apply --filename rs.yaml --namespace phpinfo 210 | ``` 211 | 1. https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.24/#daemonset-v1-apps 212 | ``` 213 | # ds.yaml 214 | apiVersion: apps/v1 215 | kind: DaemonSet 216 | metadata: 217 | name: phpinfo-ds 218 | namespace: phpinfo 219 | spec: 220 | selector: 221 | matchLabels: 222 | app: phpinfo 223 | template: 224 | metadata: 225 | labels: 226 | app: phpinfo 227 | spec: 228 | containers: 229 | - args: 230 | - -f 231 | - index.php 232 | - -S 233 | - 0.0.0.0:8080 234 | command: 235 | - php 236 | image: php:alpine 237 | name: phpinfo-container 238 | ports: 239 | - containerPort: 8080 240 | protocol: TCP 241 | volumeMounts: 242 | - mountPath: /app/index.php 243 | name: phpinfo-volume 244 | readOnly: true 245 | subPath: index.php 246 | workingDir: /app/ 247 | restartPolicy: Always 248 | securityContext: 249 | runAsUser: 65534 250 | volumes: 251 | - name: phpinfo-volume 252 | secret: 253 | defaultMode: 0444 254 | items: 255 | - key: index.php 256 | mode: 0444 257 | path: index.php 258 | secretName: phpinfo-secret 259 | # kubectl apply --filename ds.yaml --namespace phpinfo 260 | ``` 261 | 262 | -------------------------------------------------------------------------------- /networking2.md: -------------------------------------------------------------------------------- 1 | # Let us create the container: 2 | ``` 3 | mkdir --parents ${HOME}/phpinfo/ 4 | 5 | tee ${HOME}/phpinfo/index.php 0< 10 | 11 | EOF 12 | 13 | docker network create phpinfo-network --driver bridge 14 | 15 | docker run --cpus 0.100 --detach --env AUTHOR=Sebastian --memory 100M --memory-reservation 100M --name phpinfo-container --network phpinfo-network --read-only --restart always --user nobody:nogroup --volume ${HOME}/phpinfo/index.php:/data/index.php:ro --workdir /data/ docker.io/library/php:alpine php -f index.php -S 0.0.0.0:8080 16 | 17 | docker ps | grep phpinfo-container 18 | ``` 19 | # How to connect to the container network 20 | The Docker container is running but only accessible from inside: 21 | ``` 22 | $ docker exec phpinfo-container wget localhost:8080 -O - -q --spider -S 23 | HTTP/1.1 200 OK 24 | Host: localhost:8080 25 | Date: Sat, 15 Oct 2022 14:46:11 GMT 26 | Connection: close 27 | X-Powered-By: PHP/8.0.24 28 | Content-type: text/html; charset=UTF-8 29 | 30 | $ wget localhost:8080 -O - -q --spider -S 31 | $ 32 | ``` 33 | In order to connect from outside I need to publish a Node Port from the host machine mapped into the container network: 34 | ``` 35 | docker rm phpinfo-container --force 36 | 37 | docker run --cpus 0.100 --detach --env AUTHOR=Sebastian --memory 100M --memory-reservation 100M --name phpinfo-container --network phpinfo-network --publish 8080 --read-only --restart always --user nobody:nogroup --volume ${HOME}/phpinfo/index.php:/data/index.php:ro --workdir /data/ 172.31.10.220/library/php:alpine php -f index.php -S 0.0.0.0:8080 38 | 39 | docker exec phpinfo-container wget localhost:8080 -O - -q --spider -S 40 | 41 | NODE_PORT=$( docker port phpinfo-container | grep --max-count=1 tcp | cut --delimiter : --field 2 ) 42 | 43 | wget localhost:${NODE_PORT} -O - -q --spider -S 44 | ``` 45 | It is recommended to keep the Node Port as a random value but if you want to fix the Node Port then you can do that with the following command: 46 | ``` 47 | docker rm phpinfo-container --force 48 | 49 | NODE_PORT=30000 50 | 51 | docker run --cpus 0.100 --detach --env AUTHOR=Sebastian --memory 100M --memory-reservation 100M --name phpinfo-container --network phpinfo-network --publish ${NODE_PORT}:8080 --read-only --restart always --user nobody:nogroup --volume ${HOME}/phpinfo/index.php:/data/index.php:ro --workdir /data/ docker.io/library/php:alpine php -f index.php -S 0.0.0.0:8080 52 | 53 | docker exec phpinfo-container wget localhost:8080 -O - -q --spider -S 54 | 55 | wget localhost:${NODE_PORT} -O - -q --spider -S 56 | ``` 57 | # Docker networking 58 | There are three main kind of network drivers in Docker: 59 | * bridge 60 | * host 61 | * null 62 | 63 | The default network driver for Docker is bridge. 64 | Any Docker installation will create a default bridge network also called "bridge". 65 | Any container will be attached to that network by default. 66 | That means that any container will be able to talk to any other container in that bridge network (by default). 67 | Therefore, for security reasons, it is better not to use the default network. 68 | The recommended practice is to always create a custom bridge for your containers: 69 | ``` 70 | docker network create phpinfo-network --driver bridge 71 | ``` 72 | Only containers in the same network will be able to talk to each other. 73 | # Bridge Docker network 74 | It is the default Docker network. 75 | Docker will create a default bridge network after first installation. 76 | Any Docker container will connect to this bridge network by default. 77 | Two Docker containers need to be connected to the same Docker network in order to be able to communicate to each other. 78 | Therefore, by default any Docker container will be able to talk to each other through this default bridge network. 79 | 80 | 1. List the default Docker networks: 81 | 82 | ``` 83 | docker network ls 84 | ``` 85 | 2. Create one container connected to the default bridge network: 86 | 87 | ``` 88 | docker run --detach --name test1 --network bridge --tty index.docker.io/library/busybox:latest 89 | ``` 90 | 3. Test the container network pinging different targets: 91 | 92 | ``` 93 | docker exec test1 ping -c 1 localhost 94 | 95 | docker exec test1 ping -c 1 google.com 96 | 97 | docker exec test1 ping -c 1 8.8.8.8 98 | ``` 99 | 1. Check the IP address of this first container: 100 | 101 | ``` 102 | docker inspect test1 103 | ``` 104 | 3. Create a second container in the same default bridge network: 105 | 106 | ``` 107 | docker run --detach --name test2 --network bridge --tty index.docker.io/library/busybox:latest 108 | ``` 109 | 1. Test the container network pinging the IP address of the previous test container (172.17.0.2 in my case): 110 | 111 | ``` 112 | docker exec test2 ping -c 1 172.17.0.2 113 | ``` 114 | 1. Create a custome bridge network: 115 | 116 | ``` 117 | docker network create my_bridge --driver bridge 118 | ``` 119 | 1. Check the list of networks: 120 | 121 | ``` 122 | docker network ls 123 | ``` 124 | 1. Create two containers connected to this custom bridge: 125 | 126 | ``` 127 | docker run --detach --name custom1 --network my_bridge --tty busybox 128 | 129 | docker run --detach --name custom2 --network my_bridge --tty busybox 130 | ``` 131 | 1. Test the container network pinging different targets: 132 | 133 | ``` 134 | docker exec custom1 ping -c 1 localhost 135 | 136 | docker exec custom1 ping -c 1 google.com 137 | 138 | docker exec custom1 ping -c 1 8.8.8.8 139 | 140 | docker exec custom1 ping -c 1 custom2 141 | 142 | docker exec custom1 ping -c 1 test1 143 | 144 | docker exec custom1 ping -c 1 172.17.0.2 145 | ``` 146 | 1. Compare the following two commands: 147 | 148 | ``` 149 | docker exec custom1 ping -c 1 custom2 150 | 151 | docker exec test2 ping -c 1 test1 152 | 153 | docker exec test2 ping -c 1 172.17.0.2 154 | ``` 155 | 1. Let us create a second custom bridge: 156 | 157 | ``` 158 | docker network create my_bridge_2 --driver bridge 159 | ``` 160 | 1. Let us create a container in this network: 161 | 162 | ``` 163 | docker run --detach --name custom3 --network my_bridge_2 --tty index.docker.io/library/busybox:latest 164 | ``` 165 | 1. Check the isolation of this container: 166 | 167 | ``` 168 | docker exec custom3 ping -c 1 localhost 169 | 170 | docker exec custom3 ping -c 1 google.com 171 | 172 | docker exec custom3 ping -c 1 8.8.8.8 173 | 174 | docker exec custom3 ping -c 1 172.17.0.2 175 | 176 | docker exec custom3 ping -c 1 custom1 177 | ``` 178 | 1. Let us connect this new container to both custom networks: 179 | 180 | ``` 181 | docker exec custom3 ifconfig 182 | 183 | docker network connect my_bridge custom3 184 | 185 | docker exec custom3 ifconfig 186 | ``` 187 | 1. Check the connectivity between the containers: 188 | 189 | ``` 190 | docker exec custom3 ping -c 1 custom1 191 | 192 | docker exec custom3 ping -c 1 custom2 193 | 194 | docker exec custom3 ping -c 1 172.17.0.2 195 | ``` 196 | 1. Inspect the Docker network: 197 | 198 | ``` 199 | docker inspect my_bridge 200 | 201 | docker inspect my_bridge_2 202 | ``` 203 | 1. Bridges are isolated by IPtables firewall: 204 | 205 | ``` 206 | iptables -S -t filter | grep A.DOCKER-ISOLATION-STAGE-2.*DROP 207 | ``` 208 | # Experimenting with the Host network: 209 | 1. List the network interface configuration of the host machine: 210 | 211 | ``` 212 | ifconfig 213 | ``` 214 | 3. Create a container connected to the Host network: 215 | 216 | ``` 217 | docker run --detach --name host1 --network host --tty busybox 218 | ``` 219 | 1. List the network interface configuration from inside the container. It will be the same configuration as the host machine because the container is connected to the Host network: 220 | 221 | ``` 222 | docker exec host1 ifconfig 223 | ``` 224 | # Experimenting with the Null network: 225 | 1. Create a container using the null network: 226 | 227 | ``` 228 | docker run --detach --name none1 --network none --tty busybox 229 | ``` 230 | 1. When we create a container attached to the null network there is no network interface defined inside the container: 231 | 232 | ``` 233 | docker exec none1 ifconfig 234 | ``` 235 | 1. You can still send or retrieve files from the isolated container using `cp` and `exec`: 236 | 237 | ``` 238 | docker cp examples.desktop none1:/tmp/ 239 | 240 | sudo docker exec none1 ls /tmp/ 241 | ``` 242 | -------------------------------------------------------------------------------- /caltech_2022-09/2022-09-25.md: -------------------------------------------------------------------------------- 1 | # Experimenting with Docker bridge networks 2 | 1. List the networks: 3 | 4 | ``` 5 | sudo docker network ls 6 | ``` 7 | 2. Inspect the networks in order to obtain the IP addresses: 8 | 9 | ``` 10 | sudo docker inspect bridge 11 | 12 | sudo docker inspect my_bridge 13 | ``` 14 | 1. We need to use IP adresses when communicating containers in the default bridge: 15 | 16 | ``` 17 | sudo docker exec test1 ping -c 1 172.17.0.3 18 | ``` 19 | 1. We can use Docker internal DNS resolution when connecting containers in the custom bridge: 20 | 21 | ``` 22 | sudo docker exec custom1 ping -c 1 custom2 23 | ``` 24 | 1. We can connect one container to many networks. Let us connect the test container to the custom bridge: 25 | 26 | ``` 27 | sudo docker network connect my_bridge test1 28 | ``` 29 | 1. We can use internal DNS resolution for any container in the custom bridge: 30 | 31 | ``` 32 | sudo docker exec custom1 ping -c 1 test1 33 | ``` 34 | 1. Bridges are isolated by IPtables firewall: 35 | 36 | ``` 37 | sudo iptables -S -t filter | grep A.DOCKER-ISOLATION-STAGE-2.*DROP 38 | ``` 39 | # Experimenting with the Host network: 40 | 1. List the network interface configuration of the host machine: 41 | 42 | ``` 43 | ifconfig 44 | ``` 45 | 3. Create a container connected to the Host network: 46 | 47 | ``` 48 | sudo docker run --detach --name host1 --network host --tty busybox 49 | ``` 50 | 1. List the network interface configuration from inside the container. It will be the same configuration as the host machine because the container is connected to the Host network: 51 | 52 | ``` 53 | sudo docker exec host1 ifconfig 54 | ``` 55 | # Experimenting with the Null network: 56 | 1. Create a container using the null network: 57 | 58 | ``` 59 | sudo docker run --detach --name none1 --network none --tty busybox 60 | ``` 61 | 1. When we create a container attached to the null network there is no network interface defined inside the container: 62 | 63 | ``` 64 | sudo docker exec none1 ifconfig 65 | ``` 66 | 1. You can still send or retrieve files from the isolated container using `cp` and `exec`: 67 | 68 | ``` 69 | sudo docker cp examples.desktop none1:/tmp/ 70 | 71 | sudo docker exec none1 ls /tmp/ 72 | ``` 73 | # Sample application written in Python 74 | 1. Create a folder for the project: 75 | 76 | ``` 77 | mkdir --parents ${HOME}/anagrams/ 78 | ``` 79 | 1. Download the Python script for our application: 80 | 81 | ``` 82 | wget https://raw.githubusercontent.com/sebastian-colomar/anagrams/docker/src/anagrams.py --output-document ${HOME}/anagrams/anagrams.py 83 | ``` 84 | 3. Download a sample dictionary: 85 | 86 | ``` 87 | wget https://raw.githubusercontent.com/sebastian-colomar/anagrams/docker/data/words.txt --output-document ${HOME}/anagrams/words.txt 88 | ``` 89 | 1. Run the Python script that will analyze the data in the sample dictionary showing the number of Anagrams: 90 | 91 | * https://en.wikipedia.org/wiki/Anagram 92 | 93 | ``` 94 | cd ${HOME}/anagrams/ 95 | 96 | python3 anagrams.py 97 | ``` 98 | # How to deploy our Python sample application using Docker Compose 99 | 1. Create a Docker Compose file to describe the services we want to run: 100 | 101 | ``` 102 | tee ${HOME}/anagrams/docker-compose.yaml 0< 161 | ``` 162 | ``` 163 | apiVersion: v1 164 | kind: Service 165 | metadata: 166 | name: phpinfo-svc-1 167 | spec: 168 | ports: 169 | - port: 8080 170 | protocol: TCP 171 | selector: 172 | app: phpinfo-deploy-1 173 | ``` 174 | ``` 175 | apiVersion: v1 176 | kind: Service 177 | metadata: 178 | name: phpinfo-svc-2 179 | spec: 180 | ports: 181 | - port: 8080 182 | protocol: TCP 183 | selector: 184 | app: phpinfo-deploy-2 185 | ``` 186 | ``` 187 | apiVersion: apps/v1 188 | kind: Deployment 189 | metadata: 190 | name: phpinfo-deploy-1 191 | spec: 192 | replicas: 2 193 | selector: 194 | matchLabels: 195 | app: phpinfo-deploy-1 196 | template: 197 | metadata: 198 | labels: 199 | app: phpinfo-deploy-1 200 | spec: 201 | containers: 202 | - args: 203 | - php 204 | - -f 205 | - index.php 206 | - -S 207 | - 0.0.0.0:8080 208 | env: 209 | - name: OWNER 210 | value: Sebastian 211 | image: index.docker.io/library/php:8.0-alpine 212 | name: phpinfo-container 213 | resources: 214 | limits: 215 | cpu: 200m 216 | memory: 200M 217 | requests: 218 | cpu: 200m 219 | memory: 200M 220 | securityContext: 221 | readOnlyRootFilesystem: true 222 | volumeMounts: 223 | - mountPath: /data/index.php 224 | name: phpinfo-volume 225 | readOnly: true 226 | subPath: index.php 227 | workingDir: /data/ 228 | volumes: 229 | - name: phpinfo-volume 230 | configMap: 231 | defaultMode: 0400 232 | items: 233 | - key: index.php 234 | mode: 0400 235 | path: index.php 236 | name: phpinfo-cm 237 | ``` 238 | ``` 239 | apiVersion: apps/v1 240 | kind: Deployment 241 | metadata: 242 | name: phpinfo-deploy-2 243 | spec: 244 | replicas: 2 245 | selector: 246 | matchLabels: 247 | app: phpinfo-deploy-2 248 | template: 249 | metadata: 250 | labels: 251 | app: phpinfo-deploy-2 252 | spec: 253 | containers: 254 | - args: 255 | - php 256 | - -f 257 | - index.php 258 | - -S 259 | - 0.0.0.0:8080 260 | env: 261 | - name: OWNER 262 | value: Sebastian 263 | image: index.docker.io/library/php:8.1-alpine 264 | name: phpinfo-container 265 | resources: 266 | limits: 267 | cpu: 200m 268 | memory: 200M 269 | requests: 270 | cpu: 200m 271 | memory: 200M 272 | securityContext: 273 | readOnlyRootFilesystem: true 274 | volumeMounts: 275 | - mountPath: /data/index.php 276 | name: phpinfo-volume 277 | readOnly: true 278 | subPath: index.php 279 | workingDir: /data/ 280 | volumes: 281 | - name: phpinfo-volume 282 | configMap: 283 | defaultMode: 0400 284 | items: 285 | - key: index.php 286 | mode: 0400 287 | path: index.php 288 | name: phpinfo-cm 289 | ``` 290 | 1. In this link you will find the Reference for the Kubernetes API syntax: 291 | 292 | * https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.23/ 293 | -------------------------------------------------------------------------------- /caltech_2022-09/2022-10-02.md: -------------------------------------------------------------------------------- 1 | # Kubernetes Playground 2 | 1. Login to the Kubernetes playground with your Docker account and start a new session: 3 | 4 | * https://labs.play-with-k8s.com/ 5 | 1. Add a new instance and initialize the kubernetes cluster master node: 6 | 7 | ``` 8 | kubeadm init --apiserver-advertise-address $(hostname -i) --pod-network-cidr 10.5.0.0/16 9 | ``` 10 | 1. Initialize the cluster networking: 11 | 12 | ``` 13 | kubectl apply -f https://raw.githubusercontent.com/cloudnativelabs/kube-router/master/daemonset/kubeadm-kuberouter.yaml 14 | ``` 15 | 1. Deploy a sample application: 16 | 17 | ``` 18 | kubectl apply -f https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/application/nginx-app.yaml 19 | ``` 20 | 1. Check the status of the deployment: 21 | 22 | ``` 23 | kubectl get all 24 | ``` 25 | 1. As you can see in the logs, the Service is not working. It is because the Type of the Service is Loadbalancer. This type of Service requieres cloud credentials to create a Load Balancer in the Cloud of your choice. The best way to solve this problem is to change the type of the Service. In order to do so, we first download the YAML manifest: 26 | 27 | ``` 28 | curl -O https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/application/nginx-app.yaml 29 | ``` 30 | 1. Now we can modify the Service configuration with the following command: 31 | 32 | ``` 33 | sed -i /LoadBalancer/d nginx-app.yaml 34 | ``` 35 | 1. Redeploy the manifest with the changes: 36 | 37 | ``` 38 | kubectl apply -f nginx-app.yaml 39 | ``` 40 | 1. Check again the status of the deployment: 41 | 42 | ``` 43 | kubectl get all 44 | ``` 45 | 1. Now the Service has been correctly created but there is still an issue: all the Pods are in a Pending state. To troubleshoot the problem we check the events with th e following command: 46 | 47 | ``` 48 | kubectl get ev 49 | ``` 50 | 1. The latest event shows that there is a Taint that forbids the deployment of the Pods: 51 | 52 | ``` 53 | 0/1 nodes are available: 1 node(s) had taint {node-role.kubernetes.io/master: }, that the pod didn't tolerate 54 | ``` 55 | 1. Taints are labels (or marks) applied to the nodes in order to forbid the deployment of containers. By default the master node is tainted in order to secure the cluster. To solve this situation, the easiest way is to create a worker node for the deployment. For that purpose, you will need to add a new instance and join the cluster. But you will need a token in order to join the cluster. That token was created during the installation. If you lost the token you can generate a new one running the following command on the master node: 56 | 57 | ``` 58 | kubeadm token create --print-join-command 59 | ``` 60 | 61 | 3. Add a new instance and join the master node pasting the previously generated token in the newly created instance: 62 | 63 | ``` 64 | kubeadm join 192.168.0.13:6443 --token vz62an.xxx --discovery-token-ca-cert-hash sha256:xxx 65 | ``` 66 | 1. Check again the status of the deployment: 67 | 68 | ``` 69 | kubectl get all 70 | ``` 71 | 1. Now all the Pods should be correctly running and the Deployment completely Ready. But the application is still not available from outside because the container network is isolated by default. In order to be able to connect to the container network from an external web browser, we need to map a port of the external host network to a port of the internal container network. To do so, we will download again the YAML manifest, modify it and redeploy it afterwards: 72 | 73 | ``` 74 | rm -f nginx-app.yaml 75 | 76 | curl -O https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/application/nginx-app.yaml 77 | 78 | sed -i s/LoadBalancer/NodePort/ nginx-app.yaml 79 | 80 | kubectl apply -f nginx-app.yaml 81 | ``` 82 | 83 | 3. In order to remove the previous deployment run the following command: 84 | 85 | ``` 86 | kubectl delete -f https://raw.githubusercontent.com/kubernetes/website/master/content/en/examples/application/nginx-app.yaml 87 | ``` 88 | # Docker Swarm production cluster in AWS 89 | 1. Use the following template to create the cluster infrastructure in AWS: 90 | 91 | * https://raw.githubusercontent.com/secobau/docker-aws/master/etc/aws/cluster-docker.yaml 92 | 93 | 1. Connect to first master instance and initialize the Docker Swarm: 94 | 95 | ``` 96 | sudo docker swarm init --advertise-addr 10.168.1.100 97 | ``` 98 | 1. Connect to the three worker instances and join the master node 99 | 2. Connect to the leader master and generate the token for the manager nodes: 100 | 101 | ``` 102 | sudo docker swarm join-token manager 103 | ``` 104 | 1. Connect to the other two master instances and join the master node 105 | 2. Check the current members of the cluster: 106 | 107 | ``` 108 | sudo docker node ls 109 | ``` 110 | 1. Deploy a sample application (using Node Port 30001): 111 | 112 | 1. First download the configuration file: 113 | ``` 114 | wget https://raw.githubusercontent.com/sebastian-colomar/phpinfo/main/src/index.php 115 | ``` 116 | 1. Now create the Docker Compose file for the deployment: 117 | ``` 118 | tee ${PWD}/docker-swarm-1.yaml 0< 232 | 233 | EOF 234 | 235 | sudo docker network create phpinfo-network --driver bridge 236 | 237 | sudo docker run --cpus 0.100 --detach --env AUTHOR=Sebastian --memory 100M --memory-reservation 100M --name phpinfo-container --network phpinfo-network --read-only --restart always --user nobody:nogroup --volume ${HOME}/phpinfo/index.php:/data/index.php:ro --workdir /data/ 172.31.10.220/library/php:alpine php -f index.php -S 0.0.0.0:8080 238 | 239 | sudo docker ps | grep phpinfo-container 240 | ``` 241 | -------------------------------------------------------------------------------- /images/vm.svg: -------------------------------------------------------------------------------- 1 | --------------------------------------------------------------------------------