├── .gitignore ├── assets ├── 1566547690833.png ├── 1566550570921.png ├── 1567496371287.png ├── 1567575976437.png ├── container-layers.jpg ├── whale20logo332_5.png ├── COFFEE BUTTON ヾ(°∇°^).png ├── types-of-mounts-volume.png ├── Are-containers-..-vms-image-2.png ├── docker-filesystems-busyboxrw.png ├── Blog.-Are-containers-..VM-Image-1.png ├── 011f3ef6-d824-4d43-8b2c-36dab8eaaa72-2.png └── Kernel_Layout.svg ├── LICENSE ├── README.md └── 01 Docker Crash Course.md /.gitignore: -------------------------------------------------------------------------------- 1 | 2 | *.no_toc 3 | -------------------------------------------------------------------------------- /assets/1566547690833.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenSUTD/docker-reference/master/assets/1566547690833.png -------------------------------------------------------------------------------- /assets/1566550570921.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenSUTD/docker-reference/master/assets/1566550570921.png -------------------------------------------------------------------------------- /assets/1567496371287.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenSUTD/docker-reference/master/assets/1567496371287.png -------------------------------------------------------------------------------- /assets/1567575976437.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenSUTD/docker-reference/master/assets/1567575976437.png -------------------------------------------------------------------------------- /assets/container-layers.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenSUTD/docker-reference/master/assets/container-layers.jpg -------------------------------------------------------------------------------- /assets/whale20logo332_5.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenSUTD/docker-reference/master/assets/whale20logo332_5.png -------------------------------------------------------------------------------- /assets/COFFEE BUTTON ヾ(°∇°^).png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenSUTD/docker-reference/master/assets/COFFEE BUTTON ヾ(°∇°^).png -------------------------------------------------------------------------------- /assets/types-of-mounts-volume.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenSUTD/docker-reference/master/assets/types-of-mounts-volume.png -------------------------------------------------------------------------------- /assets/Are-containers-..-vms-image-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenSUTD/docker-reference/master/assets/Are-containers-..-vms-image-2.png -------------------------------------------------------------------------------- /assets/docker-filesystems-busyboxrw.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenSUTD/docker-reference/master/assets/docker-filesystems-busyboxrw.png -------------------------------------------------------------------------------- /assets/Blog.-Are-containers-..VM-Image-1.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenSUTD/docker-reference/master/assets/Blog.-Are-containers-..VM-Image-1.png -------------------------------------------------------------------------------- /assets/011f3ef6-d824-4d43-8b2c-36dab8eaaa72-2.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/OpenSUTD/docker-reference/master/assets/011f3ef6-d824-4d43-8b2c-36dab8eaaa72-2.png -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2019 methylDragon 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /assets/Kernel_Layout.svg: -------------------------------------------------------------------------------- 1 | CPUMemoryDevicesKernelApplications -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # docker-reference 2 | 3 | It's time to have a **whale** of a time. 4 | 5 | ![img](assets/whale20logo332_5.png) 6 | 7 | > Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications. By taking advantage of Docker’s methodologies for shipping, testing, and deploying code quickly, you can significantly reduce the delay between writing code and running it in production. 8 | > 9 | > 10 | 11 | 12 | 13 | ## Introduction 14 | 15 | This repo is a set of personal notes, code snippets, and a spellbook of frequent command invocations that I've collected over time. Shared from me to you! 16 | 17 | As I mainly deal with robotics applications, some sections might focus on that, but quite a lot of this reference will generalise outside of the robotics domain! **But as I use Docker with Ubuntu, I cannot guarantee generalisability across other systems or distributions**. 18 | 19 | 20 | 21 | ## Credits 22 | 23 | All credits and sources are listed inside the tutorials and references themselves. 24 | 25 | 26 | 27 | ## Support My Efforts! 28 | 29 | [![Yeah! Buy the DRAGON a COFFEE!](assets/COFFEE%20BUTTON%20%E3%83%BE(%C2%B0%E2%88%87%C2%B0%5E).png)](https://www.buymeacoffee.com/methylDragon) 30 | 31 | [Or leave a tip! ヾ(°∇°*)](https://www.paypal.me/methylDragon) 32 | 33 | 34 | 35 | ``` 36 | . . 37 | . |\-^-/| . 38 | /| } O.=.O { |\ 39 | ``` -------------------------------------------------------------------------------- /01 Docker Crash Course.md: -------------------------------------------------------------------------------- 1 | # Docker Crash Course 2 | 3 | Author: methylDragon 4 | Contains a syntax reference and code snippets for Docker! 5 | Lots of personal notes and stuff compiled from all over the internet! 6 | 7 | ------ 8 | 9 | ## Pre-Requisites 10 | 11 | ### Required 12 | 13 | - Linux terminal fluency 14 | - This tutorial will be written and tested in Ubuntu 18.04, so generalisations are not guaranteed 15 | 16 | 17 | 18 | ## Table Of Contents 19 | 20 | 1. [Introduction](#1) 21 | 2. [Docker Setup](#2) 22 | 2.1 [Install via Convenience Script](#2.1) 23 | 2.2 [Install via Repository](#2.2) 24 | 2.3 [Post-Installation Steps](#2.3) 25 | 2.4 [Install Verification](#2.4) 26 | 2.5 [Uninstallation](#2.5) 27 | 3. [Docker Basics](#3) 28 | 3.1 [What is Docker?](#3.1) 29 | 3.2 [The Docker Command](#3.2) 30 | 3.3 [Images and the Docker Hub](#3.3) 31 | 3.4 [Image Commands](#3.4) 32 | 3.5 [Docker System Commands](#3.5) 33 | 3.6 [Running Containers](#3.6) 34 | 3.7 [Volumes](#3.7) 35 | 3.8 [Basic Container Operations](#3.8) 36 | 3.9 [Executing Commands](#3.9) 37 | 3.10 [Committing and Pushing Images](#3.10) 38 | 4. [Dockerfiles](#4) 39 | 4.1 [Introduction](#4.1) 40 | 4.2 [Building Dockerfiles](#4.2) 41 | 4.3 [Build Context](#4.3) 42 | 4.4 [Dockerignore](#4.4) 43 | 4.5 [Image Layers](#4.5) 44 | 4.6 [Dockerfile Comments](#4.6) 45 | 4.7 [Dockerfile Command Overview](#4.7) 46 | 4.8 [Shell vs Exec Form](#4.8) 47 | 4.9 [An Example Dockerfile](#4.9) 48 | 4.10 [The Proper Way to Layer](#4.10) 49 | 4.11 [Nifty Multi-Stage Build Tricks](#4.11) 50 | 51 | 52 | 53 | 54 | ## 1. Introduction 55 | 56 | ![img](assets/whale20logo332_5.png) 57 | 58 | > Docker is an open platform for developing, shipping, and running applications. Docker enables you to separate your applications from your infrastructure so you can deliver software quickly. With Docker, you can manage your infrastructure in the same ways you manage your applications. By taking advantage of Docker’s methodologies for shipping, testing, and deploying code quickly, you can significantly reduce the delay between writing code and running it in production. 59 | > 60 | > 61 | 62 | Ever had to spend hours upon hours setting up an environment for a project, only to have to suddenly switch to a project and find out that your environments are clashing? 63 | 64 | Ever wish that there were a much more isolated way of **containing** your environments so that you can send it over to someone else's computer and have your project work very nicely just by running those containers? **Well Docker lets you do that**! 65 | 66 | > **Note**: This tutorial will presume that you are following along on a Linux system (preferably Ubuntu), and does not guarantee generalisability across other platforms (though most of the things should apply). 67 | > 68 | > **Additionally**: Some of the steps and ways of doing things in Docker mentioned in this tutorial are geared towards developers doing active development. There might be security concerns if one were to do things the same way in production environments, and I will try to call them out as often as possible when they become relevant, but you should be aware of the presence of these caveats. 69 | > 70 | > **Finally**: We will be using **Docker CE**, the community edition, not the Enterprise edition. 71 | 72 | This tutorial aims to give you a working knowledge of Docker, and not an all-encompassing one. 73 | 74 | 75 | 76 | ## 2. Docker Setup 77 | 78 | [Source]() 79 | 80 | There's a couple of ways to install Docker, but most of the recommended ways have you taking the distribution from the Docker repo directly, not from the Ubuntu repos. 81 | 82 | 83 | 84 | ### 2.1 Install via Convenience Script 85 | [go to top](#top) 86 | 87 | 88 | > **Some Warnings** 89 | > 90 | > - Using these scripts is not recommended for production environments 91 | > - Ensure you check the scripts first before running them, especially since they use `root` or `sudo` privileges to run 92 | > - **If you've installed Docker before, do not use the convenience script, as it can introduce problems** (eg. from having duplicate repository sources.) 93 | 94 | ```shell 95 | # Download the script, run it, then delete the script 96 | curl -fsSL https://get.docker.com -o get-docker.sh 97 | sudo sh get-docker.sh 98 | rm get-docker.sh 99 | ``` 100 | 101 | After you've installed Docker CE using the convenience script, **do not run the convenience script again, even if you want to update Docker**. Instead, you may use the package manager to update Docker. 102 | 103 | ```shell 104 | sudo apt-get install docker-ce docker-ce-cli containerd.io 105 | ``` 106 | 107 | 108 | 109 | ### 2.2 Install via Repository 110 | [go to top](#top) 111 | 112 | 113 | You can also just set up your package manager to use Docker's repository, which should be more updated than the default one. 114 | 115 | ```shell 116 | sudo apt-get update 117 | 118 | # Install packages to allow use of repositories over HTTPS 119 | sudo apt-get install \ 120 | apt-transport-https \ 121 | ca-certificates \ 122 | curl \ 123 | gnupg-agent \ 124 | software-properties-common 125 | 126 | # Add Docker's GPG key 127 | curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add - 128 | 129 | # Verify key 130 | sudo apt-key fingerprint 0EBFCD88 131 | 132 | # Add repository 133 | sudo add-apt-repository \ 134 | "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ 135 | $(lsb_release -cs) \ 136 | stable" 137 | 138 | # Install! 139 | sudo apt-get update 140 | sudo apt-get install docker-ce docker-ce-cli containerd.io 141 | ``` 142 | 143 | If you want to install a specific version of Docker, use 144 | 145 | ```shell 146 | sudo apt-get install docker-ce= docker-ce-cli= containerd.io 147 | ``` 148 | 149 | 150 | 151 | ### 2.3 Post-Installation Steps 152 | [go to top](#top) 153 | 154 | 155 | [Source]() 156 | 157 | Now, because Docker binds to a UNIX socket in order to work, it requires `sudo` privileges to run, which causes the Docker daemon to run as the `root` user. 158 | 159 | Here, we'll look through some steps to create a user group with sudo privileges so we can avoid having to constantly call `sudo` with Docker. 160 | 161 | > **Note**: This can lead to security issues, especially if you have a user group that grants default `sudo` privileges to Docker! 162 | > 163 | > The issues are listed [here](). 164 | > 165 | > But it boils down to: 166 | > 167 | > - **Only trusted users should be allowed to control your Docker daemon** 168 | > - This is because Docker allows you to share directories between the host, **without limiting the rights of the container**. 169 | > - You should use permission checks or secure API endpoints if using Docker with an internet facing server. 170 | 171 | With those warnings out of the way, let's go! 172 | 173 | ```shell 174 | sudo groupadd docker # Create docker group 175 | sudo usermod -aG docker $USER # Add yourself to the group 176 | 177 | # Then either run this (on Linux), or log in and out again to have the changes apply! 178 | newgrp docker 179 | ``` 180 | 181 | You can verify if you're part of the group by using 182 | 183 | ```shell 184 | # groups 185 | groups 186 | 187 | # Or id, either should work 188 | id -nG 189 | ``` 190 | 191 | 192 | 193 | **You will also want to configure Docker to start on boot!** 194 | 195 | ```shell 196 | sudo systemctl enable docker 197 | ``` 198 | 199 | If you don't want this, disable it instead 200 | 201 | ```shell 202 | sudo systemctl disable docker 203 | ``` 204 | 205 | 206 | 207 | ### 2.4 Install Verification 208 | [go to top](#top) 209 | 210 | 211 | Assuming you did the post installation step, you should then be able to run the Docker hello world. 212 | 213 | ![1566547690833](assets/1566547690833.png) 214 | 215 | ```shell 216 | docker run hello-world 217 | ``` 218 | 219 | Or if you had issues with the post-install steps, then at least 220 | 221 | ```shell 222 | sudo docker run hello-world 223 | ``` 224 | 225 | 226 | 227 | ### 2.5 Uninstallation 228 | [go to top](#top) 229 | 230 | 231 | This is just in case you want to uninstall! 232 | 233 | ```shell 234 | sudo apt-get purge docker-ce 235 | sudo rm -rf /var/lib/docker # And remove any Docker images 236 | ``` 237 | 238 | 239 | 240 | ## 3. Docker Basics 241 | 242 | ### 3.1 What is Docker? 243 | [go to top](#top) 244 | 245 | 246 | To first be able to explain what Docker is, we need to understand what a kernel is. 247 | 248 | #### **OS Kernels** 249 | 250 | ![Image result for os kernel](assets/Kernel_Layout.svg) 251 | 252 | [Image Source]() 253 | 254 | A kernel connects the application layer of a computer to the hardware of a computer. 255 | 256 | Some functions of an OS kernel are: 257 | 258 | - Schedule programs and threads 259 | - Manage memory 260 | - Handle hardware 261 | 262 | #### **Virtual Machines** 263 | 264 | So when we normally think of the types of things that we'd like Docker to do (simulate and attempt to isolate OS environments), the first thing that pops to mind is complete machine and OS simulation. That is, we'd think of virtual machines. 265 | 266 | ![1566550570921](assets/1566550570921.png) 267 | 268 | [Image Source]() 269 | 270 | Virtual machines will simulate their own OSes, **all the way down to the kernel level**, in order to run their applications. This added layer of simulation presents a significant overhead because the use of a [hypervisor]() and having to simulate the entire OS. 271 | 272 | This means that virtual machines can get quite **slow** and **use a lot of memory**. The benefit, however, is that you can host multiple operating systems with differing kernels. (So, running Windows alongside a Linux distribution becomes possible.) 273 | 274 | #### **Containers** 275 | 276 | Containers are different though! 277 | 278 | ![Introduction to Docker](assets/011f3ef6-d824-4d43-8b2c-36dab8eaaa72-2.png) 279 | 280 | [Image Source]() 281 | 282 | So the cool thing about Docker is that it **runs in the application space**, and leverages a common kernel across all containers. So all taskings actually run on the host system! **This removes the overhead associated with simulating an OS**, because you only have to double up on all the OS specific applications! 283 | 284 | Of course, the downside to this is that **you're quite gimped if you need to simulate an OS that requires a different kernel than the kernel on your host system**. So you can't realistically run a Windows container on a Linux system. And cross-architecture stuff will be hard without a proper emulator (like the one the Docker folks are making for [ARM]().) 285 | 286 | There are also issues with security, since each container shares the kernel and have root access. So the lack of isolation means you need to tighten up your security in all other aspects of your system. Docker does have some security features that try to help with isolation, but you should be aware of its limitations. 287 | 288 | #### **Sum Up** 289 | 290 | ![img](assets/Blog.-Are-containers-..VM-Image-1.png) 291 | 292 | [Image Source]() 293 | 294 | So at the **cost of security, and some cross-OS functionality**, containers offer **speed, a lower memory footprint, and faster development time**. 295 | 296 | And besides, if you ever find yourself needing multiple OSes, **why not combine the two solutions**? 297 | 298 | ![img](assets/Are-containers-..-vms-image-2.png) 299 | 300 | [Image Source]() 301 | 302 | 303 | 304 | ### 3.2 The Docker Command 305 | [go to top](#top) 306 | 307 | 308 | To see all available subcommands, simply type 309 | 310 | ```shell 311 | docker 312 | ``` 313 | 314 | This should give you a whole list of commands, and it can get quite lengthy. So in order to see what each command does, you may also use 315 | 316 | ```shell 317 | docker --help 318 | ``` 319 | 320 | **Then in order to run the commands, the signature is** 321 | 322 | ```shell 323 | docker 324 | ``` 325 | 326 | 327 | 328 | ### 3.3 Images and the Docker Hub 329 | [go to top](#top) 330 | 331 | 332 | Everything in Docker relies on **images**. In fact, you can think of images as snapshots of Docker containers, and containers as instances of images. 333 | 334 | ![Docker filesystems Busyboxrw](assets/docker-filesystems-busyboxrw.png) 335 | 336 | [Image Source]() 337 | 338 | More importantly, **images can be layered on top of one another** to produce a resulting container that you can then go ahead and interact with. This saves space, and simplifies the development of new images! 339 | 340 | You can download `dockerfiles` from the [Docker Hub](), which are files that Docker can use to create images. The Hub lists dockerfiles that can be used to produce images of all sorts of different OSes, distributions, and environments! 341 | 342 | There's dockerfiles for ROS, machine learning, web development, and a whole host of other environments out there for you to explore! 343 | 344 | 345 | 346 | ### 3.4 Image Commands 347 | [go to top](#top) 348 | 349 | 350 | Here are some common commands you can use with images 351 | 352 | **Search** 353 | 354 | ```shell 355 | docker search 356 | ``` 357 | 358 | **Get** 359 | 360 | ```shell 361 | docker pull 362 | docker pull : 363 | ``` 364 | 365 | **Remove Images** 366 | 367 | ```shell 368 | docker rmi 369 | ``` 370 | 371 | **See All Images in System** 372 | 373 | ```shell 374 | docker images 375 | ``` 376 | 377 | 378 | 379 | ### 3.5 Docker System Commands 380 | [go to top](#top) 381 | 382 | 383 | ```shell 384 | docker system info # Get info 385 | docker system df # Show disk usage 386 | 387 | docker prune -a # Delete all unused images 388 | docker prune -a --volumes # Delete all unused images, volumes included 389 | ``` 390 | 391 | > **A note on docker system df** 392 | > 393 | > ![1567496371287](assets/1567496371287.png) 394 | > 395 | > You might have noticed that there is a column called **RECLAIMABLE** space when using the `docker system df` command. This is simply space that is consumed by "unused images", that is, images with no containers based off of them running. (It's the size of images you can remove without breaking anything.) 396 | 397 | 398 | 399 | ### 3.6 Running Containers 400 | [go to top](#top) 401 | 402 | 403 | [Reference]() 404 | 405 | > **Note**: When you run a container, you are in fact instantiating a **new** container from the image you are running it from. Every run command creates a new container, which will take up space on your hard drive. 406 | > 407 | > If you instead wanted to execute commands inside a pre-existing, running container, use the `docker exec` command instead. 408 | 409 | **Basic Run** 410 | 411 | This command should drop you into a handy terminal inside the container from the get-go. 412 | 413 | ```shell 414 | docker run -it 415 | ``` 416 | 417 | **Useful Flags** 418 | 419 | Non-exhaustive list 420 | 421 | | Flag | Description | 422 | | --------------------------------------- | ------------------------------------------------------------ | 423 | | `-e=""` | Set environment variables. (Eg. `-e DISPLAY=$DISPLAY`) | 424 | | `--entrypoint=""` | Specify a command that will always be executed when the container starts. | 425 | | `--name=NAME` | Specify name for container | 426 | | `--net=NETWORK` or `--network=NETWORK` | `NETWORK` can be: bridge, none, host, network-name, network-id | 427 | | `--rm` | Delete the container once the daemon or container exits | 428 | | `-u ` | Set default user to run. (Note that this does not confer root privileges unless you grant them.) (A good note is to use `$(id -u)` to use the host's user ID.) | 429 | | `-v host_directory:container_directory` | Mount volumes/file directories inside the container. In this way, any changes made to the host directory will be reflected inside the container. | 430 | | `-w="WORKDIR"` | Set default working directory for container | 431 | 432 | > **Note**: You may come across the `docker create` command. 433 | > 434 | > All you need to know is that 435 | > 436 | > `docker run` = `docker create` + `docker start` + `docker attach` 437 | > 438 | > Which causes the container to be created, started, and then run in the foreground. 439 | 440 | 441 | 442 | ### 3.7 Volumes 443 | [go to top](#top) 444 | 445 | 446 | [Reference]() 447 | 448 | ![volumes on the Docker host](assets/types-of-mounts-volume.png) 449 | 450 | [Image Source]() 451 | 452 | Not only can containers mount volumes from your host system, but you can also create volumes for use with containers, and share volumes amongst containers too! 453 | 454 | 455 | 456 | #### **Container Volumes** 457 | 458 | Of course, you can only easily mount the volumes at container creation. 459 | 460 | ```shell 461 | docker run -it -v /volume_dir 462 | ``` 463 | 464 | So now a volume named `volume_dir` should be visible from the root. This will persist on your host filesystem! 465 | 466 | If you want to see where in your filesystem the volume is stored, use `docker inspect`, which should net you a couple of mount data. 467 | 468 | ```shell 469 | docker inspect 470 | ``` 471 | 472 | > **Note**: Using `docker rm` to delete a container will not delete its volumes unless you pass the `-v` flag. 473 | > 474 | > `docker rm -v ` 475 | 476 | 477 | 478 | #### **Mounting Volumes from Other Containers** 479 | 480 | **You can also mount volumes from other containers!** You might like to create containers which store volumes only! 481 | 482 | To mount volumes from another container, use 483 | 484 | ```shell 485 | docker run -it --volumes-from 486 | ``` 487 | 488 | 489 | 490 | #### **Standalone Volumes** 491 | 492 | You can also create standalone volumes. 493 | 494 | ```shell 495 | docker volume create 496 | docker volume ls # List volumes 497 | docker volume rm # Delete volumes 498 | ``` 499 | 500 | Once you do that you can mount volumes to specific directories within your containers, and then these can be shared appropriately. 501 | 502 | ```shell 503 | docker run -it -v :/mount_dir 504 | ``` 505 | 506 | You can do cool stuff like specifying volumes as read-only, and other cool stuff too! Just check out [the docs](). 507 | 508 | 509 | 510 | ### 3.8 Basic Container Operations 511 | [go to top](#top) 512 | 513 | 514 | Once you've created a container with the `docker run` command, we can then use some commands to mess around with our containers! 515 | 516 | ```shell 517 | # Start and stop container 518 | docker start 519 | docker stop 520 | 521 | # Copy data from container to host 522 | docker cp 523 | 524 | # Copy data from host to container 525 | docker cp 526 | 527 | # Delete container 528 | docker rm 529 | 530 | # List all running containers 531 | docker ps -a # Show all container statuses 532 | docker ps -as # Show all container statuses, with size 533 | docker ps -l # Show last container created 534 | ``` 535 | 536 | > **A note on virtual size** 537 | > 538 | > The virtual size of an image or container is the total on-disk size of all images it is composed of. But as images can be built on each other, and track diffs, your actual disk usage can be a lot smaller if you have many containers based off of the same image. 539 | > 540 | > > For example, if you have two images, `app-1` and `app-2`, and both are based on a common `distro` image/layer whose total size is 100MB, and `app-1` adds an additional 10MB but `app-2`adds an additional 20MB, the virtual sizes will be 110MB and 120MB respectively, but the total disk usage will only be 130MB since that base layer is shared between the two. 541 | > > 542 | > > [Source]() 543 | 544 | 545 | 546 | ### 3.9 Executing Commands 547 | [go to top](#top) 548 | 549 | 550 | Once you have a container up and running, you may execute commands in the container! 551 | 552 | ```shell 553 | # Run command in container 554 | docker exec 555 | 556 | # Start a shell in the container 557 | docker exec -it bash 558 | 559 | # Start a shell in the container with environment variables set 560 | docker exec -it -e VAR=VALUE bash 561 | ``` 562 | 563 | 564 | 565 | ### 3.10 Committing and Pushing Images 566 | [go to top](#top) 567 | 568 | 569 | If you know `git` you should find the terms `commit`, `push`, and `pull` familiar. :thinking: 570 | 571 | So if you recall, there's the Docker Hub where images can be pulled from into your machine for you to build containers off of. 572 | 573 | **Commit** 574 | 575 | Once you've created your own containers and modified it, you too can also create a new image on your system by **committing** it, so that you are able to make new containers based off of that committed image. 576 | 577 | > **Note**: Committing images will not save volumes! 578 | 579 | ```shell 580 | # Stop the container first 581 | docker stop 582 | 583 | # Commit image locally 584 | docker commit -m "Commit message" -a "Author name" 585 | 586 | # Commit image locally, but tied to a repository 587 | docker commit -m "Commit message" -a "Author name" / 588 | ``` 589 | 590 | The committed image should then show up in your list of images. 591 | 592 | ```shell 593 | docker images 594 | ``` 595 | 596 | **Push** 597 | 598 | Once you've committed an image, you may then **push** it to the Docker Hub! 599 | 600 | > **Note**: Committing and pushing an image in this way creates a tar file that will be uploaded to the Docker Hub, which is not a recommended way of pushing images to the Docker Hub. The recommended way is using `dockerfiles`, which requires a bit more planning and foresight, and will be talked about in a separate section. 601 | 602 | First we need to log in to the Docker Hub 603 | 604 | ```shell 605 | docker login -u 606 | ``` 607 | 608 | Then we can push our committed image appropriately! (repo_id is usually your username.) 609 | 610 | ```shell 611 | docker push / 612 | ``` 613 | 614 | Once pushed, you should be able to pull on another computer 615 | 616 | ```shell 617 | docker pull / 618 | ``` 619 | 620 | Nifty! 621 | 622 | 623 | 624 | ## 4. Dockerfiles 625 | 626 | ### 4.1 Introduction 627 | [go to top](#top) 628 | 629 | 630 | You can build new images off of Dockerfiles instead of `.tar` files! A Dockerfile simply tells Docker how to build an image. They're always named `Dockerfile`, with no extension. 631 | 632 | Here's an example of the Dockerfile that was used to build the hello-world image! 633 | 634 | ```docker 635 | FROM scratch 636 | COPY hello / 637 | CMD ["/hello"] 638 | ``` 639 | 640 | We can see that hello-world was build off of the `scratch` image (which is supposed to be explicitly empty image, and does not create a new image layer, actually. It's also a pun!) 641 | 642 | Then it copies the `hello` directory from the `scratch` image into your new image's `/` directory. And finally it runs the `/hello` program. 643 | 644 | > **Note**: Dockerfiles are run from top to bottom, and might result in a multi-layered image 645 | 646 | It should be fairly obvious that this is an **exceedingly powerful tool**! We can essentially reduce entire systems and environments to a couple of setup lines, and have Docker pull the necessary images or run the necessary setup steps to build containers! 647 | 648 | 649 | 650 | ### 4.2 Building Dockerfiles 651 | [go to top](#top) 652 | 653 | 654 | The command to use to build an image from a Dockerfile is quite simple. 655 | 656 | ```shell 657 | # Where the current directory is the directory in which the Dockerfile resides 658 | docker build . 659 | 660 | # Or if it isn't there... 661 | docker build -f path/to/Dockerfile . 662 | 663 | # Specify repository and tag to build image to 664 | docker build -t : . 665 | docker build -t : -t : . # Multiple works too! 666 | ``` 667 | 668 | 669 | 670 | ### 4.3 Build Context 671 | [go to top](#top) 672 | 673 | 674 | So actually there's a bit more subtlety in the `docker build` command. 675 | 676 | What actually happens when you run the command, is that **two** things actually get sent to the Docker daemon to build your image. 677 | 678 | 1. Your Dockerfile 679 | 2. A build context 680 | - This is the directory (and all subdirectories) specified in the build command. 681 | - `docker build ` or `docker build -f dockerfile_path ` 682 | 683 | The build context is important because any file you want to `ADD` or `COPY` into your container has to be included in the build context, which is uploaded to the Docker daemon. 684 | 685 | It's also important to ensure that no sensitive information is stored in the build context in case you accidentally send them into your container. You can exclude files from inclusion into the build context using **dockerignore** files. 686 | 687 | And if anything, it helps with the speed of the build if you can shrink the size of the build context so less stuff has to be transferred to the Docker daemon before it can actually start building your image. 688 | 689 | 690 | 691 | ### 4.4 Dockerignore 692 | [go to top](#top) 693 | 694 | 695 | You can create a `.dockerignore` file in the root directory of your build context. The exclusion pattern is the same as [.gitignore files](), and will ignore files in the build context, and any files added using the `ADD` or `COPY` commands, if it matches the pattern. 696 | 697 | This is a handy tool to help you prevent accidentally sending credentials or passwords into your containers! 698 | 699 | 700 | 701 | ### 4.5 Image Layers 702 | [go to top](#top) 703 | 704 | 705 | So if you remember from the introduction to Docker, images are made of layers. 706 | 707 | ![Docker filesystems Busyboxrw](assets/docker-filesystems-busyboxrw.png) 708 | 709 | But what actually happens during the building of an image, is each successive layer is built in its own intermediate container to produce a read-only layer, and then successive layers are built and stacked on top of each other until you have your final image with a writable container layer on top. 710 | 711 | ![Image result for docker layers](assets/container-layers.jpg) 712 | 713 | [Image Source]() 714 | 715 | With Dockerfiles, you have extra control over this process! Because you can use Dockerfile commands to specify when new layers should begin. And this allows you to do cool stuff like utilising the build cache to speed up your builds. But let's take things slow first. 716 | 717 | 718 | 719 | ### 4.6 Dockerfile Comments 720 | [go to top](#top) 721 | 722 | 723 | Comments are specified with `#` 724 | 725 | ```dockerfile 726 | # Like so! 727 | ``` 728 | 729 | 730 | 731 | ### 4.7 Dockerfile Command Overview 732 | [go to top](#top) 733 | 734 | 735 | [Reference]() 736 | 737 | [Cheatsheet]() 738 | 739 | All Dockerfile commands are written in full uppercase form. 740 | 741 | The list is non-exhaustive. 742 | 743 | | Command | Description | Makes new Layer | 744 | | :-----------: | :----------------------------------------------------------: | :-------------: | 745 | | `FROM` | Specifies a parent image (you can specify multiple parent images! They'll just form multiple layers.) | * | 746 | | `MAINTAINER` | (Depreciated) Specify maintainer | | 747 | | `LABEL` | Metadata in key=value form | | 748 | | `ENV` | Set environment variable | | 749 | | `RUN` | Run command, and create image layer (triggers on build) | * | 750 | | `COPY` | Copy files and directories to container | * | 751 | | `ADD` | Like `COPY`, but can untar and download online files | * | 752 | | `CMD` | Specify the default command that is run when the built image is launched (only one can exist) | | 753 | | `WORKDIR` | Specify working directory for instructions that follow | | 754 | | `ARG` | Build-time variable | | 755 | | `ENTRYPOINT` | Specifies default command that is run when the built image is launched (but this is **not ignored** when the Docker container runs with command line parameters.) (Use this is the container is meant to be used like an executable.) | | 756 | | `EXPOSE` | Expose a port | | 757 | | `VOLUME` | Mount a volume | | 758 | | `ONBUILD` | Specify a Dockerfile command that is executed when the image specified by this Dockerfile is used as a base for another build. (Triggered immediately after the `FROM` instruction.) | | 759 | | `STOPSIGNAL` | Specify the system call that is sent to the container to exit. | | 760 | | `HEALTHCHECK` | Specify a command to check if a container is healthy. You can set number of retires, check interval, and check timeout. | | 761 | | `SHELL` | Change the default shell that is used to run the `RUN`, `CMD`, and `ENTRYPOINT` commands. | | 762 | 763 | 764 | 765 | ### 4.8 Shell vs Exec Form 766 | [go to top](#top) 767 | 768 | 769 | There are multiple ways to specify how Docker should run a command with the `RUN` and `CMD` commands. Namely the **shell** and **exec** forms. 770 | 771 | - shell: `RUN ` 772 | - exec: `["executable", "param_1", "param_2", ...]` 773 | 774 | The shell form is run in a shell (`/bin/sh -c` on Linux, and `cmd /S /C` on Windows.) 775 | 776 | **Important Differences** 777 | 778 | - The exec form is **preferred** 779 | - The exec form **requires words to be enclosed by double quotes only** (as it is parsed as a JSON array) 780 | - The exec form does not invoke a command shell, so variable substitution does not happen 781 | - If you wanted it to happen, use either the shell form, or use something like `RUN ["sh", "-c", "echo $HOME"]` 782 | 783 | 784 | 785 | ### 4.9 An Example Dockerfile 786 | [go to top](#top) 787 | 788 | 789 | Let's build a Dockerfile, shall we? Let's do one that'll let you print dragons :eyes: 790 | 791 | Remember again that `FROM`, `RUN`, `ADD`, and `COPY` will create new layers. 792 | 793 | > **Note**: Due to caching issues, if you need to run apt-get update, always do it in the same command as the apt-get install. 794 | 795 | ```dockerfile 796 | # Layer 1 797 | FROM ubuntu 798 | 799 | # Layer 2, install deps 800 | RUN apt-get update && apt-get install -y \ 801 | cowsay \ 802 | fortune 803 | 804 | # Since cowsay installs to /usr/games 805 | ENV PATH $PATH:/usr/games 806 | WORKDIR /usr/share/cowsay/cows 807 | 808 | # Layer 3 and 4, download methylDragon ASCII and put them in the working directory 809 | ADD https://raw.githubusercontent.com/methylDragon/fun-stuff/master/cowsay/methylDragon/methyldragon.cow . 810 | ADD https://raw.githubusercontent.com/methylDragon/fun-stuff/master/cowsay/methylDragon/methylhalf.cow . 811 | 812 | # Layer 5 and 6, add aliases 813 | RUN echo 'alias methyl="fortune | cowsay -f methyldragon"' >> ~/.bashrc 814 | RUN echo 'alias methylhalf="fortune | cowthink -f methylhalf"' >> ~/.bashrc 815 | 816 | # Reset the working directory 817 | WORKDIR / 818 | CMD fortune | cowsay -f methyldragon && bash 819 | ``` 820 | 821 | ![1567575976437](assets/1567575976437.png) 822 | 823 | We'll build and run some commands! 824 | 825 | ```shell 826 | # Build the image 827 | docker build -t methyldragon/methylsay . 828 | ``` 829 | 830 | And spin up the container! 831 | 832 | ```shell 833 | # Then we can spin up a container and see what it does 834 | # It should start with a fortune from me to begin with, and drop you into a terminal 835 | # No worries about deleting the container, since it deletes itself on exit 836 | docker run -it --rm methyldragon/methylsay 837 | 838 | # And now that we're inside the container, we can try some commands! 839 | methyl # Get a fortune with the full methyl 840 | methylhalf # Get a random fortune with the half methyl 841 | 842 | methyl Rawr 843 | methylhalf Nice! 844 | ``` 845 | 846 | 847 | 848 | ### 4.10 The Proper Way to Layer 849 | [go to top](#top) 850 | 851 | 852 | If you actually rebuilt the Dockerfile again with some changes nearing the end of the file, you'll notice that it will build waaaaay faster than the initial build. 853 | 854 | This is because, again, images are layered on top of each other, and Docker will only rebuild images starting from the layer with a change in it, all the way to the outermost image. 855 | 856 | **As such, it is a best practice to order your Dockerfile from less frequently changed to more frequently changed**. 857 | 858 | A good order suggested by the Docker docs is, starting from the top: 859 | 860 | > 1. Install tools you need to build your application 861 | > 2. Install or update library dependencies 862 | > 3. Generate your application 863 | 864 | 865 | 866 | ### 4.11 Nifty Multi-Stage Build Tricks 867 | [go to top](#top) 868 | 869 | 870 | So with each `FROM` command you can actually specify a name for the build stage. 871 | 872 | ```dockerfile 873 | FROM scratch AS stage_name_1 874 | FROM ubuntu AS stage_name_2 875 | ``` 876 | 877 | This means that you can start copying from the previous built images by name 878 | 879 | ```shell 880 | COPY --from=stage_name /src_dir /dest_dir 881 | ``` 882 | 883 | And you can also **stop building at a particular build stage** using the `--target` argument 884 | 885 | ```shell 886 | docker build --target stage_name_1 . 887 | ``` 888 | 889 | 890 | 891 | ``` 892 | . . 893 | . |\-^-/| . 894 | /| } O.=.O { |\ 895 | ``` 896 | 897 | ​ 898 | 899 | ------ 900 | 901 | [![Yeah! Buy the DRAGON a COFFEE!](assets/COFFEE%20BUTTON%20%E3%83%BE(%C2%B0%E2%88%87%C2%B0%5E).png)](https://www.buymeacoffee.com/methylDragon) 902 | 903 | --------------------------------------------------------------------------------