├── src ├── version ├── lib │ ├── dockerutils.bashc │ └── plugins.bashc ├── importcon.bashc ├── mergecon.bashc ├── minicon.bashc └── minidock.bashc ├── usecases ├── uc4 │ ├── Dockerfile │ └── uc4.sh ├── uc2 │ ├── Dockerfile │ └── uc2.sh ├── uc5 │ ├── Dockerfile │ └── uc5.sh ├── uc3 │ ├── Dockerfile │ └── uc3.sh └── uc1 │ └── uc1.sh ├── minicon.bashcbuild ├── doc ├── man │ ├── mergecon.1 │ ├── importcon.1 │ ├── minidock.1 │ └── minicon.1 ├── mergecon.md ├── importcon.md ├── minidock.md └── minicon.md ├── LICENSE └── README.md /src/version: -------------------------------------------------------------------------------- 1 | VERSION=1.4-2 -------------------------------------------------------------------------------- /usecases/uc4/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu 2 | RUN apt-get update && apt-get install -y ffmpeg 3 | -------------------------------------------------------------------------------- /usecases/uc2/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu 2 | RUN apt-get update && apt-get install -y ssh iproute2 iputils-ping wget 3 | -------------------------------------------------------------------------------- /usecases/uc5/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu 2 | RUN apt-get update && apt-get install -y --force-yes apache2 3 | EXPOSE 80 443 4 | VOLUME ["/var/www", "/var/log/apache2", "/etc/apache2"] 5 | ENTRYPOINT ["/usr/sbin/apache2ctl", "-D", "FOREGROUND"] 6 | -------------------------------------------------------------------------------- /usecases/uc3/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM node:latest 2 | 3 | # Create app directory 4 | RUN mkdir -p /usr/src/app 5 | WORKDIR /usr/src/app 6 | 7 | # Install app dependencies 8 | COPY miniapp/package.json /usr/src/app/ 9 | RUN npm install 10 | 11 | # Bundle app source 12 | COPY miniapp /usr/src/app 13 | 14 | EXPOSE 3000 -------------------------------------------------------------------------------- /minicon.bashcbuild: -------------------------------------------------------------------------------- 1 | PACKAGE=minicon 2 | VERSION=1.4-2 3 | SUMMARY="MiniCon - Minimization of Container Filesystems" 4 | LICENSE="Apache 2.0" 5 | URL="https://github.com/grycap/minicon" 6 | DESCRIPTION="MiniCon - Minimization of Container Filesystems 7 | **minicon** aims at reducing the footprint of the filesystem for arbitrary 8 | the container, just adding those files that are needed. That means that the 9 | other files in the original container are removed. 10 | **minidock** is a helper to use minicon for Docker containers." 11 | PACKAGER="Carlos de Alfonso " 12 | # DEPENDS is for DEB packages 13 | DEPENDS="bash, jq, tar, libc-bin, coreutils, tar, rsync, file, strace" 14 | # REQUIRES is for RPM packages 15 | REQUIRES="bash, jq, tar, coreutils, tar, rsync, file, strace, glibc-common, which" 16 | 17 | /usr/share/$PACKAGE/;src/version:LICENSE 18 | /usr/share/$PACKAGE/doc;doc/*.md 19 | ;doc/man/*.1;gzip -fk ${FILEPATH} 20 | /usr/share/man/man1;doc/man/*.gz 21 | /usr/bin/;src/minicon:src/minidock:src/importcon:src/mergecon;bashc -o $SOURCEFOLDER/$FILEPATH -cCR $SOURCEFOLDER/${FILEPATH}.bashc;chmod +x $SOURCEFOLDER/$FILEPATH 22 | -------------------------------------------------------------------------------- /usecases/uc5/uc5.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # minicon - Minimization of filesystems for containers 4 | # https://github.com/grycap/minicon 5 | # 6 | # Copyright (C) GRyCAP - I3M - UPV 7 | # Developed by Carlos A. caralla@upv.es 8 | # 9 | # Licensed under the Apache License, Version 2.0 (the "License"); 10 | # you may not use this file except in compliance with the License. 11 | # You may obtain a copy of the License at 12 | # 13 | # http://www.apache.org/licenses/LICENSE-2.0 14 | # 15 | # Unless required by applicable law or agreed to in writing, software 16 | # distributed under the License is distributed on an "AS IS" BASIS, 17 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 18 | # See the License for the specific language governing permissions and 19 | # limitations under the License. 20 | # 21 | 22 | UC=5 23 | UCFOLDER=usecases/uc${UC} 24 | REFIMAGE=minicon:uc${UC}fat 25 | MINICONIMAGE=minicon:uc${UC} 26 | 27 | CURDIR=$(dirname $0) 28 | MINICONDIR=$(readlink -f $CURDIR/../..) 29 | 30 | set -x 31 | docker build $UCFOLDER -t $REFIMAGE 32 | docker images $REFIMAGE 33 | $MINICONDIR/minidock --apt -i $REFIMAGE -t $MINICONIMAGE --plugin-all 34 | docker images $MINICONIMAGE 35 | -------------------------------------------------------------------------------- /usecases/uc1/uc1.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # minicon - Minimization of filesystems for containers 4 | # https://github.com/grycap/minicon 5 | # 6 | # Copyright (C) GRyCAP - I3M - UPV 7 | # Developed by Carlos A. caralla@upv.es 8 | # 9 | # Licensed under the Apache License, Version 2.0 (the "License"); 10 | # you may not use this file except in compliance with the License. 11 | # You may obtain a copy of the License at 12 | # 13 | # http://www.apache.org/licenses/LICENSE-2.0 14 | # 15 | # Unless required by applicable law or agreed to in writing, software 16 | # distributed under the License is distributed on an "AS IS" BASIS, 17 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 18 | # See the License for the specific language governing permissions and 19 | # limitations under the License. 20 | # 21 | 22 | UC=1 23 | UCFOLDER=usecases/uc${UC} 24 | CURDIR=$(dirname $0) 25 | MINICONDIR=$(readlink -f $CURDIR/../..) 26 | 27 | set -x 28 | docker images ubuntu:latest 29 | docker run --rm -it -v $MINICONDIR:/tmp/minicon ubuntu:latest /tmp/minicon/minicon -t /tmp/minicon/$UCFOLDER/uc${UC}.tar -E bash -E ls -E mkdir -E less -E cat -E find 30 | docker import $MINICONDIR/$UCFOLDER/uc${UC}.tar minicon:uc${UC} 31 | docker images minicon:uc${UC} 32 | -------------------------------------------------------------------------------- /usecases/uc4/uc4.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # minicon - Minimization of filesystems for containers 4 | # https://github.com/grycap/minicon 5 | # 6 | # Copyright (C) GRyCAP - I3M - UPV 7 | # Developed by Carlos A. caralla@upv.es 8 | # 9 | # Licensed under the Apache License, Version 2.0 (the "License"); 10 | # you may not use this file except in compliance with the License. 11 | # You may obtain a copy of the License at 12 | # 13 | # http://www.apache.org/licenses/LICENSE-2.0 14 | # 15 | # Unless required by applicable law or agreed to in writing, software 16 | # distributed under the License is distributed on an "AS IS" BASIS, 17 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 18 | # See the License for the specific language governing permissions and 19 | # limitations under the License. 20 | # 21 | 22 | UC=4 23 | UCFOLDER=usecases/uc${UC} 24 | REFIMAGE=minicon:uc${UC}fat 25 | MINICONIMAGE=minicon:uc${UC} 26 | 27 | CURDIR=$(dirname $0) 28 | MINICONDIR=$(readlink -f $CURDIR/../..) 29 | 30 | set -x 31 | docker build $UCFOLDER -t $REFIMAGE 32 | docker images $REFIMAGE 33 | docker run --rm -it -v $MINICONDIR:/tmp/minicon $REFIMAGE /tmp/minicon/minicon -t /tmp/minicon/$UCFOLDER/uc${UC}.tar -E ffmpeg 34 | docker import $MINICONDIR/$UCFOLDER/uc${UC}.tar $MINICONIMAGE 35 | docker images $MINICONIMAGE 36 | -------------------------------------------------------------------------------- /usecases/uc3/uc3.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # minicon - Minimization of filesystems for containers 4 | # https://github.com/grycap/minicon 5 | # 6 | # Copyright (C) GRyCAP - I3M - UPV 7 | # Developed by Carlos A. caralla@upv.es 8 | # 9 | # Licensed under the Apache License, Version 2.0 (the "License"); 10 | # you may not use this file except in compliance with the License. 11 | # You may obtain a copy of the License at 12 | # 13 | # http://www.apache.org/licenses/LICENSE-2.0 14 | # 15 | # Unless required by applicable law or agreed to in writing, software 16 | # distributed under the License is distributed on an "AS IS" BASIS, 17 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 18 | # See the License for the specific language governing permissions and 19 | # limitations under the License. 20 | # 21 | 22 | UC=3 23 | UCFOLDER=usecases/uc${UC} 24 | REFIMAGE=minicon:uc${UC}fat 25 | MINICONIMAGE=minicon:uc${UC} 26 | 27 | CURDIR=$(dirname $0) 28 | MINICONDIR=$(readlink -f $CURDIR/../..) 29 | 30 | set -x 31 | express -f ./usecases/uc3/miniapp 32 | docker build $UCFOLDER -t $REFIMAGE 33 | docker images $REFIMAGE 34 | docker run --rm -it -v $MINICONDIR:/tmp/minicon $REFIMAGE /tmp/minicon/minicon -t /tmp/minicon/$UCFOLDER/uc${UC}.tar -E node -I /usr/src/app 35 | docker import $MINICONDIR/$UCFOLDER/uc${UC}.tar $MINICONIMAGE 36 | docker images $MINICONIMAGE 37 | -------------------------------------------------------------------------------- /src/lib/dockerutils.bashc: -------------------------------------------------------------------------------- 1 | function generate_dockerimagename() { 2 | local NEWNAME 3 | NEWNAME="$(cat /proc/sys/kernel/random/uuid)" 4 | NEWNAME="${NEWNAME%%-*}" 5 | echo "$NEWNAME" 6 | } 7 | 8 | function correct_dockerimagename() { 9 | local IMNAME="$1" 10 | local BASE TAG 11 | 12 | # Correct NEWNAME 13 | IFS=':' read BASE TAG <<< "${IMNAME}" 14 | if [ "$TAG" == "" ]; then 15 | IMNAME="${IMNAME}:latest" 16 | fi 17 | echo "$IMNAME" 18 | } 19 | 20 | _INSPECT_STRING= 21 | _INSPECT_IMAGE= 22 | function get_config_field() { 23 | local IMAGE="$1" 24 | local FIELD="$2" 25 | local RAW_JQ="$3" 26 | local C_RESULT 27 | 28 | if [ "$_INSPECT_IMAGE" != "$IMAGE" ]; then 29 | _INSPECT_STRING="$(docker inspect $IMAGE)" 30 | _INSPECT_IMAGE="$IMAGE" 31 | fi 32 | 33 | C_RESULT="$(echo "$_INSPECT_STRING" | jq -r "if .[].Config.${FIELD} != null then if .[].Config.${FIELD}${RAW_JQ} | type == \"array\" then .[].Config.${FIELD}${RAW_JQ} | .[] else .[].Config.${FIELD}${RAW_JQ} end else null end")" 34 | echo "$(bashc.trim "$C_RESULT")" 35 | } 36 | 37 | function get_config_field_raw() { 38 | local IMAGE="$1" 39 | local FIELD="$2" 40 | local RAW_JQ="$3" 41 | local C_RESULT 42 | 43 | if [ "$_INSPECT_IMAGE" != "$IMAGE" ]; then 44 | _INSPECT_STRING="$(docker inspect $IMAGE)" 45 | _INSPECT_IMAGE="$IMAGE" 46 | fi 47 | 48 | C_RESULT="$(echo "$_INSPECT_STRING" | jq -r "if .[].Config.${FIELD} != null then .[].Config.${FIELD}${RAW_JQ} else null end")" 49 | echo "$(bashc.trim "$C_RESULT")" 50 | } 51 | -------------------------------------------------------------------------------- /usecases/uc2/uc2.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # minicon - Minimization of filesystems for containers 4 | # https://github.com/grycap/minicon 5 | # 6 | # Copyright (C) GRyCAP - I3M - UPV 7 | # Developed by Carlos A. caralla@upv.es 8 | # 9 | # Licensed under the Apache License, Version 2.0 (the "License"); 10 | # you may not use this file except in compliance with the License. 11 | # You may obtain a copy of the License at 12 | # 13 | # http://www.apache.org/licenses/LICENSE-2.0 14 | # 15 | # Unless required by applicable law or agreed to in writing, software 16 | # distributed under the License is distributed on an "AS IS" BASIS, 17 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 18 | # See the License for the specific language governing permissions and 19 | # limitations under the License. 20 | # 21 | 22 | UC=2 23 | UCFOLDER=usecases/uc${UC} 24 | REFIMAGE=minicon:uc${UC}fat 25 | MINICONIMAGE=minicon:uc${UC} 26 | 27 | CURDIR=$(dirname $0) 28 | MINICONDIR=$(readlink -f $CURDIR/../..) 29 | 30 | docker build $UCFOLDER -t $REFIMAGE 31 | 32 | cat > $UCFOLDER/execfile-cmd <<\EOF 33 | ssh localhost 34 | /usr/bin/ssh localhost 35 | /bin/ping -c 1 www.google.es 36 | EOF 37 | 38 | set -x 39 | docker images $REFIMAGE 40 | docker run --privileged --rm -it -v $MINICONDIR:/tmp/minicon $REFIMAGE bash -c "apt-get install -y strace && /tmp/minicon/minicon -t /tmp/minicon/$UCFOLDER/uc${UC}.tar --plugin=strace:execfile=/tmp/minicon/$UCFOLDER/execfile-cmd -E bash -E ssh -E ip -E id -E cat -E ls -E mkdir -E ping -E wget" 41 | docker import $MINICONDIR/$UCFOLDER/uc${UC}.tar $MINICONIMAGE 42 | docker images $MINICONIMAGE 43 | -------------------------------------------------------------------------------- /doc/man/mergecon.1: -------------------------------------------------------------------------------- 1 | .\" Manpage for mergecon. 2 | .\" Contact caralla@upv.es to correct errors or typos. 3 | .TH man 1 "01 Mar 2018" "1.2-1" "mergecon man page" 4 | .SH NAME 5 | mergecon - merge Docker containers 6 | .SH SYNOPSIS 7 | mergecon 8 | 9 | .SH DESCRIPTION 10 | Docker containers are built from different layers, and 11 | .B mergecon 12 | is a tool that merges the filesystems of two different container images. It creates a new container image that is built from the combination of the layers of the filesystems of the input containers. 13 | 14 | .SH WHY MERGECON? 15 | 16 | If you create a minimal application filesystem (i.e. using minicon), you will be able to run your application, but you will not have any other application available for (e.g.) debugging your application. 17 | 18 | Using mergecon, you will be able to overlay the files of your container image to other existing container image. In this way, you can overlay your minicon resulting application over a whole ubuntu:latest container. The effect is that you will have the support of a whole operating environment over the minimal container. 19 | 20 | On the other side, if you have an application that needs to be compiled and installed, you can create a one-liner installation, and combine that layer with other container. That means that you will be able to compile such image in (e.g.) ubuntu, and create a final container with other flavor (e.g. CentOS). 21 | 22 | .SH OPTIONS 23 | .B --first | -1 24 | Name of the first container image (will use docker save to dump it) 25 | 26 | .B --second | -2 27 | Name of the second container image (will use docker save to dump it) the default behaviour gives more priority to the second image. I.e. in case of overlapping files in both input images, the files in the second image will be exposed to the final image. 28 | 29 | .B --tag | -t 30 | Name of the resulting container image. If not provided, the resulting name will be the concatenation of the names of the two input images: : (the : in the input image names is removed). 31 | 32 | .B --working | -w 33 | Working folder. If not provided will create a temporary folder in /tmp 34 | 35 | .B --list | -l 36 | Lists the layers in the images (useful for filesystem composition). 37 | 38 | .B --file | -f 39 | tar file where the resulting image will be created. If not provided, the image will be loaded into docker. 40 | 41 | .B --keeptemporary | -k 42 | Keeps the temporary folder. Otherwise, the folder is removed (if it is created by mergecon). 43 | 44 | .B --verbose | -v 45 | Gives more information about the procedure. 46 | 47 | .B --debug 48 | Gives a lot more information about the procedure. 49 | 50 | .SH SEE ALSO 51 | minicon(1) 52 | 53 | .SH AUTHOR 54 | Carlos de Alfonso (caralla@upv.es) -------------------------------------------------------------------------------- /doc/man/importcon.1: -------------------------------------------------------------------------------- 1 | .\" Manpage for importcon. 2 | .\" Contact caralla@upv.es to correct errors or typos. 3 | .TH man 1 "01 Mar 2018" "1.2-1" "importcon man page" 4 | .SH NAME 5 | importcon - IMPORT CONtainer copying features 6 | .SH SYNOPSIS 7 | importcon 8 | .SH DESCRIPTION 9 | When a container is running, it is possible to export its filesystem to a tar file, using the command 10 | .I docker export . 11 | Later, it is possible to import that filesystem into Docker to be used as a Docker image, using a command like 12 | .I docker import . 13 | The problem is that the new container has lost all the parameters from the original image (i.e. ENTRYPOINT, USER, CMD, etc.). 14 | 15 | .B importcon 16 | is a script that enables to import a filesystem exported using 17 | .I docker export 18 | into Docker, and to copy the parameters from the original image (i.e. ENTRYPOINT, USER, CMD, VOLUME, etc.) 19 | 20 | 21 | .SH OPTIONS 22 | .B --image | -i 23 | Name of the existing image to copy the parameters. 24 | 25 | .B --tag | -t 26 | Tag for the image that will be created from the tarfile (random if not provided) 27 | 28 | .B --env | -E 29 | Copy ENV settings 30 | 31 | .B --entrypoint | -e 32 | Copy ENTRYPOINT settings 33 | 34 | .B --expose | -x 35 | Copy EXPOSE settings 36 | 37 | .B --onbuild | -o 38 | Copy ONBUILD settings 39 | 40 | .B --user | -u 41 | Copy USER settings 42 | 43 | .B --volume | -V 44 | Copy VOLUME settings 45 | 46 | .B --workdir | -w 47 | Copy WORKDIR settins 48 | 49 | .B --cmd | -c 50 | Copy CMD settings 51 | 52 | .B --all | -A 53 | Copy all the previous settings: ENV, ENTRYPOINT, EXPOSE, ONBUILD, USER, VOLUME, WORKDIR and CMD. 54 | 55 | .B --version 56 | Displays the version of the package and exits. 57 | 58 | .B --help | -h 59 | Shows this help and exits. 60 | 61 | .SH EXAMPLES 62 | 63 | If we take the next Dockerfile 64 | 65 | FROM ubuntu 66 | RUN apt-get update && apt-get install -y --force-yes apache2 67 | EXPOSE 80 443 68 | VOLUME ["/var/www", "/var/log/apache2", "/etc/apache2"] 69 | ENTRYPOINT ["/usr/sbin/apache2ctl", "-D", "FOREGROUND"] 70 | 71 | we can build an image using the command 72 | 73 | .I docker build . -t tests:apache 74 | 75 | Then we can run a container 76 | 77 | .I docker run --rm -id -p 10000:80 tests:apache 78 | 79 | And export its filesystem to a file: 80 | 81 | .I docker export hungry_payne -o myapache.tar 82 | 83 | If we simply import the file back to a Docker image 84 | 85 | .I docker import myapache.tar tests:myapache 86 | 87 | we can check the differences between the configuration of each image for the new containers: 88 | 89 | .I docker inspect tests:apache | jq '.[0].Config' 90 | { 91 | ... 92 | "Entrypoint": [ 93 | "/usr/sbin/apache2ctl", 94 | "-D", 95 | "FOREGROUND" 96 | ], 97 | ... 98 | } 99 | 100 | .I docker inspect tests:myapache | jq '.[0].Config' 101 | { 102 | ... 103 | "Entrypoint": null, 104 | ... 105 | } 106 | 107 | 108 | We can see that our new image has all the setting set to empty. We can use 109 | .B importcon 110 | to import the container, taking the original image as reference: 111 | 112 | .I importcon -t tests:apacheimportcon -i tests:apache myapache.tar -A 113 | .I docker inspect tests:apacheimportcon | jq '.[0].Config' 114 | { 115 | ... 116 | "Entrypoint": [ 117 | "/usr/sbin/apache2ctl", 118 | "-D", 119 | "FOREGROUND" 120 | ], 121 | ... 122 | } 123 | 124 | .SH SEE ALSO 125 | minicon(1), minidock(1), jq(1) 126 | 127 | .SH AUTHOR 128 | Carlos de Alfonso (caralla@upv.es) -------------------------------------------------------------------------------- /doc/mergecon.md: -------------------------------------------------------------------------------- 1 | # mergecon - MERGE CONtainer filesystems 2 | 3 | Docker containers are built from different layers, and **mergecon** is a tool that merges the filesystems of two different container images. It creates a new container image that is built from the combination of the layers of the filesystems of the input containers. 4 | 5 | ## 1. Why mergecon? 6 | 7 | If you create a minimal application filesystem (i.e. using **minicon**), you will be able to run your application, but you will not have any other application available for (e.g.) debugging your application. 8 | 9 | Using **mergecon**, you will be able to overlay the files of your container image to other existing container image. In this way, you can overlay your **minicon** resulting application over a whole **ubuntu:latest** container. The effect is that you will have the support of a whole operating environment over the minimal container. 10 | 11 | ### 1.2 Other use cases 12 | 1. Even getting the resulting image by applying **minicon** over an **ubuntu:latest** derived container, you can overlay such image over an **alpine:latest** image (or other). The effect is that you will have your application over an **alpine** host. This is of interest in case that you want to run GNU applications over non-GNU systems(e.g. alpine). 13 | 14 | 1. If you have an application that needs to be compiled and installed, you can create a one-liner installation, and combine that layer with other container. That means that you will be able to compile such image in (e.g.) ubuntu, and create a final container with other flavor (e.g. CentOS). 15 | 16 | ## 2. Installation 17 | 18 | ### 2.1 From packages 19 | 20 | You can get the proper package (.deb o .rpm) from the [Releases page](https://github.com/grycap/minicon/releases) and install it using the appropriate package manager. 21 | 22 | **Ubuntu/Debian** 23 | 24 | ```bash 25 | $ apt update 26 | $ apt install ./minicon-1.2-1.deb 27 | ``` 28 | 29 | **CentOS/Fedora/RedHat** 30 | 31 | ```bash 32 | $ yum install epel-release 33 | $ yum install ./minicon-1.2-1.noarch.rpm 34 | ``` 35 | 36 | ### 2.2 From sources 37 | 38 | **mergecon** is a bash script that deals with **docker** commands. **mergecon** is part of the **minicon** package, and so you just simply need to have a working linux with bash installed and get the code: 39 | 40 | ```bash 41 | $ git clone https://github.com/grycap/minicon 42 | ``` 43 | 44 | In that folder you'll have the **mergecon** application. I would suggest to put it in the _/opt_ folder. Otherwise leave it in a folder of your choice: 45 | 46 | ```bash 47 | $ mv minicon /opt 48 | ``` 49 | 50 | #### 2.2.1 Dependencies 51 | 52 | **mergecon** depends on the commands _tar_ and _jq_. So, you need to install the proper packages in your system. 53 | 54 | **Ubuntu** 55 | 56 | ```bash 57 | $ apt-get install tar jq 58 | ``` 59 | 60 | **CentOS** 61 | ```bash 62 | $ yum install tar jq 63 | ``` 64 | ## 3. Usage 65 | 66 | **mergecon** has a lot of options. You are advised to run ```./mergecon --help``` to get the latest information about the usage of the application. 67 | 68 | The basic syntax is 69 | 70 | ```bash 71 | $ ./mergecon 72 | ``` 73 | 74 | Some options are: 75 | - **--first | -1 **: Name of the first container image (will use docker save to dump it) 76 | - **--second | -2 **: Name of the second container image (will use docker save to dump it) the default behaviour gives more priority to the second image. I.e. in case of overlapping files in both input images, the files in the second image will be exposed to the final image. 77 | - **--tag | -t **: Name of the resulting container image. If not provided, the resulting name will be the concatenation of the names of the two input images: : (the : in the input image names is removed). 78 | - **--working | -w **: Working folder. If not provided will create a temporary folder in /tmp 79 | - **--list | -l**: Lists the layers in the images (useful for filesystem composition). 80 | - **--file | -f **: tar file where the resulting image will be created. If not provided, the image will be loaded into docker. 81 | - **--keeptemporary | -k**: Keeps the temporary folder. Otherwise, the folder is removed (if it is created by mergecon). 82 | - **--verbose | -v**: Gives more information about the procedure. 83 | - **--debug**: Gives a lot more information about the procedure. 84 | 85 | -------------------------------------------------------------------------------- /doc/man/minidock.1: -------------------------------------------------------------------------------- 1 | .\" Manpage for minidock. 2 | .\" Contact caralla@upv.es to correct errors or typos. 3 | .TH man 1 "01 Mar 2018" "1.2-1" "minidock man page" 4 | .SH NAME 5 | minidock - minimization of Docker containers 6 | .SH SYNOPSIS 7 | minidock [ --docker-opts ] -- 8 | 9 | .SH DESCRIPTION 10 | When you run Docker containers, you usually run a system that has a whole Operating System and your specific application. The result is that the footprint of the container is bigger than needed. 11 | 12 | .B minidock 13 | aims at reducing the footprint of the Docker containers, by just including in the container those files that are needed. That means that the other files in the original container are removed. 14 | 15 | .B minidock 16 | is based on [**minicon**](doc/minicon.md), [**importcon**](doc/importcon.md) and [**mergecon**](doc/mergecon.doc), and hides the complexity of creating a container, mapping minicon, guessing parameters such as the entrypoint or the default command, creating the proper commandline, etc. 17 | 18 | .SH WHY MINIDOCK? 19 | 20 | Reducing the footprint of one container is of special interest, to redistribute the container images and saving the storage space in your premises. There are also security reasons to minimize the unneeded application or environment available in one container image (e.g. if the container does not need a compiler, why should it be there? maybe it would enable to compile a rootkit). 21 | 22 | .B minicon 23 | is a tool that enables a fine grain minimization for any type of filesystem, but it is possible to use it to reduce Docker images following the next pipeline: 24 | 25 | - Preparing a Docker container with the dependencies of **minicon** 26 | - Guessing the entrypoint and the default command for the container. 27 | - Running **minicon** for these commands (maping the proper folders to get the resulting tar file). 28 | - Using **importcon** to import the resulting file to copy the entrypoint and other settings. 29 | - etc. 30 | 31 | .B minidock 32 | is a one-liner that automates that procedure to make that reducing a container consist in just to convert a 33 | 34 | .IP 35 | .B $ docker run --rm -it myimage myapp 36 | .RE 37 | 38 | into 39 | 40 | .IP 41 | .B $ minicon -i myimage -t myimage:minicon -- myapp 42 | .RE 43 | 44 | .SH OPTIONS 45 | 46 | .B 47 | Is the whole commandline to be analised in the run. These are the same parameters that you would pass to "docker run ... ". If no run is provided, minidock will use the default command. And if the image as no default command, it will try to use one of the -R or -E commands. 48 | 49 | .B 50 | If you need them, you can include some options that will be raw-passed to the docker run command used during the analysis. (i.e. minidock will execute docker run ...). 51 | 52 | .B 53 | If you need to, you can add some minicon-specific options. The supported options are --include --exclude --plugin 54 | 55 | .B --image | -i 56 | Name of the existing image to minimize 57 | 58 | .B --tag | -t 59 | Tag for the resulting image (random if not provided) 60 | 61 | .B --default-cmd | -d 62 | Analyze the default command for the containers in the original image 63 | 64 | .B --apt 65 | Install the dependencies from minicon using apt-get commands (in the container used for the simulation). 66 | 67 | .B --yum 68 | Install the dependencies from minicon using yum commands (in the container used for the simulation). 69 | 70 | .B --execution | -E 71 | Commandline to analyze when minimizing the container (i.e. that commandline should be able to be executed in the resulting container so the files, libraries, etc. needed should be included). The difference with -R parameter is that in this case, the Entrypoint is not prepended to the commandline (docker exec vs docker run). 72 | 73 | .B --run | -R 74 | Command to analyze when minimizing the container (i.e. that command should be able to be run in the resulting container so the files, libraries, etc. needed should be included). The difference with -E parameter is that in this case, the Entrypoint is prepended to the commandline (docker exec vs docker run). 75 | 76 | .B -2 77 | If needed, you can merge the resulting minimized image with other. This is very specific for the "mergecon" tool. It is useful for (e.g.) adding a minimal Alpine distro (with ash and so on) to the minimized filesystem. 78 | 79 | .B --version | -V 80 | Shows the version number and finalizes. 81 | 82 | .B --verbose | -v 83 | Shows more information about the procedure. 84 | 85 | .B --debug 86 | Shows a lot more information about the procedure. 87 | 88 | .B --help | -h 89 | Shows the help and exits. 90 | 91 | .SH PLUGINS 92 | 93 | Refer to the documentation of minicon(1) to see how to configure the different plugins from minicon. It is of special interest the 94 | .B strace 95 | plugin, to increase the time that an execution is being simulated. 96 | 97 | .SH EXAMPLES 98 | 99 | Starting from a Docker image that has the commands available, fetting a minimal filesyste that includes bash, mkdir, ls, ping, ssh, etc. commands 100 | 101 | .RS 3 102 | .B $ minidock -i minicon:uifat -t minicon:ui --apt -E bash -E 'ssh localhost' \ 103 | -E ip -E id -E cat -E ls -E mkdir \ 104 | -E 'ping -c 1 www.google.es' -- wget -q -O- www.google.es 105 | .RE 106 | 107 | Starting from the Docker image mynodeapp that runs a NodeJS application, which is in /usr/app/myapp 108 | 109 | .RS 3 110 | .B $ minidock --apt -i mynodeapp -t mynodeapp:skinny -I /usr/app/myapp 111 | .RE 112 | 113 | 114 | .SH SEE ALSO 115 | minicon(1) 116 | 117 | .SH AUTHOR 118 | Carlos de Alfonso (caralla@upv.es) -------------------------------------------------------------------------------- /src/importcon.bashc: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # importcon - Imports a filesystem to a container (copying the features from other container) 4 | # https://github.com/grycap/minicon 5 | # 6 | # Copyright (C) GRyCAP - I3M - UPV 7 | # Developed by Carlos A. caralla@upv.es 8 | # 9 | # Licensed under the Apache License, Version 2.0 (the "License"); 10 | # you may not use this file except in compliance with the License. 11 | # You may obtain a copy of the License at 12 | # 13 | # http://www.apache.org/licenses/LICENSE-2.0 14 | # 15 | # Unless required by applicable law or agreed to in writing, software 16 | # distributed under the License is distributed on an "AS IS" BASIS, 17 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 18 | # See the License for the specific language governing permissions and 19 | # limitations under the License. 20 | # 21 | 22 | function usage() { 23 | cat < 30 | 31 | --image | -i Name of the existing image to copy the parameters 32 | --tag | -t Tag for the imported image (random if not provided) 33 | --env | -E Copy ENV settings 34 | --entrypoint | -e Copy ENTRYPOINT settings 35 | --expose | -x Copy EXPOSE settings 36 | --onbuild | -o Copy ONBUILD settings 37 | --user | -u Copy USER settings 38 | --volume | -V Copy VOLUME settings 39 | --workdir | -w Copy WORKDIR settins 40 | --cmd | -c Copy CMD settings 41 | --all | -A Copy all the previous settings: ENV, ENTRYPOINT, EXPOSE, 42 | ONBUILD, USER, VOLUME, WORKDIR and CMD. 43 | --keeptemporary | -k Keeps the temporary folder. Otherwise, the folder is removed (if it is 44 | created by mergecon). 45 | --version | -V Shows the version number and finalizes. 46 | --verbose | -v Shows more information about the procedure. 47 | --debug Shows a lot more information about the procedure. 48 | --help | -h Shows this help and exits. 49 | 50 | EOF 51 | } 52 | 53 | function verify_dependencies() { 54 | if ! docker --version > /dev/null 2> /dev/null; then 55 | bashc.finalize 1 "docker command is needed to import the image" 56 | fi 57 | if ! jq --version > /dev/null 2> /dev/null; then 58 | bashc.finalize 1 "jq command is needed" 59 | fi 60 | } 61 | 62 | source all.bashc 63 | source lib/dockerutils.bashc 64 | source version 65 | 66 | OUTFILE= 67 | # Parse the commandline into an array 68 | bashc.parameter_parse_commandline "$@" 69 | bashc.parameters_start 70 | 71 | while bashc.parameters_next; do 72 | PARAM="$(bashc.parameters_current)" 73 | case "$PARAM" in 74 | --image|-i) bashc.parameters_next 75 | FROMIMAGE="$(bashc.parameters_current)";; 76 | --tag|-t) bashc.parameters_next 77 | NEWNAME="$(bashc.parameters_current)";; 78 | --env|-E) COPY_ENV=true;; 79 | --entrypoint|-e) COPY_ENTRYPOINT=true;; 80 | --expose|-x) COPY_EXPOSE=true;; 81 | --onbuild|-o) COPY_ONBUILD=true;; 82 | --user|-u) COPY_USER=true;; 83 | --volume|-V) COPY_VOLUME=true;; 84 | --workdir|-w) COPY_WORKDIR=true;; 85 | --cmd|-c) COPY_CMD=true;; 86 | --all|-A) COPY_ENV=true 87 | COPY_ENTRYPOINT=true 88 | COPY_EXPOSE=true 89 | COPY_ONBUILD=true 90 | COPY_USER=true 91 | COPY_VOLUME=true 92 | COPY_WORKDIR=true 93 | COPY_CMD=true;; 94 | --simulate|-s) SIMULATEONLY=true;; 95 | --verbose|-v) VERBOSE=true;; 96 | --debug) DEBUG=true;; 97 | --version | -V) echo "$VERSION" && bashc.finalize;; 98 | --help | -h) usage && bashc.finalize;; 99 | --keeptemporary|-k) KEEPTEMPORARY="true";; 100 | *) [ "$FILENAME" != "" ] && usage && bashc.finalize 1 "invalid parameter $PARAM. already provided a filename" 101 | FILENAME="$PARAM";; 102 | esac 103 | done 104 | 105 | verify_dependencies 106 | 107 | if [ "$FILENAME" != "" ]; then 108 | VALIDFILENAME="$(readlink -e "$FILENAME")" 109 | fi 110 | 111 | if [ "$VALIDFILENAME" == "" ]; then 112 | bashc.finalize 1 "filename '$FILENAME' is invalid" 113 | fi 114 | 115 | FILENAME="$VALIDFILENAME" 116 | 117 | if [ "$NEWNAME" == "" ]; then 118 | NEWNAME="$(generate_dockerimagename)" 119 | fi 120 | NEWNAME=$(correct_dockerimagename "$NEWNAME") 121 | 122 | p_out "$NEWNAME" 123 | 124 | function build_docker_cmdline() { 125 | local LINES="$1" 126 | local KW="$2" 127 | 128 | while read L; do 129 | if [ "$L" != "" ]; then 130 | if [ "$L" != "null" ]; then 131 | CMDLINE+=("-c") 132 | CMDLINE+=("$KW $L") 133 | p_debug "adding -c $KW $L" 134 | else 135 | p_debug "ignoring field $KW because it has value '$L' in the original container" 136 | fi 137 | fi 138 | done <<< "$LINES" 139 | } 140 | 141 | parameters="\ 142 | COPY_ENV Env ENV 143 | COPY_USER User USER 144 | COPY_ENTRYPOINT Entrypoint ENTRYPOINT join raw 145 | COPY_ONBUILD OnBuild ONBUILD 146 | COPY_VOLUME Volumes VOLUME nojoin raw |keys[] 147 | COPY_WORKDIR WorkingDir WORKDIR raw 148 | COPY_CMD Cmd CMD join raw 149 | COPY_EXPOSE ExposedPorts EXPOSE nojoin raw |keys[]" 150 | 151 | while read L; do 152 | read VARCOND FIELD KW JOIN RAW XTRA <<< "$L" 153 | if [ "${!VARCOND}" == "true" ]; then 154 | if [ "$RAW" == "raw" ]; then 155 | RES="$(get_config_field_raw "$FROMIMAGE" "$FIELD" "$XTRA")" 156 | else 157 | RES="$(get_config_field "$FROMIMAGE" "$FIELD" "$XTRA")" 158 | fi 159 | if [ "$JOIN" == "join" ]; then 160 | RES="$(echo "$RES" | tr '\n' ' ')" 161 | fi 162 | build_docker_cmdline "$RES" "$KW" 163 | fi 164 | done <<< "${parameters}" 165 | 166 | p_debug docker import "${CMDLINE[@]}" "$FILENAME" "$NEWNAME" 167 | 168 | if [ "$SIMULATEONLY" != "true" ]; then 169 | docker import "${CMDLINE[@]}" "$FILENAME" "$NEWNAME" > /dev/null 170 | else 171 | p_warning "not executed because of commandline options" 172 | fi -------------------------------------------------------------------------------- /doc/importcon.md: -------------------------------------------------------------------------------- 1 | # importcon - IMPORT CONtainer copying features 2 | 3 | When a container is running, it is possible to export its filesystem to a tar file, using the command ```docker export ```. Later, it is possible to import that filesystem into Docker to be used as a Docker image, using a command like ```docker import ```. The problem is that the new container has lost all the parameters from the original image (i.e. ENTRYPOINT, USER, CMD, etc.). 4 | 5 | **importcon** is a script that enables to import a filesystem exported using ```docker export``` into Docker, and to copy the parameters from the original image (i.e. ENTRYPOINT, USER, CMD, VOLUME, etc.) 6 | 7 | ## 1. Why importcon? 8 | 9 | If you create a minimal application filesystem (i.e. using **minicon**), you will get a tarfile that contains the minified filesystem. Then you will probably import it into Docker using the command ```docker import``` (as in the examples). The problem is that the new container will not keep the settings such as ENTRYPOINT, CMD, WORKDIR, etc. 10 | 11 | Using **importcon**, you will be able to import the obtainer tarfile into Docker, but it is possible to provide the name of an existing image as a reference, to copy its parameters (ENV, ENTRYPOINT, CMD, WORKDIR, etc.). 12 | 13 | ## 2. Installation 14 | 15 | ### 2.1 From packages 16 | 17 | You can get the proper package (.deb o .rpm) from the [Releases page](https://github.com/grycap/minicon/releases) and install it using the appropriate package manager. 18 | 19 | **Ubuntu/Debian** 20 | 21 | ```bash 22 | $ apt update 23 | $ apt install ./minicon-1.2-1.deb 24 | ``` 25 | 26 | **CentOS/Fedora/RedHat** 27 | 28 | ```bash 29 | $ yum install epel-release 30 | $ yum install ./minicon-1.2-1.noarch.rpm 31 | ``` 32 | 33 | ### 2.2. From sources 34 | 35 | **importcon** is a bash script that deals with **docker** commands. **importcon** is part of the **minicon** package, and so you just simply need to have a working linux with bash installed and get the code: 36 | 37 | ```bash 38 | $ git clone https://github.com/grycap/minicon 39 | ``` 40 | 41 | In that folder you'll have the **importcon** application. I would suggest to put it in the _/opt_ folder. Otherwise leave it in a folder of your choice: 42 | 43 | ```bash 44 | $ mv minicon /opt 45 | ``` 46 | 47 | #### 2.2.1 Dependencies 48 | 49 | **importcon** depends on the commands _jq_. So, you need to install the proper packages in your system. 50 | 51 | **Ubuntu** 52 | 53 | ```bash 54 | $ apt-get install jq 55 | ``` 56 | 57 | **CentOS** 58 | ```bash 59 | $ yum install jq 60 | ``` 61 | ## 3. Usage 62 | 63 | **importcon** has a lot of options. You are advised to run ```./importcon --help``` to get the latest information about the usage of the application. 64 | 65 | The basic syntax is 66 | 67 | ```bash 68 | $ ./importcon 69 | ``` 70 | 71 | - **--image | -i **: Name of the existing image to copy the parameters. 72 | - **--tag | -t **: Tag for the image that will be created from the tarfile (random if not provided) 73 | - **--env | -E**: Copy ENV settings 74 | - **--entrypoint | -e**: Copy ENTRYPOINT settings 75 | - **--expose | -x**: Copy EXPOSE settings 76 | - **--onbuild | -o**: Copy ONBUILD settings 77 | - **--user | -u**: Copy USER settings 78 | - **--volume | -V**: Copy VOLUME settings 79 | - **--workdir | -w**: Copy WORKDIR settins 80 | - **--cmd | -c**: Copy CMD settings 81 | - **--all | -A**: Copy all the previous settings: ENV, ENTRYPOINT, EXPOSE, ONBUILD, USER, VOLUME, WORKDIR and CMD. 82 | - **--help | -h**: Shows this help and exits. 83 | 84 | ## 4. Example 85 | 86 | If we take the next Dockerfile 87 | 88 | ```docker 89 | FROM ubuntu 90 | RUN apt-get update && apt-get install -y --force-yes apache2 91 | EXPOSE 80 443 92 | VOLUME ["/var/www", "/var/log/apache2", "/etc/apache2"] 93 | ENTRYPOINT ["/usr/sbin/apache2ctl", "-D", "FOREGROUND"] 94 | ``` 95 | 96 | and we build it using the command 97 | 98 | ```bash 99 | docker build . -t tests:apache 100 | Sending build context to Docker daemon 37.89kB 101 | Step 1/5 : FROM ubuntu 102 | ---> 20c44cd7596f 103 | Step 2/5 : RUN apt-get update && apt-get install -y --force-yes apache2 104 | ... 105 | Successfully built ff6f2573d73b 106 | Successfully tagged tests:apache 107 | ``` 108 | 109 | Then we can run a container 110 | 111 | ```bash 112 | $ docker run --rm -id -p 10000:80 tests:apache 113 | 54cd115ab56afc0446b796336ffbfe4415a27f27c6379d0f3d526d79b7e0396b 114 | 115 | $ docker ps 116 | CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 117 | 54cd115ab56a tests:apache "/usr/sbin/apache2..." 4 seconds ago Up 2 seconds 443/tcp, 0.0.0.0:10000->80/tcp hungry_payne 118 | ``` 119 | 120 | And export its filesystem to a file: 121 | 122 | ```bash 123 | $ docker export hungry_payne -o myapache.tar 124 | ``` 125 | 126 | If we import the file back to a Docker image 127 | ```bash 128 | $ docker import myapache.tar tests:myapache 129 | sha256:397cc8e14785e4b5819348f99850cca6b641602bac319457d539ec51ec468a9e 130 | ``` 131 | 132 | Now we can check the differences between the configuration of each image for the new containers: 133 | 134 | ```bash 135 | $ docker inspect tests:apache | jq '.[0].Config' 136 | { 137 | "Hostname": "85a5c35750d6", 138 | "Domainname": "", 139 | "User": "", 140 | "AttachStdin": false, 141 | "AttachStdout": false, 142 | "AttachStderr": false, 143 | "ExposedPorts": { 144 | "443/tcp": {}, 145 | "80/tcp": {} 146 | }, 147 | "Tty": false, 148 | "OpenStdin": false, 149 | "StdinOnce": false, 150 | "Env": [ 151 | "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" 152 | ], 153 | "Cmd": null, 154 | "ArgsEscaped": true, 155 | "Image": "sha256:9bd5b7f04adcd3b7c8fce216e7c3b190abb8941e62acbba72a9172b0b50adf00", 156 | "Volumes": { 157 | "/etc/apache2": {}, 158 | "/var/log/apache2": {}, 159 | "/var/www": {} 160 | }, 161 | "WorkingDir": "", 162 | "Entrypoint": [ 163 | "/usr/sbin/apache2ctl", 164 | "-D", 165 | "FOREGROUND" 166 | ], 167 | "OnBuild": [], 168 | "Labels": {} 169 | } 170 | 171 | $ docker inspect tests:myapache | jq '.[0].Config' 172 | { 173 | "Hostname": "", 174 | "Domainname": "", 175 | "User": "", 176 | "AttachStdin": false, 177 | "AttachStdout": false, 178 | "AttachStderr": false, 179 | "Tty": false, 180 | "OpenStdin": false, 181 | "StdinOnce": false, 182 | "Env": null, 183 | "Cmd": null, 184 | "Image": "", 185 | "Volumes": null, 186 | "WorkingDir": "", 187 | "Entrypoint": null, 188 | "OnBuild": null, 189 | "Labels": null 190 | } 191 | ``` 192 | 193 | We can see that our new image has all the setting set to empty. So if we run the container from our new image, the command will fail (because of the lack of the ENTRYPOINT setting): 194 | 195 | ```bash 196 | $ docker run --rm -id -p 10001:80 tests:myapache 197 | docker: Error response from daemon: No command specified. 198 | See 'docker run --help'. 199 | ``` 200 | 201 | If we import the image using **importcon** and we inspect the configuration: 202 | 203 | ```bash 204 | $ ./importcon -t tests:apacheimportcon -i tests:apache myapache.tar -A 205 | tests:apacheimportcon 206 | sha256:2319eb385a2be7be1e7f0409ec923643b3273427963ecc0e33f2d067f969a66a 207 | 208 | $ docker inspect tests:apacheimportcon | jq '.[0].Config' 209 | { 210 | "Hostname": "", 211 | "Domainname": "", 212 | "User": "", 213 | "AttachStdin": false, 214 | "AttachStdout": false, 215 | "AttachStderr": false, 216 | "ExposedPorts": { 217 | "443/tcp": {}, 218 | "80/tcp": {} 219 | }, 220 | "Tty": false, 221 | "OpenStdin": false, 222 | "StdinOnce": false, 223 | "Env": [ 224 | "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin" 225 | ], 226 | "Cmd": null, 227 | "Image": "", 228 | "Volumes": { 229 | "/etc/apache2": {}, 230 | "/var/log/apache2": {}, 231 | "/var/www": {} 232 | }, 233 | "WorkingDir": "", 234 | "Entrypoint": [ 235 | "/usr/sbin/apache2ctl", 236 | "-D", 237 | "FOREGROUND" 238 | ], 239 | "OnBuild": null, 240 | "Labels": null 241 | } 242 | ``` 243 | 244 | We see that the settings (in special, the ENTRYPOINT) have been copied from the original _tests:apache_ image. And now we are able to run the container as we did in the original one: 245 | 246 | ```bash 247 | $ docker run --rm -id -p 10001:80 tests:apacheimportcon 248 | 6081218d246e0106915a876374a276378912335d3a843f09305939bdd4cd4832 249 | $ docker ps 250 | CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 251 | 6081218d246e tests:apacheimportcon "/usr/sbin/apache2..." 7 seconds ago Up 6 seconds 443/tcp, 0.0.0.0:10001->80/tcp naughty_brattain 252 | ``` 253 | -------------------------------------------------------------------------------- /doc/man/minicon.1: -------------------------------------------------------------------------------- 1 | .\" Manpage for minicon. 2 | .\" Contact caralla@upv.es to correct errors or typos. 3 | .TH man 1 "01 Mar 2018" "1.2-1" "minicon man page" 4 | .SH NAME 5 | minicon - MINImization of COntainers 6 | .SH SYNOPSIS 7 | minicon --
8 | 9 | minicon is a general tool to analyze applications and executions of these applications to obtain a filesystem that only contains the dependencies that have been detected. It aims at reducing the footprint of the filesystem for the container, just adding those files that are needed. That means that the other files in the original container are removed. In particular, it can be used to reduce Docker containers (see minidock). 10 | 11 | .SH DESCRIPTION 12 | When you run containers (e.g. in Docker), you usually run a system that has a whole Operating System, documentation, extra packages, and your specific application. The result is that the footprint of the container is bigger than needed. 13 | 14 | .B minicon 15 | aims at reducing the footprint of the filesystem for the container, just adding those files that are needed. 16 | That means that the other files in the original container are removed. 17 | 18 | .B minicon 19 | is a general tool to analyze applications and executions of these applications to obtain a filesystem 20 | that contains all the dependencies that have been detected. In particular, it can be used to reduce Docker 21 | containers. The 22 | .B minicon 23 | package includes 24 | .B minidock 25 | which will help to reduce Docker containers by hiding the underlying complexity of running 26 | .B minicon 27 | inside a Docker container. 28 | 29 | .SH WHY MINICON? 30 | 31 | Reducing the footprint of one container is of special interest, to redistribute the container images. 32 | 33 | It is of special interest in cases such as [SCAR](https://github.com/grycap/scar), that executes containers out of Docker images in AWS Lambda. In that case, the use cases are limited by the size of the container (the ephemeral storage space is limited to 512 Mb., and SCAR needs to pull the image from Docker Hub into the ephemeral storage and then uncompress it; so the maximum size for the container is even more restricted). 34 | 35 | But there are also security reasons to minimize the unneeded application or environment available in one container image. In the case that the application fails, not having other applications reduces the impact of an intrusion (e.g. if the container does not need a compiler, why should it be there? maybe it would enable to compile a rootkit). 36 | 37 | In this sense, the publication of the NIST "Application Container Security Guide" (https://doi.org/10.6028/NIST.SP.800-190) suggests that 38 | .I """An image should only include the executables and libraries required by the app itself; all other OS functionality is provided by the OS kernel within the underlying host OS""". 39 | 40 | .SH OPTIONS 41 | .B --rootfs | -r 42 | Create the filesystem in a specific folder. 43 | 44 | .B --tar-file | -t 45 | Generate a tar file that contains the resulting filesystem. This is ideal to import it into docker using the command "docker import". If not specified the --rootfs parameter, minicon will use a temporary folder. 46 | 47 | .B --exclude | -e 48 | Exclude all paths that begin with 'F' (it accepts texts for regular expressions). The files are excluded from the final filesystem, but if an execution depends on an app in any of these paths, it will be executed anyway (e.g. -E '/tmp/myapp' -e '/tmp' will analyze myapp, but will not appear in the final filesystem). 49 | 50 | .B --no-exclude-common | -C 51 | The default behavior of minicon is to exclude /tmp, /proc and /dev. If you do not want to exclude them, you should include this flag. The only excluded folders will be those included in the commandline. 52 | 53 | .B --include | -I 54 | Force to consider the file (or folder) pointed by 'F'. Take into account that --include has priority over --exclude. 55 | 56 | .B --execution | -E 57 | Executions to analyze, appart from the main execution (you can include a whole execution with parameters between quotes). It can appear as many time as executions are needed. This is specially useful for the strace plugin. In other case, the effect will be the same of including the command from the execution in the executables to analyze. The executions will be made in order of appearance, AFTER the main execution. 58 | 59 | .B --no-ldconfig | -L 60 | Do not generate the /etc/ldconfig.so file, adjusted to the new filesystem. 61 | 62 | .B --plugin 63 | Activates some plugins and sets the options for them. The syntax is --plugin=:=:=... (see section PLUGINS). 64 | 65 | .B --plugin-all 66 | Activates all the available plugins, using their default options. 67 | 68 | .B --logfile | -g 69 | Outputs the information in file F instead of stdout and stderr 70 | 71 | .B --quiet | -q 72 | Makes the tasks silently 73 | 74 | .B --version | -V 75 | Shows the version number and finalizes. 76 | 77 | .B --verbose | -v 78 | Shows more information about the procedure. 79 | 80 | .B --debug 81 | Shows a lot more information about the procedure. 82 | 83 | .B --help | -h 84 | Shows this help and exits. 85 | 86 | .SH PLUGINS 87 | 88 | The current version of 89 | .B minicon 90 | includes the next plugins: link, which, folder, ldd, scripts and strace. 91 | 92 | .SS link, which and folder 93 | These plugins are activated by default and deal with links, getting the proper executable application from the path and copying whole folders. 94 | 95 | .SS ldd 96 | This plugin makes use of the command 97 | .I ldd 98 | to check the dependencies of an application or library. The resulting dependencies will be added to the resulting filesystem. 99 | 100 | .SS scripts 101 | This plugins tries to guess if a command is an interpreted script. If it is guessed to be, the interpreter will be also analyzed. It makes use of the command 102 | .I file 103 | and the analysis of the shebang line of text files. It accepts the next optional parameter: 104 | 105 | .B includefolders 106 | If it is set to true, the scripts plugin will include in the final filesystem the whole folders in which the interpreter will search for packages (i.e. using @inc or include). The default value is 107 | .B false. 108 | .B 3. 109 | 110 | .SS strace 111 | This plugin analyzes the execution of an application and detects which files have been used. It is tightened to the 112 | .B -E 113 | parameter from minicon. It accepts several parameters and the syntax is: 114 | 115 | --plugin=strace:param=value:param=value... 116 | 117 | .B seconds 118 | the number of seconds that strace will be analyzing the execution. The default value is 119 | .B 3. 120 | 121 | .B mode 122 | decides which files will be included in the filesystem. The possible values are: skinny (includes only the opened, checked, etc. files and creates the opened, checked, etc. folders), slim (also includes the whole opened or created folders), regular (also includes the whole folder in which the opened files are stored; useful for included libraries) and loose (also includes the whole opened, checked, etc. folder). The default value is 123 | .B skinny. 124 | 125 | .B execfile 126 | points to a file that includes commandline examples of different applications. These commandlines will be used for analyzing the executables. E.g. analyzing a plain 127 | .I ping 128 | command has no sense, because it does nothing. But analyzing 129 | .I ping www.google.es 130 | makes use of libraries, name resolution, etc. The default value is 131 | .B none. 132 | 133 | .B showoutput 134 | If set to 135 | .I true 136 | , strace will output the output of the simulations to stdout and stderr. Otherwise, the simulation is hidden. If it the parameter appears without value, it will be interpreted to be 137 | .I true 138 | (i.e. `--plugin=strace:showoutput` is the same than `--plugin=strace:showoutput=true`). The default value is 139 | .B false. 140 | 141 | .SH EXAMPLES 142 | 143 | Getting a minimal filesyste that includes bash, mkdir, ls, etc. commands 144 | 145 | .RS 3 146 | .B minicon -t ./minimal.tar --plugin=strace -E bash -E 'ssh localhost' -E 'ip addr' -E id -E cat -E ls -E mkdir -E 'ping -c 1 www.google.es' -- wget -q -O- www.google.es 147 | .RE 148 | 149 | Then it is possible to import such filesystem into Docker with a command like 150 | 151 | .RS 3 152 | .B docker import minimal.tar tests:minicon 153 | .RE 154 | 155 | The same run of minicon, but running it inside a Docker ubuntu-based container: 156 | 157 | .RS 3 158 | .B docker run --cap-add SYS_PTRACE --rm -it -v /bin/minicon:/tmp/minicon -v $PWD:/tmp/work ubuntu:latest bash -c 'apt-get install -y strace && /tmp/minicon/minicon -t /tmp/work/minimal.tar --plugin=strace -E bash -E "ssh localhost" -E "ip addr" -E id -E cat -E ls -E mkdir -E "ping -c 1 www.google.es" -- wget -q -O- www.google.es' 159 | .RE 160 | 161 | 162 | .SH SEE ALSO 163 | minidock(1), ldd(1), strace(1), file(1), which(1) 164 | 165 | .SH AUTHOR 166 | Carlos de Alfonso (caralla@upv.es) -------------------------------------------------------------------------------- /src/mergecon.bashc: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # mergecon - Merge container file systems 4 | # https://github.com/grycap/minicon 5 | # 6 | # Copyright (C) GRyCAP - I3M - UPV 7 | # Developed by Carlos A. caralla@upv.es 8 | # 9 | # Licensed under the Apache License, Version 2.0 (the "License"); 10 | # you may not use this file except in compliance with the License. 11 | # You may obtain a copy of the License at 12 | # 13 | # http://www.apache.org/licenses/LICENSE-2.0 14 | # 15 | # Unless required by applicable law or agreed to in writing, software 16 | # distributed under the License is distributed on an "AS IS" BASIS, 17 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 18 | # See the License for the specific language governing permissions and 19 | # limitations under the License. 20 | # 21 | 22 | function usage() { 23 | cat < 29 | --first | -1 name of the first container image (will use docker save to dump it) 30 | --second | -2 name of the second container image (will use docker save to dump it) 31 | the default behaviour gives more priority to the second image. I.e. in 32 | case of overlapping files in both input images, the files in the second 33 | image will be exposed to the final image. 34 | --tag | -t name of the resulting container image. If not provided, the resulting 35 | name will be the concatenation of the names of the two input images: 36 | : (the : in the input image names is removed). 37 | --working | -w working folder. If not provided will create a temporary folder in /tmp 38 | --list | -l lists the layers in the images (useful for filesystem composition) 39 | --file | -f tar file where the resulting image will be created. If not provided, 40 | the image will be loaded into docker. 41 | --keeptemporary | -k Keeps the temporary folder. Otherwise, the folder is removed (if it is 42 | created by mergecon). 43 | --version | -V Shows the version number and finalizes. 44 | --verbose | -v Shows more information about the procedure. 45 | --debug Shows a lot more information about the procedure. 46 | --help | -h Shows this help and exits. 47 | EOF 48 | } 49 | 50 | source all.bashc 51 | source lib/dockerutils.bashc 52 | source version 53 | 54 | function verify_dependencies() { 55 | if ! docker --version > /dev/null 2> /dev/null; then 56 | bashc.finalize 1 "docker command is needed" 57 | fi 58 | if ! jq --version > /dev/null 2> /dev/null; then 59 | bashc.finalize 1 "jq command is needed" 60 | fi 61 | if ! tar --version > /dev/null 2> /dev/null; then 62 | bashc.finalize 1 "tar command is needed" 63 | fi 64 | } 65 | 66 | function read_lines_in_array() { 67 | local MANIFEST="$1" 68 | local VARNAME="$2" 69 | 70 | local R n=0 71 | while read R; do 72 | read ${VARNAME}[$n] <<< "$R" 73 | n=$((n+1)) 74 | done <<<"$(echo "$MANIFEST" | jq -r '.[] | .Layers | .[]')" 75 | } 76 | 77 | function join() { 78 | local CHAR="$1" 79 | shift 80 | 81 | local n=0 82 | while [ $# -gt 0 ]; do 83 | if [ "$n" != "0" ]; then 84 | echo -n "$CHAR" 85 | fi 86 | echo -n "$1" 87 | n=$((n+1)) 88 | shift 89 | done 90 | echo 91 | } 92 | 93 | function extract_image() { 94 | local FOLDER="$1" 95 | local IMAGENAME="$2" 96 | 97 | if [ "$IMAGENAME" != "" ]; then 98 | mkdir -p "$FOLDER" 99 | local T="$(bashc.tempfile '.')" 100 | if ! docker inspect "$IMAGENAME" 2> /dev/null; then 101 | p_debug "retrieving image $IMAGENAME because it is not in the registry" 102 | if ! docker pull "$IMAGENAME" 2> /dev/null; then 103 | bashc.finalize 1 "image $IMAGENAME could not be retrieved" 104 | fi 105 | fi 106 | if ! docker save "$IMAGENAME" -o "$T" 2> /dev/null; then 107 | bashc.finalize 1 "could not retrieve image $IMAGENAME" 108 | fi 109 | tar xf "$T" -C "$FOLDER" 110 | rm -f "$T" 111 | fi 112 | } 113 | 114 | TMPDIR= 115 | OUTFILE= 116 | # Parse the commandline into an array 117 | bashc.parameter_parse_commandline "$@" 118 | bashc.parameters_start 119 | 120 | while bashc.parameters_next; do 121 | PARAM="$(bashc.parameters_current)" 122 | case "$PARAM" in 123 | --file|-f) bashc.parameters_next 124 | OUTFILE="$(bashc.parameters_current)";; 125 | --continue|-c) CONTINUE=true;; 126 | --working|-w) bashc.parameters_next 127 | TMPDIR="$(bashc.parameters_current)";; 128 | --tag|-t) bashc.parameters_next 129 | MERGEDNAME="$(bashc.parameters_current)";; 130 | --first|-1) bashc.parameters_next 131 | FIRSTIMAGE="$(bashc.parameters_current)";; 132 | --second|-2) bashc.parameters_next 133 | SECONDIMAGE="$(bashc.parameters_current)";; 134 | --list|-l) LISTLAYERS=true;; 135 | --verbose|-v) VERBOSE=true;; 136 | --debug) DEBUG=true;; 137 | --version | -V) echo "$VERSION" && bashc.finalize;; 138 | --help | -h) usage && bashc.finalize;; 139 | --keeptemporary|-k) KEEPTEMPORARY="true";; 140 | *) usage && bashc.finalize 1;; 141 | esac 142 | done 143 | 144 | verify_dependencies 145 | 146 | if [ "$OUTFILE" != "" ]; then 147 | OUTFILE="$(readlink -m "$OUTFILE")" 148 | fi 149 | 150 | if [ "$TMPDIR" == "" ]; then 151 | TMPDIR="$(bashc.tempdir)" 152 | else 153 | # To avoid removing the folder in case that is is provided in the commandline 154 | KEEPTEMPORARY=false 155 | fi 156 | 157 | FIRSTIMAGE=$(correct_dockerimagename "$FIRSTIMAGE") 158 | SECONDIMAGE=$(correct_dockerimagename "$SECONDIMAGE") 159 | 160 | if [ "$MERGEDNAME" == "" ]; then 161 | MERGEDNAME="$(generate_dockerimagename)" 162 | fi 163 | 164 | MERGEDNAME=$(correct_dockerimagename "$MERGEDNAME") 165 | p_debug "merging layers of image $SECONDIMAGE in $FIRSTIMAGE, into $MERGEDNAME" 166 | p_out "$MERGEDNAME" 167 | 168 | cd $TMPDIR 2> /dev/null || bashc.finalize 1 "could not change to folder $TMPDIR" 169 | p_info "working folder: $TMPDIR" 170 | 171 | if [ "$CONTINUE" != "true" ]; then 172 | extract_image "1" "$FIRSTIMAGE" 173 | extract_image "2" "$SECONDIMAGE" 174 | fi 175 | 176 | LAYERS_1=() 177 | LAYERS_2=() 178 | MANIFEST_1="$(cat 1/manifest.json 2> /dev/null)" 179 | read_lines_in_array "$MANIFEST_1" "LAYERS_1" 180 | MANIFEST_2="$(cat 2/manifest.json 2> /dev/null)" 181 | read_lines_in_array "$MANIFEST_2" "LAYERS_2" 182 | 183 | if [ "$LISTLAYERS" == "true" ]; then 184 | if [ "$FIRSTIMAGE" != "" ]; then 185 | for ((n=0;n<${#LAYERS_1[@]};n++)); do 186 | p_out $n ${LAYERS_1[$n]} 187 | done 188 | fi 189 | if [ "$SECONDIMAGE" != "" ]; then 190 | for ((n=0;n<${#LAYERS_2[@]};n++)); do 191 | p_out $n ${LAYERS_2[$n]} 192 | done 193 | fi 194 | bashc.finalize 195 | fi 196 | 197 | if [ "$FIRSTIMAGE" == "" ]; then 198 | p_warning "missing starting image" 199 | fi 200 | 201 | if [ "$SECONDIMAGE" == "" ]; then 202 | p_warning "missing image to merge" 203 | fi 204 | 205 | if [ "$MANIFEST_1" == "" -a "$MANIFEST_2" == "" ]; then 206 | bashc.finalize 1 "missing manifests" 207 | fi 208 | 209 | p_debug "layers for image $FIRSTIMAGE: ${LAYERS_1[@]}" 210 | p_debug "layers for image $SECONDIMAGE: ${LAYERS_2[@]}" 211 | 212 | IMGID="$(cat /proc/sys/kernel/random/uuid | sha256sum)" 213 | IMGID="${IMGID%% *}" 214 | 215 | p_info "adding new layers" 216 | for ((n=0;n<${#LAYERS_2[@]};n++)); do 217 | LAYERFOLDER="$(dirname ${LAYERS_2[$n]})" 218 | p_debug "copying folder 2/$LAYERFOLDER to 1/$LAYERFOLDER" 219 | cp -r 2/$LAYERFOLDER 1/ 220 | done 221 | 222 | p_info "updating manifest" 223 | NLAYERS_S="\"$(join "\",\"" "${LAYERS_2[@]}")\"" 224 | echo "$MANIFEST_1" | jq "[ .[] | .Layers += [ $NLAYERS_S ] | .RepoTags = [ \"$MERGEDNAME\" ] ] " > 1/manifest.json 225 | 226 | CONFIGFILE_1="1/$(echo "$MANIFEST_1" | jq -r ".[] | .Config")" 227 | CONFIGCONTENT_1="$(cat $CONFIGFILE_1)" 228 | CONFIGFILE_2="2/$(echo "$MANIFEST_2" | jq -r ".[] | .Config")" 229 | CONFIGCONTENT_2="$(cat $CONFIGFILE_2)" 230 | 231 | DIFFIDS="$(echo "$CONFIGCONTENT_2" | jq ".rootfs.diff_ids")" 232 | echo "$CONFIGCONTENT_1" | jq "del(.history)|del(.container_config)|del(.container)|.config.Image=\"sha256:$IMGID\"|.created=(now|todate)|.rootfs.diff_ids+=$DIFFIDS" > "${CONFIGFILE_1}" 233 | 234 | rm 1/repositories 235 | 236 | if [ "$OUTFILE" != "" ]; then 237 | p_debug "outputting image to file $OUTFILE" 238 | tar cf "$OUTFILE" -C 1 . 239 | else 240 | p_debug "loading new image in docker" 241 | docker load < <(tar c -C 1 .) 242 | fi -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | Apache License 2 | Version 2.0, January 2004 3 | http://www.apache.org/licenses/ 4 | 5 | TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 6 | 7 | 1. Definitions. 8 | 9 | "License" shall mean the terms and conditions for use, reproduction, 10 | and distribution as defined by Sections 1 through 9 of this document. 11 | 12 | "Licensor" shall mean the copyright owner or entity authorized by 13 | the copyright owner that is granting the License. 14 | 15 | "Legal Entity" shall mean the union of the acting entity and all 16 | other entities that control, are controlled by, or are under common 17 | control with that entity. For the purposes of this definition, 18 | "control" means (i) the power, direct or indirect, to cause the 19 | direction or management of such entity, whether by contract or 20 | otherwise, or (ii) ownership of fifty percent (50%) or more of the 21 | outstanding shares, or (iii) beneficial ownership of such entity. 22 | 23 | "You" (or "Your") shall mean an individual or Legal Entity 24 | exercising permissions granted by this License. 25 | 26 | "Source" form shall mean the preferred form for making modifications, 27 | including but not limited to software source code, documentation 28 | source, and configuration files. 29 | 30 | "Object" form shall mean any form resulting from mechanical 31 | transformation or translation of a Source form, including but 32 | not limited to compiled object code, generated documentation, 33 | and conversions to other media types. 34 | 35 | "Work" shall mean the work of authorship, whether in Source or 36 | Object form, made available under the License, as indicated by a 37 | copyright notice that is included in or attached to the work 38 | (an example is provided in the Appendix below). 39 | 40 | "Derivative Works" shall mean any work, whether in Source or Object 41 | form, that is based on (or derived from) the Work and for which the 42 | editorial revisions, annotations, elaborations, or other modifications 43 | represent, as a whole, an original work of authorship. For the purposes 44 | of this License, Derivative Works shall not include works that remain 45 | separable from, or merely link (or bind by name) to the interfaces of, 46 | the Work and Derivative Works thereof. 47 | 48 | "Contribution" shall mean any work of authorship, including 49 | the original version of the Work and any modifications or additions 50 | to that Work or Derivative Works thereof, that is intentionally 51 | submitted to Licensor for inclusion in the Work by the copyright owner 52 | or by an individual or Legal Entity authorized to submit on behalf of 53 | the copyright owner. For the purposes of this definition, "submitted" 54 | means any form of electronic, verbal, or written communication sent 55 | to the Licensor or its representatives, including but not limited to 56 | communication on electronic mailing lists, source code control systems, 57 | and issue tracking systems that are managed by, or on behalf of, the 58 | Licensor for the purpose of discussing and improving the Work, but 59 | excluding communication that is conspicuously marked or otherwise 60 | designated in writing by the copyright owner as "Not a Contribution." 61 | 62 | "Contributor" shall mean Licensor and any individual or Legal Entity 63 | on behalf of whom a Contribution has been received by Licensor and 64 | subsequently incorporated within the Work. 65 | 66 | 2. Grant of Copyright License. Subject to the terms and conditions of 67 | this License, each Contributor hereby grants to You a perpetual, 68 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 69 | copyright license to reproduce, prepare Derivative Works of, 70 | publicly display, publicly perform, sublicense, and distribute the 71 | Work and such Derivative Works in Source or Object form. 72 | 73 | 3. Grant of Patent License. Subject to the terms and conditions of 74 | this License, each Contributor hereby grants to You a perpetual, 75 | worldwide, non-exclusive, no-charge, royalty-free, irrevocable 76 | (except as stated in this section) patent license to make, have made, 77 | use, offer to sell, sell, import, and otherwise transfer the Work, 78 | where such license applies only to those patent claims licensable 79 | by such Contributor that are necessarily infringed by their 80 | Contribution(s) alone or by combination of their Contribution(s) 81 | with the Work to which such Contribution(s) was submitted. If You 82 | institute patent litigation against any entity (including a 83 | cross-claim or counterclaim in a lawsuit) alleging that the Work 84 | or a Contribution incorporated within the Work constitutes direct 85 | or contributory patent infringement, then any patent licenses 86 | granted to You under this License for that Work shall terminate 87 | as of the date such litigation is filed. 88 | 89 | 4. Redistribution. You may reproduce and distribute copies of the 90 | Work or Derivative Works thereof in any medium, with or without 91 | modifications, and in Source or Object form, provided that You 92 | meet the following conditions: 93 | 94 | (a) You must give any other recipients of the Work or 95 | Derivative Works a copy of this License; and 96 | 97 | (b) You must cause any modified files to carry prominent notices 98 | stating that You changed the files; and 99 | 100 | (c) You must retain, in the Source form of any Derivative Works 101 | that You distribute, all copyright, patent, trademark, and 102 | attribution notices from the Source form of the Work, 103 | excluding those notices that do not pertain to any part of 104 | the Derivative Works; and 105 | 106 | (d) If the Work includes a "NOTICE" text file as part of its 107 | distribution, then any Derivative Works that You distribute must 108 | include a readable copy of the attribution notices contained 109 | within such NOTICE file, excluding those notices that do not 110 | pertain to any part of the Derivative Works, in at least one 111 | of the following places: within a NOTICE text file distributed 112 | as part of the Derivative Works; within the Source form or 113 | documentation, if provided along with the Derivative Works; or, 114 | within a display generated by the Derivative Works, if and 115 | wherever such third-party notices normally appear. The contents 116 | of the NOTICE file are for informational purposes only and 117 | do not modify the License. You may add Your own attribution 118 | notices within Derivative Works that You distribute, alongside 119 | or as an addendum to the NOTICE text from the Work, provided 120 | that such additional attribution notices cannot be construed 121 | as modifying the License. 122 | 123 | You may add Your own copyright statement to Your modifications and 124 | may provide additional or different license terms and conditions 125 | for use, reproduction, or distribution of Your modifications, or 126 | for any such Derivative Works as a whole, provided Your use, 127 | reproduction, and distribution of the Work otherwise complies with 128 | the conditions stated in this License. 129 | 130 | 5. Submission of Contributions. Unless You explicitly state otherwise, 131 | any Contribution intentionally submitted for inclusion in the Work 132 | by You to the Licensor shall be under the terms and conditions of 133 | this License, without any additional terms or conditions. 134 | Notwithstanding the above, nothing herein shall supersede or modify 135 | the terms of any separate license agreement you may have executed 136 | with Licensor regarding such Contributions. 137 | 138 | 6. Trademarks. This License does not grant permission to use the trade 139 | names, trademarks, service marks, or product names of the Licensor, 140 | except as required for reasonable and customary use in describing the 141 | origin of the Work and reproducing the content of the NOTICE file. 142 | 143 | 7. Disclaimer of Warranty. Unless required by applicable law or 144 | agreed to in writing, Licensor provides the Work (and each 145 | Contributor provides its Contributions) on an "AS IS" BASIS, 146 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or 147 | implied, including, without limitation, any warranties or conditions 148 | of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A 149 | PARTICULAR PURPOSE. You are solely responsible for determining the 150 | appropriateness of using or redistributing the Work and assume any 151 | risks associated with Your exercise of permissions under this License. 152 | 153 | 8. Limitation of Liability. In no event and under no legal theory, 154 | whether in tort (including negligence), contract, or otherwise, 155 | unless required by applicable law (such as deliberate and grossly 156 | negligent acts) or agreed to in writing, shall any Contributor be 157 | liable to You for damages, including any direct, indirect, special, 158 | incidental, or consequential damages of any character arising as a 159 | result of this License or out of the use or inability to use the 160 | Work (including but not limited to damages for loss of goodwill, 161 | work stoppage, computer failure or malfunction, or any and all 162 | other commercial damages or losses), even if such Contributor 163 | has been advised of the possibility of such damages. 164 | 165 | 9. Accepting Warranty or Additional Liability. While redistributing 166 | the Work or Derivative Works thereof, You may choose to offer, 167 | and charge a fee for, acceptance of support, warranty, indemnity, 168 | or other liability obligations and/or rights consistent with this 169 | License. However, in accepting such obligations, You may act only 170 | on Your own behalf and on Your sole responsibility, not on behalf 171 | of any other Contributor, and only if You agree to indemnify, 172 | defend, and hold each Contributor harmless for any liability 173 | incurred by, or claims asserted against, such Contributor by reason 174 | of your accepting any such warranty or additional liability. 175 | 176 | END OF TERMS AND CONDITIONS 177 | 178 | APPENDIX: How to apply the Apache License to your work. 179 | 180 | To apply the Apache License to your work, attach the following 181 | boilerplate notice, with the fields enclosed by brackets "[]" 182 | replaced with your own identifying information. (Don't include 183 | the brackets!) The text should be enclosed in the appropriate 184 | comment syntax for the file format. We also recommend that a 185 | file or class name and description of purpose be included on the 186 | same "printed page" as the copyright notice for easier 187 | identification within third-party archives. 188 | 189 | Copyright [yyyy] [name of copyright owner] 190 | 191 | Licensed under the Apache License, Version 2.0 (the "License"); 192 | you may not use this file except in compliance with the License. 193 | You may obtain a copy of the License at 194 | 195 | http://www.apache.org/licenses/LICENSE-2.0 196 | 197 | Unless required by applicable law or agreed to in writing, software 198 | distributed under the License is distributed on an "AS IS" BASIS, 199 | WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 200 | See the License for the specific language governing permissions and 201 | limitations under the License. 202 | -------------------------------------------------------------------------------- /src/minicon.bashc: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # minicon - Minimization of filesystems for containers 4 | # https://github.com/grycap/minicon 5 | # 6 | # Copyright (C) GRyCAP - I3M - UPV 7 | # Developed by Carlos A. caralla@upv.es 8 | # 9 | # Licensed under the Apache License, Version 2.0 (the "License"); 10 | # you may not use this file except in compliance with the License. 11 | # You may obtain a copy of the License at 12 | # 13 | # http://www.apache.org/licenses/LICENSE-2.0 14 | # 15 | # Unless required by applicable law or agreed to in writing, software 16 | # distributed under the License is distributed on an "AS IS" BASIS, 17 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 18 | # See the License for the specific language governing permissions and 19 | # limitations under the License. 20 | # 21 | 22 | function usage() { 23 | cat < --
29 | 30 | --rootfs | -r Create the filesystem in a specific folder. 31 | --tar-file | -t Generate a tar file that contains the resulting filesystem. This is 32 | ideal to import it into docker using the command "docker import". If 33 | not specified the --rootfs parameter, minicon will use a temporary 34 | folder. 35 | --exclude | -e Exclude all paths that begin with 'F' (it accepts texts for regular 36 | expressions). The files are excluded from the final filesystem, but if 37 | an execution depends on an app in any of these paths, it will be executed 38 | anyway (e.g. -E '/tmp/myapp' -e '/tmp' will analyze myapp, but will not 39 | appear in the final filesystem). 40 | --no-exclude-common | -C 41 | The default behavior of minicon is to exclude /tmp, /proc and /dev. If you 42 | do not want to exclude them, you should include this flag. The only excluded 43 | folders will be those included in the commandline. 44 | --include | -I Force to consider the file (or folder) pointed by 'F'. Take into account that 45 | --include has priority over --exclude. 46 | --execution | -E 47 | Executions to analyze, appart from the main execution (you can include a 48 | whole execution with parameters between quotes). It can appear as many time 49 | as executions are needed. 50 | This is specially useful for the strace plugin. In other case, the effect 51 | will be the same of including the command from the execution in the executables 52 | to analyze. The executions will be made in order of appearance, AFTER the 53 | main execution. 54 | --no-ldconfig | -L Do not generate the /etc/ldconfig.so file, adjusted to the new filesystem. 55 | --ldconfig | -l (deprecated) generate the /etc/ldconfig.so file, adjusted to the new 56 | filesystem. This is deprecated because now it is the default behavior if 57 | flag -L is not used. 58 | --plugin Activates some plugins and sets the options for them. The syntax is 59 | --plugin=:=:=... 60 | --plugin-all Activates all the available plugins, using their default options. 61 | --logfile | -g Outputs the information in file F instead of stdout and stderr 62 | --quiet | -q Makes the tasks silently 63 | --version | -V Shows the version number and finalizes. 64 | --verbose | -v Shows more information about the procedure. 65 | --debug Shows a lot more information about the procedure. 66 | --help | -h Shows this help and exits. 67 | 68 | EOF 69 | } 70 | 71 | # Checking dependencies 72 | function remove_from_plugins() { 73 | local P="$1" 74 | local PP= 75 | 76 | while [ "$PP" != "$PLUGINS_ACTIVATED" ]; do 77 | PP="$PLUGINS_ACTIVATED" 78 | PLUGINS_ACTIVATED="$(echo "$PLUGINS_ACTIVATED" | sed "s/\(^\|,\)\($P\(\|:[^,]*\)\)\(,\|$\)/\1\4/g")" 79 | done 80 | } 81 | 82 | function verify_dependencies() { 83 | if ! strace -V > /dev/null 2> /dev/null; then 84 | remove_from_plugins "strace" 85 | p_warning "disabling strace plugin because strace command is not available" 86 | fi 87 | if ! file --version > /dev/null 2> /dev/null; then 88 | remove_from_plugins "scripts" 89 | p_warning "disabling scripts plugin because file command is not available" 90 | fi 91 | if [ "$TARFILE" != "" ]; then 92 | if ! tar --version > /dev/null 2> /dev/null; then 93 | bashc.finalize 1 "tar command is needed to create tar files" 94 | fi 95 | fi 96 | if ! hash ldd > /dev/null 2> /dev/null; then 97 | bashc.finalize 1 "ldd command is needed for scripts plugin" 98 | fi 99 | } 100 | 101 | source all.bashc 102 | source lib/plugins.bashc 103 | source version 104 | 105 | function is_protected() { 106 | local SRC="$1" 107 | local PROTECTED="/proc.*,/dev.*,/sys.*,/" 108 | local CURRENT 109 | 110 | while read -d ',' CURRENT; do 111 | CURRENT="${CURRENT}\$" 112 | if [[ "$SRC" =~ ^$CURRENT ]]; then 113 | return 0 114 | fi 115 | done <<<"${PROTECTED}," 116 | return 1 117 | } 118 | 119 | function _build_rsync() { 120 | local CURRENT="$1" 121 | local RES= 122 | while [ "$CURRENT" != "/" -a "$CURRENT" != "." -a "$CURRENT" != ".." ]; do 123 | RES="--include ${CURRENT} ${RES}" 124 | CURRENT="$(dirname "$CURRENT")" 125 | done 126 | local i 127 | for ((i=0;i<${#EXCLUDED_PATHS[@]};i=i+1)); do 128 | RES="${RES} --exclude ${EXCLUDED_PATHS[$i]} --exclude ${EXCLUDED_PATHS[$i]}/\*" 129 | done 130 | echo "$RES" 131 | } 132 | 133 | function cp_rsync() { 134 | local SRC="$1" 135 | local ROOTFS="$2" 136 | local RECURSE="$3" 137 | 138 | if [ "${SRC:0:2}" == "./" ]; then 139 | SRC="${PWD}${SRC:1}" 140 | fi 141 | 142 | local SRCDIR="$(dirname "$SRC")" 143 | DST="$ROOTFS/$SRCDIR" 144 | 145 | p_out "$SRC" 146 | if [ "$RECURSE" == "true" ]; then 147 | p_debug "copying $SRC to $DST (recurse)" 148 | rsync -a $(_build_rsync "$SRCDIR") --include "${SRC}" --include "${SRC}/**" --exclude '*' / "$ROOTFS" 149 | else 150 | p_debug "copying $SRC to $DST" 151 | rsync -a $(_build_rsync "$SRCDIR") --include "${SRC}" --exclude '*' / "$ROOTFS" 152 | fi 153 | } 154 | 155 | function cp_cp() { 156 | local SRC="$1" 157 | local ROOTFS="$2" 158 | local RECURSE="$3" 159 | 160 | if [ "${SRC:0:2}" == "./" ]; then 161 | SRC="${PWD}${SRC:1}" 162 | fi 163 | 164 | local DST= 165 | DST="$ROOTFS/$(dirname "$SRC")" 166 | mkdir -p "$DST" 167 | 168 | p_out "$SRC" 169 | if [ -d "$SRC" ]; then 170 | if [ "$RECURSE" == "true" ]; then 171 | p_debug "copying $SRC to $DST (recurse)" 172 | cp -p -n -r "$SRC" "$DST" 173 | else 174 | p_debug "copying $SRC to $DST (mkdir)" 175 | mkdir -p "${DST}/$(basename "$SRC")" 176 | fi 177 | else 178 | p_debug "copying $SRC to $DST" 179 | cp -p -n "$SRC" "$DST" 180 | fi 181 | } 182 | 183 | _FILES_COPIED="" 184 | 185 | function cp_wrapper() { 186 | if [ "$CP_FUNC" == "" ]; then 187 | rsync --version 2> /dev/null > /dev/null 188 | if [ $? -eq 0 ]; then 189 | CP_FUNC=cp_rsync 190 | else 191 | p_warning "rsync is not available... some file permissions will be lost" 192 | CP_FUNC=cp_cp 193 | fi 194 | fi 195 | 196 | local SRC ROOTFS RECURSE=false 197 | while [ $# -gt 0 ]; do 198 | case "$1" in 199 | -r) RECURSE=true;; 200 | *) if [ "$SRC" == "" ]; then 201 | SRC="$1" 202 | else 203 | if [ "$ROOTFS" == "" ]; then 204 | ROOTFS="$1" 205 | fi 206 | fi;; 207 | esac 208 | shift 209 | done 210 | 211 | local EXISTING 212 | EXISTING="$(echo "$_FILES_COPIED" | grep "^[tf]:${SRC}$")" 213 | local METHOD="${EXISTING:0:1}" 214 | EXISTING="${EXISTING:2}" 215 | if [ "$EXISTING" == "$SRC" -a "${METHOD}" == "${RECURSE:0:1}" ]; then 216 | p_debug "skipping file $SRC because it has been already copied" 217 | return 218 | fi 219 | 220 | $CP_FUNC "$SRC" "$ROOTFS" "$RECURSE" 221 | _FILES_COPIED="${_FILES_COPIED} 222 | ${RECURSE:0:1}:${SRC}" 223 | } 224 | 225 | function copy() { 226 | # copies one file (or folder) to the same destination in the root filesystem 227 | # - it does not overwrite contents 228 | local SRC="$1" 229 | local RECURSE="$2" 230 | if [ "$RECURSE" != "-r" ]; then 231 | RECURSE= 232 | fi 233 | 234 | local n EP 235 | 236 | # now check whether it is a link or not... if it is a link, we'll copy the actual file and will create the link 237 | SRC="$(_linked_file "$SRC")" 238 | 239 | if is_protected "$SRC"; then 240 | p_info "excluding file $SRC because it is in a protected path" 241 | return 0 242 | fi 243 | 244 | local ALREADY_INCLUDED=false 245 | 246 | #for ((n=0;n<${#INCLUDED_PATHS[@]};n++)); do 247 | # EP="${INCLUDED_PATHS[$n]}" 248 | # if [[ "$SRC" =~ ^$EP ]]; then 249 | # ALREADY_INCLUDED=true 250 | # break 251 | # fi 252 | #done 253 | 254 | if [ "$ALREADY_INCLUDED" == "false" ]; then 255 | for ((n=0;n<${#EXCLUDED_PATHS[@]};n++)); do 256 | EP="${EXCLUDED_PATHS[$n]}" 257 | if [[ "$SRC" =~ ^$EP ]]; then 258 | p_info "excluding ${SRC} because it matches pattern ${EP}" 259 | return 0 260 | fi 261 | done 262 | fi 263 | 264 | if [ "$SRC" == "." -o "$SRC" == ".." ]; then 265 | p_warning "cowardly refusing to copy folder $SRC" 266 | return 267 | fi 268 | 269 | local DST 270 | if [ "$VERBOSE" == "true" ]; then 271 | p_info " processing $SRC... " 272 | fi 273 | if [ ! -e "$SRC" ]; then 274 | p_error 1 "failed to read $SRC" 275 | fi 276 | 277 | cp_wrapper $RECURSE "$SRC" "$ROOTFS" 278 | } 279 | 280 | function is_plugin_active() { 281 | # Checks whether a plugin is activated or not 282 | if [[ "$PLUGINS_ACTIVATED" =~ (^|,)$1(|:[^,]+)(,|$) ]]; then 283 | return 0 284 | fi 285 | return 1 286 | } 287 | 288 | function add_command() { 289 | local CMDTOADD="$1" 290 | local n=0 291 | local ALREADY_EXISTS=false 292 | while [ $n -lt ${#COMMANDS_TO_ADD[@]} ]; do 293 | if [ "$CMDTOADD" == "${COMMANDS_TO_ADD[$n]}" ]; then 294 | ALREADY_EXISTS=true 295 | break 296 | fi 297 | n=$((n+1)) 298 | done 299 | if [ "$ALREADY_EXISTS" == "false" ]; then 300 | p_debug "need to analyze $CMDTOADD" 301 | COMMANDS_TO_ADD+=( "$CMDTOADD" ) 302 | fi 303 | } 304 | 305 | # Now we are activating the basic plugins 306 | EXCLUDED_PATHS=() 307 | INCLUDED_PATHS=() 308 | PLUGINS_ACTIVATED=link,which,folder,ldd,scripts 309 | COMMANDS_TO_ADD=() 310 | EXECUTIONS=() 311 | FORCEFOLDER=false 312 | ROOTFS= 313 | LDCONFIGFILE=/etc/ld.so.conf 314 | MAINEXECUTION=() 315 | 316 | # Parse the commandline into an array 317 | bashc.parameter_parse_commandline "$@" 318 | bashc.parameters_start 319 | 320 | while bashc.parameters_next; do 321 | PARAM="$(bashc.parameters_current)" 322 | case "$PARAM" in 323 | --plugin-all) PLUGINS_ACTIVATED="$(plugin_list),${PLUGINS_ACTIVATED}";; 324 | --plugin=*) PLUGINS_ACTIVATED="${PLUGINS_ACTIVATED},${PARAM:9}";; 325 | --exclude|-e) bashc.parameters_next 326 | PARAM="$(bashc.parameters_current)" 327 | EXCLUDED_PATHS+=("$PARAM");; 328 | --no-exclude-common|-C) NOEXCLUDECOMMON=true;; 329 | --include|-I) bashc.parameters_next 330 | PARAM="$(bashc.parameters_current)" 331 | INCLUDED_PATHS+=("$PARAM");; 332 | --logfile|-g) bashc.parameters_next 333 | LOGFILE="$(bashc.parameters_current)";; 334 | --quiet|-q) QUIET=true;; 335 | --tarfile|-t) bashc.parameters_next 336 | TARFILE="$(bashc.parameters_current)";; 337 | --ldconfig|-l) LDCONFIGFILE=/etc/ld.so.conf;; 338 | --no-ldconfig|-L) LDCONFIGFILE=;; 339 | --verbose|-v) VERBOSE=true;; 340 | --debug) DEBUG=true;; 341 | --force|-f) FORCEFOLDER=true;; 342 | --rootfs|-r) bashc.parameters_next 343 | ROOTFS="$(bashc.parameters_current)" 344 | if [ ! -d "$(dirname $ROOTFS)" ]; then 345 | bashc.finalize 1 "invalid folder (parent folder must exist)" 346 | fi;; 347 | --execution|-E) bashc.parameters_next 348 | PARAM="$(bashc.parameters_current)" 349 | EXECUTIONS+=("$PARAM");; 350 | --version | -V) echo "$VERSION" && bashc.finalize;; 351 | --help | -h) usage && bashc.finalize;; 352 | --) while bashc.parameters_next; do 353 | PARAM="$(bashc.parameters_current)" 354 | MAINEXECUTION+=("$PARAM") 355 | done;; 356 | *) usage && bashc.finalize 1 "unexpected parameter: $PARAM";; 357 | esac 358 | done 359 | 360 | [ "$QUIET" == "true" ] && DEBUG= VERBOSE= 361 | 362 | # Exclude the common non-wanted folder (if not prevented) 363 | if [ "$NOEXCLUDECOMMON" != "true" ]; then 364 | EXCLUDED_PATHS+=("/sys") 365 | EXCLUDED_PATHS+=("/tmp") 366 | EXCLUDED_PATHS+=("/dev") 367 | EXCLUDED_PATHS+=("/proc") 368 | fi 369 | 370 | # Add the main execution in first place 371 | CURRENT_CMD=() 372 | if [ ${#MAINEXECUTION} -gt 0 ]; then 373 | EXECUTIONS=("$(bashc.build_cmdline "${MAINEXECUTION[@]}")" "${EXECUTIONS[@]}") 374 | fi 375 | 376 | # Now manage the executions 377 | if [ ${#EXECUTIONS[@]} -gt 0 ]; then 378 | for ((n=0;n<${#EXECUTIONS[@]};n=n+1)); do 379 | bashc.arrayze_cmd CURRENT_CMD "${EXECUTIONS[$n]}" 380 | 381 | # Remove the possible options to the strace command 382 | _CURRENT_CMD="${CURRENT_CMD[0]}" 383 | _CURRENT_CMD_N_PATH="${_CURRENT_CMD##*,}" 384 | 385 | p_debug "adding command ${_CURRENT_CMD_N_PATH} to analyze" 386 | add_command "$_CURRENT_CMD_N_PATH" 387 | done 388 | fi 389 | 390 | if [ "$TARFILE" != "" -a "$ROOTFS" == "" ]; then 391 | ROOTFS=$(bashc.tempdir) 392 | FORCEFOLDER=true 393 | fi 394 | 395 | [ "$ROOTFS" == "" ] && bashc.finalize 1 "you must provide a folder (--rootfs) to create the root filesystem or request a tar file (--tarfile)" 396 | 397 | ROOTFS="$(readlink -f "$ROOTFS")" 398 | [ "$ROOTFS" == "" ] && bashc.finalize 1 "invalid folder to create the root filesystem (the parent folder MUST exist)" 399 | 400 | DANGER_FOLDERS="/ /etc /var /sys /proc" 401 | for F in $DANGER_FOLDERS; do 402 | [ "$ROOTFS" == "$F" ] && bashc.finalize "refusing to build root filesystem in folder $ROOTFS" 403 | done 404 | 405 | [ -e "$ROOTFS" -a ! -d "$ROOTFS" ] && bashc.finalize 1 "$ROOTFS exists but it is not a folder" 406 | 407 | if [ "$FORCEFOLDER" != "true" -a -d "$ROOTFS" ]; then 408 | read -p "folder for root filesystem ($ROOTFS) already exists. Are you sure to use it? (y/N) " CONFIRM 409 | if [ "${CONFIRM^^}" != "Y" ]; then 410 | bashc.finalize 1 "aborting" 411 | fi 412 | fi 413 | 414 | # Check dependencies first 415 | verify_dependencies 416 | 417 | p_debug "creating $ROOTFS" 418 | mkdir -p "$ROOTFS" 419 | mkdir -p "$ROOTFS/tmp" 420 | 421 | p_info "copying forced paths" 422 | 423 | for ((n=0;n<${#INCLUDED_PATHS[@]};n=n+1)); do 424 | copy "${INCLUDED_PATHS[$n]}" -r 425 | done 426 | 427 | if is_plugin_active "strace"; then 428 | for ((n=0;n<${#EXECUTIONS[@]};n=n+1)); do 429 | STRACE_command "${EXECUTIONS[$n]}" 430 | done 431 | fi 432 | 433 | p_info "analyzing..." 434 | 435 | i_current=0 436 | while [ $i_current -lt ${#COMMANDS_TO_ADD[@]} ]; do 437 | CURRENT_CMD="${COMMANDS_TO_ADD[$i_current]}" 438 | for p in $(PLUGIN_funcs); do 439 | if is_plugin_active "${p##*_}"; then 440 | p_debug "invoking plugin ${p##*_}" 441 | _C_PLUGIN="${p##*_}" 442 | bashc.set_logger "${_C_PLUGIN^^}" 443 | if ! $p "$CURRENT_CMD"; then 444 | break 445 | fi 446 | fi 447 | done 448 | bashc.set_logger 449 | i_current=$(($i_current+1)) 450 | done 451 | 452 | if [ "$LDCONFIGFILE" != "" -a -e "$ROOTFS/$LDCONFIGFILE" ]; then 453 | p_debug "creating ldconfig" 454 | TMPFILE=$(bashc.tempfile) 455 | awk '!a[$0]++' "$ROOTFS/$LDCONFIGFILE" > "$TMPFILE" 456 | mv "$TMPFILE" "$ROOTFS/$LDCONFIGFILE" 457 | ldconfig -r "$ROOTFS" 458 | p_out "ldconfig recreated" 459 | fi 460 | 461 | if [ "$TARFILE" != "" ]; then 462 | if [ "$TARFILE" != "-" ]; then 463 | p_info "producing tar file $TARFILE" 464 | tar cf "$TARFILE" -C "$ROOTFS" . 465 | else 466 | tar c -C "$ROOTFS" . 467 | fi 468 | fi 469 | -------------------------------------------------------------------------------- /src/minidock.bashc: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | # 3 | # importcon - Imports a filesystem to a container (copying the features from other container) 4 | # https://github.com/grycap/minicon 5 | # 6 | # Copyright (C) GRyCAP - I3M - UPV 7 | # Developed by Carlos A. caralla@upv.es 8 | # 9 | # Licensed under the Apache License, Version 2.0 (the "License"); 10 | # you may not use this file except in compliance with the License. 11 | # You may obtain a copy of the License at 12 | # 13 | # http://www.apache.org/licenses/LICENSE-2.0 14 | # 15 | # Unless required by applicable law or agreed to in writing, software 16 | # distributed under the License is distributed on an "AS IS" BASIS, 17 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. 18 | # See the License for the specific language governing permissions and 19 | # limitations under the License. 20 | # 21 | 22 | BASEFOLDER=$(dirname $0) 23 | MINICON="$(which minicon)" 24 | if [ "$MINICON" == "" ]; then 25 | MINICON="$BASEFOLDER/minicon" 26 | fi 27 | 28 | IMPORTCON="$(which importcon)" 29 | if [ "$IMPORTCON" == "" ]; then 30 | IMPORTCON="$BASEFOLDER/importcon" 31 | fi 32 | 33 | MERGECON="$(which mergecon)" 34 | if [ "$MERGECON" == "" ]; then 35 | MERGECON="$BASEFOLDER/mergecon" 36 | fi 37 | 38 | function usage() { 39 | cat < [ --docker-opts ] -- 45 | 46 | Is the whole commandline to be analised in the run. These are the same parameters 47 | that you would pass to "docker run ... ". 48 | If no run is provided, minidock will use the default command. And if the image 49 | as no default command, it will try to use one of the -R or -E commands. 50 | * the aim is that you run "minidock" as if you used a "docker run" for your container. 51 | If you need them, you can include some options that will be raw-passed to the 52 | docker run command used during the analysis. (i.e. minidock will execute 53 | docker run ...). 54 | If you need to, you can add some minicon-specific options. The supported options 55 | are --include --exclude --plugin 56 | --image | -i Name of the existing image to minimize 57 | --tag | -t Tag for the resulting image (random if not provided) 58 | --default-cmd | -d Analyze the default command for the containers in the original image 59 | --apt Install the dependencies from minicon using apt-get commands (in the container 60 | used for the simulation). 61 | --yum Install the dependencies from minicon using yum commands (in the container used 62 | for the simulation). 63 | --execution | -E 64 | Commandline to analyze when minimizing the container (i.e. that commandline should 65 | be able to be executed in the resulting container so the files, libraries, etc. needed 66 | should be included). The difference with -R parameter is that in this case, the 67 | Entrypoint is not prepended to the commandline (docker exec vs docker run). 68 | --run | -R 69 | Command to analyze when minimizing the container (i.e. that command should be able 70 | to be run in the resulting container so the files, libraries, etc. needed 71 | should be included). The difference with -E parameter is that in this case, the 72 | Entrypoint is prepended to the commandline (docker exec vs docker run). 73 | -2 If needed, you can merge the resulting minimized image with other. This is very specific 74 | for the "mergecon" tool. It is useful for (e.g.) adding a minimal Alpine distro (with 75 | ash and so on) to the minimized filesystem. 76 | --version | -V Shows the version number and finalizes. 77 | --verbose | -v Shows more information about the procedure. 78 | --debug Shows a lot more information about the procedure. 79 | --help | -h Shows this help and exits. 80 | 81 | EOF 82 | } 83 | 84 | function verify_dependencies() { 85 | if [ ! -x "$MINICON" ]; then 86 | bashc.finalize 1 "cannot find minicon" 87 | fi 88 | if [ ! -x "$IMPORTCON" ]; then 89 | bashc.finalize 1 "cannot find importcon" 90 | fi 91 | if [ ! -x "$MERGECON" ]; then 92 | bashc.finalize 1 "cannot find mergecon" 93 | fi 94 | if ! docker --version > /dev/null 2> /dev/null; then 95 | bashc.finalize 1 "docker command is needed to import the image" 96 | fi 97 | if ! jq --version > /dev/null 2> /dev/null; then 98 | bashc.finalize 1 "jq command is needed" 99 | fi 100 | } 101 | 102 | source all.bashc 103 | source lib/dockerutils.bashc 104 | source version 105 | 106 | TMPDIR= 107 | IMPORTCONOPTS=() 108 | MERGECONOPTS=() 109 | 110 | # The strace plugin needs the ability of process tracking (or privileged, but we do not want too much permissions) 111 | DOCKEROPTS=("--cap-add" "SYS_PTRACE" "-t") 112 | 113 | # minidock runs minicon with all the plugins activated 114 | MINICONOPTS=("--plugin-all") 115 | 116 | # The main execution: it enables running minicon as if it was docker run ... 117 | MAINEXEC=() 118 | 119 | # Other executions. They need to be captured to add them the entrypoint 120 | SECONDARYEXECS=() 121 | 122 | # Run the default command as a secondary execution 123 | EXECUTEDEFAULTCMD=false 124 | 125 | # Parse the commandline into an array 126 | bashc.parameter_parse_commandline "$@" 127 | bashc.parameters_start 128 | 129 | while bashc.parameters_next; do 130 | PARAM="$(bashc.parameters_current)" 131 | case "$PARAM" in 132 | --default-cmd|-d) EXECUTEDEFAULTCMD=true;; 133 | --apt) DEPENDENCIES_APT=true;; 134 | --yum) DEPENDENCIES_YUM=true;; 135 | --plugin=*) MINICONOPTS+=("$PARAM");; 136 | --exclude|-e|--include|-I) 137 | MINICONOPTS+=("$PARAM") 138 | bashc.parameters_next 139 | PARAM="$(bashc.parameters_current)" 140 | MINICONOPTS+=("$PARAM");; 141 | --mode=*) MINICONOPTS+=("--plugin=strace:mode=${PARAM:7}") 142 | ;; 143 | --mode) bashc.parameters_next 144 | PARAM="$(bashc.parameters_current)" 145 | MINICONOPTS+=("--plugin=strace:mode=$PARAM") 146 | ;; 147 | --show-cmd-output|-O) MINICONOPTS+=("--plugin=strace:showoutput=true");; 148 | --execution|-E) bashc.parameters_next 149 | PARAM="$(bashc.parameters_current)" 150 | SECONDARYEXECS+=("-E") 151 | SECONDARYEXECS+=("$PARAM");; 152 | --run|-R) bashc.parameters_next 153 | PARAM="$(bashc.parameters_current)" 154 | SECONDARYEXECS+=("-R") 155 | SECONDARYEXECS+=("$PARAM");; 156 | --second|-2) bashc.parameters_next 157 | PARAM="$(bashc.parameters_current)" 158 | SECONDIMAGE="$PARAM";; 159 | --image|-i) bashc.parameters_next 160 | PARAM="$(bashc.parameters_current)" 161 | FROMIMAGE="$PARAM";; 162 | --tag|-t) bashc.parameters_next 163 | PARAM="$(bashc.parameters_current)" 164 | NEWNAME="$PARAM";; 165 | --simulate|-s) SIMULATEONLY=true;; 166 | --verbose|-v) MINICONOPTS+=("--verbose") 167 | IMPORTCONOPTS+=("--verbose") 168 | MERGECONOPTS+=("--verbose") 169 | VERBOSE=true;; 170 | --debug) MINICONOPTS+=("--debug") 171 | IMPORTCONOPTS+=("--debug") 172 | MERGECONOPTS+=("--debug") 173 | DEBUG=true;; 174 | --version | -V) echo "$VERSION" && bashc.finalize;; 175 | --help | -h) usage && bashc.finalize;; 176 | --keeptemporary|-k) KEEPTEMPORARY="true";; 177 | --docker-opts) while bashc.parameters_next; do 178 | PARAM="$(bashc.parameters_current)" 179 | if [ "$PARAM" == "--" ]; then 180 | bashc.parameters_prev 181 | break 182 | fi 183 | DOCKEROPTS+=("$PARAM") 184 | done;; 185 | --) while bashc.parameters_next; do 186 | PARAM="$(bashc.parameters_current)" 187 | MAINEXEC+=("$PARAM") 188 | done;; 189 | *) usage && bashc.finalize 1 "invalid parameter $PARAM" 190 | esac 191 | done 192 | 193 | verify_dependencies 194 | 195 | if [ "$FROMIMAGE" == "" ]; then 196 | bashc.finalize 1 "you have to state the image to minimize (flag -i)" 197 | fi 198 | 199 | if [ "$NEWNAME" == "" ]; then 200 | p_warning "No image provided. A random one will be created (it will be echoed at the end of the procedure)" 201 | fi 202 | 203 | # Get the Entrypoint and the default command 204 | CMDLINE_CMD=() 205 | CMDLINE_EP=() 206 | bashc.lines_to_array CMDLINE_CMD "$(get_config_field "$FROMIMAGE" "Cmd")" 207 | bashc.lines_to_array CMDLINE_EP "$(get_config_field "$FROMIMAGE" "Entrypoint")" 208 | 209 | if [ "$CMDLINE_EP" == "null" ]; then 210 | CMDLINE_EP=() 211 | else 212 | p_debug "found Entrypoint: ${CMDLINE_EP[@]}" 213 | fi 214 | 215 | if [ "$CMDLINE_CMD" == "null" ]; then 216 | CMDLINE_CMD=() 217 | else 218 | p_debug "found Cmd: ${CMDLINE_CMD[@]}" 219 | fi 220 | 221 | TMPDIR=$(bashc.tempdir) 222 | REMOTEMINICON="/bin/minicon" 223 | MINICONFOLDER="-v $(readlink -e $MINICON):$REMOTEMINICON:ro" 224 | REMOTEWORKING="/minicon/work" 225 | WORKINGFOLDER="-v $TMPDIR:$REMOTEWORKING" 226 | 227 | p_info "minimizing container using minicon" 228 | 229 | # Add dependencies (if they are requested) 230 | DEPENDENCIES= 231 | if [ "$DEPENDENCIES_APT" == "true" ]; then 232 | if [ "$DEBUG" == "true" ]; then 233 | DEPENDENCIES="$(bashc.build_cmdline apt-get -y update \&\& apt-get -y install bash libc-bin tar rsync file strace \&\&)" 234 | else 235 | DEPENDENCIES="$(bashc.build_cmdline apt-get -y update \> /dev/null 2\> /dev/null \&\& apt-get -y install bash libc-bin tar rsync file strace \> /dev/null 2\> /dev/null \&\&)" 236 | fi 237 | else 238 | if [ "$DEPENDENCIES_YUM" == "true" ]; then 239 | if [ "$DEBUG" == "true" ]; then 240 | DEPENDENCIES="$(bashc.build_cmdline yum install -y bash tar rsync file strace which \&\&)" 241 | else 242 | DEPENDENCIES="$(bashc.build_cmdline yum install -y bash tar rsync file strace which \> /dev/null 2\> /dev/null \&\&)" 243 | fi 244 | fi 245 | fi 246 | 247 | if [ "$DEPENDENCIES" == "" ]; then 248 | p_warning "Dependencies are not installed. Please make sure that the dependencies for the minicon plugins are installed in the image $FROMIMAGE. Otherwise, please use --apt or --yum flags" 249 | fi 250 | 251 | # The default behavior is to add the Entrypoint to the MAINEXEC... unless it uses the first -E command 252 | ADDEPTOMAINEXEC=true 253 | 254 | # If we do not have a main execution, we'll try to use the default command 255 | if [ "${MAINEXEC}" == "" ]; then 256 | MAINEXEC=() 257 | 258 | if [ "$CMDLINE_EP" != "" ]; then 259 | ADDEPTOMAINEXEC=false 260 | MAINEXEC+=("${CMDLINE_EP[@]}") 261 | fi 262 | 263 | if [ "$CMDLINE_CMD" != "" ]; then 264 | MAINEXEC+=("${CMDLINE_CMD[@]}") 265 | 266 | # If we grab the default command as the main execution, discard the main command 267 | CMDLINE_CMD=() 268 | fi 269 | 270 | if [ "${MAINEXEC}" != "" ]; then 271 | p_info "using the default command as the main execution" 272 | fi 273 | fi 274 | 275 | # If we still do not have a main execution, let's try if the user added additional executions or runs and we'll get the first one as the main one 276 | if [ "${MAINEXEC}" == "" ]; then 277 | if [ ${#SECONDARYEXECS[@]} -gt 0 ]; then 278 | bashc.arrayze_cmd MAINEXEC "${SECONDARYEXECS[@]:1:1}" 279 | if [ "${SECONDARYEXECS[0]}" == "-E" ]; then 280 | ADDEPTOMAINEXEC=false 281 | p_info "using the first execution as the main execution" 282 | else 283 | p_info "using the first run as the main execution" 284 | fi 285 | SECONDARYEXECS=("${SECONDARYEXECS[@]:2}") 286 | fi 287 | fi 288 | 289 | # If we have additional executions, add them 290 | if [ ${#SECONDARYEXECS[@]} -gt 0 ]; then 291 | n=0 292 | while ((n<${#SECONDARYEXECS[@]})); do 293 | RUNOREXEC="${SECONDARYEXECS[$n]}" 294 | n=$((n+1)) 295 | MINICONOPTS+=("-E") 296 | CUR_EXECUTION= 297 | 298 | # An execution (-E) does not add the entrypoint, while a run (-R) must add it: docker run ... vs docker exec ... 299 | # TODO: at this moment each run is independent from the other... it means that an application will not be running after 300 | # it has been analyzed. 301 | if [ "$RUNOREXEC" == "-R" ]; then 302 | if [ "$CMDLINE_EP" != "" ]; then 303 | CUR_EXECUTION="${CMDLINE_EP[@]} " 304 | fi 305 | fi 306 | CUR_EXECUTION="${CUR_EXECUTION}${SECONDARYEXECS[$n]}" 307 | MINICONOPTS+=("$CUR_EXECUTION") 308 | n=$((n+1)) 309 | done 310 | fi 311 | 312 | # If the user stated that additionally add the default command, we'll add it 313 | if [ "$EXECUTEDEFAULTCMD" == "true" ]; then 314 | if [ "$CMDLINE_CMD" != "" ]; then 315 | MINICONOPTS+=("-E") 316 | CUR_EXECUTION=() 317 | if [ "$CMDLINE_EP" != "" ]; then 318 | CUR_EXECUTION=("${CMDLINE_EP[@]}") 319 | fi 320 | CUR_EXECUTION+=("${CMDLINE_CMD[@]}") 321 | CUR_EXECUTION="${CUR_EXECUTION[@]}" 322 | MINICONOPTS+=("${CUR_EXECUTION}") 323 | else 324 | p_warning "no default execution is additionally run" 325 | fi 326 | fi 327 | 328 | # Now we have to state the main execution (whether stated explicitly, implicitly by not adding a default execution or the first execution) 329 | if [ "$CMDLINE_EP" != "" -a "$ADDEPTOMAINEXEC" == "true" ]; then 330 | MAINEXEC=("${CMDLINE_EP[@]}" "${MAINEXEC[@]}") 331 | fi 332 | 333 | if [ ${#MAINEXEC} -eq 0 ]; then 334 | bashc.finalize 1 "no execution available (neither default execution, main execution or additional execution)" 335 | fi 336 | 337 | # Add the main execution 338 | MINICONOPTS+=("--") 339 | MINICONOPTS+=("${MAINEXEC[@]}") 340 | 341 | p_debug docker run --rm $MINICONFOLDER $WORKINGFOLDER --entrypoint "" "${DOCKEROPTS[@]}" "$FROMIMAGE" \ 342 | sh -c "${DEPENDENCIES}$(bashc.build_cmdline "$REMOTEMINICON" "-t" "$REMOTEWORKING/image.tar" "${MINICONOPTS[@]}")" 343 | 344 | if [ "$SIMULATEONLY" == "true" ]; then 345 | p_warning "not executing because only wanted to simulate" 346 | else 347 | docker run --rm $MINICONFOLDER $WORKINGFOLDER --entrypoint "" "${DOCKEROPTS[@]}" "$FROMIMAGE" \ 348 | sh -c "${DEPENDENCIES}$(bashc.build_cmdline "$REMOTEMINICON" "-t" "$REMOTEWORKING/image.tar" "${MINICONOPTS[@]}")" 349 | 350 | if [ $? -ne 0 ]; then 351 | bashc.finalize 1 "failed to run minicon" 352 | fi 353 | fi 354 | 355 | # Now we import the resulting filesystem into docker (copying the entrypoint, etc.) 356 | IMPORTCONOPTS="${IMPORTCONOPTS[@]}" 357 | p_debug $IMPORTCON $IMPORTCONOPTS -i "$FROMIMAGE" -t "$NEWNAME" -A $TMPDIR/image.tar 358 | 359 | if [ "$SIMULATEONLY" == "true" ]; then 360 | p_warning "not executing because only wanted to simulate" 361 | if [ "$NEWNAME" == "" ]; then 362 | NEWNAME="$(generate_dockerimagename)" 363 | fi 364 | IMAGENAME="$NEWNAME" 365 | else 366 | IMAGENAME="$($IMPORTCON $IMPORTCONOPTS -i "$FROMIMAGE" -t "$NEWNAME" -A $TMPDIR/image.tar)" 367 | 368 | if [ $? -ne 0 ]; then 369 | bashc.finalize 1 "failed to run importcon" 370 | fi 371 | fi 372 | 373 | p_info "image $IMAGENAME imported" 374 | p_out "$IMAGENAME" 375 | 376 | if [ "$SECONDIMAGE" == "" ]; then 377 | bashc.finalize 378 | fi 379 | 380 | # Merge the container (if requested) 381 | p_info "merging with container $SECONDIMAGE using mergecon" 382 | p_debug $MERGECON "${MERGECONOPTS[@]}" -1 "$IMAGENAME" -2 "$SECONDIMAGE" -t "$IMAGENAME" 383 | 384 | if [ "$SIMULATEONLY" == "true" ]; then 385 | p_warning "not executing because only wanted to simulate" 386 | else 387 | $MERGECON "${MERGECONOPTS[@]}" -1 "$IMAGENAME" -2 "$SECONDIMAGE" -t "$IMAGENAME" 388 | 389 | if [ $? -ne 0 ]; then 390 | bashc.finalize 1 "failed to run mergecon" 391 | fi 392 | fi -------------------------------------------------------------------------------- /src/lib/plugins.bashc: -------------------------------------------------------------------------------- 1 | function plugin_parameter() { 2 | # Gets the value of a parameter passed to a plugin 3 | # the format is: :=:=... 4 | local PLUGIN="$1" 5 | local PARAMETER="$2" 6 | local PARAMS PP PP2 K V 7 | 8 | while read -d ',' PP; do 9 | if [[ "$PP" =~ ^$PLUGIN\: ]]; then 10 | PARAMS="${PP:$((${#PLUGIN}+1))}" 11 | while read -d ':' PP2; do 12 | IFS='=' read K V <<< "$PP2" 13 | if [ "$K" == "$PARAMETER" ]; then 14 | p_debug "found param $K with value $V" 15 | echo "$V" 16 | return 0 17 | fi 18 | done <<< "${PARAMS}:" 19 | fi 20 | done <<< "${PLUGINS_ACTIVATED}," 21 | return 1 22 | } 23 | 24 | # Credits fot this function go to https://stackoverflow.com/a/18898782 25 | # Return relative path from canonical absolute dir path $1 to canonical 26 | # absolute dir path $2 ($1 and/or $2 may end with one or no "/"). 27 | # Does only need POSIX shell builtins (no external command) 28 | function relPath () { 29 | local common path up 30 | common=${1%/} path=${2%/}/ 31 | while test "${path#"$common"/}" = "$path"; do 32 | common=${common%/*} up=../$up 33 | done 34 | path=$up${path#"$common"/}; path=${path%/}; printf %s "${path:-.}" 35 | } 36 | 37 | function buildpaths() { 38 | local S_PATH="$1" 39 | local L_PATH="$S_PATH" 40 | local C_PATH="$S_PATH" 41 | 42 | while [ "$C_PATH" != "/" -a "$C_PATH" != "." -a "$C_PATH" != ".." ]; do 43 | C_PATH="$(dirname "$C_PATH")" 44 | L_PATH="$C_PATH 45 | $L_PATH" 46 | done 47 | echo "$L_PATH" 48 | } 49 | 50 | function checklinks() { 51 | local L_PATH="$1" 52 | local C_PATH 53 | local D_PATH 54 | local FOUND 55 | while read C_PATH; do 56 | if [ "$C_PATH" != "" -a "$C_PATH" != "." -a "$C_PATH" != ".." -a -h "$C_PATH" ]; then 57 | D_PATH="$(readlink "$C_PATH")" 58 | if [ "${D_PATH:0:1}" != "/" ]; then 59 | D_PATH="$(dirname "${C_PATH}")/${D_PATH}" 60 | fi 61 | D_PATH="$(realpath -Ls "$D_PATH")" 62 | FOUND=true 63 | break 64 | fi 65 | done <<< "$L_PATH" 66 | if [ "$FOUND" == "true" ]; then 67 | local F_PATH="$(echo "$L_PATH" | tail -n 1)" 68 | local N_PATH="${#C_PATH}" 69 | N_PATH="${D_PATH}${F_PATH:$N_PATH}" 70 | echo "$F_PATH:$C_PATH:$D_PATH:$N_PATH" 71 | fi 72 | } 73 | 74 | function _linked_file() { 75 | local L_PATH="$1" 76 | local L_PATHS 77 | local RES 78 | local C_FILE="$L_PATH" 79 | local FINAL_FILE="$L_PATH" 80 | 81 | # Special cases that we do not want to handle 82 | if [ "$L_PATH" == "." -o "$L_PATH" == ".." ]; then 83 | echo "$L_PATH" 84 | return 85 | fi 86 | 87 | while [ "$C_FILE" != "" ]; do 88 | L_PATHS="$(buildpaths "$C_FILE")" 89 | RES="$(checklinks "$L_PATHS")" 90 | if [ "$RES" != "" ]; then 91 | local SRC DST L_ORIG L_DST 92 | IFS=':' read SRC L_ORIG L_DST DST <<< "${RES}:" 93 | p_debug "$L_ORIG -> $L_DST" 94 | L_DST="$(relPath "$(dirname $L_ORIG)" "$L_DST")" 95 | 96 | local EFF_FILE="$ROOTFS/$L_ORIG" 97 | mkdir -p "$(dirname "$EFF_FILE")" 98 | ln -s "$L_DST" "$EFF_FILE" 2> /dev/null 99 | C_FILE="$DST" 100 | else 101 | FINAL_FILE="$C_FILE" 102 | C_FILE= 103 | fi 104 | done 105 | echo "$FINAL_FILE" 106 | } 107 | 108 | function PLUGIN_00_link() { 109 | # If the path is a link to other path, we will create the link and analyze the real path 110 | local L_PATH="$1" 111 | 112 | local L_PATH="$(_linked_file "$1")" 113 | if [ "$L_PATH" != "$1" ]; then 114 | add_command "$L_PATH" 115 | return 1 116 | fi 117 | return 0 118 | } 119 | 120 | function PLUGIN_01_which() { 121 | # This plugin tries to guess whether the command to analize is in the path or not. 122 | # If the command can be obtained calling which, we'll analyze the actual command and not the short name. 123 | local S_PATH="$1" 124 | local W_PATH="$(which $S_PATH)" 125 | 126 | if [ "$W_PATH" != "" -a "$W_PATH" != "$S_PATH" ]; then 127 | p_debug "$1 is $W_PATH" 128 | add_command "$W_PATH" 129 | return 1 130 | fi 131 | } 132 | 133 | function PLUGIN_02_folder() { 134 | # If it is a folder, just copy it to its location in the new FS 135 | local S_PATH="$1" 136 | 137 | if [ -d "$S_PATH" ]; then 138 | p_debug "copying the whole folder $S_PATH" 139 | copy "$S_PATH" -r 140 | return 1 141 | fi 142 | 143 | return 0 144 | } 145 | 146 | function PLUGIN_09_ldd() { 147 | # Checks the list of dynamic libraries using ldd and copy them to the proper folder 148 | local S_PATH="$1" 149 | local LIBS= LIB= 150 | local COMMAND="$(which -- $S_PATH)" 151 | local LIB_DIR= 152 | if [ "$COMMAND" == "" ]; then 153 | COMMAND="$S_PATH" 154 | fi 155 | 156 | COMMAND="$(readlink -e $COMMAND)" 157 | if [ "$COMMAND" == "" ]; then 158 | p_debug "cannot analize $S_PATH using ldd" 159 | return 0 160 | fi 161 | 162 | p_info "inspect command $COMMAND" 163 | ldd "$COMMAND" > /dev/null 2> /dev/null 164 | if [ $? -eq 0 ]; then 165 | LIBS="$(ldd "$COMMAND" | grep -v 'linux-vdso' | grep -v 'statically' | sed 's/^[ \t]*//g' | sed 's/^.* => //g' | sed 's/(.*)//' | sed '/^[ ]*$/d')" 166 | for LIB in $LIBS; do 167 | # Here we build the ld config file to add the new paths where the libraries are located 168 | if [ "$LDCONFIGFILE" != "" ]; then 169 | LIB_DIR="$(dirname "$LIB")" 170 | mkdir -p "$ROOTFS/$(dirname $LDCONFIGFILE)" 171 | echo "$LIB_DIR" >> "$ROOTFS/$LDCONFIGFILE" 172 | fi 173 | add_command "$LIB" 174 | done 175 | fi 176 | 177 | copy "$COMMAND" 178 | } 179 | 180 | function analyze_strace_strings() { 181 | local STRINGS="$1" 182 | local S _S 183 | while read S; do 184 | S="${S:1:-1}" 185 | if [ "$S" != "!" ]; then 186 | if [ "$S" != "" -a "${S::1}" != "-" -a -e "$S" ]; then 187 | _S="$(readlink -e -- ${S})" 188 | if [ "$_S" != "" -a -e "$_S" ]; then 189 | if [ -f "$_S" -o -d "$_S" ]; then 190 | p_debug "file $S was used" 191 | echo "$S" 192 | fi 193 | fi 194 | fi 195 | fi 196 | done <<< "$STRINGS" 197 | } 198 | 199 | _ALREADY_STRACED=() 200 | 201 | function MARK_straced() { 202 | local CMD="$@" 203 | _ALREADY_STRACED+=("$CMD") 204 | } 205 | 206 | function already_straced() { 207 | local i 208 | local CMD="$@" 209 | for ((i=0;i<${#_ALREADY_STRACED[@]};i=i+1)); do 210 | if [ "${_ALREADY_STRACED[$i]}" == "$CMD" ]; then 211 | return 0 212 | fi 213 | done 214 | return 1 215 | } 216 | 217 | function _strace_mode() { 218 | local MODE="$(plugin_parameter "strace" "mode")" 219 | local ERR=0 220 | case "$MODE" in 221 | loose|regular|slim|skinny|default) 222 | ;; 223 | "") MODE=default;; 224 | *) p_error "invalid mode '$MODE' for strace" 225 | MODE= 226 | ERR=1;; 227 | esac 228 | if [ "$MODE" == "default" ]; then 229 | MODE=skinny 230 | fi 231 | echo "$MODE" 232 | return $ERR 233 | } 234 | 235 | function _strace_copy_folder() { 236 | local _FILE="$1" 237 | local _STRACE_EXCLUDED_PATHS=' 238 | ^//$ 239 | ^\./$ 240 | ^\.\./$ 241 | ^/tmp/$ 242 | ^/boot/$ 243 | ^/home/$ 244 | ^/sys/ 245 | ^/usr/lib(32|64|)/ 246 | ^/lib(32|64|)/ 247 | ^/etc/$ 248 | ^/var/$ 249 | ^/proc/$ 250 | ^/dev/$ 251 | ^/usr/$ 252 | ^/bin/$ 253 | ^/usr/bin/$ 254 | ^/sbin/$ 255 | ^/usr/sbin/$ 256 | ' 257 | 258 | local DN 259 | if [ -f "$_FILE" ]; then 260 | DN="$(dirname "$_FILE")/" 261 | else 262 | DN="$(realpath -Lsm "$_FILE")/" 263 | fi 264 | 265 | if [ ! -d "$DN" ]; then 266 | return 1 267 | fi 268 | 269 | while read EP; do 270 | if [ "$EP" != "" ]; then 271 | if [[ "$DN" =~ $EP ]]; then 272 | p_debug "excluding ${DN} because it matches pattern ${EP}" 273 | return 0 274 | fi 275 | fi 276 | done <<< "$_STRACE_EXCLUDED_PATHS" 277 | 278 | p_debug "copying $DN because $_FILE is in it" 279 | copy "$DN" -r 280 | return 0 281 | } 282 | 283 | function _strace_exec() { 284 | 285 | if already_straced "${COMMAND[@]}"; then 286 | p_debug "command ${COMMAND[@]} already straced" 287 | return 288 | fi 289 | 290 | MARK_straced "${COMMAND[@]}" 291 | 292 | # Execute the app without any parameter, using strace and see which files does it open 293 | local SECONDSSIM=$(plugin_parameter "strace" "seconds") 294 | if [[ ! $SECONDSSIM =~ ^[0-9]*$ ]]; then 295 | SECONDSSIM=3 296 | fi 297 | if [ "$SECONDSSIM" == "" ]; then 298 | SECONDSSIM=3 299 | fi 300 | 301 | local SHOWSTRACE 302 | SHOWSTRACE=$(plugin_parameter "strace" "showoutput") 303 | if [ $? -eq 0 ]; then 304 | if [ "$SHOWSTRACE" == "" ]; then 305 | SHOWSTRACE=true 306 | fi 307 | fi 308 | 309 | local MODE 310 | MODE="$(_strace_mode)" 311 | 312 | if [ $? -ne 0 ]; then 313 | finalize 1 "error in strace parameter" 314 | fi 315 | 316 | p_info "analysing ${COMMAND[@]} using strace and $SECONDSSIM seconds ($MODE)" 317 | 318 | local TMPFILE=$(bashc.tempfile) 319 | if [ "$SHOWSTRACE" == "true" ]; then 320 | timeout -s 9 $SECONDSSIM strace -qq -e file -fF -o "$TMPFILE" "${COMMAND[@]}" 321 | else 322 | { 323 | timeout -s 9 $SECONDSSIM strace -qq -e file -fF -o "$TMPFILE" "${COMMAND[@]}" > /dev/null 2> /dev/null 324 | } > /dev/null 2> /dev/null 325 | fi 326 | 327 | # Now we'll inspect the files that the execution has used 328 | local EXEC_FUNCTIONS="exec.*" 329 | local STRINGS 330 | local L BN DN 331 | 332 | # Add all the folders and files that are used, but analyze libraries or executable files 333 | #FUNCTIONS="open|mkdir" 334 | #STRINGS="$(cat "$TMPFILE" | grep -E "($FUNCTIONS)\(" | grep -o '"[^"]*"' | sort -u)" 335 | #while read L; do 336 | # if [ "$L" != "" ]; then 337 | # BN="$(basename $L)" 338 | # if [ "${BN::3}" == "lib" -o "${BN: -3}" == ".so" ]; then 339 | # add_command "$L" 340 | # else 341 | # copy "$L" -r 342 | # fi 343 | # fi 344 | #done <<< "$(analyze_strace_strings "$STRINGS")" 345 | 346 | # Add all the folders and files that checked to exist (folders are not copied, just may 347 | # appear in the resulting filesystem, but libraries are analyzed) 348 | # FUNCTIONS="stat|lstat" 349 | 350 | bashc.push_logger "PLAIN" 351 | STRINGS="$(cat "$TMPFILE" | grep -vE "($EXEC_FUNCTIONS)\(" | grep -o '"[^"]*"' | sort -u)" 352 | while read L; do 353 | if [ "$L" != "" -a "$L" != "/" -a "$L" != "." -a "$L" != ".." ]; then 354 | if [ -f "$L" ]; then 355 | BN="$(basename $L)" 356 | if [ "${BN::3}" == "lib" -o "${BN: -3}" == ".so" ]; then 357 | add_command "$L" 358 | else 359 | copy "$L" 360 | fi 361 | else 362 | copy "$L" 363 | fi 364 | fi 365 | done <<< "$(analyze_strace_strings "$STRINGS")" 366 | bashc.pop_logger 367 | 368 | # If the mode is slim, we'll also copy the whole opened (or created) folders 369 | if [ "$MODE" == "slim" -o "$MODE" == "regular" -o "$MODE" == "loose" ]; then 370 | bashc.push_logger "OPENDIRS" 371 | local FUNCTIONS="open|mkdir" 372 | STRINGS="$(cat "$TMPFILE" | grep -E "($FUNCTIONS)\(" | grep -o '"[^"]*"' | sort -u)" 373 | while read L; do 374 | if [ "$L" != "" ]; then 375 | if [ -d "$L" ]; then 376 | copy "$L" -r 377 | fi 378 | fi 379 | done <<< "$(analyze_strace_strings "$STRINGS")" 380 | bashc.pop_logger 381 | fi 382 | 383 | # If the mode is regular, we'll copy the whole folders used 384 | if [ "$MODE" == "loose" ]; then 385 | bashc.push_logger "COPYDIRS" 386 | STRINGS="$(cat "$TMPFILE" | grep -vE "($EXEC_FUNCTIONS)\(" | grep -o '"[^"]*"' | sort -u)" 387 | while read L; do 388 | if [ "$L" != "" ]; then 389 | if [ -d "$L" ]; then 390 | _strace_copy_folder "$L" 391 | # copy "$L" -r 392 | fi 393 | fi 394 | done <<< "$(analyze_strace_strings "$STRINGS")" 395 | bashc.pop_logger 396 | fi 397 | 398 | # If the mode is slim, we'll also copy the whole opened (or created) folders 399 | if [ "$MODE" == "regular" -o "$MODE" == "loose" ]; then 400 | bashc.push_logger "DIRSFROMFILES" 401 | local FUNCTIONS="open" 402 | STRINGS="$(cat "$TMPFILE" | grep -E "($FUNCTIONS)\(" | grep -o '"[^"]*"' | sort -u)" 403 | while read L; do 404 | if [ "$L" != "" ]; then 405 | _strace_copy_folder "$L" 406 | fi 407 | done <<< "$(analyze_strace_strings "$STRINGS")" 408 | bashc.pop_logger 409 | fi 410 | 411 | # Add all the executables that have been executed (they are analyzed). 412 | # FUNCTIONS="exec.*" 413 | 414 | bashc.push_logger "EXEC" 415 | STRINGS="$(cat "$TMPFILE" | grep -E "($EXEC_FUNCTIONS)\(" | grep -o '"[^"]*"' | sort -u)" 416 | while read L; do 417 | [ "$L" != "" ] && add_command "$L" 418 | done <<< "$(analyze_strace_strings "$STRINGS")" 419 | bashc.pop_logger 420 | 421 | rm "$TMPFILE" 422 | 423 | copy "${COMMAND[0]}" 424 | } 425 | 426 | function PLUGIN_10_strace() { 427 | # Execute the app without any parameter, using strace and see which files does it open 428 | 429 | # A file that contains examples of calls for the commands to be considered (e.g. this is because 430 | # some commands will not perform any operation if they do not have parameters; e.g. echo) 431 | local EXECFILE=$(plugin_parameter "strace" "execfile") 432 | 433 | local S_PATH="$1" 434 | local COMMAND="$(which -- $S_PATH)" 435 | if [ "$COMMAND" == "" ]; then 436 | p_debug "cannot analize $S_PATH using strace" 437 | return 0 438 | fi 439 | 440 | # Let's see if there is a specific commandline (with parameters) for this command in the file 441 | local CMDTOEXEC CMDLINE=() 442 | if [ -e "$EXECFILE" ]; then 443 | local L 444 | while read L; do 445 | CMDTOEXEC= 446 | bashc.arrayze_cmd CMDLINE "$L" 447 | n=0 448 | while [ $n -lt ${#CMDLINE[@]} ]; do 449 | if [ "${CMDLINE[$n]}" == "$COMMAND" ]; then 450 | CMDTOEXEC="$L" 451 | break 452 | fi 453 | n=$((n+1)) 454 | done 455 | if [ "$CMDTOEXEC" != "" ]; then 456 | break 457 | fi 458 | done < "$EXECFILE" 459 | fi 460 | 461 | COMMAND=($COMMAND) 462 | 463 | # If there is a specific commandline, we'll use it; otherwise we'll run the command as-is 464 | if [ "$CMDTOEXEC" != "" ]; then 465 | p_debug "will run $CMDTOEXEC" 466 | COMMAND=( ${CMDLINE[@]} ) 467 | fi 468 | 469 | _strace_exec 470 | } 471 | 472 | function STRACE_command() { 473 | local CMDLINE=() 474 | local COMMAND 475 | 476 | bashc.push_logger "STRACE" 477 | 478 | bashc.arrayze_cmd CMDLINE "$1" 479 | local _PLUGINS_ACTIVATED="${PLUGINS_ACTIVATED}" 480 | 481 | if [ "${CMDLINE[0]}" != "" ]; then 482 | local S_PATH="${CMDLINE[0]}" 483 | local OPTIONS="${S_PATH%,*}" 484 | local N_PATH="${S_PATH##*,}" 485 | 486 | if [ "$N_PATH" != "" -a "$OPTIONS" != "" ]; then 487 | # Remove the possible leading spaces before the options 488 | OPTIONS="$(bashc.trim "$OPTIONS")" 489 | p_info "specific options to strace: $OPTIONS" 490 | PLUGINS_ACTIVATED="${PLUGINS_ACTIVATED},strace:${OPTIONS}" 491 | S_PATH="$N_PATH" 492 | fi 493 | 494 | COMMAND="$(which -- $S_PATH)" 495 | if [ "$COMMAND" == "" ]; then 496 | p_debug "cannot analize $S_PATH using strace" 497 | bashc.pop_logger 498 | return 0 499 | fi 500 | CMDLINE[0]="$COMMAND" 501 | fi 502 | COMMAND=( "${CMDLINE[@]}" ) 503 | _strace_exec 504 | 505 | PLUGINS_ACTIVATED="${_PLUGINS_ACTIVATED}" 506 | 507 | bashc.pop_logger 508 | } 509 | 510 | function PLUGIN_11_scripts() { 511 | # Checks the output of the invocation to the "file" command and guess whether it is a interpreted script or not 512 | # If it is, adds the interpreter to the list of commands to add to the container 513 | p_debug "trying to guess if $1 is a interpreted script" 514 | 515 | local INCLUDEFOLDERS 516 | INCLUDEFOLDERS=$(plugin_parameter "scripts" "includefolders") 517 | if [ $? -eq 0 ]; then 518 | if [ "$INCLUDEFOLDERS" == "" ]; then 519 | INCLUDEFOLDERS=true 520 | fi 521 | else 522 | # The default value is to include the folders that the interpreter may use 523 | INCLUDEFOLDERS=false 524 | fi 525 | 526 | local S_PATH="$(which $1)" 527 | local ADD_PATHS= 528 | 529 | if [ "$S_PATH" == "" -o ! -x "$S_PATH" ]; then 530 | p_debug "$1 cannot be executed (if it should, please check the path)" 531 | return 0 532 | fi 533 | 534 | local FILE_RES="$(file $S_PATH | grep -o ':.* script')" 535 | if [ "$FILE_RES" == "" ]; then 536 | p_debug "$S_PATH is not recognised as a executable script" 537 | return 0 538 | fi 539 | 540 | FILE_RES="${FILE_RES:2:-7}" 541 | FILE_RES="${FILE_RES,,}" 542 | local SHELL_EXEC= 543 | local SHBANG_LINE=$(cat $S_PATH | sed '/^#!.*/q' | tail -n 1 | sed 's/^#![ ]*//') 544 | local INTERPRETER="${SHBANG_LINE%% *}" 545 | ADD_PATHS="$INTERPRETER" 546 | local ENV_APP= 547 | if [ "$(basename $INTERPRETER)" == "env" ]; then 548 | ADD_PATHS="$INTERPRETER" 549 | ENV_APP="${SHBANG_LINE#* }" # This is in case there are parameters for the interpreter e.g. #!/usr/bin/env bash -c 550 | ENV_APP="${ENV_APP%% *}" 551 | local W_ENV_APP="$(which "$ENV_APP")" 552 | if [ "$W_ENV_APP" != "" ]; then 553 | ENV_APP="$W_ENV_APP" 554 | fi 555 | fi 556 | 557 | case "$(basename "$INTERPRETER")" in 558 | perl) ;; 559 | python) ;; 560 | bash) ;; 561 | sh) ;; 562 | env) ADD_PATHS="${ADD_PATHS} 563 | ${ENV_APP}";; 564 | *) p_warning "interpreter $INTERPRETER not recognised" 565 | return 0;; 566 | esac 567 | 568 | # If we want to include the 'include' folders of the scripts (to also include libraries), let's get them 569 | if [ "$INCLUDEFOLDERS" == "true" ]; then 570 | case "$(basename "$INTERPRETER")" in 571 | perl) ADD_PATHS="${ADD_PATHS} 572 | $(perl -e "print qq(@INC)" | tr ' ' '\n' | grep -v -e '^/home' -e '^\.')";; 573 | python) ADD_PATHS="${ADD_PATHS} 574 | $(python -c 'import sys;print "\n".join(sys.path)' | grep -v -e '^/home' -e '^\.')";; 575 | esac 576 | fi 577 | 578 | if [ "$ADD_PATHS" != "" ]; then 579 | p_debug "found that $S_PATH needs $ADD_PATHS" 580 | local P 581 | while read P; do 582 | [ "$P" != "" ] && add_command "$P" 583 | done <<< "$ADD_PATHS" 584 | fi 585 | return 0 586 | } 587 | 588 | function PLUGIN_funcs() { 589 | # Gets the list of plugins available for the app (those functions named PLUGIN_xxx_) 590 | echo "$(typeset -F | grep PLUGIN_ | awk '{print $3}' | grep -v 'PLUGIN_funcs')" 591 | } 592 | 593 | function plugin_list() { 594 | local P 595 | while read P; do 596 | echo -n "${P##*_}," 597 | done <<< "$(PLUGIN_funcs)" 598 | echo 599 | } -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # minicon - minimization containers 2 | 3 | When you run containers (e.g. in Docker), you usually run a system that has a whole Operating System, documentation, extra packages, etc. and your specific application. The result is that the footprint of the container is bigger than needed. 4 | 5 | **minicon** is a general tool to analyze applications and executions of these applications to obtain a filesystem that contains all the dependencies that have been detected. In particular, it can be used to reduce Docker containers. The **minicon** package includes **minidock** 6 | which will help to reduce Docker containers by hiding the underlying complexity of running **minicon** inside a Docker container. 7 | 8 | The purpose of **minicon** and **minidock** is better understood with the use cases explained in depth in the section "[Examples](#4-examples)": the size of a basic UI that contains bash, ip, wget, ssh, etc. commands is _reduced from 211MB to 10.9MB_; the size of a NodeJS application along with the server is _reduced from 686 MB (using the official node image) to 45.6MB_; the size of an Apache server is _reduced from 216MB to 50.4MB_, and the size of a Perl application in a Docker container is _reduced from 206MB to 5.81MB_. 9 | 10 | > [**minidock**](doc/minidock.md) is based on [**minicon**](doc/minicon.md), [**importcon**](doc/importcon.md) and [**mergecon**](doc/mergecon.md), and hides the complexity of creating a container, mapping minicon, guessing parameters such as the entrypoint or the default command, creating the proper commandline, etc. 11 | 12 | ## 1. Why **minicon** and **minidock**? 13 | 14 | Reducing the footprint of one container is of special interest, to redistribute the container images. 15 | 16 | It is of special interest in cases such as [SCAR](https://github.com/grycap/scar), that executes containers out of Docker images in AWS Lambda. In that case, the use cases are limited by the size of the container (the ephemeral storage space is limited to 512 Mb., and SCAR needs to pull the image from Docker Hub into the ephemeral storage and then uncompress it; so the maximum size for the container is even more restricted). 17 | 18 | But there are also security reasons to minimize the unneeded application or environment available in one container image. In the case that the application fails, not having other applications reduces the impact of an intrusion (e.g. if the container does not need a compiler, why should it be there? maybe it would enable to compile a rootkit). 19 | 20 | In this sense, the recent publication of the NIST "[Application Container Security Guide](https://doi.org/10.6028/NIST.SP.800-190)" suggests that "_An image should only include the executables and libraries required by the app itself; all other OS functionality is provided by the OS kernel within the underlying host OS_". 21 | 22 | **minicon** is a tool that enables a fine grain minimization for any type of filesystem, but it is possible to use it to reduce Docker images following the next pipeline: 23 | 24 | 1. Preparing a Docker container with the dependencies of **minicon** 25 | 1. Guessing the entrypoint and the default command for the container. 26 | 1. Running **minicon** for these commands (maping the proper folders to get the resulting tar file). 27 | 1. Using **importcon** to import the resulting file to copy the entrypoint and other settings. 28 | 1. etc. 29 | 30 | **minidock** is a one-liner that automates that procedure to make that reducing a container consist in just to convert a 31 | 32 | ```bash 33 | $ docker run --rm -it myimage myapp 34 | ``` 35 | 36 | into 37 | 38 | ```bash 39 | $ minidock -i myimage -t myimage:minicon -- myapp 40 | ``` 41 | 42 | ## 2. Installation 43 | 44 | ### 2.1 From packages 45 | 46 | You can get the proper package (.deb o .rpm) from the [Releases page](https://github.com/grycap/minicon/releases) and install it using the appropriate package manager. 47 | 48 | **Ubuntu/Debian** 49 | 50 | ```bash 51 | $ apt update 52 | $ apt install ./minicon-1.2-1.deb 53 | ``` 54 | 55 | [![asciicast](https://asciinema.org/a/165792.png)](https://asciinema.org/a/165792) 56 | 57 | **CentOS/Fedora/RedHat** 58 | 59 | ```bash 60 | $ yum install epel-release 61 | $ yum install ./minicon-1.2-1.noarch.rpm 62 | ``` 63 | 64 | [![asciicast](https://asciinema.org/a/166107.png)](https://asciinema.org/a/166107) 65 | 66 | ### 2.2 From sources 67 | 68 | **minicon** are a set of bash scripts. So you just simply need to have a working linux with bash and the other dependencies installed and get the code: 69 | 70 | ```bash 71 | $ git clone https://github.com/grycap/minicon 72 | ``` 73 | 74 | In that folder you'll have the **minicon**, **minidock** and other applications. The commands in the _minicon_ distribution must be in the PATH. So I would suggest to put it in the _/opt_ folder and set the proper PATH var. Otherwise leave it in a folder of your choice and set the PATH variable: 75 | 76 | ```bash 77 | $ mv minicon /opt 78 | $ export PATH=$PATH:/opt/minicon 79 | ``` 80 | 81 | #### 2.2.1 Dependencies 82 | 83 | **minidock** depends on the commands _minicon_, _importcon_ and _mergecon_, and the packages _jq_, _tar_ and _docker_. **minicon** and the others have dependencies on other packages. So, you need to install the proper packages in your system: 84 | 85 | **Ubuntu** 86 | 87 | ```bash 88 | $ apt-get install jq tar libc-bin tar file strace rsync 89 | ``` 90 | 91 | **CentOS** 92 | ```bash 93 | $ yum install tar jq glibc-common file strace rsync which 94 | ``` 95 | 96 | ## 3. Usage 97 | 98 | The **minidock suite** enables to prepare filesystems for running containers. The suite consists in the next commands: 99 | 100 | 1. **minidock** ([doc](doc/minidock.md)) analyzes one existing Docker image, reduces its footprint and leaves the new version in the local Docker registry. It makes use of the other tools in the _minicon_ package. 101 | 102 | 1. **minicon** ([doc](doc/minicon.md)) aims at reducing the footprint of the filesystem for the container, just adding those files that are needed. That means that the other files in the original container are removed. 103 | 104 | 1. **importcon** ([doc](doc/importcon.md)) importcon is a tool that imports the contents from a tarball to create a filesystem image using the "docker import" command. But it takes as reference an existing docker image to get parameters such as ENV, USER, WORKDIR, etc. to set them for the new imported image. 105 | 106 | 1. **mergecon** ([doc](doc/mergecon.md)) is a tool that merges the filesystems of two different container images. It creates a new container image that is built from the combination of the layers of the filesystems of the input containers. 107 | 108 | Please refer to the documentation of each command to get help about them. 109 | 110 | ## 4. Examples 111 | 112 | In this section we are including examples on using **minidock**, that makes use of all the other commands. Please refer to the documentation of each command to get examples about each individual command. 113 | 114 | ### 4.1 Basic Ubuntu UI in less than 11 Mb. 115 | 116 | In this example we will create a basic user interface, from ubuntu, that include commands like `wget`, `ssh`, `cat`, etc. 117 | 118 | The `ubuntu:latest` image do not contain such commands. So we need to create a Docker file that installs `wget`, `ssh`, `ping` and others. We will use this Dockerfile: 119 | 120 | ```dockerfile 121 | FROM ubuntu:latest 122 | RUN apt-get update && apt-get install -y ssh iproute2 iputils-ping wget 123 | ``` 124 | 125 | And now, we will build the image by issuing the next command: 126 | 127 | ```bash 128 | $ docker build . -t minicon:ex1fat 129 | ``` 130 | 131 | > At this point you can check the image, and the commands that it has. You just need to create a container and issue the commands that you want to check: `docker run --rm -it minicon:ex1fat bash` 132 | 133 | Once that we have the image, we will minimize it by issuing the next command: 134 | 135 | ```bash 136 | $ minidock -i minicon:ex1fat -t minicon:ex1 --apt -E bash -E 'ssh localhost' \ 137 | -E ip -E id -E cat -E ls -E mkdir \ 138 | -E 'ping -c 1 www.google.es' -- wget www.google.es 139 | ``` 140 | 141 | * Each `-E` flag includes an example of the execution that we want to be able to make in the minimized image. 142 | * The `--apt` flag is included because we want to minimize an apt-based image (that instructs **minidock** to resolve the dependencies inside the container, using apt commands) 143 | * The command after `--` is one of the command lines that we should be able to execute in the resulting image. 144 | 145 | Finally you can verify that the image has drammatically reduced its size: 146 | 147 | ```bash 148 | $ docker images minicon 149 | REPOSITORY TAG IMAGE ID CREATED SIZE 150 | minicon ex1 42a532b9c262 28 minutes ago 10.9MB 151 | minicon ex1fat d3498d9cf260 30 minutes ago 211MB 152 | ``` 153 | 154 | At this point you should be able to run one container, using the resulting image: 155 | 156 | ```dockerfile 157 | $ docker run --rm -it minicon:ex1 bash 158 | ``` 159 | 160 | The whole procedure can be seen in the next asciicast: 161 | 162 | [![asciicast](https://asciinema.org/a/165798.png)](https://asciinema.org/a/165798) 163 | 164 | ### 4.2 Basic CentOS 7 in about 16 Mb. 165 | 166 | In this example we will create the same use-case than in the previous one, but based on a CentOS image: a basic CentOS-based user interface, that include commands like `wget`, `ssh`, `cat`, etc. 167 | 168 | The `centos:latest` image do not contain the needed commands. So we need to create a Docker file that installs `wget`, `ssh`, `ping` and others. We will use this Dockerfile: 169 | 170 | ```dockerfile 171 | FROM centos:latest 172 | RUN yum -y update && yum install -y iproute iputils openssh-clients wget 173 | ``` 174 | 175 | And now, we will build the image by issuing the next command: 176 | 177 | ```bash 178 | $ docker build . -t minicon:ex1fat 179 | ``` 180 | 181 | > At this point you can check the image, and the commands that it has. You just need to create a container and issue the commands that you want to check: `docker run --rm -it minicon:ex1fat bash` 182 | 183 | Once that we have the image, we will minimize it by issuing the next command: 184 | 185 | ```bash 186 | $ minidock -i minicon:ex1fat -t minicon:ex1 --yum -E bash -E 'ssh localhost' \ 187 | -E ip -E id -E cat -E ls -E mkdir \ 188 | -E 'ping -c 1 www.google.es' -- wget www.google.es 189 | ``` 190 | 191 | * Each `-E` flag includes an example of the execution that we want to be able to make in the minimized image. 192 | * The `--yum` flag is included because we want to minimize a yum-based image (that instructs **minidock** to resolve the dependencies inside the container used for simulation, using yum commands) 193 | * The command after `--` is one of the command lines that we should be able to execute in the resulting image. 194 | 195 | Finally you can verify that the image has drammatically reduced its size: 196 | 197 | ```bash 198 | $ docker images 199 | REPOSITORY TAG IMAGE ID CREATED SIZE 200 | minicon ex1 43d11b4837dd About a minute ago 16MB 201 | minicon ex1fat 66c5aa5bb77b 3 minutes ago 362MB 202 | centos latest ff426288ea90 7 weeks ago 207MB 203 | ``` 204 | 205 | At this point you should be able to run one container, using the resulting image: 206 | 207 | ```dockerfile 208 | $ docker run --rm -it minicon:ex1 bash 209 | ``` 210 | 211 | The whole procedure can be seen in the next asciicast: 212 | 213 | [![asciicast](https://asciinema.org/a/166112.png)](https://asciinema.org/a/166112) 214 | 215 | ### 4.3 NodeJS application 216 | 217 | In this example, we will start from the default NodeJS image and will pack our freshly created application. 218 | 219 | In first place we are creating an application using express (for our purposes, we are using the default application): 220 | 221 | ```bash 222 | $ express myapp 223 | ``` 224 | 225 | To dockerize this nodejs application, you can use the default [node image at docker hub](https://hub.docker.com/_/node/), which is based on Debian, and use the next Dockerfile: 226 | 227 | ```dockerfile 228 | FROM node 229 | COPY myapp /usr/app/myapp 230 | WORKDIR /usr/app/myapp 231 | RUN npm install 232 | ENTRYPOINT node ./bin/www 233 | EXPOSE 3000 234 | ``` 235 | 236 | Now we can build our application and test it: 237 | 238 | ```bash 239 | $ docker build . -t minicon:ex2fat 240 | $ docker run --rm -id -p 10000:3000 minicon:ex2fat 241 | 5cb83644120c074f799e2ba802f09690054eae48fdb44d92094550de4f895702 $ wget -q -O- http://localhost:10000 242 | Express

Express

Welcome to Express

243 | ``` 244 | 245 | Once that we have our application, we can minimize it: 246 | 247 | ```bash 248 | $ minidock --apt -i minicon:ex2fat -t minicon:ex2 -I /usr/app/myapp 249 | ``` 250 | 251 | * The `--apt` flag is included because the original image is based on debian (that instructs **minidock** to resolve the dependencies inside the container, using apt commands) 252 | * We do not need to include any command to simulate because the original image has an entrypoint defined, which will be simulated. 253 | * In this example we are not running all the possibilities of our application during the simulation, but we know that the application is stored in `/usr/app/myapp` and that the global modules 254 | 255 | We can test the image: 256 | 257 | ```bash 258 | $ docker run --rm -id -p 10001:3000 minicon:ex2 259 | fedb5c972e8e47ac02c09661f767156aa88328b1ce72646e717bd60624adefda 260 | $ wget -q -O- http://localhost:10001 261 | Express

Express

Welcome to Express

calfonso@ubuntu:~/ex2$ 262 | ``` 263 | 264 | If we check the size of the original and the minimized images, we can see that it has been reduced from 686 MB. to 45.6MB. (which is even less than the official node:alpine image). 265 | ```bash 266 | $ docker images 267 | REPOSITORY TAG IMAGE ID CREATED SIZE 268 | minicon ex2 1080e761a83c 38 seconds ago 45.6MB 269 | minicon ex2fat 7f8bef02d321 4 minutes ago 686MB 270 | node alpine a88ff852e3d4 4 days ago 68MB 271 | node latest 29831ba76d93 4 days ago 676MB 272 | ``` 273 | 274 | The whole procedure can be seen in the next asciicast: 275 | 276 | [![asciicast](https://asciinema.org/a/166058.png)](https://asciinema.org/a/166058) 277 | 278 | 279 | ### 4.4 Apache server 280 | 281 | In order to have an apache server, according to the Docker docs, you can create the following Dockerfile: 282 | 283 | ```docker 284 | FROM ubuntu 285 | RUN apt-get update && apt-get install -y --force-yes apache2 286 | EXPOSE 80 443 287 | VOLUME ["/var/www", "/var/log/apache2", "/etc/apache2"] 288 | ENTRYPOINT ["/usr/sbin/apache2ctl", "-D", "FOREGROUND"] 289 | ``` 290 | 291 | Then you can build it and run it: 292 | 293 | ```bash 294 | $ docker build . -t minicon:uc5fat 295 | ... 296 | $ docker run -id -p 10000:80 minicon:uc5fat 297 | fe20ebce12f2d5460bb0191975450833117528987c32c95849315bc4330c0f2a 298 | $ wget -q -O- localhost:10000 | head -n 3 299 | 300 | 301 | 302 | ``` 303 | 304 | In this case, the size of the image is about 261MB: 305 | 306 | ```bash 307 | $ docker images 308 | REPOSITORY TAG IMAGE ID CREATED SIZE 309 | minicon uc5fat ff6f2573d73b 9 days ago 261MB 310 | ``` 311 | 312 | In order to reduce it, you just need to issue the next command: 313 | ```bash 314 | $ ./minidock -i minicon:uc5fat -t minicon:uc5 --apt 315 | ... 316 | ``` 317 | 318 | > The flag _--apt_ instructs **minidock** to install the dependencies of minicon using apt-get commands, inside one ephemeral container that will be used for the analysis. It is also possible to use _--yum_, instead of _--apt_. 319 | 320 | And you will have the minimized apache ready to be run: 321 | 322 | ```bash 323 | $ docker run --rm -id -p 10001:80 minicon:uc5 324 | 0e0ef746586fd632877f1c9344b42b4dbb00f52dc2a5d06028cbfa72bd297d6c 325 | $ wget -q -O- localhost:10001 | head -n 3 326 | 327 | 328 | 329 | ``` 330 | 331 | But in this case, the footprint of the apache image has been reduced to 50.4MB: 332 | 333 | ```bash 334 | $ docker images 335 | REPOSITORY TAG IMAGE ID CREATED SIZE 336 | minicon uc5 f577e1f6e3f8 About a minute ago 50.4MB 337 | minicon uc5fat ff6f2573d73b 9 days ago 261MB 338 | ``` 339 | 340 | ### 4.5 cowsay: Docker image with Entrypoint with parameters 341 | 342 | In order to have a simple cowsay application you can create the following Dockerfile: 343 | 344 | ```docker 345 | FROM ubuntu 346 | RUN apt-get update && apt-get install -y cowsay 347 | ENTRYPOINT ["/usr/games/cowsay"] 348 | ``` 349 | 350 | Then you can build it and run it: 351 | 352 | ```bash 353 | $ docker build . -t minicon:uc6fat 354 | ... 355 | $ docker run --rm -it minicon:uc6fat i am a cow in a fat container 356 | _______________________________ 357 | < i am a cow in a fat container > 358 | ------------------------------- 359 | \ ^__^ 360 | \ (oo)\_______ 361 | (__)\ )\/\ 362 | ||----w | 363 | || || 364 | ``` 365 | 366 | In this case, the entrypoint needs some parameters to be run. If you try to analyze the container simply issuing a command like the next one: 367 | 368 | ```bash 369 | $ ./minidock -i minicon:uc6fat -t minicon:uc6 --apt 370 | ... 371 | $ docker run --rm -it minicon:uc6 i am a cow in a not properly minimized container 372 | cowsay: Could not find default.cow cowfile! 373 | ``` 374 | 375 | It does not work properly, because the execution of the entrypoint has not been successfully simulated (cowsay needs some parameters to run). 376 | 377 | In this case, you should run a **minidock** commandline that include the command that we used to test it, and we will be able to run it: 378 | 379 | ```bash 380 | $ ./minidock -i minicon:uc6fat -t minicon:uc6 --apt -- i am a cow in a fat container 381 | ... 382 | $ docker run --rm -it minicon:uc6 i am a cow in a minimized container 383 | _____________________________________ 384 | < i am a cow in a minimized container > 385 | ------------------------------------- 386 | \ ^__^ 387 | \ (oo)\_______ 388 | (__)\ )\/\ 389 | ||----w | 390 | || || 391 | ``` 392 | 393 | > after the -- flag, we can include those parameters that we use in a docker run execution. 394 | 395 | We can check the differences in the sizes: 396 | 397 | ```bash 398 | $ docker images minicon 399 | REPOSITORY TAG IMAGE ID CREATED SIZE 400 | minicon uc6 7c85b5a104f5 5 seconds ago 5.81MB 401 | minicon uc6fat 1c8179d3ba94 4 hours ago 206MB 402 | ``` 403 | 404 | In this case, the size has been reduced from 206MB to about 5.81MB. 405 | -------------------------------------------------------------------------------- /doc/minidock.md: -------------------------------------------------------------------------------- 1 | # minidock - minimization of Docker containers 2 | 3 | When you run Docker containers, you usually run a system that has a whole Operating System and your specific application. The result is that the footprint of the container is bigger than needed. 4 | 5 | **minidock** aims at reducing the footprint of the Docker containers, by just including in the container those files that are needed. That means that the other files in the original container are removed. 6 | 7 | The purpose of **minidock** is better understood with the use cases explained in depth in the section "[Examples](#4-examples)": the size of an Apache server is reduced from 216MB. to 50.4MB., and the size of a Perl application in a Docker container is reduced from 206MB to 5.81MB. 8 | 9 | 10 | > **minidock** is based on [**minicon**](minicon.md), [**importcon**](importcon.md) and [**mergecon**](mergecon.md), and hides the complexity of creating a container, mapping minicon, guessing parameters such as the entrypoint or the default command, creating the proper commandline, etc. 11 | 12 | ## 1. Why **minidock**? 13 | 14 | Reducing the footprint of one container is of special interest, to redistribute the container images and saving the storage space in your premises. There are also security reasons to minimize the unneeded application or environment available in one container image (e.g. if the container does not need a compiler, why should it be there? maybe it would enable to compile a rootkit). 15 | 16 | In this sense, the publication of the NIST "[Application Container Security Guide](https://doi.org/10.6028/NIST.SP.800-190)" suggests that "_An image should only include the executables and libraries required by the app itself; all other OS functionality is provided by the OS kernel within the underlying host OS_". 17 | 18 | **minicon** is a tool that enables a fine grain minimization for any type of container (it is even interesting for non containerized boxes). Using it for Docker images consist in a simple pipeline: 19 | 20 | 1. Preparing a Docker container with the dependencies of **minicon** 21 | 1. Guessing the entrypoint and the default command for the container. 22 | 1. Running **minicon** for these commands (maping the proper folders to get the resulting tar file). 23 | 1. Using **importcon** to import the resulting file to copy the entrypoint and other settings. 24 | 1. etc. 25 | 26 | **minidock** is a one-liner for that procedure whose aim is just to convert a 27 | 28 | ```bash 29 | $ docker run --rm -it myimage myapp 30 | ``` 31 | 32 | into 33 | 34 | ```bash 35 | $ minicon -i myimage -t myimage:minicon -- myapp 36 | ``` 37 | 38 | To obtain the minimized Docker image, and hiding the internal procedure. 39 | 40 | ## 2. Installation 41 | 42 | ### 2.1 From packages 43 | 44 | You can get the proper package (.deb o .rpm) from the [Releases page](https://github.com/grycap/minicon/releases) and install it using the appropriate package manager. 45 | 46 | **Ubuntu/Debian** 47 | 48 | ```bash 49 | $ apt update 50 | $ apt install ./minicon-1.2-1.deb 51 | ``` 52 | 53 | [![asciicast](https://asciinema.org/a/165792.png)](https://asciinema.org/a/165792) 54 | 55 | **CentOS/Fedora/RedHat** 56 | 57 | ```bash 58 | $ yum install epel-release 59 | $ yum install ./minicon-1.2-1.noarch.rpm 60 | ``` 61 | 62 | [![asciicast](https://asciinema.org/a/166107.png)](https://asciinema.org/a/166107) 63 | 64 | ### 2.2 From sources 65 | 66 | **minidock** is a bash script that runs the other applications in the _minicon_ package, to analyze the docker containers. So you just simply need to have a working linux with bash and the other dependencies installed and get the code: 67 | 68 | ```bash 69 | $ git clone https://github.com/grycap/minicon 70 | ``` 71 | 72 | In that folder you'll have the **minidock** application. Then the commands in the _minicon_ distribution must be in the PATH. So I would suggest to put it in the _/opt_ folder and set the proper PATH var. Otherwise leave it in a folder of your choice and set the PATH variable: 73 | 74 | ```bash 75 | $ mv minicon /opt 76 | $ export PATH=$PATH:/opt/minicon 77 | ``` 78 | 79 | #### 2.2.1 Dependencies 80 | 81 | **minidock** depends on the commands _minicon_, _importcon_ and _mergecon_, and the packages _jq_, _tar_ and _docker_. So, you need to install the proper packages in your system. 82 | 83 | **Ubuntu** 84 | 85 | ```bash 86 | $ apt-get install jq tar 87 | ``` 88 | 89 | **CentOS** 90 | ```bash 91 | $ yum install tar jq which 92 | ``` 93 | ## 3. Usage 94 | 95 | **minidock** has a lot of options. You are advised to run ```./minidock --help``` to get the latest information about the usage of the application. 96 | 97 | The basic syntax is 98 | 99 | ```bash 100 | $ ./minidock [ --docker-opts ] -- 101 | ``` 102 | 103 | Some of the options are: 104 | - \: Is the whole commandline to be analised in the run. These are the same parameters that you would pass to "docker run ... ". 105 | > * the aim is that you run "minidock" as if you used a "docker run" for your container. 106 | - **\**: If you need them, you can include some options that will be raw-passed to the docker run command used during the analysis. (i.e. minidock will executedocker run ...). Some examples are mapping volumes (i.e. **-v** Docker flag) 107 | - **\**: If you need to, you can add some minicon-specific options. The supported options are --include --exclude --plugin: --exclude will exclude some path, --include will include specific files or folder, and --plugin can be used to configure the _minicon_ plugins. 108 | - **--image \**: Name of the existing Docker image to minimize. 109 | - **--tag \**: Tag for the resulting image (random if not provided). 110 | - **--mode \**: Sets the mode to include the used files in the filesystem. There are 4 modes available _skinny_ (default), _slim_, _regular_ and _loose_. 111 | * skinny only includes those files that have been used during the simulation of the commands. 112 | * slim also includes some whole folders that have been opened. 113 | * regular also includes any whole folder that have been checked to exist (e.g. stat). 114 | * loose also includes the whole folders in which are stored the files that have been opened during the simulation. 115 | - **--default-cmd**: Analyze the default command for the containers in the original image. 116 | - **--apt**: Install the dependencies from minicon using apt-get commands (in the container used for the simulation). 117 | - **--yum**: Install the dependencies from minicon using yum commands (in the container used for the simulation). 118 | - **--execution \**: Commandline to analyze when minimizing the container (i.e. that commandline should be able to be executed in the resulting container so the files, libraries, etc. needed should be included). 119 | - **--run \**: Similar to _--execution_, but in this case, the Entrypoint is prepended to the commandline (docker exec vs docker run). 120 | - **-2 \**: If needed, you can merge the resulting minimized image with other. This is very specific for the "mergecon" tool. It is useful for (e.g.) adding a minimal Alpine distro (with _ash_ and so on) to the minimized filesystem. 121 | - **--verbose | -v**: Gives more information about the procedure. 122 | - **--debug**: Gives a lot more information about the procedure. 123 | 124 | ## 4. Examples 125 | 126 | ### 4.1 Basic Ubuntu UI in less than 11 Mb. 127 | 128 | In this example we will create a basic user interface, from ubuntu, that include commands like `wget`, `ssh`, `cat`, etc. 129 | 130 | The `ubuntu:latest` image do not contain such commands. So we need to create a Docker file that installs `wget`, `ssh`, `ping` and others. We will use this Dockerfile: 131 | 132 | ```dockerfile 133 | FROM ubuntu:latest 134 | RUN apt-get update && apt-get install -y ssh iproute2 iputils-ping wget 135 | ``` 136 | 137 | And now, we will build the image by issuing the next command: 138 | 139 | ```bash 140 | $ docker build . -t minicon:ex1fat 141 | ``` 142 | 143 | > At this point you can check the image, and the commands that it has. You just need to create a container and issue the commands that you want to check: `docker run --rm -it minicon:ex1fat bash` 144 | 145 | Once that we have the image, we will minimize it by issuing the next command: 146 | 147 | ```bash 148 | $ minidock -i minicon:ex1fat -t minicon:ex1 --apt -E bash -E 'ssh localhost' \ 149 | -E ip -E id -E cat -E ls -E mkdir \ 150 | -E 'ping -c 1 www.google.es' -- wget www.google.es 151 | ``` 152 | 153 | * Each `-E` flag includes an example of the execution that we want to be able to make in the minimized image. 154 | * The `--apt` flag is included because we want to minimize an apt-based image (that instructs **minidock** to resolve the dependencies inside the container, using apt commands) 155 | * The command after `--` is one of the command lines that we should be able to execute in the resulting image. 156 | 157 | Finally you can verify that the image has drammatically reduced its size: 158 | 159 | ```bash 160 | $ docker images minicon 161 | REPOSITORY TAG IMAGE ID CREATED SIZE 162 | minicon ex1 42a532b9c262 28 minutes ago 10.9MB 163 | minicon ex1fat d3498d9cf260 30 minutes ago 211MB 164 | ``` 165 | 166 | At this point you should be able to run one container, using the resulting image: 167 | 168 | ```dockerfile 169 | $ docker run --rm -it minicon:ex1 bash 170 | ``` 171 | 172 | The whole procedure can be seen in the next asciicast: 173 | 174 | [![asciicast](https://asciinema.org/a/165798.png)](https://asciinema.org/a/165798) 175 | 176 | ### 4.2 Basic CentOS 7 in about 16 Mb. 177 | 178 | In this example we will create the same use-case than in the previous one, but based on a CentOS image: a basic CentOS-based user interface, that include commands like `wget`, `ssh`, `cat`, etc. 179 | 180 | The `centos:latest` image do not contain the needed commands. So we need to create a Docker file that installs `wget`, `ssh`, `ping` and others. We will use this Dockerfile: 181 | 182 | ```dockerfile 183 | FROM centos:latest 184 | RUN yum -y update && yum install -y iproute iputils openssh-clients wget 185 | ``` 186 | 187 | And now, we will build the image by issuing the next command: 188 | 189 | ```bash 190 | $ docker build . -t minicon:ex1fat 191 | ``` 192 | 193 | > At this point you can check the image, and the commands that it has. You just need to create a container and issue the commands that you want to check: `docker run --rm -it minicon:ex1fat bash` 194 | 195 | Once that we have the image, we will minimize it by issuing the next command: 196 | 197 | ```bash 198 | $ minidock -i minicon:ex1fat -t minicon:ex1 --yum -E bash -E 'ssh localhost' \ 199 | -E ip -E id -E cat -E ls -E mkdir \ 200 | -E 'ping -c 1 www.google.es' -- wget www.google.es 201 | ``` 202 | 203 | * Each `-E` flag includes an example of the execution that we want to be able to make in the minimized image. 204 | * The `--yum` flag is included because we want to minimize a yum-based image (that instructs **minidock** to resolve the dependencies inside the container used for simulation, using yum commands) 205 | * The command after `--` is one of the command lines that we should be able to execute in the resulting image. 206 | 207 | Finally you can verify that the image has drammatically reduced its size: 208 | 209 | ```bash 210 | $ docker images 211 | REPOSITORY TAG IMAGE ID CREATED SIZE 212 | minicon ex1 43d11b4837dd About a minute ago 16MB 213 | minicon ex1fat 66c5aa5bb77b 3 minutes ago 362MB 214 | centos latest ff426288ea90 7 weeks ago 207MB 215 | ``` 216 | 217 | At this point you should be able to run one container, using the resulting image: 218 | 219 | ```dockerfile 220 | $ docker run --rm -it minicon:ex1 bash 221 | ``` 222 | 223 | The whole procedure can be seen in the next asciicast: 224 | 225 | [![asciicast](https://asciinema.org/a/166112.png)](https://asciinema.org/a/166112) 226 | 227 | ### 4.3 NodeJS application 228 | 229 | In this example, we will start from the default NodeJS image and will pack our freshly created application. 230 | 231 | In first place we are creating an application using express (for our purposes, we are using the default application): 232 | 233 | ```bash 234 | $ express myapp 235 | ``` 236 | 237 | To dockerize this nodejs application, you can use the default [node image at docker hub](https://hub.docker.com/_/node/), which is based on Debian, and use the next Dockerfile: 238 | 239 | ```dockerfile 240 | FROM node 241 | COPY myapp /usr/app/myapp 242 | WORKDIR /usr/app/myapp 243 | RUN npm install 244 | ENTRYPOINT node ./bin/www 245 | EXPOSE 3000 246 | ``` 247 | 248 | Now we can build our application and test it: 249 | 250 | ```bash 251 | $ docker build . -t minicon:ex2fat 252 | $ docker run --rm -id -p 10000:3000 minicon:ex2fat 253 | 5cb83644120c074f799e2ba802f09690054eae48fdb44d92094550de4f895702 $ wget -q -O- http://localhost:10000 254 | Express

Express

Welcome to Express

255 | ``` 256 | 257 | Once that we have our application, we can minimize it: 258 | 259 | ```bash 260 | $ minidock --apt -i minicon:ex2fat -t minicon:ex2 -I /usr/app/myapp 261 | ``` 262 | 263 | * The `--apt` flag is included because the original image is based on debian (that instructs **minidock** to resolve the dependencies inside the container, using apt commands) 264 | * We do not need to include any command to simulate because the original image has an entrypoint defined, which will be simulated. 265 | * In this example we are not running all the possibilities of our application during the simulation, but we know that the application is stored in `/usr/app/myapp` and that the global modules 266 | 267 | We can test the image: 268 | 269 | ```bash 270 | $ docker run --rm -id -p 10001:3000 minicon:ex2 271 | fedb5c972e8e47ac02c09661f767156aa88328b1ce72646e717bd60624adefda 272 | $ wget -q -O- http://localhost:10001 273 | Express

Express

Welcome to Express

calfonso@ubuntu:~/ex2$ 274 | ``` 275 | 276 | If we check the size of the original and the minimized images, we can see that it has been reduced from 686 MB. to 45.6MB. (which is even less than the official node:alpine image). 277 | ```bash 278 | $ docker images 279 | REPOSITORY TAG IMAGE ID CREATED SIZE 280 | minicon ex2 1080e761a83c 38 seconds ago 45.6MB 281 | minicon ex2fat 7f8bef02d321 4 minutes ago 686MB 282 | node alpine a88ff852e3d4 4 days ago 68MB 283 | node latest 29831ba76d93 4 days ago 676MB 284 | ``` 285 | 286 | The whole procedure can be seen in the next asciicast: 287 | 288 | [![asciicast](https://asciinema.org/a/166058.png)](https://asciinema.org/a/166058) 289 | 290 | 291 | ### 4.4 Apache server 292 | 293 | In order to have an apache server, according to the Docker docs, you can create the following Dockerfile: 294 | 295 | ```docker 296 | FROM ubuntu 297 | RUN apt-get update && apt-get install -y --force-yes apache2 298 | EXPOSE 80 443 299 | VOLUME ["/var/www", "/var/log/apache2", "/etc/apache2"] 300 | ENTRYPOINT ["/usr/sbin/apache2ctl", "-D", "FOREGROUND"] 301 | ``` 302 | 303 | Then you can build it and run it: 304 | 305 | ```bash 306 | $ docker build . -t minicon:uc5fat 307 | ... 308 | $ docker run -id -p 10000:80 minicon:uc5fat 309 | fe20ebce12f2d5460bb0191975450833117528987c32c95849315bc4330c0f2a 310 | $ wget -q -O- localhost:10000 | head -n 3 311 | 312 | 313 | 314 | ``` 315 | 316 | In this case, the size of the image is about 261MB: 317 | 318 | ```bash 319 | $ docker images 320 | REPOSITORY TAG IMAGE ID CREATED SIZE 321 | minicon uc5fat ff6f2573d73b 9 days ago 261MB 322 | ``` 323 | 324 | In order to reduce it, you just need to issue the next command: 325 | ```bash 326 | $ minidock -i minicon:uc5fat -t minicon:uc5 --apt 327 | ... 328 | ``` 329 | 330 | > The flag _--apt_ instructs **minidock** to install the dependencies of minicon using apt-get commands, inside one ephemeral container that will be used for the analysis. It is also possible to use _--yum_, instead of _--apt_. 331 | 332 | And you will have the minimized apache ready to be run: 333 | 334 | ```bash 335 | $ docker run --rm -id -p 10001:80 minicon:uc5 336 | 0e0ef746586fd632877f1c9344b42b4dbb00f52dc2a5d06028cbfa72bd297d6c 337 | $ wget -q -O- localhost:10001 | head -n 3 338 | 339 | 340 | 341 | ``` 342 | 343 | But in this case, the footprint of the apache image has been reduced to 50.4MB: 344 | 345 | ```bash 346 | $ docker images 347 | REPOSITORY TAG IMAGE ID CREATED SIZE 348 | minicon uc5 f577e1f6e3f8 About a minute ago 50.4MB 349 | minicon uc5fat ff6f2573d73b 9 days ago 261MB 350 | ``` 351 | 352 | ### 4.5 cowsay: Docker image with Entrypoint with parameters 353 | 354 | In order to have a simple cowsay application you can create the following Dockerfile: 355 | 356 | ```docker 357 | FROM ubuntu 358 | RUN apt-get update && apt-get install -y cowsay 359 | ENTRYPOINT ["/usr/games/cowsay"] 360 | ``` 361 | 362 | Then you can build it and run it: 363 | 364 | ```bash 365 | $ docker build . -t minicon:uc6fat 366 | ... 367 | $ docker run --rm -it minicon:uc6fat i am a cow in a fat container 368 | _______________________________ 369 | < i am a cow in a fat container > 370 | ------------------------------- 371 | \ ^__^ 372 | \ (oo)\_______ 373 | (__)\ )\/\ 374 | ||----w | 375 | || || 376 | ``` 377 | 378 | In this case, the entrypoint needs some parameters to be run. If you try to analyze the container simply issuing a command like the next one: 379 | 380 | ```bash 381 | $ minidock -i minicon:uc6fat -t minicon:uc6 --apt 382 | ... 383 | $ docker run --rm -it minicon:uc6 i am a cow in a not properly minimized container 384 | cowsay: Could not find default.cow cowfile! 385 | ``` 386 | 387 | It does not work properly, because the execution of the entrypoint has not been successfully simulated (cowsay needs some parameters to run). 388 | 389 | In this case, you should run a **minidock** commandline that include the command that we used to test it, and we will be able to run it: 390 | 391 | ```bash 392 | $ minidock -i minicon:uc6fat -t minicon:uc6 --apt -- i am a cow in a fat container 393 | ... 394 | $ docker run --rm -it minicon:uc6 i am a cow in a minimized container 395 | _____________________________________ 396 | < i am a cow in a minimized container > 397 | ------------------------------------- 398 | \ ^__^ 399 | \ (oo)\_______ 400 | (__)\ )\/\ 401 | ||----w | 402 | || || 403 | ``` 404 | 405 | > after the -- flag, we can include those parameters that we use in a docker run execution. 406 | 407 | We can check the differences in the sizes: 408 | 409 | ```bash 410 | $ docker images minicon 411 | REPOSITORY TAG IMAGE ID CREATED SIZE 412 | minicon uc6 7c85b5a104f5 5 seconds ago 5.81MB 413 | minicon uc6fat 1c8179d3ba94 4 hours ago 206MB 414 | ``` 415 | 416 | In this case, the size has been reduced from 206MB to about 5.81MB. 417 | 418 | # 5. Flexible Manipulation of Container Filesystems 419 | 420 | The **minidock suite** enables to prepare filesystems for running containers. The suite consists in **minidock**, [**minicon**](doc/minicon.md), [**mergecon**](doc/mergecon.md) and [**importcon**](doc/importcon.md): 421 | 422 | 1. **minidock** ([direct link](minidock---minimization-of-docker-containers)) analyzes one existing Docker image, reduces its footprint and leaves the new version in the local Docker registry. It makes use of the other tools in the _minicon_ package. 423 | 424 | 1. **minicon** ([direct link](doc/minicon.md)) aims at reducing the footprint of the filesystem for the container, just adding those files that are needed. That means that the other files in the original container are removed. 425 | 426 | 1. **importcon** ([direct link](doc/importcon.md)) importcon is a tool that imports the contents from a tarball to create a filesystem image using the "docker import" command. But it takes as reference an existing docker image to get parameters such as ENV, USER, WORKDIR, etc. to set them for the new imported image. 427 | 428 | 1. **mergecon** ([direct link](doc/mergecon.md)) is a tool that merges the filesystems of two different container images. It creates a new container image that is built from the combination of the layers of the filesystems of the input containers. 429 | 430 | -------------------------------------------------------------------------------- /doc/minicon.md: -------------------------------------------------------------------------------- 1 | # minicon - MINImization of the filesystems for CONtainers 2 | 3 | When you run containers (e.g. in Docker), you usually run a system that has a whole Operating System, documentation, extra packages, and your specific application. The result is that the footprint of the container is bigger than needed. 4 | 5 | **minicon** aims at reducing the footprint of the filesystem for the container, just adding those files that are needed. That means that the other files in the original container are removed. 6 | 7 | The purpose of **minicon** is better understood with the use cases explained in depth in the section [Use Cases](#24-use-cases). 8 | 9 | 1. **Basic Example ([direct link](#use-case-basic-example))**, that distributes only the set of tools, instead of distributing a whole Linux image. In this case the size is reduced from 123Mb. to about 5.54Mb. 10 | 1. **Basic _user interface_ that need to access to other servers ([direct link](#use-case-basic-user-interface-ssh-cli-wget))**. In this case we have reduced from 222Mb. to about 11Mb., and also we have made that the users only can use a reduced set of tools (ssh, ping, wget, etc.). 11 | 1. **Node.JS+Express application ([direct link](#use-case-nodejsexpress-application))**: The size of the defaut NodeJS Docker image (i.e. node:latest), ready to run an application is about from 691MB. Applying **minicon** to that container, the size is reduced to about 45.3MB. 12 | 1. **Use case: FFMPEG ([direct link](#use-case-ffmpeg))**: The size of a common _Ubuntu+FFMPEG_ image is about 388Mb., but if you apply **minicon** on that image, you will get a working _ffmpeg_ container whose size is only about 119Mb. 13 | 14 | ## 1. Why **minicon**? 15 | 16 | Reducing the footprint of one container is of special interest, to redistribute the container images. 17 | 18 | It is of special interest in cases such as [SCAR](https://github.com/grycap/scar), that executes containers out of Docker images in AWS Lambda. In that case, the use cases are limited by the size of the container (the ephemeral storage space is limited to 512 Mb., and SCAR needs to pull the image from Docker Hub into the ephemeral storage and then uncompress it; so the maximum size for the container is even more restricted). 19 | 20 | But there are also security reasons to minimize the unneeded application or environment available in one container image. In the case that the application fails, not having other applications reduces the impact of an intrusion (e.g. if the container does not need a compiler, why should it be there? maybe it would enable to compile a rootkit). 21 | 22 | In this sense, the recent publication of the NIST "[Application Container Security Guide](https://doi.org/10.6028/NIST.SP.800-190)" suggests that "_An image should only include the executables and libraries required by the app itself; all other OS functionality is provided by the OS kernel within the underlying host OS_". 23 | 24 | ## 2. Installation 25 | 26 | ### 2.1 From packages 27 | 28 | You can get the proper package (.deb o .rpm) from the [Releases page](https://github.com/grycap/minicon/releases) and install it using the appropriate package manager. 29 | 30 | **Ubuntu/Debian** 31 | 32 | ```bash 33 | $ apt update 34 | $ apt install ./minicon-1.2-1.deb 35 | ``` 36 | 37 | **CentOS/Fedora/RedHat** 38 | 39 | ```bash 40 | $ yum install epel-release 41 | $ yum install ./minicon-1.2-1.noarch.rpm 42 | ``` 43 | 44 | ### 2.2 From sources 45 | 46 | **minicon** is a bash script that tries to analize an application (or a set of applications) using other tools such as _ldd_ or _strace_. So you just simply need to have a working linux with bash installed and get the code: 47 | 48 | ```bash 49 | $ git clone https://github.com/grycap/minicon 50 | ``` 51 | 52 | In that folder you'll have the **minicon** application. I would suggest to put it in the _/opt_ folder. Otherwise leave it in a folder of your choice: 53 | 54 | ```bash 55 | $ mv minicon /opt 56 | ``` 57 | 58 | ### 2.2.1 Dependencies 59 | 60 | **minicon** depends on the commands _ldd_, _file_, _strace_, _rsync_ and _tar_. So, you need to install the proper packages in your system. 61 | 62 | **Ubuntu** 63 | 64 | ```bash 65 | $ apt-get install libc-bin tar file strace rsync 66 | ``` 67 | 68 | **CentOS** 69 | ```bash 70 | $ yum install glibc-common tar file strace rsync which 71 | ``` 72 | ## 3. Usage 73 | 74 | **minicon** has a lot of options. You are advised to run ```./minicon --help``` to get the latest information about the usage of the application. 75 | 76 | The basic syntax is 77 | 78 | ```bash 79 | $ ./minicon