├── .github ├── AUTHORS.md ├── CODE_OF_CONDUCT.md ├── CONTRIBUTING.md ├── FUNDING.yml ├── ISSUE_TEMPLATE │ ├── bug_report.md │ ├── feature_request.md │ └── question-or-documentation-request.md └── workflows │ └── build-containers.yaml ├── CHANGELOG.md ├── Dockerfile ├── LICENSE ├── README.md ├── VERSION ├── addLabel.py ├── docker2singularity.sh ├── img └── logo.png ├── scripts ├── actions │ ├── exec │ ├── run │ ├── shell │ ├── start │ └── test └── env │ ├── 01-base.sh │ ├── 90-environment.sh │ ├── 94-appsbase.sh │ ├── 95-apps.sh │ ├── 99-base.sh │ └── 99-runtimevars.sh └── test.sh /.github/AUTHORS.md: -------------------------------------------------------------------------------- 1 | ## Maintainers 2 | 3 | - [@chrisfilo](https://www.github.com/chrisfilo) 4 | - [@vsoch](https://www.github.com/vsoch) 5 | 6 | ## Contributors 7 | 8 | - [@pvanheus](https://www.github.com/pvanheus) 9 | - [@kaczmarj](https://www.github.com/kaczmarj) 10 | - [@alanhoyle](https://www.github.com/alanhoyle) 11 | - [@justbennet](https://www.github.com/justbennet) 12 | - [@pditommaso](https://www.github.com/pditommaso) 13 | - [@jdidion](https://www.github.com/jdidion) 14 | - [@alaindomissy](https://www.github.com/alaindomissy) 15 | -------------------------------------------------------------------------------- /.github/CODE_OF_CONDUCT.md: -------------------------------------------------------------------------------- 1 | # Contributor Covenant Code of Conduct 2 | 3 | ## Our Pledge 4 | 5 | In the interest of fostering an open and welcoming environment, we as 6 | contributors and maintainers pledge to making participation in our project and 7 | our community a harassment-free experience for everyone, regardless of age, body 8 | size, disability, ethnicity, sex characteristics, gender identity and expression, 9 | level of experience, education, socio-economic status, nationality, personal 10 | appearance, race, religion, or sexual identity and orientation. 11 | 12 | ## Our Standards 13 | 14 | Examples of behavior that contributes to creating a positive environment 15 | include: 16 | 17 | * Using welcoming and inclusive language 18 | * Being respectful of differing viewpoints and experiences 19 | * Gracefully accepting constructive criticism 20 | * Focusing on what is best for the community 21 | * Showing empathy towards other community members 22 | 23 | Examples of unacceptable behavior by participants include: 24 | 25 | * The use of sexualized language or imagery and unwelcome sexual attention or 26 | advances 27 | * Trolling, insulting/derogatory comments, and personal or political attacks 28 | * Public or private harassment 29 | * Publishing others' private information, such as a physical or electronic 30 | address, without explicit permission 31 | * Other conduct which could reasonably be considered inappropriate in a 32 | professional setting 33 | 34 | ## Our Responsibilities 35 | 36 | Project maintainers are responsible for clarifying the standards of acceptable 37 | behavior and are expected to take appropriate and fair corrective action in 38 | response to any instances of unacceptable behavior. 39 | 40 | Project maintainers have the right and responsibility to remove, edit, or 41 | reject comments, commits, code, wiki edits, issues, and other contributions 42 | that are not aligned to this Code of Conduct, or to ban temporarily or 43 | permanently any contributor for other behaviors that they deem inappropriate, 44 | threatening, offensive, or harmful. 45 | 46 | ## Scope 47 | 48 | This Code of Conduct applies both within project spaces and in public spaces 49 | when an individual is representing the project or its community. Examples of 50 | representing a project or community include using an official project e-mail 51 | address, posting via an official social media account, or acting as an appointed 52 | representative at an online or offline event. Representation of a project may be 53 | further defined and clarified by project maintainers. 54 | 55 | ## Enforcement 56 | 57 | Instances of abusive, harassing, or otherwise unacceptable behavior may be 58 | reported by contacting the project team leader @vsoch. All 59 | complaints will be reviewed and investigated and will result in a response that 60 | is deemed necessary and appropriate to the circumstances. The project team is 61 | obligated to maintain confidentiality with regard to the reporter of an incident. 62 | Further details of specific enforcement policies may be posted separately. 63 | 64 | Project maintainers who do not follow or enforce the Code of Conduct in good 65 | faith may face temporary or permanent repercussions as determined by other 66 | members of the project's leadership. 67 | 68 | ## Attribution 69 | 70 | This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 1.4, 71 | available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html 72 | 73 | [homepage]: https://www.contributor-covenant.org 74 | -------------------------------------------------------------------------------- /.github/CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | # Contributor's Agreement 2 | 3 | This code is licensed under an open source MIT [LICENSE](../LICENSE). 4 | You should abide by the terms of the license if you use this software 5 | or include it with your own work. 6 | 7 | # Contributing 8 | 9 | When contributing to Singularity Registry Global Client, it is 10 | important to properly communicate the gist of the contribution. 11 | If it is a simple code or editorial fix, simply explaining this 12 | within the GitHub Pull Request (PR) will suffice. But if this is a larger 13 | fix or Enhancement, it should be first discussed with the project 14 | leader or developers. 15 | 16 | Please note we have a code of conduct, described below. Please follow it in 17 | all your interactions with the project members and users. 18 | 19 | ## Pull Request Process 20 | 21 | 1. Bug fix PRs should be sent to both the master, from a forked branch for the feature. 22 | 2. Follow the existing code style precedent. No tabs. 23 | 3. Test your PR locally, and provide the steps necessary to test for the 24 | reviewers. 25 | 4. The project's default copyright and header have been included in any new 26 | source files. 27 | 5. All (major) changes must be documented, and (if relevant) tested 28 | 6. If necessary, update the README.md. 29 | 7. The pull request will be reviewed by others, and the final merge must be 30 | done by the Singularity project lead, @vsoch (or approved by her). 31 | -------------------------------------------------------------------------------- /.github/FUNDING.yml: -------------------------------------------------------------------------------- 1 | github: vsoch 2 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/bug_report.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Bug report :bug: 3 | about: Create a report to help us improve 4 | --- 5 | 6 | **Describe the bug** 7 | A clear and concise description of what the bug is. 8 | 9 | **To Reproduce** 10 | Steps to reproduce the behavior: 11 | 12 | **Expected behavior** 13 | A clear and concise description of what you expected to happen. 14 | 15 | If applicable, please add any screenshots or operating system / runtime details. 16 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/feature_request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Feature request :sparkles: 3 | about: Suggest an idea for this project 4 | --- 5 | 6 | **Is your feature request related to a problem? Please describe.** 7 | A clear and concise description of what the problem is. Ex. I'm always frustrated when [...] 8 | 9 | **Describe the solution you'd like** 10 | A clear and concise description of what you want to happen. 11 | 12 | **Describe alternatives you've considered** 13 | A clear and concise description of any alternative solutions or features you've considered. 14 | 15 | **More notes?** 16 | Add any other context or screenshots about the feature request here. 17 | -------------------------------------------------------------------------------- /.github/ISSUE_TEMPLATE/question-or-documentation-request.md: -------------------------------------------------------------------------------- 1 | --- 2 | name: Question or Documentation Request 3 | about: Ask a question, or prompt an update of the docs 4 | --- 5 | 6 | Hi friend, what is your question? 7 | 8 | 9 | -------------------------------------------------------------------------------- /.github/workflows/build-containers.yaml: -------------------------------------------------------------------------------- 1 | name: Build & Deploy Docker Containers 2 | on: 3 | # Build on push to any branch 4 | push: [] 5 | 6 | jobs: 7 | deploy-images: 8 | runs-on: ubuntu-latest 9 | strategy: 10 | fail-fast: false 11 | # Just build primary Dockerfile, tag, arm and amd64 for now 12 | matrix: 13 | dockerfile: [[Dockerfile, linux/amd64, '']] 14 | 15 | name: Build ${{ matrix.dockerfile[0] }} 16 | steps: 17 | - name: Checkout 18 | uses: actions/checkout@v3 19 | 20 | - name: Set Container Tag 21 | run: | 22 | branch=$(echo ${GITHUB_REF##*/}) 23 | printf "Branch is ${branch}\n" 24 | tagged="quay.io/singularity/docker2singularity:${branch}${{ matrix.dockerfile[2] }}" 25 | if [[ "${branch}" == "master" ]]; then 26 | container="quay.io/singularity/docker2singularity:latest${{ matrix.dockerfile[2] }}" 27 | else 28 | container=${tagged} 29 | fi 30 | echo "container=${container}" >> $GITHUB_ENV 31 | echo "branch=${branch}" >> $GITHUB_ENV 32 | echo "tagged=${tagged}" >> $GITHUB_ENV 33 | 34 | - name: Log in to Quay Container Registry 35 | if: github.event_name == 'push' 36 | uses: docker/login-action@v2 37 | with: 38 | registry: quay.io 39 | username: ${{ secrets.DOCKER_USER }} 40 | password: ${{ secrets.DOCKER_PASS }} 41 | 42 | - name: Set up QEMU 43 | uses: docker/setup-qemu-action@v2 44 | 45 | - name: Set up Docker Buildx 46 | uses: docker/setup-buildx-action@v2 47 | 48 | - name: Inspect Container Names 49 | run: | 50 | printf "Tagged: ${tagged}\n" 51 | printf "Container: ${container}\n" 52 | 53 | - name: Build 54 | uses: docker/build-push-action@v3 55 | if: github.event_name != 'push' 56 | with: 57 | file: ${{matrix.dockerfile[0]}} 58 | platforms: ${{ matrix.dockerfile[1] }} 59 | push: false 60 | tags: | 61 | ${{ env.container }} 62 | ${{ env.tagged }} 63 | 64 | - name: Build & Deploy 65 | uses: docker/build-push-action@v3 66 | if: github.event_name == 'push' 67 | with: 68 | file: ${{matrix.dockerfile[0]}} 69 | platforms: ${{ matrix.dockerfile[1] }} 70 | push: true 71 | tags: | 72 | ${{ env.container }} 73 | ${{ env.tagged }} 74 | -------------------------------------------------------------------------------- /CHANGELOG.md: -------------------------------------------------------------------------------- 1 | # CHANGELOG 2 | 3 | This is a manually generated log to track changes to the repository for each release. 4 | Each section should include general headers such as **Implemented enhancements** 5 | and **Merged pull requests**. All closed issued and bug fixes should be 6 | represented by the pull requests that fixed them. Critical items to know are: 7 | 8 | - renamed commands 9 | - deprecated / removed commands 10 | - changed defaults 11 | - backward incompatible changes (recipe file format? image file format?) 12 | - migration guidance (how to convert images?) 13 | - changed behaviour (recipe sections work differently) 14 | 15 | This changelog was started with version of Singularity v2.5, and reflects changes since then. 16 | The tags here reflect tags on Docker Hub 17 | 18 | ## [master](https://github.com/singularityhub/docker2singularity/tree/master) (master) 19 | - update of Singularity from [v2.5](https://github.com/singularityware/docker2singularity/tree/v2.5) to [v2.6](https://github.com/singularityware/docker2singularity/tree/v2.6), including adding libarchive dependency, and custom name with -n (v2.6) 20 | - addition of automated builds via CircleCI, and Authors.md (v2.5) 21 | - update of Singularity from [v2.4](https://github.com/singularityware/docker2singularity/tree/v2.4) to [v2.5](https://github.com/singularityware/docker2singularity/tree/v2.5), including adding libarchive dependency, and custom name with -n 22 | -------------------------------------------------------------------------------- /Dockerfile: -------------------------------------------------------------------------------- 1 | FROM golang:1.21.10-alpine AS base 2 | 3 | ################################################################################ 4 | # 5 | # Copyright (C) 2019-2024 Vanessa Sochat. 6 | # 7 | # This program is free software: you can redistribute it and/or modify it 8 | # under the terms of the GNU Affero General Public License as published by 9 | # the Free Software Foundation, either version 3 of the License, or (at your 10 | # option) any later version. 11 | # 12 | # This program is distributed in the hope that it will be useful, but WITHOUT 13 | # ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or 14 | # FITNESS FOR A PARTICULAR PURPOSE. See the GNU Affero General Public 15 | # License for more details. 16 | # 17 | # You should have received a copy of the GNU Affero General Public License 18 | # along with this program. If not, see . 19 | # 20 | ################################################################################ 21 | 22 | FROM docker:24.0.6-git AS builder 23 | COPY --from=base /go /go 24 | COPY --from=base /usr/local/go /usr/local/go 25 | ENV GOPATH=/go 26 | ENV PATH=$GOPATH/bin:/usr/local/go/bin:$PATH 27 | ENV GOLANG_VERSION=1.21.10 28 | 29 | RUN apk update && \ 30 | apk add --virtual .build-deps autoconf automake build-base linux-headers libffi-dev 31 | RUN apk add --no-cache bash git openssh-client-default gcc squashfs-tools sudo libtool gawk ca-certificates libseccomp libseccomp-dev 32 | RUN apk add --no-cache linux-headers build-base openssl-dev util-linux util-linux-dev shadow-uidmap fuse3-dev python3 rsync cryptsetup glib-dev 33 | 34 | ENV SINGULARITY_VERSION=4.1.4 35 | RUN mkdir -p /usr/local/var/singularity/mnt && \ 36 | mkdir -p $GOPATH/src/github.com/sylabs && \ 37 | cd $GOPATH/src/github.com/sylabs && \ 38 | wget -qO- https://github.com/sylabs/singularity/releases/download/v${SINGULARITY_VERSION}/singularity-ce-${SINGULARITY_VERSION}.tar.gz | \ 39 | tar xzv && \ 40 | cd singularity-ce-${SINGULARITY_VERSION} && \ 41 | ./mconfig -p /usr/local/singularity && \ 42 | make -C builddir && \ 43 | make -C builddir install 44 | 45 | # See https://docs.docker.com/develop/develop-images/multistage-build/ 46 | # for more information on multi-stage builds. 47 | 48 | FROM docker:24.0.6-git 49 | LABEL Maintainer=@vsoch 50 | COPY --from=builder /usr/local/singularity /usr/local/singularity 51 | RUN apk add --no-cache ca-certificates libseccomp libseccomp-dev squashfs-tools bash python3 rsync 52 | ENV PATH="/usr/local/singularity/bin:$PATH" 53 | 54 | COPY docker2singularity.sh /docker2singularity.sh 55 | COPY addLabel.py /addLabel.py 56 | COPY scripts /scripts 57 | RUN chmod a+x docker2singularity.sh 58 | ENTRYPOINT ["docker-entrypoint.sh", "/docker2singularity.sh"] 59 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2016-2019 Vanessa Sochat 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # `docker2singularity` 2 | 3 | https://www.sylabs.io/guides/latest/user-guide 4 | 5 | [![CircleCI](https://circleci.com/gh/singularityhub/docker2singularity.svg?style=svg)](https://circleci.com/gh/singularityhub/docker2singularity) 6 | 7 | Are you developing Docker images and you would like to run them on an HPC cluster 8 | supporting [Singularity](https://www.sylabs.io/guides/latest/user-guide/)? 9 | Are you working on Mac or Windows with no easy access to a Linux machine? If the pull, 10 | build, and general commands to work with docker images provided by Singularity 11 | natively do not fit your needs, `docker2singularity` is an alternative way to generate Singularity images. 12 | The containers are available to you on [quay.io](https://quay.io/repository/singularity/docker2singularity), 13 | and older versions also available for you on [Docker Hub](https://hub.docker.com/r/singularityware/docker2singularity/). 14 | 15 | ## Usage 16 | 17 | ```bash 18 | $ docker run quay.io/singularity/docker2singularity 19 | USAGE: docker2singularity [-m "/mount_point1 /mount_point2"] [options] docker_image_name 20 | OPTIONS: 21 | 22 | Image Format 23 | --folder -f build development sandbox (folder) 24 | --option -o add a custom option to build (-o --fakeroot or -option 'section post' ) 25 | --writable -w non-production writable image (ext3) 26 | Default is squashfs (recommended) (deprecated) 27 | --name -n provide basename for the container (default based on URI) 28 | --mount -m provide list of custom mount points (in quotes!) 29 | --help -h show this help and exit 30 | ``` 31 | 32 | ### Options 33 | 34 | **Image Format** 35 | 36 | - `squashfs` (no arguments specified) gives you a squashfs (`*.simg`) image. This is a compressed, reliable, and read only format that is recommended for production images. Squashfs support was added to Singularity proper in [January of 2017](https://github.com/sylabs/singularity/commit/0cf00d1251ff276d5b9b7a0e4eadb783a45a6b65#diff-8405d9d311d83f009adff55c3deb112c) and thus available as early as the 2.2.1 release. 37 | - `sandbox` (`-f`) builds your image into a sandbox **folder**. This is ideal for development, as it will produce a working image in a folder on your system. 38 | - `ext3` (`-w`) builds an older format (ext3) image (`*.img`). This format is not recommended for production images as we have observed degradation of the images over time, and they tend to be upwards of 1.5x to 2x the size of squashfs. 39 | 40 | Note that you are able to convert easily from a folder or ext3 image using Singularity 2.4. If your choice is to develop, making changes, and then finalize, this approach is **not** recommended - your changes are not recorded and thus the image not reproducible. 41 | 42 | **Mount Points** 43 | 44 | - `-m` specify one or more mount points to create in the image. 45 | 46 | **Options** 47 | 48 | If you look at `singularity build --help` there are a variety of options available. 49 | You can specify some custom option to the command using the `--option` flag. Make sure 50 | that each option that you specify is captured as a single string. E.g.,: 51 | 52 | ```bash 53 | --option --fakeroot 54 | --option '--section post' 55 | ``` 56 | 57 | **Image Name** 58 | 59 | The last argument (without a letter) is the name of the docker image, as you would specify to run with Docker (e.g., `docker run ubuntu:latest`) 60 | 61 | 62 | ## Legacy 63 | 64 | If you want a legacy version, see the [repository branches](https://github.com/singularityhub/docker2singularity/branches) 65 | and [tag history](https://quay.io/repository/singularity/docker2singularity?tab=tags&tag=latest) on the registry. 66 | 67 | Containers were previous built on [Docker Hub](https://hub.docker.com/r/singularityware/docker2singularity/tags/) and 68 | now are provided on [quay.io](https://quay.io/repository/singularity/docker2singularity?tab=tags). A tag with prefix `v` corresponds to a release of the Singularity software, while the others are in reference to releases of Docker. Previously used [scripts](https://github.com/singularityhub/docker2singularity/tree/master/scripts), including environment and action files, are provided in this repository for reference. 69 | 70 | ## Requirements 71 | 72 | - Docker (native Linux or Docker for Mac or Docker for Windows) - to create the Singularity image. 73 | - Singularity >= 2.1 - to run the Singularity image (**versions 2.0 and older are not supported!**). Note that if running a 2.4 image using earlier versions, not all (later developed) features may be available. 74 | 75 | ## Examples 76 | 77 | ### Build a Squashfs Image 78 | 79 | Squashfs is the recommended image type, it is compressed and less prone to degradation over time. You don't need to specify anything special to create it: 80 | 81 | This is a path on my host, the image will be written here 82 | 83 | ```bash 84 | $ mkdir -p /tmp/test 85 | ``` 86 | 87 | And here is the command to run. Notice that I am mounting the path `/tmp/test` that I created above to `/output` in the container, where the container image will be written (and seen on my host). 88 | 89 | ```bash 90 | $ docker run -v /var/run/docker.sock:/var/run/docker.sock \ 91 | -v /tmp/test:/output \ 92 | --privileged -t --rm \ 93 | quay.io/singularity/docker2singularity \ 94 | ubuntu:14.04 95 | 96 | Image Format: squashfs 97 | Inspected Size: 188 MB 98 | 99 | (1/10) Creating a build sandbox... 100 | (2/10) Exporting filesystem... 101 | (3/10) Creating labels... 102 | (4/10) Adding run script... 103 | (5/10) Setting ENV variables... 104 | (6/10) Adding mount points... 105 | (7/10) Fixing permissions... 106 | (8/10) Stopping and removing the container... 107 | (9/10) Building squashfs container... 108 | Building image from sandbox: /tmp/ubuntu_14.04-2017-09-13-3e51deeadc7b.build 109 | Building Singularity image... 110 | Singularity container built: /tmp/ubuntu_14.04-2017-09-13-3e51deeadc7b.simg 111 | Cleaning up... 112 | (10/10) Moving the image to the output folder... 113 | 62,591,007 100% 340.92MB/s 0:00:00 (xfr#1, to-chk=0/1) 114 | Final Size: 60MB 115 | ``` 116 | 117 | We can now see the finished image! 118 | 119 | ```bash 120 | $ ls /tmp/test 121 | ubuntu_14.04-2018-04-27-c7e04ea7fa32.simg 122 | ``` 123 | 124 | And use it! 125 | 126 | ```bash 127 | $ singularity shell /tmp/test/ubuntu_14.04-2018-04-27-c7e04ea7fa32.simg 128 | Singularity: Invoking an interactive shell within container... 129 | 130 | Singularity ubuntu_14.04-2018-04-27-c7e04ea7fa32.simg:~/Documents/Dropbox/Code/singularity/docker2singularity> 131 | ``` 132 | 133 | Take a look again at the generation code above, and notice how the image went from 188MB to 60MB? 134 | This is one of the great things about the squashfs filesystem! This reduction is even more impressive when we are dealing with very large images (e.g., ~3600 down to ~1800). A few notes on the inputs shown above that you should edit: 135 | 136 | - `/tmp/test`: the path you want to have the final image reside. If you are on windows this might look like `D:\host\path\where\to\output\singularity\image`. 137 | -`ubuntu:14.04`: the docker image name you wish to convert (it will be pulled from Docker Hub if it does not exist on your host system). 138 | 139 | `docker2singularity` uses the Docker daemon located on the host system. It will access the Docker image cache from the host system avoiding having to redownload images that are already present locally. 140 | 141 | 142 | If you ever need to make changes, you can easily export the squashfs image into either a sandbox folder or ext3 (legacy) image, both of which have writable. 143 | 144 | ``` 145 | sudo singularity build --sandbox sandbox/ production.simg 146 | sudo singularity build --writable ext3.img production.simg 147 | ``` 148 | 149 | ### Custom Naming 150 | 151 | Added for version 2.5.1, you can specify the name of your container with the `-n/--name` argument, as follows: 152 | 153 | ```bash 154 | docker run -v /var/run/docker.sock:/var/run/docker.sock \ 155 | -v /tmp/test:/output \ 156 | --privileged -t --rm \ 157 | quay.io/singularity/docker2singularity \ 158 | --name meatballs ubuntu:14.04 159 | 160 | ... 161 | 162 | $ ls /tmp/test/ 163 | meatballs.simg 164 | ``` 165 | 166 | ### Inspect Your Image 167 | New with `docker2singularity` 2.4, the labels for the container are available with `inspect`: 168 | 169 | ```bash 170 | singularity inspect ubuntu_14.04-2017-09-13-3e51deeadc7b.simg 171 | { 172 | "org.label-schema.singularity.build": "squashfs", 173 | "org.label-schema.docker.version": "17.06.2-ce", 174 | "org.label-schema.schema-version": "1.0", 175 | "org.label-schema.singularity.build-type": "docker2singularity", 176 | "org.label-schema.docker.id": "sha256:dea1945146b96542e6e20642830c78df702d524a113605a906397db1db022703", 177 | "org.label-schema.build-date": "2017-10-28-17:19:18", 178 | "org.label-schema.singularity.version": "2.4-dist", 179 | "org.label-schema.docker.created": "2017-09-13" 180 | } 181 | ``` 182 | 183 | as is the runscript and environment 184 | 185 | ```bash 186 | singularity inspect --json -e -r ubuntu_14.04-2017-09-13-3e51deeadc7b.simg 187 | { 188 | "data": { 189 | "attributes": { 190 | "environment": "# Custom environment shell code should follow\n\n", 191 | "runscript": "#!/bin/sh\n/bin/bash $@\n" 192 | }, 193 | "type": "container" 194 | } 195 | } 196 | 197 | ``` 198 | 199 | ### Build a Sandbox Image 200 | A sandbox image is a folder that is ideal for development. You can view it on your desktop, cd inside and browse, and it works like a Singularity image. To create a sandbox, specify the `-f` flag: 201 | 202 | ```bash 203 | docker run -v /var/run/docker.sock:/var/run/docker.sock \ 204 | -v /host/path/change/me:/output \ 205 | --privileged -t --rm \ 206 | quay.io/singularity/docker2singularity \ 207 | -f \ 208 | ubuntu:14.04 209 | ``` 210 | Importantly, you can use `--writable`, and if needed, you can convert a sandbox folder into a production image: 211 | 212 | ```bash 213 | sudo singularity build sandbox/ production.simg 214 | ``` 215 | 216 | ### Build a Legacy (ext3) Image 217 | You can build a legacy ext3 image (with `--writable`) with the `-w` flag. This is an older image format that is more prone to degradation over time, and (building) may not be supported for future versions of the software. 218 | 219 | ```bash 220 | docker run -v /var/run/docker.sock:/var/run/docker.sock \ 221 | -v /host/path/change/me:/output \ 222 | --privileged -t --rm \ 223 | quay.io/singularity/docker2singularity \ 224 | -w \ 225 | ubuntu:14.04 226 | ``` 227 | You can also use `--writable` and convert an ext3 image into a production image: 228 | 229 | ```bash 230 | sudo singularity build ext3.img production.simg 231 | ``` 232 | 233 | ### Contributed Examples 234 | 235 | The following are a list of brief examples and tutorials generated by the Singularity community for using **docker2singularity**. If you have an example of your own, please [let us know](https://www.github.com/singularityhub/docker2singularity/issues)! 236 | 237 | - [docker2singularity-demo](https://github.com/stevekm/docker2singularity-demo): an example of using docker2singularity on MacOS and using Vagrant to test the output Singularity image, complete with notes and a nice Makefile. 238 | 239 | 240 | ### Tips for making Docker images compatible with Singularity 241 | 242 | - Define all environmental variables using the `ENV` instruction set. Do not rely on `.bashrc`, `.profile`, etc. 243 | - Define an `ENTRYPOINT` instruction set pointing to the command line interface to your pipeline 244 | - Do not define `CMD` - rely only on `ENTRYPOINT` 245 | - You can interactively test the software inside the container by overriding the `ENTRYPOINT` 246 | `docker run -i -t --entrypoint /bin/bash bids/example` 247 | - Do not rely on being able to write anywhere other than the home folder and `/scratch`. Make sure your container runs with the `--read-only --tmpfs /run --tmpfs /tmp` parameters (this emulates the read-only behavior of Singularity) 248 | - Don’t rely on having elevated user permissions 249 | - Don’t use the USER instruction set 250 | 251 | ## FAQ 252 | Here are some frequently asked questions if you run into trouble! 253 | 254 | ### "client is newer than server" error 255 | If you are getting the following error: 256 | `docker: Error response from daemon: client is newer than server` 257 | 258 | You need to use the `docker info` command to check your docker version and use it to grab the correct corresponding version of `docker2singularity`. For example: 259 | 260 | ```bash 261 | docker run \ 262 | -v /var/run/docker.sock:/var/run/docker.sock \ 263 | -v D:\host\path\where\to\output\singularity\image:/output \ 264 | --privileged -t --rm \ 265 | singularityware/docker2singularity:1.11 \ 266 | ubuntu:14.04 267 | ``` 268 | 269 | Currently only the 1.10, 1.11, 1.12, and 1.13 versions are supported. If you are using an older version of Docker you will need to upgrade. 270 | 271 | 272 | ### My cluster/HPC requires Singularity images to include specific mount points 273 | If you are getting `WARNING: Non existant bind point (directory) in container: '/shared_fs'` or a similar error when running your Singularity image that means that your Singularity images require custom mount points. To make the error go away you can specify the mount points required by your system when creating the Singularity image: 274 | 275 | ```bash 276 | docker run \ 277 | -v /var/run/docker.sock:/var/run/docker.sock \ 278 | -v D:\host\path\where\to\output\singularity\image:/output \ 279 | --privileged -t --rm \ 280 | quay.io/singularity/docker2singularity \ 281 | -m "/shared_fs /custom_mountpoint2" \ 282 | ubuntu:14.04 283 | ``` 284 | 285 | ## Development 286 | 287 | ### 1. Build the container 288 | 289 | You can build a development container as follows. First, update the [VERSION](VERSION) 290 | to be correct. 291 | 292 | ```bash 293 | VERSION=$(cat VERSION) 294 | image="quay.io/singularity/docker2singularity:${VERSION}" 295 | docker build -t ${image} . 296 | ``` 297 | 298 | ### 2. Test the container 299 | 300 | We have a [Circle CI](.circleci/config.yml) builder that tests generation of the final 301 | image, and basic running to ensure the entrypoint is functioning. Since we cannot run 302 | the priviledged Docker daemon on Circle, a [test.sh](test.sh) script is provided for local testing. 303 | 304 | ```bash 305 | chmod u+x 306 | /bin/bash test.sh 307 | ``` 308 | 309 | If there are missing tests or you have added new features, please add the test here! 310 | 311 | ### 3. Documentation 312 | 313 | If you have added new features, please describe usage in the [README.md](README.md) here. 314 | Don't forget to read the [CONTRIBUTING.md](.github/CONTRIBUTING.md) along with the 315 | [code of conduct](.github/CODE_OF_CONDUCT.md) and add yourself to the [authors file](.github/AUTHORS.md). 316 | 317 | ## Acknowledgements 318 | 319 | This work is heavily based on the `docker2singularity` work done by [vsoch](https://github.com/vsoch) 320 | and [gmkurtzer](https://github.com/gmkurtzer). The original record of the work can be read about 321 | in [this commit](https://github.com/singularityhub/docker2singularity/commit/d174cadefd90f77f302f4bef5a8cd089eb2da2e4). 322 | Thank you kindly to all the [contributors](.github/AUTHORS.md), and please open an issue if you need help. 323 | -------------------------------------------------------------------------------- /VERSION: -------------------------------------------------------------------------------- 1 | 4.0.1 2 | -------------------------------------------------------------------------------- /addLabel.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | 3 | ''' 4 | 5 | add.py: wrapper for "add" of a key to a json file for 6 | Singularity Hub command line tool. 7 | 8 | This function takes input arguments of the following: 9 | 10 | --key: should be the key to lookup from the json file 11 | --value: the value to add to the key 12 | --file: should be the json file to read 13 | 14 | Copyright (c) 2019, Vanessa Sochat. All rights reserved. 15 | 16 | ''' 17 | 18 | import sys 19 | import os 20 | import json 21 | import argparse 22 | 23 | def get_parser(): 24 | 25 | parser = argparse.ArgumentParser(description="GET key from json") 26 | 27 | parser.add_argument("--key", 28 | dest='key', 29 | help="key to add to json", 30 | type=str, 31 | default=None) 32 | 33 | parser.add_argument("--value", 34 | dest='value', 35 | help="value to add to the json", 36 | type=str, 37 | default=None) 38 | 39 | parser.add_argument("--file", 40 | dest='file', 41 | help="Path to json file to add to", 42 | type=str, 43 | default=None) 44 | 45 | parser.add_argument('-f', dest="force", 46 | help="force add (overwrite if exists)", 47 | default=False, action='store_true') 48 | 49 | parser.add_argument('--quiet', dest="quiet", 50 | help="do not display debug", 51 | default=False, action='store_true') 52 | 53 | return parser 54 | 55 | 56 | def write_json(json_obj, filename, mode="w", print_pretty=True): 57 | '''write_json will (optionally,pretty print) a json object to file 58 | ''' 59 | bot.verbose2("Writing json file %s with mode %s." % (filename, mode)) 60 | with open(filename, mode) as filey: 61 | if print_pretty is True: 62 | filey.writelines(print_json(json_obj)) 63 | else: 64 | filey.writelines(json.dumps(json_obj)) 65 | return filename 66 | 67 | 68 | def read_json(filename, mode='r'): 69 | '''read_json reads in a json file and returns 70 | the data structure as dict. 71 | ''' 72 | with open(filename, mode) as filey: 73 | data = json.load(filey) 74 | return data 75 | 76 | 77 | def ADD(key, value, jsonfile, force=False, quiet=False): 78 | '''ADD will write or update a key in a json file 79 | ''' 80 | 81 | # Check that key is not empty 82 | if key.strip() in ['#', '', None]: 83 | bot.verbose('Empty key %s, skipping' % key) 84 | sys.exit(0) 85 | 86 | key = format_keyname(key) 87 | print("Adding label: '%s' = '%s'" % (key, value)) 88 | print("ADD %s from %s" % (key, jsonfile)) 89 | 90 | if os.path.exists(jsonfile): 91 | contents = read_json(jsonfile) 92 | if key in contents: 93 | msg = 'Warning, %s is already set. ' % key 94 | msg += 'Overwrite is set to %s' % force 95 | if not quiet: 96 | print(msg) 97 | if force is True: 98 | contents[key] = value 99 | else: 100 | msg = '%s found in %s ' % (key, jsonfile) 101 | msg += 'and overwrite set to %s.' % force 102 | print(msg) 103 | sys.exit(1) 104 | else: 105 | contents[key] = value 106 | else: 107 | contents = {key: value} 108 | 109 | print('%s is %s' % (key, value)) 110 | write_json(contents, jsonfile) 111 | return value 112 | 113 | 114 | def main(): 115 | 116 | parser = get_parser() 117 | 118 | try: 119 | (args, options) = parser.parse_args() 120 | except Exception: 121 | sys.exit(0) 122 | 123 | if args.key is not None and args.file is not None: 124 | if args.value is not None: 125 | 126 | value = ADD(key=args.key, 127 | value=args.value, 128 | jsonfile=args.file, 129 | force=args.force, 130 | quiet=args.quiet) 131 | 132 | else: 133 | bot.error("--key and --file and --value must be defined for ADD.") 134 | sys.exit(1) 135 | 136 | 137 | if __name__ == '__main__': 138 | main() 139 | -------------------------------------------------------------------------------- /docker2singularity.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | # 3 | # docker2singularity.sh will convert a docker image into a singularity 4 | # Must be run with sudo to use docker commands (eg aufs) 5 | # 6 | # NOTES: 7 | # If the docker image uses both ENTRYPOINT and CMD the latter will be ignored 8 | # 9 | # KNOWN ISSUES: 10 | # Currently ENTRYPOINTs and CMDs with commas in the arguments are not supported 11 | # 12 | # USAGE: docker2singularity.sh ubuntu:14.04 13 | # 14 | # 15 | # Copyright (c) 2016-2024 Vanessa Sochat, All Rights Reserved 16 | # Copyright (c) 2017 Singularityware LLC and AUTHORS 17 | # 18 | # Permission is hereby granted, free of charge, to any person obtaining a copy 19 | # of this software and associated documentation files (the "Software"), to deal 20 | # in the Software without restriction, including without limitation the rights 21 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 22 | # copies of the Software, and to permit persons to whom the Software is 23 | # furnished to do so, subject to the following conditions: 24 | # 25 | # The above copyright notice and this permission notice shall be included in all 26 | # copies or substantial portions of the Software. 27 | # 28 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 29 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 30 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 31 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 32 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 33 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 34 | # SOFTWARE. 35 | 36 | set -o errexit 37 | set -o nounset 38 | 39 | function usage() { 40 | 41 | echo "USAGE: docker2singularity [-m \"/mount_point1 /mount_point2\"] [options] docker_image_name" 42 | echo "" 43 | echo "OPTIONS: 44 | 45 | Image Format 46 | --folder -f build development sandbox (folder) 47 | --option -o add a custom option to build (-o --fakeroot or -option 'section post' ) 48 | --writable -w non-production writable image (ext3) 49 | Default is squashfs (recommended) (deprecated) 50 | --name -n provide basename for the container (default based on URI) 51 | --mount -m provide list of custom mount points (in quotes!) 52 | --help -h show this help and exit 53 | " 54 | } 55 | 56 | # --- Option processing -------------------------------------------- 57 | if [ $# == 0 ] ; then 58 | usage 59 | exit 0; 60 | fi 61 | 62 | mount_points="/oasis /projects /scratch /local-scratch /work /home1 /corral-repl /corral-tacc /beegfs /share/PI /extra /data /oak" 63 | image_format="squashfs" 64 | new_container_name="" 65 | options="" 66 | 67 | while true; do 68 | case ${1:-} in 69 | -h|--help|help) 70 | usage 71 | exit 0 72 | ;; 73 | -n|--name) 74 | shift 75 | new_container_name="${1:-}" 76 | shift 77 | ;; 78 | -m|--mount) 79 | shift 80 | mount_points="${1:-}" 81 | shift 82 | ;; 83 | -o|--option) 84 | shift 85 | options="${1:-} ${options}" 86 | shift 87 | ;; 88 | -f|--folder) 89 | shift 90 | image_format="sandbox" 91 | ;; 92 | -w|--writable) 93 | shift 94 | image_format="writable" 95 | ;; 96 | :) printf "missing argument for -%s\n" "$option" >&2 97 | usage 98 | exit 1 99 | ;; 100 | \?) printf "illegal option: -%s\n" "$option" >&2 101 | usage 102 | exit 1 103 | ;; 104 | -*) 105 | printf "illegal option: -%s\n" "$option" >&2 106 | usage 107 | exit 1 108 | ;; 109 | *) 110 | break; 111 | ;; 112 | esac 113 | done 114 | 115 | image=${1} 116 | 117 | echo "" 118 | echo "Image Format: ${image_format}" 119 | echo "Docker Image: ${image}" 120 | 121 | if [ "${new_container_name}" != "" ]; then 122 | echo "Container Name: ${new_container_name}" 123 | fi 124 | 125 | echo "" 126 | 127 | ################################################################################ 128 | ### CONTAINER RUNNING ID ####################################################### 129 | ################################################################################ 130 | 131 | runningid=`docker run -d $image tail -f /dev/null` 132 | 133 | # Full id looks like 134 | # sha256:d59bdb51bb5c4fb7b2c8d90ae445e0720c169c553bcf553f67cb9dd208a4ec15 135 | 136 | # Take the first 12 characters to get id of container 137 | container_id=`echo ${runningid} | cut -c1-12` 138 | 139 | # Network address, if needed 140 | network_address=`docker inspect --format="{{.NetworkSettings.IPAddress}}" $container_id` 141 | 142 | 143 | ################################################################################ 144 | ### IMAGE NAME ################################################################# 145 | ################################################################################ 146 | 147 | image_name=`docker inspect --format="{{.Config.Image}}" $container_id` 148 | 149 | # using bash substitution 150 | # removing special chars [perhaps echo + sed would be better for other chars] 151 | image_name=${image_name//\//_} 152 | image_name=${image_name/:/_} 153 | 154 | # following is the date of the container, not the docker image. 155 | #creation_date=`docker inspect --format="{{.Created}}" $container_id` 156 | creation_date=`docker inspect --format="{{.Created}}" $image` 157 | 158 | 159 | ################################################################################ 160 | ### IMAGE SIZE ################################################################# 161 | ################################################################################ 162 | 163 | size=`docker inspect --format="{{.Size}}" $image` 164 | # convert size in MB 165 | size=`echo $(($size/1000000+1))` 166 | echo "Inspected Size: $size MB" 167 | echo "" 168 | 169 | ################################################################################ 170 | ### IMAGE CREATION ############################################################# 171 | ################################################################################ 172 | TMPDIR=$(mktemp -u -d) 173 | mkdir -p $TMPDIR 174 | 175 | creation_date=`echo ${creation_date} | cut -c1-10` 176 | 177 | # The user has not provided a custom name 178 | if [ "${new_container_name}" == "" ]; then 179 | new_container_name=/tmp/$image_name-$creation_date-$container_id 180 | 181 | # The user has provided a custom name 182 | else 183 | new_container_name=/tmp/$(basename $new_container_name) 184 | new_container_name="${new_container_name%.*}" 185 | fi 186 | 187 | 188 | build_sandbox="${new_container_name}.build" 189 | 190 | 191 | echo "(1/10) Creating a build sandbox..." 192 | mkdir -p ${build_sandbox} 193 | echo "(2/10) Exporting filesystem..." 194 | docker export $container_id >> $build_sandbox.tar 195 | tar -C $build_sandbox -xf $build_sandbox.tar 196 | docker inspect $container_id >> $build_sandbox/singularity.json 197 | 198 | 199 | ################################################################################ 200 | ### METADATA ################################################################### 201 | ################################################################################ 202 | 203 | # Quiet Singularity debug output when adding labels 204 | SINGULARITY_MESSAGELEVEL=0 205 | export SINGULARITY_MESSAGELEVEL 206 | 207 | # For docker2singularity, installation is at /usr/local 208 | echo "(3/10) Creating labels..." 209 | LABELS=$(docker inspect --format='{{json .Config.Labels}}' $image) 210 | LABELFILE=$(printf "%q" "$build_sandbox/.singularity.d/labels.json") 211 | ADD_LABEL="/addLabel.py -f --file ${LABELFILE}" 212 | 213 | # Labels could be null 214 | if [ "${LABELS}" == "null" ]; then 215 | LABELS="{}" 216 | fi 217 | 218 | mkdir -p $build_sandbox/.singularity.d 219 | touch ${LABELFILE} 220 | 221 | # Extract some other "nice to know" metadata from docker 222 | SINGULARITY_version=`singularity --version` 223 | SINGULARITY_VERSION=$(printf "%q" "$SINGULARITY_version") 224 | DOCKER_VERSION=$(docker inspect --format='{{json .DockerVersion}}' $image) 225 | DOCKER_ID=$(docker inspect --format='{{json .Id}}' $image) 226 | 227 | # Add labels from Docker, then relevant to Singularity build 228 | echo $LABELS > $LABELFILE; 229 | eval $ADD_LABEL --key "org.label-schema.schema-version" --value "1.0" 230 | eval $ADD_LABEL --key "org.label-schema.singularity.build-type" --value "docker2singularity" 231 | eval $ADD_LABEL --key "org.label-schema.singularity.build" --value "${image_format}" 232 | eval $ADD_LABEL --key "org.label-schema.build-date" --value $(date +%Y-%m-%d-%H:%M:%S) 233 | eval $ADD_LABEL --key "org.label-schema.singularity.version" --value "${SINGULARITY_VERSION}" 234 | eval $ADD_LABEL --key "org.label-schema.docker.version" --value "${DOCKER_VERSION}" 235 | eval $ADD_LABEL --key "org.label-schema.docker.Created" --value "${creation_date}" 236 | eval $ADD_LABEL --key "org.label-schema.docker.Id" --value "${DOCKER_ID}" 237 | 238 | unset SINGULARITY_MESSAGELEVEL 239 | 240 | ################################################################################ 241 | ### SINGULARITY RUN SCRIPT ##################################################### 242 | ################################################################################ 243 | echo "(4/10) Adding run script..." 244 | 245 | function shell_escape () { 246 | python -c 'import json, pipes, sys; print(" ".join(pipes.quote(a) for a in json.load(sys.stdin) or []))' 247 | } 248 | 249 | CMD=$(docker inspect --format='{{json .Config.Cmd}}' $image | shell_escape) 250 | ENTRYPOINT=$(docker inspect --format='{{json .Config.Entrypoint}}' $image | shell_escape) 251 | 252 | echo '#!/bin/sh' > $build_sandbox/.singularity.d/runscript 253 | 254 | # Take working directory into account 255 | WORKINGDIR=$(docker inspect --format='{{json .Config.WorkingDir}}' $image) 256 | if [[ $WORKINGDIR != '""' ]]; then 257 | echo cd $WORKINGDIR >> $build_sandbox/.singularity.d/runscript 258 | fi 259 | 260 | # First preference goes to both entrypoint / cmd, then individual 261 | if [ -n "$ENTRYPOINT" ] && [ -n "$CMD" ]; then 262 | echo exec "$ENTRYPOINT" "$CMD" '"$@"' >> $build_sandbox/.singularity.d/runscript; 263 | elif [ -n "$ENTRYPOINT" ]; then 264 | echo exec "$ENTRYPOINT" '"$@"' >> $build_sandbox/.singularity.d/runscript; 265 | elif [ -n "$CMD" ]; then 266 | echo exec "$CMD" '"$@"' >> $build_sandbox/.singularity.d/runscript; 267 | fi 268 | 269 | chmod +x $build_sandbox/.singularity.d/runscript; 270 | 271 | ################################################################################ 272 | ### SINGULARITY ENVIRONMENT #################################################### 273 | ################################################################################ 274 | 275 | echo "(5/10) Setting ENV variables..." 276 | 277 | # Removed copying of template files to avoid scripts permissions issue (see #66) 278 | # Create env scripts directory 279 | mkdir -p $build_sandbox/.singularity.d/env 280 | chmod 755 $build_sandbox/.singularity.d/env 281 | 282 | # Then customize by adding Docker variables 283 | docker run --rm --entrypoint="/usr/bin/env" $image > $TMPDIR/docker_environment 284 | # do not include HOME and HOSTNAME - they mess with local config 285 | sed -i '/^HOME/d' $TMPDIR/docker_environment 286 | sed -i '/^HOSTNAME/d' $TMPDIR/docker_environment 287 | sed -i 's/^/export /' $TMPDIR/docker_environment 288 | # add quotes around the variable names 289 | sed -i 's/=/="/' $TMPDIR/docker_environment 290 | sed -i 's/$/"/' $TMPDIR/docker_environment 291 | cp $TMPDIR/docker_environment $build_sandbox/.singularity.d/env/10-docker.sh 292 | chmod +x $build_sandbox/.singularity.d/env/10-docker.sh; 293 | rm -rf $TMPDIR 294 | 295 | ################################################################################ 296 | ### Permissions ################################################################ 297 | ################################################################################ 298 | if [ "${mount_points}" ] ; then 299 | echo "(6/10) Adding mount points..." 300 | for mount_point in ${mount_points}; do 301 | mkdir -p "${build_sandbox}/${mount_point}" 302 | done 303 | else 304 | echo "(6/10) Skipping mount points..." 305 | fi 306 | 307 | 308 | # making sure that any user can read and execute everything in the container 309 | echo "(7/10) Fixing permissions..." 310 | 311 | find ${build_sandbox}/* -maxdepth 0 -not -path '${build_sandbox}/dev*' -not -path '${build_sandbox}/proc*' -not -path '${build_sandbox}/sys*' -exec chmod a+r -R '{}' \; 312 | find ${build_sandbox}/* -type f -or -type d -perm -u+x,o-x -not -path '${build_sandbox}/dev*' -not -path '${build_sandbox}/proc*' -not -path '${build_sandbox}/sys*' -exec chmod a+x '{}' \; 313 | 314 | echo "(8/10) Stopping and removing the container..." 315 | docker stop $container_id >> /dev/null 316 | docker rm $container_id >> /dev/null 317 | 318 | # Build a final image from the sandbox 319 | echo "(9/10) Building ${image_format} container..." 320 | if [ "$image_format" == "squashfs" ]; then 321 | new_container_name=${new_container_name}.sif 322 | singularity build ${options} ${new_container_name} $build_sandbox 323 | elif [ "$image_format" == "writable" ]; then 324 | new_container_name=${new_container_name}.simg 325 | singularity build ${options} --writable ${new_container_name} $build_sandbox 326 | else 327 | mv $build_sandbox $new_container_name 328 | fi 329 | 330 | echo "(10/10) Moving the image to the output folder..." 331 | finalsize=`du -shm $new_container_name | cut -f1` 332 | rsync --info=progress2 -a $new_container_name /output/ 333 | echo "Final Size: ${finalsize}MB" 334 | -------------------------------------------------------------------------------- /img/logo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/singularityhub/docker2singularity/008ec30b3c6f86eff06bc9505136e8b0bb16d729/img/logo.png -------------------------------------------------------------------------------- /scripts/actions/exec: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | for script in /.singularity.d/env/*.sh; do 4 | if [ -f "$script" ]; then 5 | . "$script" 6 | fi 7 | done 8 | 9 | exec "$@" 10 | -------------------------------------------------------------------------------- /scripts/actions/run: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | for script in /.singularity.d/env/*.sh; do 4 | if [ -f "$script" ]; then 5 | . "$script" 6 | fi 7 | done 8 | 9 | if test -n "${SINGULARITY_APPNAME:-}"; then 10 | 11 | if test -x "/scif/apps/${SINGULARITY_APPNAME:-}/scif/runscript"; then 12 | exec "/scif/apps/${SINGULARITY_APPNAME:-}/scif/runscript" "$@" 13 | else 14 | echo "No Singularity runscript for contained app: ${SINGULARITY_APPNAME:-}" 15 | exit 1 16 | fi 17 | 18 | elif test -x "/.singularity.d/runscript"; then 19 | exec "/.singularity.d/runscript" "$@" 20 | else 21 | echo "No Singularity runscript found, executing /bin/sh" 22 | exec /bin/sh "$@" 23 | fi 24 | -------------------------------------------------------------------------------- /scripts/actions/shell: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | for script in /.singularity.d/env/*.sh; do 4 | if [ -f "$script" ]; then 5 | . "$script" 6 | fi 7 | done 8 | 9 | if test -n "$SINGULARITY_SHELL" -a -x "$SINGULARITY_SHELL"; then 10 | exec $SINGULARITY_SHELL "$@" 11 | 12 | echo "ERROR: Failed running shell as defined by '\$SINGULARITY_SHELL'" 1>&2 13 | exit 1 14 | 15 | elif test -x /bin/bash; then 16 | SHELL=/bin/bash 17 | PS1="Singularity $SINGULARITY_NAME:\\w> " 18 | export SHELL PS1 19 | exec /bin/bash --norc "$@" 20 | elif test -x /bin/sh; then 21 | SHELL=/bin/sh 22 | export SHELL 23 | exec /bin/sh "$@" 24 | else 25 | echo "ERROR: /bin/sh does not exist in container" 1>&2 26 | fi 27 | exit 1 28 | -------------------------------------------------------------------------------- /scripts/actions/start: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | for script in /.singularity.d/env/*.sh; do 4 | if [ -f "$script" ]; then 5 | . "$script" 6 | fi 7 | done 8 | 9 | if test -x "/.singularity.d/startscript"; then 10 | exec "/.singularity.d/startscript" "$@" 11 | fi 12 | -------------------------------------------------------------------------------- /scripts/actions/test: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | for script in /.singularity.d/env/*.sh; do 4 | if [ -f "$script" ]; then 5 | . "$script" 6 | fi 7 | done 8 | 9 | 10 | if test -n "${SINGULARITY_APPNAME:-}"; then 11 | 12 | if test -x "/scif/apps/${SINGULARITY_APPNAME:-}/scif/test"; then 13 | exec "/scif/apps/${SINGULARITY_APPNAME:-}/scif/test" "$@" 14 | else 15 | echo "No Singularity tests for contained app: ${SINGULARITY_APPNAME:-}" 16 | exit 1 17 | fi 18 | elif test -x "/.singularity.d/test"; then 19 | exec "/.singularity.d/test" "$@" 20 | else 21 | echo "No Singularity container test found, executing /bin/sh -c true" 22 | exec /bin/sh -c true 23 | fi 24 | -------------------------------------------------------------------------------- /scripts/env/01-base.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. 4 | # 5 | # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. 6 | # 7 | # Copyright (c) 2016-2017, The Regents of the University of California, 8 | # through Lawrence Berkeley National Laboratory (subject to receipt of any 9 | # required approvals from the U.S. Dept. of Energy). All rights reserved. 10 | # 11 | # This software is licensed under a customized 3-clause BSD license. Please 12 | # consult LICENSE.md file distributed with the sources of this project regarding 13 | # your rights to use or distribute this software. 14 | # 15 | # NOTICE. This Software was developed under funding from the U.S. Department of 16 | # Energy and the U.S. Government consequently retains certain rights. As such, 17 | # the U.S. Government has been granted for itself and others acting on its 18 | # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software 19 | # to reproduce, distribute copies to the public, prepare derivative works, and 20 | # perform publicly and display publicly, and to permit other to do so. 21 | # 22 | # 23 | 24 | 25 | -------------------------------------------------------------------------------- /scripts/env/90-environment.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | #Custom environment shell code should follow 3 | 4 | -------------------------------------------------------------------------------- /scripts/env/94-appsbase.sh: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/singularityhub/docker2singularity/008ec30b3c6f86eff06bc9505136e8b0bb16d729/scripts/env/94-appsbase.sh -------------------------------------------------------------------------------- /scripts/env/95-apps.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. 4 | # 5 | # See the COPYRIGHT.md file at the top-level directory of this distribution and at 6 | # https://github.com/sylabs/singularity/blob/master/COPYRIGHT.md. 7 | # 8 | # This file is part of the Singularity Linux container project. It is subject to the license 9 | # terms in the LICENSE.md file found in the top-level directory of this distribution and 10 | # at https://github.com/sylabs/singularity/blob/master/LICENSE.md. No part 11 | # of Singularity, including this file, may be copied, modified, propagated, or distributed 12 | # except according to the terms contained in the LICENSE.md file. 13 | 14 | 15 | if test -n "${SINGULARITY_APPNAME:-}"; then 16 | 17 | # The active app should be exported 18 | export SINGULARITY_APPNAME 19 | 20 | if test -d "/scif/apps/${SINGULARITY_APPNAME:-}/"; then 21 | SCIF_APPS="/scif/apps" 22 | SCIF_APPROOT="/scif/apps/${SINGULARITY_APPNAME:-}" 23 | export SCIF_APPROOT SCIF_APPS 24 | PATH="/scif/apps/${SINGULARITY_APPNAME:-}:$PATH" 25 | 26 | # Automatically add application bin to path 27 | if test -d "/scif/apps/${SINGULARITY_APPNAME:-}/bin"; then 28 | PATH="/scif/apps/${SINGULARITY_APPNAME:-}/bin:$PATH" 29 | fi 30 | 31 | # Automatically add application lib to LD_LIBRARY_PATH 32 | if test -d "/scif/apps/${SINGULARITY_APPNAME:-}/lib"; then 33 | LD_LIBRARY_PATH="/scif/apps/${SINGULARITY_APPNAME:-}/lib:$LD_LIBRARY_PATH" 34 | export LD_LIBRARY_PATH 35 | fi 36 | 37 | # Automatically source environment 38 | if [ -f "/scif/apps/${SINGULARITY_APPNAME:-}/scif/env/01-base.sh" ]; then 39 | . "/scif/apps/${SINGULARITY_APPNAME:-}/scif/env/01-base.sh" 40 | fi 41 | if [ -f "/scif/apps/${SINGULARITY_APPNAME:-}/scif/env/90-environment.sh" ]; then 42 | . "/scif/apps/${SINGULARITY_APPNAME:-}/scif/env/90-environment.sh" 43 | fi 44 | 45 | export PATH 46 | else 47 | echo "Could not locate the container application: ${SINGULARITY_APPNAME}" 48 | exit 1 49 | fi 50 | fi 51 | 52 | -------------------------------------------------------------------------------- /scripts/env/99-base.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # 3 | # Copyright (c) 2017, SingularityWare, LLC. All rights reserved. 4 | # 5 | # Copyright (c) 2015-2017, Gregory M. Kurtzer. All rights reserved. 6 | # 7 | # Copyright (c) 2016-2017, The Regents of the University of California, 8 | # through Lawrence Berkeley National Laboratory (subject to receipt of any 9 | # required approvals from the U.S. Dept. of Energy). All rights reserved. 10 | # 11 | # This software is licensed under a customized 3-clause BSD license. Please 12 | # consult LICENSE.md file distributed with the sources of this project regarding 13 | # your rights to use or distribute this software. 14 | # 15 | # NOTICE. This Software was developed under funding from the U.S. Department of 16 | # Energy and the U.S. Government consequently retains certain rights. As such, 17 | # the U.S. Government has been granted for itself and others acting on its 18 | # behalf a paid-up, nonexclusive, irrevocable, worldwide license in the Software 19 | # to reproduce, distribute copies to the public, prepare derivative works, and 20 | # perform publicly and display publicly, and to permit other to do so. 21 | # 22 | # 23 | 24 | 25 | if [ -z "$LD_LIBRARY_PATH" ]; then 26 | LD_LIBRARY_PATH="/.singularity.d/libs" 27 | else 28 | LD_LIBRARY_PATH="$LD_LIBRARY_PATH:/.singularity.d/libs" 29 | fi 30 | 31 | PS1="Singularity> " 32 | export LD_LIBRARY_PATH PS1 33 | -------------------------------------------------------------------------------- /scripts/env/99-runtimevars.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | # Copyright (c) 2017-2018, SyLabs, Inc. All rights reserved. 3 | # 4 | # This software is licensed under a customized 3-clause BSD license. Please 5 | # consult LICENSE.md file distributed with the sources of this project regarding 6 | # your rights to use or distribute this software. 7 | # 8 | # 9 | 10 | if [ -n "${SING_USER_DEFINED_PREPEND_PATH:-}" ]; then 11 | PATH="${SING_USER_DEFINED_PREPEND_PATH}:${PATH}" 12 | fi 13 | 14 | if [ -n "${SING_USER_DEFINED_APPEND_PATH:-}" ]; then 15 | PATH="${PATH}:${SING_USER_DEFINED_APPEND_PATH}" 16 | fi 17 | 18 | if [ -n "${SING_USER_DEFINED_PATH:-}" ]; then 19 | PATH="${SING_USER_DEFINED_PATH}" 20 | fi 21 | 22 | unset SING_USER_DEFINED_PREPEND_PATH \ 23 | SING_USER_DEFINED_APPEND_PATH \ 24 | SING_USER_DEFINED_PATH 25 | 26 | export PATH 27 | -------------------------------------------------------------------------------- /test.sh: -------------------------------------------------------------------------------- 1 | #! /bin/bash 2 | # 3 | # test.sh is provided for local testing, since we likely cannot connect to 4 | # priviledged daemon on circleci easily. 5 | # 6 | # USAGE: ./test.sh ubuntu:14.04 7 | # 8 | # 9 | # Copyright (c) 2018 Vanessa Sochat, All Rights Reserved 10 | # 11 | # Permission is hereby granted, free of charge, to any person obtaining a copy 12 | # of this software and associated documentation files (the "Software"), to deal 13 | # in the Software without restriction, including without limitation the rights 14 | # to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 15 | # copies of the Software, and to permit persons to whom the Software is 16 | # furnished to do so, subject to the following conditions: 17 | # 18 | # The above copyright notice and this permission notice shall be included in all 19 | # copies or substantial portions of the Software. 20 | # 21 | # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 22 | # IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 23 | # FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 24 | # AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 25 | # LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 26 | # OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 27 | # SOFTWARE. 28 | 29 | set -o errexit 30 | set -o nounset 31 | 32 | testing_container="vanessa/salad" 33 | testing_version=v$(cat VERSION) 34 | 35 | if [ $# == 1 ] ; then 36 | testing_container="${1:-}" 37 | fi 38 | 39 | image="singularityware/singularity2docker:${testing_version}" 40 | 41 | echo "" 42 | echo "Testing Container: ${testing_container}"; 43 | echo "singularity2docker: v${testing_version}"; 44 | echo "" 45 | 46 | echo "Pulling ${testing_container}"; 47 | docker pull "${testing_container}"; 48 | 49 | ################################################################################ 50 | # Test 1. Entrypoint functions 51 | ################################################################################ 52 | 53 | echo "1. Testing simple run of container to verify entrypoint is working." 54 | echo "docker run ${image}" 55 | docker run "${image}" 56 | test "$?" -eq "0" && echo "PASS" || echo "FAIL" 57 | 58 | ################################################################################ 59 | # Test 2. Basic Build 60 | ################################################################################ 61 | 62 | echo "2. Testing docker2singularity build with ${testing_container}" 63 | TMPDIR=$(mktemp -d) 64 | echo "docker run -v /var/run/docker.sock:/var/run/docker.sock -v ${TMPDIR}:/output --privileged -t --rm ${image} ${testing_container}" 65 | docker run -v /var/run/docker.sock:/var/run/docker.sock -v ${TMPDIR}:/output --privileged -t --rm ${image} "${testing_container}" 66 | ls ${TMPDIR}; 67 | test "$?" -eq "0" && echo "PASS" || echo "FAIL" 68 | 69 | ################################################################################ 70 | # Test 3. Build with Name 71 | ################################################################################ 72 | 73 | echo "3. Testing docker2singularity build with custom name" 74 | echo "docker run -v /var/run/docker.sock:/var/run/docker.sock -v ${TMPDIR}:/output --privileged -t --rm ${image} --name chimichanga.simg ${testing_container}" 75 | docker run -v /var/run/docker.sock:/var/run/docker.sock -v ${TMPDIR}:/output --privileged -t --rm ${image} --name chimichanga.simg "${testing_container}" 76 | ls ${TMPDIR}; 77 | test "$?" -eq "0" && echo "PASS" || echo "FAIL" 78 | 79 | ################################################################################ 80 | # Test 4. Testing that container runs! 81 | ################################################################################ 82 | 83 | echo "4. Testing that container runs!" 84 | echo "singularity run --pwd /go/src/github.com/vsoch/salad $TMPDIR/chimichanga.simg fork" 85 | singularity run --pwd /go/src/github.com/vsoch/salad $TMPDIR/chimichanga.simg fork 86 | test "$?" -eq "0" && echo "PASS" || echo "FAIL" 87 | 88 | echo "Cleaning up..." 89 | rm -rf "${TMPDIR}" 90 | --------------------------------------------------------------------------------