└── README.md /README.md: -------------------------------------------------------------------------------- 1 | # The Container Ecosystem Project 2 | 3 | The ecosystem of awesome new technologies emerging around containers and microservices can be a little overwhelming, to say the least. We thought we might be able to help: welcome to the Container Ecosystem Project. The goals of this project are (1) to clearly lay out the different types technologies that make up the growing container ecosystem and the microservices technology stack – starting from the lowest levels of core container technology, and rising up through layers of abstraction to full-blown container platforms and support tools – and (2) to put forth the latest and greatest examples of each type of technology. 4 | 5 | This project is a living document - please submit edits/pull requests for suggested changes. 6 | 7 | View this project as a webpage here: 8 | http://www.sysdig.com/the-container-ecosystem-project/ 9 | 10 | ## Table of Contents 11 | 12 | * The Container Ecosystem 13 | * [Core Container Technologies](https://github.com/draios/sysdig-container-ecosystem#core-container-technologies) 14 | * [Container specifications](https://github.com/draios/sysdig-container-ecosystem#container-specifications) 15 | * [Container runtimes](https://github.com/draios/sysdig-container-ecosystem#container-runtimes) 16 | * [Container management](https://github.com/draios/sysdig-container-ecosystem#container-management) 17 | * [Container definition](https://github.com/draios/sysdig-container-ecosystem#container-definition) 18 | * [Registries](https://github.com/draios/sysdig-container-ecosystem#registries) 19 | * [Operating systems](https://github.com/draios/sysdig-container-ecosystem#operating-systems) 20 | * [VM management](https://github.com/draios/sysdig-container-ecosystem#vm-management) 21 | * [Distributed Container Technologies](https://github.com/draios/sysdig-container-ecosystem#distributed-container-technologies) 22 | * [Scheduling](https://github.com/draios/sysdig-container-ecosystem#scheduling) 23 | * [Cluster definition](https://github.com/draios/sysdig-container-ecosystem#cluster-definition) 24 | * [Service discovery / Distributed configuration storage](https://github.com/draios/sysdig-container-ecosystem#service-discovery--distributed-configuration-storage) 25 | * [Dynamic configuration management](https://github.com/draios/sysdig-container-ecosystem#dynamic-configuration-management) 26 | * [Container Platform Technologies](https://github.com/draios/sysdig-container-ecosystem#container-platform-technologies) 27 | * [Container orchestration platform](https://github.com/draios/sysdig-container-ecosystem#container-orchestration-platform) 28 | * [Hosted container platform](https://github.com/draios/sysdig-container-ecosystem#hosted-container-platform) 29 | * [Container platform management](https://github.com/draios/sysdig-container-ecosystem#container-platform-management) 30 | * [Container-based PaaS](https://github.com/draios/sysdig-container-ecosystem#container-based-paas) 31 | * [Container-Native Support Technologies](https://github.com/draios/sysdig-container-ecosystem#container-native-support-technologies) 32 | * [Networking](https://github.com/draios/sysdig-container-ecosystem#networking) 33 | * [Monitoring / Visibility](https://github.com/draios/sysdig-container-ecosystem#monitoring--visibility) 34 | * [Data layer](https://github.com/draios/sysdig-container-ecosystem#data-layer) 35 | * [CI/CD](https://github.com/draios/sysdig-container-ecosystem#cicd) 36 | * [Security](https://github.com/draios/sysdig-container-ecosystem#security) 37 | * [Getting started aides](https://github.com/draios/sysdig-container-ecosystem#getting-started-aides) 38 | * [About the Container Ecosystem Project](https://github.com/draios/sysdig-container-ecosystem#about-the-container-ecosystem-project) 39 | * [Further Reading](https://github.com/draios/sysdig-container-ecosystem#further-reading) 40 | 41 | 42 | 43 | ## Core Container Technologies 44 | 45 | *Use these tools to run a small number of containers on a single host* 46 | 47 | #### Container specifications 48 | 49 | An abstract definition of a standard "container", allowing an ecosystem of technologies to support a standard container with potentially multiple, interchangeable runtime implementations 50 | 51 | * **Docker open source** 52 | * [Open Container spec](https://github.com/appc/spec): open industry standard for container runtimes; supported by Docker, CoreOS, and most industry leaders; backed by the [Open Container Initiative (OCI)](http://www.opencontainers.org/) (run by the Linux Foundation); currently absorbing CoreOS's AppC standard 53 | 54 | * **CoreOS open source** 55 | * [AppC](https://github.com/appc/spec) (deprecated): CoreOS is now supporting the OCI 56 | 57 | 58 | #### Container runtimes 59 | 60 | This is your actual running container (essentially an abstraction of Linux kernel components like namespaces and cgroups that allow virtualization on top of a shared kernel) 61 | 62 | * **Docker open source** 63 | * [runc](https://github.com/opencontainers/runc): Docker's container runtime, now donated to the OCI as the initial implementation of the standard; essentially a repackaging of libcontainer 64 | * [libcontainer](https://github.com/opencontainers/runc/tree/master/libcontainer): a Linux container library; enables and abstracts interactions with Linux kernel components to create and control containers 65 | 66 | * **CoreOS open source** 67 | * [rkt](https://github.com/coreos/rkt): CoreOS's container runtime; initially an implementation of the AppC specification, which is now being rolled into the OCI spec 68 | 69 | * **Other open source** 70 | * [LXC](https://linuxcontainers.org/): a Linux container library; originally utilized by runc until release of libcontainer 71 | * [OpenVZ](https://openvz.org/Main_Page): a Linux container library 72 | 73 | 74 | #### Container management 75 | 76 | These tools abstract low level control of your container runtime adding further functionality and usability 77 | 78 | * **Docker open source** 79 | * [Docker Engine](https://www.docker.com/docker-engine) (aka "Docker"): the core of Docker and its primary interface; creates and runs Docker containers; includes: 80 | * Docker daemon: runs as a process on the host machine and provides an API that abstracts basic container control functions 81 | * Docker client: a CLI for interacting with the Docker daemon 82 | 83 | * **CoreOS open source** 84 | * [rkt CLI](https://coreos.com/rkt/docs/latest/commands.html): rkt's container management functionality is delivered on-demand by a binary, rather than a daemon background process 85 | 86 | * **Other open source** 87 | * [LXD](https://linuxcontainers.org/lxd/): daemon and UI for LXC 88 | * [libvirt](http://libvirt.org/): container and virtualization mgmt library that supports LXC, OpenVZ, and a variety of hypervisor technologies 89 | 90 | 91 | #### Container definition 92 | 93 | These tools allow you to define specific containers, so they can be saved, shared and reproduced 94 | 95 | * **Docker open source** 96 | * [Docker image](https://docs.docker.com/userguide/dockerimages/): a template representing a fully configured container; Docker container runtimes are created from these images; images are created with Dockerfiles and shared over registries 97 | * [Dockerfile](https://docs.docker.com/articles/dockerfile_best-practices/): text file containing all the commands needed to build a Docker image 98 | 99 | * **CoreOS open source** 100 | * [ACI (App Container Image)](https://github.com/coreos/rkt#rkt-basics): rkt's native container image format (note, rkt also supports Docker images) 101 | 102 | 103 | #### Registries 104 | 105 | Repositories for storing and sharing container images 106 | 107 | * **Docker open source** 108 | * [Docker Registry](https://github.com/docker/distribution): open source Docker image registry that can be hosted in your own environment 109 | 110 | * **Commercial** 111 | * Hosted 112 | * [Amazon EC2 Container Registry (ECR)](https://aws.amazon.com/ecr/): still in beta 113 | * [Docker Hub](https://hub.docker.com/): hosted registry with free paid tiers, private public repositories, and a collection of "official" images 114 | * [Google Container Registry](https://cloud.google.com/container-registry/) 115 | * [Quay.io](https://quay.io/): CoreOS's hosted registry 116 | * On-premise 117 | * [Docker Trusted Registry](https://www.docker.com/docker-trusted-registry) 118 | * [CoreOS Enterprise Registry](https://coreos.com/products/enterprise-registry) 119 | 120 | 121 | #### Operating systems 122 | 123 | OS's that are designed for hosting containers 124 | 125 | * **Docker open source** 126 | * [boot2docker](http://boot2docker.io/) (basically deprecated by Docker Machine): minimalist Linux for running Docker on PC and Mac in a VM; now used by Docker Machine in certain environments 127 | 128 | * **CoreOS open source** 129 | * [CoreOS](https://coreos.com/): minimalist OS built for running distributed, containerized apps; includes etcd and fleet 130 | 131 | * **Other open source** 132 | * [RancherOS](https://github.com/rancher/os): minimalist, fully containerized OS 133 | * [Project Atomic](http://www.projectatomic.io/): minimalist Red Hat Linux; versions include RHEL Atomic, CentOS Atomic, and Fedora Atomic 134 | * [Ubuntu Core "Snappy"](https://developer.ubuntu.com/en/snappy/): minimalist Ubuntu 135 | * [SmartOS](https://smartos.org/): Solaris-based OS from Joyent that includes [Zones](http://wiki.smartos.org/display/DOC/Zones) (ie. Solaris containers) 136 | * [Photon OS](https://vmware.github.io/photon/): minimalist OS from VMWare 137 | 138 | 139 | #### VM management 140 | 141 | These tools help you manage the host virtual environments in which you run your containers 142 | 143 | * **Docker open source** 144 | * [Docker Machine](https://github.com/docker/machine): creates and manages host VMs running Docker, including local VMs (eg. VirtualBox) and cloud VMs (eg. Amazon AWS, Google GCP) 145 | 146 | * **Other open source** 147 | * [Hashicorp Vagrant](https://www.vagrantup.com/): creates pre-configured VMs for dev environments based on a variety of "Providers" (virtualization technologies) including Docker containers 148 | * [Hashicorp Otto](https://www.ottoproject.io/): extends Vagrant to deploy and manage VMs across many platforms 149 | 150 | 151 | ## Distributed Container Technologies 152 | 153 | *Use these technologies to run applications on a distributed cluster of containers* 154 | 155 | #### Scheduling 156 | 157 | These tools manage placement of new containers across abstracted underlying resources 158 | 159 | * **Docker open source** 160 | * [Docker Swarm](https://github.com/docker/swarm/): designed to extend Docker API to a cluster; includes scheduling and service discovery 161 | 162 | * **CoreOS open source** 163 | * [fleet](https://github.com/coreos/fleet): low level orchestration included in CoreOS; supports basic scheduling; can be used to bootstrap Kubernetes for higher level orchestration 164 | 165 | * **Other open source** 166 | * [Chronos](https://github.com/mesos/chronos): framework for scheduling on Mesos 167 | 168 | 169 | #### Cluster definition 170 | 171 | These tools allow you to define and manage a cluster of dependent containers as a single composable entity 172 | 173 | * **Docker open source** 174 | * [Docker Compose](https://github.com/docker/docker/issues/9694): text files used to define and configure a distributed application across a cluster of Docker containers 175 | 176 | * **CoreOS open source** 177 | * [fleet unit file](https://github.com/coreos/fleet/blob/master/Documentation/unit-files-and-scheduling.md): fleet uses a specialized version of systemd unit files to define a distributed application across containers 178 | 179 | 180 | #### Service discovery / Distributed configuration storage 181 | 182 | These tools allow applications within different containers to discover each other and share configuration information (eg. IP addresses or application settings); usually implemented as a globally distributed key-value store 183 | 184 | * **Docker open source** 185 | * Docker Swarm comes with built in service discovery, but can also use etcd, Consul, Zookeeper 186 | 187 | * **CoreOS open source** 188 | * [etcd](https://github.com/coreos/etcd): globally distributed key-value store; included with CoreOS for service discovery 189 | 190 | * **Other open source** 191 | * [Marathon](https://github.com/mesosphere/marathon): framework for initializing long running jobs on Mesos; includes service discovery and cluster management functionality 192 | * [Hashicorp Consul](https://www.consul.io/): service discovery, key/value store, and cluster health checking; uses [Serf](https://www.serfdom.io/) 193 | * [Apache ZooKeeper](https://zookeeper.apache.org/): globally distributed key-value store 194 | 195 | 196 | #### Dynamic configuration management 197 | 198 | These tools let you dynamically update application settings based on changes to your distributed key-value store in applications that don't natively support this 199 | 200 | * **CoreOS open source** 201 | * [confd](http://www.confd.io/): originally built for etcd, but now supports Consul and ZooKeeper 202 | 203 | * **Other open source** 204 | * [Consul Template](https://github.com/hashicorp/consul-template): built natively for Consul 205 | 206 | 207 | ## Container Platform Technologies 208 | 209 | *Use these technologies as complete platforms for running distributed applications across clusters of containers* 210 | 211 | #### Container orchestration platforms 212 | 213 | These platforms include or abstract away all of the core functionality (listed above) needed for container cluster management ("orchestration"), including container management, scheduling, cluster definition, and service discovery 214 | 215 | * **Docker open source** 216 | * Docker Swarm, Compose, and Machine can all run together to create a complete orchestration platform (still beta); Docker Swarm can also support more advanced orchestration tools like Kubernetes 217 | 218 | * **Other open source** 219 | * [Apache Mesos](http://mesos.apache.org/): mature, highly scalable service that abstracts a pool of underlying resources and distributes "tasks" (including Docker images) from various application frameworks; uses Marathon and Chronos to add cluster management, scheduling, and service discovery; also can support Kubernetes 220 | * [Kubernetes](http://kubernetes.io/): orchestration platform designed specifically for running microservices on clusters of containers; includes scheduling, cluster management and service discovery through abstractions such as "pods", "replication controllers (RCs)", and "services"; originally from Google, now donated to the [CNCF](https://cncf.io/) 221 | * [Hashicorp Nomad](https://nomadproject.io/): uses Consul 222 | 223 | 224 | #### Hosted container platforms 225 | 226 | These platforms offer container hosting and orchestration as a service 227 | 228 | * **Commercial** 229 | * [Amazon EC2 Container Service (ECS)](https://aws.amazon.com/ecs/) 230 | * [Google Container Engine](https://cloud.google.com/container-engine/): uses Kubernetes 231 | * Docker [Tutum](https://www.tutum.co/): acquired by Docker; still beta 232 | * [Redhat Openshift](https://enterprise.openshift.com/): uses Kubernetes 233 | * [Joyent – Triton](https://www.joyent.com/) 234 | * [Giant Swarm](https://giantswarm.io/): still beta 235 | * [ProfitBricks](https://www.profitbricks.com/docker): still beta 236 | * [Modulus](https://modulus.io/) 237 | * Rackspace [Carina](https://getcarina.com/): still beta 238 | 239 | 240 | #### Container platform management 241 | 242 | These technologies add further abstracted management and control layers to distributed container environments, often through GUIs 243 | 244 | * **Docker open source** 245 | * [Project Orca](https://youtu.be/BKVKc_xFnw8?list=PLenh213llmcbpJ78mZdh5pnJ_feVT9bezt=5237): opinionated management GUI built on top of full stack of Docker technologies; still alpha 246 | 247 | * **Other open source** 248 | * [Rancher](https://github.com/rancher/rancher): still beta 249 | * [ContainerShip](https://github.com/containership/containership) 250 | * [Panamax](http://panamax.io/) 251 | * [Shipyard](https://github.com/shipyard/shipyard) 252 | * [Joyent SmartDataCenter](https://github.com/joyent/sdc): uses SmartOS 253 | 254 | * **Commercial** 255 | * [Mesosphere DCOS](https://mesosphere.com/): uses Mesos 256 | * [CoreOS Tectonic](https://tectonic.com/): uses CoreOS+Kubernetes; still beta 257 | * [Nirmata](http://nirmata.com/): multi-cloud container management; built in scheduling, policy-based orchestration, service discovery, dynamic load balancing, and infrastructure optimization. 258 | * [ContainerShip Enterprise](http://containership.io/): still beta 259 | * [StackEngine](http://stackengine.com/) 260 | * [AppFormix](http://www.appformix.com/) 261 | 262 | 263 | #### Container-based PaaS 264 | 265 | These platforms further abstract container-based infrastructures by managing application code deployment and offering PaaS-like user experiences 266 | 267 | * **Other open source** 268 | * [Deis](https://github.com/deis/deis): container based PaaS; uses CoreOS 269 | * [Flynn](https://github.com/flynn/flynn): container based PaaS; uses etcd 270 | * [RedHat Openshift Origin](http://www.openshift.org/) 271 | * [Cisco Mantl](https://github.com/CiscoCloud/microservices-infrastructure): uses Mesos 272 | * [Dokku](https://github.com/progrium/dokku): minimalist PaaS 273 | * [Empire](https://github.com/remind101/empire): PaaS built for Amazon's ECS 274 | 275 | 276 | ## Container-Native Support Technologies 277 | 278 | *Use these additional container-native tools to support your container-based infrastructure* 279 | 280 | #### Networking 281 | 282 | * **Docker open source** 283 | * [Docker port expose](https://docs.docker.com/articles/networking/): Docker feature that links a container port to a host port 284 | * [Docker linking](https://docs.docker.com/userguide/dockerlinks/): Docker feature offering a basic connection between containers on the same host 285 | * [libnetwork](https://github.com/docker/libnetwork): advanced container networking library (still "under heavy development") 286 | 287 | * **CoreOS open source** 288 | * [flannel](https://github.com/coreos/flannel): overlay network built using etcd that gives each host a separate subnet for its containers 289 | 290 | * **Other open source** 291 | * [Weave](https://github.com/weaveworks/weave): overlay network that puts all containers in a distributed system onto a single virtual network; also includes service discovery functionality 292 | * [Calico](http://www.projectcalico.org/): layer 3 virtual network that provides each container with an IP address 293 | 294 | 295 | #### Monitoring / Visibility 296 | 297 | * **Docker open source** 298 | * Docker ps/top/stats: runtime commands 299 | * Docker stats API: remote API for streaming basic container metrics; utilized by the Docker Ecosystem Technology Partners for Monitoring 300 | 301 | * **Other open source** 302 | * [sysdig](http://www.sysdig.org/): CLI for deep system/containers visibility; includes curses-based "csysdig" interface 303 | * [cAdvisor](https://github.com/google/cadvisor): basic container metrics exporter from Google; includes web GUI; [Heapster](https://github.com/kubernetes/heapster) adds Kubernetes support 304 | * [Weave Scope](https://github.com/weaveworks/scope): container network topologies 305 | 306 | * **Commercial** 307 | * [Sysdig Cloud](https://sysdig.com/): uses sysdig; includes web-based UI, application topologies, and support for all major container formats and orchestration platforms 308 | 309 | 310 | #### Data layer 311 | 312 | 313 | * **Other open source** 314 | * [CusterHQ Flocker](https://clusterhq.com/): data volume manager for running stateful services like databases in containers 315 | 316 | 317 | #### Log management 318 | 319 | 320 | * **Docker open source** 321 | * [Docker logs](https://docs.docker.com/reference/commandline/logs/): runtime command 322 | 323 | * **Other open source** 324 | * [logspout](https://github.com/gliderlabs/logspout): log router for Docker containers 325 | 326 | 327 | #### CI/CD 328 | 329 | * **Commercial** 330 | * [Shippable](https://app.shippable.com/) 331 | * [Wercker](http://wercker.com/) 332 | 333 | 334 | #### Security 335 | 336 | * **Other open source** 337 | * [OpenSCAP](https://github.com/OpenSCAP/container-compliance) 338 | 339 | * **Commercial** 340 | * [Twistlock](https://www.twistlock.com/) 341 | * [Scalock](http://scalock.com/) 342 | * [Conjur](http://www.conjur.net/) 343 | * [Lynis](https://cisofy.com/lynis/plugins/docker-containers) 344 | 345 | #### Getting started aides 346 | 347 | * **Docker open source** 348 | * [Docker Kitematic](https://www.docker.com/docker-kitematic): basic Docker GUI designed for getting started with Docker 349 | * [Docker Toolbox](https://www.docker.com/toolbox): installer for a package of core Docker tools 350 | 351 | 352 | 353 | ## About the Container Ecosystem Project 354 | 355 | Here at [Sysdig](https://sysdig.com/), the container-native visibility company, we talk to a lot of people in the container ecosystem: both consumers and producers of technology. And wow, there is a LOT of cool technology out there – and so much more coming out all the time. It can be hard to keep up with, even if you're a seasoned expert, much less as a curious newcomer just trying to figure out where to start. There are plenty of great guides out there for various container technologies and use cases (see below for some links). But we had yet to find a clearly organized survey of the different core technologies that make up the container ecosystem and the typical microservices stack. So we decided to make one: the Container Ecosystem Project. 356 | 357 | For each type of technology, we've tried to provide a brief description, as well as list examples currently available for that technology. We've separated out open source solutions from commercial offerings. Two of the leading open source container technology producers, Docker and CoreOS, each get their own sections. Throughout the doc, we've tried to mark beta technologies and parent technologies accordingly. 358 | 359 | This framework is not, of course, a perfect science, and this will be a work in progress. If you have any suggested edits, please [tweet us](https://twitter.com/sysdig) or submit edits/pull requests. We'll do our best to keep this document up to date and prune off deprecated or abandoned technologies as the ecosystem evolves. 360 | 361 | ## Further Reading 362 | 363 | * Docker ecosystem introduction from Digital Ocean: https://www.digitalocean.com/community/tutorial_series/the-docker-ecosystem 364 | * Lists of Docker ecosystem technologies 365 | * https://www.mindmeister.com/389671722/runc-open-container-ecosystem 366 | * https://github.com/weihanwang/docker-ecosystem-survey 367 | * https://github.com/veggiemonk/awesome-docker 368 | * Docker docs: https://docs.docker.com/ 369 | * CoreOS docs: https://coreos.com/docs/ 370 | 371 | Project created and maintained by Sysdig, the container-native visibility company. 372 | http://www.sysdig.com/ 373 | http://www.sysdig.org/ 374 | 375 | --------------------------------------------------------------------------------