├── .gitignore
├── README.md
├── application_stacks.md
├── atlas.json
├── ci-servers.md
├── cloud-as-default.md
├── codebase.md
├── containers.md
├── contributors.md
├── copyright.html
├── cover.html
├── cover.jpg
├── images
├── chatbot.png
├── dds-cover.png
└── field-guide-sticky.png
├── index.html
├── index.json
├── local-dev.md
├── manage-environment.md
├── monitoring.md
├── realtime-chat-and-chatbots.md
├── signup.html
├── survey-form.html
├── testing-tools.md
├── theme
├── html
│ ├── html.css
│ └── layout.html
└── pdf
│ ├── .gitkeep
│ └── pdf.css
├── titlepage.html
└── toc.html
/.gitignore:
--------------------------------------------------------------------------------
1 | .DS_Store
2 | *.zip
3 | *.pdf
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Introduction
2 |
3 | This project began while we were developing [Atlas](https://atlas.oreilly.com/), O'Reilly Media's next-generation publishing tool. It seemed like every day we were finding interesting new tools in the DevOps space, so I started a "Sticky" for the most interesting-looking tools so I could explore them later.
4 |
5 | Copyright © 2014 Andrew Odewahn. All rights reserved. Printed in the United States of America. Published by O'Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472. O'Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles ( http://safaribooksonline.com ). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com . While the publisher and the author(s) have used good faith efforts to ensure that the information and instructions contained in this work are accurate, the publisher and the author(s) disclaim all responsibility for errors or omissions, including without limitation responsibility for damages resulting from the use of or reliance on this work. Use of the information and instructions contained in this work is at your own risk. If any code samples or other technology this work contains or describes is subject to open source licenses or the intellectual property rights of others, it is your responsibility to ensure that your use thereof complies with such licenses and/or rights. 978-1-491-91658-2 [LSI]
6 |
7 | At first, this worked fine. I was content to simply keep a list, where my only ordering criteria was "Huh, that looks cool. Someday when I have time, I'll take a look at that," in the same way you might buy an exercise DVD and then only occasionally pull it out and think "Huh, someday I'll get to that." But, as anyone who has watched DevOps for any length of time can tell you, it's a space bursting with interesting and exciting new tools, so my list and guilt quickly got out of hand.
8 |
9 | Once I reached the limits of the Sticky as a medium, I started to look for patterns in my list. Some were obvious. For example, many of the tools, like Ansible, Salt, or (to a certain extent) Dockerfiles, fit into a clear infrastructure-automation group pioneered by Chef, CFEngine, and Puppet. So, too, the many cloud services.
10 |
11 | But where would something like CoreOS, Docker, or Mesos fit? As I thought about how to group them, they seemed somehow tied up with the notion of containerization, but that just seemed too narrow. Rather, these projects and tools were part of a much larger trend — enabling clustering and distributed computing—and containerization was just a piece. So, rather than group by technology, it made sense to me to group by trend—in other words, what did the tool enable, and why was that trend important?
12 |
13 | Simultaneously, other people at O'Reilly were also exploring this same question, but from a different perspective. In "[Everything is distributed](http://radar.oreilly.com/2014/05/everything-is-distributed.html)," Courtney Nash, the chair of [Velocity](http://velocityconf.com/), was asking "how do we manage systems that are too large to understand, too complex to control, and that fail in unpredictable ways." In "[Beyond the stack](http://radar.oreilly.com/2014/05/beyond-the-stack.html)," Mike Loukides was thinking about how "a new toolset has grown up to support the development of massively distributed applications," and described the profound consequences that the shift from well-tended, internal servers to disposable VMs was having on the traditional "LAMP" stack. (As well as its hipster cousin, the [MEAN stack](http://meanjs.org/).)
14 |
15 | So, it's from this context that my Sticky list grew into this [Field Guide to the Distributed Development Stack](http://sites.oreilly.com/odewahn/dds-field-guide/). The Guide is organized into buckets based on a general observation, such as:
16 |
17 | * [The cloud is the default platform](http://sites.oreilly.com/odewahn/dds-field-guide/ch02.html)
18 | * [CI servers deploy code, not ops](http://sites.oreilly.com/odewahn/dds-field-guide/ch03.html)
19 | * [The codebase is in git](http://sites.oreilly.com/odewahn/dds-field-guide/ch04.html)
20 | * [The entire application runs locally in development](http://sites.oreilly.com/odewahn/dds-field-guide/ch05.html)
21 | * [The environment is automated in the code](http://sites.oreilly.com/odewahn/dds-field-guide/ch06.html)
22 | * [The monitoring infrastructure is critical](http://sites.oreilly.com/odewahn/dds-field-guide/ch07.html)
23 | * [Tests done in code, not by a QA department](http://sites.oreilly.com/odewahn/dds-field-guide/ch08.html)
24 |
25 | In addition to being a (hopefully) useful framework, the Guide is also meant to be a living resource. So, [we've put the source on GitHub](https://github.com/odewahn/dds-field-guide) and invite you to contribute. If you feel like we've missed a tool (which we most certainly have, since new things are popping up every day) or a major theme, then fork the repo and send me a pull request. We'll be keeping this document up to date and republishing it as we watch this trend continue to grow. We'll use [O'Reilly Atlas](https://atlas.oreilly.com) to pull in the contributions and periodically republish the guide.
26 |
27 | This is still very much a work-in-progress, but I hope it will be a resource you'll add to your own Sticky collection.
28 |
29 | ## How to Contribute
30 |
31 | To contribute to the DDS field guide:
32 | * Fork this repo
33 | * Agree to the [O'Reilly Contributor License Agreement](http://contributor-agreements.oreilly.com/)
34 | * Add your tool / contribution
35 | * Submit a pull request
36 |
37 | If your request is accepted, we'll add you to the Contributors page.
38 |
39 | ### Making a larger contribution
40 |
41 | If you want to make a suggestion or contribution that is larger than just a single tool, it might make sense to begin the conversation as a GitHub issue, rather than a pull request. For example, if you want to add a new theme, or want to add a major narrative section, it would be good to discuss that first to make sure it's suitable for the guide. While I certainly don't want to limit what people contribute in any way, it's also the case that this guide will be centrally curated by me and other O'Reilly contributors.
42 |
43 |
--------------------------------------------------------------------------------
/application_stacks.md:
--------------------------------------------------------------------------------
1 | # Appendix A: Language Stack Quick Reference
2 |
3 |
4 |
5 | No language is an island. Rather, every successful language (Ruby, Python, etc.) comes with an ecosystem of tools and frameworks. The goal of the following table is to map some of the key elements in these language ecosystems so that you can quickly cross-reference an unfamiliar stack against one you're familiar with.
6 |
7 | Why would you care, you ask? Well, even if you don't need to know a language in depth, you still might want to install and use some great open source package, so it helps to have an idea of what is what. This is not meant to be an exhaustive list, just an overview of the biggies in each ecosystem.
8 |
9 | ## Ruby
10 |
11 | * Frameworks
12 | * [Rails](http://rubyonrails.org/)
13 | * [Sinatra](http://www.sinatrarb.com/)
14 | * Package management
15 | * [RubyGems](http://rubygems.org/ "")
16 | * gem
17 | * [Bundler](http://bundler.io/)
18 | * Testing tools
19 | * RSpec
20 | * [factory\_girl](https://github.com/thoughtbot/factory_girl)
21 |
22 | ## Python
23 |
24 | * Frameworks
25 | * [Django](https://www.djangoproject.com/). Django is a modular, high-level Python Web framework that ships with an ORM, a template engine and lots of helpers.
26 | * [Flask](http://flask.pocoo.org/) is a popular wsgi micro-framework, with embeded development server and debugger, template support (via [Jinja2](http://jinja.pocoo.org/) ) and restfull handlers. Flask supports [extensions](http://flask.pocoo.org/extensions/) to add functionnalities.
27 | * Package management
28 | * [Setup\_tools](https://pypi.python.org/pypi/setuptools) is a python packaging library for distribution, test, installation, dependency management, and metadata handling of python projects.
29 | * [pip](https://pypi.python.org/pypi/pip). Pip is a package manager, to install, remove, and manage packages and dependency. It can handle [build distribution](https://www.python.org/dev/peps/pep-0427/) to ship and deploy a specific distribution of your code and dependencies.
30 | * [Virtualenv](https://virtualenv.pypa.io/en/latest/) is a Python environment isolation, to manage application dependencies in a folder, outside of system-wise packages.
31 | * Testing tools
32 | * [Nose](https://nose.readthedocs.org/en/latest/) is a testing tool and library to help test Python code.
33 | * [Coverage](https://pypi.python.org/pypi/coverage) provides code coverage.
34 | * [Mock](https://pypi.python.org/pypi/mock) is a mocking library for Python. It is now included in Python's standard library sinc 3.3.
35 |
36 | ## node
37 |
38 | * Frameworks
39 | * [express](http://expressjs.com/)
40 | * [Meteor](https://www.meteor.com/)
41 | * [Ember.js](http://emberjs.com/)
42 | * Package management
43 | * npm
44 | * [npmjs](https://npmjs.org/)
45 | * Testing tools
46 |
47 | ## Java (and JVM-based languages)
48 |
49 | * Frameworks
50 | * [Play](http://www.playframework.com/) Play is a framework for Scala and Java.
51 | * [Spring](http://projects.spring.io/spring-framework/) A Dependency Injection based framework for just about everything.
52 | * [Dropwizard](http://dropwizard.io/)Dropwizard is a Java framework for developing ops-friendly, high-performance, RESTful web services.
53 | * Package management
54 | * [Maven](https://maven.apache.org/)
55 | * [Ant](http://ant.apache.org/)
56 | * [Gradle](http://www.gradle.org/)
57 | * [sbt](http://www.scala-sbt.org/) Scala-based interactive build tool.
58 | * Testing Tools
59 | * [JUnit](http://junit.org/)
60 |
61 | ## Go
62 |
63 | * Frameworks
64 | * [Gorilla Web Toolkit](http://www.gorillatoolkit.org/) is a collection of libraries to provide context, more powerfull router and dispatcher (mux), RPC and session management.
65 | * [Negroni](http://negroni.codegangsta.io/) is a HTTP middleware library over net/http. It provides middlewares and makes it easy to create your own.
66 | * [gocraft/web](https://github.com/gocraft/web) is a router (mux) and middleware package.
67 | * Package manager
68 | * [go get](https://golang.org/cmd/go/#hdr-Download_and_install_packages_and_dependencies) is go builtin package and dependencies manager
69 | * [godep](https://github.com/tools/godep) is a go dependencies manager.
70 | * Testing tools
71 | * [testing](http://golang.org/pkg/testing/) is go standard library testing library (to use with 'go-test')
72 |
73 |
--------------------------------------------------------------------------------
/atlas.json:
--------------------------------------------------------------------------------
1 | {
2 | "branch": "master",
3 | "files": [
4 | "titlepage.html",
5 | "copyright.html",
6 | "toc.html",
7 | "README.md",
8 | "cloud-as-default.md",
9 | "ci-servers.md",
10 | "codebase.md",
11 | "local-dev.md",
12 | "manage-environment.md",
13 | "monitoring.md",
14 | "testing-tools.md",
15 | "containers.md",
16 | "realtime-chat-and-chatbots.md",
17 | "contributors.md",
18 | "survey-form.html",
19 | "application_stacks.md"
20 | ],
21 | "formats": {
22 | "pdf": {
23 | "version": "web",
24 | "index": true,
25 | "toc": true,
26 | "syntaxhighlighting": false,
27 | "show_comments": false
28 | },
29 | "epub": {
30 | "index": false,
31 | "toc": true,
32 | "epubcheck": true,
33 | "syntaxhighlighting": true,
34 | "show_comments": false
35 | },
36 | "mobi": {
37 | "index": false,
38 | "toc": true,
39 | "syntaxhighlighting": true,
40 | "show_comments": false
41 | },
42 | "html": {
43 | "index": false,
44 | "toc": true,
45 | "syntaxhighlighting": false,
46 | "show_comments": false,
47 | "consolidate": false,
48 | "consolidated": false
49 | }
50 | },
51 | "theme": "git@github.com:odewahn/simple_theme.git",
52 | "title": "A Field Guide to the Distributed Development Stack ",
53 | "github_url": "https://github.com/odewahn/dds-field-guide",
54 | "practice-area": "webops",
55 | "ga_code": "UA-39014713-4",
56 | "forum_shortname": "dockerjumpstart"
57 | }
--------------------------------------------------------------------------------
/ci-servers.md:
--------------------------------------------------------------------------------
1 | # CI Servers Deploy Code, Not Ops
2 |
3 |
4 |
5 | Martin Fowler defines [continuous integration](http://www.martinfowler.com/articles/continuousIntegration.html) as "a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day." This seminal article defines the key best practices as:
6 |
7 | * Maintain a single source repository.
8 | * Automate the build.
9 | * Make your build self-testing.
10 | * Everyone commits to the mainline every day.
11 | * Every commit should build the mainline on an integration machine.
12 | * Keep the build fast.
13 | * Test in a clone of the production environment.
14 | * Make it easy for anyone to get the latest executable.
15 | * Everyone can see what's happening.
16 | * Automate deployment.
17 |
18 | The CI server executes a specific action on a repository when triggered by a commit hook. For example, if a developer makes a commit against a repository called foo, the CI server might:
19 |
20 | * Clone a local copy of foo.
21 | * Execute foo's test suites (see the section on application stacks for more about this).
22 | * If the tests fail, send an alert to the development team and halt the process.
23 | * If the test suite passes, deploy the code to a staging or even production server.
24 |
25 | ## Tools
26 |
27 | Here are a few of the CI servers you might encounter:
28 |
29 | * [Hudson](http://hudson-ci.org/). Hudson is a CI server from Oracle written in Java.
30 | * [Jenkins](http://jenkins-ci.org/). Jenkins, a fork of Hudson, is one of the leading open source CI servers. It has a host of useful plug-ins for tasks like build tasks, error reporting, and repository management.
31 | * [Buildbot](http://buildbot.net/). Buildbot is an open source CI server written in Python.
32 | * [Travis](https://travis-ci.org/). Travis is a hosted CI solution that is used primarily by the Ruby community, particularly Rails.
33 | * [Hubot](http://hubot.github.com/). Hubot is a chatbot from GitHub. It allows you to easily create scripts that you can use inside your chatroom (e.g., Campfire or HipChat) to deploy new code, receive messages from the build server, or get messages from your monitoring tools when things go wrong.
34 | * [Shippable](http://shippable.com/) Shippable is a hosted CI solution built on Docker with Webhooks.
35 | * [Wercker](http://wercker.com/). Wercker is a hosted CI solution built on Docker with Webhooks.
36 | * [Circle CI](https://circleci.com/) is a hosted Continuous Integration and Deployment with Docker support.
37 | * [Drone](https://github.com/drone/drone) is an open-source (and [hosted](https://drone.io/) ) Continuous Integration platform built on Docker
38 | * [Bamboo](https://www.atlassian.com/software/bamboo) Bamboo is an Atlassian CI/CD tool that integrates seamlessly with other Atlassian products such as JIRA, Confluence and Stash.
39 | * [Codeship](https://codeship.com) Codeship is a hosted CI/CD service. Codeship will run your project's unit tests, and can also deploy new code to your servers via Capistrano, Amazon EBS, Heroku, custom scripts, and many others.
40 | * [DeployBot](http://deploybot.com). DeployBot is a hosted service that provides many repository, environment and service integrations. It also features automated code compilation.
41 |
42 |
43 | ## Continuous Deployment
44 | * [Distelli](http://www.distelli.com/). Deployment and process management.
45 | * [Capistrano](http://capistranorb.com/). Automation and deployment tool. [Webistrano](https://github.com/peritor/webistrano) provides a web frontend to Capistrano.
46 | * [GoCD](http://www.go.cd/) is an open-source continuous delivery tool built around a pipeline model.
47 |
--------------------------------------------------------------------------------
/cloud-as-default.md:
--------------------------------------------------------------------------------
1 | # The Cloud Is the Default Platform
2 |
3 |
4 |
5 | The accelerating transition to distributed, cloud-based platforms is one of the main drivers of the DDS trend. Organizations have adopted these services for a number of reasons: cost savings, increased speed for launching new projects, and scalability, to name just a few.
6 |
7 | But, whatever the reasons for adoption, the default platform for many applications is increasingly assumed to be a transient, virtual, cloud-based platform, rather than a traditional server maintained by an internal IT group. Even in cases where the virtualization/PaaS solution is maintained in an internal cloud, the net effect is much the same.
8 |
9 | The various platforms you're likely to encounter in this new world can be divided into three main groups:
10 |
11 | * Traditional cloud providers. These allow you to quickly create storage or computing power as needed.
12 | * Hosted PaaS services. These are value-added services built on top of raw hosting providers. For example, a PaaS might allow you to easily spin up a machine based on a particular stack when you deploy your application.
13 | * Internal cloud and PaaS services.
14 |
15 |
16 | These are tools and services you're likely to encounter here:
17 |
18 | ## Traditional Cloud Providers
19 |
20 | There are lots of hosting services. Here are some of the more popular:
21 |
22 | * [Amazon Web Services](http://aws.amazon.com/). AWS is probably the original model for pay-as-you-go infrastructure and remains one of the leading cloud platforms.
23 | * [Google Compute Engine](https://cloud.google.com/products/compute-engine/). Google's cloud platform, which has become much more compelling now that they have [open sourced their cluster management tools](http://googlecloudplatform.blogspot.com/2014/06/an-update-on-container-support-on-google-cloud-platform.html).
24 | * [Azure](http://www.windowsazure.com/). Microsoft's cloud offering.
25 | * [Rackspace cloud](https://mycloud.rackspace.com/). The cloud offering from Rackspace is mostly about compute power and storage. The API is well done, and the customer service is generally outstanding. Overall, though, it's not as full-featured as AWS.
26 | * [Digital Ocean](https://www.digitalocean.com/). Cloud provider with SSD, low prices, snapshot and pre-build images
27 | * [Linode](https://www.linode.com/) is a hosting service that offers SSD for really fast access.
28 | * [SoftLayer](http://www.softlayer.com/). Offers virtual and bare-metal servers. API covers the full feature-set. Global routed private network.
29 | * [DreamCompute](https://www.dreamhost.com/cloud/). Long a mainstay of shared hosting, DreamHost offers a public cloud platform based on OpenStack.
30 |
31 | ## Hosted PaaS
32 |
33 | Hosted PaaS services add a layer on top of the raw offerings of hosting providers:
34 |
35 | * [Heroku](https://www.heroku.com/). A PaaS service built on top of AWS. Unlike AWS, which gives you a raw machine, Heroku allows the developer to push an application into the service and have a corresponding application stack provision for the machine.
36 | * [OpenShift](https://www.openshift.com/) is a cloud-hosted PaaS solution developer by Red Hat, the company behind [Red Hat Enterprise Linux](http://www.redhat.com/products/enterprise-linux/) distribution.
37 | * [Google App Engine](https://cloud.google.com/appengine/) is Google PaaS service, that supports Python, Go, PHP and Java.
38 | * [AppFog](https://www.appfog.com/) is a PaaS provider with fine-grained stack management.
39 | * [dotCloud](https://www.dotcloud.com/) is a PaaS provider from the creators of Docker, the famous container open-source tool.
40 | * [Pivotal Web Services](http://run.pivotal.io) A PaaS based on Cloud Foundry hosted by Pivotal, the largest contributor to the Cloud Foundry Open Source project.
41 | * [Bluemix](https://bluemix.net/) A PaaS based on Cloud Foundry hosted by IBM.
42 | * [Helion](http://www8.hp.com/us/en/cloud/helion-overview.html) A PaaS based on OpenStack and Cloud Foundry hosted by HP.
43 |
44 | ## Internal Services and Build-your-own Cloud
45 |
46 | These are tools that create virtual public or internal clouds (private,i.e. on premise). While they're technically running in your own (internal) datacenter, they enable the concept of scalable, on-demand resources:
47 |
48 | * [OpenStack](https://www.openstack.org/). Open source software for building private and public clouds.
49 | * [VMware vCloud Suite](http://www.vmware.com/products/vcloud-suite/). A tool for running and managing VMware images in your own data center.
50 | * [Mesos](http://mesos.apache.org/) / [Marathon](https://github.com/mesosphere/marathon) / [Chronos](https://github.com/airbnb/chronos) are a trio of technologies for managing and scheduling processes across a cluster of machines. Apache Mesos provides the core clustering technology for the stack. Marathon, from [Mesosphere](http://mesosphere.io/), is a distributed tool for starting, stopping, and managing individual jobs on a Mesos cluster. (So, it's like a distributed version of [init](http://en.wikipedia.org/wiki/Init) or [upstart](http://upstart.ubuntu.com/)). Chronos, developed by Airbnb's engineering team, is a distributed, fault-tolerant replacement for cron (the classic UNIX job scheduling tool) for scheduling when jobs will start.
51 | * [OpenShift Origin](https://openshift.github.io/) is the upstream version of Red Hat's open source OpenShift platform.
52 | * [Deis](http://deis.io/) is an open-source PaaS for public and private clouds, built on [Docker](https://www.docker.com/) and [CoreOS](https://coreos.com/).
53 | * [Flynn](https://flynn.io/) is an open-source PaaS for public and private clouds, built on [Docker](https://www.docker.com/).
54 | * [Dokku](https://github.com/progrium/dokku) is a Docker-powered small PaaS, written in bash, with modularity in mind.
55 | * [Kubernetes](https://github.com/googlecloudplatform/kubernetes) is an open-source container cluster management, built by Google.
56 | * [Eucalyptus](https://www.eucalyptus.com) is an open source, AWS compatible private cloud software.
57 | * [Cloud Foundry](http://www.cloudfoundry.org) is an open source PaaS backed by the Cloud Foundry Foundation with companies such as Pivotal, IBM, HP, VMware, SAP and tens of others.
58 |
59 | ## For More Information
60 |
61 | You can find more important background at [The Twelve Factor App](http://12factor.net/).
62 |
--------------------------------------------------------------------------------
/codebase.md:
--------------------------------------------------------------------------------
1 | # The Codebase Is in Git
2 |
3 |
4 |
5 | The version control system (VCS) is the heart of the process. At the most basic level, a VCS allows developers to keep track of all the changes made to a set of files and enables them to roll back to specific points in time in case something screws up. In some systems, like [Subversion](http://subversion.apache.org/), the code is checked out and then checked back in from a central repository. If there is a conflict between two developers' files (for example, both of them edited the same line of code), then the two version must be merged. This can be a painful process.
6 |
7 | In contrast, distributed version control systems (DVCS), like [Git](http://git-scm.com/), are the heart of most new development processes. Rather than having a central, master copy that makes it difficult and expensive to merge a lot of contributions from developers, a DVCS makes it simple (well, simpler!) to have multiple people all working on the same codebase simultaneously in different _branches_, and these branches can be easily merged into a _master_ branch.
8 |
9 | While there are many different work styles, such as [Git flow](http://nvie.com/posts/a-successful-git-branching-model/), the basic DVCS process is:
10 |
11 | * There is an agreed-upon master repository, which is often on a public service like [GitHub](https://github.com/) or [BitBucket](https://bitbucket.org/), or an internal server like [GitLab](https://www.gitlab.com/) or [Mercurial](http://mercurial.selenic.com/).
12 | * Each developer clones the master repository to his or her local machine.
13 | * The developer creates a new branch, usually for a specific feature.
14 | * The developer makes commits against the local copy.
15 | * Once the feature is done, he or she merges the branch back into the master branch and pushes the change back to the master copy of the repository.
16 | * Other developers pull from the master branch and merge their branch.
17 | * The merged copy preserves the full version history of all the distributed copies.
18 |
19 | In addition to these coordination functions, most version control systems also offer a feature called a _hook_. A hook is a process that fires once a specific event, like a commit, happens to the repository. Hooks can be defined in the repo itself, but also in the hosting service. For example, GitHub lets you define "service" hooks that are called whenever a specific event occurs. These hooks are the tie-in to the continuous integration (CI server).
20 |
21 | ## Tools
22 |
23 | Here are the key version control systems:
24 |
25 | * [Git](http://git-scm.com/). "Git is a [free and open source](http://git-scm.com/about/free-and-open-source) distributed version control system designed to handle everything from small to very large projects with speed and efficiency."
26 | * [Mercurial](http://mercurial.selenic.com/). "Mercurial is a free, distributed source control management tool. It efficiently handles projects of any size and offers an easy and intuitive interface."
27 |
28 | Hosting services provide a central point where you can manage and store all your code repositories. In addition to raw code storage, they usually offer features like issue tracking, collaborator management, and other process-oriented services.
29 |
30 | The following table lists hosting services managed by a 3rd party. The pricing model is typically based on a block of repositories for a monthly fee.
31 |
32 | * [GitHub](https://github.com/). One of the largest and most successful Git hosting services.
33 | * [BitBucket](https://bitbucket.org/). [Atlassian](https://www.atlassian.com/)'s Git hosting solution.
34 | * [GitLab.com](https://www.gitlab.com/). A hosting service based on the popular open source project GitLab HQ.
35 | * [Gitorious](https://gitorious.org/). Similar to GitLab, a hosted version of an open source tool that you can install and maintain yourself.
36 | * [Beanstalk](http://beanstalkapp.com). Beanstalk hosts Git and SVN repositories. It also provides webhooks and many built-in integrations for continuous deployment and automation.
37 |
38 | These are services that you can install and manage in your own environment:
39 |
40 | * [GitLab](https://github.com/gitlabhq/gitlabhq). "Project management and code hosting application."
41 | * [Gitosis](https://github.com/tv42/gitosis). "software for hosting Git repositories"
42 | * [Gitorious](https://gitorious.org/gitorious). The self-hosted version of gitorious.org. (It's a Rails app.)
43 | * [Kallithea](https://kallithea-scm.org/). Source code management system for Git and Mercurial.
44 | * [Stash](https://www.atlassian.com/software/stash). On-premises source code management for Git by [Atlassian](https://www.atlassian.com/).
45 |
--------------------------------------------------------------------------------
/containers.md:
--------------------------------------------------------------------------------
1 | # Containerization for Production Services
2 |
3 |
4 |
5 | Containerization is the idea that an application and all its dependencies can be packaged and shipped in a standardized way that is the same for any platform. This enables you, for example, to package a container you created and built on your development machine directly to a production server. Be aware that the container only needs the dependencies and code from your app; other systems (like, oh, the operating system) can be shared with the host environment via a [Linux container](https://linuxcontainers.org/). There are many, many tools in this rapidly emerging space, such as:
6 |
7 |
8 | * [CoreOS](https://coreos.com/). CoreOS is a Linux distro built for running and manageing applications that are packaged as Docker images. The key components are:
9 | * Docker as the way you run apps.
10 | * [etcd](https://github.com/coreos/etcd), which is a distributed key value database; it's sort of the "registry" you can use to share data between instances. It's bundled with the OS so that you can always count on it being there.
11 | * [systemd](http://coreos.com/using-coreos/systemd/), a distributed job system for scheduling and process management. I don't really quite understand this yet but plan to dive in soon.
12 | * [fleet](http://coreos.com/docs/launching-containers/launching/launching-containers-fleet/). Fleet is a tool for managing processes on a CoreOS cluster.
13 | * [Deis](http://deis.io/). Deis is a self-hosted PaaS platform based on Docker and CoreOS. Basically, it allows you to create your own Heroku-like service based on [buildpacks](https://devcenter.heroku.com/articles/buildpacks) or "raw" Docker containers.
14 | * [Docker](http://www.docker.com). Written in Go, Docker is an Open Source project that provides a clean and simple way to create system images based on a known filesystem, layer new elements onto those images, and then spin up running instances of what you've done. For example, you might start with a base images like "base/ubuntu", add a service like Redis, and then start the image to have a running Redis instance. Docker handles process management, networking, and other services for you, allowing you to focus on adding just the parts you need. There is also a company called (conveniently enough) Docker that maintains Docker, as well as providing a hosting service called the Index, where you can publish and maintain your images. Like GitHub, they have a "free for public/paid for private" model.
15 | * [Flynn.io](https://flynn.io/). Flynn, like Deis, provides a self-hosted PaaS.
16 | * [geard](http://openshift.github.io/geard/) is a tool for creating and managing Docker containers in [OpenShift](https://openshift.github.io/), RedHast's PaaS solution.
17 | * [kubernetes](https://github.com/GoogleCloudPlatform/kubernetes). Kubernetes is an open source version of Google's internal cluster management toolchain. You can use Kubernetes to run your own Docker containers on [Google Compute Engine](https://cloud.google.com/products/compute-engine/), Google's cloud platform.
18 | * [Mesos](http://mesos.apache.org). Apache Mesos is a cluster resource manager that simplifies running applications on a shared pool of servers. Mesos supports containerized workloads via linux cgroups and now supports running tasks in Docker containers natively as of August 2014 with the release of version `0.20.0`.
19 | * [Panamax](http://panamax.io/). "Panamax is a containerized app creator with an open-source app marketplace hosted in GitHub. Panamax provides a friendly interface for users of Docker, Fleet & CoreOS. With Panamax, you can easily create, share and deploy any containerized app no matter how complex it might be"
20 |
--------------------------------------------------------------------------------
/contributors.md:
--------------------------------------------------------------------------------
1 | # Appendix: Contributors
2 |
3 | * Mike Loukides — @mikeloukides
4 | * Courney Nash — @courtneynash
5 | * Paco Nathan — @pacoid | https://github.com/ceteri
6 | * Andrew Odewahn — @odewahn | https://github.com/odewahn
7 | * Jean-Christophe Bohin — @bohinjc | https://github.com/jcbohin
8 | * Jason Yee — @gitbisect | https://github.com/jyee
9 | * Matt Williams - @matty_jwilliams | https://github.com/mattyw
10 | * Raghavan "Rags" N. Srinivas - @ragss | https://github.com/ragsns
11 | * James Bayer - @jambay | http://github.com/jbayer
12 | * Torstein Gjengedal - @gjengeda | https://github.com/gjengeda
13 | * Phil Jackson - @underscorephil | https://github.com/underscorephil
14 | * Duncan Lamb - https://github.com/dethkultur
15 | * Kyle Smith - @kylesm | http://github.com/kylesm
16 | * Alfonso Cabrera - @opsguynextdoor | https://github.com/alfonso-cabrera
17 | * Jordan Stone - @cheddz | https://github.com/jmstone617
18 |
--------------------------------------------------------------------------------
/copyright.html:
--------------------------------------------------------------------------------
1 |
A Field Guide to the Distributed Development Stack
3 |
4 |
5 |
6 |
15 |
21 |
22 |
23 |
25 |
26 |
27 | Revision History for the First Edition
29 |
30 |
31 |
33 |
38 | 39 | The definitive source for mainstream technology practitioners to discover and understand the tools and concepts prominent in the shift to the distributed development stack (DDS).
40 |41 |
38 | 39 | The definitive source for mainstream technology practitioners to discover and understand the tools and concepts prominent in the shift to the distributed development stack (DDS).
40 |41 |
49 |This field guide is a work in progress, and to make it a really useful tool, we really need your feedback. Please take a few minutes to fill out this short survey. Many thanks for your help.
5 | 6 | {% if format == "html" %} 7 | 8 | 9 | 10 | {% else %} 11 | 12 | http://oreil.ly/FGFeedback 13 | 14 | {% endif %} 15 | 16 |