├── .gitignore ├── README.md ├── application_stacks.md ├── atlas.json ├── ci-servers.md ├── cloud-as-default.md ├── codebase.md ├── containers.md ├── contributors.md ├── copyright.html ├── cover.html ├── cover.jpg ├── images ├── chatbot.png ├── dds-cover.png └── field-guide-sticky.png ├── index.html ├── index.json ├── local-dev.md ├── manage-environment.md ├── monitoring.md ├── realtime-chat-and-chatbots.md ├── signup.html ├── survey-form.html ├── testing-tools.md ├── theme ├── html │ ├── html.css │ └── layout.html └── pdf │ ├── .gitkeep │ └── pdf.css ├── titlepage.html └── toc.html /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store 2 | *.zip 3 | *.pdf -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Introduction 2 | 3 | This project began while we were developing [Atlas](https://atlas.oreilly.com/), O'Reilly Media's next-generation publishing tool. It seemed like every day we were finding interesting new tools in the DevOps space, so I started a "Sticky" for the most interesting-looking tools so I could explore them later. 4 | 5 | 6 | 7 | At first, this worked fine. I was content to simply keep a list, where my only ordering criteria was "Huh, that looks cool. Someday when I have time, I'll take a look at that," in the same way you might buy an exercise DVD and then only occasionally pull it out and think "Huh, someday I'll get to that." But, as anyone who has watched DevOps for any length of time can tell you, it's a space bursting with interesting and exciting new tools, so my list and guilt quickly got out of hand. 8 | 9 | Once I reached the limits of the Sticky as a medium, I started to look for patterns in my list. Some were obvious. For example, many of the tools, like Ansible, Salt, or (to a certain extent) Dockerfiles, fit into a clear infrastructure-automation group pioneered by Chef, CFEngine, and Puppet. So, too, the many cloud services. 10 | 11 | But where would something like CoreOS, Docker, or Mesos fit? As I thought about how to group them, they seemed somehow tied up with the notion of containerization, but that just seemed too narrow. Rather, these projects and tools were part of a much larger trend — enabling clustering and distributed computing—and containerization was just a piece. So, rather than group by technology, it made sense to me to group by trend—in other words, what did the tool enable, and why was that trend important? 12 | 13 | Simultaneously, other people at O'Reilly were also exploring this same question, but from a different perspective. In "[Everything is distributed](http://radar.oreilly.com/2014/05/everything-is-distributed.html)," Courtney Nash, the chair of [Velocity](http://velocityconf.com/), was asking "how do we manage systems that are too large to understand, too complex to control, and that fail in unpredictable ways." In "[Beyond the stack](http://radar.oreilly.com/2014/05/beyond-the-stack.html)," Mike Loukides was thinking about how "a new toolset has grown up to support the development of massively distributed applications," and described the profound consequences that the shift from well-tended, internal servers to disposable VMs was having on the traditional "LAMP" stack. (As well as its hipster cousin, the [MEAN stack](http://meanjs.org/).) 14 | 15 | So, it's from this context that my Sticky list grew into this [Field Guide to the Distributed Development Stack](http://sites.oreilly.com/odewahn/dds-field-guide/). The Guide is organized into buckets based on a general observation, such as: 16 | 17 | * [The cloud is the default platform](http://sites.oreilly.com/odewahn/dds-field-guide/ch02.html) 18 | * [CI servers deploy code, not ops](http://sites.oreilly.com/odewahn/dds-field-guide/ch03.html) 19 | * [The codebase is in git](http://sites.oreilly.com/odewahn/dds-field-guide/ch04.html) 20 | * [The entire application runs locally in development](http://sites.oreilly.com/odewahn/dds-field-guide/ch05.html) 21 | * [The environment is automated in the code](http://sites.oreilly.com/odewahn/dds-field-guide/ch06.html) 22 | * [The monitoring infrastructure is critical](http://sites.oreilly.com/odewahn/dds-field-guide/ch07.html) 23 | * [Tests done in code, not by a QA department](http://sites.oreilly.com/odewahn/dds-field-guide/ch08.html) 24 | 25 | In addition to being a (hopefully) useful framework, the Guide is also meant to be a living resource. So, [we've put the source on GitHub](https://github.com/odewahn/dds-field-guide) and invite you to contribute. If you feel like we've missed a tool (which we most certainly have, since new things are popping up every day) or a major theme, then fork the repo and send me a pull request. We'll be keeping this document up to date and republishing it as we watch this trend continue to grow. We'll use [O'Reilly Atlas](https://atlas.oreilly.com) to pull in the contributions and periodically republish the guide. 26 | 27 | This is still very much a work-in-progress, but I hope it will be a resource you'll add to your own Sticky collection. 28 | 29 | ## How to Contribute 30 | 31 | To contribute to the DDS field guide: 32 | * Fork this repo 33 | * Agree to the [O'Reilly Contributor License Agreement](http://contributor-agreements.oreilly.com/) 34 | * Add your tool / contribution 35 | * Submit a pull request 36 | 37 | If your request is accepted, we'll add you to the Contributors page. 38 | 39 | ### Making a larger contribution 40 | 41 | If you want to make a suggestion or contribution that is larger than just a single tool, it might make sense to begin the conversation as a GitHub issue, rather than a pull request. For example, if you want to add a new theme, or want to add a major narrative section, it would be good to discuss that first to make sure it's suitable for the guide. While I certainly don't want to limit what people contribute in any way, it's also the case that this guide will be centrally curated by me and other O'Reilly contributors. 42 | 43 | -------------------------------------------------------------------------------- /application_stacks.md: -------------------------------------------------------------------------------- 1 | # Appendix A: Language Stack Quick Reference 2 | 3 | 4 | 5 | No language is an island. Rather, every successful language (Ruby, Python, etc.) comes with an ecosystem of tools and frameworks. The goal of the following table is to map some of the key elements in these language ecosystems so that you can quickly cross-reference an unfamiliar stack against one you're familiar with. 6 | 7 | Why would you care, you ask? Well, even if you don't need to know a language in depth, you still might want to install and use some great open source package, so it helps to have an idea of what is what. This is not meant to be an exhaustive list, just an overview of the biggies in each ecosystem. 8 | 9 | ## Ruby 10 | 11 | * Frameworks 12 | * [Rails](http://rubyonrails.org/) 13 | * [Sinatra](http://www.sinatrarb.com/) 14 | * Package management 15 | * [RubyGems](http://rubygems.org/ "") 16 | * gem 17 | * [Bundler](http://bundler.io/) 18 | * Testing tools 19 | * RSpec 20 | * [factory\_girl](https://github.com/thoughtbot/factory_girl) 21 | 22 | ## Python 23 | 24 | * Frameworks 25 | * [Django](https://www.djangoproject.com/). Django is a modular, high-level Python Web framework that ships with an ORM, a template engine and lots of helpers. 26 | * [Flask](http://flask.pocoo.org/) is a popular wsgi micro-framework, with embeded development server and debugger, template support (via [Jinja2](http://jinja.pocoo.org/) ) and restfull handlers. Flask supports [extensions](http://flask.pocoo.org/extensions/) to add functionnalities. 27 | * Package management 28 | * [Setup\_tools](https://pypi.python.org/pypi/setuptools) is a python packaging library for distribution, test, installation, dependency management, and metadata handling of python projects. 29 | * [pip](https://pypi.python.org/pypi/pip). Pip is a package manager, to install, remove, and manage packages and dependency. It can handle [build distribution](https://www.python.org/dev/peps/pep-0427/) to ship and deploy a specific distribution of your code and dependencies. 30 | * [Virtualenv](https://virtualenv.pypa.io/en/latest/) is a Python environment isolation, to manage application dependencies in a folder, outside of system-wise packages. 31 | * Testing tools 32 | * [Nose](https://nose.readthedocs.org/en/latest/) is a testing tool and library to help test Python code. 33 | * [Coverage](https://pypi.python.org/pypi/coverage) provides code coverage. 34 | * [Mock](https://pypi.python.org/pypi/mock) is a mocking library for Python. It is now included in Python's standard library sinc 3.3. 35 | 36 | ## node 37 | 38 | * Frameworks 39 | * [express](http://expressjs.com/) 40 | * [Meteor](https://www.meteor.com/) 41 | * [Ember.js](http://emberjs.com/) 42 | * Package management 43 | * npm 44 | * [npmjs](https://npmjs.org/) 45 | * Testing tools 46 | 47 | ## Java (and JVM-based languages) 48 | 49 | * Frameworks 50 | * [Play](http://www.playframework.com/) Play is a framework for Scala and Java. 51 | * [Spring](http://projects.spring.io/spring-framework/) A Dependency Injection based framework for just about everything. 52 | * [Dropwizard](http://dropwizard.io/)Dropwizard is a Java framework for developing ops-friendly, high-performance, RESTful web services. 53 | * Package management 54 | * [Maven](https://maven.apache.org/) 55 | * [Ant](http://ant.apache.org/) 56 | * [Gradle](http://www.gradle.org/) 57 | * [sbt](http://www.scala-sbt.org/) Scala-based interactive build tool. 58 | * Testing Tools 59 | * [JUnit](http://junit.org/) 60 | 61 | ## Go 62 | 63 | * Frameworks 64 | * [Gorilla Web Toolkit](http://www.gorillatoolkit.org/) is a collection of libraries to provide context, more powerfull router and dispatcher (mux), RPC and session management. 65 | * [Negroni](http://negroni.codegangsta.io/) is a HTTP middleware library over net/http. It provides middlewares and makes it easy to create your own. 66 | * [gocraft/web](https://github.com/gocraft/web) is a router (mux) and middleware package. 67 | * Package manager 68 | * [go get](https://golang.org/cmd/go/#hdr-Download_and_install_packages_and_dependencies) is go builtin package and dependencies manager 69 | * [godep](https://github.com/tools/godep) is a go dependencies manager. 70 | * Testing tools 71 | * [testing](http://golang.org/pkg/testing/) is go standard library testing library (to use with 'go-test') 72 | 73 | -------------------------------------------------------------------------------- /atlas.json: -------------------------------------------------------------------------------- 1 | { 2 | "branch": "master", 3 | "files": [ 4 | "titlepage.html", 5 | "copyright.html", 6 | "toc.html", 7 | "README.md", 8 | "cloud-as-default.md", 9 | "ci-servers.md", 10 | "codebase.md", 11 | "local-dev.md", 12 | "manage-environment.md", 13 | "monitoring.md", 14 | "testing-tools.md", 15 | "containers.md", 16 | "realtime-chat-and-chatbots.md", 17 | "contributors.md", 18 | "survey-form.html", 19 | "application_stacks.md" 20 | ], 21 | "formats": { 22 | "pdf": { 23 | "version": "web", 24 | "index": true, 25 | "toc": true, 26 | "syntaxhighlighting": false, 27 | "show_comments": false 28 | }, 29 | "epub": { 30 | "index": false, 31 | "toc": true, 32 | "epubcheck": true, 33 | "syntaxhighlighting": true, 34 | "show_comments": false 35 | }, 36 | "mobi": { 37 | "index": false, 38 | "toc": true, 39 | "syntaxhighlighting": true, 40 | "show_comments": false 41 | }, 42 | "html": { 43 | "index": false, 44 | "toc": true, 45 | "syntaxhighlighting": false, 46 | "show_comments": false, 47 | "consolidate": false, 48 | "consolidated": false 49 | } 50 | }, 51 | "theme": "git@github.com:odewahn/simple_theme.git", 52 | "title": "A Field Guide to the Distributed Development Stack ", 53 | "github_url": "https://github.com/odewahn/dds-field-guide", 54 | "practice-area": "webops", 55 | "ga_code": "UA-39014713-4", 56 | "forum_shortname": "dockerjumpstart" 57 | } -------------------------------------------------------------------------------- /ci-servers.md: -------------------------------------------------------------------------------- 1 | # CI Servers Deploy Code, Not Ops 2 | 3 | 4 | 5 | Martin Fowler defines [continuous integration](http://www.martinfowler.com/articles/continuousIntegration.html) as "a software development practice where members of a team integrate their work frequently, usually each person integrates at least daily - leading to multiple integrations per day." This seminal article defines the key best practices as: 6 | 7 | * Maintain a single source repository. 8 | * Automate the build. 9 | * Make your build self-testing. 10 | * Everyone commits to the mainline every day. 11 | * Every commit should build the mainline on an integration machine. 12 | * Keep the build fast. 13 | * Test in a clone of the production environment. 14 | * Make it easy for anyone to get the latest executable. 15 | * Everyone can see what's happening. 16 | * Automate deployment. 17 | 18 | The CI server executes a specific action on a repository when triggered by a commit hook. For example, if a developer makes a commit against a repository called foo, the CI server might: 19 | 20 | * Clone a local copy of foo. 21 | * Execute foo's test suites (see the section on application stacks for more about this). 22 | * If the tests fail, send an alert to the development team and halt the process. 23 | * If the test suite passes, deploy the code to a staging or even production server. 24 | 25 | ## Tools 26 | 27 | Here are a few of the CI servers you might encounter: 28 | 29 | * [Hudson](http://hudson-ci.org/). Hudson is a CI server from Oracle written in Java. 30 | * [Jenkins](http://jenkins-ci.org/). Jenkins, a fork of Hudson, is one of the leading open source CI servers. It has a host of useful plug-ins for tasks like build tasks, error reporting, and repository management. 31 | * [Buildbot](http://buildbot.net/). Buildbot is an open source CI server written in Python. 32 | * [Travis](https://travis-ci.org/). Travis is a hosted CI solution that is used primarily by the Ruby community, particularly Rails. 33 | * [Hubot](http://hubot.github.com/). Hubot is a chatbot from GitHub. It allows you to easily create scripts that you can use inside your chatroom (e.g., Campfire or HipChat) to deploy new code, receive messages from the build server, or get messages from your monitoring tools when things go wrong. 34 | * [Shippable](http://shippable.com/) Shippable is a hosted CI solution built on Docker with Webhooks. 35 | * [Wercker](http://wercker.com/). Wercker is a hosted CI solution built on Docker with Webhooks. 36 | * [Circle CI](https://circleci.com/) is a hosted Continuous Integration and Deployment with Docker support. 37 | * [Drone](https://github.com/drone/drone) is an open-source (and [hosted](https://drone.io/) ) Continuous Integration platform built on Docker 38 | * [Bamboo](https://www.atlassian.com/software/bamboo) Bamboo is an Atlassian CI/CD tool that integrates seamlessly with other Atlassian products such as JIRA, Confluence and Stash. 39 | * [Codeship](https://codeship.com) Codeship is a hosted CI/CD service. Codeship will run your project's unit tests, and can also deploy new code to your servers via Capistrano, Amazon EBS, Heroku, custom scripts, and many others. 40 | * [DeployBot](http://deploybot.com). DeployBot is a hosted service that provides many repository, environment and service integrations. It also features automated code compilation. 41 | 42 | 43 | ## Continuous Deployment 44 | * [Distelli](http://www.distelli.com/). Deployment and process management. 45 | * [Capistrano](http://capistranorb.com/). Automation and deployment tool. [Webistrano](https://github.com/peritor/webistrano) provides a web frontend to Capistrano. 46 | * [GoCD](http://www.go.cd/) is an open-source continuous delivery tool built around a pipeline model. 47 | -------------------------------------------------------------------------------- /cloud-as-default.md: -------------------------------------------------------------------------------- 1 | # The Cloud Is the Default Platform 2 | 3 | 4 | 5 | The accelerating transition to distributed, cloud-based platforms is one of the main drivers of the DDS trend. Organizations have adopted these services for a number of reasons: cost savings, increased speed for launching new projects, and scalability, to name just a few. 6 | 7 | But, whatever the reasons for adoption, the default platform for many applications is increasingly assumed to be a transient, virtual, cloud-based platform, rather than a traditional server maintained by an internal IT group. Even in cases where the virtualization/PaaS solution is maintained in an internal cloud, the net effect is much the same. 8 | 9 | The various platforms you're likely to encounter in this new world can be divided into three main groups: 10 | 11 | * Traditional cloud providers. These allow you to quickly create storage or computing power as needed. 12 | * Hosted PaaS services. These are value-added services built on top of raw hosting providers. For example, a PaaS might allow you to easily spin up a machine based on a particular stack when you deploy your application. 13 | * Internal cloud and PaaS services. 14 | 15 | 16 | These are tools and services you're likely to encounter here: 17 | 18 | ## Traditional Cloud Providers 19 | 20 | There are lots of hosting services. Here are some of the more popular: 21 | 22 | * [Amazon Web Services](http://aws.amazon.com/). AWS is probably the original model for pay-as-you-go infrastructure and remains one of the leading cloud platforms. 23 | * [Google Compute Engine](https://cloud.google.com/products/compute-engine/). Google's cloud platform, which has become much more compelling now that they have [open sourced their cluster management tools](http://googlecloudplatform.blogspot.com/2014/06/an-update-on-container-support-on-google-cloud-platform.html). 24 | * [Azure](http://www.windowsazure.com/). Microsoft's cloud offering. 25 | * [Rackspace cloud](https://mycloud.rackspace.com/). The cloud offering from Rackspace is mostly about compute power and storage. The API is well done, and the customer service is generally outstanding. Overall, though, it's not as full-featured as AWS. 26 | * [Digital Ocean](https://www.digitalocean.com/). Cloud provider with SSD, low prices, snapshot and pre-build images 27 | * [Linode](https://www.linode.com/) is a hosting service that offers SSD for really fast access. 28 | * [SoftLayer](http://www.softlayer.com/). Offers virtual and bare-metal servers. API covers the full feature-set. Global routed private network. 29 | * [DreamCompute](https://www.dreamhost.com/cloud/). Long a mainstay of shared hosting, DreamHost offers a public cloud platform based on OpenStack. 30 | 31 | ## Hosted PaaS 32 | 33 | Hosted PaaS services add a layer on top of the raw offerings of hosting providers: 34 | 35 | * [Heroku](https://www.heroku.com/). A PaaS service built on top of AWS. Unlike AWS, which gives you a raw machine, Heroku allows the developer to push an application into the service and have a corresponding application stack provision for the machine. 36 | * [OpenShift](https://www.openshift.com/) is a cloud-hosted PaaS solution developer by Red Hat, the company behind [Red Hat Enterprise Linux](http://www.redhat.com/products/enterprise-linux/) distribution. 37 | * [Google App Engine](https://cloud.google.com/appengine/) is Google PaaS service, that supports Python, Go, PHP and Java. 38 | * [AppFog](https://www.appfog.com/) is a PaaS provider with fine-grained stack management. 39 | * [dotCloud](https://www.dotcloud.com/) is a PaaS provider from the creators of Docker, the famous container open-source tool. 40 | * [Pivotal Web Services](http://run.pivotal.io) A PaaS based on Cloud Foundry hosted by Pivotal, the largest contributor to the Cloud Foundry Open Source project. 41 | * [Bluemix](https://bluemix.net/) A PaaS based on Cloud Foundry hosted by IBM. 42 | * [Helion](http://www8.hp.com/us/en/cloud/helion-overview.html) A PaaS based on OpenStack and Cloud Foundry hosted by HP. 43 | 44 | ## Internal Services and Build-your-own Cloud 45 | 46 | These are tools that create virtual public or internal clouds (private,i.e. on premise). While they're technically running in your own (internal) datacenter, they enable the concept of scalable, on-demand resources: 47 | 48 | * [OpenStack](https://www.openstack.org/). Open source software for building private and public clouds. 49 | * [VMware vCloud Suite](http://www.vmware.com/products/vcloud-suite/). A tool for running and managing VMware images in your own data center. 50 | * [Mesos](http://mesos.apache.org/) / [Marathon](https://github.com/mesosphere/marathon) / [Chronos](https://github.com/airbnb/chronos) are a trio of technologies for managing and scheduling processes across a cluster of machines. Apache Mesos provides the core clustering technology for the stack. Marathon, from [Mesosphere](http://mesosphere.io/), is a distributed tool for starting, stopping, and managing individual jobs on a Mesos cluster. (So, it's like a distributed version of [init](http://en.wikipedia.org/wiki/Init) or [upstart](http://upstart.ubuntu.com/)). Chronos, developed by Airbnb's engineering team, is a distributed, fault-tolerant replacement for cron (the classic UNIX job scheduling tool) for scheduling when jobs will start. 51 | * [OpenShift Origin](https://openshift.github.io/) is the upstream version of Red Hat's open source OpenShift platform. 52 | * [Deis](http://deis.io/) is an open-source PaaS for public and private clouds, built on [Docker](https://www.docker.com/) and [CoreOS](https://coreos.com/). 53 | * [Flynn](https://flynn.io/) is an open-source PaaS for public and private clouds, built on [Docker](https://www.docker.com/). 54 | * [Dokku](https://github.com/progrium/dokku) is a Docker-powered small PaaS, written in bash, with modularity in mind. 55 | * [Kubernetes](https://github.com/googlecloudplatform/kubernetes) is an open-source container cluster management, built by Google. 56 | * [Eucalyptus](https://www.eucalyptus.com) is an open source, AWS compatible private cloud software. 57 | * [Cloud Foundry](http://www.cloudfoundry.org) is an open source PaaS backed by the Cloud Foundry Foundation with companies such as Pivotal, IBM, HP, VMware, SAP and tens of others. 58 | 59 | ## For More Information 60 | 61 | You can find more important background at [The Twelve Factor App](http://12factor.net/). 62 | -------------------------------------------------------------------------------- /codebase.md: -------------------------------------------------------------------------------- 1 | # The Codebase Is in Git 2 | 3 | 4 | 5 | The version control system (VCS) is the heart of the process. At the most basic level, a VCS allows developers to keep track of all the changes made to a set of files and enables them to roll back to specific points in time in case something screws up. In some systems, like [Subversion](http://subversion.apache.org/), the code is checked out and then checked back in from a central repository. If there is a conflict between two developers' files (for example, both of them edited the same line of code), then the two version must be merged. This can be a painful process. 6 | 7 | In contrast, distributed version control systems (DVCS), like [Git](http://git-scm.com/), are the heart of most new development processes. Rather than having a central, master copy that makes it difficult and expensive to merge a lot of contributions from developers, a DVCS makes it simple (well, simpler!) to have multiple people all working on the same codebase simultaneously in different _branches_, and these branches can be easily merged into a _master_ branch. 8 | 9 | While there are many different work styles, such as [Git flow](http://nvie.com/posts/a-successful-git-branching-model/), the basic DVCS process is: 10 | 11 | * There is an agreed-upon master repository, which is often on a public service like [GitHub](https://github.com/) or [BitBucket](https://bitbucket.org/), or an internal server like [GitLab](https://www.gitlab.com/) or [Mercurial](http://mercurial.selenic.com/). 12 | * Each developer clones the master repository to his or her local machine. 13 | * The developer creates a new branch, usually for a specific feature. 14 | * The developer makes commits against the local copy. 15 | * Once the feature is done, he or she merges the branch back into the master branch and pushes the change back to the master copy of the repository. 16 | * Other developers pull from the master branch and merge their branch. 17 | * The merged copy preserves the full version history of all the distributed copies. 18 | 19 | In addition to these coordination functions, most version control systems also offer a feature called a _hook_. A hook is a process that fires once a specific event, like a commit, happens to the repository. Hooks can be defined in the repo itself, but also in the hosting service. For example, GitHub lets you define "service" hooks that are called whenever a specific event occurs. These hooks are the tie-in to the continuous integration (CI server). 20 | 21 | ## Tools 22 | 23 | Here are the key version control systems: 24 | 25 | * [Git](http://git-scm.com/). "Git is a [free and open source](http://git-scm.com/about/free-and-open-source) distributed version control system designed to handle everything from small to very large projects with speed and efficiency." 26 | * [Mercurial](http://mercurial.selenic.com/). "Mercurial is a free, distributed source control management tool. It efficiently handles projects of any size and offers an easy and intuitive interface." 27 | 28 | Hosting services provide a central point where you can manage and store all your code repositories. In addition to raw code storage, they usually offer features like issue tracking, collaborator management, and other process-oriented services. 29 | 30 | The following table lists hosting services managed by a 3rd party. The pricing model is typically based on a block of repositories for a monthly fee. 31 | 32 | * [GitHub](https://github.com/). One of the largest and most successful Git hosting services. 33 | * [BitBucket](https://bitbucket.org/). [Atlassian](https://www.atlassian.com/)'s Git hosting solution. 34 | * [GitLab.com](https://www.gitlab.com/). A hosting service based on the popular open source project GitLab HQ. 35 | * [Gitorious](https://gitorious.org/). Similar to GitLab, a hosted version of an open source tool that you can install and maintain yourself. 36 | * [Beanstalk](http://beanstalkapp.com). Beanstalk hosts Git and SVN repositories. It also provides webhooks and many built-in integrations for continuous deployment and automation. 37 | 38 | These are services that you can install and manage in your own environment: 39 | 40 | * [GitLab](https://github.com/gitlabhq/gitlabhq). "Project management and code hosting application." 41 | * [Gitosis](https://github.com/tv42/gitosis). "software for hosting Git repositories" 42 | * [Gitorious](https://gitorious.org/gitorious). The self-hosted version of gitorious.org. (It's a Rails app.) 43 | * [Kallithea](https://kallithea-scm.org/). Source code management system for Git and Mercurial. 44 | * [Stash](https://www.atlassian.com/software/stash). On-premises source code management for Git by [Atlassian](https://www.atlassian.com/). 45 | -------------------------------------------------------------------------------- /containers.md: -------------------------------------------------------------------------------- 1 | # Containerization for Production Services 2 | 3 | 4 | 5 | Containerization is the idea that an application and all its dependencies can be packaged and shipped in a standardized way that is the same for any platform. This enables you, for example, to package a container you created and built on your development machine directly to a production server. Be aware that the container only needs the dependencies and code from your app; other systems (like, oh, the operating system) can be shared with the host environment via a [Linux container](https://linuxcontainers.org/). There are many, many tools in this rapidly emerging space, such as: 6 | 7 | 8 | * [CoreOS](https://coreos.com/). CoreOS is a Linux distro built for running and manageing applications that are packaged as Docker images. The key components are: 9 | * Docker as the way you run apps. 10 | * [etcd](https://github.com/coreos/etcd), which is a distributed key value database; it's sort of the "registry" you can use to share data between instances. It's bundled with the OS so that you can always count on it being there. 11 | * [systemd](http://coreos.com/using-coreos/systemd/), a distributed job system for scheduling and process management. I don't really quite understand this yet but plan to dive in soon. 12 | * [fleet](http://coreos.com/docs/launching-containers/launching/launching-containers-fleet/). Fleet is a tool for managing processes on a CoreOS cluster. 13 | * [Deis](http://deis.io/). Deis is a self-hosted PaaS platform based on Docker and CoreOS. Basically, it allows you to create your own Heroku-like service based on [buildpacks](https://devcenter.heroku.com/articles/buildpacks) or "raw" Docker containers. 14 | * [Docker](http://www.docker.com). Written in Go, Docker is an Open Source project that provides a clean and simple way to create system images based on a known filesystem, layer new elements onto those images, and then spin up running instances of what you've done. For example, you might start with a base images like "base/ubuntu", add a service like Redis, and then start the image to have a running Redis instance. Docker handles process management, networking, and other services for you, allowing you to focus on adding just the parts you need. There is also a company called (conveniently enough) Docker that maintains Docker, as well as providing a hosting service called the Index, where you can publish and maintain your images. Like GitHub, they have a "free for public/paid for private" model. 15 | * [Flynn.io](https://flynn.io/). Flynn, like Deis, provides a self-hosted PaaS. 16 | * [geard](http://openshift.github.io/geard/) is a tool for creating and managing Docker containers in [OpenShift](https://openshift.github.io/), RedHast's PaaS solution. 17 | * [kubernetes](https://github.com/GoogleCloudPlatform/kubernetes). Kubernetes is an open source version of Google's internal cluster management toolchain. You can use Kubernetes to run your own Docker containers on [Google Compute Engine](https://cloud.google.com/products/compute-engine/), Google's cloud platform. 18 | * [Mesos](http://mesos.apache.org). Apache Mesos is a cluster resource manager that simplifies running applications on a shared pool of servers. Mesos supports containerized workloads via linux cgroups and now supports running tasks in Docker containers natively as of August 2014 with the release of version `0.20.0`. 19 | * [Panamax](http://panamax.io/). "Panamax is a containerized app creator with an open-source app marketplace hosted in GitHub. Panamax provides a friendly interface for users of Docker, Fleet & CoreOS. With Panamax, you can easily create, share and deploy any containerized app no matter how complex it might be" 20 | -------------------------------------------------------------------------------- /contributors.md: -------------------------------------------------------------------------------- 1 | # Appendix: Contributors 2 | 3 | * Mike Loukides — @mikeloukides 4 | * Courney Nash — @courtneynash 5 | * Paco Nathan — @pacoid | https://github.com/ceteri 6 | * Andrew Odewahn — @odewahn | https://github.com/odewahn 7 | * Jean-Christophe Bohin — @bohinjc | https://github.com/jcbohin 8 | * Jason Yee — @gitbisect | https://github.com/jyee 9 | * Matt Williams - @matty_jwilliams | https://github.com/mattyw 10 | * Raghavan "Rags" N. Srinivas - @ragss | https://github.com/ragsns 11 | * James Bayer - @jambay | http://github.com/jbayer 12 | * Torstein Gjengedal - @gjengeda | https://github.com/gjengeda 13 | * Phil Jackson - @underscorephil | https://github.com/underscorephil 14 | * Duncan Lamb - https://github.com/dethkultur 15 | * Kyle Smith - @kylesm | http://github.com/kylesm 16 | * Alfonso Cabrera - @opsguynextdoor | https://github.com/alfonso-cabrera 17 | * Jordan Stone - @cheddz | https://github.com/jmstone617 18 | -------------------------------------------------------------------------------- /copyright.html: -------------------------------------------------------------------------------- 1 |
2 |

A Field Guide to the Distributed Development Stack

3 | 4 |

by Andrew Odewahn

5 | 6 | 7 | 8 |

Printed in the United States of America.

9 | 10 |

Published by O'Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.

11 | 12 |

O'Reilly books may be purchased for educational, business, or sales promotional use. Online editions are also available for most titles ( http://safaribooksonline.com ). For more information, contact our corporate/institutional sales department: 800-998-9938 or corporate@oreilly.com .

13 | 14 | 21 | 22 | 25 | 26 | 27 |
28 |

Revision History for the First Edition

29 | 30 | 33 |
34 | 35 | 36 | 42 | 43 | 48 |
49 | -------------------------------------------------------------------------------- /cover.html: -------------------------------------------------------------------------------- 1 |
cover
2 | -------------------------------------------------------------------------------- /cover.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/odewahn/dds-field-guide/cd358cbf63715cd37354397fe1d442d3f9b7667e/cover.jpg -------------------------------------------------------------------------------- /images/chatbot.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/odewahn/dds-field-guide/cd358cbf63715cd37354397fe1d442d3f9b7667e/images/chatbot.png -------------------------------------------------------------------------------- /images/dds-cover.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/odewahn/dds-field-guide/cd358cbf63715cd37354397fe1d442d3f9b7667e/images/dds-cover.png -------------------------------------------------------------------------------- /images/field-guide-sticky.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/odewahn/dds-field-guide/cd358cbf63715cd37354397fe1d442d3f9b7667e/images/field-guide-sticky.png -------------------------------------------------------------------------------- /index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | The Distributed Developer Stack Field Guide 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 |
26 | 27 |

The Distributed Developer Stack Field Guide

28 |
29 | 30 | 31 | 32 | 33 |
34 | 35 |
36 |
37 |

38 | 39 | The definitive source for mainstream technology practitioners to discover and understand the tools and concepts prominent in the shift to the distributed development stack (DDS).

40 |

41 |

42 |
43 | Read online 44 |
45 |
46 | Download 47 |
48 |
49 |
50 |
 
51 |
52 |

How to contribute

53 |
    54 |
  1. Agree to the O'Reilly contributor license agreement.
  2. 55 |
  3. Fork the repo.
  4. 56 |
  5. Send a pull request.
  6. 57 |
  7. Add yourself to the contributor page.
  8. 58 |
59 |
60 | 61 |
62 |
63 | 64 |
65 |

Some key ideas we're exploring...

66 | 67 | 68 | 69 |
70 | 74 |
 
75 | 79 |
80 | 81 | 82 | 83 |
84 |
85 | 86 | The codebase is in git. 87 |
88 |
 
89 | 93 |
94 | 95 | 96 | 97 |
98 | 102 |
 
103 | 107 | 108 |
109 | 110 |
111 | 115 |
 
116 | 120 |
121 | 122 |
123 |
124 | 125 | Realtime chat and chatbots. 126 |
127 |
 
128 |
129 |   130 |
131 |
132 | 133 | 134 | 135 |
136 | 137 | 138 | 139 | 140 | 141 | 142 | 143 | 144 | 145 | 146 | 147 | 148 | -------------------------------------------------------------------------------- /index.json: -------------------------------------------------------------------------------- 1 | { 2 | "themes": [ 3 | { 4 | "icon": "fa-cloud", 5 | "description": "The cloud is the default platform.", 6 | "link": "ch02.html" 7 | }, 8 | { 9 | "icon": "fa-cogs", 10 | "description": "CI servers deploy code, not ops.", 11 | "link": "ch03.html" 12 | }, 13 | { 14 | "icon": "fa-github", 15 | "description": "The codebase is in git.", 16 | "link": "ch04.html" 17 | }, 18 | { 19 | "icon": "fa-laptop", 20 | "description": "The application runs locally on development.", 21 | "link": "ch05.html" 22 | }, 23 | { 24 | "icon": "fa-wrench", 25 | "description": "The environment is automated in the code.", 26 | "link": "ch06.html" 27 | }, 28 | { 29 | "icon": "fa-stethoscope", 30 | "description": "The monitoring infrastructure is critical.", 31 | "link": "ch07.html" 32 | }, 33 | { 34 | "icon": "fa-thumbs-o-up", 35 | "description": "Tests done in code, not by a QA department.", 36 | "link": "ch08.html" 37 | }, 38 | { 39 | "icon": "fa-th-large", 40 | "description": "Containers are the default deployment target.", 41 | "link": "ch09.html" 42 | }, 43 | { 44 | "icon": "fa-comments", 45 | "description": "Realtime chat and chatbots.", 46 | "link": "ch10.html" 47 | } 48 | ] 49 | } -------------------------------------------------------------------------------- /local-dev.md: -------------------------------------------------------------------------------- 1 | # The Entire Application Runs Locally in Development 2 | 3 | 4 | 5 | One of the key tenets of the DDS movement is that developers should have a simple way to install and run the entire app on their local machine. Being able to run it on their own system encourages creativity and flexibility and makes development much more fun and productive. 6 | 7 | [Vagrant](http://www.vagrantup.com/) is the key tool here. Basically, it takes the recipes you created with your environment tool (e.g., your Chef or Puppet files) and _provisions_ (creates) a virtual machine that runs in a tool like [VirtualBox](https://www.virtualbox.org/) or [VMware](http://www.vmware.com/). Vagrant automatically maps a drive from the virtual instance back to the host machine, allowing the developer to use his or her favorite editor / IDE but still run the application in an environment that matches the production environment as closely as possible. 8 | 9 | ## Tools 10 | 11 | The following table summarizes tools you will encounter: 12 | 13 | * [Vagrant](http://www.vagrantup.com/). "Create and configure lightweight, reproducible, and portable development environments." 14 | * [VirtualBox](https://www.virtualbox.org/). "VirtualBox is a powerful x86 and AMD64/Intel64  [Virtualization](https://www.virtualbox.org/wiki/Virtualization) product for enterprise as well as home use." Free and open source. This tool allows you to run a full image of another operating system (called the "guest") on your own machine (called the "host"). 15 | * [VMware](http://www.vmware.com/). One of the original virtualization solutions; Vagrant has a paid version that targets VMware Fusion and Workstation. 16 | * [ngrok](https://ngrok.com/). ngrok allows you to share applications running on your local machine to other users on the Internet. It works by setting up a tunnel to ngrok, which then acts as a reverse proxy to allow you to have a public URL. It uses a donation model with a suggestion of $25, but you can pay what you want. In the paid version you can have it use a custom domain. 17 | * [Vagrant Cloud](https://vagrantcloud.com/). A service from the creator of Vagrant that allows you to share versioned Vagrant images. 18 | 19 | ### Replicating third-party APIs locally 20 | 21 | The develop "everything on localhost" approach breaks down somewhat when an application makes extensive use of third-party APIs. Clearly, you could not simply install Twitter or Facebook on your local machine. To get around this, there are a number of tools for mocking up the API results returned by these services. These include: 22 | 23 | * [Canned](https://github.com/sideshowcoder/canned). "Server to respond with fake API responses, by using a directory of files for finding out what to say." 24 | * [WireMock](http://wiremock.org/). "WireMock is a flexible library for stubbing and mocking web services." 25 | -------------------------------------------------------------------------------- /manage-environment.md: -------------------------------------------------------------------------------- 1 | # The Environment Is Automated in the Code 2 | 3 | 4 | 5 | A key idea (maybe _the_ key idea) of DevOps is that the environment in which your code will run should be modeled as code, and not be some separate thing that is a black box. (And, as we get a bit further down the stack, should be versioned with the code as well.) It seems pretty basic, but the idea is that you should have a "recipe" that allows you to recreate the environment at any moment. Some of the key parts of managing the environment include: 6 | 7 | * _General configuration_. General configuration includes setting up the basic requirements for the app to even run, things like ensuring that whatever directory it will live in actually exists, creating any required users, setting up security groups, specifying where log files should be stored, exposing (or blocking) the proper ports, setting any required permissions, installing any license or cert files, and updating packages. In short, anything and everything that an app needs at the basic operating system level. 8 | * _Installation of the application stack_. If you're writing a Rails app, for example, you'll need to have the required versions of Ruby, Rails, bundler, and other stack-specific dependencies. The same goes for any other stack. 9 | * _Installation and configuration of the required backing services_. The [12 Factor App](http://12factor.net) describes [backing services](http://12factor.net/backing-services) as "any service the app consumes over the network as part of its normal operation. Examples include datastores (such as MySQL or CouchDB), messaging/queueing systems (such as RabbitMQ or Beanstalkd), SMTP services for outbound email (such as Postfix), and caching systems (such as Memcached)." Backing services can also include third-party services, like Amazon AWS (SQS, dynamodb, etc.), [GitHub](http://developer.github.com/v3/), [Twitter](https://dev.twitter.com/), and [Parse](https://www.parse.com/). Ideally, the "code for a twelve-factor app makes no distinction between local and third party services." 10 | 11 | ## Tools 12 | 13 | The following table lists some of the configuration and environment automation tools you might encounter: 14 | 15 | * [Chef](http://www.getchef.com/chef/). "Chef is built to address the hardest infrastructure challenges on the planet. By modeling IT infrastructure and application delivery as code, Chef provides the power and flexibility to compete in the digital economy." You can find thousands of recipes for installing and configuring a range of services at their [community site](https://www.getchef.com/community/). 16 | * [Fabric](http://docs.fabfile.org/en/1.8/). "Fabric is a Python (2.5-2.7) library and command-line tool for streamlining the use of SSH for application deployment or systems administration tasks." 17 | * [Puppet](http://puppetlabs.com/). "Puppet Labs' software gives systems administrators the operational agility, efficiency, and insight they need to manage dynamic infrastructure, on-premise or in the cloud." 18 | * [CFEngine](http://cfengine.com/community). "CFEngine Community is the Open Source foundation of CFEngine's innovative configuration management technology that helps systems administrators automate and ensure the availability, security and compliance of mission-critical applications and services." 19 | * [Ansible](http://www.ansible.com/home). "Ansible is the simplest way to automate apps and IT infrastructure." 20 | * [Salt](http://www.saltstack.com/). "Fast, scalable and flexible software for data center automation, from infrastructure and any cloud, to the entire application stack" 21 | * [Docker](https://www.docker.io/learn/dockerfile/level1/). "If you're building a Docker image, you can specify a lot of the dependencies by specifying a [dockerfile](https://www.docker.io/learn/dockerfile/level1/) for the container." 22 | * [Juju](https://juju.ubuntu.com/). "Juju lets you define applications as charms that know how to setup your app and relate it to other charms. It has a charmstore which contains many community reviewed charms for existing services. Using juju these charms can be deployed to many existing cloud providers or locally using LXC" 23 | * [Terraform](https://terraform.io). "Terraform allows you to effortlessly combine high-level system providers with your own or with each other – from physical and virtual servers to email and DNS providers." 24 | 25 | ## For More Information 26 | 27 | * [Snowflake Server](http://martinfowler.com/bliki/SnowflakeServer.html) 28 | * [Pets vs. Cattle](http://www.slideshare.net/randybias/pets-vs-cattle-the-elastic-cloud-story) 29 | -------------------------------------------------------------------------------- /monitoring.md: -------------------------------------------------------------------------------- 1 | # The Monitoring Infrastructure Is Critical 2 | 3 | 4 | 5 | The monitoring infrastructure is perhaps the most foreign element in the DDS stack. Of all the parts of the software infrastructure, this was perhaps the most opaque. A disk would fill up, the monitoring system would alert the systems group, and they would quietly fix it without the developer being any the wiser. 6 | 7 | However, as more of the traditional admin functions spread to other parts of the organization, there's an increasing need for developers to view the monitoring infrastructure as just another part of the app. In addition to providing the sorts of critical alerts on failures, many monitoring tools are tailored to the application stack and can be used to pinpoint performance bottlenecks. Beyond architecture, logging key events within the application itself to generate operational-level reporting (from very granular conversions, bevaior tracking, workflow bottlenecks, etc) can yield enormous insight, and frameworks exist to record events at any point within the code. 8 | 9 | Most of these systems have two components: a client and a server. The client is an agent that runs on the server you want to monitor; it is typically installed on the machine by a Chef or Puppet recipe (or Ansible or Salt or whatever tool you are using). Whether it's a daemon or a cron job, the client periodically reports back key metrics to the central server. The server provides the reporting interface, notification systems, and other functions that are required to smoothly maintain a large number of systems. 10 | 11 | ## Tools 12 | 13 | Here are some monitoring tools you might encounter: 14 | 15 | * [New Relic](http://newrelic.com/). New Relic is an application monitoring tool that enables you to simply and easily report metrics from within your app. Once you do, however, you can get a wealth of information about the bottlenecks in your application. 16 | * [Scout](https://scoutapp.com/). Scout is a hosted monitoring tool. It's very simple to set up, although it has far fewer plug-ins than Nagios. 17 | * [PagerDuty](http://www.pagerduty.com/). PagerDuty is an alert system that's designed to allow you to create groups and roles that should be notified for a variety of configurable scenarios. 18 | * [loader.io](http://loader.io/). "Loader.io is a free load-testing service that allows you to stress-test your web-apps/apis with thousands of concurrent connections." 19 | * [Hubot](http://hubot.github.com/). Hubot is a chatbot from GitHub. It allows you to easily create scripts that you can use inside your chatroom (e.g., Campfire or HipChat) to deploy new code, receive messages from the build server, or get messages from your monitoring tools when things go wrong. 20 | * [Nagios](http://www.nagios.org/). Nagios is an open source monitoring tool that has been around for a long time. It has hundreds of client plug-ins that can report all manner of system performance metrics. 21 | * [Icinga](http://www.icinga.org/). Icinga is a next generation Nagios re-implementation designed for distributed monitoring, scalability and extensibility. It also follows perhaps the single most important of principal in all of Monitoring, re-using all those millions of existing Nagios plugins around the world! See also the [Advanced Nagios Plugins Collection](https://github.com/harisekhon/nagios-plugins) for advanced API level monitoring integration covering nearly every major NoSQL & Big Data technology. 22 | * [Kale](http://codeascraft.com/2013/06/11/introducing-kale/). Kale is Etsy's monitoring platform and is "designed to solve the problem of metrics overload." 23 | * [Graphite](http://graphite.readthedocs.org/). Graphite is a tool for "scalable realtime graphing." Once you have a data feed, graphite makes it simpler to get nice charts to spot anomalies. 24 | * [StatsD](https://github.com/etsy/statsd/) is a tool from [Etsy](http://codeascraft.com/2011/02/15/measure-anything-measure-everything/). It's essentially a daemon process that can receive messages from your applications via a UDP port. StatsD receives and parses the message and then aggregates it so that it can be analyzed by another tool (like Graphite). 25 | * [Ganglia](http://ganglia.sourceforge.net/) is a BSD-licensed monitoring tool that provide more granular detail than Nagios. 26 | * [InfluxDB](http://influxdb.com/) is a time series, events, and metrics database. 27 | * [Grafana](http://grafana.org/) is an open source, feature-rich metrics dashboard and graph editor for Graphite, InfluxDB & OpenTSDB. 28 | * [Apache Logging Services](http://logging.apache.org/) includes the popular logging libraries log4j, log4net, etc, as well as the Chainsaw viewer, that developers can use to quickly log in-application events to files, databases table, or external services. 29 | * [Loggly](https://www.loggly.com/) is a cloud logging service (from developers that worked on Splunk) with a generous free tier. 30 | * [Splunk](http://www.splunk.com/). Splunk is a data collection and analytics tool. Splunk offers three versions: Enterprise (on-prem or self-hosted), Cloud (SaaS) and Light (either self-hosted or as a service). 31 | * [Zenoss](http://www.zenoss.org/). Zenoss Core is an open source tool that provides full stack monitoring. Zenoss, Inc. also offers the commercial product [Zenoss Service Dynamics](http://www.zenoss.com) for both on-prem and as-a-service. 32 | * [ELK Stack](https://www.elastic.co). Elastic's open source products Elasticsearch, Logstash and Kibana are commonly used together and known as the "ELK Stack." Logstash provides data collection and transport; Elasticsearch provides storage, data search and analytics; and Kibana is a data vizualization platform. 33 | -------------------------------------------------------------------------------- /realtime-chat-and-chatbots.md: -------------------------------------------------------------------------------- 1 | # Real-time Chat and Chatbots 2 | 3 | 4 | 5 | Once considered a relic of a bygone era, real-time chat systems are now a huge part of the workflow of most distributed teams. Like texting, chat allows developers to send each other messages in real time and is the place where most routine team communication takes place. (It can be the place where the team culture is formed and maintained.) 6 | 7 | 8 | 9 | In addition to human participants, team chats can include a _chatbot_, which is a program that performs useful functions by monitoring the chatroom for special commands. The chatbot, which appears like anyone else in the room, can do routine tasks such as kicking off a build, locking the production server, reporting errors from the monitoring tools, or providing manpages. And, programmers being programmers, the chatbot can also do things that reflect the culture of the team, like showing pictures of a pug, displaying a squirrel whenever you type "ship it," or providing responses in a Magic 8 Ball form when you ask it questions. The chatbot usually reflects the culture of the team that's using it and can often become an informal mascot for the group. 10 | 11 | 12 | 13 | ## Chat 14 | 15 | These are some of the more popular chat services and resources: 16 | 17 | * [Campfire](https://campfirenow.com/) 18 | * [HipChat](https://www.hipchat.com) 19 | * [Slack](https://slack.com/) 20 | * [SoCoCo](https://www.sococo.com/) 21 | * Google Chat 22 | * [The Emoji Cheatsheet](http://www.emoji-cheat-sheet.com/) is a fun resource that lists text shortcuts that correspond to the emoji characters used in many chat services. For example, using the code ":boom:" creates a small explosion icon when it's rendered in chat. 23 | 24 | ## Chatbots 25 | 26 | These are resources for creating chatbots: 27 | 28 | * [Hubot](https://hubot.github.com/) is a Node.js chatbot engine from GitHub. It provides the essential services like logging into the chat service, listening for commands directed its way, executing corresponding scripts, and returning output. 29 | * [Hubot Script Catalog](http://hubot-script-catalog.herokuapp.com/) is a directory of useful (and totally useless!) scripts that can be plugged into the Hubot framework. 30 | * [Lita] (https://www.lita.io/) is a chat bot similar to Hubot but written in Ruby with persistent storage provided by Redis. It uses a plugin system to connect to different chat services and to provide new behavior. The plugin system uses the familiar tools of the Ruby ecosystem: RubyGems and Bundler. 31 | 32 | 33 | ## For More Information 34 | 35 | * [Say "Hello" to Hubot](https://github.com/blog/968-say-hello-to-hubot) is a blog post from GitHub, the company that developed the popular Hubot chatbot. 36 | * [ChatOps at GitHub](https://www.youtube.com/watch?v=NST3u-GjjFw) is a video explaining the chatbot-based ops workflow at GitHub. 37 | -------------------------------------------------------------------------------- /signup.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | The Distributed Developer Stack Field Guide 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 22 | 23 | 24 | 25 |
26 | 27 |

The Distributed Developer Stack Field Guide

28 |
29 | 30 | 31 | 32 | 33 |
34 | 35 |
36 |
37 |

38 | 39 | The definitive source for mainstream technology practitioners to discover and understand the tools and concepts prominent in the shift to the distributed development stack (DDS).

40 |

41 |

42 |
 
43 |
44 | Read online 45 |
46 |
 
47 | 48 |
49 |
50 |
 
51 | 96 | 97 |
98 |
99 | 100 |
101 |

Some key ideas we're exploring...

102 | 103 | 104 | 105 | 106 | 107 | 108 |
109 | 113 |
 
114 | 118 |
119 | 120 | 121 | 122 |
123 |
124 | 125 | The codebase is in git. 126 |
127 |
 
128 | 132 |
133 | 134 | 135 | 136 |
137 | 141 |
 
142 | 146 | 147 |
148 | 149 |
150 | 154 |
 
155 | 159 |
160 | 161 |
162 |
163 | 164 | Realtime chat and chatbots. 165 |
166 |
 
167 |
168 |   169 |
170 |
171 | 172 | 173 |
174 | 175 | 176 | 177 | 178 | 179 | 180 | 181 | 182 | 183 | 184 | 185 | 186 | -------------------------------------------------------------------------------- /survey-form.html: -------------------------------------------------------------------------------- 1 |
2 |

Survey

3 | 4 |

This field guide is a work in progress, and to make it a really useful tool, we really need your feedback. Please take a few minutes to fill out this short survey. Many thanks for your help. 

5 | 6 | {% if format == "html" %} 7 | 8 | 9 | 10 | {% else %} 11 | 12 | http://oreil.ly/FGFeedback 13 | 14 | {% endif %} 15 | 16 |
17 | -------------------------------------------------------------------------------- /testing-tools.md: -------------------------------------------------------------------------------- 1 | # Tests Done in Code, Not by a QA Department 2 | 3 | 4 | 5 | As described in the section on continuous deployment servers, running automated tests on each deploy or commit is an important way to ensure software quality. 6 | 7 | ## Tools 8 | 9 | The following table lists testing tools you're likely to encounter: 10 | 11 | * [CasperJS](http://casperjs.org/). "CasperJS is an open source navigation scripting and testing utility written in Javascript for the PhantomJS WebKit headless browser and SlimerJS (Gecko). It eases the process of defining a full navigation scenario and provides useful high-level functions, methods, and syntactic sugar for doing common tasks." 12 | * [PhantomJS](http://phantomjs.org/). "PhantomJS is a headless WebKit scriptable with a JavaScript API. " 13 | * [Canned](https://github.com/sideshowcoder/canned). "Server to respond with fake API responses by using a directory of files for finding out what to say." 14 | * [Selenium](http://www.seleniumhq.org). "Selenium automates browsers. Primarily, it is for automating web applications for testing purposes, but is certainly not limited to just that." 15 | * [Watir](http://watir.com). "Watir, pronounced water, is an open-source (BSD) family of Ruby libraries for automating web browsers." 16 | * [Cucumber](http://cukes.info). "Cucumber is a tool that executes plain-text functional descriptions as automated tests. Cucumber itself is written in Ruby, but it can be used to “test” code written in Ruby or other languages including but not limited to Java, C# and Python." 17 | * [Behat](http://docs.behat.org). "Behat is an open source behavior-driven development framework for PHP 5.3 and 5.4." 18 | -------------------------------------------------------------------------------- /theme/html/html.css: -------------------------------------------------------------------------------- 1 | body { 2 | margin: 0; 3 | padding: 0; 4 | font-family: 'Open Sans', sans-serif; 5 | font-size: 16px; 6 | } 7 | 8 | /* Fork me banner 9 | ----------------------------------------------------------------- */ 10 | .fork-on-github { 11 | z-index: 999999999; 12 | position: absolute; 13 | top: 0; 14 | right: 0; 15 | border: 0; 16 | } 17 | 18 | /* Logo 19 | ----------------------------------------------------------------- */ 20 | .logo { 21 | margin-left: 80px; 22 | margin-right: 40px; 23 | width: 120px; 24 | float: left; 25 | } 26 | 27 | 28 | /* Fonts 29 | ----------------------------------------------------------------- */ 30 | 31 | a { 32 | color: #2972E0; 33 | text-decoration: none; 34 | } 35 | 36 | /* Header 37 | ----------------------------------------------------------------- */ 38 | 39 | header { 40 | position: relative; 41 | border-bottom: 1px solid #E0E0E0; 42 | } 43 | 44 | header h1 { 45 | font-weight: 300; 46 | font-size: 20px; 47 | color: gray; 48 | } 49 | 50 | header #buttons { 51 | position: absolute; 52 | top: 3px; 53 | left: 20px; 54 | } 55 | 56 | header #buttons a { 57 | color: gray; 58 | font-weight: bold; 59 | font-size: 15px; 60 | text-decoration: none; 61 | } 62 | 63 | /* Skeleton 64 | ----------------------------------------------------------------- */ 65 | 66 | #flex-box { 67 | display: flex; 68 | flex: 1 100%; 69 | } 70 | 71 | #flex-box #sidebar { 72 | flex: 0 0 250px; 73 | border-right: 1px solid #E0E0E0; 74 | } 75 | 76 | #flex-box #content { 77 | flex: 1; 78 | } 79 | 80 | /* TOC 81 | ----------------------------------------------------------------- */ 82 | 83 | /* turn sidebar off by default */ 84 | #sidebar { 85 | display: none; 86 | } 87 | 88 | /* First Level */ 89 | 90 | nav[data-type="toc"] ol { 91 | list-style: none; 92 | margin: 0; 93 | padding: 0; 94 | margin-top: 10px; 95 | } 96 | 97 | nav[data-type="toc"] li { 98 | padding: 8px 10px 8px 15px; 99 | } 100 | 101 | nav[data-type="toc"] a { 102 | color: #686868; 103 | font-size: 14px; 104 | } 105 | 106 | /* Second Level */ 107 | 108 | nav[data-type="toc"] li li { 109 | display: none; 110 | font-size: 13px; 111 | list-style-type: lower-roman; 112 | list-style-position: inside; 113 | } 114 | 115 | nav[data-type="toc"] li li a { 116 | font-size: 13px; 117 | } 118 | 119 | /* Third Level */ 120 | 121 | nav[data-type="toc"] li ol ol { 122 | display: none; 123 | } 124 | 125 | /* Page Navigation 126 | ----------------------------------------------------------------- */ 127 | 128 | ul.page-navigation { 129 | list-style: none; 130 | margin: 0; 131 | padding: 0; 132 | margin-top: 15px; 133 | margin-bottom: 15px; 134 | } 135 | 136 | ul.page-navigation li { 137 | margin: 0; 138 | padding: 0; 139 | display: inline-block; 140 | width: 45%; 141 | } 142 | 143 | ul.page-navigation li:nth-child(2) { 144 | text-align: right; 145 | } 146 | 147 | /* Content 148 | ----------------------------------------------------------------- */ 149 | 150 | #content-body { 151 | width: 750px; 152 | margin: 0 auto; 153 | } 154 | 155 | #content-body h1 { 156 | font-size: 35px; 157 | margin-bottom: 0.8em; 158 | } 159 | 160 | #content-body p { 161 | line-height: 1.6em; 162 | } 163 | 164 | #content-body pre { 165 | background-color: #F5F5F5; 166 | border: 1px solid #CCC; 167 | border-radius: 5px; 168 | padding: 10px; 169 | white-space: pre-wrap; 170 | } 171 | 172 | #content-body img { 173 | width: 100%; 174 | } 175 | 176 | -------------------------------------------------------------------------------- /theme/html/layout.html: -------------------------------------------------------------------------------- 1 | {{ doctype }} 2 | 3 | 4 | 5 | {{ title }} 6 | 7 | 8 | 9 | 10 | 11 | 19 | {{ header }} 20 | 21 | 22 | {% if github_url %} 23 | 24 | 25 | 26 | {% endif %} 27 |
28 |
29 | 30 |
31 | 32 |

{{ title }}

33 |
34 |
35 | 38 |
39 |
40 | 44 | {{ content }} 45 | 49 |
50 |
51 |
52 | 53 | -------------------------------------------------------------------------------- /theme/pdf/.gitkeep: -------------------------------------------------------------------------------- 1 | . -------------------------------------------------------------------------------- /theme/pdf/pdf.css: -------------------------------------------------------------------------------- 1 | /* customized TOC for shorter content */ 2 | nav[data-type="toc"] li[data-type="chapter"], 3 | nav[data-type="toc"] li[data-type="preface"], 4 | nav[data-type="toc"] li[data-type="appendix"], 5 | nav[data-type="toc"] li[data-type="glossary"], 6 | nav[data-type="toc"] li[data-type="index"] 7 | /*nav[data-type="toc"] ul li.preface.afterword*/ { 8 | padding-right: 0.25in; /* long chapter names */ 9 | } 10 | 11 | /* hide "Chapter" in TOC, etc. */ 12 | nav[data-type="toc"] li[data-type="chapter"]:before { 13 | content: none; 14 | } 15 | 16 | nav[data-type="toc"] li[data-type="appendix"]::marker { 17 | content: none; 18 | } 19 | 20 | /* customize footers for shorter content */ 21 | @page chapter:left { 22 | @bottom-left { 23 | content: counter(page)"\2003\007C\2003"string(ChapterTitle); 24 | } 25 | } 26 | 27 | @page chapter:right { 28 | @bottom-right { 29 | content: string(SectionTitle)"\2003\007C\2003"counter(page); 30 | } 31 | } 32 | 33 | /* customize chapter/appendix openers to match toc */ 34 | section[data-type="chapter"] > h1:before { 35 | content: none; 36 | } 37 | 38 | section[data-type="appendix"] > h1:before { 39 | content: none; 40 | } 41 | 42 | /* reset first chapter page number */ 43 | nav[data-type="toc"] + section[data-type="chapter"] { counter-reset: page 1 } 44 | 45 | /* customize title page to fit */ 46 | section[data-type="halftitlepage"] h1, 47 | section[data-type="titlepage"] h1 { 48 | 49 | font-size: 20pt; /* Animal Books @ 31.5, ~ 2.15 / 2.8 Ratio */ 50 | 51 | } 52 | 53 | /* Add Extra 1/4" trim for spiral binding */ 54 | /*Only use this for Print PDF version to be spiral bound*/ 55 | /* @page { size: 4.5in 7in } 56 | @page :left { margin-right: 0.75in } 57 | @page :right { margin-left: 0.75in } 58 | */ 59 | -------------------------------------------------------------------------------- /titlepage.html: -------------------------------------------------------------------------------- 1 |
2 |

{{ title }}

3 |

Andrew Odewahn

4 | 5 |
6 | -------------------------------------------------------------------------------- /toc.html: -------------------------------------------------------------------------------- 1 | {{ toc }} 2 | --------------------------------------------------------------------------------