├── .gitignore ├── README.md ├── Vagrantfile └── docker_mongodb_cluster ├── mongo ├── Dockerfile ├── js │ ├── addDBs.js │ ├── addIndexes.js │ ├── addShard.js │ ├── enabelSharding.js │ ├── initiate.js │ ├── setupReplicaSet1.js │ ├── setupReplicaSet2.js │ └── setupReplicaSet3.js └── start.sh └── start_cluster.sh /.gitignore: -------------------------------------------------------------------------------- 1 | .vagrant/ 2 | log/ 3 | user-data 4 | config.rb 5 | mongodata/ 6 | docker_mongodb_cluster/mongodata/ -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Instant MongoDB sharded cluster 2 | This repository provides a Vagrantfile, Dockerfile and a bootstrap script to create MongoDB Cluster using a virtual machine built on Virtualbox software hypervisor. After setup is complete shared MongoDB sharded cluster on a single virtual machine running on your local machine. 3 | 4 | MongoDB cluster consisted of the following docker containers 5 | 6 | - **mongos1-3r1-3**: Mongod server with three replica sets distributes on six mongo containers. 7 | - **configservers1-3**: Stores metadata for sharded cluster distributed on three mongo containers. 8 | - **mongos1**: Mongo routing service installed on one mongo container. 9 | - **skydock**: Used for service discovery and inserts internal docker images records into skydns. 10 | - **skydns**: Used as internal DNS for containers. 11 | 12 | Unfortunately there are some hard coded timeouts due to timing issues with MongoDB. 13 | 14 | ## Installation: 15 | 16 | ### Linux 17 | Install Virtualbox based off the [installation instructions](https://www.virtualbox.org/wiki/Linux_Downloads). 18 | 19 | ### MacOS 20 | 21 | #### Install Homebrew 22 | First, install [Homebrew](http://brew.sh/). 23 | 24 | ruby -e "$(curl -fsSL https://raw.github.com/mxcl/homebrew/go)" 25 | 26 | #### Install Virtualbox and Vagrant 27 | Install VirtualBox and Vagrant using [Brew Cask](https://github.com/phinze/homebrew-cask). 28 | 29 | brew tap phinze/homebrew-cask 30 | brew install brew-cask 31 | brew cask install virtualbox 32 | brew cask install vagrant 33 | 34 | ## Check out the repository 35 | 36 | git clone git@github.com:jacksoncage/mongo-docker.git 37 | cd mongo-docker 38 | vagrant up 39 | 40 | ## Setup Cluster 41 | This will pull all the images from [Docker index](https://index.docker.io/u/jacksoncage/mongo/) and setup Mongodb sharded cluster. Please make sure `DOCKERIP` in `docker_mongodb_cluster/start_cluster.sh` is using docker0 interface or public IP if you prefer that. 42 | 43 | vagrant ssh 44 | ./docker/start_cluster.sh" 45 | 46 | You should now be able connect to mongos1 and the new sharded cluster: 47 | 48 | MongoDB Cluster is now ready to use 49 | Connect to cluster by: 50 | $ mongo --port 49550 51 | 52 | 53 | ## Kill/restart cluster 54 | To re-initiate cluster run `start_cluster.sh` again. To rebuild vagrant `vagrant destroy` and then `vagrant up` 55 | 56 | 57 | ## Persistent storage 58 | Data is stored at `./docker_mongodb_cluster/mongodata/` and are excluded from version control. Data will be persistent even for a `vagrant destroy` as it's mounted into vagrant and then needed container as well. To remove all data `rm -rf ./docker_mongodb_cluster/mongodata/*` 59 | 60 | ## Issues 61 | - **Shard folders** - shared folder may not work at some system with certain vagrant and virtualbox version (this issue is not fixed until now), so users may need to use some other ways to upload .sh and .js files in `docker_mongodb_cluster/mongo/js` 62 | 63 | 64 | ## Built upon 65 | - [MongoDB Sharded Cluster by Sebastian Voss](https://github.com/sebastianvoss/docker) 66 | - [MongoDB](http://www.mongodb.org/) 67 | - [Skydock](https://github.com/crosbymichael/skydock) 68 | - [Skydns](https://github.com/skynetservices/skydns) 69 | - [Docker](https://github.com/dotcloud/docker/) 70 | -------------------------------------------------------------------------------- /Vagrantfile: -------------------------------------------------------------------------------- 1 | # -*- mode: ruby -*- 2 | # vi: set ft=ruby : 3 | 4 | VAGRANTFILE_API_VERSION = "2" 5 | 6 | Vagrant.configure(VAGRANTFILE_API_VERSION) do |config| 7 | config.vm.box = "phusion-open-ubuntu-14.04-amd64" 8 | config.vm.box_url = "https://oss-binaries.phusionpassenger.com/vagrant/boxes/latest/ubuntu-14.04-amd64-vbox.box" 9 | # Or, for Ubuntu 12.04: 10 | #config.vm.box = "phusion-open-ubuntu-12.04-amd64" 11 | #config.vm.box_url = "https://oss-binaries.phusionpassenger.com/vagrant/boxes/latest/ubuntu-12.04-amd64-vbox.box" 12 | 13 | config.vm.provider :vmware_fusion do |f, override| 14 | override.vm.box_url = "https://oss-binaries.phusionpassenger.com/vagrant/boxes/latest/ubuntu-14.04-amd64-vmwarefusion.box" 15 | #override.vm.box_url = "https://oss-binaries.phusionpassenger.com/vagrant/boxes/latest/ubuntu-12.04-amd64-vmwarefusion.box" 16 | end 17 | 18 | if Dir.glob("#{File.dirname(__FILE__)}/.vagrant/machines/default/*/id").empty? 19 | # Install Docker 20 | pkg_cmd = "wget -q -O - https://get.docker.io/gpg | apt-key add -;" \ 21 | "echo deb http://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list;" \ 22 | "apt-get update -qq; apt-get install -q -y --force-yes lxc-docker;" 23 | # Add vagrant user to the docker group 24 | pkg_cmd << "usermod -a -G docker vagrant; " 25 | config.vm.provision :shell, :inline => pkg_cmd 26 | end 27 | 28 | config.vm.network :private_network, ip: "172.17.8.100" 29 | config.vm.synced_folder "./docker_mongodb_cluster", "/home/vagrant/docker", id: "vagrant", :nfs => true, :mount_options => ['nolock,vers=3,udp'] 30 | # Forward docker ports 31 | for i in 49150..49299 32 | config.vm.network :forwarded_port, guest: i, host: i 33 | end 34 | config.vm.network :forwarded_port, guest: 53, host: 49300 35 | end -------------------------------------------------------------------------------- /docker_mongodb_cluster/mongo/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:14.04.1 2 | MAINTAINER Love Nyberg "love@bloglovin.com" 3 | ENV REFRESHED_AT 2014-10-18 4 | 5 | # Add 10gen official apt source to the sources list 6 | RUN apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 7F0CEB10 7 | RUN echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' | tee /etc/apt/sources.list.d/mongodb.list 8 | 9 | # Install MongoDB 10 | RUN apt-get update -qq && \ 11 | apt-get upgrade -yqq && \ 12 | apt-get -yqq install mongodb-org && \ 13 | apt-get -yqq clean 14 | 15 | # Create the MongoDB data directory 16 | RUN mkdir -p /data/db 17 | 18 | EXPOSE 27017 19 | 20 | ADD js/initiate.js /initiate.js 21 | ADD js/setupReplicaSet1.js /setupReplicaSet1.js 22 | ADD js/setupReplicaSet2.js /setupReplicaSet2.js 23 | ADD js/setupReplicaSet3.js /setupReplicaSet3.js 24 | ADD js/addShard.js /addShard.js 25 | ADD js/addDBs.js /addDBs.js 26 | ADD js/enabelSharding.js /enabelSharding.js 27 | ADD js/addIndexes.js /addIndexes.js 28 | ADD start.sh /start.sh 29 | 30 | CMD ["/start.sh"] 31 | -------------------------------------------------------------------------------- /docker_mongodb_cluster/mongo/js/addDBs.js: -------------------------------------------------------------------------------- 1 | c = { n : "m" }; 2 | db.testData.insert(c); 3 | db.testData.remove({"n" : "m"}); -------------------------------------------------------------------------------- /docker_mongodb_cluster/mongo/js/addIndexes.js: -------------------------------------------------------------------------------- 1 | db.testData.ensureIndex( { b: 1,_id:-1 } ); -------------------------------------------------------------------------------- /docker_mongodb_cluster/mongo/js/addShard.js: -------------------------------------------------------------------------------- 1 | sh.addShard("set1/mongos1r1.mongo.dev.docker:27017"); 2 | sh.addShard("set2/mongos2r1.mongo.dev.docker:27017"); 3 | sh.addShard("set3/mongos3r1.mongo.dev.docker:27017"); -------------------------------------------------------------------------------- /docker_mongodb_cluster/mongo/js/enabelSharding.js: -------------------------------------------------------------------------------- 1 | db.runCommand( { enableSharding : "database" } ); 2 | sh.shardCollection( "database.testData", { _id: "hashed" } ); -------------------------------------------------------------------------------- /docker_mongodb_cluster/mongo/js/initiate.js: -------------------------------------------------------------------------------- 1 | rs.initiate(); -------------------------------------------------------------------------------- /docker_mongodb_cluster/mongo/js/setupReplicaSet1.js: -------------------------------------------------------------------------------- 1 | rs.add('mongos1r2.mongo.dev.docker:27017'); 2 | cfg = rs.conf(); 3 | cfg.members[0].host = 'mongos1r1.mongo.dev.docker:27017'; 4 | rs.reconfig(cfg); -------------------------------------------------------------------------------- /docker_mongodb_cluster/mongo/js/setupReplicaSet2.js: -------------------------------------------------------------------------------- 1 | rs.add('mongos2r2.mongo.dev.docker:27017') 2 | cfg = rs.conf() 3 | cfg.members[0].host = 'mongos2r1.mongo.dev.docker:27017' 4 | rs.reconfig(cfg) -------------------------------------------------------------------------------- /docker_mongodb_cluster/mongo/js/setupReplicaSet3.js: -------------------------------------------------------------------------------- 1 | rs.add('mongos3r2.mongo.dev.docker:27017') 2 | cfg = rs.conf() 3 | cfg.members[0].host = 'mongos3r1.mongo.dev.docker:27017' 4 | rs.reconfig(cfg) -------------------------------------------------------------------------------- /docker_mongodb_cluster/mongo/start.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | : "${OPTIONS:=}" # Mongo opptions 3 | 4 | # Start mongo and log 5 | /usr/bin/mongo$OPTIONS -------------------------------------------------------------------------------- /docker_mongodb_cluster/start_cluster.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | # Docker interface ip 4 | DOCKERIP="10.1.42.1" 5 | LOCALPATH="/home/vagrant/docker" 6 | 7 | # Clean up 8 | containers=( skydns skydock mongos1r1 mongos1r2 mongos2r1 mongos2r2 mongos3r1 mongos3r2 configservers1 configservers2 configservers3 mongos1 ) 9 | for c in ${containers[@]}; do 10 | docker kill ${c} > /dev/null 2>&1 11 | docker rm ${c} > /dev/null 2>&1 12 | done 13 | 14 | # Uncomment to build mongo image yourself otherwise it will download from docker index. 15 | #docker build -t jacksoncage/mongo ${LOCALPATH}/mongo > /dev/null 2>&1 16 | 17 | # Setup skydns/skydock 18 | docker run -d -p ${DOCKERIP}:53:53/udp --name skydns crosbymichael/skydns -nameserver 8.8.8.8:53 -domain docker 19 | docker run -d -v /var/run/docker.sock:/docker.sock --name skydock crosbymichael/skydock -ttl 30 -environment dev -s /docker.sock -domain docker -name skydns 20 | 21 | for (( i = 1; i < 4; i++ )); do 22 | # Setup local db storage if not exist 23 | if [ ! -d "${LOCALPATH}/db/${i}-1" ]; then 24 | mkdir -p ${LOCALPATH}/mongodata/${i}-1 25 | mkdir -p ${LOCALPATH}/mongodata/${i}-2 26 | mkdir -p ${LOCALPATH}/mongodata/${i}-cfg 27 | fi 28 | # Create mongd servers 29 | docker run --dns ${DOCKERIP} --name mongos${i}r1 -P -i -d -v ${LOCALPATH}/mongodata/${i}-1:/data/db -e OPTIONS="d --replSet set${i} --dbpath /data/db --notablescan --noprealloc --smallfiles" jacksoncage/mongo 30 | docker run --dns ${DOCKERIP} --name mongos${i}r2 -P -i -d -v ${LOCALPATH}/mongodata/${i}-2:/data/db -e OPTIONS="d --replSet set${i} --dbpath /data/db --notablescan --noprealloc --smallfiles" jacksoncage/mongo 31 | sleep 20 # Wait for mongo to start 32 | # Setup replica set 33 | docker run --dns ${DOCKERIP} -P -i -t -e OPTIONS=" ${DOCKERIP}:$(docker port mongos${i}r1 27017|cut -d ":" -f2) /initiate.js" jacksoncage/mongo 34 | sleep 30 # Waiting for set to be initiated 35 | docker run --dns ${DOCKERIP} -P -i -t -e OPTIONS=" ${DOCKERIP}:$(docker port mongos${i}r1 27017|cut -d ":" -f2) /setupReplicaSet${i}.js" jacksoncage/mongo 36 | # Create configserver 37 | docker run --dns ${DOCKERIP} --name configservers${i} -P -i -d -v ${LOCALPATH}/mongodata/${i}-cfg:/data/db -e OPTIONS="d --configsvr --dbpath /data/db --notablescan --noprealloc --smallfiles --port 27017" jacksoncage/mongo 38 | done 39 | 40 | # Setup and configure mongo router 41 | docker run --dns ${DOCKERIP} --name mongos1 -P -i -d -e OPTIONS="s --configdb configservers1.mongo.dev.docker:27017,configservers2.mongo.dev.docker:27017,configservers3.mongo.dev.docker:27017 --port 27017" jacksoncage/mongo 42 | sleep 15 # Wait for mongo to start 43 | docker run --dns ${DOCKERIP} -P -i -t -e OPTIONS=" ${DOCKERIP}:$(docker port mongos1 27017|cut -d ":" -f2) /addShard.js" jacksoncage/mongo 44 | sleep 15 # Wait for sharding to be enabled 45 | docker run --dns ${DOCKERIP} -P -i -t -e OPTIONS=" ${DOCKERIP}:$(docker port mongos1 27017|cut -d ":" -f2) /addDBs.js" jacksoncage/mongo 46 | sleep 15 # Wait for db to be created 47 | docker run --dns ${DOCKERIP} -P -i -t -e OPTIONS=" ${DOCKERIP}:$(docker port mongos1 27017|cut -d ":" -f2)/admin /enabelSharding.js" jacksoncage/mongo 48 | sleep 15 # Wait sharding to be enabled 49 | docker run --dns ${DOCKERIP} -P -i -t -e OPTIONS=" ${DOCKERIP}:$(docker port mongos1 27017|cut -d ":" -f2) /addIndexes.js" jacksoncage/mongo 50 | 51 | echo "#####################################" 52 | echo "MongoDB Cluster is now ready to use" 53 | echo "Connect to cluster by:" 54 | echo "$ mongo --port $(docker port mongos1 27017|cut -d ":" -f2)" --------------------------------------------------------------------------------