├── .gitignore ├── LICENSE ├── README.md ├── Run.sh ├── bin └── .gitkeep ├── customfeeds └── .gitkeep └── dockerfile ├── Build.sh ├── Dockerfile └── diffconfig /.gitignore: -------------------------------------------------------------------------------- 1 | bin/* 2 | !bin/.gitkeep 3 | customfeeds/* 4 | !customfeeds/.gitkeep 5 | releases/ 6 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | The MIT License (MIT) 2 | 3 | Copyright (c) 2015 Doodle3D 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # openwrt-buildroot-example 2 | Example docker environment for building images for a specific project. 3 | 4 | ## Getting started 5 | 6 | ### Dependencies 7 | Make sure you have the following dependencies installed. 8 | - Git ([installation instructions](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git)) 9 | - Docker ([installation instructions](https://docs.docker.com/installation/)) ([Using Docker through Kitematic on OS X](https://github.com/Doodle3D/openwrt-buildroot/blob/master/README.md#using-docker-through-kitematic-on-os-x)) 10 | 11 | ### OpenWrt Buildroot 12 | Checkout this repository and run `Run.sh`. 13 | ```bash 14 | $ git clone https://github.com/Doodle3D/openwrt-buildroot-example.git 15 | $ cd openwrt-buildroot-example 16 | $ ./Run.sh setup 17 | ``` 18 | Use `./Run.sh help` to view available commands. 19 | 20 | ## Background 21 | This buildroot is a sharable build environment to build OpenWRT images with a custom project specific configuration and packages. The resulting image can be flashed to a OpenWRT compatible device, currently specifically the TP-Link MR3020. 22 | Instead of contaminating your machine with lots of build dependencies, which vary per operating system, we put the [OpenWRT buildroot](http://wiki.openwrt.org/about/toolchain) in a separated virtual machine using [Docker](http://docker.com/) ([Understanding docker](http://docs.docker.com/introduction/understanding-docker/)). This makes it easier for a team to work on an OpenWRT project. Because the Docker image (with most of OpenWRT precompiled) can be shared the build time will decrease dramatically. 23 | One of the goals is to make it simple enough for non Docker or Linux experts to grasp and use. 24 | 25 | We created [Run.sh](https://github.com/Doodle3D/openwrt-buildroot/blob/master/Run.sh) to automate the most common use case; building an image and flashing this to a OpenWRT device. 26 | 27 | 1. The *Run.sh* script will clone custom packages, so these can be developed locally. 28 | 2. A [container](http://docs.docker.com/introduction/understanding-docker/#docker-containers) is created the buildroot [image](http://docs.docker.com/introduction/understanding-docker/#docker-images). This image is comparable to a *.img* file used with VirtualBox or an operating system installation disk. We've already uploaded a ready made buildroot image, but you can also build it yourself, see *Image development*. 29 | 3. In this buildroot container, our [Build.sh](https://github.com/Doodle3D/openwrt-buildroot/blob/master/Build.sh) is executed. This will use the local packages (through a [shared volume](http://docs.docker.com/userguide/dockervolumes/#data-volumes)), build the actual OpenWRT image and copy the image to the shared *bin* volume. 30 | 4. Flash this OpenWRT image it to an OpenWRT device that connected to your computer. 31 | 32 | ## Image development 33 | Clone this repository 34 | ```bash 35 | $ git clone https://github.com/Doodle3D/openwrt-buildroot-example.git 36 | $ cd openwrt-buildroot-example 37 | ``` 38 | Build image from Dockerfile (normally this is downloaded from Docker Hub) 39 | This is done according to our [Dockerfile](https://github.com/Doodle3D/openwrt-buildroot/blob/master/Dockerfile). 40 | ```bash 41 | $ docker build -t yourcompany/openwrt-buildroot dockerfile/ 42 | ``` 43 | Run as interactive docker container. 44 | This enables you to access the buildroot container and make changes.
45 | Using our Run.sh: 46 | ``` bash 47 | $ ./Run.sh interactive 48 | ``` 49 | Original command: 50 | ``` bash 51 | $ docker run -t -i -v "$PWD/bin:/home/openwrt/shared/bin" -v "$PWD/customfeeds:/home/openwrt/shared/customfeeds" -u openwrt --name buildroot yourcompany/openwrt-buildroot bash 52 | ``` 53 | ### Common Docker commands 54 | ``` bash 55 | user@container: $ exit # exit container 56 | $ docker ps # show running containers 57 | $ docker ps -a -s # show all containers with size 58 | $ docker start # (re)start container 59 | $ docker attach # attach to container 60 | # docker pull # update an image 61 | ``` 62 | ### Updating image 63 | Since rebuilding the image takes a lot of time, it's usually more convenient to update the image. This can be done by creating a container from an image, updating the container and committing these changes into the image. 64 | ``` bash 65 | # Create & run an image interactively executing bash instead of Build.sh 66 | $ docker run -t -i -u openwrt --name openwrt-buildroot yourcompany/openwrt-buildroot bash 67 | # Make the changes and exit the container... 68 | # Commit the changes back into the image, while restoring the command to Build.sh. You can specify what's changed with a message using the -m flag. 69 | $ docker commit -c "CMD /home/openwrt/bin/Build.sh" -m="{change message}" openwrt-buildroot yourcompany/openwrt-buildroot 70 | ``` 71 | 72 | ## Using Docker through Kitematic on OS X 73 | [Kitematic](http://kitematic.com/) is a gui to make using Docker easier. It creates a Docker host (Virtual machine) in which the Docker deamon, images and containers live. 74 | Our Run.sh script will automatically start the Docker Host. You can also start the Docker host manually using: 75 | ``` 76 | $ docker-machine start dev 77 | ``` 78 | For Docker to access this Docker Host it needs certain Environment variables. Our Run.sh script will export these automatically for that session. You can also include the following line in your `~/.bash_profile` to have these permanently available. 79 | ``` 80 | eval "$(docker-machine env dev)" 81 | ``` 82 | Save and `$ source ~/.bash_profile` to load the changes. 83 | 84 | When docker or docker-machine isn't found, please try: 85 | - Open *Kitematic* 86 | - In the menu click: *Kitematic (Beta)* > *Install Docker Commands* 87 | -------------------------------------------------------------------------------- /Run.sh: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | 3 | VOLUME_BIN="$PWD/bin:/home/openwrt/shared/bin" 4 | VOLUME_FEEDS="$PWD/customfeeds:/home/openwrt/shared/customfeeds" 5 | DOCKER_IMAGE="yourcompany/openwrt-buildroot" 6 | DOCKER_CONTAINER_NAME="buildroot" 7 | INTERACTIVE_DOCKER_CONTAINER_NAME="buildroot-interactive" 8 | FEEDS_DIR="customfeeds" 9 | REPO_BASE="git@github.com:yourcompany/" 10 | REPO_EXT=".git" 11 | #REPO[0]="custompackage" 12 | 13 | DEVICE_NAME="devicename" # example: box 14 | DEVICE_HOSTNAME="device hostname" # example: 192.168.5.1 15 | DEVICE_USER="root" 16 | 17 | ############################################### 18 | function clone() { 19 | echo 'clone...' 20 | 21 | # if dir customfeeds does not exists: create 22 | if [ ! -d "$FEEDS_DIR" ]; then 23 | echo "[ok] creating dir $FEEDS_DIR" 24 | mkdir $FEEDS_DIR 25 | else 26 | echo "[ok] $FEEDS_DIR already exists" 27 | fi 28 | 29 | # into dir 30 | cd "$FEEDS_DIR" 31 | echo "[note] working dir: $PWD" 32 | 33 | # loop repositories, clone if not exists 34 | for repo in "${REPO[@]}" 35 | do 36 | URL="$REPO_BASE""$repo""$REPO_EXT" 37 | if [ ! -d "$repo" ]; then 38 | git clone --recursive $URL 39 | echo "[ok] cloned " $URL 40 | else 41 | echo "[note] repository already exists:" $repo 42 | fi 43 | done 44 | 45 | # back to root dir 46 | cd .. 47 | 48 | # done 49 | echo "[ok] done." 50 | } 51 | 52 | ############################################### 53 | function checkdocker() { 54 | echo 'checking Docker...' 55 | # start with docker host check? 56 | docker info > /dev/null 2>&1 57 | if [ $? -eq 0 ]; then 58 | echo '[ok] Docker is available' 59 | else 60 | echo '[note] Docker not (yet) available' 61 | # checking Docker host availablility 62 | docker-machine inspect dev > /dev/null 2>&1 63 | if [ $? -eq 0 ]; then 64 | echo '[note] Docker host found, starting host' 65 | docker-machine start dev 66 | if [ ! $DOCKER_HOST ]; then 67 | echo '[note] Exporting Docker host environment variables' 68 | eval "$(docker-machine env dev)" 69 | fi 70 | else 71 | echo '[error] Can not start Docker, please install Docker. See: https://docs.docker.com/installation/#installation' 72 | exit 1 73 | fi 74 | fi 75 | } 76 | 77 | ############################################### 78 | function image() { 79 | echo 'Check, pull or build image ...' 80 | 81 | checkdocker 82 | 83 | if [ `docker images | grep "$DOCKER_IMAGE" | wc -l` -gt 0 ]; then 84 | echo '[note] found docker image: '"$DOCKER_IMAGE" 85 | else 86 | echo '[note] image not found, checking docker hub' 87 | docker pull "$DOCKER_IMAGE" 88 | if [ $RESULT -eq 0 ]; then 89 | echo '[ok] pulled docker image successful' 90 | else 91 | echo '[note] docker image not found on docker hub' 92 | echo "> build image" 93 | docker build -t $DOCKER_IMAGE dockerfile/ 94 | fi 95 | fi 96 | } 97 | 98 | ############################################### 99 | function build() { 100 | echo 'build...' 101 | 102 | checkdocker 103 | 104 | BIN="./bin" 105 | FEEDS_DIR="customfeeds" 106 | 107 | # if dir bin does not exists: create 108 | if [ ! -d "$BIN" ]; then 109 | echo "[ok] creating dir $BIN" 110 | mkdir $BIN 111 | else 112 | echo "[ok] $BIN already exists" 113 | 114 | # clear bin 115 | cd $BIN 116 | rm -rf ./* 117 | cd - 118 | echo "[ok] emptied ""$BIN""/*" 119 | fi 120 | 121 | # check if customfeeds dir exists 122 | if [ ! -d "$FEEDS_DIR" ]; then 123 | echo "[error] no $FEEDS_DIR dirs" 124 | exit 1 125 | fi 126 | 127 | #docker 128 | echo 'docker run... (executes Build.sh inside container)' 129 | docker run --rm -v "$VOLUME_BIN" -v "$VOLUME_FEEDS" -u openwrt --name=$DOCKER_CONTAINER_NAME $DOCKER_IMAGE 130 | 131 | # check if succeeded 132 | NUM_BIN_FILES=`ls "$BIN" | wc -l` 133 | if [ $NUM_BIN_FILES -gt 0 ]; then 134 | echo "[ok] new built files are copied succesfully" 135 | else 136 | echo "[error] $BIN is empty" 137 | exit 1 138 | fi 139 | 140 | # done 141 | echo "[ok] done." 142 | } 143 | 144 | ############################################### 145 | function interactive() { 146 | echo 'docker run interactive...' 147 | 148 | if [ `docker ps -aqf "name=$INTERACTIVE_DOCKER_CONTAINER_NAME" | wc -l` -gt 0 ]; then 149 | echo '[note] restarting existing container' 150 | docker start $INTERACTIVE_DOCKER_CONTAINER_NAME 151 | docker attach $INTERACTIVE_DOCKER_CONTAINER_NAME 152 | else 153 | echo '[note] creating new container' 154 | docker run -t -i -v "$VOLUME_BIN" -v "$VOLUME_FEEDS" -u openwrt --name=$INTERACTIVE_DOCKER_CONTAINER_NAME $DOCKER_IMAGE bash 155 | fi 156 | } 157 | 158 | ############################################### 159 | function flash() { 160 | echo 'flash...' 161 | 162 | # how to 163 | echo '[note] connect an ethernet cable to $DEVICE_NAME and host computer...' 164 | read -t 20 -p "Hit ENTER or wait 20 seconds"; 165 | 166 | # make sure device is listed in ~/.ssh/config 167 | SSH_CONFIG="$HOME/.ssh/config" 168 | 169 | echo "> check ssh config file" 170 | if grep -q "$DEVICE_NAME" "$SSH_CONFIG"; then 171 | echo "'$DEVICE_NAME' is already listed in ssh config file" 172 | else 173 | echo "'$DEVICE_NAME' is NOT listed in .ssh/config file: append" 174 | echo "creating ssh config backup: "$SSH_CONFIG"_backup" 175 | sudo cp $SSH_CONFIG $SSH_CONFIG"_backup" 176 | CONFIG=$(printf '\n%s\n\t%s\n\t%s\n\t%s\n\t%s\n' 'Host '$DEVICE_NAME 'Hostname '$DEVICE_HOSTNAME 'User '$DEVICE_USER 'StrictHostKeyChecking no' 'UserKnownHostsFile=/dev/null') 177 | sudo sh -c "echo '$CONFIG' >> $SSH_CONFIG" 178 | echo "> added to ssh config file:$CONFIG" 179 | fi 180 | 181 | 182 | # copy binary to device using scp to /tmp folder 183 | # if a file is supplied use that to flash the device, otherwise choose a file from the bin folder 184 | if [ -f "$1" ]; 185 | then 186 | SOURCE_PATH="$(dirname $1)/" 187 | BINARY="$(basename $1)" 188 | else 189 | SOURCE_PATH="bin/ar71xx/" 190 | BINARY="openwrt-ar71xx-generic-tl-mr3020-v1-squashfs-sysupgrade.bin" 191 | fi 192 | 193 | FOLDER="/tmp/" 194 | 195 | if [ -n "$2" ]; 196 | then 197 | FLAGS="$2" 198 | else 199 | FLAGS="" 200 | fi 201 | 202 | # check if binary exists 203 | if [ -f "$SOURCE_PATH""$BINARY" ]; 204 | then 205 | echo "[ok] binary exists" 206 | else 207 | echo "[error] binary does not exist: ""$SOURCE_PATH""$BINARY" 208 | exit 1 209 | fi 210 | 211 | # ssh copy 212 | echo "> ssh copy ""$BINARY"" to ""$DEVICE_NAME":"$FOLDER" 213 | scp "$SOURCE_PATH""$BINARY" "$DEVICE_NAME":"$FOLDER" 214 | 215 | if [ $? -eq 0 ]; then 216 | # execute sysupgrade command on device 217 | echo "> excecute sysupgrade on ""$DEVICE_NAME" 218 | CMD="sysupgrade -v $FLAGS ""$FOLDER""$BINARY" 219 | ssh $DEVICE_NAME "$CMD" 220 | 221 | # done 222 | echo "[ok] done." 223 | else 224 | echo "[error] Couldn't copy file to $DEVICE_NAME" 225 | exit 1 226 | fi 227 | } 228 | 229 | ############################################### 230 | function update() { 231 | echo 'updating...' 232 | 233 | # update buildroot 234 | git pull 235 | echo "[ok] updated buildroot" 236 | 237 | # if dir customfeeds does not exists: exit 238 | if [ ! -d "$FEEDS_DIR" ]; then 239 | echo "[error] no '$FEEDS_DIR' folder available" 240 | exit 1 241 | fi 242 | 243 | # go into dir 244 | cd "$FEEDS_DIR" 245 | echo "[note] working dir: $PWD" 246 | 247 | # loop repositories, clone if not exists 248 | for repo in "${REPO[@]}" 249 | do 250 | URL="$REPO_BASE""$repo""$REPO_EXT" 251 | if [ -d "$repo" ]; then 252 | cd "$repo" 253 | git pull 254 | echo "[ok] git pulled (updated)" $URL 255 | cd .. 256 | else 257 | echo "[error] repo '$repo' not available" 258 | fi 259 | done 260 | 261 | # back to root dir 262 | cd .. 263 | 264 | # update docker image 265 | docker pull $DOCKER_IMAGE 266 | echo "[ok] updated docker image" 267 | 268 | # done 269 | echo "[ok] done." 270 | 271 | } 272 | 273 | ############################################### 274 | function help() { 275 | echo 'help...' 276 | echo '' 277 | echo 'Usage: ./Run.sh {COMMAND}' 278 | echo '' 279 | echo 'Commands:' 280 | echo " setup clone repo's, build Docker image (when unavailable) and OpenWRT image" 281 | echo ' clone clone repositories in ./customfeeds dir' 282 | echo ' image build Docker image (when unavailable)' 283 | echo ' build build OpenWRT image' 284 | echo ' flash [file [-n]] flash a built OpenWrt image to device over ssh' 285 | echo ' deploy build OpenWRT image and flash to device' 286 | echo ' interactive Create or start and existing buildroot container interactively' 287 | echo ' update Update the buildroot, git pull latest custom feeds and update docker image' 288 | echo ' help show this help' 289 | echo '' 290 | } 291 | 292 | ############################################### 293 | case "$1" in 294 | setup) clone && image && build ;; 295 | clone) clone ;; 296 | build) build ;; 297 | image) image ;; 298 | flash) flash $2 $3 ;; 299 | deploy) build && flash ;; 300 | interactive) interactive ;; 301 | update) update ;; 302 | *) help ;; 303 | esac 304 | -------------------------------------------------------------------------------- /bin/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Doodle3D/openwrt-buildroot-example/6573301105d700d7d481355666853b03e2b9cfde/bin/.gitkeep -------------------------------------------------------------------------------- /customfeeds/.gitkeep: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Doodle3D/openwrt-buildroot-example/6573301105d700d7d481355666853b03e2b9cfde/customfeeds/.gitkeep -------------------------------------------------------------------------------- /dockerfile/Build.sh: -------------------------------------------------------------------------------- 1 | #!/bin/sh 2 | 3 | # set workdir 4 | echo "> set workdir" 5 | cd /home/openwrt/trunk/ 6 | echo "> pwd: $PWD" 7 | 8 | # update feeds 9 | echo "> update and install customfeeds" 10 | scripts/feeds update customfeeds # download from source into feeds folder and parse info to index 11 | scripts/feeds install -a -p customfeeds # make all packages from this feed available for install 12 | 13 | #update .config file 14 | echo "> update config file" 15 | cat /home/openwrt/config/diffconfig >> .config 16 | make defconfig 17 | 18 | echo "> clear bin" 19 | cd bin 20 | rm -rf * 21 | cd - 22 | 23 | # build 24 | # DOCS: V=s : s is stdout+stderr (equal to the old V=99) 25 | # DOCS: -j [jobs]: devide build process over multiple processors 26 | echo "> make" 27 | make V=s -j $(($(getconf _NPROCESSORS_ONLN)+1)) 28 | RESULT=$? 29 | if [ $RESULT -eq 0 ]; then 30 | echo '[ok] Build successful' 31 | 32 | # copy binary to shared folder 33 | echo "> copy bin folder to shared folder" 34 | echo "> path: /home/openwrt/shared/bin" 35 | cp -r bin /home/openwrt/shared/ 36 | 37 | # complete 38 | echo "> complete" 39 | else 40 | echo '[error] Build failed' 41 | fi 42 | -------------------------------------------------------------------------------- /dockerfile/Dockerfile: -------------------------------------------------------------------------------- 1 | FROM ubuntu:14.10 2 | 3 | MAINTAINER info@yourcompany.com 4 | 5 | # install dependencies 6 | RUN apt-get update &&\ 7 | apt-get install -y git-core subversion build-essential gcc-multilib \ 8 | libncurses5-dev zlib1g-dev gawk flex gettext wget unzip python nano &&\ 9 | apt-get clean 10 | 11 | # add user 12 | RUN useradd -m openwrt 13 | RUN echo 'openwrt ALL=NOPASSWD: ALL' > /etc/sudoers.d/openwrt 14 | 15 | # set user 16 | USER openwrt 17 | 18 | # set workdir 19 | WORKDIR /home/openwrt 20 | 21 | # checkout openwrt 22 | RUN git clone git://git.openwrt.org/14.07/openwrt.git trunk 23 | 24 | # set workdir 25 | WORKDIR /home/openwrt/trunk/ 26 | 27 | # copy build script into container 28 | COPY Build.sh /home/openwrt/bin/ 29 | RUN sudo chmod +x /home/openwrt/bin/Build.sh 30 | 31 | # update and install feeds 32 | RUN scripts/feeds update -a 33 | RUN scripts/feeds install -a 34 | 35 | # define .config file 36 | RUN echo CONFIG_TARGET_ar71xx=y > .config #defconfig always needs a target 37 | RUN make defconfig 38 | 39 | # update config file 40 | # NOTE: copy custom config changes (diffconfig) to the default config 41 | RUN mkdir -p /home/openwrt/config/ 42 | COPY diffconfig /home/openwrt/config/diffconfig 43 | RUN cat /home/openwrt/config/diffconfig >> .config 44 | 45 | # NOTE: run defconfig again to apply overrides 46 | RUN make defconfig 47 | 48 | # download sources before make 49 | RUN make download 50 | 51 | # NOTE: run defconfig again to apply overrides (again after download) 52 | RUN make defconfig 53 | 54 | # build in verbose mode 55 | RUN make V=s 56 | 57 | # prepare customfeeds folder (virtual mounted folder) 58 | RUN cp feeds.conf.default feeds.conf 59 | RUN echo 'src-link customfeeds /home/openwrt/shared/customfeeds' >> feeds.conf 60 | RUN cat /home/openwrt/trunk/feeds.conf 61 | RUN mkdir -p /home/openwrt/shared/customfeeds 62 | 63 | # launch at docker run 64 | CMD ["/home/openwrt/bin/Build.sh"] 65 | -------------------------------------------------------------------------------- /dockerfile/diffconfig: -------------------------------------------------------------------------------- 1 | CONFIG_TARGET_ar71xx=y 2 | CONFIG_TARGET_ar71xx_generic=y 3 | CONFIG_TARGET_ar71xx_generic_TLMR3020=ys --------------------------------------------------------------------------------