├── LICENSE ├── README.md ├── defaults └── main.yml ├── files └── etc │ ├── config │ └── openvpn │ └── crontabs │ └── root ├── meta └── main.yml ├── tasks ├── 0-check.yml ├── 1-apt.yml ├── 2-compile.yml ├── 3-extract.yml ├── 4-generator.yml ├── 5-build.yml ├── 6-upload.yml └── main.yml └── templates ├── build └── build-images.sh.jinja2 ├── compile ├── .config.jinja2 ├── compile.sh.jinja2 ├── feeds.conf.jinja2 └── find.sh.jinja2 ├── generator └── shadow.jinja2 ├── uci ├── luci_openwisp.jinja2 └── openwisp.jinja2 └── upload └── upload_firmware.py.j2 /LICENSE: -------------------------------------------------------------------------------- 1 | Copyright (c) 2016, Federico Capoano 2 | All rights reserved. 3 | 4 | Redistribution and use in source and binary forms, with or without 5 | modification, are permitted provided that the following conditions are met: 6 | 7 | * Redistributions of source code must retain the above copyright notice, this 8 | list of conditions and the following disclaimer. 9 | 10 | * Redistributions in binary form must reproduce the above copyright notice, 11 | this list of conditions and the following disclaimer in the documentation 12 | and/or other materials provided with the distribution. 13 | 14 | * Neither the name of OpenWISP nor the names of its 15 | contributors may be used to endorse or promote products derived from 16 | this software without specific prior written permission. 17 | 18 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 19 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 20 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 21 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 22 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 23 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 24 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 25 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 26 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 27 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | ansible-openwisp2-imagegenerator 2 | ================================ 3 | 4 | [![Galaxy](http://img.shields.io/badge/galaxy-openwisp.openwisp2--imagegenerator-blue.svg?style=flat-square)](https://galaxy.ansible.com/openwisp/openwisp2-imagegenerator/) 5 | 6 | This ansible role allows to build several openwisp2 firmware images for different organizations while 7 | keeping track of their configurations. 8 | 9 | **NOTE**: this role has not been tested in the wild yet. 10 | If you intend to use it, try it out and if something goes wrong please proceed to report your issue, 11 | but bear in mind you need to be willing to understand the [build process](#build-process) 12 | and it's inner working in order to make it work for you. 13 | 14 | Required role variables 15 | ======================= 16 | 17 | The following variables are required: 18 | 19 | * `openwisp2fw_source_dir`: indicates the directory of the [OpenWrt](https://openwrt.org/) source that is used during [compilation](#2-compilation) 20 | * `openwisp2fw_generator_dir`: indicates the directory used for the [preparation of generators](#3-preparation-of-generators) 21 | * `openwisp2fw_bin_dir`: indicates the directory used when [building the final images](#4-building-of-final-images) 22 | * `openwisp2fw_organizations`: a list of organizations; see the example `playbook.yml` file in the 23 | [create playbook section](#5-create-playbook-file) to understand its structure 24 | 25 | Usage (tutorial) 26 | ================ 27 | 28 | If you don't know how to use ansible, don't panic, this procedure will guide you step by step. 29 | 30 | If you already know how to use ansible, you can skip this section and jump straight to the 31 | "Install this role" section. 32 | 33 | First of all you need to understand two key concepts: 34 | 35 | * for **"compilation server"** we mean a server which is used to compile the images 36 | * for **"local machine"** we mean the host from which you launch ansible, eg: your own laptop or CI server 37 | 38 | Ansible is a configuration management/automation tool that works by entering the compilation server 39 | via SSH and executing a series of commands. 40 | 41 | ### 1. Install ansible 42 | 43 | Install ansible **on your local machine** if you haven't done already, there are various ways 44 | in which you can do this, but we prefer to use the official python package manager, eg: 45 | 46 | sudo pip install ansible 47 | 48 | If you don't have pip installed see [Installing pip](https://pip.pypa.io/en/stable/installing/) 49 | on the pip documentation website. 50 | 51 | [Installing ansible in other ways](http://docs.ansible.com/ansible/intro_installation.html#latest-release-via-yum) 52 | is fine too, just make sure to install a version of the `2.0.x` series (which is the version with 53 | which we have tested this playbook). 54 | 55 | ### 2. Install this role 56 | 57 | For the sake of simplicity, the easiest thing is to install this role **on your local machine** 58 | via `ansible-galaxy` (which was installed when installing ansible), therefore run: 59 | 60 | sudo ansible-galaxy install openwisp.openwisp2-imagegenerator 61 | 62 | ### 3. Choose a working directory 63 | 64 | Choose a working directory **on your local machine** where to put the configuration of 65 | your firmware images. 66 | 67 | Eg: 68 | 69 | mkdir ~/my-openwisp2-firmware-conf 70 | cd ~/my-openwisp2-firmware-conf 71 | 72 | Putting this working directory under version control is also a very good idea, it will help you to 73 | track change, rollback, set up a CI server to automatically build the images for you and so on. 74 | 75 | ### 4. Create inventory file 76 | 77 | The inventory file is where group of servers are defined. In our simple case we can get away with 78 | defining a group in which we will put just one server. 79 | 80 | Create a new file `hosts` **on your local machine** with the following contents: 81 | 82 | [myserver] 83 | mycompiler.mydomain.com ansible_user= ansible_become_pass= 84 | 85 | Substitute `mycompiler.mydomain.com` with your hostname (ip addresses are allowed as well). 86 | 87 | Also put your SSH user and password respectively in place of `` and `` (must be sudoer and non-root). 88 | These credentials are used during the [Installation of dependencies step](#1-installation-of-dependencies). 89 | 90 | ### 5. Create playbook file 91 | 92 | Create a new playbook file `playbook.yml` **on your local machine** with the following contents: 93 | 94 | ```yaml 95 | # playbook.yml 96 | - hosts: your_host_here 97 | roles: 98 | - openwisp.openwisp2-imagegenerator 99 | vars: 100 | openwisp2fw_source_dir: /home/user/openwisp2-firmware-source 101 | openwisp2fw_generator_dir: /home/user/openwisp2-firmware-generator 102 | openwisp2fw_bin_dir: /home/user/openwisp2-firmware-builds 103 | openwisp2fw_source_targets: 104 | - system: ar71xx 105 | subtarget: generic 106 | profile: Default 107 | - system: x86 108 | subtarget: generic 109 | profile: Generic 110 | openwisp2fw_organizations: 111 | - name: snakeoil # name of the org 112 | flavours: # supported flavours 113 | - standard 114 | luci_openwisp: # /etc/config/luci_openwisp 115 | # other config keys can be added freely 116 | username: "operator" 117 | # clear text password that will be encrypted in /etc/config/luci_openwisp 118 | password: "" 119 | openwisp: # /etc/config/openwisp 120 | # other config keys can be added freely 121 | url: "https://my-openwisp2-instance.com" 122 | shared_secret: "my-openwisp2-secret" 123 | unmanaged: "{{ openwisp2fw_default_unmanaged }}" 124 | # clear text password that will be encrypted in /etc/shadow 125 | root_password: "" 126 | ``` 127 | 128 | This playbook will let you compile firmware images for an organization named `snakeoil` using only 129 | the `standard` flavour (which includes a default OpenWrt image with the standard OpenWISP2 modules) 130 | for two architectures, ar71xx and x86. 131 | 132 | See the section [Role Variables](#role-variables) to know how to customize the available configurations. 133 | 134 | At this stage your directory layout should look like the following: 135 | 136 | ``` 137 | . 138 | ├── hosts 139 | └── playbook.yml 140 | ``` 141 | 142 | ### 6. Run the playbook 143 | 144 | Now is time to **start the compilation of OpenWISP2 Firmware**. 145 | 146 | Launch the playbook **from your local machine** with: 147 | 148 | ansible-playbook -i hosts playbook.yml -e "recompile=1 cores=4" 149 | 150 | You can substitute `cores=4` with the number of cores at your disposal. 151 | 152 | When the playbook is done running you will find the images on **the compilation server** located in 153 | the directory specified in `openwisp2fw_bin_dir`, which in our example is 154 | `/home/user/openwisp2-firmware-builds` with a directory layout like the following: 155 | 156 | ``` 157 | /home/user/openwisp2-firmware-builds 158 | ├── snakeoil/ # (each org has its own dir) 159 | ├── snakeoil/2016-12-02-094316/ar71xx/standard/ # (contains ar71xx images for standard flavour) 160 | ├── snakeoil/2016-12-02-094316/x86/standard/ # (contains x86 images for standard flavour) 161 | └── snakeoil/latest/ # (latest is a symbolic link to the last compilation) 162 | ``` 163 | 164 | Now, if you followed this tutorial and everything worked out, you are ready to customize your 165 | configuration to suit your needs! Read on to find out how to accomplish this. 166 | 167 | Role variables 168 | ============== 169 | 170 | There are many variables that can be customized if needed, take a look at 171 | [the defaults](https://github.com/openwisp/ansible-openwisp2-imagegenerator/blob/master/defaults/main.yml) 172 | for a comprehensive list. 173 | 174 | Some of those variables are also explained in [Organizations](#organizations) and [Flavours](#flavours). 175 | 176 | Organizations 177 | ============= 178 | 179 | If you are working with OpenWISP, there are chances you may be compiling images for different groups 180 | of people: for-profit clients, no-profit organizations or any group of people that can 181 | be defined as an "*organization*". 182 | 183 | Organizations can be defined freely in `openwisp2fw_organizations`. 184 | 185 | For an example of how to do this, refer to the "[example playbook.yml file](#5-create-playbook-file)". 186 | 187 | If you need to add specific files in the filesystem tree of the images of each organization, see 188 | "[Adding files for specific organizations](#adding-files-for-specific-organizations)". 189 | 190 | Flavours 191 | ======== 192 | 193 | A flavour is a combination of packages that are included in an image. 194 | 195 | You may want to create different flavours for your images, for example: 196 | 197 | * `standard`: the most common use case 198 | * `minimal`: an image for device which have little storage space available 199 | * `mesh`: an image with packages that are needed for implementing a mesh network 200 | 201 | By default only a `standard` flavour is available. 202 | 203 | You can define your own flavours by setting `openwisp2fw_image_flavours` - take a look at 204 | [the default variables](https://github.com/openwisp/ansible-openwisp2-imagegenerator/blob/master/defaults/main.yml) 205 | to understand its structure. 206 | 207 | Build process 208 | ============= 209 | 210 | The build process is composed of the following steps. 211 | 212 | ### 1. Installation of dependencies 213 | 214 | **Tag**: `install`. 215 | 216 | In this phase the operating system dependencies needed for the subsequent steps are installed or upgraded. 217 | 218 | ### 2. Compilation 219 | 220 | **Tag**: `compile`. 221 | 222 | The OpenWrt source is compiled in order to produce something 223 | called "[Image Generator](https://openwrt.org/docs/guide-user/additional-software/imagebuilder)". 224 | The *image generator* is an archive that contains the precompiled packages and a special 225 | `Makefile` that will be used to generate the customized images for each organization. 226 | 227 | The source is downloaded and compiled in the directory specified in 228 | `openwisp2fw_source_dir`. 229 | 230 | ### 3. Preparation of generators 231 | 232 | **Tags**: `extract`, `generator` (or `files`). 233 | 234 | During these steps the *image generators* are extracted and prepared 235 | for building different images for different [organizations](#organizations), each organization 236 | can build images for different [flavours](#flavours) (eg: full-featured, minimal, mesh, ecc); 237 | 238 | The images are extracted and prepared in the directory specified in 239 | `openwisp2fw_generator_dir`. 240 | 241 | ### 4. Building of final images 242 | 243 | **Tag**: `build`. 244 | 245 | In this phase a series of images is produced. 246 | 247 | Several images will be built for each architecture, organization and flavour. 248 | This can generate quite a lof of files, therefore use this power with caution: it's probably 249 | better to start with fewer options and add more cases as you go ahead. 250 | 251 | For example, if you choose to use 2 architectures (ar71xx and x86), 2 organizations (eg: A and B) 252 | and 2 flavours (eg: standard and mini), you will get 8 groups of images: 253 | 254 | * **organization**: A / **flavour**: standard / **arch**: ar71xx 255 | * **organization**: A / **flavour**: standard / **arch**: x86 256 | * **organization**: A / **flavour**: mini / **arch**: ar71xx 257 | * **organization**: A / **flavour**: mini / **arch**: x86 258 | * **organization**: B / **flavour**: standard / **arch**: ar71xx 259 | * **organization**: B / **flavour**: standard / **arch**: x86 260 | * **organization**: B / **flavour**: mini / **arch**: ar71xx 261 | * **organization**: B / **flavour**: mini / **arch**: x86 262 | 263 | The images will be created in the directory specified in 264 | `openwisp2fw_bin_dir`. 265 | 266 | ### 5. Upload images to OpenWISP Firmware Upgrader 267 | 268 | **Tag**: `upload`. 269 | 270 | The last step is to upload images to the 271 | [OpenWISP Firmware Upgrader module](https://github.com/openwisp/openwisp-firmware-upgrader). 272 | This step is optional and disabled by default. 273 | 274 | To enable this feature, the variables ``openwisp2fw_uploader`` 275 | and ``openwisp2fw_organizations.categories`` need to be configured 276 | as in the example below: 277 | 278 | ```yaml 279 | - hosts: 280 | - myhost 281 | roles: 282 | - openwisp.openwisp2-imagegenerator 283 | vars: 284 | openwisp2fw_controller_url: "https://openwisp.myproject.com" 285 | openwisp2fw_organizations: 286 | - name: staging 287 | flavours: 288 | - default 289 | openwisp: 290 | url: "{{ openwisp2fw_controller_url }}" 291 | shared_secret: "xxxxx" 292 | root_password: "xxxxx" 293 | categories: 294 | default: 295 | - name: prod 296 | flavours: 297 | - default 298 | openwisp: 299 | url: "{{ openwisp2fw_controller_url }}" 300 | shared_secret: "xxxxx" 301 | root_password: "xxxxx" 302 | categories: 303 | default: 304 | openwisp2fw_uploader: 305 | enabled: true 306 | url: "{{ openwisp2fw_controller_url }}" 307 | token: "" 308 | image_types: 309 | - ath79-generic-ubnt_airrouter-squashfs-sysupgrade.bin 310 | - ar71xx-generic-ubnt-bullet-m-xw-squashfs-sysupgrade.bin 311 | - ar71xx-generic-ubnt-bullet-m-squashfs-sysupgrade.bin 312 | - octeon-erlite-squashfs-sysupgrade.tar 313 | - ath79-generic-ubnt_nanostation-loco-m-xw-squashfs-sysupgrade.bin 314 | - ath79-generic-ubnt_nanostation-loco-m-squashfs-sysupgrade.bin 315 | - ath79-generic-ubnt_nanostation-m-xw-squashfs-sysupgrade.bin 316 | - ar71xx-generic-ubnt-nano-m-squashfs-sysupgrade.bin 317 | - ath79-generic-ubnt_unifiac-mesh-squashfs-sysupgrade.bin 318 | - x86-64-combined-squashfs.img.gz 319 | - x86-generic-combined-squashfs.img.gz 320 | - x86-geode-combined-squashfs.img.gz 321 | - ar71xx-generic-xd3200-squashfs-sysupgrade.bin 322 | ``` 323 | 324 | The following placeholders in the example will have to be substituted: 325 | 326 | - `` is the UUID o the firmware category in OpenWISP Firmware Upgrader 327 | - `` is the REST auth token of a user with permissions to upload images 328 | 329 | You can retrieve the REST auth token by sending a POST request using the Browsable API web interface of OpenWISP: 330 | 331 | 1. Open the browser at `https:///api/v1/user/token/`. 332 | 2. Enter username and password in the form at the bottom of the page. 333 | 3. Submit the form and you will get the REST auth token in the response. 334 | 335 | The upload script creates a new build object and then uploads the firmware images 336 | specified in `image_types`, which have to correspond to the identifiers like 337 | `ar71xx-generic-tl-wdr4300-v1-il-squashfs-sysupgrade.bin` defined in the 338 | [hardware.py file of OpenWISP Firmware Upgrader](https://github.com/openwisp/openwisp-firmware-upgrader/blob/master/openwisp_firmware_upgrader/hardware.py). 339 | 340 | Other important points to know about the `upload_firmware.py` script: 341 | 342 | - The script reads `CONFIG_VERSION_DIST` and `CONFIG_VERSION_NUMBER` 343 | from the `.config` file of the OpenWrt source code to determine the build 344 | version. 345 | - The script will find out if a build with the same version and category already exists 346 | and try to add images to that build instead of creating a new one, if duplicates 347 | are found, a failure message will be printed to the console but the script will not 348 | terminate; this allows to generate images for new hardware models and 349 | add them to existing builds 350 | 351 | Adding files to images 352 | ====================== 353 | 354 | You can add arbitrary files in every generated image by placing these files in a directory named 355 | `files/` in your playbook directory. 356 | 357 | Example: 358 | 359 | ``` 360 | . 361 | ├── hosts 362 | ├── playbook.yml 363 | └── files/etc/profile 364 | ``` 365 | 366 | `files/etc/profile` will be added in every generated image. 367 | 368 | Adding files for specific organizations 369 | ======================================= 370 | 371 | You can add files to images of specific organizations too. 372 | 373 | Let's say you have an organization called `snakeoil` and you want to add a custom banner, 374 | you can accomplish this by creating the following directory structure: 375 | 376 | ``` 377 | . 378 | ├── hosts 379 | ├── playbook.yml 380 | └── organizations/snakeoil/etc/banner 381 | ``` 382 | 383 | Since this step is one of the last steps performed before building 384 | the final images, you can use this feature to overwrite any file 385 | built automatically during the previous steps. 386 | 387 | Extra parameters 388 | ================ 389 | 390 | You can pass the following extra parameters to `ansible-playbook`: 391 | 392 | * `recompile`: wether to repeat the compilation step 393 | * `cores`: number of cores to use during the compilation step 394 | * `orgs`: comma separated list of organization names if you need to 395 | limit the generation of images to specific organiations 396 | 397 | ### Examples 398 | 399 | Recompile with 16 cores: 400 | 401 | ``` 402 | ansible-playbook -i hosts playbook.yml -e "recompile=1 cores=16" 403 | ``` 404 | 405 | Generate images only for organization ``foo``: 406 | 407 | ``` 408 | ansible-playbook -i hosts playbook.yml -e "orgs=foo" 409 | ``` 410 | 411 | Generate images only for organizations ``foo`` and ``bar``: 412 | 413 | ``` 414 | ansible-playbook -i hosts playbook.yml -e "orgs=foo,bar" 415 | ``` 416 | 417 | Run specific steps 418 | ================== 419 | 420 | Since each step in the process is tagged, you can run specific steps by using ansible tags. 421 | 422 | Example 1, run only the preparation of generators: 423 | 424 | ``` 425 | ansible-playbook -i hosts playbook.yml -t generator 426 | ``` 427 | 428 | Example 2, run only the preparation of generators and build steps: 429 | 430 | ``` 431 | ansible-playbook -i hosts playbook.yml -t generator,build 432 | ``` 433 | 434 | Targets with no subtarget 435 | ========================= 436 | 437 | This example shows how to fill `openwisp2fw_source_targets` in 438 | order to compile targets that do not specify a subtarget 439 | (eg: sunxi, ARMv8, QEMU): 440 | 441 | ```yaml 442 | openwisp2fw_source_targets: 443 | # Allwinner SOC, Lamobo R1 444 | - system: sunxi 445 | profile: sun7i-a20-lamobo-r1 446 | # QEMU ARM Virtual Image 447 | - system: armvirt 448 | profile: Default 449 | ``` 450 | 451 | Support 452 | ======= 453 | 454 | See [OpenWISP Support Channels](http://openwisp.org/support.html). 455 | -------------------------------------------------------------------------------- /defaults/main.yml: -------------------------------------------------------------------------------- 1 | # variables holding extra parameters 2 | recompile: false 3 | cores: 1 4 | org_filter: "{% if orgs is defined %}{{ orgs.split(',') }}{% endif %}" 5 | # ssl library 6 | openwisp2fw_ssl_lib: openssl 7 | # apt packages needed for compilation 8 | openwisp2fw_apt_make_dependencies: 9 | - build-essential 10 | - git 11 | - libncurses5-dev 12 | - zlib1g-dev 13 | - unzip 14 | - zstd 15 | - libssl-dev 16 | - subversion 17 | - wget 18 | - gawk 19 | - ccache 20 | - xsltproc 21 | # needed for mkpasswd 22 | - whois 23 | # base OpenWRT repository 24 | openwisp2fw_source_repo: https://github.com/openwrt/openwrt.git 25 | # base OpenWRT version 26 | # accepted values: "HEAD", a branch name, a tag name or a SHA-1 commit hash 27 | openwisp2fw_source_version: openwrt-19.07 28 | # contents of feeds.conf 29 | openwisp2fw_source_feeds: 30 | - method: src-git 31 | name: openwisp 32 | location: https://github.com/openwisp/openwisp-config.git 33 | branch: master 34 | - method: src-git 35 | name: packages 36 | location: https://github.com/openwrt/packages.git 37 | branch: openwrt-19.07 38 | - method: src-git 39 | name: luci 40 | location: https://github.com/openwrt/luci.git 41 | branch: openwrt-19.07 42 | - method: src-git 43 | name: targets 44 | location: https://github.com/openwrt/targets.git 45 | branch: master 46 | # the following packages will be compiled and added 47 | # to all the firmware images by default 48 | openwisp2fw_source_default_packages: 49 | - openwisp-config 50 | - openvpn-{{ openwisp2fw_ssl_lib }} 51 | # the following packages are compiled for convenience 52 | # so they can be added by using playbook variables if needed 53 | openwisp2fw_source_additional_packages: 54 | - wpad 55 | - iwinfo 56 | - iputils-ping 57 | - ip-full 58 | - wget 59 | - kmod-3c59x 60 | - kmod-e100 61 | - kmod-e1000 62 | - kmod-natsemi 63 | - kmod-ne2k-pci 64 | - kmod-pcnet32 65 | - kmod-8139too 66 | - kmod-r8169 67 | - kmod-sis900 68 | - kmod-tg3 69 | - kmod-via-rhine 70 | - kmod-via-velocity 71 | - kmod-button-hotplug 72 | - partx-utils 73 | # other arbitrary configurations added in the OpenWRT .config file 74 | openwisp2fw_source_other_configs: 75 | - CONFIG_BUSYBOX_CUSTOM=y 76 | - CONFIG_BUSYBOX_CONFIG_FEATURE_EDITING_SAVEHISTORY=y 77 | - CONFIG_BUSYBOX_CONFIG_FEATURE_EDITING_SAVE_ON_EXIT=y 78 | - CONFIG_BUSYBOX_CONFIG_FEATURE_REVERSE_SEARCH=y 79 | - CONFIG_BUSYBOX_CONFIG_FEATURE_VI_UNDO=y 80 | # workaround to avoid conflicts between wpad and wpad-basic 81 | - "# CONFIG_PACKAGE_wpad-basic is not set" 82 | openwisp2fw_source_targets: 83 | - system: ar71xx 84 | subtarget: generic 85 | profile: Default 86 | - system: x86 87 | subtarget: generic 88 | profile: Generic 89 | # image flavours (see README to know what a flavour is) 90 | openwisp2fw_image_flavours: 91 | standard: 92 | ar71xx: 93 | packages: "{{ openwisp2fw_default_packages }}" 94 | x86: 95 | packages: "{{ openwisp2fw_default_packages }}" 96 | # convenience variable used in openwisp2fw_image_flavours 97 | openwisp2fw_default_packages: 98 | # these packages will be removed 99 | - -iptables 100 | - -ip6tables 101 | - -ppp 102 | - -ppp-mod-pppoe 103 | - -firewall 104 | - -odhcpd 105 | - -odhcp6c 106 | - -wpad-basic 107 | # these packages will be added 108 | - wpad 109 | - iwinfo 110 | - uhttpd 111 | - uhttpd-mod-ubus 112 | - px5g 113 | - libustream-{{ openwisp2fw_ssl_lib }} 114 | - openvpn-{{ openwisp2fw_ssl_lib }} 115 | - openwisp-config 116 | # default unmanaged list for openwisp-config 117 | openwisp2fw_default_unmanaged: 118 | - system.@led 119 | - network.loopback 120 | - network.@switch 121 | - network.@switch_vlan 122 | openwisp2fw_uploader: 123 | # whether to upload the results to OpenWISP Firmware Upgrader 124 | enabled: false 125 | # OpenWISP server URL 126 | url: '' 127 | # API token of an user who is authorized to upload images 128 | token: '' 129 | # maps categories of fw-upgrader to builds generated by this tool 130 | category_map: 131 | - organization_name: '' # as in `openwisp2fw_organizations.name` 132 | - category_id: '' # the UUID of the firmware category of openwisp fw-upgrader 133 | - flavour: '' # as in `openwisp2fw_organizations. flavours` 134 | # image types to upload 135 | # eg: 136 | # x86-64-combined-squashfs.img.gz 137 | # x86-generic-combined-squashfs.img.gz 138 | # x86-geode-combined-squashfs.img.gz 139 | # ar71xx-generic-xd3200-squashfs-sysupgrade.bin 140 | # ath79-generic-ubnt_unifiac-mesh-squashfs-sysupgrade.bin 141 | image_types: [] 142 | -------------------------------------------------------------------------------- /files/etc/config/openvpn: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/openwisp/ansible-openwisp2-imagegenerator/3117b853954fbc5236b031e6c892b5d0af3fd082/files/etc/config/openvpn -------------------------------------------------------------------------------- /files/etc/crontabs/root: -------------------------------------------------------------------------------- 1 | # DO NOT EDIT THIS FILE - edit the master and reinstall. 2 | # Edit this file to introduce tasks to be run by cron. 3 | # 4 | # Each task to run has to be defined through a single line 5 | # indicating with different fields when the task will be run 6 | # and what command to run for the task 7 | # 8 | # To define the time you can provide concrete values for 9 | # minute (m), hour (h), day of month (dom), month (mon), 10 | # and day of week (dow) or use '*' in these fields (for 'any').# 11 | # Notice that tasks will be started based on the cron's system 12 | # daemon's notion of time and timezones. 13 | # 14 | # Output of the crontab jobs (including errors) is sent through 15 | # email to the user the crontab file belongs to (unless redirected). 16 | # 17 | # For example, you can run a backup of all your user accounts 18 | # at 5 a.m every week with: 19 | # 0 5 * * 1 tar -zcf /var/backups/home.tgz /home/ 20 | # 21 | # For more information see the manual pages of crontab(5) and cron(8) 22 | # 23 | # m h dom mon dow command 24 | -------------------------------------------------------------------------------- /meta/main.yml: -------------------------------------------------------------------------------- 1 | --- 2 | 3 | dependencies: [] 4 | 5 | galaxy_info: 6 | author: nemesisdesign 7 | description: Generate different OpenWISP2 firmware images for several organizations 8 | license: BSD 9 | min_ansible_version: 2.2 10 | platforms: 11 | - name: Debian 12 | versions: 13 | - wheezy 14 | - jessie 15 | galaxy_tags: 16 | - networking 17 | -------------------------------------------------------------------------------- /tasks/0-check.yml: -------------------------------------------------------------------------------- 1 | - name: Ensure required variables are defined 2 | fail: msg="Required variable '{{ item }}' is not defined" 3 | when: item not in vars 4 | with_items: 5 | - openwisp2fw_source_dir 6 | - openwisp2fw_generator_dir 7 | - openwisp2fw_bin_dir 8 | - openwisp2fw_organizations 9 | 10 | - name: Ensure directories used by the playbook exist 11 | file: 12 | dest: "{{ item }}" 13 | state: directory 14 | with_items: 15 | - "{{ openwisp2fw_source_dir }}" 16 | - "{{ openwisp2fw_generator_dir }}" 17 | - "{{ openwisp2fw_bin_dir }}" 18 | 19 | # ensure ansible user has write permissions in the directories it uses 20 | # this saves the annoiance of a "Permission Denied" error at the end of the process 21 | - name: Write test file to check permissions 22 | # this task never fails but registers a variable that is checked later 23 | failed_when: false 24 | register: permission_check 25 | command: "touch {{ item }}/__test__" 26 | with_items: 27 | - "{{ openwisp2fw_source_dir }}" 28 | - "{{ openwisp2fw_generator_dir }}" 29 | - "{{ openwisp2fw_bin_dir }}" 30 | 31 | # remove test file created in the previous task 32 | - name: Remove test file 33 | # always remove __test__ files and ignore failures 34 | failed_when: false 35 | file: 36 | dest: "{{ item }}/__test__" 37 | state: "absent" 38 | with_items: 39 | - "{{ openwisp2fw_source_dir }}" 40 | - "{{ openwisp2fw_generator_dir }}" 41 | - "{{ openwisp2fw_bin_dir }}" 42 | 43 | # fail playbook loud if the variable permission_check contains "Permission denied" 44 | - name: Verify result of permission check 45 | fail: 46 | msg: "{{ ansible_user|default('current user') }} can't write to directory: {{ item.item }}" 47 | when: item.rc != 0 48 | with_items: "{{ permission_check.results }}" 49 | -------------------------------------------------------------------------------- /tasks/1-apt.yml: -------------------------------------------------------------------------------- 1 | - name: Update APT package cache 2 | apt: update_cache=yes 3 | become: true 4 | become_user: root 5 | 6 | - name: Upgrade make dependencies 7 | apt: name={{ item }} state=latest 8 | with_items: "{{ openwisp2fw_apt_make_dependencies }}" 9 | become: true 10 | become_user: root 11 | 12 | # fixes issue described in https://docs.ansible.com/ansible/become.html#becoming-an-unprivileged-user 13 | - name: Install acl if acting as non-root user 14 | become: true 15 | become_user: root 16 | apt: name=acl state=latest 17 | when: ansible_user is not defined or ansible_user != 'root' 18 | -------------------------------------------------------------------------------- /tasks/2-compile.yml: -------------------------------------------------------------------------------- 1 | - name: Update base git repo 2 | git: 3 | repo: "{{ openwisp2fw_source_repo }}" 4 | dest: "{{ openwisp2fw_source_dir }}" 5 | version: "{{ openwisp2fw_source_version }}" 6 | 7 | - name: Set feeds.conf 8 | template: 9 | src: compile/feeds.conf.jinja2 10 | dest: "{{ openwisp2fw_source_dir }}/feeds.conf" 11 | 12 | - name: Prepare default .config 13 | template: 14 | src: compile/.config.jinja2 15 | dest: "{{ openwisp2fw_source_dir }}/.config.default" 16 | 17 | - name: Create find scripts on server 18 | template: 19 | src: compile/find.sh.jinja2 20 | dest: "{{ openwisp2fw_source_dir }}/find-{{ item.system }}-{{ item.subtarget|default('generic') }}.sh" 21 | mode: 0755 22 | with_items: "{{ openwisp2fw_source_targets }}" 23 | 24 | - name: "Remove {{ openwisp2fw_generator_dir }} (cleanup)" 25 | file: 26 | path: "{{ openwisp2fw_generator_dir }}" 27 | state: absent 28 | 29 | - name: "Create {{ openwisp2fw_generator_dir }}" 30 | file: 31 | path: "{{ openwisp2fw_generator_dir }}" 32 | state: directory 33 | 34 | - name: Prepare compile.sh 35 | template: 36 | src: compile/compile.sh.jinja2 37 | dest: "{{ openwisp2fw_source_dir }}/compile.sh" 38 | mode: 0755 39 | 40 | - name: ./compile.sh 41 | command: ./compile.sh 42 | args: 43 | chdir: "{{ openwisp2fw_source_dir }}" 44 | -------------------------------------------------------------------------------- /tasks/3-extract.yml: -------------------------------------------------------------------------------- 1 | - name: Extract copies of imagebuilder archives 2 | shell: | 3 | if [ -f "{{ openwisp2fw_generator_dir }}/{{ item.system }}-{{ item.subtarget|default('generic') }}.tar.zst" ]; then 4 | tar --use-compress-program=unzstd -xf "{{ openwisp2fw_generator_dir }}/{{ item.system }}-{{ item.subtarget|default('generic') }}.tar.zst" -C "{{ openwisp2fw_generator_dir }}" 5 | else 6 | tar -xzf "{{ openwisp2fw_generator_dir }}/{{ item.system }}-{{ item.subtarget|default('generic') }}.tar.gz" -C "{{ openwisp2fw_generator_dir }}" 7 | fi 8 | args: 9 | executable: /bin/bash 10 | creates: "{{ openwisp2fw_generator_dir }}/.archive-{{ item.system }}-{{ item.subtarget|default('generic') }}-created" 11 | with_items: "{{ openwisp2fw_source_targets }}" 12 | 13 | - name: Find imagebuilder archive files 14 | find: 15 | paths: "{{ openwisp2fw_generator_dir }}" 16 | patterns: "{{ item.system }}-{{ item.subtarget | default('generic') }}.tar.*" 17 | use_regex: true # Use regex to match the patterns 18 | register: found_files 19 | with_items: "{{ openwisp2fw_source_targets }}" 20 | 21 | - name: Remove found imagebuilder archive files 22 | file: 23 | path: "{{ item.path }}" 24 | state: absent 25 | with_items: "{{ found_files.results | selectattr('files', 'defined') | map(attribute='files') | list | flatten }}" 26 | 27 | - name: Rename ImageBuilder dirs to the name of their corresponding target system 28 | shell: > 29 | {% if item.subtarget is defined %} 30 | {% set partial = '{0}-{1}'.format(item['system'], item['subtarget']) %} 31 | {% else %} 32 | {% set partial = item['system'] %} 33 | {% endif %} 34 | mv *-{{ partial }}* {{ item.system }}-{{ item.subtarget|default('generic') }} 35 | args: 36 | chdir: "{{ openwisp2fw_generator_dir }}" 37 | creates: "{{ openwisp2fw_generator_dir }}/{{ item.system }}-{{ item.subtarget|default('generic') }}" 38 | with_items: "{{ openwisp2fw_source_targets }}" 39 | -------------------------------------------------------------------------------- /tasks/4-generator.yml: -------------------------------------------------------------------------------- 1 | - name: Create a "files" directory for each organization 2 | file: 3 | path: "{{ openwisp2fw_generator_dir }}/files/{{ item.name }}" 4 | state: directory 5 | with_items: "{{ openwisp2fw_organizations }}" 6 | # support organization filtering 7 | when: not org_filter or item.name in org_filter 8 | 9 | - name: Copy general role files to each organization 10 | copy: 11 | dest: "{{ openwisp2fw_generator_dir }}/files/{{ item.name }}" 12 | src: files/ 13 | with_items: "{{ openwisp2fw_organizations }}" 14 | # support organization filtering 15 | when: not org_filter or item.name in org_filter 16 | 17 | # the following task checks whether /files/ exists 18 | # and stores the result of this check in a variable 19 | - name: check local /files dir 20 | local_action: stat path={{ playbook_dir }}/files 21 | become: false 22 | register: file_dir 23 | 24 | - name: Copy general playbook files to each organization 25 | copy: 26 | dest: "{{ openwisp2fw_generator_dir }}/files/{{ item.name }}" 27 | src: "{{ playbook_dir }}/files/" 28 | with_items: "{{ openwisp2fw_organizations }}" 29 | # perform action only if playbook_dir contains a file directory 30 | # and support organization filtering 31 | when: > 32 | file_dir.stat.exists == true and 33 | (not org_filter or item.name in org_filter) 34 | 35 | - name: /etc/config/openwisp 36 | template: 37 | dest: "{{ openwisp2fw_generator_dir }}/files/{{ item.name }}/etc/config/openwisp" 38 | src: uci/openwisp.jinja2 39 | with_items: "{{ openwisp2fw_organizations }}" 40 | # avoid failure if not using this feature 41 | # and support organization filtering 42 | when: > 43 | item.openwisp is defined and 44 | (not org_filter or item.name in org_filter) 45 | 46 | - name: Encrypt luci passwords 47 | command: "mkpasswd -5 {{ item.luci_openwisp.password }}" 48 | args: 49 | chdir: "{{ openwisp2fw_source_dir }}" 50 | with_items: "{{ openwisp2fw_organizations }}" 51 | # avoid failure if not using this feature 52 | # and support organization filtering 53 | when: > 54 | item.luci_openwisp is defined and 55 | item.luci_openwisp.password is defined and 56 | (not org_filter or item.name in org_filter) 57 | register: encrypted_luci_passwords 58 | 59 | - name: /etc/config/luci_openwisp 60 | template: 61 | dest: "{{ openwisp2fw_generator_dir }}/files/{{ item.item.name }}/etc/config/luci_openwisp" 62 | src: uci/luci_openwisp.jinja2 63 | with_items: "{{ encrypted_luci_passwords.results }}" 64 | # avoid failure if not using this feature 65 | # and support organization filtering 66 | when: > 67 | encrypted_luci_passwords is defined and 68 | encrypted_luci_passwords.results is defined and 69 | item.item.luci_openwisp is defined and 70 | item.item.luci_openwisp.password and 71 | (not org_filter or item.item.name in org_filter) 72 | 73 | - name: Encrypt root passwords 74 | command: "mkpasswd -5 {{ item.root_password }}" 75 | args: 76 | chdir: "{{ openwisp2fw_source_dir }}" 77 | with_items: "{{ openwisp2fw_organizations }}" 78 | register: encrypted_passwords 79 | # avoid failure if not using this feature 80 | # and support organization filtering 81 | when: > 82 | item.root_password is defined and 83 | (not org_filter or item.name in org_filter) 84 | 85 | - name: /etc/shadow 86 | template: 87 | dest: "{{ openwisp2fw_generator_dir }}/files/{{ item.item.name }}/etc/shadow" 88 | src: generator/shadow.jinja2 89 | mode: 0600 90 | with_items: "{{ encrypted_passwords.results }}" 91 | # avoid failure if not using this feature 92 | # and support organization filtering 93 | when: > 94 | encrypted_passwords is defined and 95 | encrypted_passwords.results is defined and 96 | item.item.root_password is defined and 97 | (not org_filter or item.item.name in org_filter) 98 | 99 | # the following task loops over each organization and checks whether 100 | # /organizations// exists 101 | # and stores the result of this check in a variable 102 | - name: check local /organizations dir 103 | local_action: stat path={{ playbook_dir }}/organizations/{{ item.name }} 104 | become: false 105 | register: organization_dirs 106 | with_items: "{{ openwisp2fw_organizations }}" 107 | # support organization filtering 108 | when: not org_filter or item.name in org_filter 109 | 110 | # perform action only if corresponding organization directory exist 111 | # eg: 112 | # if /organizations/ exists 113 | # then include its contents 114 | - name: Copy organization specific files stored in playbook directory 115 | copy: 116 | dest: "{{ openwisp2fw_generator_dir }}/files/{{ item.item.name }}" 117 | src: "{{ playbook_dir }}/organizations/{{ item.item.name }}/" 118 | with_items: "{{ organization_dirs.results }}" 119 | when: item.stat is defined and item.stat.exists == true 120 | 121 | # Temporary workaround for probable bug in OpenWRT (lede-17) imagebuilder 122 | - name: "[WORKAROUND] Create local bin path" 123 | file: 124 | dest: "{{ openwisp2fw_generator_dir }}/{{ item.system }}-{{ item.subtarget|default('generic') }}/bin/targets/{{ item.system }}/{{ item.subtarget|default('generic') }}" 125 | state: directory 126 | with_items: "{{ openwisp2fw_source_targets }}" 127 | -------------------------------------------------------------------------------- /tasks/5-build.yml: -------------------------------------------------------------------------------- 1 | - name: Prepare build-images.sh 2 | template: 3 | src: build/build-images.sh.jinja2 4 | dest: "{{ openwisp2fw_generator_dir }}/build-images.sh" 5 | mode: 0755 6 | 7 | - name: ./build-images.sh 8 | command: ./build-images.sh 9 | args: 10 | chdir: "{{ openwisp2fw_generator_dir }}" 11 | -------------------------------------------------------------------------------- /tasks/6-upload.yml: -------------------------------------------------------------------------------- 1 | - name: Prepare upload script 2 | template: 3 | src: upload/upload_firmware.py.j2 4 | dest: "{{ openwisp2fw_source_dir }}/upload_firmware.py" 5 | mode: 0744 6 | 7 | - name: ./upload_firmware.py 8 | command: ./upload_firmware.py 9 | args: 10 | chdir: "{{ openwisp2fw_source_dir }}" 11 | -------------------------------------------------------------------------------- /tasks/main.yml: -------------------------------------------------------------------------------- 1 | # perform a check of prerequisites before proceding 2 | # "fail fast and early" 3 | - include: 0-check.yml 4 | # these checks must be always executed 5 | tags: [openwisp2fw, check, install] 6 | 7 | # install system dependencies 8 | - include: 1-apt.yml 9 | tags: [openwisp2fw, install] 10 | 11 | # compile OpenWRT in order to produce the ImageBuilder(s) 12 | # these actions are performed only if "recompile=1" is passed as an extra param 13 | # or if recompile == true 14 | - include: 2-compile.yml 15 | tags: [openwisp2fw, compile] 16 | when: recompile 17 | 18 | # extract ImageGenerator archives 19 | - include: 3-extract.yml 20 | tags: [openwisp2fw, extract] 21 | 22 | # prepare image files for all organizations 23 | # but do not compile images yet 24 | - include: 4-generator.yml 25 | tags: [openwisp2fw, generator, files] 26 | 27 | # build images for each organization and flavour 28 | # store binaries in versioned directories 29 | - include: 5-build.yml 30 | tags: [openwisp2fw, build] 31 | 32 | # upload firmware images 33 | # to openwisp-firmware-upgrader 34 | - include: 6-upload.yml 35 | tags: [openwisp2fw, upload] 36 | when: openwisp2fw_uploader.enabled 37 | -------------------------------------------------------------------------------- /templates/build/build-images.sh.jinja2: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | BASEDIR="{{ openwisp2fw_generator_dir }}" 3 | BINDIR="{{ openwisp2fw_bin_dir }}" 4 | START_TIME=$(date +"%Y-%m-%d-%H%M%S") 5 | 6 | {% for target in openwisp2fw_source_targets %} 7 | {% for org in openwisp2fw_organizations %} 8 | {% if not org_filter or org.name in org_filter %} 9 | 10 | system="{{ target.system }}" 11 | subtarget="{{ target.subtarget|default('generic') }}" 12 | profile="{{ target.profile }}" 13 | org="{{ org.name }}" 14 | files_dir=$BASEDIR/files/$org 15 | versioned_dir="$BINDIR/$org/$START_TIME" 16 | latest_link="$BINDIR/$org/latest" 17 | cd $BASEDIR/$system-$subtarget 18 | {% for flavour_string in org.flavours %} 19 | {# build image for target/org/flavour #} 20 | {% set flavour = openwisp2fw_image_flavours[flavour_string] %} 21 | {% if flavour[target.system] is defined %} 22 | 23 | flavour="{{ flavour_string }}" 24 | packages="{{ ' '.join(flavour[target.system].packages) }}" 25 | bin_dir="$versioned_dir/$system/$flavour" 26 | echo "===================================" 27 | echo "$system-$subtarget-$profile / $org / $flavour" 28 | make image PROFILE="$profile" \ 29 | PACKAGES="$packages" \ 30 | FILES="$files_dir" \ 31 | BIN_DIR="$bin_dir" 32 | 33 | {# detect failures #} 34 | 35 | exit_code=$? 36 | if [ "$exit_code" != "0" ]; then 37 | echo "FAILED: $org /$system-$subtarget-$profile / $flavour" 38 | echo "PROFILE=\"$profile\" PACKAGES=\"$packages\" FILES=\"$files_dir\" BIN_DIR=\"$bin_dir\"" 39 | rm -rf $bin_dir 40 | exit 1 41 | fi 42 | {% endif %} 43 | {% endfor %} 44 | 45 | {# link (each org) to latest compilation for convenience #} 46 | 47 | if [ -s $latest_link ]; then 48 | rm $latest_link 49 | fi 50 | ln -s $versioned_dir $latest_link 51 | {% endif %} 52 | {% endfor %} 53 | {% endfor %} 54 | -------------------------------------------------------------------------------- /templates/compile/.config.jinja2: -------------------------------------------------------------------------------- 1 | # enable image builder 2 | CONFIG_IB=y 3 | CONFIG_IB_STANDALONE=y 4 | 5 | # avoid broken feeds in /etc/opkg/distfeeds.conf 6 | # CONFIG_FEED_openwisp is not set 7 | 8 | # additional packages 9 | {% for package in openwisp2fw_source_additional_packages %} 10 | CONFIG_PACKAGE_{{ package }}=y 11 | {% endfor %} 12 | 13 | # openwisp2 related packages 14 | {% for package in openwisp2fw_source_default_packages %} 15 | CONFIG_PACKAGE_{{ package }}=y 16 | {% endfor %} 17 | 18 | # other configurations 19 | {% for config in openwisp2fw_source_other_configs %} 20 | {{ config }} 21 | {% endfor %} 22 | 23 | # allow to build curl with different ssl library 24 | CONFIG_LIBCURL_{{ openwisp2fw_ssl_lib|upper }}=y 25 | -------------------------------------------------------------------------------- /templates/compile/compile.sh.jinja2: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | 4 | # cleaning up, redownloading and reinstalling packages each time 5 | # can avoid many wierd compilation errors that may be caused by 6 | # occasional bugs in package source code; we want this automated process 7 | # to run smoothly with fewer errors as possible, and this general compilation 8 | # phase shouldn't be repeated often - because the firmware images 9 | # are generated from the ImageBuilders compiled by this script - 10 | # therefore we shall prefer a slightly longer initial compilation but 11 | # with less frequent errors 12 | ./scripts/feeds clean -a 13 | ./scripts/feeds update -a 14 | ./scripts/feeds install -a 15 | 16 | # remove eventual failed compilations artifacts and ignore errors 17 | rm .owf2_compiled_* 2>/dev/null || true 18 | 19 | {% for target in openwisp2fw_source_targets %} 20 | {% set control_file = '.owf2_compiled_{0}_{1}'.format(target['system'], 21 | target.get('subtarget', 'generic')) 22 | %} 23 | 24 | {% if target['profile'] not in ['Default', 'Generic'] %} 25 | {% set profile = 'DEVICE_{0}'.format(target['profile']) %} 26 | {% else %} 27 | {% set profile = target['profile'] %} 28 | {% endif %} 29 | 30 | # do not compile imagebuilder for same subtarget more than once because not necessary 31 | if [ ! -f {{ control_file }} ]; then 32 | cp .config.default .config 33 | echo "CONFIG_TARGET_{{ target.system }}=y" >> .config; 34 | {% if target.subtarget is defined %} 35 | echo "CONFIG_TARGET_{{ target.system }}_{{ target.subtarget }}=y" >> .config; 36 | {% set profile_config_partial = '{0}_{1}'.format(target['subtarget'], profile) %} 37 | {% else %} 38 | {% set profile_config_partial = profile %} 39 | {% endif -%} 40 | # profile information can still be necessary to avoid "image is too big" error in some cases 41 | echo "CONFIG_TARGET_{{ target.system }}_{{ profile_config_partial }}=y" >> .config; 42 | make defconfig 43 | make clean 44 | make -j {{ cores }} BUILD_LOG=1 45 | touch {{ control_file }} 46 | ./find-{{ target.system }}-{{ target.get('subtarget', 'generic') }}.sh 47 | fi; 48 | 49 | {% endfor %} 50 | 51 | rm .owf2_compiled_* 52 | -------------------------------------------------------------------------------- /templates/compile/feeds.conf.jinja2: -------------------------------------------------------------------------------- 1 | {% for feed in openwisp2fw_source_feeds %} 2 | {{ feed.method }} {{ feed.name }} {{ feed.location }}{% if feed.branch %};{{ feed.branch }}{% endif %} 3 | 4 | {% endfor %} 5 | -------------------------------------------------------------------------------- /templates/compile/find.sh.jinja2: -------------------------------------------------------------------------------- 1 | #!/bin/bash 2 | set -e 3 | {% if item.subtarget is defined %} 4 | {% set partial = '{0}-{1}'.format(item['system'], item['subtarget']) %} 5 | {% else %} 6 | {% set partial = item['system'] %} 7 | {% endif %} 8 | {% set filename = '{0}/{1}-{2}'.format( 9 | openwisp2fw_generator_dir, 10 | item['system'], 11 | item.get('subtarget', 'generic') 12 | ) %} 13 | 14 | # Move and rename the file while keeping the original extension 15 | find {{ openwisp2fw_source_dir }}/bin/ \ 16 | -iname '*-imagebuilder*{{ partial }}*' \ 17 | -exec bash -c ' 18 | for file; do 19 | base_name=$(basename "$file") 20 | # Extract the extension 21 | extension="${base_name##*.}" 22 | 23 | # Check if there is a second extension (for .tar.zst or .tar.gz) 24 | if [[ "$base_name" == *.*.* ]]; then 25 | # Get the part of the filename before the last extension 26 | base_name_without_last_extension="${base_name%.*}" 27 | # Combine the last two parts to form the full extension 28 | full_extension=".${base_name_without_last_extension##*.}.$extension" 29 | else 30 | full_extension=".$extension" 31 | fi 32 | 33 | destination="{{ filename }}$full_extension" 34 | cp "$file" "$destination" 35 | done 36 | ' bash {} + 37 | -------------------------------------------------------------------------------- /templates/generator/shadow.jinja2: -------------------------------------------------------------------------------- 1 | root:{{ item.stdout }}:17137:0:99999:7::: 2 | daemon:*:0:0:99999:7::: 3 | ftp:*:0:0:99999:7::: 4 | network:*:0:0:99999:7::: 5 | nobody:*:0:0:99999:7::: 6 | dnsmasq:x:0:0:99999:7::: 7 | -------------------------------------------------------------------------------- /templates/uci/luci_openwisp.jinja2: -------------------------------------------------------------------------------- 1 | config openwisp 'gui' 2 | option password '{{ item.stdout }}' 3 | {% for key, value in item.item.luci_openwisp.items() %} 4 | {% if key != 'password' %} 5 | option {{ key }} '{{ value }}' 6 | {% endif %} 7 | {% endfor %} 8 | -------------------------------------------------------------------------------- /templates/uci/openwisp.jinja2: -------------------------------------------------------------------------------- 1 | config controller 'http' 2 | {% for key, value in item.openwisp.items() %} 3 | {% if value is not string and value is iterable %} 4 | {% for list_value in value %} 5 | list {{ key }} '{{ list_value }}' 6 | {% endfor %} 7 | {% else %} 8 | option {{ key }} '{{ value }}' 9 | {% endif %} 10 | {% endfor %} 11 | -------------------------------------------------------------------------------- /templates/upload/upload_firmware.py.j2: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | 3 | import requests 4 | import subprocess 5 | import json 6 | import sys 7 | 8 | # insiel 9 | TOKEN = '{{ openwisp2fw_uploader.token }}' 10 | BASE_URL = '{{ openwisp2fw_uploader.url }}' 11 | # key: filesystem directory 12 | # value[0]: category uuid 13 | # value[1]: flavour 14 | CATEGORIES = { 15 | {% for organization in openwisp2fw_organizations -%} 16 | {% for flavour, category_id in organization.get('categories', {}).items() -%} 17 | '{{ organization.name }}': ('{{ category_id }}', '{{ flavour }}'), 18 | {% endfor %} 19 | {% endfor %} 20 | } 21 | BASE_PATH = '{{ openwisp2fw_bin_dir }}' 22 | 23 | IMAGE_TYPES = {{ openwisp2fw_uploader.image_types | to_json }} 24 | 25 | 26 | def jsonify(data): 27 | return json.dumps(data) 28 | 29 | 30 | def find_config_line(config_name): 31 | with open('.config') as f: 32 | config_contents = f.read() 33 | lines = config_contents.split('\n') 34 | for line in lines: 35 | if line.startswith(config_name): 36 | parts = line.split('=') 37 | return parts[1].replace('"', '') 38 | else: 39 | raise ValueError(f'{config_name} not found') 40 | 41 | 42 | API_URL = f'{BASE_URL}/api/v1/firmware-upgrader/' 43 | BUILD_URL = f'{API_URL}build/' 44 | AUTHORIZATION_HEADER = {'Authorization': f'Bearer {TOKEN}'} 45 | CONTENT_TYPE_HEADER = {'Content-Type': 'application/json'} 46 | HEADERS = AUTHORIZATION_HEADER.copy() 47 | HEADERS.update(CONTENT_TYPE_HEADER) 48 | VERSION_DIST = find_config_line('CONFIG_VERSION_DIST') 49 | VERSION_NUMBER = find_config_line('CONFIG_VERSION_NUMBER') 50 | 51 | out = subprocess.Popen( 52 | ['./scripts/getver.sh'], 53 | stdout=subprocess.PIPE, 54 | stderr=subprocess.STDOUT 55 | ) 56 | stdout, stderr = out.communicate() 57 | 58 | REVISION = str(stdout.decode('utf8').strip()) 59 | OS_IDENTIFIER = f'{VERSION_DIST} {VERSION_NUMBER} {REVISION}' 60 | IMAGE_PREFIX = OS_IDENTIFIER.lower().replace(' ', '-') 61 | 62 | for org_slug, org_data in CATEGORIES.items(): 63 | category_id = org_data[0] 64 | flavour = org_data[1] 65 | 66 | # check if build already exists 67 | response = requests.get( 68 | BUILD_URL, 69 | headers=HEADERS, 70 | params={ 71 | 'version': VERSION_NUMBER, 72 | 'category': category_id, 73 | }, 74 | ) 75 | if response.status_code != 200: 76 | print('It was not possible to fetch current builds.') 77 | print(response.content.decode()) 78 | sys.exit(1) 79 | 80 | if response.json()['count'] > 0: 81 | BUILD_ID = response.json()['results'][0]['id'] 82 | else: 83 | # create build 84 | response = requests.post( 85 | BUILD_URL, 86 | headers=HEADERS, 87 | data=jsonify({ 88 | 'version': VERSION_NUMBER, 89 | 'category': category_id, 90 | 'os': OS_IDENTIFIER 91 | }), 92 | ) 93 | 94 | if response.status_code != 201: 95 | print('It was not possible to create a build.') 96 | print(response.content.decode()) 97 | sys.exit(1) 98 | 99 | BUILD_ID = response.json()['id'] 100 | 101 | UPLOAD_IMAGE_URL = f'{BUILD_URL}{BUILD_ID}/image/' 102 | 103 | for image_file in IMAGE_TYPES: 104 | target = image_file.split('-')[0] 105 | BIN_PATH = f'{BASE_PATH}/{org_slug}/latest/{target}/{flavour}/{IMAGE_PREFIX}-{image_file}' 106 | 107 | try: 108 | BIN_FILE = open(BIN_PATH, 'rb') 109 | except FileNotFoundError as e: 110 | print(f'Skipping {image_file} because of the following error:\n{e}') 111 | continue 112 | 113 | # upload image 114 | response = requests.post( 115 | UPLOAD_IMAGE_URL, 116 | headers=AUTHORIZATION_HEADER, 117 | data={ 118 | 'type': image_file, 119 | 'build': BUILD_ID 120 | }, 121 | files={'file': BIN_FILE}, 122 | ) 123 | if response.status_code != 201: 124 | print( 125 | f'Got error for {org_slug}, {image_file}: ' 126 | f'{response.content.decode("utf8")}; Skiping...' 127 | ) 128 | --------------------------------------------------------------------------------