.
675 |
--------------------------------------------------------------------------------
/README.md:
--------------------------------------------------------------------------------
1 | # Operating System for Private and Anonymous Computation Clusters
2 |
3 | [](http://www.dyne.org)
4 |
5 |
6 |
7 | The DECODE operating system is a brand new GNU+Linux distribution
8 | designed to run on servers, embedded computers and virtual machines to
9 | automatically connect micro-services to a private and anonymous
10 | peer-to-peer network cluster.
11 |
12 |
13 |
14 |
15 |
16 | | Features | Components |
17 | |--------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
18 | | Wide compatibility with industry standards | GNU + Linux minimal base |
19 | | Anonimity and privacy by design | [Tor](https://torproject.org) hidden service family |
20 | | Very secure, restricted environment | [grsec](https://github.com/minipli/linux-unofficial_grsec/wiki) community fork |
21 | | Customisable to run different applications | [Devuan](https://devuan.org) GNU+Linux SDK |
22 | | Pluggable consensus algorithm | [Redis](https://redis.io) based consensus broker |
23 | | Read-only and authenticated system | [SquashFS](http://tldp.org/HOWTO/SquashFS-HOWTO/whatis.html) + [overlayfs](https://www.kernel.org/doc/Documentation/filesystems/overlayfs.txt) + [Btrfs](https://btrfs.wiki.kernel.org/index.php/Main_Page) |
24 | | Integrated updating mechanism | [Roundshot](https://github.com/DECODEproject/roundshot) initramfs |
25 | | Built-in Graphical dashboard | [Netdata](https://github.com/netdata/netdata) resource monitor |
26 | | Low power consumption, outdoor usage | Ports to embedded ARM boards |
27 | | Extensible platform support | Includes latest JDK, Golang, Python etc. |
28 | | Minimal resource consumption | Online with less than 64MB of RAM |
29 |
30 | ## For stable releases see files.dyne.org/decode
31 |
32 | ## For more information see the DECODE project
33 |
34 | In particular, the following publications:
35 |
36 | - Privacy Design Strategies for the DECODE Architecture
37 | - Decode OS first release
38 | - DECODE OS Software Development Kit (soon to be superseeded by the upcoming Devuan's Developer Manual)
39 |
40 | ## Usage instructions
41 |
42 | DECODE OS comes in a variety of flavors:
43 |
44 | - for ARM based boxes (embedded)
45 | - for virtual machines (cloud)
46 | - live desktop (boot from usb)
47 |
48 | Running systems provide a dashboard by connecting using a browser
49 | using HTTP on port 19999.
50 |
51 | The default username is `decode` with password `decode`
52 |
53 | The default `root` password is `toor`.
54 |
55 | ## Get in touch!
56 |
57 | Developers of the Dyne.org foundation are available to support
58 | customisations and adaptations of this operating system for particular
59 | purposes in line with the foundation's goals.
60 |
61 | You are welcome to contact us:
62 |
63 | - **#devuan-dev** on **freenode** IRC (public, logged IPs)
64 | - **#dyne** on irc.dyne.org (public and private, no IPs logged)
65 | - E-mail **info@dyne.org**
66 |
67 | This project is a work in progress proceeding along a clear roadmap
68 | agreed for the DECODE project. The DECODE OS **stable release is planned
69 | for 1st quarter 2019**.
70 |
71 |
72 |
73 | This project is receiving funding from the **European Union’s Horizon
74 | 2020 research and innovation programme under grant agreement
75 | nr. 732546**.
76 |
77 | ## Build from source
78 |
79 | The following instructions illustrate how one can build DECODE OS from
80 | scratch, eventually adding software to it. This section is a work in
81 | progress.
82 |
83 | Building can be done from any GNU+Linux distribution, it entails
84 | bootstrapping a new Devuan base and then customising it via its SDK
85 | using a "blend", root access is needed in order to operate in `chroot`
86 | and in KVM accellerated `qemu`.
87 |
88 | More information on this process is provided by the "Devuan's
89 | Developers Manual", here is an outline on the steps to be taken.
90 |
91 |
92 |
93 | ### System requirements
94 |
95 | A GNU/Linux system is required in order to build DECODE OS.
96 |
97 | Here a list of package dependencies:
98 | ```
99 | zsh sudo cgpt xz-utils qemu qemu-utils
100 | ```
101 |
102 | To clone this repository:
103 |
104 | ```
105 | git clone https://github.com/DECODEproject/os-build-system --recursive
106 | ```
107 |
108 | To update the repository:
109 |
110 | ```
111 | git pull origin master && git submodule update --init --recursive --checkout
112 | ```
113 |
114 |
115 | ### Building for ARM targets
116 |
117 | A more detailed reference for the arm-sdk can be found here: https://git.devuan.org/sdk/arm-sdk
118 |
119 | ```
120 | cd arm-sdk # (or vm-sdk or live-sdk depending from your target)
121 | zsh -f
122 | ./init.sh # and when this is done, execute the command in the bottom of the output
123 | source sdk
124 | load devuan raspi3 decode # (replace "raspi3" with your board name, from the list below)
125 | bootstrap_complete_base
126 | ```
127 |
128 | Here is the list of the supported boxes: https://git.devuan.org/sdk/arm-sdk/blob/master/sdk
129 |
130 |
131 | ### Building for VM targets
132 |
133 |
134 | To enter the build console just run `./console.sh`.
135 |
136 | To build a vagrant virtual machine, run `build_vagrant_dist`.
137 |
138 | To build a live iso image, run `build_iso_dist`.
139 |
140 | To build an ARM installer image, run `build_image_dist`.
141 |
142 | Here below the sequences of build steps executed by each target:
143 |
144 | ```sh
145 | build_image_dist() {
146 | bootstrap_complete_base
147 | blend_preinst
148 | image_prepare_raw
149 | image_partition_raw_${parted_type}
150 | build_kernel_${arch}
151 | blend_postinst
152 | rsync_to_raw_image
153 | image_pack_dist
154 | }
155 |
156 | build_iso_dist() {
157 | bootstrap_complete_base
158 | blend_preinst
159 | iso_prepare_strap
160 | build_kernel_${arch}
161 | iso_setup_isolinux
162 | iso_write_isolinux_cfg
163 | blend_postinst
164 | fill_apt_cache
165 | iso_squash_strap
166 | iso_xorriso_build
167 | }
168 |
169 | build_vagrant_dist() {
170 | image_${imageformat}_as_strapdir
171 | bootstrap_complete_base
172 | vm_inject_overrides
173 | blend_preinst
174 | vm_setup_grub
175 | blend_postinst
176 | vm_umount_${imageformat}
177 | vm_vbox_setup
178 | vm_vagrant_package
179 | vm_pack_dist
180 | }
181 | ```
182 |
183 | The `build_vagrant_dist` target is a helper that executes a sequence
184 | of steps, some of them common to other helpers (hence
185 | combinable). Here below the full list of build steps executed by
186 | `build_vagrant_dist`
187 |
188 | The `bootstrap_complete_base` step creates a base system tarball that
189 | can be reused by any target, it is found inside `*_sdk/tmp` for each
190 | sdk and to save time and computation it can be copied in place for
191 | each sdk if the base system doesn't differ.
192 |
193 | ## Acknowledgments
194 |
195 | DECODE OS is Copyright (c) 2017-2018 by the Dyne.org Foundation
196 |
197 | DECODE OS and its core components are designed, written and maintained
198 | by Denis Roio and Ivan J.
199 |
200 | Devuan is a registered trademark of the Dyne.org foundation.
201 |
202 | The Devuan SDK used to build the DECODE OS was originally conceived
203 | during a period of residency at the Schumacher college in Dartington,
204 | UK. Greatly inspired by the laborious and mindful atmosphere of its
205 | wonderful premises.
206 |
207 | Devuan SDK components are designed, written and maintained by Denis
208 | Roio, Enzo Nicosia and Ivan J.
209 |
210 | This source code is free software: you can redistribute it and/or modify it
211 | under the terms of the GNU General Public License as published by the Free
212 | Software Foundation, either version 3 of the License, or (at your option)
213 | any later version.
214 |
215 | This software is distributed in the hope that it will be useful, but
216 | WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY
217 | or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
218 | more details.
219 |
220 | You should have received a copy of the GNU General Public License along
221 | with this source code. If not, see .
222 |
--------------------------------------------------------------------------------
/config:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env zsh
2 |
3 | blend_name="decode"
4 | blend_vers="1.0.0"
5 | image_name="${blend_name}-${blend_vers}-${arch}"
6 | vm_name="${blend_name}-${blend_vers}-${arch}"
7 |
8 | dyneci_url="https://sdk.dyne.org:4443/job"
9 |
10 |
11 | tomb_version="v2.4"
12 | tomb_url="https://github.com/dyne/tomb.git"
13 |
14 | musl_version="v.1.1.19"
15 | musl_url="git://git.musl-libc.org/musl"
16 |
17 | netdata_version="1.9.0"
18 | netdata_url="https://github.com/firehol/netdata/releases/download/v${netdata_version}/netdata-${netdata_version}.tar.gz"
19 |
20 | stem_version="1.6.0"
21 | stem_url="https://git.torproject.org/stem.git"
22 |
23 | tordam_url="github.com/decodeproject/tor-dam/..."
24 |
25 | zenroom_url="${dyneci_url}/zenroom-static-${arch}/lastSuccessfulBuild/artifact/src/zenroom-static"
26 |
27 | golang_url="${dyneci_url}/go-binaries/lastBuild/architecture=${arch}/artifact/go.${arch}.tar.gz"
28 |
29 | chainspace_url="https://github.com/chainspace/chainspace"
30 | chainspacedist_url="${dyneci_url}/chainspace-jar/lastSuccessfulBuild/artifact/chainspacedist.tgz"
31 |
32 |
33 | size="4098"
34 | filesystem="btrfs"
35 |
36 | TAR_STAGE4=true
37 |
38 | extra_packages+=(
39 | openrc
40 | eudev
41 | zsh
42 | tmux
43 | curl
44 | cryptsetup
45 | openssh-server
46 | psmisc
47 | btrfs-progs
48 | openssh-client
49 | gnupg2
50 | dirmngr
51 | pcsc-tools
52 | tor
53 | redis-tools
54 | redis-server
55 | paxctl
56 | net-tools
57 |
58 | default-jre-headless
59 |
60 | apt-transport-tor
61 | apt-transport-https
62 | deb.torproject.org-keyring
63 |
64 | python2.7
65 | python3
66 |
67 | # minimalism
68 | busybox-static
69 |
70 | # utilities
71 | mlocate
72 | gettext
73 | miscfiles
74 | tree
75 |
76 | # troubleshooting
77 | lsof
78 | htop
79 | iftop
80 | iotop
81 | strace
82 |
83 | python-pip
84 | python3-pip
85 | libpython-dev
86 | libffi-dev
87 | python-tox
88 | python-cffi
89 | python-pytest
90 | python-pytest-cov
91 | python-msgpack
92 | python-numpy
93 | python3-numpy
94 | python-requests
95 | python3-requests
96 | sqlite
97 | sqlite3
98 | libssl1.0-dev
99 |
100 |
101 | # build tools
102 | build-essential
103 | cmake
104 | pkg-config
105 | zlib1g-dev
106 | uuid-dev
107 | autoconf
108 | automake
109 | libtool
110 | gcc-6-plugin-dev
111 | libncurses5-dev
112 | flex
113 | libbison-dev
114 | libtool-bin
115 | libtool
116 | libgcrypt20
117 | libgcrypt20-dev
118 | equivs
119 | )
120 |
121 | purge_packages=(
122 | dbus
123 | elogind
124 | gnome-icon-theme
125 | sysv-rc
126 | )
127 |
128 | finalize_purge_packages=(
129 | #autoconf
130 | #automake
131 | #flex
132 | #equivs
133 | )
134 |
--------------------------------------------------------------------------------
/console.sh:
--------------------------------------------------------------------------------
1 | #!/bin/sh
2 |
3 | if ! [ -r vm-sdk ]; then git submodule update --init --recursive; fi
4 |
5 | sdk="${1:-vm-sdk}"
6 |
7 | cat < $sdk/.zshrc
8 | # local zshrc for easy start of console
9 | # usage: ZDOTDIR=/path/to/vm-sdk zsh
10 | pushd \$ZDOTDIR > /dev/null
11 |
12 | source sdk
13 | load devuan decode
14 | popd > /dev/null
15 | EOF
16 |
17 | ZDOTDIR=$sdk zsh
18 |
--------------------------------------------------------------------------------
/decode.blend:
--------------------------------------------------------------------------------
1 | #!/usr/bin/env zsh
2 | # Copyright (c) 2017-2018 Dyne.org Foundation
3 | #
4 | # decode.blend is written and maintained by Ivan J.
5 | #
6 | # This source code is free software: you can redistribute it and/or modify
7 | # it under the terms of the GNU General Public License as published by
8 | # the Free Software Foundation, either version 3 of the License, or
9 | # (at your option) any later version.
10 | #
11 | # This software is distributed in the hope that it will be useful,
12 | # but WITHOUT ANY WARRANTY; without even the implied warranty of
13 | # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
14 | # GNU General Public License for more details.
15 | #
16 | # You should have received a copy of the GNU General Public License
17 | # along with this source code. If not, see .
18 |
19 | ## libdevuansdk build script for decode-os
20 |
21 | source "$R/../config"
22 |
23 | blend_bootstrap_setup() {
24 | fn blend_bootstrap_setup "(override)"
25 | req=(strapdir)
26 | ckreq || return 1
27 |
28 | sudo cp -v "$R/../extra/deb.torproject.org.asc" "$strapdir"
29 | cat </dev/null
30 | #!/bin/sh
31 | echo " * Adding deb.torproject.org repo key to apt..."
32 | cat /deb.torproject.asc | apt-key add -
33 | rm -f /deb.torproject.asc
34 | EOF
35 | chroot-script -d blend-bootstrap-setup || zerr
36 | }
37 |
38 | blend_preinst() {
39 | fn blend_preinst
40 | req=(strapdir)
41 | ckreq || return 1
42 |
43 | notice "executing $blend_name preinst"
44 |
45 | add-user decode decode
46 | }
47 |
48 | blend_postinst() {
49 | fn blend_postinst
50 | req=(strapdir)
51 | ckreq || return 1
52 |
53 | notice "executing $blend_name postinst"
54 |
55 | nopackage=(musl tomb golang tordam stem netdata zenroom chainspace)
56 | for app in $nopackage; do
57 | blend_install_${app} || zerr
58 | done || zerr
59 |
60 | notice "copying rootfs overlays"
61 | rootfs_overlay_path="$R/../rootfs-overlay"
62 | sudo rsync -raX "${rootfs_overlay_path}/common/" "${strapdir}"
63 | if [[ -n "$vmsdk_version" ]]; then
64 | _sdk="vm"
65 | elif [[ -n "$armsdk_version" ]]; then
66 | _sdk="arm"
67 | elif [[ -n "$livesdk_version" ]]; then
68 | _sdk="live"
69 | fi
70 | sudo rsync -raX "${rootfs_overlay_path}/${_sdk}/" "${strapdir}"
71 |
72 | blend_finalize || zerr
73 | }
74 |
75 | ## {{{ blend_install_musl()
76 | blend_install_musl() {
77 | fn blend_install_musl
78 | req=(strapdir musl_version musl_url)
79 | ckreq || return 1
80 |
81 | notice "cloning musl gits"
82 | sudo git clone "$musl_url" "$strapdir/root/musl" || zerr
83 |
84 | notice "installing musl in $strapdir"
85 |
86 | cat </dev/null
87 | #!/bin/sh
88 | cd /root/musl
89 | git checkout ${musl_version}
90 | ./configure && \
91 | make ${MAKEOPTS} && make install || exit 1
92 | cd ..
93 | rm -rf musl
94 | EOF
95 | chroot-script install-musl || zerr
96 | }
97 | ## }}}
98 | ## {{{ blend_install_tomb()
99 | blend_install_tomb() {
100 | fn blend_install_tomb
101 | req=(strapdir tomb_version tomb_url)
102 | ckreq || return 1
103 |
104 | notice "cloning tomb gits"
105 | sudo git clone "$tomb_url" "$strapdir/root/tomb" || zerr
106 |
107 | notice "installing tomb in $strapdir"
108 | cat </dev/null
109 | #!/bin/sh
110 | cd /root/tomb
111 | git checkout ${tomb_version}
112 | make install
113 |
114 | cd extras/kdf-keys
115 | make && make install || exit 1
116 |
117 | cd /root
118 | rm -rf tomb
119 | EOF
120 | chroot-script install-tomb || zerr
121 | }
122 | ## }}}
123 | ## {{{ blend_install_golang()
124 | blend_install_golang() {
125 | fn blend_install_golang
126 | req=(strapdir golang_url)
127 | ckreq || return 1
128 |
129 | notice "installing golang in $strapdir"
130 | cat </dev/null
131 | #!/bin/sh
132 | cd /usr/local
133 | wget ${golang_url} || exit 1
134 | tar xf $(basename ${golang_url})
135 | rm -f $(basename ${golang_url})
136 |
137 | ln -snvf /usr/local/go/bin/go /usr/local/bin/go
138 | ln -snvf /usr/local/go/bin/gofmt /usr/local/bin/gofmt
139 | EOF
140 | chroot-script install-golang || zerr
141 | }
142 | ## }}}
143 | ## {{{ blend_install_tordam()
144 | blend_install_tordam() {
145 | fn blend_install_tordam
146 | req=(strapdir tordam_url)
147 | ckreq || return 1
148 |
149 | notice "installing tor-dam in $strapdir"
150 | cat </dev/null
151 | #!/bin/sh
152 | cd /home/decode
153 | sudo -u decode go get -v -u ${tordam_url}
154 | cd /home/decode/go/src/github.com/decodeproject/tor-dam
155 |
156 | # A random password for the Tor Controlport auth
157 | torpass="\$(echo "\$(shuf -n 2 /usr/share/dict/words --random-source=/dev/urandom | tr '\n' '-')")"
158 | sed -i python/damhs.py -e "s/topkek/\${torpass}/"
159 | sed -i python/damauth.py -e "s/topkek/\${torpass}/"
160 | make install
161 | make -C contrib install-init
162 |
163 | torpass="\$(sudo -u debian-tor tor --hash-password "\${torpass}")"
164 | sed -e 's/User tor/User debian-tor/' < contrib/torrc > /etc/tor/torrc
165 | sed -e 's/HashedControlPassword .*//' -i /etc/tor/torrc
166 | echo "HashedControlPassword \${torpass}" >> /etc/tor/torrc
167 | EOF
168 | chroot-script -d install-tordam || zerr
169 | }
170 | ## }}}
171 | ## {{{ blend_install_netdata()
172 | blend_install_netdata() {
173 | fn blend_install_netdata
174 | req=(strapdir netdata_version netdata_url)
175 | ckreq || return 1
176 |
177 | notice "downloading netdata tarball"
178 | sudo wget -O "$strapdir/root/netdata.tgz" "$netdata_url" || zerr
179 |
180 | notice "installing netdata in $strapdir"
181 | cat </dev/null
182 | #!/bin/sh
183 | cd /root
184 | tar xvf netdata.tgz || exit 1
185 | cd netdata-${netdata_version}
186 | ./netdata-installer.sh --dont-wait --dont-start-it || exit 1
187 | cd ..
188 | rm -rf netdata.tgz netdata-${netdata_version}
189 | EOF
190 | chroot-script install-netdata || zerr
191 | }
192 | ## }}}
193 | ## {{{ blend_install_stem()
194 | blend_install_stem() {
195 | fn blend_install_stem
196 | req=(strapdir stem_version stem_url)
197 | ckreq || return 1
198 |
199 | notice "installing stem tor library"
200 | sudo git clone "$stem_url" "$strapdir/root/stem" || zerr
201 |
202 | notice "installing stem in $strapdir"
203 | cat </dev/null
204 | #!/bin/sh
205 | cd /root/stem
206 | git checkout ${stem_version}
207 | python3 setup.py install || exit 1
208 | cd ..
209 | rm -rf stem
210 | EOF
211 | chroot-script install-stem || zerr
212 | }
213 | ## }}}
214 | ## {{{ blend_install_zenroom()
215 | blend_install_zenroom() {
216 | fn blend_install_zenroom
217 | req=(strapdir zenroom_url)
218 | ckreq || return 1
219 |
220 | notice "installing zenroom"
221 | sudo wget -O "$strapdir/usr/local/bin/zenroom" "$zenroom_url" || zerr
222 | sudo chmod +x "$strapdir/usr/local/bin/zenroom"
223 | }
224 | ## }}}
225 | ## {{{ blend_install_chainspace()
226 | blend_install_chainspace() {
227 | fn blend_install_chainspace
228 | req=(strapdir chainspace_url chainspacedist_url)
229 | ckreq || return 1
230 |
231 | local p="$strapdir/home/decode/chainspace"
232 |
233 | notice "installing chainspace"
234 | sudo git clone "$chainspace_url" "$p" || zerr
235 | pushd "${p}"
236 | sudo wget ${chainspacedist_url} || zerr
237 | sudo tar xvf $(basename $chainspacedist_url) || zerr
238 | popd
239 | sudo chown -R 1000:1000 "$strapdir/home/decode"
240 |
241 | cat </dev/null
265 | #!/bin/sh
266 |
267 | chsh -s /bin/bash decode
268 | for i in users ; do
269 | gpasswd -a decode \$i
270 | done
271 |
272 | sed -e 's/UsePAM yes/UsePAM no/' -i /etc/ssh/sshd_config
273 |
274 | rc-update del redis-server default
275 | rc-update add dam-dir default
276 | rc-update add dam-client default
277 |
278 | ## misc
279 | sed -i -e 's/devuan/decode/' /etc/hosts
280 | echo decode > /etc/hostname
281 | mkdir -p /var/lib/tor
282 | chown -R debian-tor:debian-tor /var/lib/tor
283 | rm -rf /var/lib/tor/hidden_service
284 |
285 | ## cleanup
286 | apt-get --yes --force-yes purge ${finalize_purge_packages}
287 | apt-get --yes --force-yes autoremove
288 | apt-get clean
289 | apt-get update
290 |
291 | cleanupfiles="
292 | /var/log/bootstrap.log
293 | /var/log/dpkg.log
294 | /var/log/alternatives.log
295 | /var/log/fontconfig.log
296 | /var/log/apt
297 | /var/log/fsck
298 | /var/log/ConsoleKit
299 | /var/lib/polkit-1
300 | /usr/local/share/zsh/site-functions
301 | "
302 | echo "\$cleanupfiles" | xargs rm -rf
303 |
304 | updatedb
305 | EOF
306 |
307 | chroot-script -d finalize || zerr
308 | }
309 | ## }}}
310 | ## {{{ conf_print_sorceslist()
311 | conf_print_sourceslist() {
312 | fn conf_print_sourceslist "(override)"
313 |
314 | cat <