├── .github └── CONTRIBUTING.md ├── .gitignore ├── .travis.yml ├── LICENSE ├── README.md ├── docs ├── Makefile ├── _static │ ├── data-access-control-code-email-domains.png │ ├── data-access-control-receive-token.png │ ├── data-ingesting-added-columns.png │ ├── data-ingesting-code-inserting-data.png │ ├── data-ingesting-import-csv-rails.png │ ├── data-querying-school-data.png │ ├── data-visualization-method-comparison.png │ ├── data-visualization-webgl.png │ ├── deployguide-appservices.png │ ├── deployguide-swap-menu.png │ ├── deployguide-swap.png │ ├── github-workflow-create-new-repo.png │ ├── github-workflow-description-tags-empty.png │ ├── github-workflow-description-tags-filled.png │ ├── github-workflow-disable-features.png │ ├── github-workflow-labels-menu.png │ └── github-workflow-set-issue-metadata.png ├── admin │ ├── deployment-guide.rst │ └── github-workflow.rst ├── administrative-boundaries.rst ├── conf.py ├── data │ ├── data-access-control.rst │ ├── data-ingesting.rst │ ├── data-querying.rst │ ├── data-rules.rst │ ├── data-validation.rst │ └── data-visualization.rst ├── index.rst ├── make.bat ├── requirements.txt └── test-docs.sh ├── project-index.md └── public ├── images ├── admin_levels.png └── admin_zeros.png └── sample-data └── baseline-mobility ├── movement-day0.json ├── movement-day1.json ├── movement-day2.json ├── movement-day3.json ├── movement-day4.json ├── movement-day5.json └── movement-day6.json /.github/CONTRIBUTING.md: -------------------------------------------------------------------------------- 1 | MagicBox contributing guidelines 2 | ================================ 3 | 4 | Thank you for your interest in contributing to a UNICEF project! UNICEF believes 5 | in new approaches, partnerships, and technologies that support realizing 6 | children's rights are critical to improving their lives. People like you can 7 | help contribute to our mission, and we'd love your help! 8 | 9 | 10 | ## Table of contents 11 | 12 | * **[What is this?](https://github.com/unicef/magicbox/blob/master/.github/CONTRIBUTING.md#what-is-this)** 13 | * **[Contribute what?](https://github.com/unicef/magicbox/blob/master/.github/CONTRIBUTING.md#contribute-what)** 14 | * **[Ground rules](https://github.com/unicef/magicbox/blob/master/.github/CONTRIBUTING.md#ground-rules)** 15 | * **[Getting started](https://github.com/unicef/magicbox/blob/master/.github/CONTRIBUTING.md#getting-started)** 16 | * **[Report a bug](https://github.com/unicef/magicbox/blob/master/.github/CONTRIBUTING.md#report-a-bug)** 17 | * **[Code review process](https://github.com/unicef/magicbox/blob/master/.github/CONTRIBUTING.md#code-review-process)** 18 | * **[Community](https://github.com/unicef/magicbox/blob/master/.github/CONTRIBUTING.md#community)** 19 | 20 | 21 | ## What is this? 22 | 23 | These guidelines help contributors to our projects understand our workflow and 24 | how to contribute to our projects. Following these guidelines helps communicate 25 | that you respect the time of the developers managing and developing this open 26 | source project. In return, they should reciprocate that respect in addressing 27 | your issue, assessing changes, and helping you finalize your pull requests. 28 | 29 | 30 | ## Contribute what? 31 | 32 | There are different ways to contribute to a project. Some ways to contribute 33 | are… 34 | 35 | * Writing code 36 | * Submitting bug reports / feature requests 37 | * Improving documentation 38 | * Writing tutorials / blog posts 39 | 40 | 41 | ## Ground rules 42 | 43 | These ground rules are the basic responsibilities for contributing to the 44 | project. 45 | 46 | * **DO's** 47 | * DO run unit tests before submitting a pull request 48 | * DO create issues first for major changes / enhancements you want to make 49 | * Discuss things transparently first and get community feedback 50 | * DO keep each pull request to _one_ feature / bug fix 51 | * If you want to submit multiple things, submit multiple pull requests 52 | * DO write unit tests for new features 53 | * DO write [good commit messages](http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html) 54 | * **DON'Ts** 55 | * DON'T commit changes to files irrelevant to your pull request (e.g. 56 | `gitignore`) 57 | 58 | 59 | ## Getting started 60 | 61 | _Hint_: **Working on your first pull request?** Learn how from this _free_ 62 | series. [How to Contribute to an Open Source Project on GitHub](https://egghead.io/series/how-to-contribute-to-an-open-source-project-on-github) 63 | 64 | 1. Fork the project to your GitHub account. 65 | 2. Create changes in your fork. 66 | * Run unit tests once you're done! (`npm test`) 67 | 3. Send a pull request to our repo. 68 | 69 | 70 | ## Report a bug 71 | 72 | When filing an issue, answer these five questions: 73 | 74 | 1. What version of NodeJS / npm are you using (`npm version`)? 75 | 2. What operating system are you using? 76 | 3. What did you do? 77 | 4. What did you expect to see? 78 | 5. What did you see instead? 79 | 80 | 81 | ## Code review process 82 | 83 | The core team at the UNICEF Office of Innovation reviews pull requests on a 84 | weekly basis. The core team may vary – see an updated list in the [project index](https://github.com/unicef/magicbox/blob/master/project-index.md). 85 | 86 | Expect a response on new pull requests **within five business days** (Mon-Fri). 87 | If you don't receive any feedback, please follow up with a new comment! 88 | 89 | 90 | ## Community 91 | 92 | To be determined! 93 | 94 | 97 | 98 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Sphinx docs output 2 | docs/_build 3 | 4 | # Virtualenv 5 | # http://iamzed.com/2009/05/07/a-primer-on-virtualenv/ 6 | .Python 7 | [Bb]in 8 | [Ii]nclude 9 | [Ll]ib 10 | [Ll]ib64 11 | [Ll]ocal 12 | [Ss]cripts 13 | pyvenv.cfg 14 | .venv 15 | pip-selfcheck.json 16 | 17 | 18 | ### macOS ### 19 | # General 20 | .DS_Store 21 | .AppleDouble 22 | .LSOverride 23 | 24 | # Icon must end with two \r 25 | Icon 26 | 27 | # Thumbnails 28 | ._* 29 | 30 | # Files that might appear in the root of a volume 31 | .DocumentRevisions-V100 32 | .fseventsd 33 | .Spotlight-V100 34 | .TemporaryItems 35 | .Trashes 36 | .VolumeIcon.icns 37 | .com.apple.timemachine.donotpresent 38 | 39 | # Directories potentially created on remote AFP share 40 | .AppleDB 41 | .AppleDesktop 42 | Network Trash Folder 43 | Temporary Items 44 | .apdisk 45 | 46 | ### Windows ### 47 | # Windows thumbnail cache files 48 | Thumbs.db 49 | ehthumbs.db 50 | ehthumbs_vista.db 51 | 52 | # Dump file 53 | *.stackdump 54 | 55 | # Folder config file 56 | [Dd]esktop.ini 57 | 58 | # Recycle Bin used on file shares 59 | $RECYCLE.BIN/ 60 | 61 | # Windows Installer files 62 | *.cab 63 | *.msi 64 | *.msix 65 | *.msm 66 | *.msp 67 | 68 | # Windows shortcuts 69 | *.lnk 70 | -------------------------------------------------------------------------------- /.travis.yml: -------------------------------------------------------------------------------- 1 | language: python 2 | python: 3 | - 3.5 4 | install: 5 | - pip install -r docs/requirements.txt 6 | script: docs/test-docs.sh 7 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | BSD 3-Clause License 2 | 3 | Copyright (c) 2017, UNICEF 4 | All rights reserved. 5 | 6 | Redistribution and use in source and binary forms, with or without 7 | modification, are permitted provided that the following conditions are met: 8 | 9 | * Redistributions of source code must retain the above copyright notice, this 10 | list of conditions and the following disclaimer. 11 | 12 | * Redistributions in binary form must reproduce the above copyright notice, 13 | this list of conditions and the following disclaimer in the documentation 14 | and/or other materials provided with the distribution. 15 | 16 | * Neither the name of the copyright holder nor the names of its 17 | contributors may be used to endorse or promote products derived from 18 | this software without specific prior written permission. 19 | 20 | THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" 21 | AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE 22 | IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE 23 | DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE 24 | FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL 25 | DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR 26 | SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER 27 | CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, 28 | OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE 29 | OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 30 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | MagicBox 2 | ======== 3 | 4 | **_This repository is no longer actively maintained_**. 5 | 6 | [![Chat on Gitter](https://badges.gitter.im/unicef-innovation-dev/Lobby.png)](https://gitter.im/unicef-innovation-dev/Lobby) 7 | [![Documentation Status](https://readthedocs.org/projects/magicbox/badge/?version=latest)](https://magicbox.readthedocs.io/en/latest/?badge=latest) 8 | [![Build Status](https://travis-ci.org/unicef/magicbox.svg?branch=master)](https://travis-ci.org/unicef/magicbox) 9 | 10 | A platform that uses real-time data to inform life-saving humanitarian responses to emergency situations 11 | 12 | 13 | ## About 14 | 15 | MagicBox is an open-source platform that uses real-time information to inform life-saving humanitarian responses to emergency situations. 16 | It's composed of multiple GitHub repositories designed to ingest, aggregate, and serve data. 17 | 18 | You can read more about the project's purpose [on unicef.org](https://www.unicef.org/innovation/Magicbox). 19 | 20 | 21 | ## What's here 22 | 23 | This repo is the "parent" repo for all MagicBox-related projects. 24 | It hosts the documentation and other misc. resources for MagicBox. 25 | Code for other projects, like the [API](https://github.com/unicef/magicbox-open-api) and [front-end](https://github.com/unicef/magicbox-maps), are hosted in other repositories. 26 | See the [project index](https://github.com/unicef/magicbox/blob/master/project-index.md) for a full list of MagicBox-related repos and their respective maintainers. 27 | 28 | 29 | ## Contributing 30 | 31 | **We are looking for collaboration from the Open Source community!** 32 | There's so much we want to do, including but not limited to: 33 | enhancing existing applications with new features, optimizing the technical tools and algorithms involved to accommodate data challenges, and bringing our work closer to the public to leverage their inputs via blog posts and tutorials. 34 | 35 | * Please read our [contribution guidelines](https://github.com/unicef/magicbox/blob/master/.github/CONTRIBUTING.md) for details on what and how you can contribute. 36 | * Check out our [Development Project Board](https://github.com/orgs/unicef/projects/2). 37 | * Look for tasks labelled `good first issue` under the Issues tab in each repo. 38 | * Join our [Gitter channel](https://gitter.im/unicef-innovation-dev/Lobby). 39 | 40 | 41 | ## Documentation 42 | 43 | Project documentation is hosted on [ReadTheDocs](https://magicbox.readthedocs.io/). 44 | Its current version is mostly suited for developers who want to understand more about our underlying tech. 45 | We are hoping to establish a more user-friendly version soon, one that is readable by our potential users (e.g. data scientists and decision-makers in UNICEF country offices, state governments, other UN agencies). 46 | 47 | 48 | ## Resources 49 | 50 | To understand MagicBox better: 51 | 52 | * Read the README for each repo (full list is [here](https://github.com/unicef/magicbox/blob/master/project-index.md)). 53 | An architecture diagram of how these components are connected to one another is coming soon. 54 | * Check out our [Medium blog posts](https://medium.com/@mikefabrikant/latest). 55 | -------------------------------------------------------------------------------- /docs/Makefile: -------------------------------------------------------------------------------- 1 | # Minimal makefile for Sphinx documentation 2 | # 3 | 4 | # You can set these variables from the command line. 5 | SPHINXOPTS = 6 | SPHINXBUILD = sphinx-build 7 | SPHINXPROJ = MagicBox 8 | SOURCEDIR = . 9 | BUILDDIR = _build 10 | 11 | # Put it first so that "make" without argument is like "make help". 12 | help: 13 | @$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) 14 | 15 | .PHONY: help Makefile 16 | 17 | # Catch-all target: route all unknown targets to Sphinx using the new 18 | # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). 19 | %: Makefile 20 | @$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O) -------------------------------------------------------------------------------- /docs/_static/data-access-control-code-email-domains.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/data-access-control-code-email-domains.png -------------------------------------------------------------------------------- /docs/_static/data-access-control-receive-token.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/data-access-control-receive-token.png -------------------------------------------------------------------------------- /docs/_static/data-ingesting-added-columns.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/data-ingesting-added-columns.png -------------------------------------------------------------------------------- /docs/_static/data-ingesting-code-inserting-data.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/data-ingesting-code-inserting-data.png -------------------------------------------------------------------------------- /docs/_static/data-ingesting-import-csv-rails.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/data-ingesting-import-csv-rails.png -------------------------------------------------------------------------------- /docs/_static/data-querying-school-data.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/data-querying-school-data.png -------------------------------------------------------------------------------- /docs/_static/data-visualization-method-comparison.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/data-visualization-method-comparison.png -------------------------------------------------------------------------------- /docs/_static/data-visualization-webgl.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/data-visualization-webgl.png -------------------------------------------------------------------------------- /docs/_static/deployguide-appservices.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/deployguide-appservices.png -------------------------------------------------------------------------------- /docs/_static/deployguide-swap-menu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/deployguide-swap-menu.png -------------------------------------------------------------------------------- /docs/_static/deployguide-swap.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/deployguide-swap.png -------------------------------------------------------------------------------- /docs/_static/github-workflow-create-new-repo.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/github-workflow-create-new-repo.png -------------------------------------------------------------------------------- /docs/_static/github-workflow-description-tags-empty.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/github-workflow-description-tags-empty.png -------------------------------------------------------------------------------- /docs/_static/github-workflow-description-tags-filled.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/github-workflow-description-tags-filled.png -------------------------------------------------------------------------------- /docs/_static/github-workflow-disable-features.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/github-workflow-disable-features.png -------------------------------------------------------------------------------- /docs/_static/github-workflow-labels-menu.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/github-workflow-labels-menu.png -------------------------------------------------------------------------------- /docs/_static/github-workflow-set-issue-metadata.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/docs/_static/github-workflow-set-issue-metadata.png -------------------------------------------------------------------------------- /docs/admin/deployment-guide.rst: -------------------------------------------------------------------------------- 1 | ############################ 2 | How to deploy an application 3 | ############################ 4 | 5 | 6 | In the UNICEF Office of Innovation team, we use `Azure Web Apps`_ to deploy our applications to the cloud. 7 | 8 | This guide will walk you through the deployment process of a WebApp in the Azure environment. 9 | At the end of this guide you will be able to develop, tag and deploy the tagged version of the application to the azure servers. 10 | 11 | 12 | *************************************** 13 | Create a feature branch for development 14 | *************************************** 15 | 16 | While in development, it is useful to make all of your changes in a feature branch named accordingly to the task. 17 | 18 | For example, if we want to implement a new feature to show information about the country the user clicked in, we could name the branch accordingly to this task using the command below: 19 | 20 | .. code:: bash 21 | 22 | git checkout -b show_country_information 23 | 24 | We can implement our features in this branch and submit a Pull Request to merge it into the master branch of the app. 25 | 26 | Create a tag 27 | ============ 28 | 29 | Once your changes get merged into master, we can create a tag for the current version of the application. 30 | Make sure to name the tag accordingly with the project's convention and following the `semantic versioning`_ rules. 31 | 32 | For example, suppose you made a simple fix in the application and your changes just got merged from the branch "fix_bug_61" into master. 33 | In this project, tags are named using the convention "vX.X.X" and the last tag is "v1.1.1". 34 | 35 | Since your fix is a `patch`_, you can create the tag "v1.1.2" with the command 36 | 37 | .. code:: bash 38 | 39 | git tag v1.1.2 40 | 41 | Always remember to push the tag to the public repository to make others aware of the current project version. 42 | You can push your tags to the public repository with the command. 43 | 44 | .. code:: bash 45 | 46 | git push origin master --tags 47 | 48 | 49 | ************************* 50 | Deploying the application 51 | ************************* 52 | 53 | After our changes get merged into master, we can deploy the application to Azure. 54 | 55 | Each UNICEF's application in Azure have two main deployment slots: production and staging. 56 | 57 | In the production slot lies the code that you can see in your browser, the application as it is, available for the final user. 58 | 59 | The staging slot contains the code preparing to be released to production. 60 | Once your code is in this slot you can take a look in a private url to see how your code will respond in production. 61 | 62 | If everything looks correct in staging, you can swap the code in production with the code in the staging slot to make your changes available for all users. 63 | 64 | Adding Azure remote for repository 65 | ================================== 66 | 67 | The first thing we need to do is add the staging remote to our local git repository. 68 | 69 | Within the project directory, you can add the stagin remote for the project issuing the command below. 70 | 71 | .. code:: bash 72 | 73 | git remote add staging https://@-staging.scm.azurewebsites.net:443/.git 74 | 75 | Make sure to change with the project name you are working on and for your username set in the deployment credentials in Azure. 76 | 77 | Pushing changes to the staging slot 78 | =================================== 79 | 80 | Push changes to the staging repository issuing the command below 81 | 82 | .. code:: bash 83 | 84 | git push staging master 85 | 86 | This command will push the current version in your master branch to staging in azure. 87 | 88 | To push a specific tag to Azure, run: 89 | 90 | .. code:: bash 91 | 92 | git push staging tagname:master 93 | 94 | Replace "tagname" with the tag you want to push to azure. 95 | 96 | Swap staging to production (Web) 97 | ================================ 98 | 99 | The last step to make your changes available to the public is swap the staging environment with the production environment. 100 | 101 | Once your code is working properly in the staging environment, you can swap it with production. 102 | Swapping those slots means that the code in the staging environment now answers for the production environment and the code in the production environment now lives in the staging environment. 103 | 104 | If a bug was introduced in your changes and it was not caught in the staging environment and is now in the production environment, all you have to do to revert it to the last working version of the application is to swap it again. 105 | 106 | To make the swap, go to the `Azure Panel`_ > App Services, find the project you are working on and select it. 107 | 108 | .. image:: ../_static/deployguide-appservices.png 109 | 110 | In the overview section you will find the "Swap" button, you can click in it. 111 | 112 | .. image:: ../_static/deployguide-swap.png 113 | 114 | Make sure to select in the Swap panel "staging" as source and "production" as destination. 115 | 116 | .. image:: ../_static/deployguide-swap-menu.png 117 | 118 | Click OK to make the swap. You will get a notification once it finishes. 119 | 120 | To replace your changes for the last working version of the application, just repeat this procedure. 121 | 122 | Swap staging to production (CLI) 123 | ================================ 124 | 125 | If you have the right permissions in place you can make the swap using the `Azure CLI`_. 126 | 127 | To do this, make sure you are in the ASM mode running 128 | 129 | .. code:: bash 130 | 131 | azure config mode --help 132 | 133 | Then you can use 134 | 135 | .. code:: bash 136 | 137 | azure site swap 138 | 139 | 140 | .. _`Azure Web Apps`: https://docs.microsoft.com/en-us/azure/app-service/ 141 | .. _`semantic versioning`: https://semver.org/ 142 | .. _`patch`: https://semver.org/#spec-item-6 143 | .. _`Azure Panel`: https://portal.azure.com 144 | .. _`Azure CLI`: https://www.npmjs.com/package/azure-cli 145 | -------------------------------------------------------------------------------- /docs/admin/github-workflow.rst: -------------------------------------------------------------------------------- 1 | ############################## 2 | GitHub workflow best practices 3 | ############################## 4 | 5 | The UNICEF Innovation development team uses GitHub to host open source projects. 6 | This document explains the GitHub workflow maintainers and developers use. 7 | It also offers suggestions for best practices on using available tools and integrations. 8 | 9 | This document explains… 10 | 11 | - How to make a new GitHub repository 12 | 13 | - How to maintain a GitHub repository 14 | 15 | - How to communicate effectively 16 | 17 | 18 | *********************************** 19 | How to make a new GitHub repository 20 | *********************************** 21 | 22 | This section explains steps to follow when creating a new GitHub repository. 23 | These makes projects more organized, easier to follow from an outsider's perspective, and boosts visibility of development. 24 | 25 | Repository creation 26 | =================== 27 | 28 | These steps focus on the initial repository creation, from `github.com/new `_. 29 | 30 | #. **Set a meaningful name**: 31 | 32 | - Try to make purpose obvious in title 33 | 34 | - Use hyphens (``-``) instead of underscores (``_``) 35 | 36 | #. **Write a description**: Write one or two sentences to quickly describe the project 37 | 38 | #. **Initialize with README**: Check to add a ``README.md`` file 39 | 40 | - If one is not yet written, initialize one 41 | 42 | #. **Add .gitignore for project type**: Find project's programming language and choose its ``.gitignore`` file, if available 43 | 44 | #. **Add BSD-3 Clause license**: Standard license used for UNICEF Innovation projects 45 | 46 | .. figure:: /_static/github-workflow-create-new-repo.png 47 | :alt: Example of creating a new repository 48 | 49 | Example of creating a new repository 50 | 51 | Configure repository 52 | ==================== 53 | 54 | Now, the repository is created. 55 | Make sure these settings are updated: 56 | 57 | Disable unneeded tools 58 | ---------------------- 59 | 60 | Disable any unneeded features or repository tools. 61 | If they are needed, they can be turned on again later. 62 | Turning off unneeded features makes it easier for someone to find the useful places in the project. 63 | It can also indicate if the thing they are looking for (e.g. documentation) is somewhere else. 64 | 65 | These features are found under the *Settings* menu for every repository. 66 | 67 | .. figure:: /_static/github-workflow-disable-features.png 68 | :alt: Disable unneeded tools, like a wiki or project boards 69 | 70 | Disable unneeded tools, like a wiki or project boards. 71 | There may be documentation or project boards set up elsewhere. 72 | This does not disable organization-level project boards. 73 | 74 | Set description, URL, topic tags 75 | -------------------------------- 76 | 77 | Make sure every repository has a description and topic tags set. 78 | If there is a URL to view a demo of the project or read more about it, include it too. 79 | 80 | At the top of every GitHub repository, there are fields of metadata for a description, a URL, and topic tags. 81 | A description and URL helps someone understand the project in one or two sentences or see a live demo. 82 | Topic tags raise visibility in the GitHub ecosystem and help other people discover new projects. 83 | 84 | .. figure:: /_static/github-workflow-description-tags-empty.png 85 | :alt: Example of no description, URL, or topic tags 86 | 87 | Example of no description, URL, or topic tags 88 | 89 | Do not leave a repository empty like this. 90 | Use the *Edit* button on the right to change the description and URL. 91 | Two text boxes will appear. 92 | 93 | For topic tags, click *Add topics* towards the left. 94 | Choose tags related to your project to help identify it. 95 | Consider programming languages, frameworks, or other software used. 96 | Tags like ``humanitarian`` or ``united-nations`` are also examples of tags for types of software. 97 | 98 | .. figure:: /_static/github-workflow-description-tags-filled.png 99 | :alt: magicbox-latlon-admin-server with description and topic tags 100 | 101 | magicbox-latlon-admin-server with description and topic tags 102 | 103 | Set up useful labels 104 | ==================== 105 | 106 | Labels are visual organization tools for your GitHub project. 107 | They make issues easier to sort and prioritize tasks. 108 | Additionally, they also help new contributors identify areas of interest for your project. 109 | They can help improve awareness of different types of contribution methods in your project (e.g. design and documentation tasks). 110 | 111 | Configure each repository's labels in a way that makes sense for your project. 112 | The labels should mean something to *you* so they are easily applied for sorting later. 113 | Every repository's issue and pull request labels are found under the *Issues* tab with the *Labels* button. 114 | 115 | .. figure:: /_static/github-workflow-labels-menu.png 116 | :alt: Click the Labels button towards the right of the search bar 117 | 118 | Click the *Labels* button towards the right of the search bar 119 | 120 | A good example of labels is `here `_ on the `unicef/magicbox`_ repository. 121 | To view the color code for a given label, click the *Edit* on its row. 122 | 123 | Not all of these labels will be helpful for a new project. 124 | Take ones that make sense, and make new labels specific to the project, if needed. 125 | 126 | .. _set-up-ci: 127 | 128 | Set up continuous integration (CI) 129 | ================================== 130 | 131 | - `Why we need CI `_ 132 | - `If you get confused between travis-ci.org and travis-ci.com `_ (heads-up: use travis-ci.com for new repositories; some of our earlier reposistories are on travis-ci.org, but we will wait until Travis CI rolls out their migration plan to take actions) 133 | 134 | MagicBox repositories use `Travis CI `_ for continuous integration. 135 | Below is how you can add this service to a new repository: 136 | 137 | 1. Make sure you have the **admin access** to the repository. 138 | New core developers can gain such access across multiple repositories by joining the `MagicBox Admins team `_. 139 | Contact Mike (`@mikefab `_) or one of the lead maintainers for this permission. 140 | 141 | 2. If you are not already on Travis CI, browse to `travis-ci.com `_ and log in with your GitHub account. 142 | Along the way, you will see the option to add any repository to Travis CI. 143 | Select *Only select repositories* and choose ``unicef/`` from the drop-down list. 144 | At the moment, we still handpick repositories to be monitored by Travis CI because not all repositories will need this service. 145 | Once you arrive at your Travis CI Profile page, you will see a list of all repositories currently under Travis CI tracking, whether they belong to your personal account or the `@unicef `_ GitHub account. 146 | 147 | 3. Now go to the repository's main page on GitHub, click on the *Settings* tab, then select *Integrations & services* from the left-side menu. 148 | If everything is properly set up in the previous step, you should see Travis CI under *Installed GitHub Apps*. 149 | 150 | 4. From the root directory of the repository on GitHub, add a new file called ``.travis.yml``. 151 | Place the following content in that file: 152 | 153 | .. code-block:: javascript 154 | 155 | language: node_js 156 | node_js: 157 | - "8" 158 | cache: 159 | directories: 160 | - "node_modules" 161 | 162 | If these steps do not make sense, refer to this `Getting Started guide `_ by Travis CI. 163 | 164 | 5. The last step is to add the Travis CI badge to the repository's README. 165 | Browse to the Travis CI page of the repository - the URL probably looks like this: ``https://travis-ci.com/unicef/the-repo-of-interest``. 166 | Find the status symbol next to your repository's name (the little bar to the right of the Octocat). 167 | In the pop-up window, click the drop-down menu to select Markdown, then copy the generated code block. 168 | Paste it to the top of your README file, just under the repository's name. 169 | If unclear, see `this guide `_. 170 | 171 | .. _set-up-code-health-checks: 172 | 173 | Set up code health checks with Code Climate 174 | ============================================ 175 | 176 | `Code Climate `_ is the chosen code health checker for MagicBox projects. 177 | This automated code review service runs checks whenever a pull request is made, helping contributors and maintainers identify issues before they get merged into the code base. 178 | That makes it sound similar to Travis CI or other CI tools in general - they all perform pre-merge checks. 179 | The main difference is: one is more about the technical functionality of the code (*e.g. will my program crash?*) and the other considers how "clean" and maintainable the code is - hence the term "code health." 180 | Examples of issues that Code Climate could bring up: complex or hard-to-understand code; code duplicates; functions or classes that are too long and need refactoring; style issues raised by ESLint. 181 | 182 | Aside from that, Code Climate automates and displays test coverage results. 183 | Having a high test coverage score is encouraged for any code repository, especially open source projects since the code quality will affect and be affected by a larger group of developers. 184 | Code Climate reads output from locally run tests or coverage tools like `lcov `_, then displays the score alongside the analysis of other quality metrics. 185 | The score can be viewed via both the dashboard on Code Climate site and the README badge. 186 | 187 | Before following the steps below to activate Code Climate for a new repository, make sure to gain **admin access** first. 188 | 189 | 1. Sign into `Code Climate - Quality `_ if you are not on it yet. 190 | If this is your first time signing up, use your GitHub account. 191 | (If you already have a Code Climate account and it is not **linked with your GitHub account**, follow `these instructions `_ to set that up.) 192 | Select *Open Source* as you sign up and you can add the repository here. 193 | Handpick repositories rather than opting for *All repositories*. 194 | If you are already on Code Climate, add new repositories by clicking on the button *Add a repository* from your Dashboard. 195 | (If all of this doesn't make sense, use this `guide from CodeClimate `_.) 196 | 197 | 2. Now Code Climate has started tracking your code, but you need a bit more set-up in order to interact with this service more actively. 198 | From the list of tracked repositories on your Code Climate profile, click the repository you want to set up and navigate to its *Repo Settings* tab. Look for the following sections in the left-side navigation menu. 199 | 200 | a. **Enable Pull Request integration**: 201 | *GitHub* section. 202 | Scroll down to *Pull request status updates* and click *Install*. 203 | A little green check mark will tell you if the installation succeeds and this feature is now active. 204 | If this option is not available or nowhere to be seen, it could be because you have not installed the Code Climate GitHub app. 205 | In step 1, by signing up and linking your GitHub account, you have connected with Code Climate via OAuth authentication. 206 | Your GitHub repository now sees Code Climate as an OAuth app. 207 | However, to automatically display the check status at each pull request, Code Climate needs to have access to your GitHub repository as a GitHub app. 208 | Hence, head to `Code Climate GitHub app `_ to install it on *both* your personal account and the `@unicef `_ GitHub account. You should install it on your personal account to utilize Code Climate power when you work in your own forks. 209 | 210 | b. **Set up Webhooks**: 211 | This `guide `_ explains why we need Webhooks and how to set it up. 212 | You can verify if the setup is successful via either Code Climate (*Repo Settings* > *GitHub* > *Connections* > *Webhook on GitHub*) or GitHub (*Settings* > *Webhooks*). 213 | 214 | c. **Enable ESLint with Code Climate**: 215 | *Plugins* section. 216 | Check the box in front of *ESLint*. 217 | Since the ESLint engine by default only analyzes ``.js`` files, if the repository uses non-traditional JavaScript syntax such as JSX or ES6, you will need to add the following file to the repository's root directory. Name it ``.codeclimate.yml``: 218 | 219 | .. code-block:: javascript 220 | 221 | plugins: 222 | eslint: 223 | enabled: true 224 | channel: "eslint-4" 225 | config: 226 | extensions: 227 | - .js 228 | - .jsx 229 | nodesecurity: 230 | enabled: true 231 | 232 | This code accesses the newest ESLint release possible (channel 4, see more here: https://docs.codeclimate.com/docs/eslint) and specifies the file extensions that we want ESLint to analyze. 233 | 234 | 3. By now, most quality metrics have been taken care of except for *test coverage reporting*. 235 | As said in point number 7 `here `_, test coverage statuses are enabled by default when you enable Pull Request integration. 236 | However, the docs article also says you need to configure test coverage for the statuses to populate. 237 | Hence, go to your repository on Code Climate and navigate to *Repo Settings* > *Test coverage*. 238 | There you will find the repository's Test Reporter ID. 239 | Copy that token to clipboard. 240 | Then go to your repository's main page on GitHub. 241 | Replace the content of ``.travis-ci.yml`` with the following: 242 | 243 | .. code-block:: javascript 244 | 245 | env: 246 | global: 247 | - CC_TEST_REPORTER_ID= 248 | 249 | language: node_js 250 | node_js: 251 | - "8" 252 | cache: 253 | directories: 254 | - "node_modules" 255 | 256 | before_script: 257 | - curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter 258 | - chmod +x ./cc-test-reporter 259 | - ./cc-test-reporter before-build 260 | after_script: 261 | - ./cc-test-reporter after-build --exit-code $TRAVIS_TEST_RESULT 262 | 263 | The code above tells Code Climate to run and report on test coverage scores every time Travis CI runs checks for a new pull request. 264 | Code Climate, however, does not generate test coverage results itself - it reads output from a supported testing framework, which usually are third-party tools, as said `here `_. 265 | We, therefore, need to `set up a testing framework`_ in our code, which is covered below. 266 | 267 | Deeper reads: 268 | 269 | - If you don't have admin access to a repository but still want to track its detailed code health analysis, go to its README on GitHub, click on its Maintainability badge to open its Code Climate report, and hit *Star* to add this repository to your Code Climate dashboard. If unclear, `see this guide `_. 270 | 271 | - If you are a core developer or maintainer, `read this article `_ to make better use of Code Climate in your pull request workflow. 272 | 273 | - Explore advanced features with `review comments `_. 274 | 275 | 4. The last step is to embed the **maintainability and test coverage badges** to GitHub. 276 | Head to your repository on Code Climate and click on *Repo Settings* > *Badges*. 277 | Select the format of your choice and copy that code snippet to the top of the repository's README, just under the repository's name. (This `guide `_ has good screenshots to illustrate this step.) 278 | 279 | .. _set-up-testing-framework: 280 | 281 | Set up a testing framework 282 | =========================== 283 | 284 | .. note:: To be written. 285 | 286 | *********************************** 287 | How to maintain a GitHub repository 288 | *********************************** 289 | 290 | This section focuses on "housekeeping" with GitHub projects, including labels and project boards. 291 | 292 | Housekeeping is important to maintain a repository. 293 | This organizes bugs, feature requests, and the project itself. 294 | Organized projects help active contributors stay on track and make realistic deadlines. 295 | It also helps new contributors understand what is going on. 296 | 297 | Housekeeping has five parts: 298 | 299 | #. Issue metadata 300 | 301 | #. Adding labels 302 | 303 | #. Updating project boards 304 | 305 | #. Making pull requests 306 | 307 | #. Reviewing pull requests 308 | 309 | Update issue and pull request metadata 310 | ====================================== 311 | 312 | Every GitHub issue and pull request has four metadata properties: 313 | 314 | #. **Assignees**: Who is currently working on this and who is the best point-of-contact for updates 315 | 316 | #. **Labels**: Visual cues on task status and importance (see below) 317 | 318 | #. **Projects**: Advanced business process management (see below) 319 | 320 | #. **Milestone**: Relevant feature or version milestone for an issue or pull request 321 | 322 | Assignees and labels should always be used at a minimum. 323 | Use projects and milestones when they are available. 324 | 325 | .. figure:: /_static/github-workflow-set-issue-metadata.png 326 | :alt: Set assignees, labels, project boards, and milestones from the side column in every GitHub issue or pull request 327 | 328 | Set assignees, labels, project boards, and milestones from the side column in every GitHub issue or pull request 329 | 330 | Adding labels to issues 331 | ======================= 332 | 333 | Above, labels were mentioned as part of issue and pull request metadata. 334 | Maintaining and using labels is a good habit. 335 | An issue or pull request might have two to four labels, depending on how the project was set up. 336 | 337 | If labels are not yet configured, read `Set up useful labels`_. 338 | 339 | Once a week, check issues and pull requests to see if tags are up-to-date. 340 | Update or change any labels that are stale (such as priority labels). 341 | Add labels from the metadata sub-menu when you open an issue or pull request. 342 | 343 | Updating project boards 344 | ======================= 345 | 346 | `GitHub project boards `_ are an organizational tool for the project. 347 | They use a `kanban-style `_ approach to organizing GitHub issues and pull requests. 348 | Our workflow is explained `on Opensource.com `_. 349 | 350 | To update and maintain the project boards… 351 | 352 | #. Make sure any issues or pull requests not shown are added to the board 353 | 354 | #. Ensure important issues are organized by *In progress* or *To Do* 355 | 356 | #. Issues not yet ready for consideration go on the backlog 357 | 358 | #. All items under *In progress* or *To Do* columns should be GitHub issues, **not** note cards (note cards are okay for the backlog column) 359 | 360 | Making pull requests 361 | ==================== 362 | 363 | All major changes to the project should **always be made through a pull request** (PR). 364 | Pull requests are like a registry of changes for a project. 365 | It is easy for someone to see what is going in and out of a project. 366 | Outside contributors will always have to make pull requests, so it is good practice for core / trusted developers to use pull requests too. 367 | 368 | Follow contributing guidelines 369 | ------------------------------ 370 | 371 | The contributing guidelines for all MagicBox projects live `in the unicef/magicbox repository `_. 372 | 373 | Always follow these contributing guidelines when working in the project. 374 | These are the standards and rules we ask the community to follow when contributing. 375 | As project maintainers, it is our responsibility to hold ourselves to the same standards we ask of others. 376 | Thus, always make sure current development practices are in-line with what our guidelines. 377 | 378 | Write useful commit messages 379 | ---------------------------- 380 | 381 | Writing useful commit messages is a good practice to follow. 382 | When looking through project commits, it should be somewhat clear what has changed in the project and how. 383 | Short or nondescript commit messages are not helpful to maintainers or new contributors. 384 | Commit messages do not need to be paragraphs, but they should clearly indicate what changed or why something changed. 385 | 386 | Read `this blog post `_ for more information about keeping git history clean and tidy with ``git rebase``. 387 | 388 | Reviewing pull requests 389 | ======================= 390 | 391 | Pull requests (often abbreviated as PRs) are the cornerstone of accepting contributions to countless open source projects. 392 | All major contributions to a project, from both core contributors and new contributors, should be made as pull requests. 393 | It is important to follow consistent practices when reviewing pull requests. 394 | 395 | Triage new pull requests 396 | ------------------------ 397 | 398 | Update the metadata for all new pull requests, especially if they will be open for *longer than one work day*. 399 | Examples of metadata includes the following: 400 | 401 | - **Assignees**: Indicates whose responsibility it is to review or accept a pull request 402 | 403 | - **Labels**: Indicates what type of change the pull request is and what its priority is 404 | 405 | - **Projects**: Provides context to overall project development (if using project boards) 406 | 407 | - **Milestones**: Connects pull request to a specific goal or version (if applicable) 408 | 409 | Triaging new pull requests by updating the metadata keeps the project organized. 410 | It is easier for an outsider to understand the project workflow and development by triaging. 411 | It is also helpful to give context for a pull request if you have to update it later. 412 | For example, if a pull request cannot be merged because of an external problem, label it as **blocked**. 413 | 414 | Use continuous integration (CI) 415 | ------------------------------- 416 | 417 | Use the CI added :ref:`in the previous section ` as a basic requirement for accepting new contributions. 418 | All pull requests will run your test suite and ensure new contributions pass all tests. 419 | This prevents bad code from slipping under the cracks and making it into a production environment. 420 | It also provides quick, instant feedback for a new contribution. 421 | The contributor immediately knows their change broke the application and know test is not passing. 422 | 423 | For *all* new contributions, from both active and new contributors, ensure all CI tests pass before merging a pull request. 424 | Bypassing CI health checks by pushing directly to the repository or merging a pull request before tests finish bypasses the advantages of CI. 425 | 426 | 427 | Use code health checks 428 | ---------------------- 429 | 430 | Use the code health checks added :ref:`earlier in this section ` as another requirement for accepting new contributions. 431 | There are many ways for you to configure the code health checks. 432 | Use them as a way to set standards for code quality and enforce those standards automatically in new contributions. 433 | The code health checks offer both already active and new contributors a way to understand the impact of their changes. 434 | This results in clear code that is easier to maintain in the long-term. 435 | 436 | Ensure all new contributions receive passing grades from the code health checking tool before accepting them. 437 | 438 | Leave a review 439 | -------------- 440 | 441 | Code review is a helpful practice for any software project and team, as explained in `this Atlassian blog post `_. 442 | It is a chance to catch deeper problems before they enter the code base. 443 | It also provides a chance for mentorship and guidance for a new contributor. 444 | Additionally, it improves the overall health of your project and makes an outside contribution more likely to contribute again. 445 | Taking the time to review someone's contribution and code is also validation of their time and energy spent to make that contribution. 446 | 447 | Spending the time to review new contributions should be as regular of a practice as writing your own code. 448 | Ensure each new pull request receives a review, even if it is a passing review with no comments. 449 | If you do leave feedback, make sure it is kind and courteous – be aware of how you deliver your feedback. 450 | See `this guide `_ on unlearning toxic behavior in code reviews. 451 | 452 | Always remember to thank a contributor for their contribution too. 453 | 454 | 455 | ******************************* 456 | Communicating about development 457 | ******************************* 458 | 459 | Communication about development should be kept public as much as possible in `our Gitter chat `_. 460 | Whenever you make a new pull request, always share the link in the main Gitter chat room. 461 | This lets other developers know you made a change and also gives them an opportunity to review your code. 462 | And if you want a code review, be sure to ask for it too. 463 | 464 | .. _`unicef/magicbox`: https://github.com/unicef/magicbox 465 | -------------------------------------------------------------------------------- /docs/administrative-boundaries.rst: -------------------------------------------------------------------------------- 1 | ######################### 2 | Administrative boundaries 3 | ######################### 4 | 5 | UNICEF uses **administrative boundaries** (or administrative areas) to define geographic areas in MagicBox and related projects. 6 | Administrative boundaries (ABs) extend the concepts of a country, state, or region. 7 | Examples of an administrative boundary may be… 8 | 9 | - Country 10 | 11 | - State 12 | 13 | - Province 14 | 15 | - County 16 | 17 | - Municipality 18 | 19 | - And more… 20 | 21 | ABs come from `GADM `__ and the `Humanitarian Data Exchange `__ (HumData), databases of global administrative areas. 22 | In addition, more ABs may come from other sources like municipal governments. 23 | These databases make it easier to work with this data in GIS or related software. 24 | 25 | 26 | ****** 27 | Levels 28 | ****** 29 | 30 | Levels are a hierarchy system to explain different concepts of administrative boundaries. 31 | For example, a city may belong to a municipality, which belongs to a country. 32 | Levels help us explain this hierarchy. 33 | 34 | UNICEF uses this system of levels: 35 | 36 | - **Level 0**: Countries 37 | 38 | - **Level 1**: States 39 | 40 | - **Level 2**: Counties 41 | 42 | - **Level 3**: Municipalities 43 | 44 | - And so on… 45 | 46 | 47 | *************** 48 | Why we use them 49 | *************** 50 | 51 | Different definitions of geographic areas by different nations makes worldwide geographic mapping a challenge. 52 | Not everyone uses the same names for regions. 53 | In MagicBox projects, each region, or **shape**, is assigned an ``admin_id``. 54 | 55 | An ``admin_id`` is an interpretation of a political border for a specific time. 56 | They point to a specific shape in a shapefile that represents an individual country. 57 | Time is a part of the admin ID to ingest historical data. 58 | For example, mobility or temperature might require a series from an earlier date where a region was known by a different name. 59 | 60 | See the following example of an AB for Afghanistan, with an ``admin_id``. 61 | 62 | .. figure:: https://cdn-images-1.medium.com/max/800/1*wAatd2Qiu5vXYmatpVCcmg.png 63 | :alt: Example of an administrative boundary for Afghanistan 64 | 65 | Example of an administrative boundary for Afghanistan 66 | 67 | To understand where ``afg_1_11_102-gadm2–8`` points to, note that Afghanistan has three **levels** of administrative boundaries. 68 | The ``admin_id`` has three integers, one per admin level: 69 | 70 | .. figure:: https://cdn-images-1.medium.com/max/800/1*hGHSrnLcyXdPDDvXje_UsQ.png 71 | :alt: Afghanistan has three levels of administrative boundaries 72 | 73 | Afghanistan has three levels of administrative boundaries 74 | 75 | 76 | *********************** 77 | Naming string explained 78 | *********************** 79 | 80 | First integer 81 | ============= 82 | 83 | The first integer is ``1`` because Afghanistan is the first country in GADM's database. 84 | 85 | .. figure:: https://cdn-images-1.medium.com/max/800/1*9WTComzFpXK2yUk8lNXg8A.png 86 | :alt: The first few countries in GADM's database 87 | 88 | The first few countries in GADM's database 89 | 90 | 91 | Second and third integer 92 | ======================== 93 | 94 | Afghanistan has 34 shapes at level one and 320 shapes in level two. 95 | 96 | .. figure:: https://cdn-images-1.medium.com/max/800/1*4POtdmBjmF1JZtbuhutDOA.png 97 | :alt: Afghanistan: Higher level administrative boundaries 98 | 99 | Afghanistan: Higher level administrative boundaries 100 | 101 | 102 | Putting it together 103 | =================== 104 | 105 | Thus, ``afg_1_11_102-gadm2–8`` indicates any population value attached to it is related to: 106 | 107 | - First country in the gadm collection 108 | 109 | - 11th shape in the admin level 1 shapefile 110 | 111 | - 102nd shape in the admin level 2 shapefile 112 | 113 | .. figure:: https://cdn-images-1.medium.com/max/800/1*k6TaJ4I0zk39f24yG_d_WQ.png 114 | :alt: Putting it together: What the admin ID represents 115 | 116 | Putting it together: What the admin ID represents 117 | -------------------------------------------------------------------------------- /docs/conf.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python3 2 | # -*- coding: utf-8 -*- 3 | # 4 | # MagicBox documentation build configuration file, created by 5 | # sphinx-quickstart on Tue Jan 16 13:57:12 2018. 6 | # 7 | # This file is execfile()d with the current directory set to its 8 | # containing dir. 9 | # 10 | # Note that not all possible configuration values are present in this 11 | # autogenerated file. 12 | # 13 | # All configuration values have a default; values that are commented out 14 | # serve to show the default. 15 | 16 | # If extensions (or modules to document with autodoc) are in another directory, 17 | # add these directories to sys.path here. If the directory is relative to the 18 | # documentation root, use os.path.abspath to make it absolute, like shown here. 19 | # 20 | # import os 21 | # import sys 22 | # sys.path.insert(0, os.path.abspath('.')) 23 | 24 | 25 | # -- General configuration ------------------------------------------------ 26 | 27 | # If your documentation needs a minimal Sphinx version, state it here. 28 | # 29 | # needs_sphinx = '1.0' 30 | 31 | # Add any Sphinx extension module names here, as strings. They can be 32 | # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom 33 | # ones. 34 | extensions = ['sphinx.ext.intersphinx', 35 | 'sphinx.ext.todo'] 36 | 37 | # Add any paths that contain templates here, relative to this directory. 38 | templates_path = ['_templates'] 39 | 40 | # The suffix(es) of source filenames. 41 | # You can specify multiple suffix as a list of string: 42 | # 43 | # source_suffix = ['.rst', '.md'] 44 | source_suffix = '.rst' 45 | 46 | # The master toctree document. 47 | master_doc = 'index' 48 | 49 | # General information about the project. 50 | project = 'MagicBox' 51 | copyright = '2018, UNICEF Office of Innovation' 52 | author = 'UNICEF Office of Innovation' 53 | 54 | # The version info for the project you're documenting, acts as replacement for 55 | # |version| and |release|, also used in various other places throughout the 56 | # built documents. 57 | # 58 | # The short X.Y version. 59 | version = '' 60 | # The full version, including alpha/beta/rc tags. 61 | release = '' 62 | 63 | # The language for content autogenerated by Sphinx. Refer to documentation 64 | # for a list of supported languages. 65 | # 66 | # This is also used if you do content translation via gettext catalogs. 67 | # Usually you set "language" from the command line for these cases. 68 | language = None 69 | 70 | # List of patterns, relative to source directory, that match files and 71 | # directories to ignore when looking for source files. 72 | # This patterns also effect to html_static_path and html_extra_path 73 | exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store'] 74 | 75 | # The name of the Pygments (syntax highlighting) style to use. 76 | pygments_style = 'sphinx' 77 | 78 | # If true, `todo` and `todoList` produce output, else they produce nothing. 79 | todo_include_todos = True 80 | 81 | 82 | # -- Options for HTML output ---------------------------------------------- 83 | 84 | # The theme to use for HTML and HTML Help pages. See the documentation for 85 | # a list of builtin themes. 86 | # 87 | import sphinx_rtd_theme 88 | html_theme = 'sphinx_rtd_theme' 89 | html_theme_path = [sphinx_rtd_theme.get_html_theme_path()] 90 | 91 | # Theme options are theme-specific and customize the look and feel of a theme 92 | # further. For a list of options available for each theme, see the 93 | # documentation. 94 | # 95 | html_theme_options = { 96 | 'collapse_navigation': False, 97 | 'navigation_depth': 3, 98 | } 99 | 100 | # Add any paths that contain custom static files (such as style sheets) here, 101 | # relative to this directory. They are copied after the builtin static files, 102 | # so a file named "default.css" will overwrite the builtin "default.css". 103 | html_static_path = ['_static'] 104 | 105 | # Custom sidebar templates, must be a dictionary that maps document names 106 | # to template names. 107 | # 108 | # This is required for the alabaster theme 109 | # refs: http://alabaster.readthedocs.io/en/latest/installation.html#sidebars 110 | html_sidebars = { 111 | '**': [ 112 | 'relations.html', # needs 'show_related': True theme option to display 113 | 'searchbox.html', 114 | ] 115 | } 116 | 117 | 118 | # -- Options for HTMLHelp output ------------------------------------------ 119 | 120 | # Output file base name for HTML help builder. 121 | htmlhelp_basename = 'MagicBoxdoc' 122 | 123 | 124 | # -- Options for LaTeX output --------------------------------------------- 125 | 126 | latex_elements = { 127 | # The paper size ('letterpaper' or 'a4paper'). 128 | # 129 | # 'papersize': 'letterpaper', 130 | 131 | # The font size ('10pt', '11pt' or '12pt'). 132 | # 133 | # 'pointsize': '10pt', 134 | 135 | # Additional stuff for the LaTeX preamble. 136 | # 137 | # 'preamble': '', 138 | 139 | # Latex figure (float) alignment 140 | # 141 | # 'figure_align': 'htbp', 142 | } 143 | 144 | # Grouping the document tree into LaTeX files. List of tuples 145 | # (source start file, target name, title, 146 | # author, documentclass [howto, manual, or own class]). 147 | latex_documents = [ 148 | (master_doc, 'MagicBox.tex', 'MagicBox Documentation', 149 | 'UNICEF Office of Innovation', 'manual'), 150 | ] 151 | 152 | 153 | # -- Options for manual page output --------------------------------------- 154 | 155 | # One entry per manual page. List of tuples 156 | # (source start file, name, description, authors, manual section). 157 | man_pages = [ 158 | (master_doc, 'magicbox', 'MagicBox Documentation', 159 | [author], 1) 160 | ] 161 | 162 | 163 | # -- Options for Texinfo output ------------------------------------------- 164 | 165 | # Grouping the document tree into Texinfo files. List of tuples 166 | # (source start file, target name, title, author, 167 | # dir menu entry, description, category) 168 | texinfo_documents = [ 169 | (master_doc, 'MagicBox', 'MagicBox Documentation', 170 | author, 'MagicBox', 'One line description of project.', 171 | 'Miscellaneous'), 172 | ] 173 | 174 | 175 | 176 | # -- Options for Epub output ---------------------------------------------- 177 | 178 | # Bibliographic Dublin Core info. 179 | epub_title = project 180 | epub_author = author 181 | epub_publisher = author 182 | epub_copyright = copyright 183 | 184 | # The unique identifier of the text. This can be a ISBN number 185 | # or the project homepage. 186 | # 187 | # epub_identifier = '' 188 | 189 | # A unique identification for the text. 190 | # 191 | # epub_uid = '' 192 | 193 | # A list of files that should not be packed into the epub file. 194 | epub_exclude_files = ['search.html'] 195 | 196 | 197 | 198 | # Example configuration for intersphinx: refer to the Python standard library. 199 | intersphinx_mapping = {'https://docs.python.org/': None} 200 | -------------------------------------------------------------------------------- /docs/data/data-access-control.rst: -------------------------------------------------------------------------------- 1 | ############## 2 | Access control 3 | ############## 4 | 5 | Information about schools can be sensitive. 6 | Potential bad actors could use school data for malicious purposes and terrorism. 7 | How school data is shared outside of the API is important. 8 | 9 | MagicBox uses `Auth0`_ to authenticate users and assign them roles. 10 | 11 | 12 | ****** 13 | Tokens 14 | ****** 15 | 16 | Auth0 creates tokens for users authenticating to the API. 17 | A user may make a request with their token like this. 18 | 19 | .. code-block:: bash 20 | 21 | curl -i localhost:8000/api/v1/schools/countries/GL -H "Token: Bearer xxxxxxxxxx9gek6Z5Ilnkx" 22 | 23 | After receiving the token, UNICEF reviews the applicant before privileges are granted. 24 | Once our API receives the token, we pass it to Auth0 which returns the user's profile and roles. 25 | 26 | .. image:: /_static/data-access-control-receive-token.png 27 | 28 | 29 | ***** 30 | Rules 31 | ***** 32 | 33 | General rules can be assigned by email domain via Auth0. 34 | 35 | .. figure:: /_static/data-access-control-code-email-domains.png 36 | 37 | Code that implements checking for the email domain of a user 38 | 39 | 40 | .. seealso:: 41 | 42 | See :doc:`data-rules` for more information about data rules. 43 | 44 | 45 | .. _`Auth0`: https://auth0.com/ 46 | -------------------------------------------------------------------------------- /docs/data/data-ingesting.rst: -------------------------------------------------------------------------------- 1 | ############################ 2 | Ingesting data into MagicBox 3 | ############################ 4 | 5 | Importing new data sets to MagicBox is important for expanding coverage and how we retrieve new data insights. 6 | This document explains how we ingest and import new data to MagicBox. 7 | 8 | 9 | ********************* 10 | School data ingestion 11 | ********************* 12 | 13 | Data for schools comes in CSV files with a specific naming scheme. 14 | We developed a Ruby on Rails-based CRUD [#]_ application, `Project Connect`_ to ingest this data. 15 | 16 | There are two admin interfaces for Rails applications: 17 | 18 | - Rails Admin 19 | 20 | - Active Admin 21 | 22 | Active Admin has an `import plugin`_ for uploading CSVs with ``before_batch_import`` support. 23 | We chose Active Admin for this reason. 24 | 25 | .. image:: /_static/data-ingesting-import-csv-rails.png 26 | 27 | 28 | ******************** 29 | School data database 30 | ******************** 31 | 32 | MagicBox needed a database to manage large amounts of school data and process it quickly. 33 | To do this, MagicBox uses a relational database, `PostgreSQL`_, to store data. 34 | PostgreSQL was chosen because it was used in a tutorial for building a similar use case as ours (thus, a relational database is not a "married" idea). 35 | 36 | All school data is stored in a single table named ``schools``. 37 | This keeps things simple at the expense of some repeated values in the database. 38 | 39 | Inserted data 40 | ============= 41 | 42 | When data is imported, new data is inserted alongside the imported data. 43 | Four new types of important data are added: 44 | 45 | - Provider / owner of school CSV data 46 | 47 | - Organization that received school data 48 | 49 | - Identity of uploader 50 | 51 | - If uploader chooses to remain private 52 | 53 | .. image:: /_static/data-ingesting-added-columns.png 54 | 55 | These values are not included with the data. 56 | We use the ``before_batch_import`` hook in Active Admin to parse file names, insert this data, and assign values. 57 | 58 | .. figure:: /_static/data-ingesting-code-inserting-data.png 59 | 60 | Implementation of how data is modified before imported. See `schools.rb`_ 61 | 62 | 63 | .. [#] Create, Read, Update, Delete 64 | .. _`Project Connect`: https://github.com/unicef/project-connect 65 | .. _`import plugin`: https://github.com/activeadmin-plugins/active_admin_import 66 | .. _`PostgreSQL`: https://www.postgresql.org/ 67 | .. _`schools.rb`: https://github.com/unicef/project-connect/blob/master/app/admin/schools.rb 68 | -------------------------------------------------------------------------------- /docs/data/data-querying.rst: -------------------------------------------------------------------------------- 1 | ###################### 2 | Querying MagicBox data 3 | ###################### 4 | 5 | The MagicBox API (`magicbox-open-api`_) serves different sets of data through its API. 6 | Currently, five datasets are available. 7 | 8 | - Cases 9 | 10 | - Mobility 11 | 12 | - Mosquitoes 13 | 14 | - Population 15 | 16 | - Schools 17 | 18 | The benefit of querying this data is to analyze and cross different data sets together. 19 | For example, this is done in `magicbox-maps`_, where population and mosquito prevelance data are crossed together. 20 | 21 | This document explains how the API works but does not document how to use it. 22 | For more information on using the API, see the `magicbox-open-api`_ repository. 23 | 24 | 25 | ***** 26 | Cases 27 | ***** 28 | 29 | .. note:: 30 | 31 | This section needs to be written. 32 | 33 | ******** 34 | Mobility 35 | ******** 36 | 37 | .. note:: 38 | 39 | This section needs to be written. 40 | 41 | ********** 42 | Mosquitoes 43 | ********** 44 | 45 | .. note:: 46 | 47 | This section needs to be written. 48 | 49 | ********** 50 | Population 51 | ********** 52 | 53 | .. note:: 54 | 55 | This section needs to be written. 56 | 57 | ******* 58 | Schools 59 | ******* 60 | 61 | The MagicBox API also serves data about schools. 62 | The public repositories contain sample data from private data sets. 63 | However, in the interest of the privacy and safety of the schools, the full data sets are kept private. 64 | 65 | How we do it 66 | ============ 67 | 68 | There are three endpoints for querying school data. 69 | They serve the following data: 70 | 71 | #. List of countries with school data 72 | 73 | #. Schools in a country 74 | 75 | #. Specific school data by ID value 76 | 77 | The first endpoint returns a comma-seperated list of country codes. 78 | There is available data about schools in the countries returned in the list. 79 | 80 | The second endpoint returns an array of arrays, where 81 | 82 | - First array returns column names (e.g. ``lat``, ``lon``, ``speed_connectivity``, ``type_connectivity``) 83 | - Subsequent arrays return values for school records 84 | 85 | The third endpoint returns column values from a specific school, as selected by its ID number. 86 | Some of the values available are listed below: 87 | 88 | - Address 89 | 90 | - Network connectivity 91 | 92 | - Number of teachers 93 | 94 | - Number of students 95 | 96 | - If electricity is available 97 | 98 | - If running water is available 99 | 100 | - And more 101 | 102 | .. figure:: /_static/data-querying-school-data.png 103 | 104 | Example of output from third endpoint inside of a front-end utilizing the API 105 | 106 | 107 | ******* 108 | Caveats 109 | ******* 110 | 111 | There are some caveats with querying the MagicBox API. 112 | 113 | - Cannot specify or limit columns included in response 114 | 115 | - Cannot request values where value is null 116 | 117 | - Cannot use operators (e.g. greater than, less than, etc.) 118 | 119 | 120 | .. _`magicbox-open-api`: https://github.com/unicef/magicbox-open-api 121 | .. _`magicbox-maps`: https://github.com/unicef/magicbox-maps 122 | -------------------------------------------------------------------------------- /docs/data/data-rules.rst: -------------------------------------------------------------------------------- 1 | #################### 2 | Data ingestion rules 3 | #################### 4 | 5 | This document explains how the format we receive data in. 6 | It also details how certain flags, or rules, are applied to the data when ingested. 7 | 8 | 9 | *************** 10 | Available flags 11 | *************** 12 | 13 | There are two flags available. 14 | These flags are applied to data when ingested. 15 | 16 | - **Private data**: Data is private and is only accessible to UNICEF and uploading organization 17 | 18 | - **Anonymity**: Uploading organization chooses to remain anonymous 19 | 20 | These flags are not inside of the data sets. 21 | The flags must be applied when the data is ingested. 22 | 23 | 24 | .. seealso:: 25 | 26 | See :doc:`data-ingesting` for more information about data ingestion. 27 | 28 | 29 | ********************** 30 | School data file names 31 | ********************** 32 | 33 | School mapping data is provided in a unique file name format. 34 | Some information must be gathered from the file name. 35 | See this example:: 36 | 37 | BR-ProCo-0-MCTIC-0.csv 38 | 39 | There are five parts to the file name. 40 | 41 | #. **BR**: Country code (BR is Brazil) 42 | #. **ProCo**: Partner (ProCo is `Project Connect `_) 43 | #. **0**: Sets privacy of data (0 is public, 1 is private) 44 | #. **MCTIC**: Provider (MCTIC is `Ministério da Ciência, Tecnologia, Inovação e Comunicação `_) 45 | #. **0**: Sets privacy of uploader (0 is public, 1 is anonymous) 46 | 47 | 48 | ******* 49 | Caveats 50 | ******* 51 | 52 | Code is not yet written to enforce rules. 53 | We still need to implement permissions based on user information forwarded from Auth0. 54 | -------------------------------------------------------------------------------- /docs/data/data-validation.rst: -------------------------------------------------------------------------------- 1 | ############### 2 | Data validation 3 | ############### 4 | 5 | Data validation ensures correct and consistent data is processed and aggregated for use in MagicBox. 6 | The validated data is used by implementations of the MagicBox API, such as `magicbox-maps`_. 7 | 8 | This article explains how we verify values, identify duplicates, and merge multiple records into one. 9 | 10 | 11 | ************ 12 | How we do it 13 | ************ 14 | 15 | .. figure:: https://cdn-images-1.medium.com/max/800/1*WEBHVGUmqRVleyMNNCLp3A.gif 16 | 17 | How `magicbox-latlong-admin-server`_ validates coordinate pairs 18 | 19 | Data validation is performed by `magicbox-latlong-admin-server`_. 20 | The server verifies a pair of latitude / longtitude coordinates are located within the country indicated by the beginning of a file name. 21 | 22 | The `data validation program`_ processes every ten minutes as a cron job. 23 | It runs a programatic version of this query: 24 | 25 | .. code-block:: sql 26 | :emphasize-lines: 2 27 | 28 | // Do a select on all schools that have not been geo validated 29 | "SELECT id, lat, lon, country_code FROM schools WHERE date_geo_validated IS null AND lat IS NOT null AND lon IS NOT null" 30 | 31 | .. image:: https://cdn-images-1.medium.com/max/1000/1*5cHXnEW4C3qKKAIh0aIbhw.png 32 | 33 | 34 | Adding new attributes 35 | ===================== 36 | 37 | After the data is validated, the `magicbox-latlong-admin-server`_ updates attributes for some data. 38 | The attributes are only added if the engine returns an admin area ID for the target country (see :doc:`../administrative-boundaries`). 39 | 40 | #. ``date_geo_validated``: To ``CURRENT_TIMESTAMP`` 41 | 42 | #. ``coords_within_country``: true or false 43 | 44 | 45 | ************ 46 | Why we do it 47 | ************ 48 | 49 | Some CSV files we import and aggregate contain tens of thousands of records. 50 | Data validation is an expensive operation. 51 | We chose not to verify each school's coordinates in the ``before_batch_import`` hook for this reason. 52 | 53 | Instead, we opted for the `data validation program`_ to run as a cron job every ten minutes to accomplish this. 54 | 55 | 56 | .. _`magicbox-maps`: https://github.com/unicef/magicbox-maps 57 | .. _`magicbox-latlong-admin-server`: https://github.com/unicef/magicbox-latlong-server 58 | .. _`data validation program`: https://github.com/unicef/validate_geo_coordinates 59 | -------------------------------------------------------------------------------- /docs/data/data-visualization.rst: -------------------------------------------------------------------------------- 1 | ################## 2 | Data visualization 3 | ################## 4 | 5 | MagicBox is a full-stack application. 6 | While the back-end API (`magicbox-open-api`_) serves the data and makes it available for querying, the `magicbox-maps`_ application visualizes the data inside a world map. 7 | 8 | 9 | ************ 10 | How we do it 11 | ************ 12 | 13 | `magicbox-maps`_ draws the plot points using OpenStreetMap `Leaflet`_ and `WebGL`_. 14 | WebGL allows us to render up to `16 million clickable points`_ at once. 15 | 16 | To do this inside of a `React`_ application like ``magicbox-maps``, we created `react-webgl-leaflet`_. 17 | This React module enables us to render several plot points simultaneously, like we use for the school mapping data. 18 | Every school is represented with a unique, clickable point. 19 | 20 | .. figure:: /_static/data-visualization-webgl.png 21 | 22 | `react-webgl-leaflet`_ npm module 23 | 24 | This is currently used for `magicbox-maps`_. 25 | 26 | 27 | ************ 28 | Why we do it 29 | ************ 30 | 31 | Originally, we began using `Leaflet`_ markers. 32 | This worked well for Mauritania with 2,936 schools. 33 | However, when we moved to Colombia with 49,020 schools, performance took a significant hit. 34 | We experimented with a few other options, like heatmaps and clustering. 35 | 36 | .. image:: /_static/data-visualization-method-comparison.png 37 | 38 | In the end, we decided on using `WebGL`_, as explained above. 39 | 40 | 41 | .. _`magicbox-open-api`: https://github.com/unicef/magicbox-open-api 42 | .. _`magicbox-maps`: https://github.com/unicef/magicbox-maps 43 | .. _`Leaflet`: http://leafletjs.com/ 44 | .. _`WebGL`: https://www.khronos.org/webgl/ 45 | .. _`16 million clickable points`: https://stackoverflow.com/questions/16830824/google-maps-using-three-js-and-webgl/27653983#27653983 46 | .. _`React`: https://reactjs.org/ 47 | .. _`react-webgl-leaflet`: https://github.com/unicef/react-webgl-leaflet 48 | -------------------------------------------------------------------------------- /docs/index.rst: -------------------------------------------------------------------------------- 1 | .. MagicBox documentation master file, created by 2 | sphinx-quickstart on Tue Jan 16 13:57:12 2018. 3 | You can adapt this file completely to your liking, but it should at least 4 | contain the root `toctree` directive. 5 | 6 | MagicBox documentation 7 | ====================== 8 | 9 | MagicBox is an open-source platform that uses real-time information to inform 10 | life-saving humanitarian responses to emergency situations. It's composed of 11 | multiple GitHub repositories designed to ingest, aggregate, and serve data. 12 | 13 | .. toctree:: 14 | :maxdepth: 3 15 | :name: general 16 | :caption: General: 17 | 18 | administrative-boundaries 19 | 20 | .. toctree:: 21 | :maxdepth: 3 22 | :name: data 23 | :caption: Data: 24 | :glob: 25 | 26 | data/* 27 | 28 | .. toctree:: 29 | :maxdepth: 3 30 | :name: admin 31 | :caption: Administration: 32 | :glob: 33 | 34 | admin/* 35 | 36 | 37 | Indices and tables 38 | ================== 39 | 40 | * :ref:`genindex` 41 | * :ref:`modindex` 42 | * :ref:`search` 43 | -------------------------------------------------------------------------------- /docs/make.bat: -------------------------------------------------------------------------------- 1 | @ECHO OFF 2 | 3 | pushd %~dp0 4 | 5 | REM Command file for Sphinx documentation 6 | 7 | if "%SPHINXBUILD%" == "" ( 8 | set SPHINXBUILD=sphinx-build 9 | ) 10 | set SOURCEDIR=. 11 | set BUILDDIR=_build 12 | set SPHINXPROJ=MagicBox 13 | 14 | if "%1" == "" goto help 15 | 16 | %SPHINXBUILD% >NUL 2>NUL 17 | if errorlevel 9009 ( 18 | echo. 19 | echo.The 'sphinx-build' command was not found. Make sure you have Sphinx 20 | echo.installed, then set the SPHINXBUILD environment variable to point 21 | echo.to the full path of the 'sphinx-build' executable. Alternatively you 22 | echo.may add the Sphinx directory to PATH. 23 | echo. 24 | echo.If you don't have Sphinx installed, grab it from 25 | echo.http://sphinx-doc.org/ 26 | exit /b 1 27 | ) 28 | 29 | %SPHINXBUILD% -M %1 %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% 30 | goto end 31 | 32 | :help 33 | %SPHINXBUILD% -M help %SOURCEDIR% %BUILDDIR% %SPHINXOPTS% 34 | 35 | :end 36 | popd 37 | -------------------------------------------------------------------------------- /docs/requirements.txt: -------------------------------------------------------------------------------- 1 | ## Theme for Sphinx documentation 2 | sphinx 3 | sphinx_rtd_theme 4 | -------------------------------------------------------------------------------- /docs/test-docs.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env sh 2 | # 3 | # Script to automatically build and test the Sphinx documentation currently in 4 | # the repo. This script should always be run before submitting a new pull 5 | # request. 6 | # 7 | # If you're on Windows, please use the `make.bat` script in `docs/` directory. 8 | # 9 | 10 | cd "$(dirname "$0")" && make clean html 11 | 12 | -------------------------------------------------------------------------------- /project-index.md: -------------------------------------------------------------------------------- 1 | All MagicBox projects and repositories underneath the UNICEF GitHub organization are listed here. To learn more about each project, see the README in each repository. 2 | 3 | | Repository | Description | Maintainers | 4 | |------------|-------------|-------------| 5 | | [unicef/magicbox](https://github.com/unicef/magicbox) | Documentation and other general resources related to MagicBox | [@mikefab](https://github.com/mikefab), [@jflory7](https://github.com/jflory7) | 6 | | [unicef/magicbox-admins-to-airports](https://github.com/unicef/magicbox-admins-to-airports) | Given airports with incorrect administrative boundaries, query and check against Google API for location data and create CSV output for corrected administrative boundaries | [@mikefab](https://github.com/mikefab) | 7 | | [unicef/magicbox-aggregate-mobility](https://github.com/unicef/magicbox-aggregate-mobility) | Aggregate Amadeus mobility data by administrative boundary to produce CSV output | [@mikefab](https://github.com/mikefab), [@carloscdias](https://github.com/carloscdias) | 8 | | [unicef/-magicbox-aggregate-raster-admin](https://github.com/unicef/-magicbox-aggregate-raster-admin) | | [@mikefab](https://github.com/mikefab) | 9 | | [unicef/magicbox-aggregate-raster-shapefile](https://github.com/unicef/magicbox-aggregate-raster-shapefile) | | [@mikefab](https://github.com/mikefab) | 10 | | [unicef/magicbox-latlon-admin-server](https://github.com/unicef/magicbox-latlon-admin-server) | API that takes latitude/longitude pairs, and returns an ID and metadata for the administrative boundaries they're located within | [@mikefab](https://github.com/mikefab) | 11 | | [unicef/magicbox-maps](https://github.com/unicef/magicbox-maps) | | [@mikefab](https://github.com/mikefab), [@ayanez17](https://github.com/ayanez17) | 12 | | [unicef/magicbox-open-api](https://github.com/unicef/magicbox-open-api) | API to serve open data aggregated by administrative boundaries | [@mikefab](https://github.com/mikefab) | 13 | | [unicef/magicbox-maps-prototype](https://github.com/unicef/magicbox-maps-prototype) | A no-backend prototype that visualizes mobility, school connectivity, and other socio-economic indicators | [@mikefab](https://github.com/mikefab), [@thoat](https://github.com/thoat) | 14 | | [unicef/magicbox-kepler-demo](https://github.com/unicef/magicbox-kepler-demo) | A new UI for MagicBox Maps, making use of Uber's KeplerGL | [@mikefab](https://github.com/mikefab), [@marcellamaki](https://github.com/marcellamaki) | 15 | | [unicef/magicbox-aad-app](https://github.com/unicef/magicbox-aad-app) | Allows our ReactJS app to work with Microsoft Azure Active Directory | [@mikefab](https://github.com/mikefab) | 16 | | [unicef/magicbox-multihop-mobility](https://github.com/unicef/magicbox-multihop-mobility) | SQL code to calculate multi-hop mobility | [@marcellamaki](https://github.com/marcellamaki) | 17 | 18 | Note: Another way to find all MagicBox-related repos is to go to UNICEF GitHub mainpage and enter the search term "magicbox", like [here](https://github.com/unicef?utf8=%E2%9C%93&q=magicbox&type=&language=). There you'll be able to see which repos are under more active development than the others. 19 | -------------------------------------------------------------------------------- /public/images/admin_levels.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/public/images/admin_levels.png -------------------------------------------------------------------------------- /public/images/admin_zeros.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/unicef/magicbox/9f34639d00dffa28dc10ca82bbb42847c9f23a27/public/images/admin_zeros.png --------------------------------------------------------------------------------