├── content ├── CNAME ├── googlec9e1ed2a96ca548d.html ├── img │ ├── pypy1.PNG │ ├── blog-bg.png │ ├── troll-face.jpg │ ├── pypy-vs_cpython.png │ ├── lessons-in-dryness.jpg │ └── markdown! do you speak it?.jpg ├── BingSiteAuth.xml ├── article.md.template ├── implementing_scheme_pairs_in_python.md ├── paginator_code_review.md ├── pypy-joannah-nanjekye.md ├── quicksort-python.md ├── data-vis-brian-suda.md ├── pytest_maintaining_state.md ├── selenium-webscraping-tutorial.md ├── scrapy.md ├── tracking-vehicles-with-accesskenya-camera.md ├── decorators-in-python-mbithe-nzomo.md └── parametrizedUnitTestingWithPyTest.md ├── requirements.txt ├── .gitmodules ├── publishconf.py ├── circle.yml ├── .gitignore ├── fabfile.py ├── pelicanconf.py ├── develop_server.sh ├── README.md └── Makefile /content/CNAME: -------------------------------------------------------------------------------- 1 | blog.pynbo.or.ke 2 | -------------------------------------------------------------------------------- /requirements.txt: -------------------------------------------------------------------------------- 1 | pelican>=3.3 2 | markdown>=2.3.1 3 | ghp-import 4 | -------------------------------------------------------------------------------- /content/googlec9e1ed2a96ca548d.html: -------------------------------------------------------------------------------- 1 | google-site-verification: googlec9e1ed2a96ca548d.html -------------------------------------------------------------------------------- /content/img/pypy1.PNG: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Python-Nairobi/pynbo-blog/HEAD/content/img/pypy1.PNG -------------------------------------------------------------------------------- /content/img/blog-bg.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Python-Nairobi/pynbo-blog/HEAD/content/img/blog-bg.png -------------------------------------------------------------------------------- /content/img/troll-face.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Python-Nairobi/pynbo-blog/HEAD/content/img/troll-face.jpg -------------------------------------------------------------------------------- /content/BingSiteAuth.xml: -------------------------------------------------------------------------------- 1 | 2 | 3 | 109659DEA990D1B9D57D23FDD7455402 4 | -------------------------------------------------------------------------------- /content/img/pypy-vs_cpython.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Python-Nairobi/pynbo-blog/HEAD/content/img/pypy-vs_cpython.png -------------------------------------------------------------------------------- /content/img/lessons-in-dryness.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Python-Nairobi/pynbo-blog/HEAD/content/img/lessons-in-dryness.jpg -------------------------------------------------------------------------------- /content/img/markdown! do you speak it?.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/Python-Nairobi/pynbo-blog/HEAD/content/img/markdown! do you speak it?.jpg -------------------------------------------------------------------------------- /.gitmodules: -------------------------------------------------------------------------------- 1 | [submodule "pure-pynbo"] 2 | path = pure-pynbo 3 | url = https://github.com/Python-Nairobi/pure-pynbo.git 4 | [submodule "pelican-plugins"] 5 | path = pelican-plugins 6 | url = https://github.com/Python-Nairobi/pelican-plugins.git 7 | -------------------------------------------------------------------------------- /publishconf.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- # 3 | from __future__ import unicode_literals 4 | 5 | # This file is only used if you use `make publish` or 6 | # explicitly specify it as your config file. 7 | 8 | import os 9 | import sys 10 | sys.path.append(os.curdir) 11 | from pelicanconf import * 12 | 13 | SITEURL = 'pynbo.or.ke' 14 | RELATIVE_URLS = False 15 | 16 | FEED_ALL_ATOM = 'feeds/all.atom.xml' 17 | CATEGORY_FEED_ATOM = 'feeds/%s.atom.xml' 18 | 19 | DELETE_OUTPUT_DIRECTORY = True 20 | 21 | # Following items are often useful when publishing 22 | 23 | #DISQUS_SITENAME = "" 24 | #GOOGLE_ANALYTICS = "" 25 | -------------------------------------------------------------------------------- /circle.yml: -------------------------------------------------------------------------------- 1 | general: 2 | branches: 3 | ignore: 4 | - gh-pages 5 | 6 | checkout: 7 | post: 8 | - sed -i '/Host pure-pynbo/aHostname github.com' ~/.ssh/config 9 | - git submodule sync 10 | - git submodule update --init --recursive 11 | 12 | dependencies: 13 | override: 14 | - cd ~/pynbo-blog 15 | - pip install -r requirements.txt 16 | 17 | test: 18 | override: 19 | - ./develop_server.sh start 20 | - sleep 5 21 | - ./develop_server.sh stop 22 | 23 | deployment: 24 | master: 25 | branch: master 26 | commands: 27 | - sed -i s/circleci_github/cideploy/g ~/.ssh/config 28 | - ghp-import -p output 29 | -------------------------------------------------------------------------------- /content/article.md.template: -------------------------------------------------------------------------------- 1 | Title: { Replace with appropriate title } 2 | Date: { Replace with UTC ISO 8601 combined formated date time e.g 1969-12-31 23:59:59 } 3 | Tags: { Replace with comma separated tags e.g comma, separated, tags } 4 | Category: Relevant_Category 5 | Slug: { Replace with unique but readable slug e.g some_unique_slug } 6 | Author: { Replace with your name(s). Be consistent for subsequent articles. } 7 | Summary: { Replace this with a brief description of the article. This will show up on the articles index. } 8 | about_author: { Replace with a brief summary about you. } 9 | email: { Replace with your gravatar email. Used to display your [gravar](https://gravatar.com/) in the article. } 10 | 11 | Blog Post Content Goes Here. 12 | 13 | 14 | #INTRO 15 | Just a basic template to get you started. 16 | 17 | #STEPS 18 | 1. Replace the front mater at the beginning of this file with values relevant to your article. 19 | 2. Haxx the rest of the document to your liking in markdown. 20 | 21 | Also Feel free to Haxx this template into a full blown article creation guide. 22 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | # Byte-compiled / optimized / DLL files 2 | __pycache__/ 3 | *.py[cod] 4 | 5 | # C extensions 6 | *.so 7 | 8 | # Distribution / packaging 9 | .Python 10 | env/ 11 | build/ 12 | develop-eggs/ 13 | dist/ 14 | downloads/ 15 | eggs/ 16 | lib/ 17 | lib64/ 18 | parts/ 19 | sdist/ 20 | var/ 21 | *.egg-info/ 22 | .installed.cfg 23 | *.egg 24 | 25 | # PyInstaller 26 | # Usually these files are written by a python script from a template 27 | # before PyInstaller builds the exe, so as to inject date/other infos into it. 28 | *.manifest 29 | *.spec 30 | 31 | # Installer logs 32 | pip-log.txt 33 | pip-delete-this-directory.txt 34 | 35 | # Unit test / coverage reports 36 | htmlcov/ 37 | .tox/ 38 | .coverage 39 | .cache 40 | nosetests.xml 41 | coverage.xml 42 | 43 | # Translations 44 | *.mo 45 | *.pot 46 | 47 | # Django stuff: 48 | *.log 49 | 50 | # Sphinx documentation 51 | docs/_build/ 52 | 53 | # PyBuilder 54 | target/ 55 | 56 | # Vim, Pelican, Venv et. al. 57 | *.swp 58 | *.swo 59 | *.pid 60 | .venv 61 | venv 62 | output 63 | cache 64 | .DS_Store 65 | .ropeproject/ 66 | 67 | # emacs 68 | *~ 69 | -------------------------------------------------------------------------------- /fabfile.py: -------------------------------------------------------------------------------- 1 | from fabric.api import * 2 | import fabric.contrib.project as project 3 | import os 4 | import sys 5 | import SimpleHTTPServer 6 | import SocketServer 7 | 8 | # Local path configuration (can be absolute or relative to fabfile) 9 | env.deploy_path = 'output' 10 | DEPLOY_PATH = env.deploy_path 11 | 12 | # Remote server configuration 13 | production = 'root@localhost:22' 14 | dest_path = '/var/www' 15 | 16 | # Rackspace Cloud Files configuration settings 17 | env.cloudfiles_username = 'my_rackspace_username' 18 | env.cloudfiles_api_key = 'my_rackspace_api_key' 19 | env.cloudfiles_container = 'my_cloudfiles_container' 20 | 21 | 22 | def clean(): 23 | if os.path.isdir(DEPLOY_PATH): 24 | local('rm -rf {deploy_path}'.format(**env)) 25 | local('mkdir {deploy_path}'.format(**env)) 26 | 27 | def build(): 28 | local('pelican -s pelicanconf.py') 29 | 30 | def rebuild(): 31 | clean() 32 | build() 33 | 34 | def regenerate(): 35 | local('pelican -r -s pelicanconf.py') 36 | 37 | def serve(): 38 | os.chdir(env.deploy_path) 39 | 40 | PORT = 8000 41 | class AddressReuseTCPServer(SocketServer.TCPServer): 42 | allow_reuse_address = True 43 | 44 | server = AddressReuseTCPServer(('', PORT), SimpleHTTPServer.SimpleHTTPRequestHandler) 45 | 46 | sys.stderr.write('Serving on port {0} ...\n'.format(PORT)) 47 | server.serve_forever() 48 | 49 | def reserve(): 50 | build() 51 | serve() 52 | 53 | def preview(): 54 | local('pelican -s publishconf.py') 55 | 56 | def cf_upload(): 57 | rebuild() 58 | local('cd {deploy_path} && ' 59 | 'swift -v -A https://auth.api.rackspacecloud.com/v1.0 ' 60 | '-U {cloudfiles_username} ' 61 | '-K {cloudfiles_api_key} ' 62 | 'upload -c {cloudfiles_container} .'.format(**env)) 63 | 64 | @hosts(production) 65 | def publish(): 66 | local('pelican -s publishconf.py') 67 | project.rsync_project( 68 | remote_dir=dest_path, 69 | exclude=".DS_Store", 70 | local_dir=DEPLOY_PATH.rstrip('/') + '/', 71 | delete=True, 72 | extra_opts='-c', 73 | ) 74 | -------------------------------------------------------------------------------- /pelicanconf.py: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env python 2 | # -*- coding: utf-8 -*- # 3 | from __future__ import unicode_literals 4 | 5 | AUTHOR = u'Python Nairobi' 6 | SITENAME = u'#PyNBO' 7 | SITEURL = u'http://blog.pynbo.or.ke' 8 | TAGLINE = u'The Python-Nairobi Blog' 9 | 10 | PATH = 'content' 11 | 12 | TIMEZONE = 'Africa/Nairobi' 13 | 14 | DEFAULT_LANG = u'en' 15 | 16 | # Feed generation is usually not desired when developing 17 | FEED_ALL_ATOM = None 18 | CATEGORY_FEED_ATOM = None 19 | TRANSLATION_FEED_ATOM = None 20 | 21 | # Blogroll 22 | LINKS = (('PyNBO Meetup', 'http://meetup.com/Python-Nairobi/'), 23 | ('Python.org', 'http://python.org/'),) 24 | 25 | # Social widget 26 | SOCIAL = (('github', 'https://github.com/Python-Nairobi/'), 27 | ('meetup', 'http://www.meetup.com/Python-Nairobi/'), 28 | ('twitter', 'http://twitter.com/pynbo/'),) 29 | 30 | DEFAULT_PAGINATION = 10 31 | 32 | # Uncomment following line if you want document-relative URLs when developing 33 | RELATIVE_URLS = True 34 | 35 | # global metadata to all the contents 36 | # DEFAULT_METADATA = (('key', 'val'),) 37 | 38 | # path-specific metadata 39 | EXTRA_PATH_METADATA = { 40 | 'extra/robots.txt': {'path': 'robots.txt'}, 41 | } 42 | 43 | # static paths will be copied without parsing their contents 44 | STATIC_PATHS = [ 45 | 'img', 46 | 'js', 47 | 'css', 48 | 'extra/robots.txt', 49 | 'googlec9e1ed2a96ca548d.html', 50 | 'CNAME', 51 | 'BingSiteAuth.xml', 52 | ] 53 | 54 | # code blocks with line numbers 55 | PYGMENTS_RST_OPTIONS = {'linenos': 'table'} 56 | # DISQUS_SITENAME = "" 57 | PDF_GENERATOR = False 58 | REVERSE_CATEGORY_ORDER = True 59 | LOCALE = "C" 60 | DEFAULT_DATE = (1969, 12, 31, 23, 59, 59) 61 | THEME = "pure-pynbo" 62 | SHOW_ARTICLE_AUTHOR = True 63 | DISPLAY_PAGES_ON_MENU = False 64 | DISPLAY_CATEGORIES_ON_MENU = False 65 | DISPLAY_ARCHIVES_ON_MENU = False 66 | DEFAULT_DATE_FORMAT = ('%B %d, %Y') 67 | COVER_IMG_URL = 'img/blog-bg.png' 68 | PAGE_EXCLUDES = ['.git'] 69 | IGNORE_FILES = ['*.swp', '.git*'] 70 | 71 | PLUGIN_PATHS = ['./pelican-plugins'] 72 | PLUGINS = ['sitemap', 'gravatar'] 73 | GOOGLE_ANALYTICS = 'UA-54801851-1' 74 | READERS = {"html": None} 75 | -------------------------------------------------------------------------------- /develop_server.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | ## 3 | # This section should match your Makefile 4 | ## 5 | PY=${PY:-python} 6 | PELICAN=${PELICAN:-pelican} 7 | PELICANOPTS= 8 | 9 | BASEDIR=$(pwd) 10 | INPUTDIR=$BASEDIR/content 11 | OUTPUTDIR=$BASEDIR/output 12 | CONFFILE=$BASEDIR/pelicanconf.py 13 | 14 | ### 15 | # Don't change stuff below here unless you are sure 16 | ### 17 | 18 | SRV_PID=$BASEDIR/srv.pid 19 | PELICAN_PID=$BASEDIR/pelican.pid 20 | 21 | function usage(){ 22 | echo "usage: $0 (stop) (start) (restart) [port]" 23 | echo "This starts Pelican in debug and reload mode and then launches" 24 | echo "an HTTP server to help site development. It doesn't read" 25 | echo "your Pelican settings, so if you edit any paths in your Makefile" 26 | echo "you will need to edit your settings as well." 27 | exit 3 28 | } 29 | 30 | function alive() { 31 | kill -0 $1 >/dev/null 2>&1 32 | } 33 | 34 | function shut_down(){ 35 | PID=$(cat $SRV_PID) 36 | if [[ $? -eq 0 ]]; then 37 | if alive $PID; then 38 | echo "Stopping HTTP server" 39 | kill $PID 40 | else 41 | echo "Stale PID, deleting" 42 | fi 43 | rm $SRV_PID 44 | else 45 | echo "HTTP server PIDFile not found" 46 | fi 47 | 48 | PID=$(cat $PELICAN_PID) 49 | if [[ $? -eq 0 ]]; then 50 | if alive $PID; then 51 | echo "Killing Pelican" 52 | kill $PID 53 | else 54 | echo "Stale PID, deleting" 55 | fi 56 | rm $PELICAN_PID 57 | else 58 | echo "Pelican PIDFile not found" 59 | fi 60 | } 61 | 62 | function start_up(){ 63 | local port=$1 64 | echo "Starting up Pelican and HTTP server" 65 | shift 66 | $PELICAN --debug --autoreload -r $INPUTDIR -o $OUTPUTDIR -s $CONFFILE $PELICANOPTS & 67 | pelican_pid=$! 68 | echo $pelican_pid > $PELICAN_PID 69 | cd $OUTPUTDIR 70 | $PY -m pelican.server $port & 71 | srv_pid=$! 72 | echo $srv_pid > $SRV_PID 73 | cd $BASEDIR 74 | sleep 1 75 | if ! alive $pelican_pid ; then 76 | echo "Pelican didn't start. Is the Pelican package installed?" 77 | return 1 78 | elif ! alive $srv_pid ; then 79 | echo "The HTTP server didn't start. Is there another service using port 8000?" 80 | return 1 81 | fi 82 | echo 'Pelican and HTTP server processes now running in background.' 83 | } 84 | 85 | ### 86 | # MAIN 87 | ### 88 | [[ ($# -eq 0) || ($# -gt 2) ]] && usage 89 | port='' 90 | [[ $# -eq 2 ]] && port=$2 91 | 92 | if [[ $1 == "stop" ]]; then 93 | shut_down 94 | elif [[ $1 == "restart" ]]; then 95 | shut_down 96 | start_up $port 97 | elif [[ $1 == "start" ]]; then 98 | if ! start_up $port; then 99 | shut_down 100 | fi 101 | else 102 | usage 103 | fi 104 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | The Python Nairobi Blog Repo 2 | ========== 3 | 4 | ## INTRO 5 | This repository contains the sources used to build the python nairobi blog. 6 | We use [pelican](http://docs.getpelican.com) to build the contents of this repository 7 | into the site you see at http://blog.pynbo.or.ke or http://python-nairobi.github.io/pynbo-blog 8 | 9 | ## PREREQUIESITES 10 | To contribute, we recommend you install:- 11 | 12 | - [Python >= 2.7](https://www.python.org/download/releases/2.7) 13 | - [PIP]("http://www.pip-installer.org/en/latest/installing.html) 14 | - [virtualenv](http://www.virtualenv.org/en/latest/virtualenv.html) 15 | 16 | ## CLONING 17 | 1. Click here to fork this Repo. 18 | 2. Clone your fork and cd into it. 19 | 3. Create a virtualenv for your repo:- `mkvirtualenv -a . pynbo-blog`. 20 | 4. Install the remaining dependencies:- `pip install -r requirements.txt`. 21 | 5. Initialize & update submodule dependencies (plugins, theme & skulpt interpreter): `git submodule update --init --recursive`. 22 | 6. Then create a branch, naming it along the lines as your topic of contribution. 23 | 24 | ## EDITING 25 | Start the development server:- `./develop_server.sh start`. Open http://127.0.0.1:8000 to view. 26 | Any changes you make will automatically be reflected there. 27 | 28 | 29 | ### MARKDOWN 30 | ![Markdown, do you speak it?](content/img/markdown!%20do%20you%20speak%20it%3F.jpg) 31 | We write our articles in [MarkDown](https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet) 32 | from the [`content`](content/) folder. 33 | Your markdown files should end with a `.md`, `.markdown`, or `.mkd` file extension. 34 | You should also include some metadata at the top of your markdown files. 35 | Checkout the example bellow and adapt appropriately:- 36 | 37 | ``` 38 | Title: Blog post title 39 | Date: YYYY-MM-DD HH:MM 40 | Tags: tag1, tag2 41 | Category: Relevant_Category 42 | Slug: short_descriptive_article_name 43 | Author: Your Name (github_handle) 44 | Summary: A summary of your blog post content 45 | about_author: A brief summary about you 46 | email: used to display your [gravar](https://gravatar.com/) in the article. 47 | 48 | Blog Post Content Goes Here. 49 | ``` 50 | You can also `cp content/article.md.template content/new_article.md` and `vi content/new_article.md` appropriately. 51 | Feel free to write portions of your article in html to compensate for where markdown falls short 52 | e.g. opening links in new tabs. (Jules is ok with this) 53 | 54 | ## FINALLY 55 | 1. Add, commit and push your changes. 56 | 2. Send us a pull request against [master](https://github.com/Python-Nairobi/pynbo-blog/tree/master). 57 | 58 | We'll review your contribution and maybe ask you to make further changes before we merge. 59 | -------------------------------------------------------------------------------- /content/implementing_scheme_pairs_in_python.md: -------------------------------------------------------------------------------- 1 | --- 2 | Title: Implementing Lisp pairs in Python 3.6.1 3 | Date: 2017-09-07 09:00 4 | Categories: misc 5 | Slug: lisp_pairs_in_python 6 | Author: Bonface K. Munyoki 7 | Summary: I discuss how to implement Scheme's basic pair data structure in Python for fun using Message Passing [Scheme is a Lisp dialect] 8 | about_author: Linux enthusiast. Audiophile. Huge anime fan. "Retired" amateur competitive swimmer. Tinkeror. Former typography nerd. 9 | email: bonfacemunyoki@gmail.com 10 | 11 | --- 12 | 13 | *I wrote this article earlier on my blog here.* 14 | 15 | I've come to like Python. It makes rapid prototyping things way easier. I also appreciate that it treats functions as first class citizens, something that (maybe) many Python hackers do not know. I thought it'd be cool to implement one of Scheme's(Scheme is a Lisp variant) basic structures, the ***pair***, in Python. I'll use a form of Message Passing to do this(more on this later). 16 | 17 | First, here's what a pair looks like in Scheme: 18 | 19 | ``` lisp 20 | ;; We define a as a pair comprising 1 and 3 21 | (def a (cons 1 3)) 22 | 23 | ;; We access the first element of a pair by 24 | ;; running (car ). In our case, this 25 | ;; would give: 1 26 | (car a) 27 | 28 | ;; We access the second element of a pair by 29 | ;; running (cdr ). This will give: 3 30 | (cdr a) 31 | ``` 32 | 33 | Here's how our implementation of `cons`, `car`, `cdr` looks like: 34 | 35 | ``` 36 | def cons(x, y): 37 | def dispatch(m): 38 | if m == "car": 39 | return x 40 | elif m == "cdr": 41 | return y 42 | else: 43 | print("error dude") 44 | return dispatch 45 | ``` 46 | 47 | Here, we define an internal procedure `dispatch` that receives some "message" and acts on it. If the "message" is a `car` it will return the first element of cons' arguments. If it's a `cdr`, it'll return the second element; otherwise, a simple error message is printed. Our `cons` function returns a procedure as its return value. As we shall see later, our "message" will be passed to this return value. Now let's create our `car` and `cdr` functions. 48 | 49 | ``` 50 | def car(z): 51 | return z("car") 52 | 53 | def cdr(z): 54 | return z("cdr") 55 | ``` 56 | 57 | Both `car` and `cdr` take a pair(a cons object) as it's arguments. The right value is returned depending on the "message passed". The name "message passing" comes from the image that a data object(in our case the pairs) is an entity that receives the requested operation name as a "message". Let's create some fancy pairs :) 58 | ``` 59 | x = cons(2, 3) 60 | 61 | # Let's print the first element of x: 62 | print(car(x)) 63 | 64 | # Let's print the second element of x: 65 | print(cdr(x)) 66 | ``` 67 | -------------------------------------------------------------------------------- /content/paginator_code_review.md: -------------------------------------------------------------------------------- 1 | --- 2 | Title: Pagination Review of Django's and Pelican's Implementation 3 | Date: 2019-03-12 17:30 4 | Categories: misc 5 | Slug: pagination-review-of-django-pelican-implementation 6 | Author: John Nduli 7 | Summary: I discuss how pagination is implemented in both pelican and django, and how this implementation can generally be used 8 | about_author: Linux enthusiast, Anime + Manga Fan, Audiophile, Tinkerer 9 | email: yohanaizraeli@gmail.com 10 | --- 11 | 12 | *I wrote this article earlier on my blog here.* 13 | 14 | 15 | Pagination is the process of dividing up a document into discrete pages 16 | ([wikipedia](https://en.wikipedia.org/wiki/Pagination)). Django and 17 | Pelican have similar pagination implementations. This boils down to 18 | having a class that has accepts a sliceable object as one of its params, 19 | and returns a page containing list of items depending on number of items 20 | per page. The django [paginator 21 | docs](https://docs.djangoproject.com/en/2.1/topics/pagination/#required-arguments) 22 | describe the object list as a list, tuple, QuerySet, or other sliceable 23 | object with a count() or __len__() method. 24 | 25 | ```python 26 | paginator = Paginator(object_list=list, per_page=10) 27 | ``` 28 | 29 | 30 | To get the contents of a particular page one calls the get_page method: 31 | 32 | ```python 33 | paginator.get_page(2) 34 | ``` 35 | 36 | What this does is to first check if the number provided is valid using 37 | the validate_number method. A number is valid if its an integer, it's 38 | greater than 0 and is less that the total number of pages in the 39 | paginator (if the alloy_empty_first_page parameter was set to True, 1 40 | will be valid even if the total number of pages is 0). 41 | 42 | If the number was valid the page method is called. This creates the 43 | required start and end slices i.e.`start:end` on the object list, and 44 | returns a Page object that only contains these elements. 45 | 46 | The code below shows some boiler plate that can achieve this: 47 | 48 | ```python 49 | class Paginator: 50 | 51 | def __init__(self, object_list, per_page, 52 | allow_empty_first_page=True): 53 | self.object_list = object_list 54 | self.per_page = int(per_page) 55 | self.orphans = int(orphans) 56 | self.allow_empty_first_page = allow_empty_first_page 57 | 58 | def validate_number(self, number): 59 | # Make sure number is integer, greater than 1 and less than 60 | # the total number of pages 61 | 62 | def get_page(self, number): 63 | # if number is valid, returns a page object. 64 | # This is done by getting range of items required 65 | # e.g. 11:20, and returning an object that one can iterate 66 | # through (called a Page object) 67 | 68 | def num_pages(self): 69 | # Returns the total number of pages 70 | 71 | def count(self): 72 | # Returns total number of objects 73 | ``` 74 | 75 | To loop over the page object with a for, the Page class implements the 76 | __getitem__ for this. Pelican's implementation does not have this method 77 | though. It also has some useful methods like: 78 | 79 | + `has_next` 80 | + `has_previous` 81 | + `has_other_pages` 82 | + `next_page_number` 83 | + `previous_page_number` 84 | 85 | With these one can effectively get the contents of a page by looping 86 | through the paginator (in django only) and have various logic on which 87 | page numbers to display. 88 | 89 | The example provided in the [django docs 90 | here](https://docs.djangoproject.com/en/2.1/topics/pagination/#using-paginator-in-a-view) 91 | shows a really good usage of the various methods. 92 | -------------------------------------------------------------------------------- /Makefile: -------------------------------------------------------------------------------- 1 | PY?=python 2 | PELICAN?=pelican 3 | PELICANOPTS= 4 | 5 | BASEDIR=$(CURDIR) 6 | INPUTDIR=$(BASEDIR)/content 7 | OUTPUTDIR=$(BASEDIR)/output 8 | CONFFILE=$(BASEDIR)/pelicanconf.py 9 | PUBLISHCONF=$(BASEDIR)/publishconf.py 10 | 11 | FTP_HOST=localhost 12 | FTP_USER=anonymous 13 | FTP_TARGET_DIR=/ 14 | 15 | SSH_HOST=localhost 16 | SSH_PORT=22 17 | SSH_USER=root 18 | SSH_TARGET_DIR=/var/www 19 | 20 | S3_BUCKET=my_s3_bucket 21 | 22 | CLOUDFILES_USERNAME=my_rackspace_username 23 | CLOUDFILES_API_KEY=my_rackspace_api_key 24 | CLOUDFILES_CONTAINER=my_cloudfiles_container 25 | 26 | DROPBOX_DIR=~/Dropbox/Public/ 27 | 28 | GITHUB_PAGES_BRANCH=gh-pages 29 | 30 | DEBUG ?= 0 31 | ifeq ($(DEBUG), 1) 32 | PELICANOPTS += -D 33 | endif 34 | 35 | help: 36 | @echo 'Makefile for a pelican Web site ' 37 | @echo ' ' 38 | @echo 'Usage: ' 39 | @echo ' make html (re)generate the web site ' 40 | @echo ' make clean remove the generated files ' 41 | @echo ' make regenerate regenerate files upon modification ' 42 | @echo ' make publish generate using production settings ' 43 | @echo ' make serve [PORT=8000] serve site at http://localhost:8000' 44 | @echo ' make devserver [PORT=8000] start/restart develop_server.sh ' 45 | @echo ' make stopserver stop local server ' 46 | @echo ' make ssh_upload upload the web site via SSH ' 47 | @echo ' make rsync_upload upload the web site via rsync+ssh ' 48 | @echo ' make dropbox_upload upload the web site via Dropbox ' 49 | @echo ' make ftp_upload upload the web site via FTP ' 50 | @echo ' make s3_upload upload the web site via S3 ' 51 | @echo ' make cf_upload upload the web site via Cloud Files' 52 | @echo ' make github upload the web site via gh-pages ' 53 | @echo ' ' 54 | @echo 'Set the DEBUG variable to 1 to enable debugging, e.g. make DEBUG=1 html' 55 | @echo ' ' 56 | 57 | html: 58 | $(PELICAN) $(INPUTDIR) -o $(OUTPUTDIR) -s $(CONFFILE) $(PELICANOPTS) 59 | 60 | clean: 61 | [ ! -d $(OUTPUTDIR) ] || rm -rf $(OUTPUTDIR) 62 | 63 | regenerate: 64 | $(PELICAN) -r $(INPUTDIR) -o $(OUTPUTDIR) -s $(CONFFILE) $(PELICANOPTS) 65 | 66 | serve: 67 | ifdef PORT 68 | cd $(OUTPUTDIR) && $(PY) -m pelican.server $(PORT) 69 | else 70 | cd $(OUTPUTDIR) && $(PY) -m pelican.server 71 | endif 72 | 73 | devserver: 74 | ifdef PORT 75 | $(BASEDIR)/develop_server.sh restart $(PORT) 76 | else 77 | $(BASEDIR)/develop_server.sh restart 78 | endif 79 | 80 | stopserver: 81 | kill -9 `cat pelican.pid` 82 | kill -9 `cat srv.pid` 83 | @echo 'Stopped Pelican and SimpleHTTPServer processes running in background.' 84 | 85 | publish: 86 | $(PELICAN) $(INPUTDIR) -o $(OUTPUTDIR) -s $(PUBLISHCONF) $(PELICANOPTS) 87 | 88 | ssh_upload: publish 89 | scp -P $(SSH_PORT) -r $(OUTPUTDIR)/* $(SSH_USER)@$(SSH_HOST):$(SSH_TARGET_DIR) 90 | 91 | rsync_upload: publish 92 | rsync -e "ssh -p $(SSH_PORT)" -P -rvzc --delete $(OUTPUTDIR)/ $(SSH_USER)@$(SSH_HOST):$(SSH_TARGET_DIR) --cvs-exclude 93 | 94 | dropbox_upload: publish 95 | cp -r $(OUTPUTDIR)/* $(DROPBOX_DIR) 96 | 97 | ftp_upload: publish 98 | lftp ftp://$(FTP_USER)@$(FTP_HOST) -e "mirror -R $(OUTPUTDIR) $(FTP_TARGET_DIR) ; quit" 99 | 100 | s3_upload: publish 101 | s3cmd sync $(OUTPUTDIR)/ s3://$(S3_BUCKET) --acl-public --delete-removed --guess-mime-type 102 | 103 | cf_upload: publish 104 | cd $(OUTPUTDIR) && swift -v -A https://auth.api.rackspacecloud.com/v1.0 -U $(CLOUDFILES_USERNAME) -K $(CLOUDFILES_API_KEY) upload -c $(CLOUDFILES_CONTAINER) . 105 | 106 | github: publish 107 | ghp-import -b $(GITHUB_PAGES_BRANCH) $(OUTPUTDIR) 108 | git push origin $(GITHUB_PAGES_BRANCH) 109 | 110 | .PHONY: html help clean regenerate serve devserver publish ssh_upload rsync_upload dropbox_upload ftp_upload s3_upload cf_upload github 111 | -------------------------------------------------------------------------------- /content/pypy-joannah-nanjekye.md: -------------------------------------------------------------------------------- 1 | Title: Faster python programs with pypy by Joannah Nanjekye 2 | Date: 2017-03-06 18:11:00 3 | Tags: pypy, python 4 | Slug: pypy_by_joannah_nanjekye 5 | Author: Joannah Nanjekye 6 | Summary: A summary of the presentation on pypy Joannah Nanjekye gave. 7 | email: nanjekyejoannah@gmail.com 8 | about_author:

Software Engineer, Aeronautical Engineer to be, Straight Outta 256 , I choose Results over Reasons, Passionate Aviator, Show me the code

I share my thoughts majorly here https://nanjekyejoannah.github.io/

9 | 10 | Pypy is an alternate implementation of the python programming language. Pypy started out as a python interpreter written in python. It is aimed at being compatible with cpython and currently with experimental compatibility for CPython C API extensions. 11 | 12 | Pypy originally refered to two things. one the python interpreter and Rpython Translation toolchain but currently pypy is always used to refer to the python interpreter. The translation framework is always refered to as the Rpython Translation Framework. 13 | 14 | It is the distinct features embedded in pypy that give your python programs the magical performance. Lets have a look at some of them. 15 | 16 | # Pypy Distinct Features 17 | 18 | The pypy intepreter offers a couple of distinct features namely; 19 | 20 | * Speed 21 | * Compatibility 22 | * Memory Usage 23 | * Stackless python features 24 | 25 | ##Speed 26 | 27 | Pypy is magically faster due to its high performance Just-in-time compiler and garbage collector. 28 | 29 | It is therefore faster for programs that are JIT susceptible. This means that pypy may not always run faster, it may be for pure python code but actually run slower than Cpython where the JIT cant help. 30 | 31 | pypy may also not be as fast for small scripts that do not give the Just-in-time compiler enough warm up time. 32 | 33 | I ran this [code](https://gist.github.com/mrmoje/30bacf757e400ef28c56fdaecfb0b2a8) on pypy and on normal python and the results show pypy is actually faster. I will go on share the outcomes below; 34 | 35 | 36 | 37 | Alot of benchmarks have shown pypy performing better than Cpython and its not getting slower with time. Each new version of pypy is faster than its predecessor. The pypy team have shared good insight at their [speed center](http://speed.pypy.org/). 38 | 39 | 40 | ##Compatibility 41 | 42 | Good news is pypy is compatible with most of the python Libraries. 43 | 44 | * Django 45 | * Flask 46 | * Bottle 47 | * Pylons 48 | * Pyramid 49 | * Twisted 50 | * lxml 51 | * Beautiful Soup 52 | * Sphinx 53 | * IPyton 54 | * PIL/Pillow 55 | * Psycopg2 56 | * MySQL-Python 57 | * pysqlite 58 | * pymongo 59 | * cx_Oracle 60 | * SQLAlchemy 61 | * Gunicorn 62 | * Requests 63 | * nose 64 | * pytest 65 | * celery 66 | * Pip 67 | * Numpy 68 | * Scipy 69 | * Gevent 70 | 71 | Pypy has Experimental support for Extension modules through cpyext. It can run most C extensions these days. 72 | 73 | ##Memory Usage 74 | 75 | Memory-intensive python programs take lesser space than when running on pypy . However this may not be the case always though due to some details. 76 | 77 | ##Stackless python Features 78 | 79 | Pypy exposes language features similar to the ones present in stackless python. 80 | 81 | #Differences between Cpython and Python 82 | 83 | Lets take a look at pypy and a reference python implementation called cpython. 84 | 85 | 86 | 87 | 88 | # Way Forward 89 | 90 | “If you want your code to run faster, you should probably just use PyPy.” — Guido van Rossum (creator of Python) 91 | 92 | # Installation 93 | 94 | It is easy to install 95 | 96 | ###Linux:- 97 | 98 | `sudo apt-get install pypy` 99 | 100 | ###Mac:- 101 | 102 | `sudo brew install pypy` 103 | 104 | ###Windows:- 105 | 106 | There is rich [Documentation](http://pypy.org/download.html) on installation for windows. 107 | 108 | 109 | # Check these out for more pypy inspiration 110 | * 111 | * 112 | * 113 | * 114 | -------------------------------------------------------------------------------- /content/quicksort-python.md: -------------------------------------------------------------------------------- 1 | Title: Quicksort in python 2 | Tags: python, algorithms,quicksort 3 | Date: 2016-09-19 19:51:00 4 | Slug: quicksort-python 5 | Summary: A brief demo of quicksort in python 6 | Author: Kenny Rachuonyo 7 | email: krm@99nth.com 8 | about_author:

A computer programmer in Nairobi, and fan of Python. You can check out his homepage here. 9 | 10 | Quicksort is one of the common sorting algorithms taught in computer science. 11 | 12 | Here I shall attempt to give a brief and clear example in Python. 13 | 14 | As a divide end conquer algorithm, three main steps are involved: 15 | 16 | - Pick an element (pivot) in the list 17 | - Partition the list so that all elements smaller than the pivot 18 | come before it and those bigger than the pivot come after it 19 | - Recursively apply the previous steps to the smaller sublists 20 | before and after the pivot 21 | 22 | This should leave you with a fully sorted list at the end. 23 | 24 | Let's define a quicksort function 25 | 26 | :::python 27 | def quicksort(L, lo, hi): 28 | """Sorts elements between indices lo and hi inclusive 29 | 30 | L - a list to sort 31 | lo - index of the lower end in the range 32 | hi - index of the higher end""" 33 | 34 | # Base case: lo and hi are equal, i.e. only one element in sublist 35 | # In this case, nothing is done on the list 36 | 37 | if lo < hi: 38 | # lo is less than hi, i.e. at least two elements in sublist 39 | 40 | # the partitioning step, p is the final position of the 41 | # pivot after partitioning 42 | p = partition(L, lo, hi) 43 | 44 | # Recursively sort the 'less than' partition 45 | quicksort(L, lo, p - 1) 46 | 47 | # Recursively sort the 'greater than' partition 48 | quicksort(L, p + 1, hi) 49 | 50 | # and that's it :-) 51 | 52 | 53 | We then define the partition function that does the actual work. It picks an 54 | element in the list within the given range, and divides the list into segments 55 | less than or equal, and greater than the pivot. 56 | 57 | 58 | :::python 59 | def partition(L, lo, hi): 60 | """Partitions the list within the given range 61 | L - a list to partition 62 | lo - index of the lower end in list to start partitioning from 63 | hi - index of higher end in list to end the partitioning""" 64 | 65 | # There several schemes used to pick the pivot 66 | # Here we shall use a one known as the 'Lomuto partition scheme' 67 | # Where we simply pick the last item in the range as the pivot 68 | 69 | pivot = L[hi] 70 | 71 | # i is the next position in the list where we 72 | # place an element less than or equal to the pivot 73 | 74 | # We begin at the lower end 75 | i = lo 76 | 77 | # We iterate through the list from lo to hi - 1 (the pivot is at hi, remember?) 78 | # separating elements less than or equal to the pivot 79 | # from those greater than the pivot 80 | j = lo 81 | while j < hi: 82 | # if element at j is less than or equal to the pivot 83 | # swap it into location i 84 | if L[j] <= pivot: 85 | L[i], L[j] = L[j], L[i] 86 | i += 1 # and increment i 87 | 88 | # increment j 89 | j += 1 90 | 91 | # When the loop completes, we know that all elements before i are less than 92 | # or equal to the pivot, and all elements from i onwards are greater than 93 | # the pivot 94 | 95 | # swap the pivot into it's correct position, separating these two parts 96 | L[i], L[hi] = L[hi], L[i] 97 | 98 | # Now the pivot is at position i, and all elements after i are greater than 99 | # it 100 | 101 | # return its position 102 | return i 103 | 104 | We can now test our quicksort function. 105 | 106 | :::python 107 | import random 108 | 109 | # Create a list of 10 unsorted integers between 1 and 100 inclusive 110 | list_to_sort = [random.randint(1, 100) for i in range(10)] 111 | print("List before sorting: ", list_to_sort) 112 | 113 | # Now let's sort the list 114 | last_index = len(list_to_sort) - 1 115 | quicksort(list_to_sort, 0, last_index) 116 | 117 | print("List after sorting: ", list_to_sort) 118 | 119 | 120 | You can read more on quicksort in its 121 | [wikipedia page](https://en.wikipedia.org/wiki/Quicksort). 122 | 123 | And [here's](https://www.safaribooksonline.com/library/view/python-cookbook/0596001673/ch02s12.html) 124 | an implementation in only 3 lines of python, if your into that sort of thing. 125 | 126 | [Here's](https://github.com/krmboya/py-examples/blob/master/quicksort.py) the source code. 127 | 128 | Also posted [here](http://www.99nth.com/~krm/blog/quicksort-python.html) 129 | -------------------------------------------------------------------------------- /content/data-vis-brian-suda.md: -------------------------------------------------------------------------------- 1 | Title: Data Vizualization by Brian Suda 2 | Date: 2014-12-16 18:11:00 3 | Tags: data viz, python, deterministic design, iceland, brian suda 4 | Slug: data_viz_by_brian_suda 5 | Author: Brian Suda 6 | Summary: A round up and links from the presentation Brian Suda gave about data vizualizations 7 | email: brian@suda.co.uk 8 | about_author:

Brian Suda is an informatician currently residing in Reykjavík, Iceland. He has spent a good portion of each day connected to Internet after discovering it back in the mid-1990s.

Most recently, he has been focusing more on the mobile space and future predictions, how smaller devices will augment our every day life and what that means to the way we live, work and are entertained.

His own little patch of Internet can be found at http://suda.co.uk where many of his past projects and crazy ideas can be found.

9 | 10 | Thanks to everyone who came on their Saturday to listen to me talk about data visualisations through a computer screen several timezones and many kilometres away. I know sitting through online presentations can be boring, so I hope mine was interesting enough that not many people fell asleep or left. It doesn’t matter, I could only see my slides and hear the shuffling of chairs every once and awhile, so I knew people were still there. 11 | 12 | On Saturday, December 13th, I ran through my presentation about the basics of charts & graphs and how build-up to more complex data visualizations. It was a great group to chat with because it was the a Python meet-up. Most of the event I attend it is a mix of disciplines, so you can never get too technical, or even worse, it is a group of designers who you at you with a blank expression when you start to talk about code and scripts. 13 | 14 | Getting to share some ideas with the folks in Nairobi was an exciting experience, and I know you are the types of people who get it. You’ll take apart the code and understand what’s happening. 15 | 16 | # Deterministic Design 17 | 18 | One of the topics I’m really interested in is this idea of Deterministic Design. 19 | 20 | This is the sort of design that you do once, up front, then feed in the data and see how it looks. Then if you need to update the design, you don’t do it for that particular instance, but back upstream at the source where the code is written. That way, next time the run the code, all future designs will benefit. 21 | 22 | At the moment, I’m calling this Deterministic Design, because if you feed in the same data, you’ll get the same results every time. Much like the color picking example I showed in the presentation. Using the MD5 function, we can get a hex string, which we take the first 6 characters and use that to generate a unique color that is fully reproducible on any system or language. 23 | 24 | :::python 25 | import hashlib 26 | hh = hashlib.md5() 27 | hh.update("Hello World") 28 | hh.hexdigest()[:6] 29 | 30 | I use this equation everywhere. It saves time and thought, but the downside is that you are bound by the color that it returns, even if it isn’t pretty! This little snippet of code needs a friend to help make sure you have the highest color contrast possible. I wrote about this on [24ways Calculating Color Contrast](http://24ways.org/2010/calculating-color-contrast/). You’ll have to port the code to Python yourself. 31 | 32 | I love a lot of these little projects, to convert small ideas into some code that is repeatable over and over again. I also like the UNIX philosophy of small, reusable pieces. This means I tend to write small programs which take a CSV as data and create small SVG files as output. Then I can bring those into more complex programs to edit, annotate and layout the data. If the CSV changes, it is easy for me to reproduce the charts & graphs, because it is all done in code. 33 | 34 | Most of the simple tools that I have written are available on [GitHub](https://github.com/briansuda/Deterministic-Design). There is a lot to wade through and you’ll have to port it to Python, but they are simple, useful utilities. 35 | 36 | # Starting Small and Growing Bigger 37 | When starting off, it is important to just start small. Rather than trying to dig into a massive amount of social media data, why not track your hours each week. How much is spent at work, at home, a sleep. Then try to visualize that in an interesting way. With some practice you’ll figure out what works, what doesn’t and what tools you like to use in your workflow. 38 | 39 | After that you can begin to progress to more complex data sets. Start off with some you like and understand, build-up a small portfolio of interesting examples and then take on bigger and bigger projects. You’ll quickly see that even the biggest project is just a series of smaller ones that you probably have some experience or tools to deal with. 40 | 41 | # Inspiration 42 | To stay fresh, you need to keep an eye on interesting people in the field. There are a lot of resources out there and these are a few to get you started. 43 | 44 | * 45 | * 46 | * 47 | * 48 | 49 | If anyone has any questions, they can always contact me on twitter [@briansuda](https://twitter.com/briansuda) or follow along with what I write on [(optional.is)](http://optional.is/required) and [join the mailing list](http://optional.is/newsletter). I can’t guarantee that everything will be Data viz related, but it will be interesting! 50 | -------------------------------------------------------------------------------- /content/pytest_maintaining_state.md: -------------------------------------------------------------------------------- 1 | --- 2 | Title: Maintaining states while using Pytest 3 | Date: 2018-09-10 9:30 4 | Tags: python, pytest 5 | Category: unit testing 6 | Slug: test_states 7 | Author: Michael Bukachi 8 | Summary: Maintaining states while using pytest 9 | about_author: Michael is a Software Engineer at Mobidev Kenya Limited 10 | email: michaelbukachi@gmail.com 11 | --- 12 | 13 | ## Pre-requisites 14 | > - Beginner level Python 15 | > - Python2* or Python3* installed 16 | > - Basic knowledge on TDD and python UnitTesting 17 | > - Pytest installed 18 | 19 | 20 | #Intro 21 | 22 | ![alt text](https://cdn-images-1.medium.com/max/1600/1*7dFF3N2BwzlPz5MHA3_eyA.gif "When things go wrong") 23 | 24 | Ah, testing, a topic many devs don't like hearing. I really don't blame them. Testing is _tricky_. 25 | It's difficult to get it right and it's always evolving. I personally struggled with it for quite 26 | some time before I got the hang of it. But it is **necessary**. These days I can't start a project before making sure 27 | that there are testing frameworks available for the specific language I am using. 28 | 29 | **Disclaimer:** This guide is not an introduction to testing using pytest. For that, just open 30 | [our friendly search engine](https://www.google.com) and search "getting started with pytest". Trust me, you won't 31 | regret it. 32 | 33 | ![](https://data.whicdn.com/images/86201703/original.gif "You're welcome") 34 | 35 | #Getting started 36 | For this guide, I'll be using flask for demonstration. The demo project is available 37 | [here](https://github.com/michaelbukachi/flask_pystest_state.git). The basic structure of the project as follows: 38 | ``` 39 | ├── app 40 | │   ├── __init__.py 41 | │   └── views.py 42 | ├── requirements.txt 43 | └── test 44 | ├── conftest.py 45 | ├── __init__.py 46 | └── test_token.py 47 | 48 | ``` 49 | 50 | **app/__init__.py** 51 | ``` 52 | from flask import Flask 53 | 54 | 55 | def create_app(): 56 | app = Flask(__name__) 57 | from .views import auth 58 | app.register_blueprint(auth) 59 | return app 60 | 61 | ``` 62 | Notice that I've used the lazy pattern so that it is easy to test. 63 | 64 | **app/views.py** 65 | ``` 66 | from uuid import uuid4 67 | 68 | from flask import Blueprint, jsonify, request 69 | 70 | auth = Blueprint('auth', __name__) 71 | 72 | TOKEN = str(uuid4()) 73 | 74 | 75 | @auth.route('/token') 76 | def get_token(): 77 | return jsonify(TOKEN) 78 | 79 | 80 | @auth.route('/secure', methods=['POST']) 81 | def secure_page(): 82 | args = request.get_json(force=True) 83 | if 'token' in args: 84 | if args['token'] == TOKEN: 85 | return jsonify('This is a secure page') 86 | 87 | res = jsonify('Unauthorized') 88 | res.status_code = 401 89 | return res 90 | 91 | ``` 92 | 93 | **test/conftest.py** 94 | ``` 95 | import json 96 | 97 | import pytest 98 | from flask import Response 99 | from flask.testing import FlaskClient 100 | from werkzeug.utils import cached_property 101 | 102 | from app import create_app 103 | 104 | 105 | class JSONResponse(Response): 106 | 107 | @cached_property 108 | def json(self): 109 | return json.loads(self.get_data(as_text=True)) 110 | 111 | 112 | @pytest.fixture('session') 113 | def flask_app(): 114 | app = create_app() 115 | yield app 116 | 117 | 118 | @pytest.fixture('session') 119 | def client(flask_app): 120 | app = flask_app 121 | ctx = flask_app.test_request_context() 122 | ctx.push() 123 | app.test_client_class = FlaskClient 124 | app.response_class = JSONResponse 125 | return app.test_client() 126 | 127 | ``` 128 | 129 | **test/test_token.py** 130 | ``` 131 | import pytest 132 | from flask.testing import FlaskClient 133 | 134 | 135 | @pytest.fixture('module') 136 | def token(): 137 | return __name__ + ':token' 138 | 139 | 140 | def test_get_token(client: FlaskClient, token, request): 141 | res = client.get('/token') 142 | assert res.status_code == 200 143 | token_ = res.json 144 | request.config.cache.set(token, token_) 145 | 146 | 147 | def test_secure_page(client: FlaskClient, token, request): 148 | token_ = request.config.cache.get(token, None) 149 | res = client.post('/secure', json={}) 150 | assert res.status_code == 401 151 | res = client.post('/secure', json={'token': token_}) 152 | assert res.status_code == 200 153 | 154 | ``` 155 | 156 | For those who have not used pytest before, pytest basically uses dependency injection to 157 | provide dependencies to tests. This has numerous applications, from providing dummy data to providing 158 | configurations and constants. If you take a look at `test/conftest.py` you'll notice that the dependencies to be injected 159 | have been defined. 160 | 161 | All the magic happens in `test/test_token.py` in the following lines of code: 162 | ``` 163 | @pytest.fixture('module') 164 | def token(): 165 | return __name__ + ':token' 166 | ``` 167 | 168 | The name of the function can be anything. What's important to note is the use of `__name__ + ':token:`. Since state changes 169 | are persisted into a file, using `__name__` enables creation of unique files in case multiple test are being run in parallel. 170 | 171 | The `request` dependency is provided by pytest and is used to store and retrieve persisted data 172 | 173 | I find state persistence very useful in situations why I need to share data between tests. Yes, yes. I know. Test are supposed 174 | to be unique and independent. But sometimes it's necessary. For instance, when a token is required to access an api, like in 175 | the example above or, if you want to maintain data from a response for use in future requests. 176 | 177 | 178 | ![alt text](https://orig00.deviantart.net/6ef3/f/2015/092/e/7/that_s_all_folks_by_shootersp-d8o2m64.gif "That's all folks!") -------------------------------------------------------------------------------- /content/selenium-webscraping-tutorial.md: -------------------------------------------------------------------------------- 1 | Title: Introduction to Web Scraping using Selenium 2 | Date: 2017-09-05 10:40:00 3 | Tags: python, webscraping, selenium, chromedriver 4 | Category: Relevant_Category 5 | Slug: selenium-webscraping-tutorial 6 | Author: Roger Taracha 7 | Summary: In this tutorial you’ll learn how to scrape websites with Selenium and ChromeDriver. 8 | about_author: Taracha is a Software Developer & Mentor based in Nairobi. For more catch me on [Medium](https://medium.com/@NabweraTaracha) and on Twitter @thedancercodes. 9 | email: rojtaracha@gmail.com 10 | 11 | 12 | ### Title 13 | Introduction to Web Scraping using Selenium 14 | 15 | This article first appeared on [Medium.com](https://medium.com/@NabweraTaracha/introduction-to-web-scraping-using-selenium-7ec377a8cf72) 16 | 17 | ### What is Web Scraping? 18 | As the name suggests, this is a data extraction technique used for extracting data from websites. It is an automated process where an application processes the HTML of a Web Page to extract data for manipulation such as converting the Web page to another format and copying into a local database or spreadsheet for later retrieval or analysis. 19 | 20 | ### What will we build? 21 | In this tutorial we will build a web scraping program that will scrape a Github user profile and get the Repository Names and the Languages for the Pinned Repositories. 22 | If you would like to jump straight into the project, here is link to the repo on Github. 23 | https://github.com/TheDancerCodes/Selenium-Webscraping-Example 24 | 25 | ### What will we require? 26 | We will also use the following packages and driver. 27 | * [selenium package](https://pypi.python.org/pypi/selenium) — used to automate web browser interaction from Python 28 | * [ChromeDriver](https://sites.google.com/a/chromium.org/chromedriver/downloads) — provides a platform to open up and perform tasks in specified browser. 29 | * [Virtualenv](https://virtualenv.pypa.io/en/stable/) — to create an isolated Python environment for our project. 30 | * [Extras](http://selenium-python.readthedocs.io/) - Selenium-Python ReadTheDocs Resource. 31 | 32 | ### Steps 33 | 1. Setup project 34 | 2. Import Modules 35 | 3. Make The Request 36 | 4. Get the Response 37 | 5. Run the program 38 | 39 | 40 | ### Set up the project 41 | Create a new project folder. Within that folder create an `setup.py` file. 42 | In this file, type in our dependency `selenium`. 43 | 44 | Open up your command line & create a virtual environment using the basic command: 45 | 46 | `$ virtualenv webscraping_example` 47 | 48 | Next, install the dependency into your virtualenv by running the following command in the terminal: 49 | 50 | `$(webscraping_example) pip install -r setup.py` 51 | 52 | 53 | ### Import Required Modules 54 | Specify the modules required for the project. 55 | 56 | 57 | ```python 58 | from selenium import webdriver 59 | from selenium.webdriver.common.by import By 60 | from selenium.webdriver.support.ui import WebDriverWait 61 | from selenium.webdriver.support import expected_conditions as EC 62 | from selenium.common.exceptions import TimeoutException 63 | 64 | ``` 65 | 66 | 67 | ### Make The Request 68 | When making the request we need to consider the following: 69 | 1. Pass in the desired website url. 70 | 2. Implement a Try and Except for handling a timeout situation should it occur. 71 | 72 | 73 | 74 | ```python 75 | # Specifying incognito mode as you launch your browser[OPTIONAL] 76 | option = webdriver.ChromeOptions() 77 | option.add_argument(“ — incognito”) 78 | 79 | # Create new Instance of Chrome in incognito mode 80 | browser = webdriver.Chrome(executable_path='/Library/Application Support/Google/chromedriver', chrome_options=option) 81 | 82 | # Go to desired website 83 | browser.get("https://github.com/TheDancerCodes") 84 | 85 | # Wait 20 seconds for page to load 86 | timeout = 20 87 | 88 | try: 89 | # Wait until the final element [Avatar link] is loaded. 90 | # Assumption: If Avatar link is loaded, the whole page would be relatively loaded because it is among 91 | # the last things to be loaded. 92 | WebDriverWait(browser, timeout).until(EC.visibility_of_element_located((By.XPATH, "//img[@class='avatar width-full rounded-2']"))) 93 | 94 | except TimeoutException: 95 | print("Timed out waiting for page to load") 96 | browser.quit() 97 | 98 | ``` 99 | 100 | 101 | ### Get the Response 102 | Once we make a request and it is successful, we need to get a response. 103 | We will break the response into 2 and combine it at the end. 104 | The response is the title and language of the pinned repositories of the Github profile. 105 | 106 | 107 | ```python 108 | # find_elements_by_xpath - Returns an array of selenium objects. 109 | titles_element = browser.find_elements_by_xpath("//a[@class='text-bold']") 110 | 111 | # List Comprehension to get the actual repo titles and not the selenium objects. 112 | titles = [x.text for x in titles_element] 113 | 114 | 115 | print('TITLES:') 116 | print(titles, '\n') 117 | 118 | 119 | # Get all of the pinned repo languages 120 | language_element = browser.find_elements_by_xpath("//p[@class='mb-0 f6 text-gray']") 121 | languages = [x.text for x in language_element] # same concept as for-loop/ list-comprehension above. 122 | 123 | print("LANGUAGES:") 124 | print(languages, '\n') 125 | 126 | # Pair each title with its corresponding language using zip function and print each pair 127 | for title, language in zip(titles, languages): 128 | print("RepoName : Language") 129 | print(title + ": " + language, '\n') 130 | 131 | ``` 132 | 133 | 134 | ### Run the program 135 | Finally execute the program by running it directly in your IDE or by using the following command: 136 | 137 | 138 | ```python 139 | $ (webscraping_example) python webscraping_example.py 140 | 141 | ``` 142 | 143 | 144 | You can read more on Web Scraping here 145 | * [Wikipedia Page](https://en.wikipedia.org/wiki/Web_scraping). 146 | 147 | * If you would like to try out this example, [here's the link to the source code on github](https://github.com/TheDancerCodes/Selenium-Webscraping-Example). `Fork`, `Clone`, `Install requirements` and `Scrape`. 148 | -------------------------------------------------------------------------------- /content/scrapy.md: -------------------------------------------------------------------------------- 1 | Title: Scraping The Python Nairobi Blog 2 | Date: 2017-09-17 00:00 3 | Tags: scrapy, web, spiders 4 | Category: Web Scraping 5 | Slug: scraping-the-python-nairobi-blog 6 | Author: Gideon Kimutai (https://github.com/gr1d99) 7 | Summary: This post will show you how to scrape the Python Nairobi blog using scrapy framework. 8 | about_author: A freelance Django web developer 9 | email: [gravar](gideonkimutai9@gmail.com) 10 | 11 | Scrapy is an application framework for crawling web sites and extracting structured data which can be used for a wide 12 | range of useful applications, like data mining, information processing or historical archival. 13 | 14 | 15 | so lets get started. 16 | 17 | Requirements 18 | ------------ 19 | 1. you need to have python installed on your computer. here is the link to the official site [python.org](https://www.python.org/downloads/) 20 | 2. create a virtual enviroment, steps can be found here [click me](https://virtualenv.pypa.io/en/stable/) 21 | 3. activate your created virtualenvironment and install scrapy framework by running the command 22 | pip install scrapy. 23 | 24 | **NB** if everything is installed correctly you are good to start scraping some data. 25 | 26 | **In this tutorial we will be scraping the Python Nairobi Blog, you may apply the concepts in this tutorial to scrape 27 | other websites too. You will also have to read further on your own since i will only cover the basics** 28 | 29 | 30 | check again if `scrapy` is available in your current environment by typing `scrapy` in your terminal. If you see some 31 | bunch of help options then you are good to go. 32 | 33 | we are going to create our project. to do that type 34 | 35 | :::bash 36 | $ scrapy createproject naiblog 37 | 38 | after creating the project minimize your terminal and you should see a directory named `naiblog`. 39 | 40 | the directory structure of naiblob should look like this:- 41 | 42 | 43 | :::bash 44 | naiblog/ 45 | scrapy.cfg 46 | naiblog/ 47 | __init__.py # deploy configuration file project's Python module, you'll import your code from here 48 | items.py # project items definition file 49 | pipelines.py # project pipelines file 50 | settings.py # project settings file 51 | spiders/ 52 | __init__.py # a directory where you'll later put your spiders 53 | 54 | create a file named `naiblog_spider.py` inside the folder `naiblog/spiders/naiblog_spider.py` 55 | 56 | now copy the contents below to the new file. 57 | 58 | :::python 59 | import scrapy 60 | 61 | 62 | 63 | class NailblogSpider(scrapy.Spider): 64 | name = 'naiblog' 65 | 66 | start_urls = [ 67 | 'http://blog.pynbo.or.ke/' 68 | ] 69 | 70 | 71 | def parse(self, response): 72 | pass 73 | 74 | 75 | you need to note some few things here. 76 | 77 | * our spider subcalsses `scrapy.Spider`. 78 | * `name=naiblog` this is the name of our spider, it should be unique within this project. 79 | * `start_urls[]` is a class attribute and this where you write a list of urls that you will crawl. **this is just one of 80 | the ways**. 81 | * `parse()` is a method that will be called to handle response download from each request. 82 | 83 | now you atleast have the basics, so change the contents of the `parse()` method to be the same as the one below. 84 | 85 | :::python 86 | class NaiblogSpider(spider.Spider): 87 | ... 88 | 89 | def parse(self, response): 90 | sub_header = response.xpath("//div//h1[@class='content-subhead']/text()").extract_first() 91 | print sub_header 92 | 93 | 94 | on your terminal navigate to the root/base directory of the `naiblog` project and type 95 | 96 | :::bash 97 | $ scrapy crawl naiblog --nolog 98 | 99 | output 100 | 101 | :::bash 102 | $ Latest posts 103 | 104 | 105 | **yeey you just extracted the content subheading of the [http://blog.pynbo.or.ke/](http://blog.pynbo.or.ke/)** 106 | 107 | just open the above url in your browser and right click on the heading `Latest Posts` then select the option inspect. 108 | you will see that the `Post List` is wrapped inside 109 | 110 | :::html 111 |
113 | Latest posts 114 | 115 | ... 116 | 117 | 118 | **NB:** when extracting contents from a response we use `css` or `xpath` Selectors to select elements from the downloaded 119 | response. 120 | 121 | I find `xpath` more powerful rather than using `css`, you will also have to read more on yourself to understand `Xpath` 122 | expressions. 123 | 124 | now to the final part of our spider, comments on the code will help you get the idea of what is happening 125 | 126 | 127 | import scrapy 128 | 129 | 130 | class NaiblogSpider(scrapy.Spider): 131 | name = 'naiblog' 132 | 133 | start_urls = [ 134 | 'http://blog.pynbo.or.ke/' 135 | ] 136 | 137 | def parse(self, response): 138 | # lets start by extracting the root node /html 139 | # from the response. 140 | html = response.xpath("/html") 141 | 142 | # select the body element which is inside 143 | # the root node html 144 | body = html.xpath("//body") 145 | 146 | # lets now get the content element 147 | content = body.xpath("//div[@class='content']") 148 | 149 | # extract the posts element 150 | posts = content.xpath("//div[@class='posts']") 151 | 152 | # iterate over the posts element in order to get 153 | # each individual post by extracting section node 154 | 155 | for post in posts.xpath("section[@class='post']"): 156 | 157 | # get the node header 158 | header = post.xpath("header[@class='post-header']") 159 | # finally get the title text 160 | title = header.xpath("h3 /a/text()").extract_first() 161 | 162 | # to get the description is the hardest part, if you inspect the element 163 | # of the page you will notice that there are two

element without any attribute between them which holds a simple description 165 | # of the post. Basically it makes it hard to extract the description of the post. 166 | # I suggest the developers of the web app should look into it. 167 | # with that issue in mind I did some tweak below in order to get the empty

168 | # tag. 169 | description = header.xpath("p[@class='post-meta']|p/text()").extract()[1] 170 | 171 | # extract post category 172 | category = header.xpath("p[@class='post-meta'] /a/text()").extract_first() 173 | 174 | # extract post date 175 | date = header.xpath("p[@class='post-meta'] /text()")[-1].extract() 176 | 177 | # finally lets return some data 178 | yield { 179 | 'description': description, 'category': category, 'title': title, 'date': date 180 | } 181 | 182 | lets execute our spider 183 | 184 | :::bash 185 | $ scrapy crawl naiblog -o posts.json 186 | 187 | 188 | you will see lots of logs being displayed on your terminal, when there are no more logs displayed navigate to the root of 189 | `naiblog` project and you will see a file named `posts.json`, open it and you will see all the posts in `pynbo blog`. 190 | 191 | Thats all!! 192 | 193 | find the entire project on my git repo [naiblog](https://github.com/gr1d99/naiblog-spider.git). 194 | -------------------------------------------------------------------------------- /content/tracking-vehicles-with-accesskenya-camera.md: -------------------------------------------------------------------------------- 1 | Title: Tracking Vehicles on Access Kenya Cameras 2 | Date: 2016-01-09 18:11:00 3 | Tags: python, image analysis, opencv, access kenya, traffic 4 | Slug: image_analysis_by_chris_orwa 5 | Author: Chris Orwa 6 | Summary: Description of a python API I built to track vehicular movement on Access Kenya cameras 7 | email: chrisorwa@gmail.com 8 | about_author:

Chris Orwa is a Data Scientist currently residing in Nairobi, Kenya. He's main focus is on computational techniques for unstructured data. To that end, he has learned to code in Python and R, and retained both languages as the core-analysis software/language with an occasional dash of C and Java.

His crazy analysis monologues can be found at http://blackorwa.com

9 | 10 | Hello everyone, here's my blog post on how I built an API to return traffic conditions from Access Kenya cameras. The API would work by specifiying a road name at the URL endpoint and a json reponse would have number of cars that moved and the speed of movement. 11 | 12 | #### Data Capture 13 | To begin the process I decided to test if is possible to capture Image from each camera. After exploration of the website I realized each camera had a url with a JPEG file extension at the end. I figured the cameras wrote a new image to the JPEG file on the URL. So, can I captured every new image? Yes - I used the **urllib** library to capture the image and store it on the disk. 14 | 15 | I organized the camera urls into a dict so as to have a means of calling and referencing each road. 16 | 17 | :::python 18 | cameras = dict ( 19 | museum='http://traffic.accesskenya.com/images/traffic/feeds/purshotam.jpg', 20 | ojijo='http://traffic.accesskenya.com/images/traffic/feeds/mhcojijo.jpg', 21 | forest_limuru='http://traffic.accesskenya.com/images/traffic/feeds/forestlimuru.jpg?', 22 | kenyatta_uhuru_valley='http://traffic.accesskenya.com/images/traffic/feeds/barclaysplaza.jpg',) 23 | 24 | After that, I wrote a function that takes a url and extracts three images every 6 seconds. 25 | 26 | :::python 27 | def capture_images(self): 28 | for i in 'abc': 29 | if self.name in way: 30 | urllib.urlretrieve(way[self.name],'img_'+i+'.jpg') 31 | time.sleep(6) 32 | 33 | #### Image Processing 34 | The images captured from the camera are stored on a folder with the name of the road. Next task involves processing the images for analysis. I utlize two libraries for this task; **PIL** for loading the images and **numpy** to convert pixel values to numerical array values. All these are wrapped in a function (shown below) that takes the image folder directory as input then proceeds to load all 3 images, converts them to numpy arrays, deletes the images and returns a python dict holding arrays on all the images. 35 | 36 | :::python 37 | # load images 38 | def load(self): 39 | files = os.listdir(self.path) 40 | a = dict() 41 | b = dict() 42 | k = 0 43 | 44 | while k <= len(files): 45 | for names in files: 46 | if names != '.DS_Store': 47 | a[names] = Image.open(names).convert('L') 48 | a[names].load() 49 | b[names] = np.asarray(a[names]) 50 | k +=1 51 | 52 | # delete image folder 53 | shutil.rmtree(os.getcwd()) 54 | 55 | return b 56 | 57 | #### Motion Detection 58 | The first important step in analysis of the images is checking if there has been movements within the 6 second period. To achieve this, I utilized the concept of differential imaging - a means of measuring motion detection by subtracting the pixel values of subsquent images. In my function, I calculate the number of pixels that have moved, this helps in quantifying the movement (standstill, moderate traffic). 59 | 60 | :::python 61 | # differential imaging 62 | def diffImg(self,img1,img2,img3): 63 | 64 | # calculate absolute difference 65 | d1 = cv2.absdiff(img1,img2) 66 | d2 = cv2.absdiff(img2,img3) 67 | bit = cv2.bitwise_and(d1,d2) 68 | ret,thresh = cv2.threshold(bit,35,255,cv2.THRESH_BINARY) 69 | 70 | #get number of different pixels 71 | moving = list() 72 | for cell in thresh.flat: 73 | if cell == 255: 74 | move = 'True' 75 | moving.append(move) 76 | pixie = len(moving) 77 | 78 | return pixie 79 | 80 | #### Calibrating Movement 81 | Once movement is detected, it is important to then quantify trafficc in km/h. To aid in this calculation is the optical flow algorithm. A concept in computer vision that allow tracking features in an image. I utilized this functionality to find features to track (cars) in the first images, and get their corresponding positions in the second and third image. I then proceeded to calculate the avaerage distance (euclidean distance) that the feature has moved. Dividing the pixel distance by 12 seconds gives me speed at which the objects(cars) are moving. My function returns this value. 82 | 83 | :::python 84 | # calculate optical flow of points on images 85 | def opticalFlow(self,img1,img2,img3): 86 | 87 | #set variables 88 | lk_params = dict(winSize = (10,10), 89 | maxLevel = 5, 90 | criteria = (cv2.TERM_CRITERIA_EPS | cv2.TERM_CRITERIA_COUNT,10,0.03)) 91 | 92 | features_param = dict( maxCorners = 3000, 93 | qualityLevel = 0.5, 94 | minDistance = 3, 95 | blockSize = 3) 96 | 97 | # feature extraction of points to track 98 | pt = cv2.goodFeaturesToTrack(img1,**features_param) 99 | p0 =np.float32(pt).reshape(-1,1,2) 100 | 101 | # calaculate average movement 102 | dist = list() 103 | for loop in p0: 104 | p1,st,err =cv2.calcOpticalFlowPyrLK(img1, img2,loop, 105 | None,**lk_params) 106 | 107 | p0r,st,err =cv2.calcOpticalFlowPyrLK(img2,img1,p1, 108 | None,**lk_params) 109 | 110 | if abs(loop-p0r).reshape(-1, 2).max(-1) < 1: 111 | dst = distance.euclidean(loop,p0r) 112 | dist.append(dst) 113 | 114 | return round(max(dist)*10,2) 115 | 116 | #### The API 117 | The API (underconstruction) is based on flask. By specifying a road name at the HTTP endpoint, the API returns speed of traffic and level of movement (stanstill, moderate rate, no traffic). 118 | 119 | :::python 120 | # load required libraries 121 | import image_processing 122 | import numpy as np 123 | from flask import Flask 124 | import links 125 | import json 126 | import scipy as sp 127 | 128 | # create flask web server 129 | app = Flask(__name__) 130 | 131 | # create HTTP endpoint 132 | @app.route('/ImPro/') 133 | 134 | # main function 135 | def get_route(road): 136 | # initialize route class 137 | road = 'sarit' 138 | traffic = image_processing.route(road) 139 | 140 | # setup working directory 141 | traffic.set_dir() 142 | 143 | # get image from traffic camera 144 | traffic.capture_images() 145 | 146 | # load image stack 147 | x = traffic.load() 148 | 149 | # differential imaging 150 | y = traffic.diffImg(x['img_a.jpg'],x['img_b.jpg'],x['img_c.jpg']) 151 | 152 | # calculate optical flow 153 | z = traffic.opticalFlow(x['img_a.jpg'],x['img_b.jpg'],x['img_c.jpg']) 154 | 155 | if __name__ == "__main__": 156 | app.run() 157 | 158 | The code is available at . Feel free to modify and improve as you see fit. 159 | I hope you enjoyed the article. Feel free to drop me an e-mail at chrisorwa@gmail.com for any queries. 160 | -------------------------------------------------------------------------------- /content/decorators-in-python-mbithe-nzomo.md: -------------------------------------------------------------------------------- 1 | --- 2 | Title: Lessons in DRYness - Decorators in Python 3 | Date: 2017-08-30 09:00 4 | Tags: decorators, dry code, python 5 | Category: misc 6 | Slug: decorators_in_python 7 | Author: Mbithe Nzomo 8 | Summary: What decorators are and how to use them for DRYer code 9 | about_author: Mbithe is a software developer at Andela, and a co-organiser and coach at Django Girls Nairobi. She blogs at deCODEgirl.com and tweets at @mbithenzomo. 10 | email: mbithe.nzomo@gmail.com 11 | --- 12 | *This article first appeared 13 | on deCODEgirl.com.* 14 | 15 | Troll Face 16 | 17 | What do clothes and code (and my jokes) have in common? Answer: They're so 18 | much better when they're dry. Troll Face 19 | 20 | What does it mean for code to be **DRY**? Simple: **Don't Repeat Yourself**. 21 | The DRY Principle is key when writing efficient code. It helps keep your 22 | codebase smaller and less complex, saves you time, reduces redundancy... it's 23 | basically sugar, spice, and everything nice. :) In this post, I'm going to demonstrate how decorators in Python can help keep 24 | your code DRY. 25 | 26 | Once upon a time, I was building a RESTful API with Flask (which 28 | you can check it out on GitHub 30 | here) for an online bucket list service. Users could: 31 | 32 | 1. Sign up and login 33 | 2. Create, edit, view, and delete bucket lists 34 | 3. Create, edit, view, and delete bucket list items 35 | 36 | Some endpoints were protected and could only be accessed by authorized users. 37 | For instance, only the user who created a bucket list or bucket list item could 38 | edit, view, or delete it. 39 | 40 | Here's how I initially tackled it. First, I created a method to display an 41 | error message, with the default being "Error: You are not authorized to access 42 | this resource." 43 | 44 | ``` python 45 | def unauthorized(message=None): 46 | """ 47 | Returns an error message. 48 | """ 49 | if not message: 50 | message = "Error: You are not authorized to access this resource." 51 | return jsonify({ 52 | "message": message 53 | }), 403 54 | ``` 55 | 56 | Then, I went ahead and wrote the methods for the endpoints: 57 | 58 | ``` python 59 | class BucketListAPI(Resource): 60 | """ 61 | URL: /api/v1/bucketlists/ 62 | Request methods: GET, PUT, DELETE 63 | """ 64 | 65 | def get(self, id): 66 | """ Get a bucket list """ 67 | bucketlist = Bucketlist.query.filter_by(id=id).first() 68 | if bucketlist: 69 | if bucketlist.created_by == g.user.id: 70 | return marshal(bucketlist, bucketlist_serializer) 71 | else: 72 | return unauthorized() 73 | else: 74 | return unauthorized("Error: The bucket list specified doesn't " 75 | "exist. Please try again!") 76 | 77 | def put(self, id): 78 | """ Edit a bucket list """ 79 | bucketlist = Bucketlist.query.filter_by(id=id).first() 80 | if bucketlist: 81 | if bucketlist.created_by == g.user.id: 82 | parser = reqparse.RequestParser() 83 | parser.add_argument("title", 84 | required=True, 85 | help="No title provided.") 86 | parser.add_argument("description", type=str, default="") 87 | args = parser.parse_args() 88 | title, description = args["title"], args["description"] 89 | bucketlist.title = title 90 | bucketlist.description = description 91 | return edit_item(name="title", 92 | item=bucketlist, 93 | serializer=bucketlist_serializer, 94 | is_user=False, 95 | is_bucketlist=True, 96 | is_item=False) 97 | else: 98 | return unauthorized() 99 | else: 100 | return unauthorized("Error: The bucket list you are trying to " 101 | "edit doesn't exist. Please try again!") 102 | 103 | def delete(self, id): 104 | """ Delete a bucket list """ 105 | bucketlist = Bucketlist.query.filter_by(id=id).first() 106 | if bucketlist: 107 | if bucketlist.created_by == g.user.id: 108 | return delete_item(bucketlist, 109 | bucketlist.title, 110 | is_user=False, 111 | is_bucketlist=True, 112 | is_item=False) 113 | else: 114 | return unauthorized() 115 | else: 116 | return unauthorized("Error: The bucket list you are trying to " 117 | "delete doesn't exist. Please try again!") 118 | ``` 119 | 120 | You can see the problem already: in each of these methods, I basically repeated 121 | the same chunk of code to ensure that: 122 | 123 | 1. The bucket list existed 124 | 2. The current user was authorized to access the endpoint 125 | 3. Appropriate error messages were displayed if the above conditions were false 126 | 127 | ``` python 128 | # Check whether bucket list exists 129 | if bucketlist: 130 | # Check whether bucket list was created by the current user 131 | if bucketlist.created_by == g.user.id: 132 | # Do something here 133 | else: 134 | # Return unauthorized error message 135 | return unauthorized() 136 | else: 137 | # Return non-existent bucket list error message 138 | return unauthorized("Error: The bucket list you are trying to " 139 | "delete doesn't exist. Please try again!") 140 | ``` 141 | 142 | Not DRY at all. Fret not, though: decorators to the rescue! 143 | 144 | A decorator is basically a function that takes a callable (a function or a 145 | class) as a parameter, modifies it, and returns it. You could also say that decorators wrap a callable, modifying its behavior. 146 | 147 | Here's how I used a decorator to eliminate my duplicate code. First, I wrote 148 | the decorator function: 149 | 150 | ``` python 151 | def authorized_user_bucketlist(function): 152 | def auth_wrapper(*args, **kwargs): 153 | g.bucketlist = Bucketlist.query.filter_by(id=kwargs["id"]).first() 154 | try: 155 | if g.bucketlist.created_by == g.user.id: 156 | return function(*args, **kwargs) 157 | return unauthorized() 158 | except: 159 | return unauthorized("Error: The bucket list specified doesn't " 160 | "exist. Please try again!") 161 | return auth_wrapper 162 | ``` 163 | 164 | So here's what's going on in there: The decorator is 165 | `authorized_user_bucketlist`, and it takes some function as a parameter. It has 166 | another function, `auth_wrapper`, where a variable `g.bucketlist` is defined as 167 | the bucket list whose ID is passed to the function as a key word argument. It 168 | then checks if the bucket list was created by the current user. If so, it 169 | invokes the function and returns the result. If not, it returns an unauthorized error message. If the bucket list doesn't exist, it returns a non-existent 170 | bucket list error message. Basically, it does the same thing my duplicated 171 | chunk of code was doing, but with a little more finesse (like, using a 172 | try-except block as opposed to an if-else block, because it's better to ask for forgiveness than to ask for permission). 173 | 174 | The refactored code: 175 | 176 | ``` python 177 | from base import authorized_user_bucketlist 178 | 179 | class BucketListAPI(Resource): 180 | """ 181 | URL: /api/v1/bucketlists/ 182 | Request methods: GET, PUT, DELETE 183 | """ 184 | @authorized_user_bucketlist 185 | def get(self, id): 186 | """ Get a bucket list """ 187 | return marshal(g.bucketlist, bucketlist_serializer) 188 | 189 | @authorized_user_bucketlist 190 | def put(self, id): 191 | """ Edit a bucket list """ 192 | parser = reqparse.RequestParser() 193 | parser.add_argument("title", 194 | required=True, 195 | help="No title provided.") 196 | parser.add_argument("description", type=str, default="") 197 | args = parser.parse_args() 198 | title, description = args["title"], args["description"] 199 | g.bucketlist.title = title 200 | g.bucketlist.description = description 201 | return edit_item(name="title", 202 | item=g.bucketlist, 203 | serializer=bucketlist_serializer, 204 | is_user=False, 205 | is_bucketlist=True, 206 | is_item=False) 207 | 208 | @authorized_user_bucketlist 209 | def delete(self, id): 210 | """ Delete a bucket list """ 211 | return delete_item(g.bucketlist, 212 | g.bucketlist.title, 213 | is_user=False, 214 | is_bucketlist=True, 215 | is_item=False) 216 | ``` 217 | 218 | I imported the decorator function, and then called the decorator using the `@` 219 | symbol before each method. This is some 221 | syntactic sugar that Python provides, 222 | making the calling of decorators much simpler than it otherwise would be. 223 | 224 | The code is much DRYer now! Mission accomplished! It's not perfect, but it's 225 | way better than it was. :) This project is several months old now, and no longer 226 | being maintained, but feel free to clone the repo and test my API! 229 | 230 | Also, you can learn more about decorators in Python here and here. 234 | -------------------------------------------------------------------------------- /content/parametrizedUnitTestingWithPyTest.md: -------------------------------------------------------------------------------- 1 | --- 2 | Title: Python parametrized unit testing with Pytest 3 | Date: 2017-09-04 19:00 4 | Tags: python, pytest 5 | Category: unit testing 6 | Slug: test_parameterization 7 | Author: Paul Upendo 8 | Summary: Create pythonic parametrized tests with pytest 9 | about_author: Upendo is a Software Developer at Andela Kenya 10 | email: lovepaul46@outlook.com 11 | --- 12 | 13 | ## Pre-requisites 14 | > - Beginner level Python 15 | > - Python2* or Python3* installed 16 | > - Basic knowledge on TDD and python UnitTesting 17 | > - Pytest installed 18 | 19 | # Intro 20 | I really did not get a hang of TDD the first time I heard about it. I saw it as an acronym that was maybe too difficult to fathom. A few months down the line I have a clear and better perspective towards it, and even better a more pythonic way to use TDD in my software development. Hang on as we methodically dive into it. 21 | 22 | TDD (Test Driven Development) is a software design methodology. This generally means writing UnitTests for your code even before you write it yourself. TDD, now that you are farmiliar with the acronym, is not limited to a particular language stack but for this instance we will stick to python as our language of choice. 23 | 24 | # Parametrizing your tests with Pytest 25 | There are a couple of UnitTesting frameworks in python: 26 | - Python Unittest (that comes with the standard python library distribution) 27 | - Python Pytest 28 | - Python-nose 29 | 30 | Python UnitTest framework is a member of the XUnit family of testing tools that use a similar pattern for defining testCases. With UnitTest following XUnit Conventions, it makes Pytest and Python-nose a more - Pythonic way to write tests that follow Pep8 style guide. I will focus on paramet`rization of tests using Pytest. 31 | 32 | #### Why parametrize my tests? 33 | Let's think of a scenario where we have to write tests for a particular function but with multiple input values/arguments. Rewriting my assertion statements with the different arguments works quite well but what if we could simplify this to only one assertion statement for that function? Then run the test multiple times with an array of parameters? Smart right? Python's Pytest provides an excellent Pythonic way to achieve this. 34 | 35 | `@pytest.mark.parametrize` allows one to define multiple sets of arguments and fixtures at the test function or class. 36 | 37 | Let us write parametrized tests for a function that accepts integer input from the user and determines whether the input is odd or even. 38 | ```python 39 | # File test_parametrization.py 40 | import pytest 41 | 42 | 43 | @pytest.mark.parametrize("test_input,expected", [ 44 | (3, "odd"), 45 | (4, "even"), 46 | (9, "even"), 47 | ]) 48 | 49 | def test_even_and_odd_numbers(test_input, expected): 50 | assert (runOddEvenCheck(test_input)) == expected 51 | 52 | ``` 53 | To run this, use the command `pytest`, which executes the pytest test runner for us. The result for this would be a series of failing tests as we can see below. The beautiful thing about this is that by implementing test parameterization, Pytest created 3 individual tests for us from our test function :smile: . 54 | ```python 55 | # Commandline output 56 | (pytest_blog) ~/D/t/p/parameterized_testing ❯❯❯ pytest ⏎ develop ✱ ◼ 57 | =================================================================================================================================== test session starts =================================================================================================================================== 58 | platform darwin -- Python 3.6.1, pytest-3.2.1, py-1.4.34, pluggy-0.4.0 59 | rootdir: /Users/Upendo/Documents/tech_blogs/pytest_blog/parameterized_testing, inifile: 60 | collected 3 items 61 | 62 | test_parametrization.py FFF 63 | 64 | ======================================================================================================================================== FAILURES ========================================================================================================================================= 65 | ____________________________________________________________________________________________________________________________ test_even_and_odd_numbers[3-odd] _____________________________________________________________________________________________________________________________ 66 | 67 | test_input = 3, expected = 'odd' 68 | 69 | @pytest.mark.parametrize("test_input,expected", [ 70 | (3, "odd"), 71 | (4, "even"), 72 | (9, "even"), 73 | ]) 74 | 75 | def test_even_and_odd_numbers(test_input, expected): 76 | > assert (runOddEvenCheck(test_input)) == expected 77 | E NameError: name 'runOddEvenCheck' is not defined 78 | 79 | test_parametrization.py:11: NameError 80 | ____________________________________________________________________________________________________________________________ test_even_and_odd_numbers[4-even] ____________________________________________________________________________________________________________________________ 81 | 82 | test_input = 4, expected = 'even' 83 | 84 | @pytest.mark.parametrize("test_input,expected", [ 85 | (3, "odd"), 86 | (4, "even"), 87 | (9, "even"), 88 | ]) 89 | 90 | def test_even_and_odd_numbers(test_input, expected): 91 | > assert (runOddEvenCheck(test_input)) == expected 92 | E NameError: name 'runOddEvenCheck' is not defined 93 | 94 | test_parametrization.py:11: NameError 95 | ____________________________________________________________________________________________________________________________ test_even_and_odd_numbers[9-even] ____________________________________________________________________________________________________________________________ 96 | 97 | test_input = 9, expected = 'even' 98 | 99 | @pytest.mark.parametrize("test_input,expected", [ 100 | (3, "odd"), 101 | (4, "even"), 102 | (9, "even"), 103 | ]) 104 | 105 | def test_even_and_odd_numbers(test_input, expected): 106 | > assert (runOddEvenCheck(test_input)) == expected 107 | E NameError: name 'runOddEvenCheck' is not defined 108 | 109 | test_parametrization.py:11: NameError 110 | ================================================================================================================================ 3 failed in 0.07 seconds ================================================================================================================================= 111 | (pytest_blog) ~/D/t/p/parameterized_testing ❯❯❯ 112 | ``` 113 | This just made the process to write our tests less time consuming, not that this will always be the case as it also depends on the complexity or requirements of the project/code we plan to write after the tests for. 114 | Let us get back to fixing our failing tests as per the output we have received form Pytest test runner :bowtie: 115 | All our asserts fail as a NameError exception is raised. So let's fix that by creating a function `runOddEvenCheck` to check for number state whether odd or even when user input is provided. Then import the function for our tests to use. 116 | ``` python 117 | # File numberWork.py 118 | def runOddEvenCheck(nums): 119 | return "even" if nums % 2 == 0 else "odd" 120 | 121 | ``` 122 | Then let us import this function to our test file: 123 | ``` python 124 | import pytest 125 | from numberWork import runOddEvenCheck 126 | 127 | 128 | @pytest.mark.parametrize("test_input,expected", [ 129 | (3, "odd"), 130 | (4, "even"), 131 | (9, "even"), 132 | ]) 133 | def test_even_and_odd_numbers(test_input, expected): 134 | assert (runOddEvenCheck(test_input)) == expected 135 | 136 | ``` 137 | And we can finally run our tests with `pytest`: 138 | ``` python 139 | (pytest_blog) ~/D/t/p/parameterized_testing ❯❯❯ pytest ⏎ develop ✱ ◼ 140 | =================================================================================================================================== test session starts =================================================================================================================================== 141 | platform darwin -- Python 3.6.1, pytest-3.2.1, py-1.4.34, pluggy-0.4.0 142 | rootdir: /Users/Upendo/Documents/tech_blogs/pytest_blog/parameterized_testing, inifile: 143 | collected 3 items 144 | 145 | test_parametrization.py ..F 146 | 147 | ======================================================================================================================================== FAILURES ========================================================================================================================================= 148 | ____________________________________________________________________________________________________________________________ test_even_and_odd_numbers[9-even] ____________________________________________________________________________________________________________________________ 149 | 150 | test_input = 9, expected = 'even' 151 | 152 | @pytest.mark.parametrize("test_input,expected", [ 153 | (3, "odd"), 154 | (4, "even"), 155 | (9, "even"), 156 | ]) 157 | def test_even_and_odd_numbers(test_input, expected): 158 | > assert (runOddEvenCheck(test_input)) == expected 159 | E AssertionError: assert 'odd' == 'even' 160 | E - odd 161 | E + even 162 | 163 | test_parametrization.py:11: AssertionError 164 | =========================================================================================================================== 1 failed, 2 passed in 0.06 seconds ============================================================================================================================ 165 | (pytest_blog) ~/D/t/p/parameterized_testing ❯❯❯ 166 | ``` 167 | Wow, we do have passing and failing tests now from our Pytest report. Two tests pass, with one failing test (Fact being that 9 is not an even number from the AssertionError message above). 168 | From these example we are now able to give parametrized testing a try I hope :smile:. The topic is quite vast, this is just but a sneak peak at the good we can do with the Pythonic Pytest. 169 | If you would like to try this example out, [here is a link to the source code on github](https://github.com/paulupendo/parametrized_testing/tree/master). `Fork`, `Clone`, `install requirements` and `pytest` :smile:. 170 | 171 | Happy coding! 172 | 173 | 174 | 175 | 176 | 177 | 178 | --------------------------------------------------------------------------------