├── bashrc_snippets ├── py_mods.sh ├── bashrc_extras.sh ├── bashrc_basics.sh ├── README.md ├── bash_aliases ├── colorify.sh └── special.sh ├── python ├── README.md ├── versions_and_environments.md ├── documentation.md ├── package_dev.md ├── kedro.md └── knowledge_dump.md ├── README.md └── git_best_practices.md /bashrc_snippets/py_mods.sh: -------------------------------------------------------------------------------- 1 | # adding my own repo to the python path: 2 | export PYTHONPATH="${PYTHONPATH}:~/../repo_name" 3 | -------------------------------------------------------------------------------- /bashrc_snippets/bashrc_extras.sh: -------------------------------------------------------------------------------- 1 | # add modifications to python path etc 2 | if [ -f ~/.py_mods ]; then 3 | . ~/.py_mods 4 | fi 5 | 6 | # add hacks specific to your device 7 | if [ -f ~/.surface_hacks ]; then 8 | . ~/.surface_hacks 9 | fi 10 | -------------------------------------------------------------------------------- /bashrc_snippets/bashrc_basics.sh: -------------------------------------------------------------------------------- 1 | # Alias definitions. 2 | # You may want to put all your additions into a separate file like 3 | # ~/.bash_aliases, instead of adding them into .bashrc directly. 4 | # See /usr/share/doc/bash-doc/examples in the bash-doc package. 5 | # add these lines to .bashrc to do so: 6 | 7 | if [ -f ~/.bash_aliases ]; then 8 | . ~/.bash_aliases 9 | fi 10 | 11 | -------------------------------------------------------------------------------- /python/README.md: -------------------------------------------------------------------------------- 1 | # Python 2 | in this folder: 3 | * [Knowledge Dump](knowledge_dump.md) Contains random tidbits of knowledge about python code and theory. 4 | * [Resources for developing an open-source package](package_dev.md) 5 | * [Versions and Environments.md](versions_and_environments.md) 6 | * [Using Kedro with Poetry](kedro.md) 7 | * [Documentation](documentation.md) 8 | 9 | 10 | -------------------------------------------------------------------------------- /python/versions_and_environments.md: -------------------------------------------------------------------------------- 1 | ## Managing multiple python versions on system 2 | Great guide on how to install and manage multiple python versions on an ubuntu based OS: https://hackersandslackers.com/multiple-versions-python-ubuntu/ 3 | 4 | 5 | # Jupyter notebook Kernels 6 | 7 | ## add conda/other environment as kernel 8 | ``conda install -c anaconda ipykernel`` 9 | 10 | ``python -m ipykernel install --user --name=env_name`` 11 | -------------------------------------------------------------------------------- /bashrc_snippets/README.md: -------------------------------------------------------------------------------- 1 | # Collection of .bashrc code snippets 2 | 3 | * [basics](bashrc_snippets/bashrc_basics.sh) This should always be in your .bashrc [NOTE: Pop_OS! will already contain this.] 4 | * [bash_aliases](bashrc_snippets/bash_aliases) Some useful aliases to include in a separate file. 5 | * [colorify](bashrc_snippets/colorify.sh) Code to make your terminal more pleasant to the eyes. 6 | * [python](bashrc_snippets/py_mods.sh) Some python specific snippets. 7 | * [special](bashrc_snippets/special.sh) contains special practices, not necessary stuff. 8 | 9 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Swiss Cheese Brain 2 | Collection of useful and silly coding tricks, tips and knowledge for people with a memory like Swiss cheese (it's only qualification being: full of holes). 3 | 4 | * [git_best_practices](git_best_practices.md) covers contributing to a public repo and some git tricks. 5 | * [bashrc basics and hacks](bashrc_snippets/README.md) links to code snippets to make you more productive in the terminal. 6 | * [Python know-how and open source project management](python/README.md) many different resources. 7 | * [Python Resources](python/README.md) and open source project management 8 | -------------------------------------------------------------------------------- /bashrc_snippets/bash_aliases: -------------------------------------------------------------------------------- 1 | # git aliases 2 | alias gits="git status" 3 | alias gitx="git checkout" 4 | alias gita="git add . ; git commit -am" 5 | alias gitm="git commit -m" 6 | 7 | # some more ls aliases 8 | alias ll='ls -alF' 9 | alias la='ls -A' 10 | alias l='ls -CF' 11 | 12 | # cloud aliases 13 | alias ssh_azure="ssh ourownstory@xx.xx.xxx.xxx" 14 | alias tb_azure="tensorboard --logdir=results --host xx.xx.xxx.xxx --port 6006" 15 | alias ssh_aws="ssh -i "~/../keyfile.pem" ec2-user@xx.xxx.xxx.xxx" 16 | alias jupyter_aws="ssh -i "~/../keyfile.pem" -L 8888:localhost:8888 ec2-user@xx.xxx.xxx.xxx" 17 | 18 | # venv aliases 19 | v(){ 20 | # use: $ v myenvname 21 | source ~/venv/$1/bin/activate 22 | } 23 | 24 | v-reset(){ 25 | # delete and recreate venv 26 | rm -r ~/venv/$1 27 | python3 -m venv ~/venv/$1 28 | source ~/venv/$1/bin/activate 29 | } 30 | 31 | -------------------------------------------------------------------------------- /python/documentation.md: -------------------------------------------------------------------------------- 1 | # Documentation 2 | * using mkdocs with github pages: https://www.mkdocs.org/user-guide/deploying-your-docs/ 3 | * using custom domain: https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site 4 | * use the Sphinx documentation framework to build the documentation website, which is hosted via Github Pages. (Details: The documentation's source is enclosed in the docs folder. Whereas the main branch only contains the basic source files, the branch gh-pages entails the build data (with folders docs/html and docs/doctrees) and is used for deployment.) 5 | * Docstrings: format according to [NumPy Style](https://sphinxcontrib-napoleon.readthedocs.io/en/latest/example_numpy.html#example-numpy) in order to display their API reference correctly using Spinx. Please refer to [Pandas Docstring Guide](https://pandas.pydata.org/pandas-docs/stable/development/contributing_docstring.html#) for best practices. The length of line inside docstrings block must be limited to 80 characters to fit into Jupyter documentation popups. 6 | * Jupyter notebooks: render using the Sphinx nblink package. (Details: When you add a new tutorial notebook, please add the tutorial file to the respective section inside docs/source/contents.rst. Further, generate the corresponding .nblink files) 7 | * Principles for documentation structure: https://diataxis.fr/ 8 | * Great Readme: https://dbader.org/blog/write-a-great-readme-for-your-github-project 9 | -------------------------------------------------------------------------------- /bashrc_snippets/colorify.sh: -------------------------------------------------------------------------------- 1 | # set a fancy prompt (non-color, unless we know we "want" color) 2 | case "$TERM" in 3 | xterm-color|*-256color) color_prompt=yes;; 4 | esac 5 | 6 | # uncomment for a colored prompt, if the terminal has the capability; turned 7 | # off by default to not distract the user: the focus in a terminal window 8 | # should be on the output of commands, not on the prompt 9 | #force_color_prompt=yes 10 | 11 | if [ -n "$force_color_prompt" ]; then 12 | if [ -x /usr/bin/tput ] && tput setaf 1 >&/dev/null; then 13 | # We have color support; assume it's compliant with Ecma-48 14 | # (ISO/IEC-6429). (Lack of such support is extremely rare, and such 15 | # a case would tend to support setf rather than setaf.) 16 | color_prompt=yes 17 | else 18 | color_prompt= 19 | fi 20 | fi 21 | 22 | if [ "$color_prompt" = yes ]; then 23 | PS1='${debian_chroot:+($debian_chroot)}\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ ' 24 | else 25 | PS1='${debian_chroot:+($debian_chroot)}\u@\h:\w\$ ' 26 | fi 27 | unset color_prompt force_color_prompt 28 | 29 | # If this is an xterm set the title to user@host:dir 30 | case "$TERM" in 31 | xterm*|rxvt*) 32 | PS1="\[\e]0;${debian_chroot:+($debian_chroot)}\u@\h: \w\a\]$PS1" 33 | ;; 34 | *) 35 | ;; 36 | esac 37 | 38 | # enable color support of ls and also add handy aliases 39 | if [ -x /usr/bin/dircolors ]; then 40 | test -r ~/.dircolors && eval "$(dircolors -b ~/.dircolors)" || eval "$(dircolors -b)" 41 | alias ls='ls --color=auto' 42 | #alias dir='dir --color=auto' 43 | #alias vdir='vdir --color=auto' 44 | 45 | alias grep='grep --color=auto' 46 | alias fgrep='fgrep --color=auto' 47 | alias egrep='egrep --color=auto' 48 | fi 49 | 50 | # colored GCC warnings and errors 51 | #export GCC_COLORS='error=01;31:warning=01;35:note=01;36:caret=01;32:locus=01:quote=01' 52 | 53 | -------------------------------------------------------------------------------- /bashrc_snippets/special.sh: -------------------------------------------------------------------------------- 1 | # ~/.bashrc: executed by bash(1) for non-login shells. 2 | # see /usr/share/doc/bash/examples/startup-files (in the package bash-doc) 3 | # for examples 4 | 5 | # If not running interactively, don't do anything 6 | case $- in 7 | *i*) ;; 8 | *) return;; 9 | esac 10 | 11 | # don't put duplicate lines or lines starting with space in the history. 12 | # See bash(1) for more options 13 | HISTCONTROL=ignoreboth 14 | 15 | # append to the history file, don't overwrite it 16 | shopt -s histappend 17 | 18 | # for setting history length see HISTSIZE and HISTFILESIZE in bash(1) 19 | HISTSIZE=1000 20 | HISTFILESIZE=2000 21 | 22 | # check the window size after each command and, if necessary, 23 | # update the values of LINES and COLUMNS. 24 | shopt -s checkwinsize 25 | 26 | # If set, the pattern "**" used in a pathname expansion context will 27 | # match all files and zero or more directories and subdirectories. 28 | #shopt -s globstar 29 | 30 | # make less more friendly for non-text input files, see lesspipe(1) 31 | [ -x /usr/bin/lesspipe ] && eval "$(SHELL=/bin/sh lesspipe)" 32 | 33 | # set variable identifying the chroot you work in (used in the prompt below) 34 | if [ -z "${debian_chroot:-}" ] && [ -r /etc/debian_chroot ]; then 35 | debian_chroot=$(cat /etc/debian_chroot) 36 | fi 37 | 38 | # Add an "alert" alias for long running commands. Use like so: 39 | # sleep 10; alert 40 | alias alert='notify-send --urgency=low -i "$([ $? = 0 ] && echo terminal || echo error)" "$(history|tail -n1|sed -e '\''s/^\s*[0-9]\+\s*//;s/[;&|]\s*alert$//'\'')"' 41 | 42 | # enable programmable completion features (you don't need to enable 43 | # this, if it's already enabled in /etc/bash.bashrc and /etc/profile 44 | # sources /etc/bash.bashrc). 45 | if ! shopt -oq posix; then 46 | if [ -f /usr/share/bash-completion/bash_completion ]; then 47 | . /usr/share/bash-completion/bash_completion 48 | elif [ -f /etc/bash_completion ]; then 49 | . /etc/bash_completion 50 | fi 51 | fi 52 | 53 | -------------------------------------------------------------------------------- /git_best_practices.md: -------------------------------------------------------------------------------- 1 | # Git Best Practices 2 | [git: ELI5](https://hackernoon.com/understanding-git-fcffd87c15a3) 3 | 4 | ## contribute to a public repo 5 | Good general guidelines and steps described: 6 | https://akrabat.com/the-beginners-guide-to-contributing-to-a-github-project/ 7 | 8 | ## git rebase, squash when working on a forked repo 9 | 10 | * When rebasing based on remote repo: (assuming you added remote forked repo as upstream `git remote add upstream git@github.com:user/repo.git`) 11 | * Do all your work, commit it and stash the rest. Push to origin `git push -u origin master` 12 | * Optional: To be safe, make a copy of the current state of your origin/master before rebasing `git checkout -b origin/copy_master origin/master` 13 | * Update upstream `git fetch upstream` 14 | * Rebase with upstream `git rebase upstream/master` 15 | * Optional: Prettify your commit history `git rebase -i HEAD~3`. Change number to the amount of commits you want to squash. You can view the history to decide how many you want to rework by looking at `git log origin/master`. 16 | * Rewrite origin's history `git push origin master --force` 17 | * initiate PR on github 18 | * Optional: If your PR got a change request, and you did your updates and are ready to redo the above steps, you can first view if the upstream master has since been updated with `git log HEAD..upstream/master`. This shows the commits in upstream that are not in your current origin. Note: regardless of changes in upstream, you can directly push to your PR branch with `git push origin master`. 19 | 20 | ## reset local branch to origin 21 | 22 | * If you want to overwrite your local branch with a remote branch `git reset --hard origin/master` 23 | 24 | ## .gitignore 25 | https://labs.consol.de/development/git/2017/02/22/gitignore.html 26 | 27 | # Git Advanced Operations 28 | ## git submodules 29 | When in need of source code files from another project. Read https://github.blog/2016-02-01-working-with-submodules/ 30 | 31 | ## split a subfolder into a new repo 32 | guide from github https://docs.github.com/en/get-started/using-git/splitting-a-subfolder-out-into-a-new-repository 33 | 34 | using filter-repo (recommended) https://github.com/newren/git-filter-repo 35 | 36 | guide guide using filter-branch (not recommended) https://manishearth.github.io/blog/2017/03/05/understanding-git-filter-branch/ 37 | 38 | -------------------------------------------------------------------------------- /python/package_dev.md: -------------------------------------------------------------------------------- 1 | # Developing Python packages 2 | Note: consider using scripts like https://github.com/github/scripts-to-rule-them-all 3 | 4 | ## `__init__.py` 5 | From: https://stackoverflow.com/questions/448271/what-is-init-py-for 6 | There are 2 main reasons for `__init__.py` 7 | 8 | 1) For convenience: the other users will not need to know your functions' exact location in your package hierarchy. 9 | ```python 10 | your_package/ 11 | __init__.py 12 | file1.py 13 | file2.py 14 | ... 15 | fileN.py 16 | ``` 17 | 18 | ```python 19 | # in __init__.py 20 | from file1 import * 21 | from file2 import * 22 | ... 23 | from fileN import * 24 | ``` 25 | ```python 26 | # in file1.py 27 | def add(): 28 | pass 29 | ``` 30 | 31 | then others can call add() by 32 | 33 | `from your_package import add` 34 | 35 | without knowing file1, like 36 | 37 | `from your_package.file1 import add` 38 | 39 | 2) If you want something to be initialized; for example, logging (which should be put in the top level): 40 | ```python 41 | import logging.config 42 | logging.config.dictConfig(Your_logging_config) 43 | ``` 44 | 45 | ## Using Setuptools aka `setup.py` 46 | https://godatadriven.com/blog/a-practical-guide-to-using-setup-py/ 47 | 48 | automatically add README.md to PyPI long description: 49 | ```python 50 | # read the contents of your README file 51 | from os import path 52 | this_directory = path.abspath(path.dirname(__file__)) 53 | with open(path.join(this_directory, 'README.md'), encoding='utf-8') as f: 54 | readme = f.read() 55 | 56 | setuptools.setup( 57 | # other arguments omitted 58 | long_description=readme, 59 | long_description_content_type='text/markdown' 60 | ) 61 | ``` 62 | 63 | ## Release a package to PYPI with setuptools 64 | Assuming you already have your code (incl. setup.py) set up. If not, read this [Real Python Guide](https://realpython.com/pypi-publish-python-package/) first. 65 | 66 | ```python 67 | pip install twine 68 | python setup.py sdist bdist_wheel 69 | tar tzf dist/packagename-1.0.0.tar.gz 70 | ``` 71 | Check if tarball contains what you expect. 72 | And then another check: 73 | ```python 74 | twine check dist/* 75 | ``` 76 | Test upload: 77 | ```python 78 | twine upload --repository-url https://test.pypi.org/legacy/ dist/* 79 | ``` 80 | Check the page, is the version number and everything else right? 81 | 82 | Create a new virtualenv and install the package from test.pypi: 83 | ```python 84 | python -m pip install --extra-index-url https://test.pypi.org/simple/ 85 | ``` 86 | Check if it works. 87 | 88 | Finally, actually upload it: 89 | ```python 90 | twine upload dist/* 91 | ``` 92 | Enjoy (and test again): 93 | 94 | ```python 95 | pip install 96 | ``` 97 | 98 | ## Release a package to PYPI with poetry 99 | One-time setup: Create an API token for PyPI and add it to poetry. Guide: 100 | https://www.digitalocean.com/community/tutorials/how-to-publish-python-packages-to-pypi-using-poetry-on-ubuntu-22-04 101 | 102 | Publishing procedure: 103 | ``` 104 | poetry build 105 | ``` 106 | ``` 107 | poetry publish 108 | ``` 109 | 110 | -------------------------------------------------------------------------------- /python/kedro.md: -------------------------------------------------------------------------------- 1 | # Using Kedro with Poetry 2 | 3 | There are different ways to do this, but as of Kedro 19.2, [kedro initializes with setuptools, not poetry](https://github.com/kedro-org/kedro/issues/1722#issuecomment-1839694525). But that is simple to fix. 4 | 5 | Assuming you want to use poetry's envs, too - we first have to set up a poetry project. 6 | All we need is a minimal `pyproject.toml`: 7 | ``` 8 | [tool.poetry] 9 | name = "projectname" 10 | version = "0.1.0" 11 | description = "" 12 | authors = ["Your Name "] 13 | readme = "README.md" 14 | 15 | [tool.poetry.dependencies] 16 | python = "^3.10" 17 | kedro = "0.19.2" 18 | 19 | [build-system] 20 | requires = ["poetry-core"] 21 | build-backend = "poetry.core.masonry.api" 22 | ``` 23 | 24 | Now we can create a poetry virtual environment (using this minimal pyproject.toml): 25 | ``` 26 | > poetry shell 27 | ``` 28 | 29 | And then install our packages, in this case just kedro: 30 | ``` 31 | > poetry update 32 | ``` 33 | 34 | Next, we can set up Kedro, which will prompt us to give the project a name and few more options: 35 | ``` 36 | > kedro new 37 | ``` 38 | 39 | Assuming we called the project `projectname`, and said no to all other options, we end up with these folders and files: 40 | ``` 41 | > tree 42 | . 43 | ├── poetry.lock 44 | ├── projectname 45 | │   ├── conf 46 | │   │   ├── base 47 | │   │   │   ├── catalog.yml 48 | │   │   │   └── parameters.yml 49 | │   │   ├── local 50 | │   │   │   └── credentials.yml 51 | │   │   └── README.md 52 | │   ├── notebooks 53 | │   ├── pyproject.toml 54 | │   ├── README.md 55 | │   ├── requirements.txt 56 | │   └── src 57 | │   └── projectname 58 | │   ├── __init__.py 59 | │   ├── __main__.py 60 | │   ├── pipeline_registry.py 61 | │   ├── pipelines 62 | │   │   └── __init__.py 63 | │   └── settings.py 64 | └── pyproject.toml 65 | 66 | 8 directories, 14 files 67 | ``` 68 | 69 | Our new, kedro-generated `projectname/pyproject.toml` looks like this: 70 | ``` 71 | [build-system] 72 | requires = [ "setuptools",] 73 | build-backend = "setuptools.build_meta" 74 | 75 | [project] 76 | name = "projectname" 77 | readme = "README.md" 78 | dynamic = [ "dependencies", "version",] 79 | 80 | [project.scripts] 81 | projectname = "projectname.__main__:main" 82 | 83 | [tool.kedro] 84 | package_name = "projectname" 85 | project_name = "projectname" 86 | kedro_init_version = "0.19.2" 87 | tools = [ "None",] 88 | example_pipeline = "False" 89 | source_dir = "src" 90 | 91 | [project.entry-points."kedro.hooks"] 92 | 93 | [tool.setuptools.dynamic.dependencies] 94 | file = "requirements.txt" 95 | 96 | [tool.setuptools.dynamic.version] 97 | attr = "projectname.__version__" 98 | 99 | [tool.setuptools.packages.find] 100 | where = [ "src",] 101 | namespaces = false 102 | ``` 103 | 104 | ## Converting from `setuptools` to `poetry` 105 | Now, the manual work of converting `pyproject.toml` from `setuptools` to `poetry` begins. 106 | 107 | We can simply copy all the contents from our initial pyproject.toml, and add them in on the top. 108 | We then need to delete the pre-existing `[build-system]` section that is referring to `setuptools`. 109 | In fact, we can delete all other `[tool.setuptools.xyz]` sections, as well. 110 | 111 | Then we deal with the `[project]` sections: 112 | The main `[project]` section can be deleted (or merged) as we replaced it with our `[tool.poetry]` section. 113 | The `[project.scripts]` section we can simply rename to `[tool.poetry.scripts]`. 114 | We also rename `[project.entry-points."kedro.hooks"]` to `[tool.poetry.plugins."kedro.hooks"]`. 115 | 116 | Further, we need to add any required packages, which the template had defined in `projectname/requirements.txt` to our new `[tool.poetry.dependencies]` section. 117 | 118 | Finally, our `projectname/pyproject.toml` file looks like this: 119 | ``` 120 | [tool.poetry] 121 | name = "package-name" 122 | version = "0.1.0" 123 | description = "" 124 | authors = ["Your Name "] 125 | readme = "README.md" 126 | 127 | [tool.poetry.dependencies] 128 | python = "^3.10" 129 | kedro = "0.19.2" 130 | ipython = ">=8.10" 131 | jupyterlab = ">=3.0" 132 | notebook = ">=7.1.0" 133 | 134 | [build-system] 135 | requires = ["poetry-core"] 136 | build-backend = "poetry.core.masonry.api" 137 | 138 | [tool.poetry.scripts] 139 | projectname = "projectname.__main__:main" 140 | 141 | [tool.kedro] 142 | package_name = "projectname" 143 | project_name = "projectname" 144 | kedro_init_version = "0.19.2" 145 | tools = [ "None",] 146 | example_pipeline = "False" 147 | source_dir = "src" 148 | 149 | [tool.poetry.plugins."kedro.hooks"] 150 | ``` 151 | And that's all the heavy editing. 152 | 153 | We wrap up the conversion process by deleting `projectname/requirements.txt` and the `pyproject.toml` and `poetry.lock` files in the project root directory. 154 | 155 | Finally, we re-run poetry update to re-initialize the poetry.lock file and install all the required packages. 156 | ``` 157 | > cd projectname 158 | > poetry update 159 | ``` 160 | 161 | We can check if all is well with: 162 | ``` 163 | > kedro info 164 | ``` 165 | 166 | 167 | 168 | -------------------------------------------------------------------------------- /python/knowledge_dump.md: -------------------------------------------------------------------------------- 1 | # Practice 2 | 3 | ## Basics 4 | 5 | * `arr[::-1]` uses `start:end:step` to return the array in reverse order 6 | * `range(start, stop, hop)` 7 | * `list [:]` creates a copy 8 | * `map(lambda x: str(x), arr)` Output: `['1','2','3']` 9 | * `float(“inf”)` and `float(“-inf”)` for initializing a far bigger and smaller value 10 | * String formatting https://pyformat.info/ 11 | 12 | ## How to use stuff 13 | 14 | ### List as queue 15 | ``` 16 | arr = [1,2,3] 17 | arr.append(x) # queue.push(x) 18 | arr.pop(0) #queue.pop() 19 | arr[0] #queue.peek() 20 | ``` 21 | 22 | ### List as stack 23 | ``` 24 | arr = [1,2,3] 25 | arr.append(x) #stack.push(x) 26 | arr.pop() # stack.pop() 27 | arr[-1] # stack.peek() 28 | ``` 29 | 30 | ### Use Namedtuples: 31 | ``` 32 | Point = namedtuple('Point', ['x', 'y']) 33 | p = Point(11, y=22) 34 | x, y = p 35 | p[0] + p[1] # 33 36 | p.x + p.y # 33 37 | p # Point(x=11, y=22) 38 | ``` 39 | 40 | ### Use defaultdict 41 | from collections for saving some time initializing values for new keys. 42 | `s = [('yellow', 1), ('blue', 2), ('yellow', 3), ('blue', 4), ('red', 1)]` 43 | 44 | ### Defaultdict use: 45 | ``` 46 | d = defaultdict(list) 47 | for k, v in s: 48 | d[k].append(v) 49 | d.items() # [('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])] 50 | ``` 51 | 52 | ### Normal dict use: 53 | ``` 54 | d = {} 55 | for k, v in s: 56 | d.setdefault(k, []).append(v) 57 | d.items() # [('blue', [2, 4]), ('red', [1]), ('yellow', [1, 3])] 58 | ``` 59 | Useful for: 60 | ``` 61 | s = 'mississippi' 62 | d = defaultdict(int) 63 | for k in s: 64 | d[k] += 1 65 | d.items() # [('i', 4), ('p', 2), ('s', 4), ('m', 1)] 66 | ``` 67 | 68 | ### `else` after loops 69 | Python has a cute else statement after the loop, which executes if you finish the loop normally (say, without a break). 70 | This can be very useful for shorthand coding, but to an outside obersver it looks like part of a faulty if-else at first glance. 71 | ``` 72 | counter = 0 73 | while counter <= 5: 74 | print counter, 75 | counter += 1 76 | else: 77 | print "loop exited normally" 78 | # Output: 0 1 2 3 4 5 loop exited normally 79 | 80 | for i in range(5): 81 | print i, 82 | if i > 3: 83 | break 84 | else: 85 | print "loop exited normally" 86 | # Output: 0 1 2 3 4 87 | ``` 88 | 89 | ### pickle 90 | -> write to string for storing in file 91 | 92 | ### `map(fun, iter)` 93 | returns a list of the results after applying the given function to each item of a given iterable (list, tuple etc.) 94 | 95 | ### `numpy.argsort()` 96 | returns the indices of the elements in sorted order 97 | 98 | ## Advanced 99 | 100 | ### Set Property 101 | set property of a class with `property(fget, fset, fdel, doc)` 102 | https://www.geeksforgeeks.org/python-property-function/ 103 | 104 | 105 | # Theory 106 | ## Key features of python: 107 | - interpreted 108 | - dynamically typed 109 | - OOP 110 | - writing code quick, runing code slow 111 | - "glue" code 112 | 113 | ## Python Objects 114 | All Python objects have three things: a value, a type, and an identity. 115 | 116 | ## Mutability 117 | ### Immutable objects 118 | "An object with a fixed value. includes numbers, strings and tuples. Such an object cannot be altered. A new object has to be created if a different value has to be stored." 119 | 120 | Integers (and floats, Booleans, strings, frozen sets, and bytes) are immutable; their value doesn't change. 121 | 122 | ### Mutable Objects 123 | "Mutable objects can change their value but keep their id()." 124 | 125 | Lists (and dictionaries, sets, arrays, and bytearrays), are mutable. 126 | 127 | ## `==` vs `is` 128 | The == equality operator compares values, while the is operator compares identities. 129 | 130 | You can consider `x is y` to be shorthand for `id(x) == id(y)` 131 | 132 | ## Hashability 133 | "An object is hashable if it has a hash value which never changes during its lifetime" Immutable objects can be hashable, mutable objects can't be hashable. 134 | (only hashable objects can be used as keys in a dictionary or as items in a set.) 135 | 136 | A hash is an integer that depends on an object's value, and objects with the same value always have the same hash. 137 | (Objects with different values will occasionally have the same hash too. This is called a hash collision.) 138 | 139 | While id() will return an integer based on an object's identity, the hash() function will return an integer (the object's hash) based on the hashable object's value. 140 | Since hashes are based on values and only immutable objects can be hashable, this means that hashes will never change during the object's lifetime. 141 | 142 | Corner case: a tuple (which should be immutable) that contains a mutable list cannot be hashed. 143 | A tuple is immutable: it always contains the same identities(objects) in the same order. 144 | However, if one of those objects is mutable, the value of that object can be changed (but not it's identity). 145 | Which is why immutable objects (ints, strings, bools, ...) in a tuple can not be changed (as it would require a new identity). 146 | 147 | ## range vs xrange 148 | xrange generates numbers as they are used, not in advance. 149 | 150 | ## monkey patching: 151 | modifying methods of class after definition 152 | 153 | ## Decoarators 154 | Using decorators, you can wrap a class or function method call so that a piece of code can be executed before or after the execution of the original code. 155 | 156 | 157 | 158 | --------------------------------------------------------------------------------