├── img ├── git │ ├── github │ │ ├── github-remote-01.txt │ │ ├── github-local-01.txt │ │ ├── github-remote-02.txt │ │ ├── github-local-03.txt │ │ ├── github-local-02.txt │ │ ├── github-remote-03.txt │ │ ├── github-local-04.txt │ │ ├── github-local-05.txt │ │ ├── github-local-06.txt │ │ ├── github-remote-01.svg │ │ ├── github-remote-02.svg │ │ ├── github-local-01.svg │ │ ├── github-local-02.svg │ │ ├── github-local-03.svg │ │ └── github-remote-03.svg │ ├── branches │ │ ├── git-branch-01.txt │ │ ├── git-branch-02.txt │ │ ├── git-branch-03.txt │ │ ├── simpsons.jpg │ │ ├── mergetool.png │ │ ├── pre-ff.txt │ │ ├── git-branch-04.txt │ │ ├── git-branch-05.txt │ │ ├── git-collaborative-work.txt │ │ ├── ff.txt │ │ ├── git-branch-06.txt │ │ ├── pre-rebase.txt │ │ ├── git-branch-07.txt │ │ ├── git-reset-soft-2.txt │ │ ├── gitink.sh │ │ ├── no-ff.txt │ │ ├── git-reset-soft-1.txt │ │ ├── git-branch-08.txt │ │ ├── git-branch-09.txt │ │ ├── rebase.txt │ │ ├── git-branch-10.txt │ │ ├── git-branch-11.txt │ │ ├── git-branch-12.txt │ │ ├── git-branch-13.txt │ │ ├── git-branch-01.svg │ │ ├── git-branch-02.svg │ │ ├── git-branch-03.svg │ │ ├── git-branch-04.svg │ │ └── pre-ff.svg │ ├── intro │ │ ├── topo.jpg │ │ ├── git-fire.jpg │ │ └── phd_final.gif │ ├── distributed │ │ ├── git-pull-rebase-1.txt │ │ ├── remote-02-remote.txt │ │ ├── git-pull-rebase-2.txt │ │ ├── git-pull-rebase-3a.txt │ │ ├── remote-06-remote.txt │ │ ├── remote-03-local.txt │ │ ├── remote-05-local.txt │ │ ├── git-pull-rebase-3b.txt │ │ ├── remote-04-local.txt │ │ ├── remote-07-remote.txt │ │ ├── git-pull-rebase-3b-merge.txt │ │ ├── remote-06-local.txt │ │ ├── remote-12-remote.txt │ │ ├── remote-07-local.txt │ │ ├── git-pull-rebase-3b-rebase.txt │ │ ├── remote-13-remote.txt │ │ ├── remote-08-local.txt │ │ ├── remote-12-local.txt │ │ ├── remote-13-local.txt │ │ ├── remote-09-local.txt │ │ ├── remote-10-local.txt │ │ ├── remote-11-local.txt │ │ ├── git-pull-rebase-1.svg │ │ ├── remote-02-remote.svg │ │ ├── remote-06-remote.svg │ │ ├── git-pull-rebase-3a.svg │ │ └── git-pull-rebase-2.svg │ └── confict-resolution │ │ ├── mk1.jpg │ │ ├── mk2.jpg │ │ ├── mergetool.png │ │ ├── git-split-branch-1.txt │ │ ├── git-split-branch-2.txt │ │ ├── git-split-branch-3.txt │ │ ├── git-fix-1.txt │ │ ├── git-fix-3.txt │ │ ├── git-fix-2.txt │ │ └── git-split-branch-1.svg ├── amdahl.png ├── launcher.jpg ├── profile.png ├── unit-testing.jpg ├── chucknorris_badass.jpg ├── chucknorris_thumbup.jpg └── profile.svg ├── intro.mkd ├── README.md ├── github.mkd ├── git-branch-design.mkd ├── documentation.mkd ├── optimiziation.mkd ├── git-archaeology.mkd ├── git-submodules.mkd └── tdd.mkd /img/git/github/github-remote-01.txt: -------------------------------------------------------------------------------- 1 | [master] 2 | | 3 | c1----c2 4 | -------------------------------------------------------------------------------- /img/git/branches/git-branch-01.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,HEAD] 3 | | 4 | c1----c2 5 | -------------------------------------------------------------------------------- /img/git/github/github-local-01.txt: -------------------------------------------------------------------------------- 1 | [master,origin/master] 2 | | 3 | c1----c2 4 | -------------------------------------------------------------------------------- /img/amdahl.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/amdahl.png -------------------------------------------------------------------------------- /img/git/github/github-remote-02.txt: -------------------------------------------------------------------------------- 1 | [master] 2 | | 3 | c1----c2----c3 4 | -------------------------------------------------------------------------------- /img/launcher.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/launcher.jpg -------------------------------------------------------------------------------- /img/profile.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/profile.png -------------------------------------------------------------------------------- /img/git/github/github-local-03.txt: -------------------------------------------------------------------------------- 1 | [master,origin/master] 2 | | 3 | c1----c2----c3 4 | -------------------------------------------------------------------------------- /img/git/intro/topo.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/git/intro/topo.jpg -------------------------------------------------------------------------------- /img/unit-testing.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/unit-testing.jpg -------------------------------------------------------------------------------- /img/git/branches/git-branch-02.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,HEAD] 3 | | 4 | c1----c2 5 | | 6 | [devel] 7 | -------------------------------------------------------------------------------- /img/chucknorris_badass.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/chucknorris_badass.jpg -------------------------------------------------------------------------------- /img/chucknorris_thumbup.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/chucknorris_thumbup.jpg -------------------------------------------------------------------------------- /img/git/branches/git-branch-03.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,_HEAD] 3 | | 4 | c1----c2 5 | | 6 | [devel,HEAD] 7 | -------------------------------------------------------------------------------- /img/git/intro/git-fire.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/git/intro/git-fire.jpg -------------------------------------------------------------------------------- /img/git/intro/phd_final.gif: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/git/intro/phd_final.gif -------------------------------------------------------------------------------- /img/git/branches/simpsons.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/git/branches/simpsons.jpg -------------------------------------------------------------------------------- /img/git/distributed/git-pull-rebase-1.txt: -------------------------------------------------------------------------------- 1 | [origin/master] 2 | | 3 | c1----c2 4 | | 5 | [master,HEAD] 6 | -------------------------------------------------------------------------------- /img/git/branches/mergetool.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/git/branches/mergetool.png -------------------------------------------------------------------------------- /img/git/github/github-local-02.txt: -------------------------------------------------------------------------------- 1 | [origin/master] 2 | | 3 | c1----c2----c3 4 | | 5 | [master] 6 | -------------------------------------------------------------------------------- /img/git/github/github-remote-03.txt: -------------------------------------------------------------------------------- 1 | [master] 2 | | 3 | c1----c2----c3----c4----c5 4 | -------------------------------------------------------------------------------- /img/git/confict-resolution/mk1.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/git/confict-resolution/mk1.jpg -------------------------------------------------------------------------------- /img/git/confict-resolution/mk2.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/git/confict-resolution/mk2.jpg -------------------------------------------------------------------------------- /img/git/confict-resolution/mergetool.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/coderefinery/old-archived-lessons/HEAD/img/git/confict-resolution/mergetool.png -------------------------------------------------------------------------------- /img/git/confict-resolution/git-split-branch-1.txt: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | c1----c2 [master,HEAD] 5 | \ | 6 | c3----c4----c5 7 | -------------------------------------------------------------------------------- /img/git/distributed/remote-02-remote.txt: -------------------------------------------------------------------------------- 1 | 2 | [master] 3 | | 4 | c1----c2----c3 5 | \ 6 | b1 7 | | 8 | [dev] 9 | -------------------------------------------------------------------------------- /img/git/branches/pre-ff.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,HEAD] 3 | | 4 | c1----c2 5 | \ 6 | b1----b2----b3 7 | | 8 | [devel] 9 | -------------------------------------------------------------------------------- /img/git/distributed/git-pull-rebase-2.txt: -------------------------------------------------------------------------------- 1 | [origin/master] 2 | | 3 | c1----c2 4 | \ 5 | c3----c4 6 | | 7 | [master,HEAD] 8 | -------------------------------------------------------------------------------- /img/git/distributed/git-pull-rebase-3a.txt: -------------------------------------------------------------------------------- 1 | 2 | 3 | c1----c2 [origin/master] 4 | \ | 5 | c3----c4 6 | | 7 | [master,HEAD] 8 | -------------------------------------------------------------------------------- /img/git/branches/git-branch-04.txt: -------------------------------------------------------------------------------- 1 | 2 | [master] 3 | | 4 | c1----c2 5 | \ 6 | b1----b2----b3 7 | | 8 | [devel,HEAD] 9 | -------------------------------------------------------------------------------- /img/git/distributed/remote-06-remote.txt: -------------------------------------------------------------------------------- 1 | 2 | [master] 3 | | 4 | c1----c2----c3----c4 5 | \ 6 | b1 7 | | 8 | [dev] 9 | -------------------------------------------------------------------------------- /img/git/branches/git-branch-05.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,HEAD] 3 | | 4 | c1----c2 5 | \ 6 | b1----b2----b3 7 | | 8 | [devel,_HEAD] 9 | -------------------------------------------------------------------------------- /img/git/distributed/remote-03-local.txt: -------------------------------------------------------------------------------- 1 | 2 | [origin/master,master,HEAD] 3 | | 4 | c1----c2----c3 5 | \ 6 | b1 7 | | 8 | [origin/dev] 9 | -------------------------------------------------------------------------------- /img/git/github/github-local-04.txt: -------------------------------------------------------------------------------- 1 | [master,origin/master] 2 | | 3 | c1----c2----c3----c4----c5 4 | | 5 | [upstream/master] 6 | -------------------------------------------------------------------------------- /img/git/github/github-local-05.txt: -------------------------------------------------------------------------------- 1 | [origin/master] 2 | | 3 | c1----c2----c3----c4----c5 4 | | 5 | [master,upstream/master] 6 | -------------------------------------------------------------------------------- /img/git/branches/git-collaborative-work.txt: -------------------------------------------------------------------------------- 1 | b1----b2----b3 2 | / \ 3 | c1----c2----c3----c4----m2----m3 4 | \ \ / 5 | d1----m1----d2 6 | -------------------------------------------------------------------------------- /img/git/distributed/remote-05-local.txt: -------------------------------------------------------------------------------- 1 | 2 | [origin/master,master,HEAD] 3 | | 4 | c1----c2----c3----c4 5 | \ 6 | b1 7 | | 8 | [origin/dev] 9 | -------------------------------------------------------------------------------- /img/git/branches/ff.txt: -------------------------------------------------------------------------------- 1 | 2 | [_master,_HEAD] 3 | | 4 | c1----c2 [master,HEAD] 5 | \ | 6 | b1----b2----b3 7 | | 8 | [devel] 9 | -------------------------------------------------------------------------------- /img/git/confict-resolution/git-split-branch-2.txt: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | c1----c2 [master,HEAD] 5 | \ | 6 | c3----c4----c5 7 | | 8 | [feature] 9 | -------------------------------------------------------------------------------- /img/git/distributed/git-pull-rebase-3b.txt: -------------------------------------------------------------------------------- 1 | [origin/master] 2 | | 3 | c1----c2----c5----c6 4 | \ 5 | c3----c4 6 | | 7 | [master,HEAD] 8 | -------------------------------------------------------------------------------- /img/git/distributed/remote-04-local.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,HEAD] 3 | | 4 | c1----c2----c3----c4 5 | \ | 6 | b1 [origin/master] 7 | | 8 | [origin/dev] 9 | -------------------------------------------------------------------------------- /img/git/github/github-local-06.txt: -------------------------------------------------------------------------------- 1 | [origin/master] 2 | | 3 | c1----c2----c3----c4----c5 4 | | 5 | [master,upstream/master] 6 | -------------------------------------------------------------------------------- /img/git/branches/git-branch-06.txt: -------------------------------------------------------------------------------- 1 | 2 | [_master,_HEAD] [master,HEAD] 3 | | | 4 | c1----c2----c3----c4----c5 5 | \ 6 | b1----b2----b3 7 | | 8 | [devel] 9 | -------------------------------------------------------------------------------- /img/git/branches/pre-rebase.txt: -------------------------------------------------------------------------------- 1 | 2 | [master] 3 | | 4 | c1----c2----c3----c4----c5 5 | \ 6 | b1----b2----b3 7 | | 8 | [devel,HEAD] 9 | -------------------------------------------------------------------------------- /img/git/distributed/remote-07-remote.txt: -------------------------------------------------------------------------------- 1 | 2 | [_master] [master] 3 | | | 4 | c1----c2----c3----c4----c5----c6----c7 5 | \ 6 | b1 7 | | 8 | [dev] 9 | -------------------------------------------------------------------------------- /img/git/branches/git-branch-07.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,_HEAD] 3 | | 4 | c1----c2----c3----c4----c5 5 | \ 6 | b1----b2----b3 7 | | 8 | [devel,HEAD] 9 | -------------------------------------------------------------------------------- /img/git/branches/git-reset-soft-2.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,HEAD] 3 | | 4 | c1----c2----c3----c4----c5 5 | \ 6 | b1----b2----b3 7 | | 8 | [feature] 9 | -------------------------------------------------------------------------------- /img/git/confict-resolution/git-split-branch-3.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,HEAD] 3 | | 4 | c1----c2 [_master,_HEAD] 5 | \ | 6 | c3----c4----c5 7 | | 8 | [feature] 9 | -------------------------------------------------------------------------------- /img/git/branches/gitink.sh: -------------------------------------------------------------------------------- 1 | #!/usr/bin/env bash 2 | 3 | if [ ! -d "gitink" ]; then 4 | git clone https://github.com/bast/gitink.git 5 | fi 6 | 7 | for file in *.txt; do 8 | python gitink/gitink.py $file > ${file%txt}svg 9 | done 10 | -------------------------------------------------------------------------------- /img/git/branches/no-ff.txt: -------------------------------------------------------------------------------- 1 | 2 | [_master,_HEAD] [master,HEAD] 3 | | | 4 | c1----c2----------------m1 5 | \ / 6 | b1----b2----b3 7 | | 8 | [devel] 9 | -------------------------------------------------------------------------------- /img/git/distributed/git-pull-rebase-3b-merge.txt: -------------------------------------------------------------------------------- 1 | [origin/master] 2 | | 3 | c1----c2----c5----c6 4 | \ \ 5 | c3----c4----m1 6 | | 7 | [master,HEAD] 8 | -------------------------------------------------------------------------------- /img/git/distributed/remote-06-local.txt: -------------------------------------------------------------------------------- 1 | 2 | [origin/master] [master,HEAD] 3 | | | 4 | c1----c2----c3----c4----c5----c6----c7 5 | \ 6 | b1 7 | | 8 | [origin/dev] 9 | -------------------------------------------------------------------------------- /img/git/branches/git-reset-soft-1.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,HEAD] 3 | | 4 | c1----c2----c3----c4----m1 5 | \ / 6 | b1----b2----b3 7 | | 8 | [feature] 9 | -------------------------------------------------------------------------------- /img/git/distributed/remote-12-remote.txt: -------------------------------------------------------------------------------- 1 | 2 | [master] 3 | | 4 | c1----c2----c3----c4----c5----c6----c7----c8 5 | \ 6 | b1 7 | | 8 | [dev] 9 | -------------------------------------------------------------------------------- /img/git/distributed/remote-07-local.txt: -------------------------------------------------------------------------------- 1 | 2 | [_origin/master] [origin/master,master,HEAD] 3 | | | 4 | c1----c2----c3----c4----c5----c6----c7 5 | \ 6 | b1 7 | | 8 | [origin/dev] 9 | -------------------------------------------------------------------------------- /img/git/branches/git-branch-08.txt: -------------------------------------------------------------------------------- 1 | 2 | [master] 3 | | 4 | c1----c2----c3----c4----c5 5 | \ \ 6 | b1----b2----b3----m1 7 | | 8 | [devel,HEAD] 9 | -------------------------------------------------------------------------------- /img/git/distributed/git-pull-rebase-3b-rebase.txt: -------------------------------------------------------------------------------- 1 | [origin/master] 2 | | 3 | c1----c2----c5----c6 4 | \ 5 | c3*----c4* 6 | | 7 | [master,HEAD] 8 | -------------------------------------------------------------------------------- /img/git/confict-resolution/git-fix-1.txt: -------------------------------------------------------------------------------- 1 | 2 | [master] 3 | | 4 | c1----c2----c3----c4----c5 5 | \ 6 | b1----b2----fix----b3----b4 7 | | 8 | [feature,HEAD] 9 | -------------------------------------------------------------------------------- /img/git/distributed/remote-13-remote.txt: -------------------------------------------------------------------------------- 1 | 2 | [master] 3 | | 4 | c1----c2----c3----c4----c5----c6----c7----c8 5 | \ 6 | b1----b2----b3 7 | | | 8 | [_dev] [dev] 9 | -------------------------------------------------------------------------------- /img/git/confict-resolution/git-fix-3.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,HEAD] 3 | | 4 | c1----c2----c3----c4----c5----fix* 5 | \ 6 | b1----b2----fix----b3----b4 7 | | 8 | [feature] 9 | -------------------------------------------------------------------------------- /img/git/branches/git-branch-09.txt: -------------------------------------------------------------------------------- 1 | 2 | [master] 3 | | 4 | c1----c2----c3----c4----c5 5 | \ \ 6 | b1----b2----b3----m1----b4----b5 7 | | 8 | [devel,HEAD] 9 | -------------------------------------------------------------------------------- /img/git/branches/rebase.txt: -------------------------------------------------------------------------------- 1 | 2 | [master] 3 | | 4 | c1----c2----c3----c4----c5 5 | \ 6 | b1*----b2*----b3* 7 | | 8 | [devel,HEAD] 9 | -------------------------------------------------------------------------------- /img/git/branches/git-branch-10.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,HEAD] 3 | | 4 | c1----c2----c3----c4----c5 5 | \ \ 6 | b1----b2----b3----m1----b4----b5 7 | | 8 | [devel,_HEAD] 9 | -------------------------------------------------------------------------------- /img/git/confict-resolution/git-fix-2.txt: -------------------------------------------------------------------------------- 1 | 2 | [master] 3 | | 4 | c1----c2----c3----c4----c5---fix 5 | \ \ 6 | b1----b2----b3----b4----m1 7 | | 8 | [feature,HEAD] 9 | -------------------------------------------------------------------------------- /img/git/branches/git-branch-11.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,HEAD] 3 | | 4 | c1----c2----c3----c4----c5----c6----c7 5 | \ \ 6 | b1----b2----b3----m1----b4----b5 7 | | 8 | [devel] 9 | -------------------------------------------------------------------------------- /img/git/branches/git-branch-12.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,HEAD] 3 | | 4 | c1----c2----c3----c4----c5----c6----c7----m2 5 | \ \ / 6 | b1----b2----b3----m1----b4----b5 7 | | 8 | [devel] 9 | -------------------------------------------------------------------------------- /img/git/branches/git-branch-13.txt: -------------------------------------------------------------------------------- 1 | 2 | [master,HEAD] 3 | | 4 | c1----c2----c3----c4----c5----c6----c7----m2 5 | \ \ / 6 | b1----b2----b3----m1----b4----b5 7 | | 8 | [_devel] 9 | -------------------------------------------------------------------------------- /img/git/distributed/remote-08-local.txt: -------------------------------------------------------------------------------- 1 | 2 | [origin/master,_master,_HEAD] 3 | | 4 | c1----c2----c3----c4----c5----c6----c7 5 | \ \ 6 | b1 d8 7 | | | 8 | [origin/dev] [master,HEAD] 9 | -------------------------------------------------------------------------------- /img/git/distributed/remote-12-local.txt: -------------------------------------------------------------------------------- 1 | 2 | [origin/master] 3 | | 4 | c1----c2----c3----c4----c5----c6----c7----c8 5 | \ \ 6 | b1----b2----b3 d8* 7 | | | | 8 | [origin/dev][dev,HEAD] [master] 9 | -------------------------------------------------------------------------------- /img/git/distributed/remote-13-local.txt: -------------------------------------------------------------------------------- 1 | 2 | [origin/master] 3 | | 4 | c1----c2----c3----c4----c5----c6----c7----c8 5 | \ \ 6 | b1----b2----b3 d8* 7 | | | 8 | [origin/dev,dev,HEAD] [master] 9 | -------------------------------------------------------------------------------- /img/git/distributed/remote-09-local.txt: -------------------------------------------------------------------------------- 1 | 2 | [origin/master] 3 | | 4 | c1----c2----c3----c4----c5----c6----c7----c8 5 | \ \ \ 6 | b1 d8----m1 7 | | | 8 | [origin/dev] [master,HEAD] 9 | -------------------------------------------------------------------------------- /img/git/distributed/remote-10-local.txt: -------------------------------------------------------------------------------- 1 | 2 | [origin/master] 3 | | 4 | c1----c2----c3----c4----c5----c6----c7----c8 5 | \ \ 6 | b1 d8* 7 | | | 8 | [origin/dev] [master,HEAD] 9 | -------------------------------------------------------------------------------- /img/git/distributed/remote-11-local.txt: -------------------------------------------------------------------------------- 1 | 2 | [origin/master] 3 | | 4 | c1----c2----c3----c4----c5----c6----c7----c8 5 | \ \ 6 | b1 d8* 7 | | | 8 | [origin/dev,dev,HEAD] [master,_HEAD] 9 | -------------------------------------------------------------------------------- /intro.mkd: -------------------------------------------------------------------------------- 1 | name: inverse 2 | layout: true 3 | class: center, middle, inverse 4 | 5 | --- 6 | 7 | # Introduction and overview 8 | 9 | ## Radovan Bast 10 | 11 | --- 12 | 13 | layout: false 14 | 15 | ## Motivation 16 | 17 | "Software is a cornerstone of science. Without software, twenty-first century 18 | science would be impossible. Without better software, science cannot progress." 19 | [http://sciencecodemanifesto.org] 20 | 21 | - Tools for better scientific software 22 | - Managing complexity 23 | - Good programming practices come from experience 24 | - Experience comes from bad programming practices 25 | 26 | --- 27 | 28 | ## Teachers 29 | 30 | - Rossen Apostolov (PDC/KTH) 31 | - Radovan Bast (Department of IT, UiT The Arctic University of Norway) 32 | - Jonas Juselius (Serit IT Partner Tromso) 33 | - Erik Kaellman (Uppsala universitet) 34 | - Michael Schliephake (PDC/KTH) 35 | 36 | --- 37 | 38 | ## Practical information (memory help) 39 | 40 | - Access cards 41 | - Network 42 | - Food and microwaves 43 | - Coffee/tea/cake 44 | - SeSE format 45 | - Exercises and software 46 | - Project work 47 | - Website 48 | - Program 49 | - Slides 50 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # Links to lesson slides (legacy versions) 2 | 3 | - [Introduction to version control and Git](http://cicero.xyz/v2/remark/github/coderefinery/lessons/master/git-intro.mkd/) 4 | - [Branching and merging in Git](http://cicero.xyz/v2/remark/github/coderefinery/lessons/master/git-branches.mkd/) 5 | - [Conflict resolution in Git](http://cicero.xyz/v2/remark/github/coderefinery/lessons/master/git-conflict-resolution.mkd/) 6 | - [Collaboration via distributed version control](http://cicero.xyz/v2/remark/github/coderefinery/lessons/master/git-distributed.mkd/) 7 | - [Working with GitHub](http://cicero.xyz/v2/remark/github/coderefinery/lessons/master/github.mkd/) 8 | - [Git branch design](http://cicero.xyz/v2/remark/github/coderefinery/lessons/master/git-branch-design.mkd/) 9 | - [Archaeology with Git](http://cicero.xyz/v2/remark/github/coderefinery/lessons/master/git-archaeology.mkd/) 10 | - [Including external projects using Git submodules](http://cicero.xyz/v2/remark/github/coderefinery/lessons/master/git-submodules.mkd/) 11 | - [Modern code documentation](http://cicero.xyz/v2/remark/github/coderefinery/lessons/master/documentation.mkd/) 12 | - [Profiling and code optimization](http://cicero.xyz/v2/remark/github/coderefinery/lessons/master/optimiziation.mkd/) 13 | - [Test-driven development](http://cicero.xyz/v2/remark/github/coderefinery/lessons/master/tdd.mkd/) 14 | - [CMake basics for busy scientists](http://cicero.xyz/v2/remark/github/coderefinery/lessons/master/cmake-basics.mkd/) 15 | - [Advanced CMake Kung Fu](http://cicero.xyz/v2/remark/github/coderefinery/lessons/master/cmake-kung-fu.mkd/) 16 | 17 | Licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). 18 | Code examples: [OSI](http://opensource.org)-approved [MIT license](http://opensource.org/licenses/mit-license.html). 19 | -------------------------------------------------------------------------------- /img/git/github/github-remote-01.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 17 | 19 | 26 | 31 | 32 | 36 | 44 | c1 55 | 59 | 67 | c2 78 | 82 | 87 | 91 | 99 | master 110 | 114 | 119 | 120 | -------------------------------------------------------------------------------- /img/profile.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 18 | 20 | 38 | 40 | 41 | 43 | image/svg+xml 44 | 46 | 47 | 48 | 49 | 50 | 54 | coffeeemailscomicsdebuggingcompiling 86 | IIIIIIIIIIIIIIIIIIIII 118 | 119 | 120 | -------------------------------------------------------------------------------- /github.mkd: -------------------------------------------------------------------------------- 1 | name: inverse 2 | layout: true 3 | class: center, middle, inverse 4 | 5 | --- 6 | 7 | # Working with GitHub 8 | 9 | ## Radovan Bast 10 | 11 | Licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). 12 | Code examples: [OSI](http://opensource.org)-approved [MIT license](http://opensource.org/licenses/mit-license.html). 13 | 14 | --- 15 | 16 | layout: false 17 | 18 | ## This talk is a live demo, list below serves as memory help 19 | 20 | - Over 3 million users 21 | - Over 10 million repositories 22 | - Largest code host in the world 23 | - Today the de facto standard 24 | - GitHub activity good for your CV 25 | - Overview 26 | - Creating and deleting projects 27 | - Accessing projects (https or ssh) 28 | - README.md 29 | - Issues (tickets) 30 | - Wiki 31 | - Fork/pull-request mechanism: like peer review 32 | - Autoclosing issues 33 | - Discussing with mentions 34 | - GitHub Pages 35 | - Hooks 36 | - Gist 37 | - KTH has GitHub Enterprise 38 | 39 | --- 40 | 41 | ## Forking a repository 42 | 43 | - A fork is basically a (bare) clone 44 | - The upstream repo and the fork are in principle independent repositories 45 | - We copy all commits, all branches 46 | 47 | **https://github.com/foo/foo.git** 48 | ![](img/git/github/github-remote-01.svg) 49 | 50 | **https://github.com/user/foo.git** 51 | ![](img/git/github/github-remote-01.svg) 52 | 53 | --- 54 | 55 | ## Cloning a fork to work on it 56 | 57 | ```shell 58 | $ git clone https://github.com/user/foo.git 59 | ``` 60 | 61 | **https://github.com/foo/foo.git** 62 | ![](img/git/github/github-remote-01.svg) 63 | 64 | **https://github.com/user/foo.git** 65 | ![](img/git/github/github-remote-01.svg) 66 | 67 | **local repo** 68 | ![](img/git/github/github-local-01.svg) 69 | 70 | --- 71 | 72 | ## Working with the local repo 73 | 74 | - We do some work and make a commit 75 | 76 | **https://github.com/foo/foo.git** 77 | ![](img/git/github/github-remote-01.svg) 78 | 79 | **https://github.com/user/foo.git** 80 | ![](img/git/github/github-remote-01.svg) 81 | 82 | **local repo** 83 | ![](img/git/github/github-local-02.svg) 84 | 85 | --- 86 | 87 | ## Pushing the change to origin 88 | 89 | ```shell 90 | $ git push origin master 91 | ``` 92 | 93 | **https://github.com/foo/foo.git** 94 | ![](img/git/github/github-remote-01.svg) 95 | 96 | **https://github.com/user/foo.git** 97 | ![](img/git/github/github-remote-02.svg) 98 | 99 | **local repo** 100 | ![](img/git/github/github-local-03.svg) 101 | 102 | --- 103 | 104 | ## Pull-request 105 | 106 | - We can file a pull-request 107 | - A pull-request means: "please review my changes and if you agree, merge them with a mouseclick" 108 | 109 | **https://github.com/foo/foo.git** 110 | ![](img/git/github/github-remote-01.svg) 111 | 112 | **https://github.com/user/foo.git** 113 | ![](img/git/github/github-remote-02.svg) 114 | 115 | **local repo** 116 | ![](img/git/github/github-local-03.svg) 117 | 118 | --- 119 | 120 | ## Pull-request 121 | 122 | - If the pull-request is accepted, the change is incorporated 123 | 124 | **https://github.com/foo/foo.git** 125 | ![](img/git/github/github-remote-02.svg) 126 | 127 | **https://github.com/user/foo.git** 128 | ![](img/git/github/github-remote-02.svg) 129 | 130 | **local repo** 131 | ![](img/git/github/github-local-03.svg) 132 | 133 | --- 134 | 135 | ## Updating the fork with upstream changes 136 | 137 | - Upstream repo receives other changes (other merged pull-requests) 138 | - How do we get these changes to the forked repo? 139 | 140 | **https://github.com/foo/foo.git** 141 | ![](img/git/github/github-remote-03.svg) 142 | 143 | **https://github.com/user/foo.git** 144 | ![](img/git/github/github-remote-02.svg) 145 | 146 | **local repo** 147 | ![](img/git/github/github-local-03.svg) 148 | 149 | --- 150 | 151 | ## Updating the fork with upstream changes 152 | 153 | ```shell 154 | $ git remote add upstream https://github.com/foo/foo.git 155 | $ git fetch upstream 156 | ``` 157 | 158 | **https://github.com/foo/foo.git** 159 | ![](img/git/github/github-remote-03.svg) 160 | 161 | **https://github.com/user/foo.git** 162 | ![](img/git/github/github-remote-02.svg) 163 | 164 | **local repo** 165 | ![](img/git/github/github-local-04.svg) 166 | 167 | --- 168 | 169 | ## Updating the fork with upstream changes 170 | 171 | ```shell 172 | $ git checkout master 173 | $ git merge upstream/master 174 | ``` 175 | 176 | **https://github.com/foo/foo.git** 177 | ![](img/git/github/github-remote-03.svg) 178 | 179 | **https://github.com/user/foo.git** 180 | ![](img/git/github/github-remote-02.svg) 181 | 182 | **local repo** 183 | ![](img/git/github/github-local-05.svg) 184 | 185 | --- 186 | 187 | ## Updating the fork with upstream changes 188 | 189 | ```shell 190 | $ git push origin master 191 | ``` 192 | 193 | **https://github.com/foo/foo.git** 194 | ![](img/git/github/github-remote-03.svg) 195 | 196 | **https://github.com/user/foo.git** 197 | ![](img/git/github/github-remote-03.svg) 198 | 199 | **local repo** 200 | ![](img/git/github/github-local-06.svg) 201 | 202 | --- 203 | 204 | ## Pro-tip 205 | 206 | - Different URLs for fetch and push 207 | 208 | ```shell 209 | $ git remote add origin https://github.com/foo/foo.git 210 | $ git remote set-url --push origin https://github.com/user/foo.git 211 | ``` 212 | 213 | - Now we always fetch from the central repo and push to forked repo 214 | 215 | ```shell 216 | $ git remote -v 217 | 218 | origin https://github.com/foo/foo.git (fetch) 219 | origin https://github.com/user/foo.git (push) 220 | ``` 221 | 222 | --- 223 | 224 | ## Summary 225 | 226 | - Working with multiple remotes is not scary 227 | - `origin` and `upstream` are just aliases 228 | - We can call these aliases as we like 229 | - We can add and remove remotes 230 | 231 | ```shell 232 | $ git remote add upstream https://github.com/foo/foo.git 233 | $ git remote rm upstream 234 | ``` 235 | 236 | - We synchronize remotes via the local clone 237 | - To see all remotes 238 | 239 | ```shell 240 | $ git remote -v 241 | ``` 242 | -------------------------------------------------------------------------------- /img/git/branches/git-branch-01.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 17 | 19 | 26 | 31 | 32 | 36 | 44 | c1 55 | 59 | 67 | c2 78 | 82 | 87 | 91 | 99 | master 110 | 114 | 119 | 123 | 131 | HEAD 142 | 146 | 151 | 152 | -------------------------------------------------------------------------------- /img/git/github/github-remote-02.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 17 | 19 | 26 | 31 | 32 | 36 | 44 | c1 55 | 59 | 67 | c2 78 | 82 | 87 | 91 | 99 | c3 110 | 114 | 119 | 123 | 131 | master 142 | 146 | 151 | 152 | -------------------------------------------------------------------------------- /img/git/github/github-local-01.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 17 | 19 | 26 | 31 | 32 | 36 | 44 | c1 55 | 59 | 67 | c2 78 | 82 | 87 | 91 | 99 | master 110 | 114 | 119 | 123 | 131 | origin/master 142 | 146 | 151 | 152 | -------------------------------------------------------------------------------- /git-branch-design.mkd: -------------------------------------------------------------------------------- 1 | name: inverse 2 | layout: true 3 | class: center, middle, inverse 4 | 5 | --- 6 | 7 | # Git branch design 8 | 9 | ## Radovan Bast 10 | 11 | Licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). 12 | Code examples: [OSI](http://opensource.org)-approved [MIT license](http://opensource.org/licenses/mit-license.html). 13 | 14 | --- 15 | 16 | layout: false 17 | 18 | ## Branch naming 19 | 20 | - Name your local branches such that you will recognize them 3 months later 21 | - "test", "foo", "debug", "mybranch" are not good 22 | - Give descriptive names to remote branches 23 | - For topic branches we recommend to name them "author/topic" (example `radovan/new-integrator`) 24 | - Then everybody knows who is to be contacted about this branch (e.g. stale branches) 25 | - Also you can easily find "your" branches 26 | 27 | ```shell 28 | $ git branch -r | grep radovan 29 | ``` 30 | 31 | - Name bugfix branches after the issue/ticket (e.g. `issue-137`) 32 | - For release branches we recommend `release-2.x` or `release/2.x` or `stable/2.x` 33 | 34 | --- 35 | 36 | ## Always have only one main development line 37 | 38 | - Document where it is 39 | - Organize branches according to features, not according to groups of people 40 | - Good: branches `feature-a`, `feature-b`, `feature-c` 41 | - Bad: branches `stockholm`, `san_francisco`, `uppsala` 42 | - Reason: `stockholm`, `san_francisco`, and `uppsala` will either diverge 43 | (three main development lines) or somebody will spend a heroic effort to keep 44 | them synchronized 45 | 46 | --- 47 | 48 | ## Every commit on the main development line should compile 49 | 50 | - Sometimes you want to find a commit in the past that broke some functionality 51 | - When using `git bisect` you will see that it is very helpful if all commits compile 52 | - On the other hand you will see that it is annoying if you hit a commit that does not compile 53 | - This is why we insist so much on a compiling main development line with nice history 54 | - There is no reason to commit broken or unfinished code to the main development line: for this we have branches 55 | 56 | --- 57 | 58 | ## Branch hygiene 59 | 60 | - You committed 3 commits but would like to squash them into one 61 | 62 | ```shell 63 | $ git log --oneline 64 | 65 | 6e129cf documentation for feature C 66 | 05344f6 small fix for feature C 67 | bc11c47 save work on feature C 68 | aa25177 feature B 69 | 6b58ba4 feature A 70 | ``` 71 | 72 | - We have no uncommitted changes 73 | - We wish our last 3 commits (6e129cf, 05344f6, bc11c47) were just one 74 | - We use a soft reset: 75 | 76 | ```shell 77 | $ git reset --soft aa25177 78 | ``` 79 | 80 | - This means that we move our repository back to commit aa25177 81 | BUT we keep the working tree of the last commit (6e129cf) 82 | 83 | --- 84 | 85 | ## Squashing commits 86 | 87 | ```shell 88 | $ git reset --soft aa25177 89 | ``` 90 | 91 | - Verify the result with `git status` 92 | - Now we can commit modifications w.r.t. `aa25177` in one single nice commit 93 | 94 | ```shell 95 | $ git commit -m 'feature C' 96 | 97 | 81e100c feature C 98 | aa25177 feature B 99 | 6b58ba4 feature A 100 | ``` 101 | 102 | - The history tells us that we committed our work in a single commit `81e100c` 103 | - The final state of the actual code is identical 104 | - Alternative to `git reset --soft` is an interactive rebase 105 | - We recommend to create commits on the main development line which are nice logical 106 | units 107 | - Commits should be pickable (not too large not too small for a `cherry-pick`) 108 | - Avoid ball-of-mud commits 109 | 110 | --- 111 | 112 | ## Develop on feature branches 113 | 114 | - Keeps bugs away from the main development line 115 | - Divide and conquer: do not create a branch for "everything" 116 | - The more you do on one branch the longer it will take until you can reintegrate it 117 | - The more granular the branches, the shorter lived 118 | - Talk with your colleagues to avoid conflicts 119 | 120 | --- 121 | 122 | ## How to test combinations of features 123 | 124 | - I develop `feature-a` 125 | - My colleague develops `feature-b` 126 | - Both are not ready yet to go into the main line 127 | - How can we test them together? 128 | 129 | --- 130 | 131 | ## How to test combinations of features 132 | 133 | - Do not cross-merge feature branches 134 | - Reason: if `feature-a` becomes ready, it cannot be integrated to the main line 135 | because it is then diluted with `feature-b` 136 | - It is easy to make soup out of vegetables, it is difficult to separate a vegetable out of a soup 137 | - Test combinations on integration branches 138 | - Integration branches only integrate, we do not "work" on them 139 | - Same holds for testing combinations with the main line 140 | - The main line should ideally be an integration branch 141 | 142 | --- 143 | 144 | ## A successful Git branching model 145 | 146 | - Very popular 147 | - http://nvie.com/posts/a-successful-git-branching-model/ 148 | - Critique 149 | - Naming is IMO unfortunate (rename `develop` to `master` and `master` to `stable`) 150 | - Model is not ideal if you need to support past versions 151 | - Good if you do not distribute the stable release (e.g. if you run it on your servers) 152 | 153 | --- 154 | 155 | ## Alternative: separate branch for each major release 156 | 157 | - For release branches we recommend `release-2.x` or `release/2.x` or `stable/2.x` 158 | - It is crystal clear where the main development line is 159 | - Does not require to create new branches for patches of past versions 160 | - Good if you distribute code 161 | - Patches need to be applied to the oldest supported release branch and cherry-picked or merged 162 | "up" to the main line 163 | 164 | --- 165 | 166 | ## Document and enforce your branch naming and strategies 167 | 168 | - Document recommended branch naming 169 | - Document your branching layout/strategy 170 | - Require your developers to follow it (code review) 171 | - Write-protect your main development line and release branch(es) 172 | - Use semantic versioning: http://semver.org 173 | -------------------------------------------------------------------------------- /img/git/branches/git-branch-02.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 17 | 19 | 26 | 31 | 32 | 36 | 44 | c1 55 | 59 | 67 | c2 78 | 82 | 87 | 91 | 99 | master 110 | 114 | 119 | 123 | 131 | HEAD 142 | 146 | 151 | 155 | 163 | devel 174 | 178 | 183 | 184 | -------------------------------------------------------------------------------- /img/git/github/github-local-02.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 17 | 19 | 26 | 31 | 32 | 36 | 44 | c1 55 | 59 | 67 | c2 78 | 82 | 87 | 91 | 99 | origin/master 110 | 114 | 119 | 123 | 131 | c3 142 | 146 | 151 | 155 | 163 | master 174 | 178 | 183 | 184 | -------------------------------------------------------------------------------- /img/git/distributed/git-pull-rebase-1.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 17 | 19 | 26 | 31 | 32 | 36 | 44 | c1 55 | 59 | 67 | c2 78 | 82 | 87 | 91 | 99 | origin/master 110 | 114 | 119 | 123 | 131 | master 142 | 146 | 151 | 155 | 163 | HEAD 174 | 178 | 183 | 184 | -------------------------------------------------------------------------------- /img/git/github/github-local-03.svg: -------------------------------------------------------------------------------- 1 | 2 | 3 | 17 | 19 | 26 | 31 | 32 | 36 | 44 | c1 55 | 59 | 67 | c2 78 | 82 | 87 | 91 | 99 | c3 110 | 114 | 119 | 123 | 131 | master 142 | 146 | 151 | 155 | 163 | origin/master 174 | 178 | 183 | 184 | -------------------------------------------------------------------------------- /documentation.mkd: -------------------------------------------------------------------------------- 1 | name: inverse 2 | layout: true 3 | class: center, middle, inverse 4 | 5 | --- 6 | 7 | # Modern code documentation 8 | 9 | ## Radovan Bast 10 | 11 | Licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). 12 | Code examples: [OSI](http://opensource.org)-approved [MIT license](http://opensource.org/licenses/mit-license.html). 13 | 14 | --- 15 | 16 | ## When you look at a code project for the first time, what are the things you look for? 17 | 18 | --- 19 | 20 | layout: false 21 | 22 | ## When you look at a code project for the first time, what are the things you look for? 23 | 24 | - Website is the first thing I see 25 | - Documentation often the second 26 | - If both are repelling, I might get repelled 27 | - Sometimes both are nonexistent 28 | - How do you feel when you see that the documentation has not been updated since 2001? 29 | 30 | --- 31 | 32 | ## Motivation 33 | 34 | ### Project website 35 | 36 | - Project website is important 37 | - Companies without a website cannot exist 38 | - Soon scientific projects without website will not be able to exist (or in other words get grants) 39 | - A website is something that you can show when you apply for jobs 40 | 41 | ### Project documentation 42 | 43 | - Documentation is important 44 | - Writing and reading documentation can be fun 45 | - Documentation does not have to be outdated/incomplete 46 | - If the documentation is notoriously outdated/incomplete, it can mean 47 | that your documentation framework/workflow is not ideal 48 | - It is not always the lazy programmer, sometimes it is the framework 49 | 50 | --- 51 | 52 | template: inverse 53 | 54 | ## Why are so many websites so ugly? 55 | 56 | --- 57 | 58 | ## Why are so many websites so ugly? 59 | 60 | - Many scientific websites look like websites created by scientists in the 90s 61 | - Websites which are not suitable for phones 62 | - Lance Armstrong: "It's not the bike it's the rider." 63 | - Concerning documentation, sometimes it is not the rider, it is the bike 64 | - Some projects simply use better tools 65 | 66 | --- 67 | 68 | ## Technology 69 | 70 | - Do not write own CSS 71 | - Bootstrap http://getbootstrap.com 72 | - Jekyll http://jekyllrb.com 73 | 74 | - Do not maintain own web servers 75 | - GitHub pages https://pages.github.com for project websites 76 | - Bitbucket pages http://pages.bitbucket.org 77 | 78 | --- 79 | 80 | ## Typical GitHub pages workflow 81 | 82 | - Create `gh-pages` branch for your project which holds website sources 83 | - Every push automatically rebuilds http://myorg.github.io/myproject/ [site does not exist] 84 | - It is good practice to use Bootstrap for CSS 85 | - GitHub provides an automatic page generator: you can create a shiny project website in 5 minutes 86 | - See also https://pages.github.com for step-by-step guides 87 | 88 | --- 89 | 90 | template: inverse 91 | 92 | ## Project documentation 93 | 94 | --- 95 | 96 | ## Problems with many code projects 97 | 98 | - Parts of the documentation are outdated/incomplete 99 | - Documentation workflows undocumented 100 | - We typically program first and document later (or never) 101 | - No manual for developers (first steps, call tree, where to start, what to do and what not to do) 102 | - One documentation for different versions (either just confusing or documentation for 103 | older versions is just dropped) 104 | - Have to read a lot of text in different places just to get started 105 | - Keyword-driven (what is possible) instead of tutorial-driven (what is recommended) 106 | - With some projects it is impossible to get a result after 5 minutes of reading 107 | - Some scientific projects take pride in being inaccessible, hard to understand, undocumented, opaque 108 | 109 | --- 110 | 111 | ## README files in the source tree 112 | 113 | - advantage: versionnable (goes with the code development) 114 | - disadvantage: you need a terminal to read it 115 | - better than nothing 116 | 117 | --- 118 | 119 | ## Wikis 120 | 121 | - Dokuwiki https://www.dokuwiki.org 122 | - advantage: barrier to write and edit is low 123 | - disadvantage: difficult to have versions 124 | 125 | --- 126 | 127 | ## LaTeX/PDF 128 | 129 | - Not a lightweight markup 130 | - Barrier to write documentation is relatively high 131 | - PDF format is not ideal for copy-paste ability of examples 132 | - PDF manual is not ideal on small screens (phone) 133 | - Typically not trivial to serve and update the generated PDF 134 | 135 | --- 136 | 137 | ## Doxygen 138 | 139 | - Typically used to autogenerate API documentation 140 | - Documented directly in the source code 141 | - Popular in the C++ community (Doxygen is to C++ what Sphinx is to Python) 142 | - Has support for Fortran and Python 143 | - Many keywords are understood by Doxygen: http://www.stack.nl/~dimitri/doxygen/manual/commands.html 144 | - Can be used to also generate higher-level ("human") documentation 145 | - Can be deployed to GiHub/Bitbucket pages 146 | 147 | --- 148 | 149 | ## RST and Markdown 150 | 151 | - Both very lightweight 152 | - Looks good in the browser 153 | - Looks good in the terminal 154 | - Sphinx http://sphinx-doc.org can generate HTML from RST 155 | - Basically all Python projects use Sphinx 156 | 157 | --- 158 | 159 | ## Specs of a good documentation 160 | 161 | - Close to the code (minimize barrier) 162 | - Versions: If the project has versions, the documentation should too 163 | - Lightweight markup (LaTeX is not lightweight enough) 164 | - Readable on any device 165 | - Division into tutorials and keyword reference 166 | - Tutorials contain good defaults 167 | - Ready examples that one can copy-paste to get quickly started 168 | - Prose 169 | - Written by humans 170 | 171 | --- 172 | 173 | ## Read the Docs 174 | 175 | - https://readthedocs.org 176 | - Free Sphinx hosting 177 | - RST or Markdown 178 | - PDF can be generated on the fly 179 | - Equations and images no problem 180 | - Layout can be changed 181 | - It is no problem to serve from GitHub pages or Read the Docs using your own URL 182 | - Many projects use https://readthedocs.org as their main site 183 | 184 | --- 185 | 186 | ## Typical Read the Docs workflow 187 | 188 | - Host source code with documentation sources on GitHub 189 | - `post-receive` sends POST request to Read the Docs to rebuild the documentation 190 | 191 | ```shell 192 | $ curl -X POST http://readthedocs.org/build/myproject 193 | ``` 194 | 195 | - Read the Docs then fetches changes (typically from GitHub but can be somewhere else provided the repo is public) 196 | and rebuilds HTML and PDF 197 | - No problem to build several branches (versions) of your documentation 198 | 199 | --- 200 | 201 | ## Good practices (1/2) 202 | 203 | - Whatever solution you choose, make it automatically rebuild pages upon `git push` 204 | - If you need to ask somebody to run some scripts on the server after you have modified sources, 205 | that will not work in practice (does not scale) 206 | - Answer questions with an URL to (updated) documentation 207 | - Email answers get lost/forgotten 208 | - Answer and improve doc at the same time 209 | - Long-term lazy 210 | - Chance is that next time the user/colleague 211 | will find the answer by him/herself 212 | - Trains people to read the doc 213 | 214 | --- 215 | 216 | ## Good practices (2/2) 217 | 218 | - Write documentation that you would like to read 219 | - When using code review make sure that new functionality always comes with documentation 220 | - Some people who install and test the software know nothing about the theory behind the code 221 | - Busy user support at computing centers 222 | - Vendors who run benchmarks 223 | - Market success stories (pictures, scaling, molecules, be proud) 224 | -------------------------------------------------------------------------------- /optimiziation.mkd: -------------------------------------------------------------------------------- 1 | name: inverse 2 | layout: true 3 | class: center, middle, inverse 4 | 5 | --- 6 | 7 | # Profiling and code optimization 8 | 9 | ## Radovan Bast 10 | 11 | Licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). 12 | Code examples: [OSI](http://opensource.org)-approved [MIT license](http://opensource.org/licenses/mit-license.html). 13 | 14 | --- 15 | 16 | layout: false 17 | 18 | ## When do we want to optimize the code? 19 | 20 | - Code is too slow 21 | - Memory demand is too high 22 | 23 | --- 24 | 25 | ## When do we want to optimize the code? 26 | 27 | - Code is too slow 28 | - Memory demand is too high 29 | 30 | ### Things we can do about it 31 | 32 | - Buy faster computer 33 | - Buy more memory 34 | - Help the compiler 35 | - Change algorithm 36 | - Change data structures (optimize data access) 37 | - Port to a "faster language" 38 | - Parallelize 39 | - Port to another platform (GPU, accelerators) 40 | 41 | --- 42 | 43 | ## Profile first, optimize later 44 | 45 | - Before you invest money or time, identify the problem 46 | - Here insert famous Donald Knuth quote 47 | - Before solving the problem, verify whether it is worth it 48 | 49 | --- 50 | 51 | ## Profiling 52 | 53 | 54 | 55 | - Benchmark and profile before optimizing 56 | 57 | --- 58 | 59 | ## Profiling 60 | 61 | 62 | 63 | - Where does the program spend most time? 64 | - Dynamic program analysis (runtime) 65 | - Statistical sampling or event-based 66 | - Profiling perturbs the calculation 67 | - Profiled calculation should be representable 68 | 69 | --- 70 | 71 | ## Profilers 72 | 73 | ## Statistical (sampling) 74 | 75 | - Collect data at regular intervals 76 | - Typically small overhead 77 | - Not all code is seen 78 | - May differ between execution 79 | - Show bottlenecks in production code 80 | - Can be used for monitoring over long execution runs 81 | - Reports on external library functions 82 | 83 | ## Event-based (tracing) 84 | 85 | - Collect data (traces) for pre-defined events 86 | (entering or leaving functions, object allocation, communication, disk I/O) 87 | - Modify the code 88 | - Often large overhead 89 | - Selective 90 | 91 | --- 92 | 93 | ## Profilers 94 | 95 | - "Home use", free 96 | - Gprof 97 | - Callgrind (Valgrind) 98 | - Python: cProfile, profile 99 | - OProfile 100 | 101 | - HPC, free 102 | - HPCToolkit 103 | - DynInst 104 | - Extrae/Paraver 105 | - Scalasca 106 | - TAU 107 | 108 | - Commercial 109 | - Vampir 110 | - Allinea Performance Reports 111 | - Allinea MAP 112 | - CrayPat 113 | - VTune Amplifier (Intel) 114 | 115 | --- 116 | 117 | template: inverse 118 | 119 | ## How would you profile stranded on an island? 120 | 121 | --- 122 | 123 | ## How would you profile stranded on an island? 124 | 125 | - Insert timers! 126 | - Everybody can do that 127 | - In any language 128 | - It is good to have timings anyhow in the output 129 | 130 | --- 131 | 132 | ## Before solving the problem, verify whether it is worth it 133 | 134 | - Human time often more valuable than CPU time 135 | - We often spend more time reading and writing code than running it 136 | - Compare your programming time multiplied with your salary to the cost of CPU 137 | time that you will save 138 | - Is a 20% faster code worth 2 months of work? 139 | - Does it really matter whether the run takes 1 minute or 2 minutes 140 | - It probably matters if the computation takes 3 months 141 | - If your code is used by 5000 users then 20\% speedup may be worth it 142 | 143 | --- 144 | 145 | ## Performance vs. code obfuscation 146 | 147 | - When you optimize for speed/memory, typically you increase complexity 148 | - Imagine your code runs 40% faster but nobody understands it 149 | - If you have to complicate the code, keep the complexity localized 150 | - Define your priorities 151 | - Performance is typically not portable 152 | 153 | --- 154 | 155 | ## Complexity complexity complexity 156 | 157 | - We do not know the languages of tomorrow 158 | - We do not know the hardware of tomorrow 159 | - Code complexity may prevent us from porting our codes to soft- and hardware of tomorrow 160 | - Keep it simple as long as you can 161 | 162 | --- 163 | 164 | ## Buy faster computer 165 | 166 | - Moore's law 167 | - Frequency race 168 | - "The party isn't exactly over, but the police has arrived, and the music has been turned way down" (P. Kogge) 169 | - Today we need to think about parallelization 170 | 171 | --- 172 | 173 | ## Buy more memory 174 | 175 | - Number of cores/threads scales faster than memory 176 | - Today we need to think about parallelization 177 | - Linear scaling requires localization of memory access 178 | - Use tools to identify memory bottlenecks 179 | 180 | --- 181 | 182 | ## Help the compiler 183 | 184 | - Restrict types (Cython) 185 | - Use pragmas 186 | - Tell the compiler where the bottlenecks are 187 | - Tell the compiler where vectorization is possible 188 | - Compiler flags 189 | - Instruction set 190 | 191 | --- 192 | 193 | ## Change algorithm 194 | 195 | - Consider increase in complexity 196 | - Pre-factor vs. scaling 197 | - "Slow" algorithm may catch the worm 198 | - Easy algorithm first 199 | - YAGNI 200 | 201 | --- 202 | 203 | ## Change data structures (optimize data access) 204 | 205 | - Latency: https://gist.github.com/jboner/2841832 206 | - Cache hierarchy 207 | - Cache line 208 | - Gorilla 209 | - MPI and scaling (disk access serializing) 210 | - Swapping 211 | 212 | --- 213 | 214 | ## Refactoring 215 | 216 | - Natural process 217 | - "In the long run every program becomes rococo - then rubble." 218 | - http://pu.inf.uni-tuebingen.de/users/klaeren/epigrams.html 219 | - Needs tests 220 | 221 | --- 222 | 223 | ## Recomputing vs. remembering 224 | 225 | - Sometimes recompute faster than reading from disk 226 | - Memoization 227 | 228 | --- 229 | 230 | ## Port to a "faster language" 231 | 232 | - Every language allows to write slow code 233 | - Pre-factor vs. scaling 234 | - "Slow" language may catch the worm 235 | - Combine languages (FFI) 236 | - Functional programming vs. OOP 237 | 238 | --- 239 | 240 | ## Prototype first with "easy" language 241 | 242 | - Matlab 243 | - Python 244 | - Julia 245 | 246 | --- 247 | 248 | ## Parallelize 249 | 250 | - Amdahl's law 251 | 252 | 253 | 254 | - Serial and parallel part scale differently w.r.t. system size 255 | 256 | --- 257 | 258 | ## Typical problems with calculations that do not scale 259 | 260 | - Parallel region too small 261 | - Serialization of parallel tasks 262 | - I/O heavy code 263 | - Communication overhead 264 | - Work load imbalance 265 | - Resource imbalance 266 | - Memory-bound code 267 | - Sub-optimal use of libraries 268 | - System size too small 269 | - Wrong code 270 | 271 | --- 272 | 273 | ## Port to another platform (GPU, accelerators) 274 | 275 | - Moving target 276 | - Trend goes towards multi-processor and multi-thread 277 | - If you rewrite your data structures for GPU/accelerators, the effort 278 | is most probably not wasted 279 | 280 | ### Before porting 281 | 282 | - Restructure mathematical formulation 283 | - Innovate at algorithm level 284 | - Then maybe port 285 | 286 | --- 287 | 288 | ## Do not reinvent the wheel 289 | 290 | - Use libraries if you can 291 | - Libraries are dependencies 292 | 293 | --- 294 | 295 | ## General 296 | 297 | - Use elemental functions 298 | - Use pure functions/subroutines (Fortran) 299 | - Use intrinsics 300 | - Beware of multiple loops 301 | - Do not allocating/deallocate inside multiple loops 302 | 303 | --- 304 | 305 | ## Example: profiling performance and memory in Python 306 | 307 | - https://gist.github.com/bast/e37ee29036d4f43fee4d 308 | - https://gist.github.com/bast/1bd41989a975a999d653 309 | 310 | --- 311 | 312 | ## Conclusions 313 | 314 | - Profile first, optimize later 315 | - When you scale the system, new bottlenecks will appear 316 | -------------------------------------------------------------------------------- /git-archaeology.mkd: -------------------------------------------------------------------------------- 1 | name: inverse 2 | layout: true 3 | class: center, middle, inverse 4 | 5 | --- 6 | 7 | # Archaeology with Git 8 | 9 | ## Radovan Bast 10 | 11 | Licensed under [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/). 12 | Code examples: [OSI](http://opensource.org)-approved [MIT license](http://opensource.org/licenses/mit-license.html). 13 | 14 | --- 15 | 16 | layout: false 17 | 18 | ## Archaeology with Git 19 | 20 | - Inspecting commits 21 | - Finding out who introduced a modification 22 | - Finding out when a modification has been introduced 23 | - Grepping code 24 | - Finding removed code 25 | - Finding a developer to talk to (large projects) 26 | - Going back in time 27 | - Finding out when something broke 28 | - Finding commits not merged upstream 29 | 30 | --- 31 | 32 | ## Inspecting commits 33 | 34 | - At any moment we can inspect individual commits 35 | 36 | ```shell 37 | $ git show 49dc419 38 | 39 | commit 49dc419c8a44051cfe7826b85ee0a23e5faf3975 40 | Author: Radovan Bast 41 | Date: Sat Nov 22 15:33:15 2014 +0100 42 | 43 | do not recompute powers 44 | 45 | diff --git a/triangle.py b/triangle.py 46 | index cc52fe2..fa35eab 100644 47 | --- a/triangle.py 48 | +++ b/triangle.py 49 | @@ -6,7 +6,10 @@ m = int(sys.argv[1]) 50 | 51 | # loop over all a < b < c <= m 52 | for c in xrange(1, m+1): 53 | + cp = c*c 54 | for b in xrange(1, c): 55 | + bp = b*b 56 | for a in xrange(1, b): 57 | - if a*a + b*b == c*c: 58 | + ap = a*a 59 | + if ap + bp == cp: 60 | print("(%i, %i, %i)" % (a, b, c)) 61 | ``` 62 | 63 | - We see that the start of the hash is enough if it is unique 64 | 65 | --- 66 | 67 | ## Finding out who introduced a modification and when 68 | 69 | - `git blame` is a fun and useful command to find out when a specific line got introduced and by whom 70 | - This is important to judge the impact of a bug (When was it introduced? For how long has this bug been around? Has this bug been released?) 71 | 72 | ```shell 73 | $ git blame 74 | 75 | faa4617a (Radovan Bast 2012-07-02 11:04:45 +0200) ! get the one electron Hamiltonian 76 | e564cfe8 (A. J. Thorvaldsen 2012-12-25 00:59:21 +0100) H1 = 0*S 77 | e564cfe8 (A. J. Thorvaldsen 2012-12-25 00:59:21 +0100) call mat_ensure_alloc(H1, only_alloc=.true.) 78 | faa4617a (Radovan Bast 2012-07-02 11:04:45 +0200) call interface_scf_get_h1(H1) 79 | faa4617a (Radovan Bast 2012-07-02 11:04:45 +0200) 80 | e6cfa2cf (Radovan Bast 2012-03-01 10:06:39 +0100) ! get the two electron contribution (G) to Fock matrix 81 | e564cfe8 (A. J. Thorvaldsen 2012-12-25 00:59:21 +0100) G = 0*S 82 | e564cfe8 (A. J. Thorvaldsen 2012-12-25 00:59:21 +0100) call mat_ensure_alloc(G, only_alloc=.true.) 83 | 810e14d8 (Radovan Bast 2012-07-01 17:19:56 +0200) call interface_scf_get_g(D, G) 84 | 761d5c27 (Radovan Bast 2012-05-18 16:28:34 +0200) 85 | e6cfa2cf (Radovan Bast 2012-03-01 10:06:39 +0100) ! Fock matrix F = H1 + G 86 | 761d5c27 (Radovan Bast 2012-05-18 16:28:34 +0200) F = H1 + G 87 | ``` 88 | 89 | - "Who the %&!@!!! wrote this?!?" 90 | - In 90% of cases you yourself 91 | - It is useful for projects with 20+ people where you do not have complete overview 92 | 93 | --- 94 | 95 | ## Grepping code 96 | 97 | ```shell 98 | $ git grep fixme 99 | 100 | densfit/df_vxc.F:!fixme call XC integrator 101 | dirac/dirgrd.F:!fixme: there is some issue with one-step - investigate later 102 | dirac/dirrdn.F:! fixme num grid report should be independent of DFT 103 | dirac/dirscf.F: !fixme: michal for InteRest 104 | dirac/dirscf.F: !fixme: michal for InteRest 105 | dirac/dirscf.F: !fixme: michal for InteRest 106 | embedding/get_vemb_mat.F:!fixme irep can be < 0 in NONREL runs 107 | eri/eri2out.F:!fixme 108 | eri/eri2out.F:!fixme 109 | eri/eri2out.F:!fixme 110 | gp/gpkrmc.F:C fixme: dette ser ikke ud til at virke for complex groups. 111 | interest/module_interest_hrr.f90: !-- fixme --! 112 | interest/module_interest_interface.F90: !> fixme: contracted basis sets 113 | ``` 114 | 115 | - Greps entire repository below current directory 116 | - Super fast 117 | 118 | --- 119 | 120 | ## Finding removed code 121 | 122 | - There used to be a line containing "break" 123 | - Now it is gone 124 | - We can figure out when it disappeared 125 | 126 | ```shell 127 | $ git log -c -S'break' 128 | 129 | commit 41997dffb4c5dd47c81adb1fd1b616e72415e6e5 130 | Author: Radovan Bast 131 | Date: Sun Nov 23 22:44:31 2014 +0100 132 | 133 | remove line 134 | 135 | commit bf39f9d3f2b2867e1c66d58d3c4a32842c7f80d5 136 | Author: Radovan Bast 137 | Date: Sat Nov 22 15:33:37 2014 +0100 138 | 139 | break loop a if triple found 140 | ``` 141 | 142 | --- 143 | 144 | ## Finding removed code 145 | 146 | ```shell 147 | $ git show 41997df 148 | 149 | commit 41997dffb4c5dd47c81adb1fd1b616e72415e6e5 150 | Author: Radovan Bast 151 | Date: Sun Nov 23 22:44:31 2014 +0100 152 | 153 | remove line 154 | 155 | diff --git a/triangle.py b/triangle.py 156 | index c693105..fa35eab 100644 157 | --- a/triangle.py 158 | +++ b/triangle.py 159 | @@ -13,4 +13,3 @@ for c in xrange(1, m+1): 160 | ap = a*a 161 | if ap + bp == cp: 162 | print("(%i, %i, %i)" % (a, b, c)) 163 | - break # no need to continue loop a 164 | ``` 165 | 166 | --- 167 | 168 | ## Finding a developer to talk to (large projects) 169 | 170 | - Example: you want to know who implemented a specific keyword/functionality 171 | - `git grep` for the keyword 172 | - Then with `git blame` find the person who introduced it 173 | - With `git show` check the commit that introduced the change (it could be a file rename done by someone else) 174 | - In the latter case check out a version prior to the move and `git grep` and `git blame` there 175 | 176 | --- 177 | 178 | ## Going back in time 179 | 180 | - We do not have to branch from the tip of the branch (HEAD) 181 | - We can branch from arbitrary (earlier) hash 182 | 183 | ```shell 184 | $ git checkout -b 185 | ``` 186 | 187 | - This is the recommended mechanism to inspect old code (hash abc123): 188 | 189 | ```shell 190 | $ git checkout -b museum abc123 # create branch called "museum" from hash abc123 191 | # do some archaeology ... 192 | # "what was I thinking back then!?" 193 | $ git checkout master # after you are done switch back to "master" 194 | $ git branch -d museum 195 | ``` 196 | 197 | - Branches help us to keep the repository organized 198 | 199 | --- 200 | 201 | ## Finding out when something broke 202 | 203 | - Sometimes you realize that something broke 204 | - You know that it used to work 205 | - You do not know when it broke 206 | - First find out a commit in past when it worked 207 | - Then bisect your way until you find the commit that broke it 208 | 209 | ```shell 210 | $ git bisect start 211 | $ git bisect good 75c737cf5c9 # this is a commit that worked 212 | $ git bisect bad HEAD # last commit is broken 213 | # now compile and test 214 | # after that decide whether 215 | $ git bisect good 216 | # or 217 | $ git bisect bad 218 | # now compile and test 219 | # after that decide whether 220 | $ git bisect good 221 | # or 222 | $ git bisect bad 223 | # iterate until commit is found 224 | ``` 225 | 226 | - This can even be automatized with `git bisect run