├── mloska ├── .gitgnore ├── mliska │ └── app.py └── dev │ ├── 2_first_bug.py │ ├── 1_first_feature.ipynb │ └── 3_not_a_bug_yay.md ├── .gitignore ├── isthisnormcore.jpg ├── petit-poilu-matrioska.png ├── what-is-a-git-workflow.png ├── dev.md ├── LICENSE ├── README.md ├── index.nim └── index.html /mloska/.gitgnore: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /.gitignore: -------------------------------------------------------------------------------- 1 | .DS_Store -------------------------------------------------------------------------------- /mloska/mliska/app.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /mloska/dev/2_first_bug.py: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /mloska/dev/1_first_feature.ipynb: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /mloska/dev/3_not_a_bug_yay.md: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /isthisnormcore.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietroppeter/git-matrioska/main/isthisnormcore.jpg -------------------------------------------------------------------------------- /petit-poilu-matrioska.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietroppeter/git-matrioska/main/petit-poilu-matrioska.png -------------------------------------------------------------------------------- /what-is-a-git-workflow.png: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/pietroppeter/git-matrioska/main/what-is-a-git-workflow.png -------------------------------------------------------------------------------- /dev.md: -------------------------------------------------------------------------------- 1 | - [x] init git-matrioska repo and readme 2 | - [x] nimislides setup 3 | - [x] title slide 4 | - [x] matrioska slide 5 | - [x] preamble 6 | - [x] the problem 7 | - [x] what's a git workflow? 8 | (example git flow) 9 | - [x] solution is not git submodule 10 | - [x] the idea/solution 11 | - [x] folderStructure 12 | -------------------------------------------------------------------------------- /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2023 Pietro Peterlongo 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # git-matrioska 2 | 3 | A normcore git workflow to maintain a production (ML) repo. 4 | 5 | Only exists as a quickly whipped up set of slides: https://pietroppeter.github.io/git-matrioska/ 6 | 7 | This was my first lightning talk. At my first event where this kind of things happen. 8 | It happened at PyData Amsterdam 2023. It was... interesting. 9 | 10 | The idea (git-matrioska) is probably a bad idea or it just applies to a very small use case and 11 | I cannot really put my finger on why it is weird. 12 | But I did use this in my small team for some time and it seemed to actually work well. 13 | 14 | An even worse idea was, on the morning of the first day of the event, to decide to give a lightning talk about this possibly bad idea 15 | with the idea that, well, so people will tell me why it is bad and I will have learnt about it. 16 | 17 | Only I had very little time to prepare the slides (thanks nimislides to the rescue I did prepare them while _not really_ following the talks!), 18 | and at the end of the day with my brain almost fried by all the great content I had seen and my energies depleted by enjoying talking with a lot of great people. 19 | 20 | So I walked on stage in front of a few hundreds people (attendes were reported at 800 but hopefully not all were there), 21 | I mumble an excuse for my poor state and manage to have to rush the final slides (where I actually talk about the solution) 22 | because I did not have time anymore (hey, I timed it at 3mins when I rehearsed it...). 23 | 24 | It did seem to provide some kind of entertainment (if not technical content) and an additional chance for talking to people, which were very kind to me afterwards (my guess is they were thinking something along the lines of "_this poor lad, he clearly is confused but he does not seem a bad person, so let's just smile at him and let's hope he does not try to start a conversation!_"). 25 | 26 | All in all, although being very stressful, I will probably do it again (mmh... maybe). 27 | 28 | At some point I might improve on the slides and give a better (and longer) presentation. 29 | -------------------------------------------------------------------------------- /index.nim: -------------------------------------------------------------------------------- 1 | import nimib, nimislides 2 | 3 | template slideTitle = 4 | slide: 5 | nbText: "# `git-matrioska`" 6 | nbText: "a normcore git workflow" 7 | nbText: "for maintaining production (ML) repos" 8 | nbText: " " 9 | nbText: "[pietroppeter](https://github.com/pietroppeter)" 10 | 11 | template slideMatrioska = 12 | slide: 13 | # from https://www.petitpoilu.com/albums/amour-glace 14 | nbImage("petit-poilu-matrioska.png") 15 | 16 | template slidePreamble = 17 | slide: 18 | nbText: "## preamble" 19 | unorderedList: 20 | listItem: nbText: "I work in Supply Chain B2B company" 21 | listItem: nbText: "maintaining a production ML repo for past 7 years" 22 | listItem: nbText: "**problem**: keeping a clean production repo..." 23 | listItem: nbText: "...while documenting (in a repo!) our maintenance" 24 | # git matrioska is my solution 25 | 26 | template slideIsThisNormcore = 27 | slide: 28 | nbImage("isthisnormcore.jpg") 29 | # interested in why you think this might be a bad idea 30 | 31 | template slideWhatIsGitWorkflow = 32 | slide: 33 | nbImage("what-is-a-git-workflow.png") 34 | 35 | template slideNotSubmodule = 36 | slide: 37 | nbText: "> a repo inside a repo, the solution is..." 38 | fragment: 39 | fragment(strike): 40 | nbText: "`git submodule`" 41 | 42 | template slideIdea = 43 | slide: 44 | nbText: "## the idea" 45 | orderedList: 46 | listItem: nbText: "start with a production (ML) repo (e.g. `mliska`)" 47 | listItem: nbText: "_feel_ the problem (clean prod & track changes)" 48 | listItem: nbText: "create `mloska`" 49 | listItem: nbText: "checkout `mliska` inside `mloska`" 50 | listItem: nbText: "add `mliska` to `.gitignore`" 51 | 52 | template slideFolderStructure = 53 | slide: 54 | # https://ascii-tree-generator.com 55 | nbText: """## Example folder structure 56 | ``` 57 | mloska/ 58 | ├─ .git/ 59 | ├─ dev/ 60 | │ ├─ 1_first_feature.ipynb 61 | │ ├─ 2_first_bug.py 62 | │ ├─ 3_not_a_bug_yay.md 63 | ├─ mliska/ 64 | │ ├─ .git/ 65 | │ ├─ app.py 66 | ├─ .gitignore 67 | ``` 68 | """ 69 | 70 | when isMainModule: 71 | nbInit(theme = revealTheme) 72 | setSlidesTheme(White) 73 | slideTitle 74 | slideMatrioska 75 | slidePreamble 76 | slideIsThisNormcore 77 | slideWhatIsGitWorkflow 78 | slideNotSubmodule 79 | slideIdea 80 | slideFolderStructure 81 | nbSave -------------------------------------------------------------------------------- /index.html: -------------------------------------------------------------------------------- 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
11 |
12 |
13 |

git-matrioska

14 |

a normcore git workflow

15 |

for maintaining production (ML) repos

16 | 17 |

pietroppeter

18 |
19 |
20 |
21 | 22 |
23 |
24 |
25 |
26 |

preamble

27 |
    28 |
    29 | 30 |
  • 31 |

    I work in Supply Chain B2B company

    32 |
    • 33 |
    34 | 35 |
    36 | 37 |
  • 38 |

    maintaining a production ML repo for past 7 years

    39 |
    • 40 |
    41 | 42 |
    43 | 44 |
  • 45 |

    problem: keeping a clean production repo...

    46 |
    • 47 |
    48 | 49 |
    50 | 51 |
  • 52 |

    ...while documenting (in a repo!) our maintenance

    53 |
    • 54 |
    55 | 56 |
57 |
58 |
59 |
60 | 61 |
62 |
63 |
64 |
65 |
66 | 67 |
68 |
69 |
70 |
71 |
72 |

a repo inside a repo, the solution is...

73 |
74 |
75 | 76 |
77 | 78 |

git submodule

79 |
80 | 81 |
82 | 83 |
84 |
85 |

the idea

86 |
    87 |
    88 | 89 |
  1. 90 |

    start with a production (ML) repo (e.g. mliska)

    91 |
    2. 92 |
    93 | 94 |
    95 | 96 |
  2. 97 |

    feel the problem (clean prod & track changes)

    98 |
    3. 99 |
    100 | 101 |
    102 | 103 |
  3. 104 |

    create mloska

    105 |
    4. 106 |
    107 | 108 |
    109 | 110 |
  4. 111 |

    checkout mliska inside mloska

    112 |
    5. 113 |
    114 | 115 |
    116 | 117 |
  5. 118 |

    add mliska to .gitignore

    119 |
    6. 120 |
    121 | 122 |
123 |
124 |
125 |

Example folder structure

126 | 
mloska/
127 | ├─ .git/
128 | ├─ dev/
129 | │  ├─ 1_first_feature.ipynb
130 | │  ├─ 2_first_bug.py
131 | │  ├─ 3_not_a_bug_yay.md
132 | ├─ mliska/
133 | │  ├─ .git/
134 | │  ├─ app.py
135 | ├─ .gitignore
136 |
137 |
138 |
139 |
140 | 141 | 142 | 143 | 151 | 152 | 153 | --------------------------------------------------------------------------------