├── tiller_design_diagramm.jpg ├── code-of-conduct.md ├── Table3_Round6_Better_ways_to_curate_charts.md ├── README.md ├── helmtest.md ├── Table3_Round4_Extensions_of_existing_charts..md ├── helmplugins.md ├── Table3_Round5_Tiller_storage_backends.md ├── consistent-construction-of-charts.md ├── Table1_Round3_RBAC_friendly_Tiller.md ├── Table3_Round1_Template_Engine.md ├── helm-summit-jobs.md ├── Round1_Table2_Future_of_Charts.md └── Round3_Table1_Tiller_and_CRDs.md /tiller_design_diagramm.jpg: -------------------------------------------------------------------------------- https://raw.githubusercontent.com/helm/helm-summit-notes/HEAD/tiller_design_diagramm.jpg -------------------------------------------------------------------------------- /code-of-conduct.md: -------------------------------------------------------------------------------- 1 | # Community Code of Conduct 2 | 3 | Helm follows the [CNCF Code of Conduct](https://github.com/cncf/foundation/blob/master/code-of-conduct.md). 4 | -------------------------------------------------------------------------------- /Table3_Round6_Better_ways_to_curate_charts.md: -------------------------------------------------------------------------------- 1 | # New, Better ways to curate charts, and link back to back to owners of chart. 2 | 3 | Table 3, Round 6. 4 | 5 | ## Raised issues: 6 | 7 | 1. Maintainers are not able to update their charts fast enough 8 | 2. Kubernetes org with Prow - Aaron C said they they are happy to add people to the k8s github org 9 | 3. Contact information from ownership 10 | 4. Members or collaborators? 11 | 5. How do you find owners? 12 | 6. Chef marketplace model 13 | 14 | ## Proposals: 15 | 16 | 1. Create PRs for all charts to add committers as OWNER - @lachie83 17 | 2. Create chart adoption process 18 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | # helm-summit-notes 2 | Helm Summit notes from February 2018 3 | 4 | Each round is 30 minutes. If you took notes, please at a .md to this repo 5 | 6 | | Round | Sessions Titles | 7 | |-------|-----------------| 8 | | 1 | [Helm Plugins](helmplugins.md), [Future of Chart Repos](Round1_Table2_Future_of_Charts.md), [Templating Support Group](Table3_Round1_Template_Engine.md)| 9 | | 2 | [Helm Test](helmtest.md) | 10 | | 3 | [Tiller and CRDs](Round3_Table1_Tiller_and_CRDs.md) | 11 | | 4 | [Extensions of Existing Charts](Table3_Round4_Extensions_of_existing_charts.md) | 12 | | 5 | [Tiller Storage Backends](Table3_Round5_Tiller_storage_backends.md) | 13 | | 6 | [Better Ways to Curate Charts](Table3_Round6_Better_ways_to_curate_charts.md) | 14 | 15 | If you're looking for a new gig, see our [job board](helm-summit-jobs.md) from Helm Summit. 16 | -------------------------------------------------------------------------------- /helmtest.md: -------------------------------------------------------------------------------- 1 | # helm test working session 2 | 3 | - test directory with multiple values files 4 | - testing of image, testing of chart 5 | - on installation all resources can come up 6 | - what if you want to run helm test on production 7 | - do we want different test scenarios where you plugin different values? 8 | - out of bounds usecase: cluster audit, all security requirements are up to date 9 | - what do we want to persist? do we want structured output, be able to trace failures? 10 | - plugin for acceptance testing via a Chart? 11 | - chart validation should be separated from application validation 12 | - need for DSL to validate different values 13 | - need to figure out which personas need testing 14 | - compile-time testing, static analysis testing, dynamic testing 15 | 16 | Personas: 17 | 1. Does this create valid K8s object? 18 | 19 | 2. Helm end user 20 | 21 | 3. Helm Chart developer 22 | 23 | 24 | Action Items 25 | - create #helm-test slack channel 26 | - develop test personas 27 | -------------------------------------------------------------------------------- /Table3_Round4_Extensions_of_existing_charts..md: -------------------------------------------------------------------------------- 1 | # helm-summit-notes 2 | Helm Summit notes from February 2018 3 | Table 3, Round 4. 4 | Extensions of existing charts. 5 | 6 | ## Raised issues: 7 | 8 | 1. How to make changes into existing charts. Without forking and modifying charts repo. 9 | ⋅⋅* Add new ladles, annotations, values 10 | ..* Add/remove side car[s] 11 | 12 | ## Proposals: 13 | 14 | @omkensey mentioned that this topic was discussed in one of the previous tables (improving experince with charts) and proposal was to start using overlay. 15 | @technosophos said that Helm is supporting overlaying from the begging. 16 | 17 | Discussion log: 18 | ``` 19 | modify charts without breaking it. 20 | add, remove, delete and things. 21 | customizing charts with overlay 22 | proposal from another group: improving experince with charts. 23 | flags, apis. 24 | mostly when k8s changes it takes forever to populate those into repo. 25 | 26 | from chart aka docker file 27 | 28 | use ksonnet as backend. 29 | 30 | add external deps example: once chart is installed helm performs actions from annotation. 31 | post/get query URLs. something like webhooks 32 | ``` 33 | -------------------------------------------------------------------------------- /helmplugins.md: -------------------------------------------------------------------------------- 1 | # Helm Plugins working group 2 | 3 | 4 | Level set: 5 | --- 6 | 7 | Plugins should not be opinionated 8 | Plugins should be generic as possible 9 | 10 | Creating new flow (for Adam): 11 | - Copy old plugin, changing values, profit 12 | 13 | plugin.yaml contains: 14 | - metadata 15 | - hooks 16 | - definition 17 | 18 | Plugins are essentially aliases to longer commands 19 | Plugin hooks to directories 20 | 21 | `helm install` will fetch remote, run any hooks, link everything in cache directly to $HELM_HOME 22 | 23 | `helm plugin install` for local development pointing at directories 24 | 25 | When plugin is executed, sets up all environment variables needed 26 | 27 | Tiller-side plugins are also possible: 28 | - [rudder](https://github.com/helm/rudder-appcontroller) 29 | 30 | 31 | ## Curating Helm Plugins 32 | 33 | Currently, community asks you to 34 | - add the tag `helm-plugin` to your project in Github 35 | - submit a PR to [related section](https://github.com/kubernetes/helm/blob/master/docs/related.md#helm-plugins) 36 | 37 | 38 | Proposed items: 39 | - helm create for base template of plugins 40 | - hub.kubeapps.com but for helm plugins 41 | -------------------------------------------------------------------------------- /Table3_Round5_Tiller_storage_backends.md: -------------------------------------------------------------------------------- 1 | # helm-summit-notes 2 | Helm Summit notes from February 2018 3 | Table 3, Round 5. 4 | Tiller storage backends. 5 | 6 | ## Raised issues: 7 | 8 | 1. Use external state storage backend for Tiller. 9 | 2. CRs size is limited by 1MB (etcd limitations applied to all k8s objects) 10 | 11 | 12 | ## Proposals: 13 | 14 | 1. Helm 2. There is already a couple of projects to use Postgres, CosmosDB, and Mysql 15 | 16 | 2. Split current Tiller state into multiple CRs: Values, Chart, Release etc 17 | ..* Optimization: Modify Tiller to change/create only new CRs and reuse unchanged. 18 | 3. Store refs to external resources in CRs. 19 | ..* Possible issues with managing access info for external backends. 20 | 21 | !["Tiller design diagramm"]("Tiller design diagramm" "Tiller design diagramm") 22 | 23 | Discussion log: 24 | ``` 25 | for 2.8 there is a way of storing in DB. 26 | helm3 (aka crd controller) possible issues is crd size is 1MB max (k8s object limit ) 27 | proposal to store ref to external storage in crd. 28 | issues with access. 29 | helm scheduler (k8s-helm). 30 | reddis or Memcache for tiller 31 | multiple CRDs from tiller. different Kinds. instead of one big one. 32 | 33 | reuse of crds like compare if chart changed. 34 | v1 create all. update changes only new things and rec 35 | ``` 36 | -------------------------------------------------------------------------------- /consistent-construction-of-charts.md: -------------------------------------------------------------------------------- 1 | # Consistent Construction of Charts 2 | 3 | - Adding best practices to common chart 4 | - Can end up being its own programming languague 5 | - One user had 200 pg manual on their common chart 6 | - Is it better to do generators? 7 | - Ask folks some questions and generate the chart 8 | - JSON schema 9 | - There are challenges with validating that things are the right values. For example non-zero int 10 | - Tooling like lint extensions 11 | - Something that tests states of the manifests after rendering 12 | - Similar to [kube-lint](https://github.com/viglesiasce/kube-lint) 13 | - There is an issue open on the Helm repo to add more to the linter there 14 | - There are things that Helm doesnt support from YAML 15 | - No anchors 16 | - YQ is used to get values out of a YAML 17 | - How we check version increments 18 | - There is a proposal for overlays 19 | - kfox opened this 20 | - Can you have a overrides folder to take over resources from dependent charts 21 | - Currently in Charts repo we do conditional deployment of secrets/configmaps based on a value 22 | - customConfigMap 23 | - One customer has 200 charts and using style guide to make sure people do the right thing 24 | - Stacknetes approach 25 | - Internal linters were useful 26 | - Before and after rendering 27 | - Charts get developed ad hoc, then the best practices applied later 28 | - Should there be more flexibility in the helm lint 29 | - Maintainers are looking to add configs for YAML linting 30 | - Templating can be hard for users that don’t know Go template or Go 31 | - Do dev teams want to write charts 32 | - Yes at the beginning 33 | - After a while its too complicated to handle 34 | - Have to think about a lot of kubernetes "things" 35 | - RBAC 36 | - Network policy 37 | -------------------------------------------------------------------------------- /Table1_Round3_RBAC_friendly_Tiller.md: -------------------------------------------------------------------------------- 1 | # helm-summit-notes 2 | Helm Summit notes from February 2018 3 | Table 1, Round 3. 4 | RBAC friendly Tiller 5 | 6 | ## Raised issues: 7 | 8 | 1. Tiller requires admin permission (GOD MODE) for operations. 9 | 2. With access to Tiller, user can user Tiller's permissions, not user's own set permissions 10 | 3. There is no clear way of access/object creation auditing. 11 | 12 | ## Proposals: 13 | 14 | 1. Start using Impersonation API in Tiller. https://kubernetes.io/docs/admin/authentication/#user-impersonation 15 | Discussion log: 16 | ``` 17 | SA creation and assigning permissions 18 | tiller uses provided SA for app deployment 19 | I'm user X and go use that SA account. 20 | Impersonation API. whats missing and can it be used by tiller. there is no way of checking who was who and see what was created. 21 | Impersonation api is not stable enough. 22 | ``` 23 | 24 | 2. Pass rendered YAMLs back to the client and use kubctl to create actual objects. 25 | Discussion log: 26 | ``` 27 | passing back from tiller and run kubectl 28 | helm has to have credentials. 29 | pass credentials to use and check if allowed to use 30 | ``` 31 | 32 | 3. Tiller as CRD controller. No direct iteration with Tiller. 33 | If a user can create CRDs this automatically will allow object creation in that namespace. 34 | Discussion log: 35 | ``` 36 | tiller is controller watch CRDs and then do it. 37 | policies in tiller, check what about t create. 38 | not allowed to specify SA which elevates permissions. 39 | 2 type charts: local and global. 40 | helm release and cluster-helm-release 41 | controller opting-in its permissions. 42 | can secure only 1 namespaces 43 | ``` 44 | 45 | 4. Tiller as API server 46 | Discussion log: 47 | ``` 48 | admission controller checking names who created and who trying to do. 49 | tiller as admission controller. 50 | CR with the field of impersonation a user. 51 | API server (custom) aggregating API you need to use your own etcd. 52 | code is there for crd controller and api server. in bitnami github 53 | ``` 54 | 55 | P.S. @anguslees has working version of Tiller CRD controller and almost ready for API server 56 | -------------------------------------------------------------------------------- /Table3_Round1_Template_Engine.md: -------------------------------------------------------------------------------- 1 | # Template Engine 2 | 3 | - Factoring out the template engine 4 | 5 | - Dynamically overriding charts that are not templated 6 | 7 | ### Overriding templates 8 | 9 | The idea is to override arbitrary bits of YAML 10 | 11 | Q: Does that introduce a security issue? 12 | 13 | - No? 14 | 15 | Idea: Overriding the templates like erb — allow a template override. E.g. specify a path to override. 16 | 17 | - Merging problem when the template is not structured well 18 | 19 | - Brian L: Using KSonnet instead. 20 | 21 | - Generate a diff of one template against another 22 | 23 | - Brian G: There’s always something to override, so templating is not as good of an assumption as forking Also using overlays. Merging is done after the rendering has been done. 24 | 25 | - This requires additional tooling to detect errors as overlays change 26 | 27 | - Applying patches and the diff outcome is the main important artifact 28 | 29 | - Version of chart the patch applies to... and wheter it is overriding something new 30 | 31 | Q: What about templatizing values files? 32 | 33 | Some discussion about sealing values (making them unoverridable without some additional flag). 34 | 35 | Q: Do people want to fork packages? 36 | 37 | - “I don’t ever want to” 38 | 39 | - “In Chef it was a massive rebase problem and maintenance problem” 40 | 41 | - Parameterize over forking 42 | 43 | - “We have to fork every chart to add things like node selectors... but we don’t want to” 44 | 45 | - “Overlays might allow a fast path to upstream” 46 | 47 | - “The comparison is incorrect” 48 | 49 | - “If I as a user takes a standard config, and I override, I know I am taking responsibility” 50 | 51 | - “In the CM world, wrapping a good enough version is the main practice” 52 | 53 | - “Often we want to prune down what is configurable” 54 | 55 | Proposal: Overlay with a JSON or YAML patch that is applied after the render. 56 | 57 | - “It’s a version 0.5 of the idea, but needs cleaner tooling” 58 | 59 | Proposal: Make it possible to do other template engines 60 | 61 | **Back to template solutions:** 62 | 63 | “Is there value in giving hte user choice between pre-render/post-render?” 64 | 65 | A pre-render means you can plug in your own values 66 | 67 | Proposal: Hooks in template lifecycle hooks pre-/post-template 68 | -------------------------------------------------------------------------------- /helm-summit-jobs.md: -------------------------------------------------------------------------------- 1 | # Helm Summit Hiring Board 2 | 3 | Are you hiring? Open a PR to add opportunities open your company. 4 | 5 | ### Microsoft 6 | The Azure Container Service team (ACS) is hiring for three (3!) roles: 7 | #### AKS Engineering (Azure Managed Kubernetes) 8 | Join the team that builds and runs Azure's managed Kubernetes service. 9 | - Location: Must be in or near Boulder, CO to be considered. 10 | - Level: Open to mid-level and senior engineers. 11 | - Apply: Apply [here](https://careers.microsoft.com/jobdetails.aspx?ss=&pg=0&so=&rw=7&jid=348046&jlang=EN&pp=SS) 12 | 13 | #### Azure's upstream Kubernetes team 14 | We're looking for senior engineers experienced with Go to join our OSS team dedicated to making Azure the best 15 | place to run Kubernetes by working on the Kubernetes project itself. 16 | - Location: Redmond or remote candidates will be considered. 17 | - Level: Open to mid-level and senior engineers. 18 | - Apply: Apply [here](https://careers.microsoft.com/jobdetails.aspx?ss=&pg=0&so=&rw=3&jid=348192&jlang=EN&pp=SS) 19 | 20 | #### ACS Lean Startup Team 21 | We're building a new pilot engineering team at Microsoft San Francisco setting to prove that working in a more 22 | modern way can be done within the walls of Microsoft’s cloud team and build an esprit de corps using lean, 23 | design thinking, and agile methodologies. 24 | - Location: SF Bay Area candidates preferred. Candidates in other areas will be considered if willing to work on-site at Microsoft 25 | SF for three months. 26 | - Level: Open to mid-level and senior engineers. 27 | - Apply: Apply [here](https://careers.microsoft.com/jobdetails.aspx?ss=&pg=0&so=&rw=1&jid=347392&jlang=EN&pp=SS) 28 | 29 | ### Codefresh 30 | The DevOps Platform built for Containers, Kubernetes, and Helm is hiring: 31 | #### Developer Evangelist 32 | Do you like code and talk? Apply [here](https://codefresh.io/developer-evangelist/) 33 | 34 | ### JFrog 35 | The database of DevOps supporting transformations from traditional to cloud native 36 | #### Developer Advocate 37 | Want to travel the world helping leading organizations make the leap? 38 | - Location: Sunnyvale, CA 39 | - Apply: Read the job description [here](https://join.jfrog.com/job/?job=847935) and email your cv to craigp @ jfrog.com 40 | #### Senior Solutions Engineer 41 | Build cloud native solutions with the top technology partners 42 | - Location: Sunnyvale, CA 43 | - Apply: Read the job description [here](https://join.jfrog.com/job/?job=847935) and email your cv to craigp @ jfrog.com 44 | -------------------------------------------------------------------------------- /Round1_Table2_Future_of_Charts.md: -------------------------------------------------------------------------------- 1 |
2 |

Table of Contents

3 |
4 | 18 |
19 |
20 | 21 | 22 | 23 | # Future of Helm Charts 24 | 25 | 26 | 27 | 28 | ## Topic 1: Should repositories like ChartMuseum signal quality? 29 | 30 | For users, it was agreed that it is useful to have some sort of signalling mechanism for quality. There were various proposals made to signal quality of charts – each had its benefits and drawbacks. 31 | 32 | 33 | 34 | 35 | ### Ratings 36 | 37 | - There are certain problems that arise with using ratings as a quality signalling mechanism 38 | - The system can be gamed, folks can ask friends / colleagues to rate things, often means nothing about the actual quality. 39 | - Ratings don't work well over a period of time. For eg. if there is a chart with bad ratings, that gets fixed, the bad ratings still persist. Idea - rate every version seperately. 40 | - Maybe something along the style of Amazon reviews is better than ratings? But then again, this can be gamed – see fake Amazon reviews. 41 | 42 | 43 | 44 | 45 | ### Number of Downloads 46 | 47 | - How do you know that downloads have been by an actual user vs. something like a CI system? 48 | - kubeapps hub pulls all of the charts – how do you distinguish that from a real person? 49 | - Any information can be gamed – however if you have a million downloads / docker pulls, there is definitely a message there – people are finding the chart / image useful! 50 | - Recently Helm added user agent flags – maybe we can use something like this to differentiate from CI? This is worth a follow up. 51 | 52 | However, with either of these two options, where is the correct place to store any such info about downloads? Does this belong in the chart repository? 53 | 54 | - Big orgs prefer to use on-premise tools. They need some way of sharing knowledge. Would they need something like this? Might be overkill for them though. 55 | 56 | 57 | 58 | 59 | ## Topic 2: Changing the format of the chart index to be more performant . 60 | 61 | - Upstream issue filed for this – Issue 3557: 62 | - The original repository file format was based on Debian 63 | - It was anticipated that someday we'll reach a scale where we will run into performance issues. 64 | - If someone wants to propose and move to a v2 repository file format this would be a good contribution 65 | - There was a talk from Ankur Chadha (JFrog) that kicked off discussion here. 66 | - Will moving to a new v2 repository format be a breaking change for search? 67 | - Helm search should still work even with this move 68 | - A problem this would solve for the chart repo - have a distributed index / chart delegation (viglesias). 69 | - This would make charts a sort of meta-repo, so kubernetes/charts can have something that it can delegate to instead of growing unsustainably. 70 | 71 | 72 | 73 | 74 | ## Topic 3: Upload of charts directly from the helm CLI 75 | 76 | - It might make sense for the chart upload mechanism to be based on Helm plugins 77 | - The plugin interface is confusing today, can we make this better / easier? 78 | - Also discovery and downloading the plugins can be problematic (today at least). 79 | - Moreover, kubectl has its own plugins and plugin mechanisms 80 | - (Completely different side topic – Should helm be/have a kubectl plugin interface?) 81 | - It was mentioned that the plugin conversation is not simple – need to follow up with what conversations happend at the plugin table during the Summit (which was a later session). 82 | - Should "helm push" be added to the base API and not be a plugin? This is a bigger conversation. At the very least this deserves an issue in the Helm issue queue, where folks can weigh in on the idea. 83 | -------------------------------------------------------------------------------- /Round3_Table1_Tiller_and_CRDs.md: -------------------------------------------------------------------------------- 1 | 17 | 18 | 19 | 20 | # Future of Tiller 21 | 22 | 23 | 24 | 25 | ## Overall expectations of Users 26 | 27 | - What are the expectations from our users? Do folks usually have templates rendered, and not look at them after rendering? Or do they make / track changes after rendering? 28 | - What's the vision and final goal of the project? What are the use-cases we are trying to solve? 29 | - Helm Power Users say: Hey I need something like Mongo, or redis – moves on to how do I orchestrate this deploy – moves on to orchestrating lifecycles this way. 30 | - Using CRDs for Helm 3 and tillerless helm almost sound antithetical to each other. 31 | - Can we use composible building blocks (Bryan Grants's proposal from earlier in the day)? 32 | 33 | 34 | 35 | 36 | ## Tiller-less Helm 37 | 38 | - Do we head towards a Chef Model – do we move towards Chef Zero vs. Cookbooks? Why not both? This should be possible. 39 | - Can we use plugins for the client side pieces? 40 | - Currently authoring Helm plugins is a bit painful. There is a separate discussion on how to make this better. 41 | - Today Tiller deals with serialization, locking, etc. for multiple operations. 42 | - v1 to v2 was a move from tillerless to tillerful – do we have learnings from this move? 43 | - Some type of client side rendering can definitely help from a testing perspective 44 | 45 | 46 | 47 | 48 | ## Proposal to use CRDs for Helm 3 49 | 50 | - Taylor's proposal touches on Application Lifecycle Management () which was pitched earlier during the Summmit. 51 | - One use case to definitely address – having a component that is running all of the time that reconciles the distributed application. 52 | - Even if and when we do build something like this, the controller should consume libraries that can be injected into the client. 53 | - There was a strong proposal to eliminate the Lifecycle resource – and watching on this resource. 54 | - The reason was that we need a more consistent story about LifeCycle Hooks – at scale ordering of dependencies doesn't really work well. 55 | - We haven't really defined what it means to be healthy – so how do we define dependencies. 56 | - Can we use the concept of liveness / readiness that works so well for Pods also for the CRD level. 57 | - The right way to do this may be to put this in a Service – the semantic of Service readiness is more compelling than of aggregating Pod readiness. 58 | 59 | 60 | 61 | 62 | ## Next Steps for CRDs 63 | 64 | - Eliminate the Lifecycle resource for CRDs – and watching on this resource. 65 | - In the release object, probably makes sense to include a "status" field 66 | - Report back the status of the CRD in this field 67 | - Another idea/suggestion was to have something like an "upgrade in progress" state. 68 | - Ordering is one use case, determining if an application is healthy is another use case. This would address both to a certain extent. 69 | - Taylor: Maybe we can copy the deployment model to use going forward 70 | - Taylor: We should be able to change things, we can't remove things as this would not be backwards compatible 71 | - For current lifecycle hooks, we can possibly write an optional operator that would consume this "state" from the CRD and run currently supported hooks (eg. pre / post-install, etc.) 72 | 73 | 74 | 75 | 76 | ## Other Observations 77 | 78 | - Helm doesn't currently go all they way there for dependency management for some folks. 79 | - A lot of feedback was around keeping the scope of Helm small(er). 80 | - Open Question: How do we define the templating engines in a way that's smart that doesn't lead to drift? Not completely sure how to address this without tiller. eg. we have go-templates registered, but you can change this out to use a different one, is this possible per namespace? 81 | - Ensure that CRDs address harder problems that need to model state machines. How do multiple controllers solve the serialization problem for something like a state machine? The "state" proposed above can help. 82 | --------------------------------------------------------------------------------