├── community ├── blog-and-announcements.md ├── roadmap.md ├── status.md ├── discord-chat.md └── sourcehut.md ├── basics ├── example-content.md ├── installing-from-packages.md └── installing-from-source.md ├── connectivity ├── local-mode.md ├── agent-mode.md └── ssh-mode.md ├── inventory ├── hosts.md ├── groups.md ├── inventory-variables-on-disk.md ├── inventory-overview.md ├── file-based-inventory.md └── dynamic-cloud-inventory.md ├── modules ├── module-overview.md ├── services.md ├── access-control.md ├── external-modules.md ├── commands.md ├── package-managers.md └── control-flow.md ├── playbooks ├── handlers.md ├── playbook-overview.md ├── managing-secrets.md ├── roles.md ├── logging.md └── using-variables.md ├── SUMMARY.md ├── .gitbook └── assets │ ├── file.excalidraw (1) (1) (1).svg │ ├── file.excalidraw (11).svg │ ├── file.excalidraw (13).svg │ ├── file.excalidraw (15).svg │ ├── file.excalidraw (12).svg │ └── file.excalidraw (8).svg ├── README.md ├── appendix ├── tips.md └── security-approach.md └── developer-guides ├── development-tips-intro.md └── module-development.md /community/blog-and-announcements.md: -------------------------------------------------------------------------------- 1 | # 📰 Blog & Announcements 2 | 3 | Release information and development updates, as well as stories and tips, are shared on our [free substack](https://jetporch.substack.com/) newsletter. Sign up below and get emails in your inbox when any new posts are made. 4 | 5 | You can choose to subscribe to the _Development Blog_ (which will also include occasional tips and stories) or alternatively just _Announcements_ for a heads up when a new release is out and occasionally some other schedule-related updates. 6 | 7 | {% embed url="https://jetporch.substack.com" %} 8 | Substack Signup Page 9 | {% endembed %} 10 | -------------------------------------------------------------------------------- /basics/example-content.md: -------------------------------------------------------------------------------- 1 | # 📒 Example Content 2 | 3 | Rather than understanding Jet's documentation all at once, it's often easier to absorb the concept of Jet's playbook automation by looking at examples and documentation in parallel. 4 | 5 | ``` 6 | $ git clone https://git.sr.ht/~mpdehaan/jetporch 7 | $ cd jetporch/examples/ 8 | ``` 9 | 10 | Jet is designed to at least _approach_ being executable infrastructure policy documentation. When someone looks at Jet playbooks, regardless of familiarity with Jet, they should be able to tell at least roughly what they will do. 11 | 12 | Some of our git examples showcase and encourage experimentation with specific language features, while others model more broad and complete real-world scenarios. 13 | 14 | These examples are intended to be executed against real local and/or remote infrastructure and experimented with, or used as a starting point for your own content. 15 | 16 | To see how to execute the examples, see [Command Line Usage.](command-line-usage.md) 17 | 18 | -------------------------------------------------------------------------------- /connectivity/local-mode.md: -------------------------------------------------------------------------------- 1 | # 💻 Local 2 | 3 | Local mode means the playbook just affects the local machine that is running jetp, and no remote connectivity is involved. It is the only Jet mode designed for configuring just a single machine. 4 | 5 |

6 | 7 | See our [CLI documentation](../basics/command-line-usage.md) for details. 8 | 9 | In local mode, inventory is ignored. 10 | 11 | Local mode can be used to build VM and container images from Jet Playbooks, in addition to setup of development environments. It can also be used when executing jet from other management systems. 12 | 13 | 14 | 15 | {% hint style="info" %} 16 | **Check Mode Reminder** 17 | 18 | As per the documentation on [Command Line Usage](../basics/command-line-usage.md), local mode can be run in two ways -- "local" mode which will configure the system, and "check-local" which returns what jet thinks would happen if running "jet local". [SSH](ssh-mode.md) has this feature as well. 19 | {% endhint %} 20 | 21 | -------------------------------------------------------------------------------- /inventory/hosts.md: -------------------------------------------------------------------------------- 1 | # 💻 Hosts 2 | 3 | 4 | 5 |

6 | 7 | A Host defines an IP address or hostname that jet can talk to, give orders, or configure. 8 | 9 | [Playbooks](../playbooks/playbook-overview.md) in Jet cannot work by naming specific hosts. Playbooks address multiple hosts at the same time by referencing [Groups](groups.md). Groups contain the hosts. 10 | 11 | Hosts _can_ be given specific [variables](../playbooks/using-variables.md) that override variables specified on the groups of which they are a member, but this doesn't mean they should. 12 | 13 | Generally, you should resist defining too many variables on hosts. It's better to think in terms of groups as much as possible. 14 | 15 | Occasionally a host may run on a non-default SSH port, or need to have an address defined that is different from the name of the host. This, along with the username to login as, can be set by variables that are described on the [SSH](../connectivity/ssh-mode.md) chapter. 16 | 17 | 18 | 19 | -------------------------------------------------------------------------------- /community/roadmap.md: -------------------------------------------------------------------------------- 1 | # 🛣 Roadmap 2 | 3 | [Trello hosts our our public roadmap and planning board](https://trello.com/b/57AyE7ka/jetporch-public-roadmap), which is always evolving. 4 | 5 | 6 | 7 |

8 | 9 | No login is required to view the roadmap. 10 | 11 | We want to make running jet from the development branch an always pleasant and _stable_ experience. Further, all official releases are aimed to avoid breaking users with existing content. 12 | 13 | While our roadmap is public, it may not contain all that we have coming in any future release. As releases evolve, we will add notes so users can see what is coming and what to look forward to. Things can shift around at any time! 14 | 15 | If you don't want to keep up with trello, the [Substack](blog-and-announcements.md) will always summarize upcoming release features in the announcements category even if you are not signed up for develoment blog topics. If you have a question about upcoming features or plans, stop by the [Discord](discord-chat.md) at any time! 16 | 17 | {% embed url="https://trello.com/b/57AyE7ka/jetporch-public-roadmap" %} 18 | 19 | -------------------------------------------------------------------------------- /community/status.md: -------------------------------------------------------------------------------- 1 | # 📈 Status 2 | 3 | ## Current Release 4 | 5 | v0.1.1 - Tech Preview 1 6 | 7 | ## In Development 8 | 9 | v0.2.0 - Tech Preview 2 10 | 11 | ## Support Matrix 12 | 13 | 14 | 15 | | OS | As A Control Machine | As A Configuration Target | 16 | | ---------------------------------- | -------------------- | ------------------------- | 17 | | Mac OS X (any) | Fully Supported | Fully Supported | 18 | | Enterprise Linux Variants (>= 7.0) | Fully Supported | Fully Supported | 19 | | Fedora | Fully Supported | Fully Supported | 20 | | Debian | Fully Supported | Fully Supported | 21 | | Ubuntu | Fully Supported | Fully Supported | 22 | | Arch | Fully Supported | Fully Supported | 23 | | SuSE | Fully Supported | Fully Supported | 24 | | Alpine | Fully Supported | Fully Supported\* | 25 | | Other Linux | Fully Supported | ??? | 26 | 27 | 28 | 29 | {% hint style="info" %} 30 | **Distribution Specific Notes** 31 | 32 | Alpine users will need to run "apk add shadow" so the access control modules are functional. 33 | {% endhint %} 34 | 35 | 36 | 37 | 38 | 39 | -------------------------------------------------------------------------------- /modules/module-overview.md: -------------------------------------------------------------------------------- 1 | --- 2 | description: This page is a placeholder 3 | --- 4 | 5 | # 🔎 Module Overview 6 | 7 | Modules are the 'behavior' in Playbook [Task](../playbooks/tasks-and-task-modifiers.md) statements that cause meaningful action to happen on managed systems- they do things or make some property of a system match a particular state. 8 | 9 | Tasks appear like so in a Playbook: 10 | 11 | 12 | 13 | ``` 14 | tasks: 15 | 16 | - !shell 17 | name: optional comment about running /usr/bin/foo 18 | cmd: /usr/bin/foo --flag1 --flag2 19 | 20 | ``` 21 | 22 | This diagram was used earlier in the documentation, but we'll repeat it here: 23 | 24 | 25 | 26 |

27 | 28 | 29 | 30 | 31 | 32 | Core modules are code that is part of our Rust binary and do not require any software to be installed or transferred to remote systems. We will follow a many-batteries-included philosophy with some minor curation caveats that are detailed on each module page. 33 | 34 | 35 | 36 |

37 | 38 | 39 | 40 | 41 | 42 | [External module support](external-modules.md) (pending) involves executing modules written in any language that can read or emit JSON, which can also be used to call modules from other management tooling, regardless of programming language implementation choices. 43 | 44 | -------------------------------------------------------------------------------- /basics/installing-from-packages.md: -------------------------------------------------------------------------------- 1 | # 💾 Installing From Packages 2 | 3 | {% hint style="info" %} 4 | **Installing From Source Is Encouraged** 5 | 6 | As Jetp just made it's very first release (Tech Preview 1 on September 29th, 2024), you may want to consider following the development branch and [Installing From Source](installing-from-source.md) - releases are expected to occur approximately monthly, so you may be missing out on features and bug fixes that already exist on the development branch. For release information see [Roadmap](../community/roadmap.md), [Status](../community/status.md), and the [Blog](../community/blog-and-announcements.md). 7 | 8 | We always want to make the development branch very stable -- so there's no problem with following along with the development branch if you like. If you are not following along with [chat](../community/discord-chat.md), and even if you are, many new development features will be announced on the [Blog](../community/blog-and-announcements.md) as they are added. 9 | {% endhint %} 10 | 11 | ## Rust Packages 12 | 13 | Once Rust is installed (see instructions under [Installing From Source](installing-from-source.md)), the latest jetp release can be installed with: 14 | 15 | ``` 16 | $ cargo install jetp 17 | ``` 18 | 19 | this will install the jetp executable into \~/.cargo/bin. 20 | 21 | ``` 22 | $ which jetp 23 | $ jetp --version 24 | ``` 25 | 26 | ### Native OS Packages 27 | 28 | For the current release, official OS packages (RPMs, debs, etc) endorsed by JetPorch are not yet available. These may be available in the future. 29 | 30 | -------------------------------------------------------------------------------- /inventory/groups.md: -------------------------------------------------------------------------------- 1 | # 💻 Groups 2 | 3 | Before we talk about how to define groups later in this chapter, let's understand what Groups are. 4 | 5 | Groups map [Hosts](hosts.md) _- manageable machines specified by hostnames or IPs -_ into sets of machines that can be given particular orders at the same time. 6 | 7 | 8 | 9 | Groups Organize Hosts

10 | 11 | 12 | 13 | 14 | 15 | Some ideas to think about: 16 | 17 | * Groups do _not_ need to specify that all systems in a group are alike, whether that means their roles or operating systems. 18 | * Groups typically represent something having a common location (like a datacenter, region, or availability zone), property (hardware type), or function that would warrant special attention or variance in configuring machines in that group. 19 | * Groups can overlap, with a system being in more than one group. Any amount is fine. 20 | * Groups can have subgroups, where they automatically include all the systems in child groups below them. Subgroups can continue to arbitrary levels of depth. 21 | * Group hierarchies do not have to represent a tree, but do represent a graph. The graph technically can have cycles, but Jet will not get caught in any infinite loops. Cycles can sometimes result in some unexpected variable precedence behavior. Try to avoid cycles. 22 | * Groups can have variables assigned to them that apply to all hosts in the group, and these variables can be used in templates and the Jet playbook language. Sometimes a group will never be addressed in configuration, but is there to only apply certain variables. 23 | 24 | Examples of groups names might be things like "us-east-1", "testlab", "build-nodes", or "webservers". 25 | 26 | These concepts will be explored as this chapter continues, and more so in the [Playbooks](broken-reference) section. 27 | 28 | 29 | 30 | -------------------------------------------------------------------------------- /playbooks/handlers.md: -------------------------------------------------------------------------------- 1 | # ⌚ Handlers 2 | 3 | Handlers are special tasks that mostly work like normal tasks (that is to say, they run modules), but are not necessarily run in every playbook run on every single host. Why? They are triggered by conditions occuring from other modules changing something. 4 | 5 | The most common use of handlers is to restart a service only when the configuration file for that service changes, for example, if the configuration file is replaced by a new version of a template, we may want to restart the service using that configuration file. Many users will use handlers for no other reason than this. 6 | 7 | To phrase this differently, handlers are run only when _notified_ by some other task. These notifications happen only when changes occur on that triggering task. If the triggering task found that nothing needed to be changed or executed, the handler would not need to run. 8 | 9 | 10 | 11 | Overflow Of Handler Logic

12 | 13 | Handler lists can be defined in [Roles](roles.md) as well, but in a loose playbook section an example would look like so: 14 | 15 | ``` 16 | # playbook.yml 17 | 18 | - groups: 19 | - all 20 | 21 | tasks: 22 | 23 | - !template 24 | src: foo.conf.hb 25 | dest: /etc/foo/foo.conf 26 | and: 27 | notify: restart foo 28 | 29 | handlers: 30 | 31 | - !sd_service 32 | service: foo 33 | restarted: true 34 | with: 35 | subscribe: restart foo 36 | 37 | ``` 38 | 39 | The name of the handler must match the name of the "notify" signal string for the handler to run. 40 | 41 | {% hint style="danger" %} 42 | **subscribe vs name** 43 | 44 | The names of tasks in Jet are just optional _comments_, the "subscribe" parameter is how the handlers are identified. It is also possible for multiple handlers to subscribe to the same event messages! 45 | {% endhint %} 46 | 47 | -------------------------------------------------------------------------------- /modules/services.md: -------------------------------------------------------------------------------- 1 | # 🕒 Services 2 | 3 | Service modules manage long-running operations. OS services for all Linux/Unix platforms are definitely a good fit for this category, but other service management systems (such as supervisord) could also be relevant for future module additions. 4 | 5 | ### Service Module Index 6 | 7 | | Module | Description | 8 | | -------------------------------------- | ------------------------------------------------------------------- | 9 | | [sd\_service](services.md#sd\_service) | systemd-based service management for nearly all Linux distributions | 10 | 11 | ### sd\_service 12 | 13 | Manages services for operating systems that use systemd. 14 | 15 | 16 | 17 | ``` 18 | tasks: 19 | - !sd_service 20 | service: redis 21 | started: true 22 | enabled: true 23 | 24 | - !sd_service 25 | service: nginx 26 | started: false 27 | enabled: false 28 | 29 | - !sd_service 30 | service: httpd 31 | enabled: true 32 | restart: true 33 | 34 | ``` 35 | 36 | "enabled", "started", and "restart" are all optional parameters. 37 | 38 | The restart flag will be mostly used with handlers to restart services when configuration files change, this is described in the [Play](../playbooks/plays.md) documentation. 39 | 40 | 41 | 42 | | Parameter | Description | 43 | | ----------- | ---------------------------------------------------------------------------------------------------------------------- | 44 | | **service** | **Required**. The name of the service to manage | 45 | | **started** | Boolean. Whether the service should be started or not. Omission implies this property is not managed. | 46 | | **enabled** | Boolean. Whether the servcie should be enabled to start on boot or not. Omission implies this property is not managed | 47 | 48 | 49 | 50 | -------------------------------------------------------------------------------- /inventory/inventory-variables-on-disk.md: -------------------------------------------------------------------------------- 1 | # 📂 Inventory Variables On Disk 2 | 3 | Whether a [File-Based](file-based-inventory.md) or [Dynamic inventory](dynamic-cloud-inventory.md) is used, a "group\_vars/" and "host\_vars/" Variable folder can exist alongside an inventory source. This allows variables to be defined in YAML that are added to the inventory, regardless of where it came from. 4 | 5 | {% hint style="info" %} 6 | **Reminder: When Does Inventory Apply?** 7 | 8 | While mentioned earlier, it may help repeating that inventory in Jet is an _SSH-only_ concept and does not apply to local or future deployment modes. In fact, it is an error to specify --inventory when trying to execute a playbook locally. 9 | {% endhint %} 10 | 11 | For file-based inventory, Jet looks alongside the '_groups/_' folder to see if there are any files with the same names as the groups. When it looks for files, it looks for files _without_ an extension: 12 | 13 | ``` 14 | # directory listing 15 | # used by command like: 16 | # jetp [mode] --inventory path/to/inventory [...] 17 | 18 | path/to/inventory/ 19 | groups/ 20 | webservers 21 | boston 22 | group_vars/ 23 | webservers 24 | boston 25 | host_vars/ 26 | webservers 27 | boston 28 | ``` 29 | 30 | When using dynamic inventory, Jet looks for the group\_vars/ and host\_vars/ directories parallel to the inventory scripts used. 31 | 32 | ``` 33 | # directory listing 34 | # used by command like: 35 | # jetp [mode] --inventory path/to/inventory/dynamic_inventory_script.py [...] 36 | 37 | path/to/inventory/ 38 | groups/ 39 | webservers 40 | boston 41 | group_vars/ 42 | webservers 43 | boston 44 | dynamic_inventory_script.py 45 | dynamic_inventory_script.ini 46 | 47 | ``` 48 | 49 | File formats are the same regardless of the inventory type. 50 | 51 | Both '_group\_vars/_' and '_host\_vars/_' files are YAML hashes like so: 52 | 53 | ``` 54 | asdf: 1234 55 | a_list: 56 | - one 57 | - two 58 | - three 59 | a_nested_nash: 60 | dog: fido 61 | cat: fluffy 62 | ``` 63 | 64 | We strongly suggest keeping the inventory folder in source control, possibly as a seperate repository from the playbook content. 65 | 66 | ### 67 | 68 | 69 | 70 | -------------------------------------------------------------------------------- /SUMMARY.md: -------------------------------------------------------------------------------- 1 | # Table of contents 2 | 3 | * [JetPorch](README.md) 4 | 5 | ## Basics 6 | 7 | * [💾 Installing From Source](basics/installing-from-source.md) 8 | * [💾 Installing From Packages](basics/installing-from-packages.md) 9 | * [📒 Example Content](basics/example-content.md) 10 | * [✈ Command Line Usage](basics/command-line-usage.md) 11 | 12 | ## Community 13 | 14 | * [🔊 Discord Chat](community/discord-chat.md) 15 | * [📰 Blog & Announcements](community/blog-and-announcements.md) 16 | * [🛣 Roadmap](community/roadmap.md) 17 | * [📈 Status](community/status.md) 18 | * [🚁 Contributing](community/contributing.md) 19 | * [🛖 SourceHut](community/sourcehut.md) 20 | 21 | ## Inventory 22 | 23 | * [🔎 Inventory Overview](inventory/inventory-overview.md) 24 | * [💻 Groups](inventory/groups.md) 25 | * [💻 Hosts](inventory/hosts.md) 26 | * [📂 File-Based Inventory](inventory/file-based-inventory.md) 27 | * [☁ Dynamic/Cloud Inventory](inventory/dynamic-cloud-inventory.md) 28 | * [📂 Inventory Variables On Disk](inventory/inventory-variables-on-disk.md) 29 | 30 | ## Playbooks 31 | 32 | * [🔎 Playbook Overview](playbooks/playbook-overview.md) 33 | * [🏃♂ Plays](playbooks/plays.md) 34 | * [✅ Tasks & Task Modifiers](playbooks/tasks-and-task-modifiers.md) 35 | * [⌚ Handlers](playbooks/handlers.md) 36 | * [🎭 Roles](playbooks/roles.md) 37 | * [🔁 Using Variables](playbooks/using-variables.md) 38 | * [🔒 Managing Secrets](playbooks/managing-secrets.md) 39 | * [📒 Logging](playbooks/logging.md) 40 | 41 | ## Modules 42 | 43 | * [🔎 Module Overview](modules/module-overview.md) 44 | * [🔓 Access Control](modules/access-control.md) 45 | * [❗ Commands](modules/commands.md) 46 | * [🔀 Control Flow](modules/control-flow.md) 47 | * [📦 External Modules](modules/external-modules.md) 48 | * [📂 Files](modules/files.md) 49 | * [🎁 Package Managers](modules/package-managers.md) 50 | * [🕒 Services](modules/services.md) 51 | 52 | ## Connectivity 53 | 54 | * [💻 Local](connectivity/local-mode.md) 55 | * [▶ SSH](connectivity/ssh-mode.md) 56 | * [🌎 Agent Mode](connectivity/agent-mode.md) 57 | 58 | ## Developer Guides 59 | 60 | * [🔢 Development Tips / Intro](developer-guides/development-tips-intro.md) 61 | * [🛠 Module Development](developer-guides/module-development.md) 62 | 63 | ## Appendix 64 | 65 | * [❔ FAQ](appendix/faq.md) 66 | * [💡 Tips](appendix/tips.md) 67 | * [🔐 Security Approach](appendix/security-approach.md) 68 | -------------------------------------------------------------------------------- /inventory/inventory-overview.md: -------------------------------------------------------------------------------- 1 | # 🔎 Inventory Overview 2 | 3 | {% hint style="info" %} 4 | **Understanding When Inventory Applies** 5 | 6 | If wanting to configure machines in Jet over [SSH](../connectivity/ssh-mode.md), there are two basic key concepts to understand -- _Inventory_ and [Playbooks](../playbooks/playbook-overview.md). 7 | 8 | If wanting to use Jet [locally](../connectivity/local-mode.md) or in the future local [agent](../connectivity/agent-mode.md) modes, you only need to understand [playbooks](../playbooks/playbook-overview.md). In this section, we will assume you _do_ want to deploy with SSH. If you want to learn Jet playbooks _locally_ first, you can skip this section and come back later. 9 | 10 | To review how to select different deployment modes using _jetp_ subcommands, see [Executing A Playbook](../basics/command-line-usage.md). 11 | {% endhint %} 12 | 13 | **Inventory** tells jet (in SSH mode) which machines (arranged into [Groups](groups.md)) we want to control or talk to. 14 | 15 | There are two ways of defining inventory: 16 | 17 | 1. [**File-based inventory**](file-based-inventory.md) organizes lists of hosts into groups using YAML files on the filesystem. This structure is really best understood by having GitHub open in a new tab, so open up the link in [example repo](../basics/example-content.md) for examples when reading the chapter. File-based inventory is great for getting started and also for small development projects, though many users with production cloud configurations may not use it long term. 18 | 2. [**Dynamic inventory**](dynamic-cloud-inventory.md) is commonly used with cloud systems and "external sources of truth". With these, jet calls an executable script (or scripts) that return JSON. These scripts ask the external source (such as a cloud provider) what machines there are to manage, so no inventory information has to be described manually -- this is important because in cloud cases, in particular, inventory is always changing. 19 | 20 | No matter how we choose to load it, Inventory is defined by two types of objects in Jet -- [Groups](groups.md) and [Hosts](hosts.md), which are explained further in the upcoming sections. 21 | 22 | ## Inventory Display 23 | 24 | Inventory groups and hosts, as well as variables loaded from them, can be explored with the jetp CLI: 25 | 26 | ``` 27 | 28 | $ jetp show-inventory # shows the all group 29 | 30 | $ jetp show-inventory --show-groups webservers 31 | 32 | $ jetp show-inventory --show-hosts www01.example.com 33 | ``` 34 | 35 | -------------------------------------------------------------------------------- /connectivity/agent-mode.md: -------------------------------------------------------------------------------- 1 | # 🌎 Agent Mode 2 | 3 | **Coming Soon** 4 | 5 | Ordinarily jet works in a SSH push-based mode. This is a great fit for configuring entire service tiers. Running jet with 100 parallel threads is definitely a normal thing to do, though if configuring, say, 100k machines all at once, this no longer becomes an effective strategy. 6 | 7 | For this, jet will be adding a command called 'jetp watch', which may work as follows: 8 | 9 | ``` 10 | jetp watch --watch-config watch.yml 11 | ``` 12 | 13 | This will start a simple agent process. 14 | 15 | To understand the watch command, it's easiest to talk about the configuration file and how it effects operation: 16 | 17 | ``` 18 | check_command: /var/jetp/check.sh 19 | download_command: /var/jetp/download.sh {{ download_path }} 20 | jet_command: jetp local --playbook {{ download_path }}/pb.yml 21 | check_interval: 200 22 | splay: 100 23 | on_ok: /var/jetp/ok.sh {{ play_uuid }} 24 | on_fail: /var/jetp/fail.sh {{ play_uuid }} 25 | ``` 26 | 27 | By changing the various commands, we can decide when to run jet on a remote system using a variety of criteria, and where to get content from. 28 | 29 | For deciding when to kick off a playbook run, we set this with **check\_command** - these may include looking for watch files or checking a message bus (Kafka, NATS, RabbitMQ, etc), bucket, or REST API. The script can return 0 when jet needs to run, and 1 when it does not need to run. 30 | 31 | The **download\_command** can pull content from NFS, or a bucket like s3 or Minio, and would be triggered when the check command indicates a run is desired. A random GUID location is created with "\{{ download path \}}" for optional use. This same parameter is made available to the jet command later. If the content is already on the filesystem, this parameter could be ignored. 32 | 33 | The system can wait **check\_interval** seconds between running the check command. 34 | 35 | **splay** waits a random number of seconds (less than this value) on startup to minimize thundering herd scenarios. 36 | 37 | **on\_ok** and **on\_fail** can optionally run certain commands when failures occur. This might be to signal a REST service or upload logfiles. 38 | 39 | This feature will also be heavily useful when used with pending structured logfile support, making it integrate well with logfile aggregration services like Splunk, Logstash, so on. 40 | 41 | All of these options can be put together in various ways. For instance, to just run a playbook periodically, the check\_command can be /usr/bin/true. If content is on NFS, the download command could also be _/usr/bin/true_. 42 | 43 | 44 | 45 | -------------------------------------------------------------------------------- /playbooks/playbook-overview.md: -------------------------------------------------------------------------------- 1 | # 🔎 Playbook Overview 2 | 3 | Jet **Playbooks** select [**Groups**](../inventory/groups.md) from [**Inventory**](broken-reference) and then assign work or configurations to those selected machines. 4 | 5 | Playbook File Structure

6 | 7 | 8 | 9 | 10 | 11 | Playbooks can be used to enforce configuration, deploy applications, or orchestrate arbitrary distributed workflows. Playbooks are written in Jet's specific dialect of YAML. 12 | 13 | {% hint style="info" %} 14 | **Keeping YAML Friendly** 15 | 16 | In contrast to some YAMLOps tools, for example some in the Kubernetes ecosystem, Jet playbooks always strive for a relatively flat YAML structure and human-levels of readability. We do not deeply nest data structures or include advanced templating features directly in the language, and the document is always valid YAML itself. 17 | 18 | We feel a lot of other tools have given YAML a good name, and take a lot of time on language design on purpose. YAML is, pretty much, the best way to represent structured data that is editable by humans. However, the simplicity of the language is always in the hands of those building the tools. While we aren't perfect, we want users to have as little to remember as possible, and try to always make choices as obvious as we can. 19 | 20 | YAML tends to have features like interpreting the word "no" as false, in Jet, only true and false are legal for booleans. Similarly, we never encourage use of some confusing YAML features like anchors. We do use YAML tags, which may be new to some people, for selecting Jet [modules](broken-reference). 21 | {% endhint %} 22 | 23 | Inside of Playbooks, lists of [**Tasks**](tasks-and-task-modifiers.md) define the work or configurations to execute or apply on those selected machines. 24 | 25 | To encourage reuse and clarity, Tasks can (and usually _should_) be grouped into [**Roles**](roles.md)**.** This way, the Playbook can be mostly thought of as a mapping between Groups and Roles. 26 | 27 | When using roles consistently, a playbook just answers the question "Who does what work?" and executing a playbook makes that specific work happen -- and playbooks become very concise, with the guts of the automation abstracted away into the role definitions. This is explored further later in this section. 28 | 29 | Playbooks are probably best understood when looking at some language examples and following along with the documentation in another browser tab. See [example content](../basics/example-content.md). For information on how to run a playbook, flip back to the [CLI Instructions](../basics/command-line-usage.md). 30 | 31 | 32 | 33 | 34 | 35 | 36 | 37 | -------------------------------------------------------------------------------- /playbooks/managing-secrets.md: -------------------------------------------------------------------------------- 1 | # 🔒 Managing Secrets 2 | 3 | This chapter is about how to use secret values in production and avoid accidental leakage into logs. This works using Jet's ability to get variables from the environment. Therefore, environment variable support in jet is not technically secret specific but is designed to work _mainly_ with secrets due to some _intentionally_ designed limitations that protect against secret leakage. 4 | 5 | Here is an example of passing environment variables into Jet: 6 | 7 | ``` 8 | $ FOO=1 BAR=2 jetp ssh --playbooks pb1.yml --inventory ~/path/to/inventory 9 | ``` 10 | 11 | Contrary to this basic example (which is good for learning), when dealing with secrets, we would _not_ normally get environment variables from the shell. The better way to inject variables into Jet is "--extra-vars" (-e) as described in [Command Line Usage](../basics/command-line-usage.md). As such, when we use variables like this, we are going to be intending them for secrets. 12 | 13 | Naturally, we would not want to put secret values on the command line where they appear in shell history. Secret values would be loaded in from various secret tools like this example from 1Password's op run: 14 | 15 | ``` 16 | op run --env .env -- jetp ssh --playbooks pb1.yml --inventory ~/path/to/inventory --loadenv 17 | ``` 18 | 19 | In the above shell example, op run executes jetp in a subshell, passing extra secrets directly to jetp as unencrypted values. 20 | 21 | 22 | 23 | Where Secrets Flow

24 | 25 | Environment variables (not just "secrets") will be made available to the the [template module](../modules/files.md) only, allowing our environment variable support to be quickly tested without using a particular secret tool: 26 | 27 | ``` 28 | # foo.conf.hb 29 | 30 | [excellent_application] 31 | foo={{ENV_foo}} 32 | bar={{ENV_bar}} 33 | splines=5 34 | reticulate=1 35 | ``` 36 | 37 | Since only the template module is supported for this, we can not access secret values like so: 38 | 39 | ``` 40 | tasks: 41 | - !shell 42 | cmd: "echo hey I shouldn't tell you this but bar is {{ ENV_bar }} 43 | ``` 44 | 45 | That will produce an error. This is very intentional. 46 | 47 | Only supporting the template module is done to make sure that values cannot be leaked into build logs or shell history accidentally. 48 | 49 | Redaction features from tools like "op run" provide an extra measure of log and output protection, which could be valuable if someone attempted to execute cat on a produced template or something of that nature to help with debugging. 50 | 51 | This feature is not fool proof. 52 | 53 | It is designed to prevent accidental leakage of secrets to the wrong parties, but with access to source control or build scripts, things change quickly. 54 | 55 | Among those with sufficient access, mandatory code review prior to deployment is the best solution to possible insider threat issues, which is why Jet is designed around auditability and predictable execution. 56 | 57 | 58 | 59 | 60 | 61 | 62 | 63 | -------------------------------------------------------------------------------- /community/discord-chat.md: -------------------------------------------------------------------------------- 1 | # 🔊 Discord Chat 2 | 3 | Jet has a free chat instance on Discord, which is a great place for idea discussion and Q\&A. 4 | 5 | [**Click here to join!**](https://discord.gg/mu5DCPGJJq) 6 | 7 | Discord uses either a free [desktop](https://discord.com/download) or a mobile app that you can download from your mobile app store. 8 | 9 | ## Why Join Chat? 10 | 11 | How The Project Uses Chat

12 | 13 | Our community workflow is mostly based around Discord and [Substack](blog-and-announcements.md) and uses GitHub only to store the code, manage tickets, and pull requests. Discord is not just for casual chat -- it's our community hub -- you can read the (a bit long, sorry) explanation of this on [Contributing](contributing.md). 14 | 15 | There is _intentionally_ no email discussion list and we're not on social media. We could have used email for planning and questions, but that approach seemed a bit dated and often people to not want to subscribe to high-traffic groups. We could have used Slack, but joining and flipping between Slack channels is often confusing. 16 | 17 | Discord works well for us and is increasingly used for more and more software projects. 18 | 19 | For asynchronous and more curated information, please also join the [free substack](blog-and-announcements.md) newsletter. 20 | 21 | ## Rules 22 | 23 | We have just a few rules for Discord: 24 | 25 | 1. Be nice to each other. 26 | 2. Do not promote your products, projects, and services. 27 | 3. Our Discord is a meant to be a fun, casual channel for prospective users and those interested in directly using or working on the project _only_. For curious professional management tool developers and business employees involved in applications that could be viewed as ecosystem alternatives to Jet, this is _not_ an appropriate channel for you. We recommend following the [Substack](blog-and-announcements.md), which will have frequent detailed updates and all release information. 28 | 29 | ## Additional Chat Tips 30 | 31 | 1. Public pastebins, like gist.github.com, are often really helpful to see playbooks or ideas in context, without trying to format them from inside the Discord interface. It's also possible to allow comments on gists, which can be really useful in correcting playbook content. 32 | 2. If you think you've found a problem, descriptive language is always better than saying something doesn't work. Showing the command line used and relevant portions of output is also very helpful. This same tip goes for GitHub tickets as well. Avoid flooding the channel though, so again, pastebins are helpful. For talking about some details, screenshots are also welcome, but they don't make copy-and-paste easy. 33 | 3. While not a rule, if you'd like to privately message someone on Discord, you can make message requests without using the Discord's "Friends List" feature. Friends List requests might be cancelled, and we don't want to seem unfriendly, so we are just passing that along here first! 34 | 4. We work for GIFs and emojis! Not kidding. 35 | 36 | 37 | 38 | 39 | 40 | -------------------------------------------------------------------------------- /inventory/file-based-inventory.md: -------------------------------------------------------------------------------- 1 | # 📂 File-Based Inventory 2 | 3 | Inventory in Jet can be defined using YAML files on the filesystem. 4 | 5 | {% hint style="info" %} 6 | **When To Use File Based Inventory** 7 | 8 | While some large installations may adopt file-based inventory, many cloud users and larger installations may be interested in [dynamic inventory](dynamic-cloud-inventory.md) instead. Still, new users may find it easier to learn with this file-based inventory inventory, and it makes an excellent starting point. If this is your first time with Jet, we recommend trying file-based inventory at least once as you are getting going, before switching to dynamic inventory. 9 | {% endhint %} 10 | 11 | {% hint style="info" %} 12 | **Have Experience With Similar Tooling?** 13 | 14 | Users coming from Ansible will note that our group definition files have a different format than the INI-like or YAML-based inventory supported in Ansible. Forget what you know and read the documentation below. Our format has been designed to have less nesting and variance. _Ansible is a registered trademark of Red Hat Inc._ 15 | {% endhint %} 16 | 17 | For understanding static inventory, it is easiest to show an example. [Please open GitHub](https://github.com/jetporch/example\_content/tree/main/inventory) in another browser tab as we go. 18 | 19 | With examples open, let's break it down. Suppose we have a directory structure like this: 20 | 21 | ``` 22 | ./inventory_folder/ 23 | groups/ 24 | group1 # YAML files of a specific format 25 | group2 26 | group_vars/ 27 | group1 # arbitrary YAML maps for variables 28 | group2 29 | host_vars/ 30 | hostname1.example.com # arbitrary YAML maps for variables 31 | hostname2.exmaple.com 32 | ``` 33 | 34 | ## Group Files 35 | 36 | A group file looks like this: 37 | 38 | ``` 39 | # filename: groups/boston - do not add a .yml extension 40 | 41 | subgroups: 42 | - boston-dc1 43 | - boston-dc2 44 | 45 | hosts: 46 | - asdf.example.com 47 | - jkl.example.com 48 | 49 | 50 | ``` 51 | 52 | Basically it's just a simple list of directives: 53 | 54 | ### subgroups 55 | 56 | Subgroup declares that the group, in this case, "boston', has a group of machines below it and that the group "boston" automatically includes all hosts referenced in those group files. 57 | 58 | This keyword is _optional_ and does not have to appear in the file. 59 | 60 | ### hosts 61 | 62 | Declares a list of hosts are members of the named group. 63 | 64 | A groups file does not have to define any hosts if it only wants to define subgroups for a given group. In other words, both '_subgroups_' and '_hosts_' are optional keywords in a group definition. 65 | 66 | ## Assigning Variables 67 | 68 | What's above is all there is to group definitions. There are additional steps to add variable information. 69 | 70 | The above commentary on directory structure showed a 'group\_vars/' and 'host\_vars/ directory. 71 | 72 | See [Inventory Variables On Disk](inventory-variables-on-disk.md) for how to assign values to inventory using that structure by adding these parallel folders. This structure can also be used with [dynamic/cloud inventory](dynamic-cloud-inventory.md) so it is very much worth learning. 73 | 74 | -------------------------------------------------------------------------------- /basics/installing-from-source.md: -------------------------------------------------------------------------------- 1 | # 💾 Installing From Source 2 | 3 | 4 | 5 |

6 | 7 | 8 | 9 | 1. Jetp is made to run from one control machine -- no software needs to be installed on remote machines in order to use Jet other than sshd. To start, we'll install Rust, so we can compile Jet. If you have Rust already installed, our current _minimal supported rust version_ is 1.6.4. Any newer version will be fine, but versions packaged by OS vendors may be somewhat more dated than what you can get from the Rust website and we may decide to move faster than some OS distributions. 10 | 11 | [You can install using the official instructions on rust-lang.org](https://www.rust-lang.org/tools/install). 12 | 13 | 2. Clone the project 14 | 15 | ``` 16 | git clone https://git.sr.ht/~mpdehaan/jetporch ~/jetporch 17 | ``` 18 | 19 | 3. You're going to need to install some development dependencies. If you are running any "**Enterprise Linux"** variants or Fedora: 20 | 21 | ``` 22 | yum install -y gcc openssl openssl-devel 23 | ``` 24 | 25 | If you're on **Debian** or **Ubuntu**: 26 | 27 | ``` 28 | apt-get install make gcc libssl-dev pkg-config 29 | ``` 30 | 31 | If you are on a **Mac**, you will also need to [install homebrew](https://docs.brew.sh/Installation) if you haven't already. This in turn may require running 'xcode-select --install' _first_ to install XCode Developer tools. Once homebrew is installed, run: 32 | 33 | ``` 34 | brew install gcc openssl 35 | ``` 36 | 37 | Jet uses openssl for it's SSH client library (ssh2) when connecting to remote hosts. 38 | 39 | 4. cd into the checkout directory. If you just made a checkout, without running 'git switch', you'll be building the 'main' branch which contains all of the latest code in development. The development branch is intended to always be usable. 40 | 41 | ``` 42 | cd ~/jetporch 43 | ``` 44 | 45 | 5. Build the project by running the Makefile. The first time dependencies will be downloaded and compiled, successive builds will be much faster. 46 | 47 | ``` 48 | make 49 | ``` 50 | 51 | 6. Add the release directory to your path, so that you can rebuild jet without having to copy it over each time. The word 'release' here means the build is optimized, it does not refer to the build coming from a particular versioned _release_. Release builds take longer to build but are significantly faster than 'debug' builds, possibly by 5x-10x in some parts of the program! To enjoy the full power of Jet, release builds are the way to go! 52 | 53 | ``` 54 | $ export PATH=$PATH:~/jetporch/target/release/ 55 | ``` 56 | 57 | 7. Run jetp 58 | 59 | ``` 60 | $ jetp --version 61 | ``` 62 | 63 | The git branch and build time will appear in your terminal. Please include this information when asking questions in chat! 64 | 65 | {% hint style="info" %} 66 | **One Binary To Rule Them All** 67 | 68 | When using jet in [SSH mode](../connectivity/ssh-mode.md), you don't need to compile jet for each of your target architectures, as no binaries are transferred to remote systems. Jet works by using shell commands on the remote hosts. Some modules will require some binaries to be preinstalled, but they never require specific programming language environments like Python, Perl, or Ruby. 69 | {% endhint %} 70 | 71 | ### Staying Updated 72 | 73 | Once you have an a checkout and are running off the development branch, pulling in recent changes from a branch requires just a few steps: 74 | 75 | 1. Update git to pull in any upstream changes 76 | 77 | ``` 78 | $ git pull --rebase 79 | ``` 80 | 81 | 2. Rebuild jetp 82 | 83 | ``` 84 | $ make 85 | ``` 86 | 87 | The Rust compiler will only need to rebuild the jet sources, not all of the dependencies. 88 | 89 | ### Ready For More? 90 | 91 | If you have any problems, [stop by Discord](../community/discord-chat.md) and we'd be happy to help you out. 92 | 93 | What's next? Browse the rest of the documentation, take a look the [example content](example-content.md) and try [executing a playbook](command-line-usage.md). 94 | 95 | -------------------------------------------------------------------------------- /.gitbook/assets/file.excalidraw (1) (1) (1).svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /README.md: -------------------------------------------------------------------------------- 1 | --- 2 | description: the Jet Enterprise Professional Orchestrator, aka Jet 3 | layout: 4 | title: 5 | visible: true 6 | description: 7 | visible: true 8 | tableOfContents: 9 | visible: true 10 | outline: 11 | visible: false 12 | pagination: 13 | visible: true 14 | --- 15 | 16 | # JetPorch 17 | 18 | Jet is **a general-purpose, community-driven IT automation platform** for configuration, deployment, orchestration, patching, and arbitrary task execution workflows. Jet is for Linux and Mac systems. 19 | 20 | 21 | 22 | 23 | 24 |

25 | 26 | 27 | 28 | Jet features: 29 | 30 | * A strong emphasis on language simplicity, consistency, and stability with a minimal aesthetic 31 | * _Extremely_ fast Rust-powered multithreaded, parallel SSH, push-based configuration 32 | * Optional, agent-based pull-framework (ETA: end of year 2023) 33 | * Static HTML-based run reports (ETA: end of year 2023) 34 | * An enterprise security and audit focus 35 | * Friendly, easy-to-read documentation 36 | 37 | Jet is being lead by Michael DeHaan, the original creator of popular IT automation programs _Cobbler_ and _Ansible_. 38 | 39 |

40 | 41 | Jet's engine and included modules are implemented in Rust though soon user modules will be able to be implemented in any language that can speak JSON. To keep up with the project (or get involved), see the [community](community/discord-chat.md) [links](community/blog-and-announcements.md) in the sidebar! 42 | 43 | ### The Basics 44 | 45 |

46 | 47 | ### Additional Summary Info 48 | 49 | {% tabs %} 50 | {% tab title="Language Design" %} 51 | To make things easy for IT users with Ansible familiarity, jet uses a **YAML** dialect similar to the Ansible **playbook** language. While our playbook language is different than classic Ansible, an analogy might be to consider UK vs American English. _Glacier! Aluminum! Schedule!_ 52 | 53 | **Templating** also uses a familiar bracket syntax, this time using _Handlebars_ as the template engine, so porting content should remain straightforward. We have also made variable handling predictable by having less places to define variables. 54 | 55 | We want the Jet language to be very stable to avoid any needed content changes during user upgrades. As such, the language documentation will be treated as a **specification**. Wherever possible, we will be deliberate in the design process to eliminate any breaking changes to the existing user automation content. 56 | {% endtab %} 57 | 58 | {% tab title="Performance And Capabilities" %} 59 | Jet is implemented in **Rust**, producing efficient native binaries cable of **massive parallelism.** Rust's compiler also checks code to an _extremely_ aggressive degree due to the best-in-class type system, greatly benefitting code quality. Similarly, Jet's code itself is designed to check automation parameters aggressively to avoid runtime surprises and common human errors. 60 | 61 | While the engine is Rust, Jet's native modules rely on shell access alone on remote systems in terms of what they execute. No dynamic language runtimes need to be installed on remote systems. However, through external module features, Jet will still be able to execute JSON-based modules written in other languages. 62 | 63 | Jet's supports **local configuration** and multithreaded parallel **SSH**. An agent based pull mode is a pending feature. 64 | 65 | Additional modules and language features will be added over time. 66 | {% endtab %} 67 | 68 | {% tab title="But Wait, There's More" %} 69 | Tons of features not mentioned here are scattered throughout the documentation site. Feel free to explore the various chapters. You may also get a good idea of the feel of Jet by looking at some [example playbook content](basics/example-content.md). If you have any questions, [stop by chat](community/discord-chat.md) and we'd be glad to talk with you about almost anything! 70 | {% endtab %} 71 | {% endtabs %} 72 | 73 | ### Legal/Notes 74 | 75 | _Jetporch (also referred to herein as Jet) is a GPLv3 licensed, community-developed open source program. Jetporch is unaffiliated with Ansible or Red Hat Inc. Ansible is a registered trademark of Red Hat Inc. and is also a GPLv3 licensed open source application._ 76 | 77 | _Jetporch does not come with a warranty, expressed or implied. Usage is always at your own risk._ 78 | -------------------------------------------------------------------------------- /connectivity/ssh-mode.md: -------------------------------------------------------------------------------- 1 | # ▶ SSH 2 | 3 | Jet can connect to remote nodes to manage them using parallel SSH connections. 4 | 5 | First, see our [CLI documentation](../basics/command-line-usage.md) for more information about launching playbooks, particularly notes about _--batch-size_ (once available) and _--threads_. 6 | 7 | Additional details about SSH connections continues below: 8 | 9 | ### Variable Control 10 | 11 | In SSH mode, a playbook will attempt to all connect to all hosts not named "localhost" using SSH, while a node named "localhost" is treated as special, and still uses the [Local](local-mode.md) connection. 12 | 13 | The following inventory variables affect connection behavior: 14 | 15 | 16 | 17 | | Inventory Variable | Purpose | Default | 18 | | ---------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | 19 | | jet\_ssh\_hostname | what hostname or IP to connect to for this host | the name of the host in inventory | 20 | | jet\_ssh\_port | what SSH port to use | 22, unless set by "**ssh\_port**" keyword in a Play | 21 | | jet\_ssh\_user | what user to login as | the currently logged in username on the control machine, unless set in the [Play](../playbooks/plays.md) with the "**ssh\_user**" keyword or **--user** on the jetp command line | 22 | | jet\_ssh\_key\_comment | if multiple keys are added to SSH agent this can specify a specific one to use versus trying all keys (added in v0.2) | the environment variable **JET\_SSH\_KEY\_COMMENT** can also name the default key, or None if absent | 23 | | jet\_ssh\_private\_key\_file | Specifies the path to a specific keyfile to use instead of using keys from ssh-agent (added in v0.2) | none | 24 | | jet\_ssh\_private\_key\_passphrase | while not wildly secure compared to ssh-agent methods, this allows entering a ssh key unlock passphrase as an inventory variable when specifying jet\_ssh\_private\_key\_file (added in v0.2) | the environment variable **JET\_SSH\_PRIVATE\_KEY\_PASSPHRASE** - this requires all keys use the same default unlock password, see [Secret Management](../playbooks/managing-secrets.md) section for tips about how to access this from other tools | 25 | 26 | 27 | 28 | The parameter jet\_ssh\_key\_comment is for use with **ssh-agent only** for when multiple SSH keys are added to SSH agent and multiple different SSH keys are required to connect to all of the systems being managed. 29 | 30 | ``` 31 | ssh-agent 32 | ssh-add ~/.ssh/id_rsa 33 | ssh-add some_other_key 34 | ssh-add some_other_key2 35 | jetp ssh --inventory ~/private_inventory --playbook playbook1.yml -vvv 36 | ``` 37 | 38 | The parameters jet\_ssh\_private\_key\_file and jet\_ssh\_private\_key\_passphrase work similarly but **bypass ssh\_agent**. In general, we suggest using SSH agent, though there may be some reasons to avoid it, such as not having ssh-agent plugins for a particular CI/CD system. 39 | 40 | It is technically possible (though confusing?) to have some hosts that use SSH agent and others that do not. 41 | 42 | -------------------------------------------------------------------------------- /modules/access-control.md: -------------------------------------------------------------------------------- 1 | # 🔓 Access Control 2 | 3 | Access control modules are concerned with creating users and groups. There won't be too many of these. 4 | 5 | 6 | 7 | {% hint style="info" %} 8 | **Where's OS X?** 9 | 10 | Because OS X user management is very very different, and also mostly because developer setup rarely needs them, we do not attempt to provide OS X user/group management for the access control modules. 11 | {% endhint %} 12 | 13 | 14 | 15 | ### Access Control Module Index 16 | 17 | | Module | Description | 18 | | -------------------------------- | -------------------- | 19 | | [group](access-control.md#group) | Manages Linux groups | 20 | | [user](access-control.md#user) | Manages Linux users) | 21 | 22 | ### group 23 | 24 | adds, removes, and manages user groups 25 | 26 | 27 | 28 | | | | 29 | | ------ | ------------------------------------------------------------------------------------------------------------------------------------ | 30 | | group | Required. name of the group | 31 | | gid | Numeric GID to use when creating the group. | 32 | | users | A list of users to add to the group. | 33 | | append | Boolean, default false. If true, the users in the group parameter list will be added to the group instead of made to match the list. | 34 | | system | Boolean, default false. Creates the group as a system group. | 35 | | remove | Boolean, default false. If true, the group will be removed instead of created or modified. | 36 | 37 | ### user 38 | 39 | The user module adds, removes, and manages user accounts. 40 | 41 | 42 | 43 | 44 | 45 | | Parameter | Details | 46 | | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | 47 | | user | Required. the name of the user account to manage | 48 | | uid | Numeric user ID to assign to the account. Used on creation only. | 49 | | gid | Primary group name for the account. | 50 | | groups | List of groups the account should be a member of | 51 | | append | Boolean, default false - modifies the way existing users are treated. If supplied, any missing groups listed in 'groups' will be added to the list of groups for the user, but missing groups will not be removed. | 52 | | create\_home | Boolean, default true - when false, a home directory will not be created for the user account. Applies to creation only. | 53 | | create\_user\_group | Boolean, default true - unless false, a group with the same name as the user will be created automatically. | 54 | | gecos | sets/updates the [gecos field](https://en.wikipedia.org/wiki/Gecos\_field) | 55 | | shell | Default shell to assign to the user | 56 | | system | Boolean, default false. If true, creates the user as a system account. | 57 | | remove | Boolean, default false. If true, will remove the account specified by 'user' if it exists, and other parameters will be ignored. Modified by 'cleanup'. | 58 | | cleanup | Boolean, default false. If true, when removing user accounts the user's home directory and mail will also be removed. | 59 | 60 | 61 | 62 | ``` 63 | tasks: 64 | 65 | - !user 66 | user: bob 67 | 68 | - !user 69 | user: alice 70 | remove: true 71 | cleanup: true 72 | 73 | - !user 74 | user: frodo 75 | groups: 76 | - hobbits 77 | - shire 78 | uid: 5000 79 | shell: /bin/sh 80 | append: true 81 | 82 | ``` 83 | 84 | 85 | 86 | {% hint style="info" %} 87 | **Looking For More Features?** 88 | 89 | Additional features to the user module may make good items for future patches and contributions. These may include adding password setting capabilities, etc. 90 | {% endhint %} 91 | 92 | 93 | 94 | ## 95 | -------------------------------------------------------------------------------- /modules/external-modules.md: -------------------------------------------------------------------------------- 1 | # 📦 External Modules 2 | 3 | 4 | 5 | {% hint style="info" %} 6 | **Early Access** 7 | 8 | This feature is currently available on the git source branches in with the following minor limitations. All documented features are otherwise stable: 9 | 10 | (1) failed\_when conditions should be on top of the return code behavior, not in place of them 11 | 12 | (2) the module JSON conventions does not yet support the module determining it's own changed status by returning a 'changed' element in the JSON 13 | {% endhint %} 14 | 15 | While Jet's stock modules are all written in Rust, Jet also allows you to write modules using any programming language you want - whether that's Go, Python, Ruby, Intercal, or bash, it doesn't matter. 16 | 17 | This system works by pushing out your external module/program to the target hosts (if in SSH mode, otherwise it's just copying it to a temp directory) along with a JSON file containing the host-specific arguments to that module. The module is then run by feeding the arguments as standard input into your module. After the module is run - regardless of success or failure status - Jet will remove both input files. 18 | 19 | ### Writing External Modules 20 | 21 | [Here is a python example of a minimal module](https://git.sr.ht/\~mpdehaan/jetporch/tree/main/item/examples/playbooks/modules/demo\_external.py) - the module can be very short and there is no real requirement of using any particular libraries. If you are writing a module in bash you probably would want to use something like 'jq' to process the input. Examples for different programming languages will likely be added in the future. 22 | 23 | ```python 24 | #!/usr/bin/env python3 25 | 26 | import json 27 | import sys 28 | import fileinput 29 | 30 | data = json.loads("\n".join([ x for x in fileinput.input() ])) 31 | result = {} 32 | data_a = int(data["a"]) 33 | data_b = int(data["b"]) 34 | 35 | result['sum'] = data_a + data_b 36 | result['difference'] = data_a - data_b 37 | 38 | msg = json.dumps(result) 39 | 40 | print(msg) 41 | sys.exit(0) 42 | ``` 43 | 44 | 45 | 46 | ### Module Programming Conventions 47 | 48 | If any variables are used in module inputs, the variables will come into the JSON as strings, so the external module code needs to cast accordingly. 49 | 50 | Modules should take care to not output non-JSON text at least in non-failure conditions, as this will prevent their output from being parsed. 51 | 52 | If a module wishes to return a failure, it can also return non-0 exit status and include a human-readable value in a 'msg' JSON map attribute or just print traceback text. In crash / failure conditions, it's ok to return errors as they are (not as JSON), as non-JSON output will automatically be registered as a failure. 53 | 54 | ### Calling External Modules 55 | 56 | Modules are invoked in Jet playbooks with an '!external' module tag, like so: 57 | 58 |

tasks:
 59 |     - !external
 60 |       use: your_module_name
 61 |       params:
 62 |           asdf: 1234
 63 |           jkl: "foo"
 64 |

65 | 66 | {% hint style="info" %} 67 | **That's Meta** 68 | 69 | If you've been paying attention the above syntax, you might correctly guess that "External Modules" are implemented with a regular jet Rust module. That's why "!external" is used on every call, the name of the Jet Rust module that supports this feature is called "external". 70 | {% endhint %} 71 | 72 | ### Module Search Paths 73 | 74 | Modules will be looked for in a directory along the playbook called "modules/" by default. 75 | 76 | Additionally, the CLI flag --modules (-m) can add additional search paths, as can the environment variable JET\_MODULES\_PATH: 77 | 78 | ``` 79 | ./target/release/jetp local --modules ~/foo --playbook pb.yml 80 | ``` 81 | 82 | ``` 83 | JET_MODULES_PATH=~/foo jetp local --playbook pb.yml 84 | ``` 85 | 86 | ### Additional Features 87 | 88 | Just like the command module, results of the module can be saved to a variable, and there are arguments available for adding to what the changed and failure convention of the module would be. 89 | 90 | 91 | 92 | [Here is a more complete example of the module being used](https://git.sr.ht/\~mpdehaan/jetporch/tree/main/item/examples/playbooks/external.yml): 93 | 94 | ``` 95 | 96 | - name: demo external module feature 97 | groups: 98 | - all 99 | 100 | tasks: 101 | 102 | - !external 103 | use: demo_external.py 104 | params: 105 | a: 1 106 | b: 2 107 | save: result 108 | failed_when: (lt sum 50) 109 | changed_when: false 110 | 111 | - !debug 112 | vars: 113 | - result 114 | 115 | 116 | ``` 117 | 118 | Other than "use" and "params" the remaining parameters work like those in the "[shell](commands.md#shell)" module. 119 | 120 | | Parameter | Description | 121 | | ------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | 122 | | use | The name of the external module to use, including the file extension (if any) | 123 | | params | Arbitrary parameters to the module. This must be a JSON map/dict but can be arbitrarily nested at any level of depth. Keep in mind in playbooks it is usually clearer to limit nesting and to keep things simple. | 124 | | save | Saves the result data in a variable. All JSON data is included, in addition to a variable "rc" which includes the return code from the external module. | 125 | | failed\_when | If a supplied condition here is true, the module will record a failed status even if the module returned a zero exit code and valid JSON. | 126 | | changed\_when | A condition that controls when a change status is recorded for the module. If not supplied, a changed status is always recorded | 127 | 128 | -------------------------------------------------------------------------------- /playbooks/roles.md: -------------------------------------------------------------------------------- 1 | # 🎭 Roles 2 | 3 | You should have already seen how roles are included in playbooks in the [Play](plays.md) documentation chapter. This chapter describes how to define them, now that you have seen how to reference them. 4 | 5 | To review, Roles are a way of organizing [tasks](tasks-and-task-modifiers.md), [variables](using-variables.md), handlers, and so on together to enable reuse across [Groups](../inventory/groups.md) and different [Plays](plays.md). 6 | 7 | We _highly_ recommend using Jet roles for non-trivial automation projects once you understand the basics of Jet. 8 | 9 | 10 | 11 | 12 | 13 | Role Directory Structure

14 | 15 | 16 | 17 | {% hint style="info" %} 18 | **More Role Powers Coming Soon** 19 | 20 | Starting with Tech Preview 2, roles will also be able to package/embed [custom modules](../modules/external-modules.md) for easy distribution. 21 | {% endhint %} 22 | 23 | [The role format is shown in our GitHub examples](https://github.com/jetporch/jetporch\_examples/tree/main/roles/redis) and will be much easier to understand roles with GitHub open in a different browser tab. 24 | 25 | A role typically represents a function a given system should perform, such as "security\_policy" or "application\_server", but they can represent any kind of behavior, application, or policy. 26 | 27 | Roles are found in search paths specified by _--roles-path_ (or enviornment variable _JET\_ROLE\_PATH_) as described in the [CLI Guide](../basics/command-line-usage.md) ... additionally, roles will be looked for in a "roles/" directory alongside playbooks. Using the search path, it is possible to easily use roles in different git repositories. 28 | 29 | The roles structure is shown below. 30 | 31 | ``` 32 | rolename/ 33 | 34 | role.yml 35 | 36 | ./tasks/*.yml 37 | ./handlers/*.yml 38 | 39 | ./templates/*.hb 40 | ./files/*.* 41 | ``` 42 | 43 | 44 | 45 | {% hint style="info" %} 46 | **Role Parameters** 47 | 48 | Roles can be parameterized as shown in the example in the [Plays](plays.md) documentation, in which case this acts as if setting additional _vars_ parameters that are only applicable to the scope of that role. This is shown in the playbook examples. There is not a list of what parameters can be allowed in a role invocation, they really can set any variable name they want. 49 | {% endhint %} 50 | 51 | ### role.yml 52 | 53 | Every role must have a role.yml file in the root of the role directory which can look like the following: 54 | 55 | ``` 56 | name: an optional comment 57 | 58 | defaults: 59 | alpha_port: 2001 60 | beta_port: 2010 61 | gamma_port: 2061 62 | mogwai: "gizmo" 63 | 64 | tasks: 65 | - tasks1.yml 66 | - tasks2.yml 67 | handlers: 68 | - handlers1.yml 69 | - handlers2.yml 70 | ``` 71 | 72 | All sections other than _name_ are optional, but there must be a _role.yml_ file to have a role. The name must be exactly named _role.yml_ verbatim, and cannot be your-role-name.yml, or Jet will not find it. 73 | 74 | {% hint style="info" %} 75 | **No Include Statements** 76 | 77 | There is intentionally no _include_ statement or task in Jet that can be used in the middle of some tasks block, it is the _role.yml_ file itself does the including. This is chosen for clarity in auditing playbook content and to simplify the language. Things must be explicit and decided up front. 78 | {% endhint %} 79 | 80 | A few pathing notes: 81 | 82 | * Paths in '_tasks_' will be assumed to be relative to roles/$rolename/tasks/ unless they are absolute. 83 | * Paths in '_handlers_' will be assumed to be relative to roles/$rolename/handlers/ unless they are absolute. 84 | 85 | It is strongly encouraged to put these files in those locations. 86 | 87 | ### tasks/ 88 | 89 | The 'tasks' section refers to lists of [tasks](roles.md#tasks) that live in role\_name/tasks. 90 | 91 | The values in this file are a flat list of task objects like you might see under a 'tasks:' keyword in a playbook file, like so: 92 | 93 | ``` 94 | # example: tasks/tasks1.yml 95 | 96 | - !shell 97 | name: a first task 98 | cmd: echo foo 99 | 100 | - !template 101 | name: a second task, configuring foo! 102 | src: foo.conf.jb 103 | dest: /etc/foo/foo.conf 104 | attributes: { owner: michaeldehaan, group: michaeldehaan, mode: 0o600 } 105 | and: 106 | notify: restart foo 107 | ``` 108 | 109 | A few notes: 110 | 111 | * If a task file isn't mentioned in role.yml it will not be included. Other unreferenced files in this path will also be ignored. 112 | * Note that the type of the tasks file is a YAML list and there is no "tasks:" at the top of the file. 113 | * This directory does not have to exist unless tasks are referenced in the role.yml. 114 | 115 | ### handlers/ 116 | 117 | Here the system looks for explicitly named files listed in the "handlers:" section of the role.yml file 118 | 119 | The values in this file are also a list of [handler](roles.md#handlers) objects like you would see under the 'handlers:' keyword in a playbook file: 120 | 121 | ``` 122 | # example: handlers/handlers1.yml 123 | 124 | - !sd_service 125 | service: foo 126 | restart: true 127 | with: 128 | subscribe: restart foo 129 | 130 | ``` 131 | 132 | The same details from tasks above apply here: 133 | 134 | * If a handler file isn't mentioned in role.yml it will not be included. Other unreferenced files in this path will also be ignored. 135 | * Note that the type of the handlers file is a YAML list and there is no "tasks:" at the top of the file. 136 | * A directory does not have to exist unless tasks are referenced in the role.yml. 137 | 138 | {% hint style="info" %} 139 | **Does A Handler Have To Contain Tasks?** 140 | 141 | No. A handlers section can exist in a role without a role tasks section, but this could only serve to act on the notifications of other roles that executed previously in the current play. 142 | 143 | Similarly, it is true that event notifications in one role can be acted on in later roles in the same play. Once a new play starts, all handler notification signals are cleared. 144 | 145 | Either way, this would be a relatively uncommon way to use roles. 146 | {% endhint %} 147 | 148 | ### files/ 149 | 150 | The copy module will look for files here when processing tasks in the active role, when trying to figure out what a "src" parameter points to. 151 | 152 | Outside of roles, this would normally for a folder named 'files/' next to the playbook directory. 153 | 154 | This directory does not have to exist unless the role uses the copy module. 155 | 156 | ### templates/ 157 | 158 | The template module will look for files here when processing tasks in the active role, when trying to figure out what a "src" parameter points to. 159 | 160 | Outside of roles, this would normally look in a folder named 'templates/' next to the playbook directory. 161 | 162 | This directory does not have to exist unless the role uses the template module. 163 | 164 | ### 165 | 166 | ### 167 | 168 | 169 | 170 | -------------------------------------------------------------------------------- /.gitbook/assets/file.excalidraw (11).svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /appendix/tips.md: -------------------------------------------------------------------------------- 1 | # 💡 Tips 2 | 3 | ## Basics 4 | 5 | 6 | 7 |

8 | 9 |

Tips For Learning And Adopting Jet

10 | 11 | When learning or adopting any automation system the usual advice to write automation content for one service at a time, learn the tool as you learn what you need to deploy that particular service, and expand gradually. 12 | 13 | Starting with the [automation example content in our GitHub repo](https://github.com/jetporch/jetporch\_examples) is a good way to become familiar with the system. If you have any Jet questions or encounter any challenges, you can stop by our [Discord](../community/discord-chat.md) and we would be happy to help. 14 | 15 | There are places where many features you are seeking may not be yet available (yet?), and this does not mean they aren't things we want to add, so please express any thoughts about missing functionality especially if you do not see it on the [roadmap](../community/roadmap.md). There may be alternative solutions to many workflows. 16 | 17 | Documentation here is mostly organized in a random access way that encourages learning something when you need it. Regardless, be sure to understand concepts about [playbooks](../playbooks/plays.md), [tasks](../playbooks/tasks-and-task-modifiers.md), handlers, roles, [variables](../playbooks/using-variables.md), templates, and (for SSH modes) [inventory](broken-reference). Understanding all of the modules is definitely not required. Following along with example content (soon to be on GitHub) is definitely recommended. 18 | 19 | Learning in a lab or VM environment is probably easiest first with a few VMs and using [static inventory](../inventory/file-based-inventory.md) with IP addresses or hostnames. You can also use your local Mac or Linux computer. Using verbose mode "-v -v" may be instructive for understanding how some Jet commands operate under the hood. If you want to see the value of variables, you can use [echo](../modules/control-flow.md) or ['jetp show](../basics/command-line-usage.md)'. 20 | 21 | Once you are ready to get going, for users with cloud systems, we recommend using '[jetp ssh](../connectivity/ssh-mode.md)' using our [dynamic inventory scripts](../inventory/dynamic-cloud-inventory.md) which is compatible with any of the external inventory scripts you can download from Ansible. We may have our own versions of some of these scripts in the future. External inventory scripts can easily be tested with '_jetp show --inventory script.py --groups all_'. 22 | 23 | The future planetary scale features are designed for very large install bases, and really will not be needed for many users with medium to even large sized installations. It's ok to start with basic SSH now as our Rust-based platform should offer some very interesting performance capabilities on it's own. 24 | 25 |

26 | 27 | ## Process And Organization 28 | 29 | 30 | 31 |

32 | 33 |

Integration with CI/CD Pipelines and Build System Tooling

34 | 35 | Most users will want to gate the execution of their automation using some sort of build system tool. 36 | 37 | This is more desirable than initiating configuration from individual admin machines and can prevent problems when multiple admins are running multiple steps at once. Further, it gives centralized logs and red/green status to whether a deployment encountered any errors. 38 | 39 | Look for whether your build system tool has a plugin to work with SSH credentials and can effectively execute SSH-agent on your behalf. 40 | 41 | If the build system supports parameterized jobs (i.e. "surveys") you can pass the results of those parameters via JSON or YAML to the "-e" command, for instance, you might have a manual job that asks for a release version. 42 | 43 | Read the [Secret Management](../playbooks/managing-secrets.md) chapter and see how Jet can be used with secret management tools to keep secrets out of build logs. 44 | 45 |

46 | 47 |

48 | 49 |

Handling Environmental Differences

50 | 51 | Often different environments will use different servers and these configurations must be baked into the configurations of various machines in configuration files. 52 | 53 | When this occurs, using group variables on disk, in a group\_vars/ folder alongside your inventory is most likely the best way to set these settings so that they are correct regardless of what playbooks or roles are used. 54 | 55 | For cloud based inventory sources, you may want to use tag based groups to apply these variables to particular hosts. 56 | 57 | If there are further questions about this, [stop by the Discord](../community/discord-chat.md) and we'd be happy to discuss details! 58 | 59 |

60 | 61 |

62 | 63 |

Mixing With Other Automation Systems

64 | 65 | It is quite common for many organizations to have multiple automation systems, particularly when migrating between them, or when different departments make different technology choices. This is completely acceptable. 66 | 67 | Using Jet to call other automation tools is fine and should not present any problems, as long as the remote tools remain non-interactive and finish in a short period of time. If doing this, you may wish to execute them with "at" to make them run asynchronously, and use another playbook to collect logs. You will not get intermediate output while long configuration processes are running. 68 | 69 | It's also fine to use another tool to execute Jet in local mode if you so wish, or even executing jet locally on first boot. 70 | 71 |

72 | 73 | 74 | 75 | ## Performance And Scaling 76 | 77 | 78 | 79 |

80 | 81 |

Increasing the --threads Value

82 | 83 | The default value for SSH multithreading in Jet is 20 threads, most capable control machines should be happy with more than 100 or more. Experiment and see what works best for you. 84 | 85 |

86 | 87 |

88 | 89 |

Running Configuration From Inside The Cloud

90 | 91 | Suppose you have a cloud environment with N number of systems in it that we want to configure as quickly as possible. While Jet is smart and tries to keep connections open, it's also true that it would be nice to be as close to those systems as possible. 92 | 93 | You will get better performance from running configuration tasks from a bastion host or designated "helper" machine inside your cloud environments than from a "work from home" setup where each connection comes in from outside the cloud. Being mindful of this will maximize the performance of Jet and result in configuraton runs completing even faster 94 | 95 |

96 | 97 |

98 | 99 |

Use of Mirrors

100 | 101 | When using a configuration management system that has the ability to address thousands of machines at once, it is important to be respectful of public infrastructure the machines may be addressing. 102 | 103 | Do not use machines to download from public resources like pypi on a multitude of machines at once. Instead, look to tools like Artifactory to create private mirrors. 104 | 105 | This will also help you avoid potential problems when upstream resources are inaccessible, packages are removed, and so on. 106 | 107 | Where public resources are being accessed, you can configure --batch-size in Jet to address a small number of machines at a time that will not be effectively be working towards DDOSing a SaaS service or community resource. 108 | 109 |

110 | 111 | -------------------------------------------------------------------------------- /community/sourcehut.md: -------------------------------------------------------------------------------- 1 | # 🛖 SourceHut 2 | 3 | Jet will be moving code and collaboration to SourceHut in the very near future. 4 | 5 | This page is a temporary page to include some writeups for testing before making this move final and replacing our GitHub instructions.GitHub will remain as a read-only mirror in this case, where we disable issues (moving to the SourceHut tracker). 6 | 7 | This page is expected to be a little rough at the moment and will be edited as we get some feedback from folks testing this workflow. 8 | 9 | ## Important Web Links 10 | 11 | * Main project - [https://sr.ht/\~mpdehaan/jetporch/](https://sr.ht/\~mpdehaan/jetporch/) 12 | * See patch queue here, subscription is not required for submissions - [https://lists.sr.ht/\~mpdehaan/jetporch-devel](https://lists.sr.ht/\~mpdehaan/jetporch-devel) 13 | 14 | ## Why SourceHut? 15 | 16 | Jet will be using SourceHut to simplify workflow in merging patches from both maintainers and contributors. SourceHut uses a similar patch submission process as the Linux kernel. 17 | 18 | For those that remember pre-GitHub times, things were actually easier in many ways because you did not have to use a web interface and juggle many different copies of remote and local branches, a local branch and an email was all that was needed to collaborate. Over time, GitHub became all people knew, and these elegant weapons from a more civilized age were lost in time. 19 | 20 | For maintainers, we had more control of merging. Lately, GitHub has been making errors in attribution of some patches, and we do not like how it encourages 'web merging' pull requests and makes the manual process - which can involve more attention and manual testing more difficult. Further, we oppose some AI related training done on projects without their consent or regard for license compatibility, and fear what lazy reliance on AI assisted code will do to the industry. 21 | 22 | Often to use GitHub well we have to ask contributors to rebase or re-squash a patch, which requires an intricate dance of branch creations where on the command line resubmitting a patch is a trivial operation. This is frustrating and unfair for contributors to have to work with.With SourceHut, we can do this ourselves without having to ask. It's smoother on all sides. 23 | 24 | Finally, code review is much easier as we can simply just read the patch set in our editor and reply to it inline, where as GitHub comments carve up the code into tiny blocks and small viewports. 25 | 26 | ## Web Interface 27 | 28 | For starters, you can browse the project code at [https://sr.ht/\~mpdehaan/jetporch/](https://sr.ht/\~mpdehaan/jetporch/) 29 | 30 | ## Checking Out The Code 31 | 32 | To work on the code you do not need a SSH key or a password. An anonymous public clone is all you need! 33 | 34 | ``` 35 | $ git clone https://git.sr.ht/~mpdehaan/jetporch 36 | ``` 37 | 38 | ## Want Online Backups? 39 | 40 | If you would still like to put your own copy of the source code online (on GitHub, GitLab, GitBucket, etc) to share a fork with someone you can do this, but it is not required . Most people will not need to do this, but it's fine if you do want to. 41 | 42 | ``` 43 | git remote add my-repo my-url-here 44 | 45 | git push my-repo 46 | ``` 47 | 48 | ## Sending In A Patch 49 | 50 | Working on the code and sending in a patch is easier than GitHub after you get the hang of your first patch submission. 51 | 52 | First, you need to configure your git checkout of Jet according to the instructions on [https://git-send-email.io](https://git-send-email.io). Don't skip these steps as attaching a patch in email will not work - this will take about 3 minutes and you'll only have to do it once. Many SaaS email services will require setting up an application password to use. The online instructions are fairly brilliant and include instructions for setting up mail clients like Gmail, Fastmail, Proton, and so on. 53 | 54 | To start work, make sure you are up to date with the upstream code: 55 | 56 | ``` 57 | git checkout main 58 | 59 | git pull --rebase 60 | ``` 61 | 62 | Now, start a temporary throwaway "topic branch" for the feature you want to add: 63 | 64 | ``` 65 | git checkout -b memorable_description 66 | ``` 67 | 68 | Do your work. 69 | 70 | ``` 71 | # edit some files 72 | git commit -a 73 | 74 | # more edits 75 | git commit -a 76 | ``` 77 | 78 | If your patch is going for some time and you want to pull in upstream code changes to work along with it, you can always run: 79 | 80 | ``` 81 | git pull --rebase 82 | 83 | # OR ... 84 | 85 | git config pull.rebase true 86 | 87 | git pull 88 | ``` 89 | 90 | If you always rebase, this keeps your patches on the top of the stack, which will be important in the next step. 91 | 92 | Once you have made sure your patch works, is well tested, and free from warnings, you are ready to think about submitting it. 93 | 94 | You should make sure your patch looks correct before emailing it. Ensure sure it includes only changes related to the one idea you are working on, and does not contain work from previous efforts. 95 | 96 | Use "git log | less" to see how many commits you made, and if it is just one commit, the command you wish to run next is: 97 | 98 | ``` 99 | git format-patch HEAD^ 100 | ``` 101 | 102 | Add two carets if there are two commits, etc. 103 | 104 | Take a look at the patch and see if it contains what you want. 105 | 106 | Once you decide your patch looks good, you are ready to send it. You cannot use your mail client to do this, and instead will run git-send-email using the exact same pattern as above. Copy this command (except maybe for the number of carets). 107 | 108 | ``` 109 | git config sendemail.to "~mpdehaan/jetporch-devel@lists.sr.ht" 110 | 111 | git send-email HEAD^ 112 | ``` 113 | 114 | This will open your editor (such as vim, emacs, etc) and allow you to add a description describing the patch set. Make sure to explain what the purpose of the patch is and any relevant info. 115 | 116 | If there are any questions or comments, [Michael](https://home.laserllama.net) will reply to the patch. [Status of the patch is also shown in the web interface](https://lists.sr.ht/\~mpdehaan/jetporch-devel). If your patch needs changes, the process is to submit a new patch, which can simply include additional commits on the same topic branch. 117 | 118 | ``` 119 | git send-email --annotate -v2 HEAD^^ 120 | ``` 121 | 122 | The number of carets is how many commits to include in the patch. If you want to get fancy, you could use git commit -a --amend to update the last changeset instead. 123 | 124 | Now you can switch back to the main development branch, submit other patches that don't depend on the outstanding patches, and so on. It's easy to have multiple patches and features working at the same time this way. 125 | 126 | ## About Patch Discussion 127 | 128 | The patch submission list is a review list, not a full discussion list about future possible ideas or how internals work. As such, replies to patches should be about the patch in question, and all topics send to the patch list should contain a patch. 129 | 130 | Discussion not about code review should happen on [Discord](discord-chat.md). Be sure to join the development channels! 131 | 132 | Discussion of features ahead of submitting patches is strongly recommended, this is not required for bugfixes, though it can still be helpful depending on complexity. The more discussion happens, the less code review and modifications need to happen, so in general, up front discussion is _great_. 133 | 134 | Thank you and we look forward to patches from you! If you hit any problems or have git questions of any kind, feel free to ask in chat, we want everyone to have a good time with this process! 135 | 136 | -------------------------------------------------------------------------------- /playbooks/logging.md: -------------------------------------------------------------------------------- 1 | # 📒 Logging 2 | 3 | Starting with the Tech Preview 2 release, jetp emits structured JSON logs to /var/log/jetp/jetp.log or, alterately, a path specified by the environment variable JET\_LOG. These logs are intended for consumption by software, not humans, with one JSON entry per line. Different event times will have different levels of detail. 4 | 5 | If the path is not writeable, jet will proceed without error. 6 | 7 | Log rotation with logrotate is strongly recommended as these logs can be particularly large especially when configuring a large number of hosts, processing complex configurations, or running very many builds that all rely on jetp. 8 | 9 | From here, logfiles can be sent to various log aggregrators such as Splunk, Elastic Search, and so on. 10 | 11 | The logs are largely self explanatory and include information on playbook paths and task results, as well as summaries of playbook statuses similar to what is produced by the Jet command line. 12 | 13 | 14 | 15 | 16 | 17 |

18 | 19 | ### JSON Event Types 20 | 21 | 22 | 23 | | | | Host Specific? | 24 | | --------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- | -------------- | 25 | | PLAYBOOK\_START | Playbook began executing | No | 26 | | PLAY\_START | A particular play in a playbook began executing | No | 27 | | ROLE\_START | A particular role began executing | No | 28 | | TASK\_STATUS | Status of a particular task on a particular host that is not error related. TASK\_CHECK\_STATUS is used instead in check mode, or TASK\_FAILED for failed events. | Yes | 29 | | TASK\_CHECK\_STATUS | Same as TASK\_STATUS but is used in check mode operations. | Yes | 30 | | TASK\_FAILED | Same data as TASK\_STATUS but is used when modules fail on particular hosts and the errors are not ignored | Yes | 31 | | HOST\_CONNECT\_FAILED | An event that is registered when it is impossible to connect or login to a particular host. | Yes | 32 | | SUMMARY | A detailed report of statistics from a playbook run. Also signals the playbook is done executing. | No | 33 | 34 | 35 | 36 | 37 | 38 | ### JSON Log Event Fields 39 | 40 | 41 | 42 | Not all event types will have all fields in the JSON entries. Consulting the log examples from a live run is recommended for understanding values. 43 | 44 | 45 | 46 | | Json Field | Description | 47 | | -------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | 48 | | event | the event type string. different event types will have different JSON data members. | 49 | | run | a GUID that can be used to associate all events related to a particular playbook run | 50 | | start | the time the playbook run was initiated in rfc 2822 format (UTC) | 51 | | now | the current time in rfc 2822 format (UTC) | 52 | | elapsed | the elapsed time in seconds into the playbook run, for PLAYBOOK\_COMPLETE events this indicates the complete duration of the run | 53 | | playbook | the "name" field of the playbook being run | 54 | | playbook\_path | the full path to the playbook being run | 55 | | role | the current role name, if any | 56 | | task | the name of the current task. If no name was provided, the name of the module is used | 57 | | task\_ct | Numeric. The number of tasks "deep" the task is into the playbook run. This is not reset between roles | 58 | | cmd | the name of the cmd being run, which is provided for shell module task events and any failures involving cmd statuses. Intermediate command statuses for successful commands during normal module execution are not logged. | 59 | | cmd\_rc | The return code for any command from a shell module, or from a failed module execution | 60 | | cmd\_out | The output text from a shell module call or a failed module execution | 61 | | task\_status | The status for TASK\_STATUS events. TASK\_FAILED events are seperate. These are shown in another table. | 62 | | host | For host specific actions, this is the name of the host in inventory related to the current event. Some events are global and do not have a host entry. | 63 | | summary | a JSON map of numeric information. This gives a detailed statistics report on the playbook run, including totals for various types of host states and how many modifications were performed, similar to the table at the end of the human-readable jet output. | 64 | 65 | 66 | 67 | 68 | 69 | 70 | 71 | -------------------------------------------------------------------------------- /.gitbook/assets/file.excalidraw (13).svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /modules/commands.md: -------------------------------------------------------------------------------- 1 | # ❗ Commands 2 | 3 | Command modules execute programs or run scripts where there is no declarative module available. Imperative (aka "non-declarative") actions in Jet are _not_ even remotely discouraged, and various parameters and task keywords can be used to make them behave more like declarative resource models as is shown below. 4 | 5 | ### Command Module Index 6 | 7 | | Module | Description | 8 | | ---------------------------- | -------------------------------------------------- | 9 | | [script](commands.md#script) | Push and run scripts and dynamic language programs | 10 | | [shell](commands.md#shell) | Run individual command line commands | 11 | 12 | Command modules run commands or scripts on the indicated managed system. These are particularly useful when combined with [task keywords](../playbooks/tasks-and-task-modifiers.md). 13 | 14 | ### script 15 | 16 | runs the contents of a script over the shell. 17 | 18 | {% hint style="warning" %} 19 | **Coming Soon** 20 | 21 | This feature will be included with the "Tech Preview 2" release. A quick workaround is to use the _**copy**_ module and then execute it with the _**shell**_ module. 22 | {% endhint %} 23 | 24 | ### shell 25 | 26 | runs a shell command on a machine. 27 | 28 | ``` 29 | tasks: 30 | - !shell 31 | cmd: "/usr/bin/foo --some-arguments 1234" 32 | ``` 33 | 34 | While this is really one of the simplest modules, there is some nuance to dealing with it as it pertains to variable usage and error handling considerations. 35 | 36 |

37 | 38 |

Understanding Variable Safety

39 | 40 | To _completely_ maximize safety, variables should not be put into the "cmd" argument line, however they can be used in safe ways, especially when you are sure variables came from defintiions in the git repository and not from remotely derived values, such as command output from a remote host. When using variables in the cmd parameter, care must be taken to trust _all_ variables in the 'cmd' argument and know where they come from. Variables should also always be quoted, though the program does not enforce this. 41 | 42 | Why not? Well, it's good for accidental errors but it's not a safety feature. Shell quoting of strings containing variables is unsufficient by itself to prevent shell escapes, such as inserting a semicolon and running a stealth "bonus" command. Thankfully, Jet looks out for these potential security problems. Nearly all parameters in Jet on input are already filtered for invalid shell characters. 43 | 44 | By default, if the 'cmd' line contains variables, Jet will also _not_ permit potentially unsafe shell characters to appear in the resultant command input if they appear after the variables are evaluated. This will cause a failure in the playbook for each host where this occurs and the command will not be executed. Therefore the **unsafe** keyword is required to permit those shell characters to appear in situations where unsafe characters appear with variables, whether or not they appear in the variable expressions themselves or between them. 45 | 46 | This is a vital security consideration as information coming back from a remote host and sometimes a cloud environment (such as an instance tag variables set by another user) may be untrustworthy. 47 | 48 | **Unsafe** and remotely-sourced variables should never be used together with the "delegate\_to" keyword, because it would allow a compromised host to run shell commands of it's choosing on the control machine. 49 | 50 | Here is an example of using the unsafe keyword: 51 | 52 | ``` 53 | - !shell 54 | name: the unsafe command is very dangerous! read the warning below 55 | cmd: "/usr/bin/foo --param='{{ somevar }}' > '{{ somepath }}'.log" 56 | unsafe: true 57 | ``` 58 | 59 | When using unsafe, the variables must still be trusted by the user, this keyword only admits that the user is signing off that they are not afraid of the variables being used for shell injections. 60 | 61 |

62 | 63 |

64 | 65 |

Controlling When Failures Are Returned

66 | 67 | Often a command may return a non-zero error code in a scenario that does not indicate a failure condition. Non-zero commands normally fail the task in the Jet play and keep it from continuing. A good example of this is grep which returns 0 on matching some text and 1 on not finding a match and we may wish to ignore this error. 68 | 69 | In these scenarios, _**failed\_when**_ can be used on the task. 70 | 71 | [The following example is available on GitHub](https://github.com/jetporch/jetporch\_examples/blob/main/playbooks/shell\_tricks.yml) and can be used as the basis for other experiments -- this puts together conditionals, variable evaluation, and various shell module parameters: 72 | 73 | ``` 74 | tasks: 75 | 76 | - !shell 77 | cmd: grep -i "chickens" /etc/motd 78 | failed_when: (gt rc 1) 79 | changed_when: false 80 | save: grep_result 81 | 82 | - !echo 83 | msg: "grep result was {{ grep_result.rc }}" 84 | 85 | - !echo 86 | msg: "the motd mentioned chickens" 87 | with: 88 | condition: (eq grep_result.rc 0) 89 | 90 | - !echo 91 | msg: "the motd did not mention chickens" 92 | with: 93 | condition: (eq grep_result.rc 1) 94 | 95 | ``` 96 | 97 | The **failed\_when** parameter expression has access to all variables in the host that normal variable templating would have and is a handlebars expression as described in the documentation for [condition](../playbooks/tasks-and-task-modifiers.md). 98 | 99 | In addition to all additional variables, **failed\_when** is given two more variables which are the ones most users will want to use - **rc**, the return code of the command, and **out**, the full text output of the command, merging standard out and standard error into one string. 100 | 101 | Without **save**, also shown, the variables do not persist for access in subsequent tasks. Save requires a variable name to store the values in. 102 | 103 | **failed\_when** can also be used to completely disable return code checking, much more simply, like so: 104 | 105 | ``` 106 | tasks: 107 | - !shell 108 | name: completely ignore the error code 109 | cmd: /usr/bin/foo 110 | failed_when: false 111 | ``` 112 | 113 | 114 | 115 |

116 | 117 |

118 | 119 |

Controlling When Changes Are Reported

120 | 121 | Finally, many users like the the idea of a playbook reporting that no changes needed to be made on remote hosts as a sign that hosts match desired policy. Since the grep command does _not_ make changes on any remote hosts, we can mark it as such with **changed\_when**. This way, the playbook summary at the end of the playbook run will give better visibility of what changes Jet actually made on the host. 122 | 123 | ``` 124 | - !shell 125 | cmd: grep -i "chickens" /etc/motd 126 | failed_when: (gt rc 1) 127 | changed_when: false 128 | save: grep_result 129 | ``` 130 | 131 | Should we wish to mark another command as conditionally changing things based on return code, failed\_when uses the same conditional syntax used above and also described in detail with the '[condition](../playbooks/tasks-and-task-modifiers.md)' task keyword. 132 | 133 | ``` 134 | - !shell 135 | cmd: /usr/bin/arbitrary_cmd --example-only 136 | failed_when: (gt rc 1) 137 | changed_when: (eq rc 1) 138 | save: grep_result 139 | ``` 140 | 141 | 142 | 143 |

144 | 145 | | Parameter | Details | 146 | | ----------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------- | 147 | | **cmd** | The command to run | 148 | | **failed\_when** | Handlebars expression using 'rc' and 'out' to decide whether the command is a failure. If not provided, errors are when the return code is non-zero | 149 | | **changed\_when** | Handlebars expression to decide whether the task effected a change on the remote host. If not provided, the system will assume yes. | 150 | | **save** | Stores the result of the command in a variable, \.rc and \.out are available permanently in the host's namespace until overwritten. | 151 | 152 | -------------------------------------------------------------------------------- /developer-guides/development-tips-intro.md: -------------------------------------------------------------------------------- 1 | # 🔢 Conventions & Workflow 2 | 3 | Thanks for interest in working on Jet's source code! Understanding some of the following topics may make for a better development and contributing experience. Also consider reading [Contributing](../community/contributing.md) if you haven't already! 4 | 5 |

6 | 7 |

For New Rust Developers

8 | 9 | Whether you are familiar with Rust or not, we're glad to have you. 10 | 11 | Just as early devops tools exposed new people to Python and Ruby, we also hope Jet helps develop an interest in Rust among new audiences. Rust is largely a systems language, and sometimes development is slower, but Rust also reduces the time spent finding errors, such that written code often sometimes works the very first time like magic. We love it for this. 12 | 13 | A few concepts occur fairly commonly in Jet code and warrant review - you may be able to pick these up just by skimming the code - but may also want to read about them elsewhere online. 14 | 15 | 1. _Arc_ is a smart pointer to a value on the heap, and Arcs are used extensively throughout the program. Various values are not able to be passed around the stack or as references, particuarly with the concurrency goals of the program in SSH mode. 16 | 2. We use _Mutex_ and _RwLock_ to manage concurrent access to objects. Mutex allows for exclusive read or write access to one thread at a time. RwLock allows access to any number of readers but only one writer. We use RwLock unless a Mutex is specifically required. We use Mutex around SSH Connections and occassionally to keep output together, but that is mostly it. 17 | 3. As with most Rust programs, we use _Result_<_OkType,ErrorType_> to indicate a value may return either a given result or an error. Similarly _Option_ is used to represent a return value that may be None. 18 | 4. We rather heavily use _match_ for case-like statements, especially when working with Result and Option results. There are a lot of shortcuts in Rust like unwrap\_or and such, but often match is the clearest, and it works for everything, so we prefer using match in almost all cases. 19 | 5. We use a lot of Rust collections like _HashMap_ and _HashSet,_ as well as iterators like map. 20 | 6. We use a fair amount of _Enums_. We often use Enums instead of booleans for better clarity when reading code that calls a function that would take a boolean as a parameter, this way we don't have to remember what _true_ means for the third argument to a function call. The list of valid modules in Jet that you put under "tasks:" is implemented as an Enum. 21 | 7. The question mark operator "_?_" is used to send the error portion of a result up to the current function when an error occurs, allowing for more concise code when the error has the same return type as the outer function itself. For this reason, we try to keep the return types the same in most cases. The Rust compiler will warn you when a result is ignored, so this is not something you will likely forget to leave out, but it is always worth being in the mindset of thinking "what am I doing with the errors from this function?" 22 | 23 | If you have any Rust questions at all, or just want to talk about Rust, stop by the #rust channel in our chat. There are no bad questions! 24 | 25 |

26 | 27 |

28 | 29 |

Jet Architecture/Code Basics

30 | 31 | Regardless of development background, a few particular design details about the program are important to note: 32 | 33 | 1. Inside of modules, both Ok results and errors often use an object called _TaskResponse_ ('src/tasks/response.rs'), usually shared with an _Arc_ and generated by the _TaskHandle_. There's a fair amount of code to convert errors into TaskResponses. Noticing this pattern will help module development make sense, because you will not be able to return results/errors of other types inside the main module file. 34 | 2. Inside of modules, 'src/tasks/handle.rs' (TaskHandle) provides user access to most common functions and ensures consistency between modules. Think of this as the "power tool" interface to making modules easy. Using Task Handle also makes sure modules stay on the right track and have common conventions. Various features of task handle are namespaced into other structs hanging off of task handle, such as template functions for processing parameters, or 'response' for shortcuts around creating return objects. 35 | 3. Outside of modules, such as when parsing CLI arguments, most errors are simple strings. 36 | 4. The best way to understand the internal control flow of the entire program is to start with main.rs and notice that based on the "cli/cli\_parser.rs" code, various functions in main.rs are selected that behave differently. For CLI modes involving playbooks (most of them), common traversal code in "playbooks/traversal.rs" is used to walk over the playbook tree, and these are set up in cli/playbooks.rs. Execution of tasks, including parallel execution of SSH tasks, lives in "tasks/task\_fsm.rs" - and this is what makes the "dispatch" functions of modules work. These are all of the guts of the system before you get to modules, which are the easy part that can remain blissfully oblivious of everything behind the scenes! See the [module development guide](module-development.md) for module info as this will be the most frequently adjusted part of the program! 37 | 5. Inventory is a critical concept for SSH management only, but there is still a "localhost" object and "local connections" that are used for local management options. This local management is still used in SSH operations for some internal tasks, like identifying local checksums of files for the copy module. Some familiarity with connections may be useful. When you see references to "remote", even in local context, it refers to the connection object. Local always refers to the machine running 'jetp'. 38 | 6. The Rust code is almost never allowed to call panic! except in cases of a module being coded incorrectly according to the contract, or code that is essentially impossible to execute without a major coding error in the program. Panic would stop all tasks vs just failing the current thread. Module execution in SSH is distributed among many threads, we return an Err(TaskResponse) instead when we encounter an error situation. This allows us to report the failure and continue on other hosts. Sometimes you will see panic in some places of the code that are impossible to reach conditions that need some code there to satisfy the compiler. 39 | 40 | With that out of the way, see the other sections in the docs for details about other aspects of the program. If your question is still unanswered, hop by Discord and we'd be glad to explain and talk about anything. 41 | 42 |

43 | 44 |

45 | 46 |

Development Practices

47 | 48 | 49 | 50 | Editor choice is up to you. Michael uses a completely stock VSCode install as he finds some of the overlays from popular Rust plugins distracting. The "rainbow" colored parens in VSCode can be useful when dealing with some nested quasi-functional-programming expressions. 51 | 52 | 1. Remember that all style and preferences are subjective. That being said, to get contributions accepted you need to **turn off auto formatting in your editor** and do not run the code through tools like rustfmt. Reformatting an entire file is to be avoided. Generally try to match the style of the rest of the program, which tries to be a bit more concise than rustfmt. 53 | 2. Make sure all code is free of rust compiler warnings on compilation. Compile by running "make" in the root of the source checkout. 54 | 3. We generally do not care about warnings from tools like clippy. Fixing some of them may be nice but we disagree with others, so leave these changes up to project leadership and do not include them in pull requests. We prefer explicit return statements. 55 | 4. Try to make the code as clean, clear, concise, and self-explanatory as possible. Interior comments on difficult to understand code are welcome, especially when the Rust compiler wishes to make things look a little strange. 56 | 5. Think about future developers other than yourself and make things easy for them. Choose high quality variable names and function names. Matching Ok(x) is more than fine as is sometimes using "x" on an iterator, but it is especially important to give "let" variables descriptive variable names. There is no need to comment things that would be obvious to those that know rust, or to explain the purposes of function names that are already self explanatory, but a one or two line comment per function never hurts. 57 | 6. Consider the error paths, return types, and security implications of your code very carefully. This will be a major part of code review. The Rust compiler helps here a lot though! 58 | 7. Try very hard to avoid introducing new crate dependencies, especially those that are not widely adopted. Introduction of new depedencies is always going to be a needs-chat-discussion point. The good news is, dependencies in module development in particular are seldom useful, as in SSH mode we do not push code or programs out to remote machines. Where we can interface with a stable OS CLI tool, that is always preferred. Since all code in SSH mode runs on the control machine, we cannot use dependencies to effect change directly on remote systems, even though that would technically work well in local configuration modes, it is not allowed to have module features that only work locally. 59 | 8. See the Contributing notes in full and understand all of the items. Per our [Contributing page](../community/contributing.md), we strongly suggest joining Discord chat if you wish to submit a feature or something more than a simple bugfix. 60 | 9. There are no wrong questions! Ask questions if you want to understand how _anything_ works or get stuck in understanding something! We enjoy conversation and want to hear from you, and also want to make your experience with jet fun and successful. 61 | 62 |

63 | 64 | If there are any questions at all, join the development channels on [Discord](../community/discord-chat.md) and we would be glad to help! 65 | 66 | -------------------------------------------------------------------------------- /modules/package-managers.md: -------------------------------------------------------------------------------- 1 | # 🎁 Package Managers 2 | 3 | The package management modules wrap package manager commands for easier, more declarative use. 4 | 5 | ### Package Manager Module Index 6 | 7 | 8 | 9 | | Module | Description | 10 | | ---------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------- | 11 | | [apt](package-managers.md#apt) | Package management for Debian and derivatives like Ubuntu | 12 | | [dnf](package-managers.md#dnf) | Package management for current "Enterprise Linux" OSes and Fedora | 13 | | [homebrew](package-managers.md#homebrew) | Package management for OS X | 14 | | [pacman](package-managers.md#pacman) | Package management for Arch Linux | 15 | | [yum](package-managers.md#yum) | Package management for "Enterprise Linux" 7 and before. (Actually an alias, this is the same underlying Jet module as 'dnf') | 16 | | [zypper](package-managers.md#zypper) | Package Management for SuSE distributions | 17 | 18 | ### apt 19 | 20 | Installs packages on Debian and Ubuntu systems with the apt package manager. 21 | 22 | {% hint style="warning" %} 23 | **Apt-Cache Information** 24 | 25 | the apt-module does not run "apt update" for you, if this is important (and it should be), use the shell module to do it prior to running apt module tasks. 26 | {% endhint %} 27 | 28 | ``` 29 | tasks: 30 | 31 | - !apt 32 | package: redis 33 | 34 | - !apt 35 | package: redis 36 | version: 5:6.0.16-1ubuntu1 37 | 38 | - !apt 39 | package: redis 40 | update: true 41 | 42 | - !apt 43 | package: redis 44 | remove: true 45 | ``` 46 | 47 | 48 | 49 | | Parameter | Description | 50 | | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | 51 | | **package** | Required. Name of the package to install or remove | 52 | | **version** | Optional version of the package to install or upgrade to. Must be matched exactly. | 53 | | **update** | Boolean, default false. If true, will attempt to update an already installed package to the latest version in the repo or the given version, if version is specified. | 54 | | **remove** | Boolean, default false. If true, will remove the package if installed rather than attempting installation if not installed. | 55 | 56 | ### dnf 57 | 58 | Installs packages for "Enterprise Linux" flavor operating systems using the dnf package manager. Specific versions can be specified, or not. 59 | 60 | "remove" will remove the package if currently installed. 61 | 62 |

tasks:
 63 | 
 64 |    - !dnf
 65 |      package: tree
 66 |      
 67 |    - !dnf
 68 |      package: tree
 69 |      version: 1.8.0
 70 |      
 71 |    - !dnf
 72 |      package: tree
 73 |      update: true
 74 |      
 75 |    - !dnf
 76 |      package: tree
 77 |      remove: true
 78 |

79 | 80 | | Parameter | Description | 81 | | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | 82 | | **package** | Required. Name of the package to install or remove | 83 | | **version** | Optional version of the package to install or upgrade to. Must be matched exactly. | 84 | | **update** | Boolean, default false. If true, will attempt to update an already installed package to the latest version in the repo or the given version, if version is specified. | 85 | | **remove** | Boolean, default false. If true, will remove the package if installed rather than attempting installation if not installed. | 86 | 87 | ### homebrew 88 | 89 | Installs Mac packages with homebrew. 90 | 91 |

tasks:
 92 | 
 93 |    
 94 |    - !homebrew
 95 |      package: tree
 96 |      
 97 |    - !homebrew
 98 |      package: tree
 99 |      remove: true
100 |

101 | 102 |

Parameter	Description
package	Required. Name of the package to install or remove
version	Optional version of the package to install or upgrade to. Must be matched exactly.
update	Boolean, default false. If true, will attempt to update an already installed package to the latest version in the repo or the given version, if version is specified.
remove	Boolean, default false. If true, will remove the package if installed rather than attempting installation if not installed.

103 | 104 | ### pacman 105 | 106 | Installs packages for Arch Linux operating systems using the pacman package manager. With pacman, specifying specific versions and using 'update' may potentially break system packages because of Arch's rolling release system. Consider running "pacman -Syu" for updates. 107 | 108 |

tasks:
109 | 
110 |    
111 |    - !pacman
112 |      package: extra/tree
113 |      
114 |    - !pacman
115 |      package: extra/tree
116 |      remove: true
117 |

118 | 119 |

Parameter	Description
package	Required. Name of the package to install or remove
version	Optional version of the package to install or upgrade to. Must be matched exactly.
update	Boolean, default false. If true, will attempt to update an already installed package to the latest version in the repo or the given version, if version is specified.
remove	Boolean, default false. If true, will remove the package if installed rather than attempting installation if not installed.

120 | 121 | ### yum 122 | 123 | At some point at time "Enterprise Linux" distros and Fedora changed their package manager while keeping the commands and interface the same. Since they mostly work the same, internally in Jet, we can run mostly the same code to support both modules. Internally the Jet module will see if the /usr/bin/yum or /usr/bin/dnf command is available and run the correct command as appropriate. This means the module may be referenced as "!dnf" or "!yum" in Jet playbooks and it makes no difference, it will run the correct command either way. 124 | 125 | There is no need to write playbook logic to figure out whether to use !dnf or !yum when configuring a server, nor to use the facts module prior to using either module. Take care though - various configuration files for the package manager still do have different locations. 126 | 127 | ``` 128 | tasks: 129 | 130 | - !yum 131 | package: tree 132 | 133 | # .. and so on, see examples for the dnf module 134 | 135 | ``` 136 | 137 | 138 | 139 | | Parameters | Description | 140 | | ---------------------------------- | ---------------------------------- | 141 | | See [dnf](package-managers.md#dnf) | See [dnf](package-managers.md#dnf) | 142 | 143 | ### zypper 144 | 145 | Installs packages on SuSE using the zypper package manager. 146 | 147 | ``` 148 | tasks: 149 | 150 | - !zypper 151 | package: tree 152 | 153 | # arguments work like all the other packaging modules 154 | 155 | ``` 156 | 157 | 158 | 159 | | Parameter | Description | 160 | | ----------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------- | 161 | | **package** | Required. Name of the package to install or remove | 162 | | **version** | Optional version of the package to install or upgrade to. Must be matched exactly. | 163 | | **update** | Boolean, default false. If true, will attempt to update an already installed package to the latest version in the repo or the given version, if version is specified. | 164 | | **remove** | Boolean, default false. If true, will remove the package if installed rather than attempting installation if not installed. | 165 | 166 | ### 167 | 168 | -------------------------------------------------------------------------------- /.gitbook/assets/file.excalidraw (15).svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /playbooks/using-variables.md: -------------------------------------------------------------------------------- 1 | # 🔁 Using Variables 2 | 3 | Variables are values that can be substituted in templates (via the template module) and logic throughout Jet's [playbook](playbook-overview.md) language. _Most_ strings in Jet are templated for evaluation, so variables can be used almost anywhere in the system. 4 | 5 | Variables can be set in [inventory](broken-reference) -- in which case they can come from on disk or discovered from cloud parameters or in [playbooks](broken-reference). 6 | 7 | 8 | 9 | 10 | 11 |

12 | 13 | 14 | 15 | 16 | 17 | ### Use In Tasks 18 | 19 | \ 20 | Variables set elsewhere in the playbook can be directly used in playbook statements like so: 21 | 22 | 23 | 24 | 25 | 26 | ``` 27 | tasks: 28 | - !shell 29 | cmd: /usr/bin/example --argument1 {{ argval }} --argument2 asdf 30 | ``` 31 | 32 | 33 | 34 | The templates are processed by the Rust [handlebars](https://docs.rs/handlebars/latest/handlebars/) library, passing through all variables that are defined in the Jet system. 35 | 36 | 37 | 38 | {% hint style="info" %} 39 | **Where Variables Can Be Used** 40 | 41 | Templating in Jet is very deliberate by design to encourage predictable and highly auditable content. 42 | 43 | Only module _parameters_ in Jet are templated. For reasons of clarity, it is not possible to use variable expressions in the "_**Play**_" object (like the _**vars**_ keyword, or in paths to **vars\_files**) or nest the values of variables inside _other_ variables. In other words, variable evaluation is not recursive. 44 | 45 | All of these decisions were made to make playbooks extremely predictable, avoiding many possibilities of hidden errors or surprises. 46 | 47 | Use variables when needed, but it is important to also realize that Jet is not meant to be a programming language, and the system is designed to enforce "best practices" in automation design by sometimes limiting the ability to do confusing things. There may be times when you might want to generate a _**vars\_files**_ file for Jet from a script if you need data denormalized in many different ways. 48 | 49 | Use of templates parameters in unsupported areas will produce largely obvious and up-front errors messages, such as a YAML boolean field not accepting a string value, allowing for easy correction. 50 | {% endhint %} 51 | 52 | 53 | 54 | 55 | 56 | ### Use In Templates 57 | 58 | 59 | 60 | The template module also uses Rust's handlebars library to describe the files it wants to write on the remote machines. An example source template might use basic variables but it can also use other constructs. 61 | 62 | 63 | 64 | ``` 65 | # foo.config template example 66 | 67 | [config section] 68 | xyz = {{ xyz }} 69 | backup_enabled = true 70 | 71 | {{#if (eq abcd debian}} 72 | tribbles = auto 73 | {{else}} 74 | tribbles = 5000 75 | {{/if}} 76 | 77 | ``` 78 | 79 | 80 | 81 | The Rust handlebars library provides the some compatible block expressions that are described in the Javascript handelbarjs documentation. To keep things simple, these are the most commonly used and appropriate. 82 | 83 | * `{{{{raw}}}} ... {{{{/raw}}}}` 84 | * `{{#if ...}} ... {{else}} ... {{/if}}` 85 | * `{{#unless ...}} ... {{else}} .. {{/unless}}` 86 | * `{{#each ...}} ... {{/each}}` 87 | * `{{#with ...}} ... {{/with}}` 88 | 89 | 90 | 91 | As described with cond in [Tasks](tasks-and-task-modifiers.md), if statements can use the following boolean expressions: 92 | 93 | * `eq` 94 | * `ne` 95 | * `gt` 96 | * `gte` 97 | * `lt` 98 | * `lte` 99 | * `and` 100 | * `or` 101 | * `not` 102 | 103 | 104 | 105 | The '_if_' template example above shows the example of using if with _eq_, which must be surrounded in parentheses. This is also shown in the examples for the 'cond' [task](tasks-and-task-modifiers.md) keyword. The other expressions work the same way. 106 | 107 | In addition to the built-in expressions, Jet adds the following expressions 108 | 109 | 110 | 111 |

Expression	Description	Version Added	Example
contains	is a substring in a string?	0.2	{{ #if (contains x "foo") }}
ends_with	does a string variable end with a given string?	0.2	{{ #if (ends_with x "foo") }}
isdefined	is the given variable name defined?	0.2	{{ #if (isdefined foo) }}
to_lower_case	returns the string in lowercase	0.2	{{ to_lowercase foo }}
to_upper_case	returns the string in uppercase	0.2	{{ to_uppercase foo }}
starts_with	does the string variable start with the given string	0.2	{{ #if (starts_with x "foo") }}
trim	return the string without leading or trailing spaces	0.2	{{ trim foo }}
trim_start	return the string without leading spaces	0.2	{{ trim_start foo }}
trim_end	return the string without tailing spaces	0.2	{{ trim_end foo }}

112 | 113 | With the exception of isdefined, these functions directly mirror their behavior in the Rust language. 114 | 115 | A combined usage example showing some of these concepts put together: 116 | 117 | ``` 118 | {{#if (isdefined foo) }} 119 | foo={{ to_lower_case foo }} 120 | {{/if }} 121 | 122 | {{#if (contains bar "elephants") }} 123 | elephants=1 124 | {{/if }} 125 | ``` 126 | 127 | {% hint style="warning" %} 128 | **No Recursive Evaluation** 129 | 130 | To encourage predictability and easily auditable content, templates in Jet are _**not**_ evaluated recursively, but only a single time. If you think you need variable indirection where a variable has another variable name in it, consider restructuring your data. Where neccessary, variables such as paths that were stored this way... 131 | 132 | base\_dir: /opt/apps 133 | 134 | app\_path: "\{{ base\_dir \}}/foo/bar" 135 | 136 | Can be recomposed like so: 137 | 138 | base\_dir: "/opt/apps" 139 | 140 | app\_sub\_dir: "/foo/bar" 141 | 142 | And then inside of the template file or module, just do this: 143 | 144 | app\_path=\{{ base\_dir}/\{{ app\_sub\_dir \}} 145 | {% endhint %} 146 | 147 | 148 | 149 | Users familiar with Jinja2 will note that the variable dereferencing (double curly brackets) are the same but the other operations are not. Handlebars is a deliberately more minimal system, and is not designed to be used in a 'programming' type capacity. Over time we may provide some additional helper functions for templates, but will try to keep this very constrained to encourage predictable read-throughs of automation content. 150 | 151 | ### Advanced Topics 1: Variable Precedence 152 | 153 | If learning Jet, you may not need to understand this immediately. However, as you get further along, you will probably need to be aware of these details. 154 | 155 | 156 | 157 | Variable precedence works like this: 158 | 159 | 160 | 161 |

162 | 163 | It is possible to define a variable in many locations throughout Jet. When a variable is referenced, how do we decide what value is used? The variable source closer to the top of this list wins: 164 | 165 | 1. Variables used on the command line always win. 166 | 2. Playbook variables have the next highest priority. 167 | 1. vars\_files 168 | 2. vars 169 | 3. Inventory variables have middle priority, and the is based on where the variable is in the inventory tree: 170 | 1. Variables defined on a [Host](../inventory/hosts.md) system will beat out _Group_ variables. Host variables include variables stored as the result of a 'save' task action when running a module. 171 | 2. Variables defined on a [Group](../inventory/groups.md) will be used if the same variable isn't set on the Host. If there are subgroups that repeat variable names, the most specific group wins over the larger parent group. For example values in "london" (a city group) would win over "england" (a country group). 172 | 4. Default variables have the lowest priority. 173 | 1. Variables defined in the 'default' keyword of a playbook 174 | 2. Variables defined in the 'default' folder for a [Role](roles.md) 175 | 176 | 177 | 178 | {% hint style="info" %} 179 | **How can I avoid commiting this to memory?** 180 | 181 | We hear you! - variable precedence is a neccessary evil because it exists and people will want to ask questions about it, but really, you're not meant to rely on it too much. Defining variables in the right place makes things easier, so we might suggest: 182 | 183 | Keep variables about _behavior and purpose_ inside of roles and role defaults. 184 | 185 | Keep variables about _location and environment_ (stage vs prod, or customer vs customer) in inventory. Understand precedence hierarchy with respect to inventory. 186 | 187 | Use vars/vars\_files for things that are essentially constants that never change between location, environment, or type of system, and do not need to be repeated between playbooks, such as things to be used in simple on-off playbooks. 188 | 189 | Use -e for things like release versions, but otherwise only sporadically. 190 | {% endhint %} 191 | 192 | ### Advanced Topics 2: Variable Merging 193 | 194 | Here's another topic you may not need to even use, but should be aware of if you try it. 195 | 196 | If a _list-type_ variable is defined in multiple places, the contents of both lists will be added together. 197 | 198 | For instance, a default group variable could define a default list of base packages and an inventory subgroup variable would add more base packages to it rather than replacing the original list. 199 | 200 | Similarly, if a hash/mapping/dictionary variable is defined in multiple places, the keys of the hash/mapping/dictionary will be merged together creating a larger hash. If any keys are used between the two variables, the one with greater precedence will overwrite the keys of the lesser. 201 | 202 | -------------------------------------------------------------------------------- /inventory/dynamic-cloud-inventory.md: -------------------------------------------------------------------------------- 1 | # ☁ Dynamic/Cloud Inventory 2 | 3 | In nearly all real-world production cases, managed computers, nodes, or systems (whatever we call them) dynamically come and go at all times, and what is currently present is rarely defined by files on disk or in version control. Cloud tools may create systems from CI/CD workflows, or systems may be controlled by autoscaling groups. 4 | 5 | In these cases, Jet's [on-disk inventory format](file-based-inventory.md) cannot be the "source of truth" for the list of groups, hostnames, and often even variable information. 6 | 7 | {% hint style="info" %} 8 | **Reminder About When Inventory Is Needed** 9 | 10 | In local mode and in the future [agent](../connectivity/agent-mode.md) modes, inventory is **not required** and this section can be skipped. Inventory is an [SSH](../connectivity/ssh-mode.md)-only feature. 11 | {% endhint %} 12 | 13 | SSH Dynamic inventory in Jet works by calling one or more executable scripts or programs that return a specific JSON format to tell jet about groups, hosts, and variables. 14 | 15 | 16 | 17 |

18 | 19 | 20 | 21 | To use an inventory script, we simply invoke jet passing the executable file as a parameter: 22 | 23 | ``` 24 | $ chmod +x ~/some_script.py 25 | $ jetp show-inventory --inventory ~/some-script.py --show-groups all 26 | $ jetp ssh --inventory ~/some_script.py --playbook pb1.yml 27 | ``` 28 | 29 | These scripts could contact a cloud provider, commercial CMDB, in-house database, or essentially any program. All that matters is they return the correct JSON data on standard output. Development of your own in-house inventory scripts (or forking our own) is encouraged, should available ones (detailed below) not work exactly how you like. 30 | 31 | {% hint style="info" %} 32 | **Adding Variables From Files** 33 | 34 | Inventory variables from dynamic inventory can still be supplemented with variable information from the filesystem. 35 | 36 | If the inventory script has a "group\_vars/" or "host\_vars/" directory in the same directory as the inventory script and parallel to it, they will added to the groups and hosts returned by the inventory script. 37 | 38 | This is sometimes much easier and clearer than trying to assign variables through something like cloud tags, which some inventory scripts support. 39 | 40 | See [Inventory Variables On Disk](inventory-variables-on-disk.md) for how this works. 41 | {% endhint %} 42 | 43 | ## Where To Find Inventory Scripts 44 | 45 | While you can write your own inventory scripts, you can find several for popular inventory sources in the ['contrib/inventory' directory of the main repo checkout](https://git.sr.ht/\~mpdehaan/jetporch/tree/main/item/contrib/inventory). 46 | 47 | Because the inventory script specification is based on JSON, it is relatively easy to write inventory scripts in a wide variety of programming languages. If you write an inventory script for a popular computing platform, we would be interested in adding it to the collection. 48 | 49 | For ease of collective maintenance, we would accept python and Rust in tree. Because these modules will usually be waiting on external APIs, there is _not_ much need for them to be implemented in Rust, but Rust's compiler checks are great to have and encourage more refactoring and keeping things up to date compared to python. Still, you may find python easier to write and that is also fine! 50 | 51 | {% hint style="info" %} 52 | **Legacy Code Still Present?** 53 | 54 | This folder of inventory scripts was started as a fork of the Ansible inventory scripts from Ansible 2.9, the last Ansible release that contained stand-alone inventory script examples in tree before moving to an ansible-specific plugin architecture. 55 | 56 | Over time, any legacy references to "module\_utils" from the ansible codebase can be removed until it ends up being a self-contained set of scripts without any dependencies on "lib/jeti". Most of "lib/jeti" deals with (unwanted) python 2.X support, which is not important to the Jetporch project, since we can assume a newer control machine and do not require python to be installed on remote systems. 57 | 58 | Also, when looking at scripts in the project you may see many different parsing command line options. Jet does _not_ use these, and many of these were added for debugging purposes or to support a legacy Ansible parameter "--host" that Jet does not use. These can also be removed over time from Jeti. [See the readme in contrib/inventory](https://git.sr.ht/\~mpdehaan/jetporch/tree/main/item/contrib/inventory/README.md) for more information. 59 | 60 | _Ansible is a registered trademark of Red Hat, Inc._ 61 | {% endhint %} 62 | 63 | ## Dynamic Inventory Specification 64 | 65 | With some discussion of Jeti out of the way, let's describe the JSON format if you want to write your own inventory script instead of using one that has already been developed. 66 | 67 | At the most basic level, the inventory script should return a JSON map (also called a "hash table" or "dictionary") with each top level key being the name of a Jet group. 68 | 69 | Each element can contain different elements, _all_ of which are optional. We'll start off with an example that just shows _children_ and _hosts_. 70 | 71 |

{
 72 |     "webservers": {
 73 |         "children" : [   
 74 |             "webservers_raleigh",
 75 |             "webservers_newyork"
 76 |         ],
 77 |     },
 78 |     "webservers_raleigh" : {
 79 |         "hosts" : [
 80 |             "raleigh-www-01.example.com",
 81 |             "raleigh-www-02.example.com"
 82 |         ],
 83 |     },
 84 |     "webservers_newyork" : {
 85 |         "hosts" : [
 86 |             "newyork-www-01.example.com",
 87 |             "newyork-www-02.example.com"
 88 |         ]
 89 |     },
 90 |     "dbservers" : {
 91 |         "hosts" : [
 92 |              "db1.example.com",
 93 |              "db2.example.com"
 94 |         ]
 95 |     }
 96 | }
 97 |

98 | 99 | In the above example, the JSON defines multiple groups - in this example, _webservers, webservers\_raleigh, webservers\_newyork_, and _dbservers_. 100 | 101 | The valid subkeys "_children_" and "_hosts_" are both optional. For example, the group "webservers" has no hosts as direct members, but it does contain subgroups that have hosts. 102 | 103 | {% hint style="info" %} 104 | **More About Calling Conventions, Input, and Output** 105 | 106 | How should an inventory script behave? It should merely output JSON when called. 107 | 108 | There is no need to have sorted keys or pretty-print the JSON, this is only shown for explaining the format. It might be helpful to pretty-print JSON if called with a particular flag, but Jet will not pass any flags to the script or program. Inventory programs/scripts in Jet do not pass any arguments or set any environment variables. 109 | 110 | Any data loading from a configuration file, environment variable, and so on is up to that particular inventory script/program to decide. Typically they will load a file with the same base filename as the inventory script and a different extension. 111 | 112 | If the program/script would like to return an error, it can return a non-zero exit code and any standard out or standard error messages from that failure will be shown to the user. 113 | {% endhint %} 114 | 115 | ### Adding Group Variables 116 | 117 | Variables can be added to any group by adding a "vars" keyword to the group definition. The "vars'" must be a map/hashtable/dictionary as shown below, and can contain nested values, including hashes and arrays: 118 | 119 | ``` 120 | "webservers_raleigh" : { 121 | "hosts" : [ 122 | "raleigh-www-01.example.com", 123 | "raleigh-www-02.example.com" 124 | ], 125 | "vars" : { 126 | "foo" : "bar", 127 | "baz_port" : "8080" 128 | } 129 | } 130 | ``` 131 | 132 | 133 | 134 | {% hint style="info" %} 135 | **Must Groups Have Hosts In Them?** 136 | 137 | Groups without hosts in them can still have variables assigned to them, just like groups can have hosts assigned to them without variables. Having a completely empty group without subgroups is not very useful, but it will also not result in an error either. The inventory script loading code in Jet will _not_ warn if a group with variables is never assigned hosts, because it is expected for the script code to construct inventory correctly. 138 | {% endhint %} 139 | 140 | ### Adding Host Variables 141 | 142 | Host specific variables can be added by adding a "hostvars" section to any group. 143 | 144 | ``` 145 | "webservers_raleigh" : { 146 | "hostvars" : { 147 | "raleigh-www-01.example.com" : { 148 | "asdf" : "example value" 149 | } 150 | "raleigh-www-02.example.com" : { 151 | "asdf" : "example value 2" 152 | } 153 | } 154 | } 155 | ``` 156 | 157 | {% hint style="info" %} 158 | **Shortcuts** 159 | 160 | When using "hostvars" in the script's per-group JSON, it is _not_ required to have a seperate "hosts" JSON element, though both will be processed if present. _Hostvars_ could be considered an 'advanced' way to specify hosts. This value can also be an empty dictionary "{}". 161 | {% endhint %} 162 | 163 | {% hint style="info" %} 164 | **Magic Groups?** 165 | 166 | To support compatibility with legacy ansible inventory scripts, host specific variables can also be added to an imaginary top-level group named "\__meta"._ If writing a new inventory script, don't do this. 167 | 168 | Group names that start with a hyphen _will not be created as real addressable groups in Jet._ Jet interprets hyphen-meta as referring to what Jet calls the "all" group, though explicitly saying "all" instead of "\_meta" is preferred. 169 | 170 | Every group in Jet is implicitly added to the "all" group, there is no need to define an "all" group unless wanting to assign variables to that group, in which case they basically would serve as default variables, having a precedence level just a tiny amount higher than the defaults keyword in a playbook. 171 | {% endhint %} 172 | 173 | {% hint style="info" %} 174 | **Magic Variables?** 175 | 176 | Very few variables in Jet are considered magic and these are described in the [SSH chapter](../connectivity/ssh-mode.md). They are all host-specific inventory variables. 177 | {% endhint %} 178 | 179 | 180 | 181 | 182 | 183 | -------------------------------------------------------------------------------- /.gitbook/assets/file.excalidraw (12).svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /appendix/security-approach.md: -------------------------------------------------------------------------------- 1 | # 🔐 Security Approach 2 | 3 | Jet seeks clarity and auditability of automation content, correctness, and predictability in automation content first and then performance second. 4 | 5 | These design goals includes reducing opportunity for accidentally insecure practices with the tool, secrets disclosure, local disk exploits, cryptographic problems, and not over-enabling insider threat risk. This focus will always dominate over other design decisions. 6 | 7 | ### Readability And Insider Threat Mitigation 8 | 9 | This is probably the most important thing we can do - we want _**predictability**_ vs _cleverness_. 10 | 11 | Language flexibility is important, but _**predictability**_ will always be chosen over flexibility. 12 | 13 | We want _anyone_ who does not know Jet but knows Linux/Unix administration to be able to read and mostly understand what a playbook would do. 14 | 15 | We also want it to be hard to obfuscate malicious configuration intent. Further, Jet's internal code itself should be similarly readable. Features that can tend to turn automation into a loose programming environment or make content hard to read will be rejected in the language design process. Compelling features that are hard to maintain will be similarly _not_ be accepted even at the cost of increased user adoption. 16 | 17 | With IT automation in general, there are some (honestly many) theoretical possible cases where disguised 'bad' automation content could be used against a local control system or be used to deceptively automate the deployment of insecure configurations on remote systems, and we want these cases to be more easily caught during automation code review. 18 | 19 | To ensure templates are not abused and remain easy to understand, we use the intentionally constrainted _Handlebars_ as the [template system](../playbooks/using-variables.md). Handlebars is a relatively simple templating system that does not allow as much programming capability in templates by design. Further, in Jet, variable expressions are evaluated only once, so it is easier to tell what the value of a variable could be, rather than a possible recursive evaluation where one variable can contain the value from another (potentially several levels deep). It is similarly not possible to insert plugin code at runtime into the jetp runtime environment, such as by including it in an automation content repository. 20 | 21 | There are also intentionally very few [variable precedence](../playbooks/using-variables.md) rules in Jet so it is easy to understand what values will apply to automation content - _vars_ and _vars\_files_ content in playbooks are always used, _inventory_ variables are next up, and _defaults_ are last. 22 | 23 | [Conditional](../playbooks/tasks-and-task-modifiers.md) (if) expressions use handlebars rather than a more fully featured system that has access to many functions. Handlebars conditional statements look a bit like the RPN notation from an HP calculator, but we are ok with that. 24 | 25 | There is also no dynamic import feature of instruction lists using a variable path, instead dependent files are listed explicitly in [roles.yml](../playbooks/roles.md), and there is no other include mechanism. Dynamic includes could mean that when auditing automation content the reader would have a hard time predicting what instructions were actually executed by the automation content, especially if a remote system were allowed to decide an include path by using a variable. 26 | 27 | Since actions on the local machine could leak information (even by executing 'cat' on files in other directories in verbose mode!) or create symlinks or hardlinks, executing local (control machine) actions during SSH push modes requires an opt-in flag to enable the local action and delegation features (_--allow-localhost-delegation_). 28 | 29 | ### Cryptography 30 | 31 | Jet does not (and will not ever) implement any of it's own cryptographic code and relies on SSL, OS tools, and so on. 32 | 33 | In SSH modes, jet _does_ in-fact support the insecure SSH access by password (--ask-login-password, which is there for mostly for home lab bootstrapping), but since SSH passwords can be grabbed from a hacked sshd binary on a compromised remote target, we strongly recommend only using key based authentication in any organizational setting. 34 | 35 | For future planetary scale modes, we we will rely on the security features and key infrastructure provided by the supported technologies and vendors. 36 | 37 | Our file and copy template operations use SHA-512 to decide when to replace/transfer some files, and there is no other use of these algorithms cryptographically in the main program. This is shared here for those that notice the package dependency and wonder what it is for. 38 | 39 | ### Secret And Variable Management 40 | 41 | Rather than implementing it's own secret management, [jet can load secrets from the environment](security-approach.md#secret-management), allowing interaction with secret management tools such as but not limited to OnePassword's "op run". 42 | 43 | Many secret management programs will be able to redact the values of secrets that appear in a program (in this case, jetp's) output, preventing the chance of leaking information in normal output, though jet also works to almost completely eliminate accidental leakage risk by only making variables available to the template module and not to playbook templating. 44 | 45 | Future modules that take passwords as parameters will have a way to take the variable name as a secret variable as a parameter (coming from the same environment facility) without templating. 46 | 47 | ### Sudo And Other Access Control Tools 48 | 49 | Sudo support in jet allows for configurable execution such that 'sudo' support work with multiple privilege managers other than sudo, using a template pattern where the desired account name is a parameter. 50 | 51 | {% hint style="info" %} 52 | Sudo access with sudo passwords will be added in the Tech Preview 2 release 53 | {% endhint %} 54 | 55 | Because jet modules written in Rust primarily work by executing command line tools on the remote system, it _is_ possible to constrain access via /etc/sudoers to specific commands only for particular remote users - allowing them access, potentially, to only a subject of jet modules and not a run arbitrary commands or arbitrary code with elevated rights. 56 | 57 | While we do not suspect many users will take advantage of this capability, it will be available and is easy to implement using a sudoers file. 58 | 59 | ### Shell Usage 60 | 61 | To prevent shell command injection, jet filters any arguments sent to the shell involving variables to screen for dangerous characters. This is described in the shell module documentation. 62 | 63 | This filtering includes code to prevent spaces in shell arguments to confuse module orders or add additional arguments. The [shell module](../modules/commands.md#understanding-variable-safety) in jet _does_ potentially allow arbitrary execution when variables are used as a parameter to it's 'cmd' argument, but performs strict screening for dangerous shell characters unless an '_unsafe_' parameter is used or no variables are present in the parameters to the module. 64 | 65 | Still, variables should be used with caution within the 'shell' parameter especially when they are used to form command names. This is true of any automation system. As Jet requires relatively full access to the remote system in order to make _meaningful_ changes, such authors of automation content should be trusted personnel. 66 | 67 | ### Disk Usage 68 | 69 | When using the [file and template](../modules/files.md) module in sudo or sudo-replacement modes, jet must write some temporary files to disk, because SFTP does not allow priveledge escalation. In these situations, we transfer files to \~/.jet/tmp in the (remote) user's home directory (using random temporary filenames) before moving them with sudo (or any sudo replacement) and _never_ use /tmp. 70 | 71 | Also in terms of local disk usage, for clarity, Jet _also_ does not have _any_ configuration files, so you always know only the command line options give to Jet affect it's behavior and what Jet will do when executed. 72 | 73 | Jet also tries to have a bare minimum number of places where other input files could be, always explicitly set, so it is clear exactly what files are loaded and used. 74 | 75 | ### Rust 76 | 77 | Rust is known for memory safety, making a large class of insecure coding errors to be basically impossible. 78 | 79 | The compiler is also extremely advanced so we have extensive error handling awareness forcing all errors produced by functions to be manually considered by the developers and can tell when errors are not handled. Similarly, dead code detection is very robust so we can make sure there is no unused code in the program. Jet intends to always be 100% free of compiler warnings, even on the development branch. 80 | 81 | All of this language support allows the code itself to be refactored more aggressively and everything can be designed and modified with a higher degree of intent than if we had to trust in test-driven-development (TDD) coverage percentages to enable refactoring, where we could possibly break error paths and lesser-used features during development. 82 | 83 | Rust does use single-binary static-binary builds (with _dynamic_ linking to openssl), though we will always make it easy to build your own version of the package from source, regardless of whether you have any development knowledge of Rust itself. 84 | 85 | ### Disclaimers, Disclosure, and Security Announcements 86 | 87 | Perhaps the most obvious thing -- deploying secure systems is your responsibility, no framework can do it completely for you. You must understand how to secure that which you deploy. We can only make the framework itself protect your data and be predictable. This system will absolutely let you automate the installation of a wide-open database server on the public internet and many other similar things, and Jet won't know you are doing it. Jet provides plumbing and the ability to automate/move content, it does not check the content itself. 88 | 89 | As is common for open source projects, none of the above represents any sort of claim, warranty, or guarantee on security features. **Usage of Jet is always at your own risk**. No claims about security here are guaranteed, they are merely our efforts to set up a good foundation. Open source code does help, however. 90 | 91 | We can always make errors. Express any security concerns that may affect users directly to michael@michaeldehaan.net. 92 | 93 | Users interested in following security-related announcements (along with general release announcements) can sign up for the _announcements_ subcategory of our newsletter at [https://jetporch.substack.com/](https://jetporch.substack.com). 94 | 95 | 96 | 97 | 98 | 99 | -------------------------------------------------------------------------------- /.gitbook/assets/file.excalidraw (8).svg: -------------------------------------------------------------------------------- 1 | -------------------------------------------------------------------------------- /modules/control-flow.md: -------------------------------------------------------------------------------- 1 | # 🔀 Control Flow 2 | 3 | ### Control Flow Modules Index 4 | 5 | Control flow commands don't _manage_ remote systems, but they can share information with the user and alter the execution of the playbook. 6 | 7 | These are somewhat similar to many of the [logic keywords in tasks](../playbooks/tasks-and-task-modifiers.md), and are frequently used in conjunction with those keywords. 8 | 9 | 10 | 11 | | Module | Description | 12 | | -------------------------------- | ------------------------------------------------------------------------- | 13 | | [assert](control-flow.md#assert) | testing and safeguards | 14 | | [debug](control-flow.md#debug) | help writing playbooks | 15 | | [echo](control-flow.md#echo) | help writing playbooks and explaining steps in playbook output | 16 | | [fail](control-flow.md#fail) | adding safeguards to playbooks or early stops when debugging | 17 | | [facts](control-flow.md#facts) | gathering info about remote systems for use in conditionals and templates | 18 | | [set](control-flow.md#set) | combining variables and conditionally setting variables | 19 | 20 | ### assert 21 | 22 | The assert module can fail a play for a given host if a certain expression criteria applies. 23 | 24 | The assert module, as shown below, works on the same conditional syntax described with _cond_ on the [task page](../playbooks/tasks-and-task-modifiers.md), so we do not need to explain the handlebars expression syntax again here. 25 | 26 | [An example is available in our examples repo for you to play with](https://github.com/jetporch/jetporch\_examples/blob/main/playbooks/learning\_assert.yml) 27 | 28 | ``` 29 | tasks: 30 | 31 | - !assert 32 | name: the OS must be Linux 33 | true: (eq jet_os_type "Linux") 34 | 35 | - !assert 36 | name: the OS must not be MacOS 37 | false: (eq jet_os_type "MacOS") 38 | 39 | - !assert 40 | name: various things must all be true 41 | all_true: 42 | - (eq dog "scooby") 43 | - (eq ghost "blinky") 44 | 45 | - !assert 46 | name: none of these things may be true 47 | all_false: 48 | - (eq jet_os_type "Atari") 49 | - (eq ghost "Slimer") 50 | 51 | - !assert 52 | name: one of these things must be true 53 | msg: something is wrong, the value of ghost is {{ ghost }} 54 | some_true: 55 | - 56 | ``` 57 | 58 | {% hint style="info" %} 59 | **Why Does 'msg' exist when there is 'name'?** 60 | 61 | While template expressions cannot appear in 'name' - this is a banner that appears above all tasks and variables can be different per host - variables can appear in 'msg', which may be helpful in displaying the value of some variable without a seperate _echo_ task. 62 | {% endhint %} 63 | 64 | 65 | 66 | | Parameter | Details | 67 | | -------------- | --------------------------------------------------------- | 68 | | **msg** | Display this string when the assert fails | 69 | | **true** | A single handlebars expression that must be true | 70 | | **false** | A single handlebars expression that must be true | 71 | | **all\_true** | Multiple handlebars expressions that must all be true | 72 | | **all\_false** | Multiple handlebars expressions that must all be false | 73 | | **some\_true** | Multiple handlebars expressions of which one must be true | 74 | 75 | ### debug 76 | 77 | The debug module shows all variables available to a host in templating, in the context of each host. This can be quite a lot of output, so the optional variables parameter can pare down the output to show just certain variables: 78 | 79 | ``` 80 | tasks: 81 | 82 | - !debug 83 | 84 | # OR ... 85 | 86 | - !debug 87 | vars: 88 | - foo 89 | - bar 90 | ``` 91 | 92 | 93 | 94 | | Parameters | Description | 95 | | ------------- | ------------------------------------------------------------------------------------ | 96 | | **variables** | An optional list of variable names to print, otherwise all variables will be printed | 97 | 98 | ### echo 99 | 100 | Displays messages in the middle of a play. This command does not involve any communication with the managed host and can be used to debug variable values in a basic way. A more fully featured debug module will be available later. 101 | 102 | ``` 103 | tasks: 104 | - !echo 105 | msg: "For those about to rock, we salute you!" 106 | 107 | - !echo: 108 | msg: "The value of X is {{ x }}" 109 | ``` 110 | 111 | 112 | 113 | | Parameters | Description | 114 | | ---------- | ----------------------------------------- | 115 | | msg | **Required**. String to show to the user. | 116 | 117 | ### fail 118 | 119 | Fail unconditionally fails the rest of the play, which will result in all of the tasks in the play failing for every host, which will in turn result in the entire playbook execution being stopped. 120 | 121 | This is not really a useful construct to leave in the automation, but it may be useful content when debugging some automation and wanting the automation to stop before it gets any further along in the playbook content. 122 | 123 | {% hint style="info" %} 124 | **Failing With Flexibility** 125 | 126 | If you want to fail a playbook run for a host conditionally - when certain situations are true but not always - use the _assert_ module instead 127 | {% endhint %} 128 | 129 | ``` 130 | tasks: 131 | 132 | - !echo 133 | msg: "hello" 134 | 135 | - !fail 136 | msg: "this message is optional" 137 | 138 | - !echo 139 | msg: "things will never get this far" 140 | 141 | ``` 142 | 143 | 144 | 145 | | Parameter | Description | 146 | | --------- | ------------------------------------- | 147 | | **msg** | Required. String to show to the user. | 148 | 149 | ### facts 150 | 151 | The fact module reaches out to the remote system, determines various properties about the OS and state of that system, and inserts variables into the 'host' namespace for use in templates, conditionals, and so on. 152 | 153 | {% hint style="info" %} 154 | **When Are Facts Gathered And When Are Fact Variables In Scope?** 155 | 156 | Fact gathering in Jet is never done implicitly. If you do not have a "!facts" task in a play, facts will not be gathered. This is because lots of playbook uses do not need them. 157 | 158 | If the playbook has multiple plays, variables will persist between plays that involve the same hosts. If you would like to use the _facts_ module, you can put it at any point in the tasks list, but at the top is a good idea. It's fine to re-run _facts_ later to re-gather facts as well, should you think something has changed! 159 | {% endhint %} 160 | 161 | ``` 162 | tasks: 163 | 164 | - !facts 165 | 166 | - !echo "the Linux OS type is {{ jet_os_flavor }}" 167 | ``` 168 | 169 | The core set of variables supported by Jet is intentionally somewhat small. Values include: 170 | 171 | | Target OS | Variables | | 172 | | --------- | -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | 173 | | Mac | jet\_os\_type | MacOS | 174 | | Mac | jet\_os\_flavor | OSX | 175 | | Linux | jet\_os\_type | Linux | 176 | | Linux | jet\_os\_flavor | EL, Debian, Arch, etc | 177 | | Linux | jet\_os\_release\_\* | variables from /etc/os-release namespaced and lowercased, such as: jet\_os\_release\_is\_like, jet\_os\_release\_name, etc, cat this file on your distribution for understanding keys and values | 178 | | All | jet\_arch | The architecture, such as x86\_64, as provided by 'uname' | 179 | 180 | 181 | 182 | Again, Jet's core set of facts is deliberately minimal. 183 | 184 | To extend the levels of facts provided, Jet can use installed tools like facter and ohai (and soon, osquery) on remote hosts, by supplying parameters to the fact module: 185 | 186 | 187 | 188 | | Parameter | Description | 189 | | --------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | 190 | | facter | **Boolean**. If true, runs 'facter' on the remote host, saving facts in a variable called 'facter'. Facter is a product of Puppet Labs. Facter comes from the configuration management tool Puppet. | 191 | | ohai | **Boolean**. If true, runs 'ohai' on the remote hosts, saving facts in a variable called 'ohai'. Ohai comes from the configuration management tool Chef. | 192 | 193 | Here is an example of facter being used: 194 | 195 | ``` 196 | tasks: 197 | 198 | - !facts 199 | facter: true 200 | 201 | - !debug 202 | # so many facts!!! 203 | 204 | - !echo 205 | msg: "uptime is {{ facter.system_uptime.days }} days" 206 | 207 | ``` 208 | 209 | 210 | 211 | {% hint style="info" %} 212 | **Need More Facts?** 213 | 214 | More fact options will be added in Tech Preview 2, most likely including support for a /etc/jet/facts.d directory on the remote host. 215 | {% endhint %} 216 | 217 | ### set 218 | 219 | set allows variables to be set conditionally or as the result of combination of other variables, something that is not generally allowed elsewhere in the program. Use this sparingly as it may impede playbook readability and obviousness. This could be be used in roles that want to target multiple operating systems but the packaging between them is different in terms of file locations or desired default settings. 220 | 221 | A conditonal set example: 222 | 223 | ``` 224 | - name: set example 225 | 226 | groups: 227 | - all 228 | 229 | defaults: 230 | debian_basedir: "/etc/foo" 231 | conf_location: "/etc/foo.conf" 232 | 233 | tasks: 234 | 235 | - !facts 236 | 237 | - !set 238 | vars: 239 | conf_location: "{{ debian_basedir }}/foo.conf" 240 | with: 241 | condition: (eq jet_os_flavor "Debian") 242 | 243 | ``` 244 | 245 | {% hint style="info" %} 246 | **Need More Flexibility?** 247 | 248 | Depending on demand a more structured case statement may also be available in the future, but we want to avoid having branching constructs in the automation language beyond skipping tasks. 249 | {% endhint %} 250 | 251 | {% hint style="warning" %} 252 | **Pay Attention To Vars Vs Defaults** 253 | 254 | If the word "defaults" was changed to "vars" above, the location would not be overridden correctly because "vars" has a much [higher precedence](../playbooks/using-variables.md) than host variables. 255 | {% endhint %} 256 | 257 | | Parameter | Description | 258 | | --------- | -------------------------------------------------------- | 259 | | **vars** | A mapping of string variable names to variables to set. | 260 | 261 | -------------------------------------------------------------------------------- /developer-guides/module-development.md: -------------------------------------------------------------------------------- 1 | # 🛠 Module Development 2 | 3 | is chapter is about developing [modules](broken-reference) for contribution into jetp. 4 | 5 | ## When To Develop Built-In Modules Vs External Modules 6 | 7 | It is important to understand internal modules are part of the core program and are not exactly plugins but are very intentionally baked _into_ the program. It's very hard to break a module, even a seldom used one, by making a code change elsewhere in the program. While modules are baked in, they are however still very _modular_ in that they all have a common interface (traits) and ways of working. 8 | 9 | The purpose of a Jet Rust module is to provide an extremely fast, tight, and accurate implementation of a OS/configuration/deployment capability that is instantly available to all users of Jet _without_ searching, downloading, or thinking about it. This may be considered a 'batteries included' philosophy. 10 | 11 | For modules that may be a bit more site-specific, such as hitting your own internal APIs, [External Modules](../modules/external-modules.md) will be a faster means of development that allow you to work in any language. It will be very difficult to maintain a fork of Jet, and it is also frankly _slower_ to write modules in Jet properly, but we like it this way. 12 | 13 | Generally, we mostly want to consider new module contributions particularly in areas of: 14 | 15 | * Increasing the quality of support for configuring and working with operating systems, including services and package managers 16 | * Suppporting general purpose automation needs and extremely commonly used functions 17 | 18 | We do not currently include modules for: 19 | 20 | * Cloud topology configuration, though simple operations useful in OS configuration like downloading from buckets and such are fair game. 21 | * Talking to new/particular SaaS services that are likely to frequently change APIs 22 | 23 | With many exceptions, modules typically like to work by wrapping command line tools with stable interfaces. The emphasis here is upon _**stable**_ which is the critera for inclusion. 24 | 25 | For instance, suppose we wanted to make a module for downloading content from a FooCloud object store and FooCloud was already used by 25% of the world. This makes it a pretty good idea to include support for FooCloud! 26 | 27 | What we would want to do is see if FooCloud has a CLI tool. If the CLI is already great, we might not need a module at all. However, if the CLI is hard to use, requires multiple steps, or is not declarative in nature (such as the 'get' commands being different from 'update') it may be a good fit for module development because having a module would improve our quality of life in writing repeated automation tasks around these items. 28 | 29 | As another example, assume we have a new monitoring startup company that only has an API that is available via submitting rapidly evolving JSON to a web service endpoint. Because this is both a new and small company, as well as there not being a stable command line tool to interface with the program, it does not make sense to add a module for Jet. Should this monitoring company evolve to become widely used, a module might make sense provided there was a _**stable**_ command line tool. Because the tool would be ideally be frequently updated, there is no concern about forever maintaining API interfaces with the vendor. 30 | 31 | Controlling module quality is important to Jet while also maintaining a "batteries included" feel. Since modules are written in Rust, we have strong guarantees that they will be maintained correctly when written correctly, and the various details of the finite state machine underpinning modules, as well as internal API design, ensures these can remain high quality. 32 | 33 | Inclusion of a module is always a discussion, see [Contributing](../community/contributing.md) for some rationale about that and how to discuss module ideas on chat. 34 | 35 | With that out of the way, let's get to details! 36 | 37 | ## Identifying An Exemplar Module 38 | 39 | While the API contracts of modules will ensure mostly correct implementation, you will want to identify a module that is closest in nature to the module you want to develop. 40 | 41 | For modules working with files on the local filesystem, this may be _**template**_ or _**copy**_ or _**file**_. Realistically, you should study all three, and more so. There is something to be learned from any module's implementation. These live in modules/ in the git checkout. 42 | 43 | For adding support for a package manager - or most modules that mostly wrap an OS command and provide declarative/idempotent state to an imperative command, a good example module to look at is **dnf** or **apt.** 44 | 45 | ### Early Steps 46 | 47 | Create a new file for your module in src/modules/foo.rs, replacing foo with the name of your module. 48 | 49 | Consider copying the code from an exemplar module and adapting it. Think about what steps it would need to do to fulfill the "query", "create", "modify", "remove", and/or execute legs of the state machine from the module code you are looking at. 50 | 51 | Read the steps below to learn more about completing the module and wiring it in to jetp. 52 | 53 | ### Adding A Module To The Module Registry 54 | 55 | This is a tiny bit weird. 56 | 57 | Because of some interplay between polymorphic dispatch macros, our YAML serializer, and Rust's rules about data-inheritance in structs (which does not exist currently), we have a tiny bit more boilerplate to wire modules into the "module registry" than what we would like. This will possibly improve in the future with more build magic, but please forgive this and you'll quickly be in happy territory in just a few minutes! 58 | 59 | 1. Open modules/registry/list.rs and search for "Echo". This shows you every occurance of the Echo module being added to the registry. 60 | 2. Add the name of your module to the list everywhere you see Echo, but keep things alphabetized everywhere to keep the system neat. There are currently only four places you have to add this - and they are each very simple one-line entries 61 | 3. Open src/modules/(category)/mod.rs and add your module import there, and also keep it alphabetized. 62 | 63 | Your module is now part of the Rust code and will be embedded in the single-binary that compilation produces. Of course, it won't compile yet, most likely, so let's dive into the individual parts of module development in greater detail! 64 | 65 | ### Implementing Your Module 66 | 67 | Using your exemplar module for reference, module implemntation is best understood by looking at the module code in various stages, and here we'll explain the design decisions of one particular module, in this case _**dnf**_ ... modules/package/dnf.rs 68 | 69 | {% hint style="info" %} 70 | A GitHub link will be inserted here after release 71 | {% endhint %} 72 | 73 | The dnf module wraps various OS commands to provide a declarative layer over the package manager. While dnf is specific to certain platforms, you will be able to learn from it just the same. 74 | 75 | Each module has some normal structure, though the details of each module may be very different: 76 | 77 | 78 | 79 |

80 | 81 | ### Module Name 82 | 83 | ``` 84 | const MODULE: &str = "dnf"; 85 | ``` 86 | 87 | The name of the module is only used when there is no 'name' element on a task in the playbook, and this is shown to the user in this case. The "!tag" name of the module instead comes from the name used in the enum in registry/list.rs, but these should match. 88 | 89 | 90 | 91 | ### Inputs And Outputs 92 | 93 | 94 | 95 | ``` 96 | pub struct DnfTask { 97 | pub name: Option, 98 | pub package: String, 99 | pub version: Option, 100 | pub update: Option, 101 | pub remove: Option, 102 | pub with: Option, 103 | pub and: Option 104 | } 105 | 106 | struct DnfAction { 107 | pub package: String, 108 | pub version: Option, 109 | pub update: bool, 110 | pub remove: bool, 111 | } 112 | 113 | ``` 114 | 115 | Every module will define a Task struct and an Action struct. The Task struct corresponds to the input in the YAML playbook. The Action struct is the rendered and evaluated version of the Task, after substituting all template variables, casting all parameters, and so on. 116 | 117 | This is done by the evaluate function: 118 | 119 | ``` 120 | 121 | fn evaluate(&self, 122 | handle: &Arc, 123 | request: &Arc, 124 | tm: TemplateMode) -> Result> 125 | ``` 126 | 127 | The evaluation code for every module will be different, but it doesn't actually execute any behaviors on the remote system, it merely checks and evaluates all of the arguments. The various features used on each parameter are mostly from the "handle.template" namespace, so browsing src/handle/template.rs will give a good idea of what is offered here. 128 | 129 | Handle is an object that basically a power tool or swiss army knife that modules will work with extensively, here, it's mostly "handle.template" or "handle.repsonse" that is used here. Refer to the source code for specifics. 130 | 131 | Request is the request object, which is needed to construct a response. Later down the module will perform different types of actions based on the type of request, but for evaluate, the request object is mostly uninteresting. 132 | 133 | tm (TemplateMode) is mostly an internalism you don't need to think about. 99% of the time, the value of this is "Strict", which means all variables in an object need to be defined. There are some ways this is used internally ahead-of-time in the call chain, and that's not really important for module authors to understand. 134 | 135 | ### Getting Things Done 136 | 137 | When the 'main' piece of the code is walking tasks, it will call evaluate on each to produce an action, and then on each action, it will call dispatch. 138 | 139 | Dispatch in each module looks like this: 140 | 141 | 142 | 143 | ``` 144 | fn dispatch(&self, 145 | handle: &Arc, 146 | request: &Arc) -> Result, Arc> 147 | ``` 148 | 149 | 150 | 151 | And each module will perform different actions based on request.request\_type, returning different variations of return objects from handle.response. 152 | 153 | There is a finite state machine behind the scenes that will ensure certain classes of responses are not allowed for certain types of requests. 154 | 155 | This is best understood by browsing various modules. In the case of dnf, we have responses that occur for varying behaviors: 156 | 157 | 158 | 159 | 160 | 161 |

162 | 163 | 164 | 165 | | | | 166 | | ------ | ------------------------------------------------------------------------------------------------------------------------------------ | 167 | | Query | return what type of action should occur based on the current state of the system | 168 | | Create | the resource was missing, so perform these actions to create it from scratch. In the case of packages, create means 'install' | 169 | | Modify | the resource did not match the specification, so perform some actions to modify it. In the case of packages, modify means 'upgrade' | 170 | | Remove | the resource exists but should not. In the case of packages, remove means 'uninstall'. | 171 | 172 | 173 | 174 | In addition some additional categories are not used by the package module but appear elsewhere: 175 | 176 | 177 | 178 | | | | 179 | | ------- | ---------------------------------------------------------------------------------------------------------------------------------------------- | 180 | | Passive | used by modules that perform some action that does not really change the remote system, and may only provide some output or set some variables | 181 | | Execute | used by modules that run commands | 182 | 183 | Modules are to return errors if they are called to perform actions they cannot support, but this will also not happen, because the query module will not return transitions into those states. 184 | 185 | --------------------------------------------------------------------------------