├── LICENSE ├── facts.md ├── types-and-providers.md ├── classes.md └── tests.md /LICENSE: -------------------------------------------------------------------------------- 1 | MIT License 2 | 3 | Copyright (c) 2016 Ashley Penney 4 | 5 | Permission is hereby granted, free of charge, to any person obtaining a copy 6 | of this software and associated documentation files (the "Software"), to deal 7 | in the Software without restriction, including without limitation the rights 8 | to use, copy, modify, merge, publish, distribute, sublicense, and/or sell 9 | copies of the Software, and to permit persons to whom the Software is 10 | furnished to do so, subject to the following conditions: 11 | 12 | The above copyright notice and this permission notice shall be included in all 13 | copies or substantial portions of the Software. 14 | 15 | THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR 16 | IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, 17 | FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE 18 | AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER 19 | LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 20 | OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE 21 | SOFTWARE. 22 | -------------------------------------------------------------------------------- /facts.md: -------------------------------------------------------------------------------- 1 | #Introduction 2 | 3 | [RESEARCH: Facter 2 and the changes within. ] 4 | 5 | In the [previous section](Types and Providers) we extended the ssh 6 | module we began in the [Classes, Params, and APIs](Same) section to 7 | allow it to create ssh_tunnels. Now we’re going to extend the module 8 | to add two new facts. The first is `private_key_users`, a list of 9 | users that have known private keys in their .ssh/ directory. The 10 | second is `public_key_comments`, a collection of all the comments from 11 | all *.pub files, in case you need a quick way to confirm that a key 12 | exists on a server. 13 | 14 | These aren’t the most practical examples, but they serve as 15 | demonstrations for what you can do in facts. 16 | 17 | The recently released Facter 2.0 has dramatically changed the 18 | functionality and ability to write sophisticated facts. This document 19 | has been written against Facter 2.0 and contains non-backwards 20 | compatible code. 21 | 22 | #Background 23 | 24 | When generating private keys via OpenSSH, a pair is created with the 25 | filename id_x and id_x.pub. The X differs by encryption method used, 26 | but generally it’s id_rsa and id_dsa. Public keys look like: 27 | 28 | ``` 29 | ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVwwxl6dz6Y7karwyS8S+za4qcu99Ra8H8N3cVHanEB+vuigtbhLOSb+bk6NjxFtC/jF+Usf5FM5fGIYd51L7RE9BbzbKiWb9giFnNqhKWclO5CY4sQTyUyYiJTQKLuVtkmiFeArV+jIuthxm6JrdOeFx8lJpcgGlZjlcBGxp27EbZNGWIlAdvW0ZXy0JqS9M/vj71NBBDfkrpyzAPC0aBa9+FmywOH6HXbyeFooHLOw+mfzP87jwDDQ2yXIehDoC1BsLYXD+j+kdnR0CNltJh1PYOFNpbKQpfnPhfdw4Oc0hZ34n+kfBPavKlbwxoVAoisBWWo4c9ZnUoe2OBRHAX comment at the end 30 | ``` 31 | 32 | #Supporting decisions 33 | 34 | Puppet is a declarative system, which means we describe the end 35 | configuration state desired for your nodes. New puppet users often 36 | make the mistake of writing facts to allow them to build complex 37 | logic based on the local state of the agent being managed. They then 38 | use these facts to determine what actions to take. An example of this 39 | might be a fact called `apache_installed` and a manifest that says: 40 | 41 | ```puppet 42 | if $apache_installed { 43 | include 'something' 44 | } 45 | ``` 46 | 47 | This is a dangerous way to use facts. Facts should be used to find 48 | out the information needed to enforce a decision that the module 49 | designer has already decided on, rather than being used to make those 50 | decisions. In the above example you should decide at a higher level 51 | if apache should be installed or not, and then make it so, not query 52 | the agent to find out if it was. 53 | 54 | Writing facts to make decisions rather than supporting them causes the 55 | unfortunate side effect of making your catalog nondeterministic. This 56 | means that it is hard to reason about infrastructure because it is 57 | constantly changing based on the local state of the systems. It's best 58 | to leave facts in a supporting role of helping you enforce policy that 59 | was predetermined based on the role or profile of the agent, not the 60 | local state of the server being managed. 61 | 62 | #Logic 63 | 64 | Another common mistake made is to move logic into facts, as opposed to 65 | modeling your logic within the manifest. Facts should, when 66 | possible, simply return information. This is then consumed within the 67 | manifest to decide the end state. Internal logic within the fact 68 | which is used to help obtain the information is fine, but you 69 | shouldn't write a fact called `ssh_required` that returns true/false. 70 | Instead you should write a fact called `ssh_installed` that returns 71 | true/false and then write the appropriate logic to determine if ssh is 72 | required within the manifest based on the knowledge that it's 73 | currently installed or not. 74 | 75 | #Confines 76 | 77 | One of the mechanisms facts supports is confines, allowing you to 78 | write: 79 | 80 | ``` 81 | confine :kernel => ‘Linux’ 82 | ``` 83 | 84 | But you can also get more sophisticated and pass in a block to 85 | confines, such as: 86 | 87 | ``` 88 | confine { File.exists?(keyfile) } 89 | ``` 90 | 91 | An example of a fact that relies on a confine block can be found in 92 | the `gid.rb` fact distributed with Facter. 93 | 94 | ##Demonstration 95 | 96 | We start our fact by requiring etc, a ruby gem, in order to iterate 97 | through the users on the system. We then iterate through each entry 98 | of the password file. 99 | 100 | ``` 101 | require 'etc' 102 | 103 | Facter.add(:private_key_users) do 104 | setcode do 105 | users = [] 106 | # This generates an object for each entry in /etc/passwd 107 | Etc.passwd do |user| 108 | ``` 109 | 110 | We now look for known private keys and if one is found we add the 111 | users name to the list of users. 112 | 113 | ``` 114 | # Look for known private key names. 115 | ['id_rsa', 'id_dsa'].each do |file| 116 | if File.exists?("#{user.dir}/.ssh/#{file}") 117 | users << user.name 118 | # Once we've found a key we can skip the rest of the tests 119 | break 120 | end 121 | end 122 | end 123 | ``` 124 | 125 | Finally, at the end we return the list of users. 126 | 127 | ``` 128 | # Facts must be strings, so return a list of users comma separated. 129 | users.join(',') 130 | end 131 | end 132 | ``` 133 | 134 | Our second fact is similar to the first fact, except that we extract 135 | public-key comments and return a comma separated list of these. 136 | 137 | ``` 138 | require 'etc' 139 | 140 | Facter.add(:public_key_comments) do 141 | setcode do 142 | comments = [] 143 | # This generates an object for each entry in /etc/passwd 144 | Etc.passwd do |user| 145 | # Generate a list of *.pub files in ~/.ssh/ 146 | keyfiles = Dir["#{user.dir}/.ssh/*.pub"] 147 | # Check each .pub file in ~/.ssh/ in turn 148 | 149 | ``` 150 | 151 | Lastly, we check each keyfile in turn by reading it and taking the 152 | last element as the comment. This is fairly error prone as it simply 153 | relies on a space separator so any comment with a space fails, but 154 | it’s concise and matches the default style comments generated with 155 | ssh-keygen. 156 | 157 | ``` 158 | keyfiles.each do |file| 159 | contents = File.read("#{user.dir}/.ssh/#{file}") 160 | contents.each_line do |line| 161 | comments << line.split(' ').last 162 | end 163 | end 164 | end 165 | # Facts must be strings, so return a comma seperated list. 166 | comments.uniq.join(',') 167 | end 168 | end 169 | 170 | ``` 171 | 172 | #Conclusion 173 | 174 | [TODO], wrap up text. 175 | -------------------------------------------------------------------------------- /types-and-providers.md: -------------------------------------------------------------------------------- 1 | #Advanced Guide to Modules: Types & Providers 2 | 3 | ##Introduction 4 | 5 | Types and providers are the heart of how Puppet works. They allow you to 6 | express the "how to do something" through raw commands, outside of the 7 | constraints of the Puppet DSL. With that information Puppet can then figure 8 | out how to transition from what you have to what you want. They are written 9 | in Ruby, like Puppet. 10 | 11 | Types and providers are preferred over execs and definitions due the ability to 12 | do things difficult within the Puppet DSL. It's not possible to read the 13 | results of an exec{} and do different things based on that output. They are 14 | also ideally suited for other forms of complex logic that is difficult to 15 | manage within the constraints of the DSL. For instance, a provider that 16 | 17 | An example of this is a provider that manages local redis instances. It would 18 | need to make an actual TCP connection to the database to retrieve the current 19 | state of the properties you wish to manage with Puppet. This would be impossible 20 | within the DSL and therefore modeled as a type and provider. 21 | 22 | To highlight these practices, we will expand the basic ssh module we built in 23 | the [previous section](link to Classes, Params, and APIs) to add a type and 24 | provider to support the creation of ssh tunnels, as well as SOCKS proxies. 25 | 26 | ###Types & Providers vs. Defines 27 | 28 | It would not be unusual to model something like ssh tunnels or SOCKS proxies 29 | within a definition in Puppet to allow the creation of multiple instances. While 30 | this may work in the most simplistic of cases it usually breaks down in production 31 | quality modules due to the complexities of managing the underlying system state 32 | with the limitations of `exec{}`. You cannot do sophisticated logic easily and 33 | are limited to chaining together `exec{}` resources to try and do various things 34 | based on exit codes and shell commands. 35 | 36 | To most effectively model ssh tunnels and SOCKS proxies using types and 37 | providers, we must be able to: 38 | 39 | * Create an SSH tunnel based on various parameters. 40 | * Validate the parameters to make sure they conform to the needs of the command 41 | that runs to build the tunnel. 42 | * Test if a tunnel is running and only start it if not. 43 | * Close an SSH tunnel if `ensure => absent` is set. 44 | 45 | Because `exec{}` resources are just simple wrappers for shell 46 | commands, it would be extremely complex to model the workflow of an 47 | SSH tunnel and SOCKS proxy using one. In order to successfully use an 48 | `exec{}` resource in this case, all of the logic of closing tunnels, 49 | testing the processes, and tracking the existing processes within 50 | Puppet would need to be modeled within the `definition`. The logic 51 | model would require many `exec{}` statements and custom facts to try 52 | and query the existing state of the machine, as there is no easy way 53 | within Puppet to use the results of an `exec{}` statement within 54 | another `exec{}` statement. This is because Puppet is intended to 55 | declare the end state, not the steps required to reach the end state. 56 | That's the entire purpose of a type and provider, to handle the step 57 | by step actions to transition between states. 58 | 59 | To provide a concrete example of what this might look like, here is an 60 | example `unless` property of a hypothetical exec{} to manage ssh tunnels. 61 | 62 | ```puppet 63 | unless => ‘ps auxww | grep “-L ${local_port}:${forward_host}:${forward_port}”’` 64 | ``` 65 | 66 | This would attempt to stop duplicate tunnels being started. It has a 67 | number of problems, being unable to handle systems that don't take 68 | 'auxww' as arguments to ps. You’d also have to ensure you handle -D as 69 | well for SOCKS proxies, and this would make the logic complex. 70 | 71 | This also wouldn't work if you wanted to change one of the properties of the 72 | definition. If you changed the port you passed into the resource it would be 73 | unable to find the previous tunnel anymore thanks to the change in the grep 74 | string. It would then create a second tunnel. You would have to keep every 75 | tunnel definition you created, changing the ensure parameter to absent, and 76 | never reuse names to ensure it correctly managed them all. 77 | 78 | In addition to the above problems the validation of the parameters 79 | would have to be done with various functions within the manifest. 80 | None of this is impossible, but you’d end up with a difficult to reuse 81 | define. Moreover, it would be tied very specifically to the 82 | implementation and it would be difficult to extend it for various 83 | different systems and use cases. 84 | 85 | As you’re reading the types and providers section you can correctly 86 | infer that we recommend the use of types and providers for this kind 87 | of functionality [link-to-start]. Types and providers open up the 88 | full power of Ruby on the agent, allow for multiple providers to serve 89 | as the backends to a single type, and moreover, allow you to control 90 | resources directly from the CLI with `puppet resource` or from an 91 | orchestration system like MCollective. In the ssh module, using types 92 | and providers allows us to easily audit SSH tunnels across the entire 93 | infrastructure, while also opening up more sophisticated use cases 94 | than simple execs{} can generate. 95 | 96 | NOTE (ideally this should be in a box of some kind): 97 | [Types and Providers](http://www.amazon.com/Puppet-Types-Providers-Dan-Bode/dp/1449339328?tag=88fd3e53f8c9c-20), 98 | is the best reference material available for writing types and 99 | providers. It covers all the pieces of functionality we’ll use below 100 | in vastly more detail and is an invaluable guide. 101 | 102 | 103 | ###Intro to SSH Tunnels 104 | 105 | We’re going to extend our ssh module by adding a type and provider capable of 106 | managing ssh tunnels. If you’re not familiar with the concept, an ssh tunnel 107 | allows you to tunnel TCP/IP traffic from a local port through a remote server 108 | to a destination. An example of this might be: 109 | 110 | ``` 111 | ssh -L 8080:localhost:80 user@remoteserver 112 | ``` 113 | 114 | Running `ssh -L` would send any traffic from 8080 on your machine to the local 115 | port 80 on user@remotewebserver. Or, rather than localhost, you could use 116 | another hostname to have ‘remoteserver’ act as a kind of proxy for your 117 | traffic. You can even use ssh tunneling to set up a SOCKS proxy in order to 118 | forward traffic of all kinds: 119 | 120 | ``` 121 | ssh -D 8080 user@remoteserver 122 | ``` 123 | 124 | For further reading on SSH tunnels you can [TODO: find a resource] 125 | 126 | ##Type 127 | 128 | Types represent the resource description. They list out all the 129 | parameters that are required, their validation, any munging (forcing 130 | passed in parameters to conform to certain needs, like forcing 131 | everything to be lowercase.), or special casing of how the parameters 132 | discovered on the system are compared to the required ones. They are 133 | written in Ruby and placed in lib/puppet/type/type_name.rb. 134 | 135 | In our ssh module, we’re going to write `ssh_tunnel`, a type that sets 136 | up tunnels. We have two use cases we want to target for now, one is: 137 | 138 | ``` 139 | ssh_tunnel { ‘bypass firewall’: 140 | ensure => present, 141 | local_port => ‘8080’, 142 | forward_server => ‘localhost’, 143 | forward_port => ‘80’, 144 | remote_server => ‘remoteserver’, 145 | } 146 | ``` 147 | 148 | As well as the SOCKS proxy option: 149 | 150 | ``` 151 | ssh_tunnel { ‘socks proxy’: 152 | ensure => present, 153 | local_port => ‘8080’, 154 | socks => true, 155 | remote_server => ‘remoteserver’, 156 | } 157 | ``` 158 | 159 | This means that our required parameters are `remote_server`, `local_port`, and 160 | `ensure`. Other parameters are optional depending on which mode we’re using. 161 | 162 | ###Demonstration 163 | 164 | Type declarations declare the name of the resource we're creating. 165 | Best practices recommend that you try to keep the name short and 166 | sweet. 167 | 168 | In our ssh module, we begin with a simple type declaration at the 169 | top. To write your own types, you just need to change ':ssh_tunnel' to 170 | ':type_name' and then update @doc to contain an appropriate 171 | documentation string. 172 | 173 | ``` 174 | Puppet::Type.newtype(:ssh_tunnel) do 175 | @doc = 'Manage SSH tunnels' 176 | ``` 177 | 178 | Type validations ACCOMPLISH X and SHOULD/MUST BE USED IN Z WAY/FOR Y PURPOSE. 179 | Validation has two sorts (I will fix that word later): regular and global. 180 | Regular validation, referred to as simply 'validation', does/is used to X. 181 | Global validation does/is used to Y. Best practices HAS SOME THOUGHTS ABOUT 182 | WHAT'S BEST. 183 | 184 | Type validations allow you to verify that the contents of your 185 | resource parameters conform to various validation rules. There are 186 | two sorts of validation, parameter and global. Parameter validation, 187 | referred to as simply 'validation' is for inspecting and verifying the 188 | contents of a single parameter. Global validation can be used to 189 | ensure that multiple parameters conform. An example might be making 190 | sure that two conflicting parameters are never set together. Best 191 | practices recommends validating as much of your potential input as 192 | possible. 193 | 194 | First we'll look at some global validation in our ssh module. We do 195 | this by calling the validate method and passing it a block of things 196 | to test. In our type we just need to make sure that `forward_server` 197 | and `forward_port` are set if we’re not in SOCKS mode. 198 | 199 | If either of these conditions aren’t true, we call fail() with a 200 | reason. 201 | 202 | ``` 203 | validate do 204 | # If we're not in SOCKS mode, we need a server and port to forward to. 205 | if self[:socks] == :false 206 | fail('forward_server is not set') if self[:forward_server].nil? 207 | fail('forward_port is not set') if self[:forward_port].nil? 208 | end 209 | end 210 | ``` 211 | 212 | One benefit to writing a type and provider is that there are various 213 | helper methods that make common tasks easier. When we call ensurable 214 | it'll automatically create an ensure property with appropriate 215 | validation handling. 216 | 217 | ``` 218 | # This automatically creates the ensure parameter. 219 | ensurable 220 | ``` 221 | 222 | Next, we create our first “parameter”. Puppet distinguishes between 223 | properties and parameters, and the difference is just that properties 224 | can be managed and discovered from the system, whereas parameters 225 | aren’t discoverable and can’t be managed. In our type the name is 226 | really not something you “manage” on the local system, it’s just a 227 | reference point for Puppet. That means it’s a parameter, not a 228 | property. 229 | 230 | In order to allow this type to work with `puppet resource` we’ll need 231 | to define some way for Puppet to automatically name discovered 232 | resources. The provider we create after the type will be able to 233 | discover the existing instances that were created on the machine, such 234 | as: 235 | 236 | ``` 237 | ssh -L 8080:localhost:80 user@remote 238 | ``` 239 | 240 | It'll then translate that into an appropriate name value such as: 241 | 242 | ``` 243 | ssh_tunnel { ‘8080:localhost:80-user@remote’: } 244 | ``` 245 | 246 | This is a little clumsy but we need the name to be unique, and 247 | changing any of the information about the tunnel means it’s a 248 | completely new one. For other kinds of resources it’s easier to 249 | provide a short and memorable name, as only the properties change and 250 | there’s a static name to refer to (like package{}). 251 | 252 | ``` 253 | newparam(:name, :namevar => true) do 254 | desc "The SSH tunnel name, must be unique." 255 | end 256 | ``` 257 | 258 | Our first property introduces ‘newvalues()’ and ‘defaultto()’. These 259 | methods are some of the helpers Puppet contains to make it easier to 260 | validate input and provide defaults. The first method, newvalues(), 261 | accepts a comma seperated list of values to accept. These can be 262 | symbols, strings, or even regular expressions. It’s fine to have 263 | newvalues(/^\//) in order to enforce an absolute path. The second 264 | method, defaultto() just specifies the default value that Puppet 265 | should set if you don’t pass the parameter in when defining the 266 | resource in your manifest. 267 | 268 | ``` 269 | newproperty(:socks) do 270 | desc 'Should this be a SOCKS proxy' 271 | newvalues(:true, :false) 272 | defaultto :false 273 | end 274 | ``` 275 | 276 | Our next property demonstrates an alternative, more sophisticated, use 277 | of validation. In validate we’re binding the value passed into 278 | validate to the variable ‘port’, making it available for checking 279 | within. If you were to pass an array to local_port it would iterate 280 | over that array and pass each element to validate. 281 | 282 | Inside the block you can do whatever ruby code you like. We’re trying 283 | to verify that port can be converted to an integer and then tested to 284 | make sure it’s a valid port range. 285 | 286 | ``` 287 | newproperty(:local_port) do 288 | desc 'The local port to forward traffic from' 289 | validate do |port| 290 | fail("Port is not in the range 1-65535") unless port.to_i >= 1 and port.to_i <= 65535 291 | end 292 | end 293 | ``` 294 | 295 | Afterwards this results in output such as: 296 | 297 | ``` 298 | # puppet resource ssh_tunnel 8080-test@localhost local_port=65536 299 | Error: Could not run: Parameter local_port failed on Ssh_tunnel[8080-test@localhost]: Port is not in the range 1-65535 300 | ``` 301 | 302 | The rest of our properties are all similar to the above explanations, 303 | so we won’t explain these further. 304 | 305 | ``` 306 | newproperty(:forward_port) do 307 | desc 'Port to forward traffic to' 308 | validate do |port| 309 | fail("Port is not in the range 1-65535") unless port.to_i >= 1 and port.to_i <= 65535 310 | end 311 | end 312 | 313 | newproperty(:remote_server) do 314 | desc 'Remote server to connect to' 315 | newvalue(/\w+/) 316 | end 317 | 318 | newproperty(:forward_server) do 319 | desc 'Server to forward traffic to' 320 | newvalue(/\w+/) 321 | end 322 | end 323 | ``` 324 | 325 | #Provider 326 | 327 | Once the type has been written you need a provider to ‘provide’ the services to 328 | the type. In this case the type is tied specifically to the provider, but we 329 | could have tried to make a generic “tunnel” type that took ssh and other 330 | providers. That’s good practice when feasible, but it’s ok to make a specific 331 | type/provider as well when it makes sense. Don’t over abstract before you know 332 | abstraction has a benefit. 333 | 334 | #Demonstration 335 | 336 | We start our provider by creating the Puppet::Type definition with 337 | .provide(:ssh) added. We also set up a command. Behind the scenes 338 | this finds appropriate executables within the system path. It does so 339 | in a generic, cross operating system, way so this works for Linux or 340 | Windows: 341 | 342 | ``` 343 | Puppet::Type.type(:ssh_tunnel).provide(:ssh) do 344 | desc 'Create ssh tunnels' 345 | 346 | commands :ssh => 'ssh' 347 | ``` 348 | 349 | `instances` is the heart of a provider. It’s the method that Puppet 350 | runs to discover all the existing instances of the thing the provider 351 | is managing. In the case of `user{}` that would be all the users. In 352 | our case it means that it needs to check `ps` and find all running 353 | tunnels. We do this by calling a method we haven't written yet, 354 | `ssh_processes`. This'll take a string to search for and attempt to 355 | discover it on the server. We'll write this later. 356 | 357 | Once we've discovered each of the processes we'll split the string into 358 | various pieces and use those to call new() which all the properties that 359 | were returned from ssh_processes. We'll build an array of all these 360 | new() calls and return that to Puppet as the list of instances we found. 361 | 362 | ``` 363 | def self.instances 364 | instances = [] 365 | tunnel_connections = ssh_processes("#{ssh_opts} -L") 366 | socks_connections = ssh_processes("#{ssh_opts} -D") 367 | 368 | tunnel_connections.each do |name, pid| 369 | rest, remote_host = name.split(' ') 370 | local_port, forward_server, forward_port = rest.split(':') 371 | instances << new( 372 | :ensure => :present, 373 | :name => "#{rest}-#{remote_host}", 374 | :local_port => local_port, 375 | :forward_server => forward_server, 376 | :forward_port => forward_port, 377 | :remote_host => remote_host, 378 | :socks => :false, 379 | :pid => pid 380 | ) 381 | end 382 | 383 | socks_connections.each do |name, pid| 384 | local_port, remote_host = name.split(' ') 385 | 386 | instances << new( 387 | :ensure => :present, 388 | :name => "#{local_port}-#{remote_host}", 389 | :local_port => local_port, 390 | :remote_host => remote_host, 391 | :socks => :true, 392 | :pid => pid 393 | ) 394 | end 395 | 396 | return instances 397 | end 398 | ``` 399 | 400 | Prefetch is called the first time Puppet encounters a resource in the 401 | catalog that matches the type that the provider is written for. This 402 | allows you to run certain logic before any resource of that type is 403 | applied. Maybe you need to run a command that temporarily allows 404 | changes to be applied to the thing being 405 | managed. 406 | 407 | The most common use for `prefetch` is shown below. When Puppet calls 408 | `prefetch` it passes in a hash of all the resources of the matching 409 | type. This can be used to set this provider as managing each of the 410 | instances we discover on the system. 411 | 412 | This is difficult to explain and complex to understand. For now you 413 | can simply cut and paste this into providers and tweak it a little to 414 | be appropriate to speed things up. 415 | 416 | ``` 417 | def self.prefetch(resources) 418 | # Obtain a hash of all the instances we can discover 419 | tunnels = instances 420 | resources.keys.each do |name| 421 | # If we find the resource in the catalog in the discovered list 422 | # then set the provider for the resource to this one. 423 | if provider = tunnels.find { |tunnel| tunnel.name == name } 424 | resources[name].provider = provider 425 | end 426 | end 427 | end 428 | ``` 429 | 430 | Puppet runs `exists?` to determine if a resource actually exists on 431 | the agent. This allows you to determine the existence of a resource 432 | in cases where it's not practical to create a list of all of the 433 | resources on the agent with `instances`. Files would be an example 434 | of this, it would be expensive to create a resource for every file 435 | on the system through `instances`. 436 | 437 | If Puppet contains a resource in the catalog with an ensure property 438 | set to present and can't find it as existing via `exists?` it'll run 439 | `create` to call the appropriate logic to create the resource. 440 | 441 | In our `create` method we’re checking if the @resource[:socks] (aka 442 | socks => x in the resource definition) is true. If it’s true then we 443 | call ssh() with appropriate parameters to create a socks proxy. If 444 | @resource[:socks] is false then we just create a regular tunnel. 445 | 446 | ssh() is created by the earlier `commands ssh => :ssh` statement. The 447 | reason for this rather than calling commands directly is to add some 448 | additional features like showing calls via command() when you use 449 | --debug, automatic confining of the provider to only systems that 450 | contain the commands, and consistent command failure handling. 451 | 452 | You pass in arguments to ssh() one at a time, separated by spaces, and 453 | it safely builds up the appropriate command string to use. In our 454 | case we’re mostly just passing in other bits of information from 455 | @resource, which is full of the values from the defined resource in a 456 | manifest. 457 | 458 | By setting the @property_hash[:x] values to the @resource[:x] value, 459 | our provider is able to automatically create ‘getters’ (methods that 460 | get various bits of information) later in the code. 461 | 462 | ``` 463 | def create 464 | if @resource[:socks] == :true 465 | # Create a SOCKS proxy 466 | ssh(ssh_opts, '-D', @resource[:local_port], @resource[:remote_server]) 467 | @property_hash[:ensure] = :present 468 | @property_hash[:local_port] = @resource[:local_port] 469 | @property_hash[:remote_server] = @resource[:remote_server] 470 | else 471 | # Create an SSH tunnel 472 | ssh(ssh_opts, '-L', "#{@resource[:local_port]}:#{@resource[:forward_server]}:#{@resource[:forward_port]}", @resource[:remote_server]) 473 | @property_hash[:ensure] = :present 474 | @property_hash[:local_port] = @resource[:local_port] 475 | @property_hash[:forward_server] = @resource[:forward_server] 476 | @property_hash[:forward_port] = @resource[:forward_port] 477 | @property_hash[:remote_server] = @resource[:remote_server] 478 | end 479 | 480 | exists? ? (return true) : (return false) 481 | end 482 | ``` 483 | 484 | Destroy is, unsurprisingly, the method Puppet runs if ensure => absent 485 | and `exists?` returned true. In this case it simply checks the pid 486 | value contained within the @property\_hash and sends it a SIGTERM to 487 | shut it down. Once it does that it clears out the @property_hash to 488 | reflect that the resource, and all its properties, is gone. 489 | 490 | ``` 491 | def destroy 492 | Process.kill('SIGTERM', @property_hash[:pid].to_i) 493 | @property_hash.clear 494 | end 495 | ``` 496 | 497 | Here is how Puppet determines if a resource already exists. It just simply 498 | checks the property\_hash we've built up when running `instances` and `create` 499 | to see if the resource is present. If we had not written an `instances` method 500 | due to it being too expensive to run on the agent, we'd have had to check ps 501 | directly within `exists?` to determine if this resource was on the agent. 502 | 503 | ``` 504 | def exists? 505 | @property_hash[:ensure] == :present || false 506 | end 507 | ``` 508 | 509 | Here is a piece of ruby metaprogramming that creates other methods within the 510 | provider. It understands the properties and parameters contained within the 511 | type definition and automatically creates `property` and `property=` for each 512 | discovered from the type. By default these check the @property_hash to find 513 | the appropriate value for the property. 514 | 515 | If you find that `mk_resource_methods` is appropriate for most of your methods 516 | but you sometimes need to tweak one of them (perhaps a tunnel=) then you can 517 | simply define another method right after this call to override the generated 518 | one. 519 | 520 | ``` 521 | mk_resource_methods 522 | ``` 523 | 524 | Here we have a helper method that passes the appropriate ssh_opts in 525 | to ssh() commands so that we don’t have to remember to add them 526 | everywhere. We create it as self.ssh_opts to make it a class method 527 | rather than an instance method, meaning there's just a single method 528 | that lives in the class rather than each instance of this provider. 529 | 530 | If this sounds confusing to you then you can read more about this on 531 | various ruby sites or in ruby books. An example of this can be found 532 | at 533 | (rubymonks)[http://rubymonk.com/learning/books/4-ruby-primer-ascent/chapters/45-more-classes/lessons/113-class-variables]. 534 | 535 | ``` 536 | # -f tells ssh to go into the background just before command execution 537 | # -N tells ssh not to execute remote commands 538 | def self.ssh_opts 539 | '-fN' 540 | end 541 | # This allows ssh_opts to be used within the instance. 542 | def ssh_opts 543 | self.class.ssh_opts 544 | end 545 | ``` 546 | 547 | Beyond this we wish to make the methods in our provider private so 548 | they cannot be accidentally called from outside of the provider itself. 549 | Any method defined below this word will be considered private. 550 | 551 | ``` 552 | private 553 | ``` 554 | 555 | Last, but not least, we have the `ssh_processes` method that actually 556 | determines what tunnels are running on the agent. It calls `ps` and 557 | attempts to use regular expressions to extract the appropriate pid and 558 | process details needed to populate the instances in self.instances(). 559 | 560 | One of the difficulties with writing types and providers is avoiding 561 | dependencies on ruby libraries that aren't already part of the 562 | standard ruby distribution, or dependencies of Puppet. We could have 563 | written code here that called out to any of the process handling 564 | libraries of ruby, which would have been far more portable. However, 565 | that would mean the ssh module would need to distribute or install 566 | this library on all systems and sometimes that's not desired. 567 | 568 | This is why the code below manually runs and processes the output of 569 | `ps`. It's unportable to Windows and systems with a different `ps` 570 | command. For real world usage we would want to consider finding 571 | better ways to handle this. 572 | 573 | ``` 574 | # Find and return the ssh tunnel/proxy names and their associated pid. 575 | def self.ssh_processes(pattern) 576 | ssh_processes = {} 577 | ps = Facter["ps"].value 578 | IO.popen(ps) do |table| 579 | table.each_line do |line| 580 | # Attempts to match 'user (pid).*ssh {sshopts} (port:host:port host)' 581 | if match = line.match(/^\w+\s+(\d+)\s.*ssh\s#{pattern}\s(\d+:?\w*:?\d*\s\w.+)/) 582 | pid, name = match.captures 583 | ssh_processes[name] = pid 584 | end 585 | end 586 | end 587 | return ssh_processes 588 | end 589 | 590 | end 591 | ``` 592 | -------------------------------------------------------------------------------- /classes.md: -------------------------------------------------------------------------------- 1 | Naming them 2 | Deprecating them 3 | Good API examples vs Bad? 4 | Semantic Versioning quick guide in the context of modules. (Fix a bug - 1.0.x. Add parameters - 1.x.0. Change existing parameters. x.0.0) 5 | 6 | #Classes, parameters, and APIs 7 | 8 | As mentioned in the [Beginner's Guide to 9 | Modules](http://docs.puppetlabs.com/guides/module_guides/bgtm.html), 10 | *classes* are used to aggregate and organize *parameters*, the 11 | publicly-consumable *API*-like interface of your module. This first 12 | section of the Advanced Guide to Modules will walk through the best 13 | practices for developing a basic module: from determining how many and 14 | which classes to use to figuring out what your parameters will do and 15 | which should be public vs. private. To help illustrate these 16 | practices, we'll build the framework of a basic ssh module. 17 | 18 | ##Getting Started 19 | 20 | ###Intro to SSH 21 | 22 | SSH is software designed to secure connections to remote servers. It 23 | can be used to provide a secure shell, as well as to transfer files 24 | and tunnel TCP traffic securely through machines. For more 25 | information please visit 26 | [Wikipedia](http://en.wikipedia.org/wiki/Secure_Shell). 27 | 28 | ###Classes 29 | 30 | A module is, at its heart, a place to create a number of classes that 31 | belong together. We create classes to separate out the logic of a 32 | module into a number of smaller, self-contained tasks/functions that 33 | are organized around a single purpose. The most common pattern is 34 | 35 | * class 36 | * class::install 37 | * class::config 38 | * class::service 39 | * class::other 40 | 41 | Visualize this pattern as a division between private and public 42 | interfaces: You have a single public class (i.e. `class`) that allows 43 | you to centralize the management of the parameters and logic required 44 | for the configuration of the software the module manages, then smaller 45 | private subclasses that exist purely to batch together similar 46 | functionality and make it easier for developers to isolate the 47 | internal implementations of module functionality (such as configuring 48 | all the configuration files). It's almost the same as developing 49 | functions in real languages, they exist to isolate functionality and 50 | enable safe refactoring without having to change other pieces of the 51 | system. 52 | 53 | Where possible, names of classes should be self-evident and organized 54 | underneath the public class. For instance, 55 | 56 | * class::server 57 | * class::server::private_class 58 | 59 | ##Parameters 60 | 61 | One of the hardest parts of building a module is designing the Public 62 | API, the parameters for all the classes. A module with too few 63 | parameters is inflexible, whilst a module with too many parameters 64 | loses focus and thus efficacy because it has no opinions. Ideally 65 | your module would be opinionated without losing all flexibility, 66 | keeping in mind the scope of your module. This might mean creating 67 | parameters for the most common options you wish to configure, as well 68 | as with escape-hatch parameters that allow a user to pass in custom 69 | files or hashes. 70 | 71 | In this section, we will cover Puppet Labs' best practices for 72 | [naming](#naming) parameters, deciding how [many](#amount) parameters 73 | to create, and appropriately 74 | [separating](#separating-logic-from-configuration) module parameters. 75 | 76 | ###Naming 77 | 78 | The first rule of naming parameters is simple: "Make them obvious." 79 | Parameters with obvious names include 80 | 81 | * package_ensure 82 | * service_enable 83 | * service_manage 84 | 85 | All of the above parameters rely on combining the resource type with 86 | the property to make obvious, self-documenting names. For 87 | application-specific parameters, we recommend using the terms 88 | contained within the product, such as 89 | 90 | * keys_file 91 | * panic 92 | * restrict 93 | * servers 94 | 95 | The above parameters were taken from the [ntp 96 | module](http://forge.puppetlabs.com/puppetlabs/ntp) and are mostly 97 | examples of single-word parameters. We could have used names like 98 | `servers_list`, `restrict_array`, or`panic_boolean`, but we must 99 | assume that the user generally understands the configuration file 100 | options commonly used in NTP and knows what type of data they require 101 | (a list of servers vs a true/false value, for example). 102 | 103 | It's a mistake to make parameters with names that don't reflect the 104 | knowledge of experienced users of the software you're managing in 105 | order to increase the ease of use for inexperienced users. It's 106 | easier for them to learn the technology than for experienced users to 107 | learn your terminology. Naming a parameter `local_clock_boolean` 108 | instead of `udlc` would confuse an experienced NTP user and force them 109 | to check the README or code to ensure they understood what the 110 | parameter did. 111 | 112 | In order to assist users reading the code of your module, you should 113 | separate out the parameters that control configuration options from 114 | those parameters that control the logical flow of the module. This 115 | means you may end up with a class that looks like: 116 | 117 | ``` 118 | class software( 119 | # Logic 120 | $service_manage, 121 | $epel, 122 | $dev_enable, 123 | # Configuration 124 | $servers_list, 125 | $blacklist, 126 | $whitelist, 127 | ) {} 128 | ``` 129 | 130 | ###Scoping (size/responsibility/different word) 131 | 132 | Scoping the purpose and size of your module is critical. In order to 133 | build a module to manage SSH we must consider what functionality falls 134 | under “managing SSH”. It’s relatively easy to create a list of things 135 | that would fall under this category: 136 | 137 | * Managing the ssh packages. 138 | * Managing the ssh service. 139 | * Global ssh client configuration. 140 | * Global ssh server configuration. 141 | * Individual user ssh configuration. 142 | 143 | Scoping gets harder when you consider related functionality like: 144 | 145 | * Managing ssh host keys. 146 | * Managing individual user ssh keys. 147 | 148 | However, both of those tasks can be done independently of managing ssh 149 | as a daemon and therefore would be better off in a separate module, 150 | allowing them to be used by someone who already has an existing method 151 | by which they manage the ssh daemon. 152 | 153 | For the purposes of this guide we’ll be working to manage the five 154 | items listed above. 155 | 156 | ###Managing the ssh packages. 157 | 158 | We'll start our ssh module using the puppet module tool (PMT): 159 | 160 | ``` 161 | $ puppet module generate puppetlabs-ssh 162 | Notice: Generating module at /Users/apenney/modules/puppetlabs-ssh 163 | puppetlabs-ssh 164 | puppetlabs-ssh/Modulefile 165 | puppetlabs-ssh/README 166 | puppetlabs-ssh/manifests 167 | puppetlabs-ssh/manifests/init.pp 168 | puppetlabs-ssh/spec 169 | puppetlabs-ssh/spec/spec_helper.rb 170 | puppetlabs-ssh/tests 171 | puppetlabs-ssh/tests/init.pp 172 | ``` 173 | 174 | Using the PMT is the suggested way of creating modules, as it provides 175 | a quick and easy way of obtaining all of the basic module files we'll 176 | need. 177 | 178 | We’ll start by creating some skeleton classes for our module so we 179 | have a place to manage packages from. The below tree output shows the 180 | tree of classes for both client and manifests server. 181 | 182 | ``` 183 | $ tree 184 | . 185 | ├── client 186 | │ └── install.pp 187 | ├── client.pp 188 | ├── init.pp 189 | ├── params.pp 190 | ├── server 191 | │ ├── install.pp 192 | └── server.pp 193 | ``` 194 | 195 | This includes the main ssh::server and ssh::client classes as well as 196 | the specific install classes for both pieces. We split out the server 197 | and client right at the start so that they can be individually 198 | managed. The main client and server classes look like: 199 | 200 | ``` 201 | class ssh::client ( 202 | $package_ensure = $ssh::params::client_package_ensure, 203 | $package_name = $ssh::params::client_package_name, 204 | ) inherits ssh::params { 205 | 206 | contain ssh::client::install 207 | 208 | } 209 | ``` 210 | 211 | ```puppet 212 | class ssh::server( 213 | $package_ensure = $ssh::params::server_package_ensure, 214 | $package_name = $ssh::params::server_package_name, 215 | ) inherits ssh::params { 216 | 217 | contain ssh::server::install 218 | 219 | } 220 | ``` 221 | 222 | As discussed in the parameters section we’ve used the pattern of 223 | “resource_property” to create these parameter names. 224 | 225 | If `contain` is new to you then a brief description is that it 226 | replaces the anchor{} resource of the past and allows you to properly 227 | contain classes within the scope of other classes so that they don’t 228 | drift. Further information can be found at [Language 229 | Containment](http://docs.puppetlabs.com/puppet/latest/reference/lang_containment.html) 230 | 231 | This leaves us with ::install classes that look like: 232 | 233 | ``` 234 | class ssh::client::install inherits ssh::client { 235 | 236 | if $caller_module_name != $module_name { 237 | fail("Use of private class ${name} by ${caller_module_name}") 238 | } 239 | 240 | package { 'ssh': 241 | ensure => $package_ensure, 242 | name => $package_name, 243 | } 244 | 245 | } 246 | 247 | class ssh::server::install inherits ssh::server { 248 | 249 | if $caller_module_name != $module_name { 250 | fail("Use of private class ${name} by ${caller_module_name}") 251 | } 252 | 253 | package { 'sshd': 254 | ensure => $package_ensure, 255 | name => $package_name, 256 | } 257 | 258 | } 259 | ``` 260 | 261 | While most of these classes are unsurprising we do two things that are 262 | less common in existing modules. First, these classes inherit the 263 | main class. This brings the parameters from ssh::server and 264 | ssh::client in scope to these private classes, allowing us to simply 265 | reference $package_ensure rather than having to pass the parameters 266 | into these classes. Secondly, we explicitly mark these classes as 267 | being private with: 268 | 269 | ``` 270 | if $caller_module_name != $module_name { 271 | fail("Use of private class ${name} by ${caller_module_name}") 272 | } 273 | ``` 274 | 275 | Which causes compilation of the Puppet catalog to fail if the class 276 | was included anywhere by in the scope of a class within the module. 277 | If you were to add ssh::server::install to site.pp or to the console 278 | of Puppet Enterprise it would immediately fail. 279 | 280 | ###Managing the ssh service 281 | 282 | We’ll extend our module by adding a new class, ssh::server::service. 283 | Earlier, when we made the decision to split the classes into ::server 284 | and ::client, it was influenced by the realization that there’s no 285 | such thing as a client service, so we would have needed to introduce 286 | logic to protect against managing the service if you were only trying 287 | to manage the client. 288 | 289 | ``` 290 | class ssh::server::service inherits ssh::server { 291 | 292 | if $caller_module_name != $module_name { 293 | fail("Use of private class ${name} by ${caller_module_name}") 294 | } 295 | 296 | service { 'sshd': 297 | ensure => $service_ensure, 298 | name => $service_name, 299 | } 300 | 301 | } 302 | ``` 303 | 304 | We then need to modify our main ssh::server class to include this: 305 | 306 | ``` 307 | class ssh::server( 308 | $package_ensure = $ssh::params::server_package_ensure, 309 | $package_name = $ssh::params::server_package_name, 310 | $service_ensure = $ssh::params::server_service_ensure, 311 | $service_name = $ssh::params::server_service_name, 312 | ) inherits ssh::params { 313 | 314 | # We declare the classes before containing them. 315 | class { 'ssh::server::install': } -> 316 | class { 'ssh::server::service': } 317 | 318 | contain ssh::server::install 319 | contain ssh::server::service 320 | } 321 | ``` 322 | 323 | The biggest change here is the introduction of the `->` chaining 324 | syntax. This sets up ordering between the classes, as documented at 325 | [docs.puppetlabs.com](http://docs.puppetlabs.com/learning/ordering.html), 326 | ensuring that the ssh::server::install class runs before the 327 | ssh::server::service class. Without this we run the risk of service 328 | attempting to run before the ssh packages are installed. 329 | 330 | We prefer to create dependencies at the class level rather than at the 331 | resource level in order to free ourselves up to internally refactor 332 | the classes without having to constantly remember to update 333 | dependencies in numerous resources. This pattern also naturally lends 334 | itself to handling external dependencies, such as: 335 | 336 | Class['apt'] -> Class['ssh::server'] 337 | 338 | This would allow you to ensure the contents of the apt module ran 339 | before any of the ssh::server subclasses attempted to do anything with 340 | packages. 341 | 342 | There will be times when it makes sense to break these rules, but it 343 | should be infrequent. By moving your composition of dependencies to 344 | the class level you shield yourself from constantly updating 345 | dependencies whenever the internals of a class changes. It is better 346 | to split a class into two if you need to depend on only half of it 347 | than it is to fall into the antipattern of requiring resources. 348 | 349 | ###Global ssh client configuration 350 | 351 | Now we wish to address the global ssh client configuration, which is 352 | stored in a file called ssh_config. We first create a 353 | ssh::client::config class: 354 | 355 | ``` 356 | class ssh::client::config inherits ssh::client { 357 | 358 | if $caller_module_name != $module_name { 359 | fail("Use of private class ${name} by ${caller_module_name}") 360 | } 361 | 362 | file { 'ssh_config': 363 | ensure => $config_ensure, 364 | path => $config_path, 365 | content => template('ssh/ssh_config.erb'), 366 | } 367 | 368 | } 369 | ``` 370 | 371 | We then modify the main ssh::client class to include these new 372 | parameters and this class: 373 | 374 | ``` 375 | class ssh::client ( 376 | $config_ensure = $ssh::params::client_config_ensure, 377 | $config_path = $ssh::params::client_config_path, 378 | $package_ensure = $ssh::params::client_package_ensure, 379 | $package_name = $ssh::params::client_package_name, 380 | ) inherits ssh::params { 381 | 382 | # We declare the classes before containing them. 383 | class { 'ssh::client::install': } -> 384 | class { 'ssh::client::config': } 385 | 386 | contain ssh::client::install 387 | contain ssh::client::config 388 | } 389 | ``` 390 | 391 | With the framework for configuring the SSH client in place we need to 392 | leverage our domain knowledge of SSH to pick out things that users are 393 | likely to configure. The ssh_config file is constructed of a Host 394 | entry, and then a number of indented parameters below that host entry. 395 | 396 | There’s a number of choices that can be made here as to how flexible 397 | to be with this module. We may wish to allow you to set any number of 398 | hosts and their parameters, and we’d need to use defines and possibly 399 | puppetlabs-concat to do this. 400 | 401 | For the purposes of this module we’ll exclusively manage ‘Host *’, 402 | parameters that apply to all hosts. We’ll keep things simple by 403 | managing the following things: 404 | 405 | * ForwardAgent 406 | * ForwardX11 407 | * PasswordAuthentication 408 | * Port 409 | * Protocol 410 | 411 | This means our template will look like: 412 | 413 | ``` 414 | ## 415 | ## Managed by Puppet 416 | ## 417 | 418 | Host * 419 | ForwardAgent <%= forward_agent %> 420 | ForwardX11 <%= forward_x11 %> 421 | PasswordAuthentication <%= password_authentication %> 422 | Port <%= port %> 423 | Protocol <%= protocol %> 424 | ``` 425 | 426 | We add these parameters to ssh::client, which becomes: 427 | 428 | ``` 429 | class ssh::client ( 430 | $config_ensure = $ssh::params::client_config_ensure, 431 | $config_path = $ssh::params::client_config_path, 432 | $package_ensure = $ssh::params::client_package_ensure, 433 | $package_name = $ssh::params::client_package_name, 434 | # Configuration 435 | $forward_agent = $ssh::params::client_forward_agent, 436 | $forward_x11 = $ssh::params::client_forward_x11, 437 | $password_authentication = $ssh::params::client_password_authentication, 438 | $port = $ssh::params::client_port, 439 | $protocol = $ssh::params::client_protocol, 440 | ) inherits ssh::params { 441 | 442 | # We declare the classes before containing them. 443 | class { 'ssh::client::install': } -> 444 | class { 'ssh::client::config': } 445 | 446 | contain ssh::client::install 447 | contain ssh::client::config 448 | } 449 | ``` 450 | 451 | This is a simplified selection, and it raises the question of why we 452 | didn’t choose to manage all of the possible parameters for 453 | sshd_config. There’s several answers here, but the most important is 454 | that a module with tens, or hundreds, of parameters is simply too 455 | unwieldy to use, or develop, and it becomes extremely difficult to 456 | maintain with time. There’s several causes of too many parameters, 457 | but two that you are most likely to face are too broad of a module 458 | scope, or reliance on individual parameters rather than more 459 | sophisticated data handling. 460 | 461 | The first case is simple; in some cases the module scope is simply too 462 | broad. You’re trying to manage too many things and you’d be better 463 | off breaking the module into several. 464 | 465 | The second case is more complex to deal with but it comes down to data 466 | structures. In the case of ssh_config we could have created a 467 | parameter per piece of configuration you can pass to a host, but that 468 | gets complex and requires you to keep updating the module as ssh 469 | changes. 470 | 471 | Given a configuration file of: 472 | 473 | Host a 474 | Parameter 1 475 | 476 | Host b 477 | Parameter 2 478 | 479 | You could create a `configuration_data` parameter in ssh::client and 480 | pass it: 481 | 482 | [ {‘a’ => {‘Parameter’ => ‘1’}}, {‘b’ => {‘Parameter’ => ‘2’}} ] 483 | 484 | This is an array of hashes, each of which contains a key that points 485 | to another sub-hash of parameters. It’s complex if you’re not used to 486 | dealing with nested data structures, but it’s arbitrarily expandable 487 | to any number of hosts, and parameters, and is contained within a 488 | single parameter meaning that any possible ssh_configuration data is 489 | expressible. 490 | 491 | To see this in action we can modify our ssh::client work as follows: 492 | 493 | ``` 494 | class ssh::client ( 495 | $config_ensure = $ssh::params::client_config_ensure, 496 | $config_path = $ssh::params::client_config_path, 497 | $package_ensure = $ssh::params::client_package_ensure, 498 | $package_name = $ssh::params::client_package_name, 499 | # Configuration 500 | $configuration_data = $ssh::params::client_configuration_data, 501 | ) inherits ssh::params { 502 | 503 | # We declare the classes before containing them. 504 | class { 'ssh::client::install': } -> 505 | class { 'ssh::client::config': } 506 | 507 | contain ssh::client::install 508 | contain ssh::client::config 509 | } 510 | ``` 511 | 512 | We need to modify the template next: 513 | 514 | ``` 515 | ## 516 | ## Managed by Puppet 517 | ## 518 | 519 | <% @configuration_data.sort.each do |hash| -%> 520 | <% hash.each do |host, parameters| -%> 521 | Host <%= host %> 522 | <% if parameters.is_a?(Hash) -%> 523 | <% parameters.sort.map do |title, value| -%> 524 | <%= title %> <%= value %> 525 | <% end -%> 526 | <% end -%> 527 | <% end -%> 528 | <% end -%> 529 | ``` 530 | 531 | While this is a little more complex, it basically boils down to 532 | iterating over each hash within the array, then printing the key out 533 | as the name of the Host, then a further iteration over all the 534 | parameters contained in the subhash. The end result looks like: 535 | 536 | ``` 537 | ## 538 | ## Managed by Puppet 539 | ## 540 | 541 | Host * 542 | Port 22 543 | ``` 544 | 545 | ###Global ssh server configuration 546 | 547 | We now extend the module in exactly the same way to cover 548 | ssh::server::config. This is slightly less complex in that we don’t 549 | need to use nested hashes, just a single hash with entries for each 550 | parameter and their value. We start by making the ssh::server::config 551 | class and modifying the ssh::server class to include it. 552 | 553 | ``` 554 | class ssh::server::config inherits ssh::server { 555 | 556 | if $caller_module_name != $module_name { 557 | fail("Use of private class ${name} by ${caller_module_name}") 558 | } 559 | 560 | file { 'sshd_config': 561 | ensure => $config_ensure, 562 | path => $config_path, 563 | owner => 'root', 564 | group => 'root', 565 | mode => '0544', 566 | content => template('ssh/sshd_config.erb'), 567 | } 568 | 569 | } 570 | 571 | class ssh::server( 572 | $config_ensure = $ssh::params::server_config_ensure, 573 | $config_path = $ssh::params::server_config_path, 574 | $package_ensure = $ssh::params::server_package_ensure, 575 | $package_name = $ssh::params::server_package_name, 576 | $service_ensure = $ssh::params::server_service_ensure, 577 | $service_name = $ssh::params::server_service_name, 578 | # Configuration parameters 579 | $configuration_data = $ssh::params::server_configuration_data, 580 | $os_configuration_data = $ssh::params::server_os_configuration_data, 581 | ) inherits ssh::params { 582 | 583 | # We declare the classes before containing them. 584 | class { 'ssh::server::install': } -> 585 | class { 'ssh::server::config': } ~> 586 | class { 'ssh::server::service': } 587 | 588 | contain ssh::server::install 589 | contain ssh::server::config 590 | contain ssh::server::service 591 | 592 | } 593 | ``` 594 | 595 | We’ve introduced two new parameters in the above class, 596 | `configuration_data` and `os_configuration_data`. This makes it 597 | easier to provide data that differs per operating system in params.pp 598 | while letting you override the other sshd settings. 599 | 600 | Next we update params.pp to include all the new data. This file has 601 | grown at this point to handle Redhat and Debian, as well as provide 602 | sensible defaults: 603 | 604 | ``` 605 | class ssh::params { 606 | 607 | # Server parameters 608 | $server_config_ensure = 'present' 609 | $server_package_ensure = 'present' 610 | $server_service_ensure = 'running' 611 | $server_configuration_data = { 612 | 'Port' => '22', 613 | 'Protocol' => '2', 614 | 'UsePrivilegeSeparation' => 'yes', 615 | 'KeyRegenerationInterval' => '3600', 616 | 'ServerKeyBits' => '768', 617 | 'SyslogFacility' => 'AUTH', 618 | 'LogLevel' => 'INFO', 619 | 'LoginGraceTime' => '120', 620 | 'PermitRootLogin' => 'yes', 621 | 'StrictModes' => 'yes', 622 | 'RSAAuthentication' => 'yes', 623 | 'PubkeyAuthentication' => 'yes', 624 | 'IgnoreRhosts' => 'yes', 625 | 'RhostsRSAAuthentication' => 'no', 626 | 'HostbasedAuthentication' => 'no', 627 | 'PermitEmptyPasswords' => 'no', 628 | 'ChallengeResponseAuthentication' => 'no', 629 | 'PasswordAuthentication' => 'yes', 630 | 'X11Forwarding' => 'yes', 631 | 'X11DisplayOffset' => '10', 632 | 'PrintMotd' => 'no', 633 | 'PrintLastLog' => 'yes', 634 | 'TCPKeepAlive' => 'yes', 635 | 'AcceptEnv' => 'LANG LC_*', 636 | 'UsePAM' => 'yes', 637 | } 638 | # Client parameters 639 | $client_config_ensure = 'present' 640 | $client_package_ensure = 'present' 641 | $client_configuration_data = [ { '*' => {} } ] 642 | 643 | case $::osfamily { 644 | 'Redhat': { 645 | $client_config_path = '/etc/ssh/ssh_config' 646 | $client_package_name = 'openssh-clients' 647 | 648 | $server_config_path = '/etc/ssh/sshd_config' 649 | $server_package_name = 'openssh-server' 650 | $server_service_name = 'sshd' 651 | $server_os_configuration_data = { 652 | 'HostKey' => [], 653 | 'Subsystem' => 'sftp /usr/libexec/openssh/sftp-server /usr/lib/openssh/sftp-server', 654 | } 655 | } 656 | 'Debian': { 657 | $client_config_path = '/etc/ssh/ssh_config' 658 | $client_package_name = 'openssh-client' 659 | 660 | $server_config_path = '/etc/ssh/sshd_config' 661 | $server_package_name = 'openssh-server' 662 | $server_service_name = 'ssh' 663 | $server_os_configuration_data = { 664 | 'HostKey' => [ '/etc/ssh/ssh_host_rsa_key', '/etc/ssh/ssh_host_dsa_key', '/etc/ssh/ssh_host_ecdsa_key' ], 665 | 'Subsystem' => 'sftp /usr/lib/openssh/sftp-server /usr/lib/openssh/sftp-server', 666 | } 667 | } 668 | default: { 669 | fail("${::module} is unsupported on ${::osfamily}") 670 | } 671 | } 672 | 673 | } 674 | ``` 675 | 676 | The final piece we need to build out is the template. This has the 677 | two hashes in it, and for each of the parameters it checks to see if 678 | the values are an array, if so it repeats the parameter name and then 679 | the entry for each element of the array. This means we can handle 680 | multiple hostkeys. 681 | 682 | ``` 683 | ## 684 | ## Managed by Puppet 685 | ## 686 | 687 | <% @configuration_data.sort.each do |parameter, value| -%> 688 | <% if value.is_a?(Array) -%> 689 | <% value.each do |v| -%> 690 | <%= parameter %> <%= v %> 691 | <% end -%> 692 | <% else -%> 693 | <%= parameter %> <%= value %> 694 | <% end -%> 695 | <% end -%> 696 | 697 | # OS specific entries. 698 | <% @os_configuration_data.sort.each do |parameter, value| -%> 699 | <% if value.is_a?(Array) -%> 700 | <% value.each do |v| -%> 701 | <%= parameter %> <%= v %> 702 | <% end -%> 703 | <% else -%> 704 | <%= parameter %> <%= value %> 705 | <% end -%> 706 | <% end -%> 707 | ``` 708 | 709 | ###Individual user ssh configuration 710 | 711 | This piece of the module is almost the same as the client 712 | configuration, but we need to be able to take in a user name to build 713 | out the appropriate configuration. As a result this makes sense to 714 | write as a define instead of a class. 715 | 716 | We only need a few parameters here and we make the assumption that 717 | users live in /home, but you can see that the only real difference 718 | is that we allow the owner/group to be set for the file. We reuse 719 | the same template as the previous ssh::client::config class. 720 | 721 | ```puppet 722 | define ssh::user( 723 | $ensure = present, 724 | $owner = $name, 725 | $group = $name, 726 | $configuration_data = [ { '*' => {} } ] 727 | ) { 728 | 729 | file { "${name}_ssh": 730 | ensure => directory, 731 | path => "/home/${name}/.ssh/", 732 | owner => $owner, 733 | group => $group, 734 | mode => '0700', 735 | } -> 736 | 737 | file { "${name}_ssh_config": 738 | ensure => $ensure, 739 | path => "/home/${name}/.ssh/ssh_config", 740 | owner => $owner, 741 | group => $group, 742 | mode => '0600', 743 | content => template('ssh/ssh_config.erb'), 744 | } 745 | 746 | } 747 | ``` 748 | 749 | ###Documentation 750 | 751 | At the end of all this work on the module itself we should be sure that we 752 | appropriately document it. We have a README template available on our (website)[whereisthereadme] 753 | which you should use as the basis of your documentation as it contains all the 754 | appropriate sections to fill out. In our case we'll document out the classes 755 | and defines we made. 756 | 757 | ``` 758 | #ssh 759 | 760 | ####Table of Contents 761 | 762 | 1. [Overview](#overview) 763 | 2. [Module Description - What the module does and why it is useful](#ssh-description) 764 | 3. [Setup - The basics of getting started with [Ssh]](#setup) 765 | * [What [Ssh] affects](#what-[ssh]-affects) 766 | * [Setup requirements](#setup-requirements) 767 | * [Beginning with [Ssh]](#beginning-with-[Ssh]) 768 | 4. [Usage - Configuration options and additional functionality](#usage) 769 | 5. [Reference - An under-the-hood peek at what the module is doing and how](#reference) 770 | 5. [Limitations - OS compatibility, etc.](#limitations) 771 | 6. [Development - Guide for contributing to the module](#development) 772 | 773 | ##Overview 774 | 775 | This module manages OpenSSH's daemon, server configuration, client 776 | configuration, and user specific client configuration. 777 | 778 | ##Module Description 779 | 780 | This module manages OpenSSH and the associated configuration with both server 781 | and clients. It'll also allow you to configure per user configuration. 782 | 783 | ##Setup 784 | 785 | ###What [Ssh] affects 786 | 787 | * sshd_config 788 | * ssh_config 789 | * ~/.ssh/ files. 790 | * sshd daemon. 791 | 792 | ###Beginning with [Ssh] 793 | 794 | In order to ensure sshd is running you just need to: 795 | 796 | ``` 797 | include ssh::server 798 | ``` 799 | 800 | ##Usage 801 | 802 | The two main classes for this module are `ssh::server` and `ssh::client`. 803 | Through these two classes you should be able to manage ssh comprehensively. 804 | 805 | ##Reference 806 | 807 | ###Classes 808 | 809 | * ssh::client - Main class for managing the ssh client global settings. 810 | * ssh::client::install - Installs the main ssh client. 811 | * ssh::client::config - Configs the main ssh client. 812 | * ssh::server - Main class for managing the ssh server. 813 | * ssh::server::install - Installs the ssh server. 814 | * ssh::server::config - Configures the ssh server. 815 | * ssh::server::service - Configures the ssh service. 816 | * ssh::params - Default parameters. 817 | 818 | ###Parameters 819 | 820 | ###ssh::server 821 | 822 | ####`config_ensure` 823 | Should the config file be present or absent. 824 | 825 | ####`config_path` 826 | 827 | The path for the configuration file. 828 | 829 | ####`package_ensure` 830 | 831 | Should the package be present or absent. 832 | 833 | ####`package_name` 834 | 835 | What is the name of the ssh server package? 836 | 837 | ####`service_ensure` 838 | 839 | Should be the service be running or stopped. 840 | 841 | ####`service_name` 842 | 843 | What is the name of the ssh server service? 844 | 845 | ####`configuration_data` 846 | 847 | This allows you to pass in arbitrary configuration to the sshd_config file. You 848 | create it in the format of: 849 | 850 | { 'Parameter' => 'Value' } 851 | 852 | To change any individual value in this hash you'll need to copy the entire 853 | hash from the params.pp file and modify it to taste. 854 | 855 | ####`os_configuration_data` 856 | 857 | Similar to the `configuration_data` parameter this is a hash containing 858 | configuration data that differs between operating systems. This separation 859 | allows you to set only OS specific configuration settings without touching the 860 | main hash if you so wish. 861 | 862 | ###ssh::client 863 | 864 | ####`config_ensure` 865 | 866 | Should the config file be present or absent. 867 | 868 | ####`config_path` 869 | 870 | The path for the configuration file. 871 | 872 | ####`package_ensure` 873 | 874 | Should the package be present or absent. 875 | 876 | ####`package_name` 877 | 878 | What is the name of the ssh client package? 879 | 880 | ####`configuration_data` 881 | 882 | This parameter allows you to create configuration entries for multiple 883 | hosts. You must create an array of hashes, containing subhashes, such as: 884 | 885 | [ { '*' => { 'Port' => '2222' }, 'testhost' => { 'Port => '22' } } ] 886 | 887 | ###Defines 888 | 889 | * ssh::user 890 | 891 | ###Parameters 892 | 893 | ###ssh::user 894 | 895 | ####`ensure` 896 | 897 | Should the config file be present or absent. 898 | 899 | ####`owner` 900 | 901 | Who should own the configuration file. 902 | 903 | ####`group` 904 | 905 | Which group should own the configuration file. 906 | 907 | ####`configuration_data` 908 | 909 | This parameter allows you to create configuration entries for multiple 910 | hosts. You must create an array of hashes, containing subhashes, such as: 911 | 912 | [ { '*' => { 'Port' => '2222' }, 'testhost' => { 'Port => '22' } } ] 913 | 914 | 915 | ##Limitations 916 | 917 | This module only supports the following osfamily: 918 | 919 | * Debian 920 | * RedHat 921 | 922 | ##Development 923 | ``` 924 | 925 | 926 | -------------------------------------------------------------------------------- /tests.md: -------------------------------------------------------------------------------- 1 | -> Test planning should be a doc before this. 2 | 3 | #Introduction 4 | 5 | Extending on from the three previous guides, 6 | [Classes, Params, APIs](link), [Types and Providers](link), and 7 | [Facts](link), we finally introduce testing into our SSH module. 8 | Normally you would write tests as you go, rather than retrofit them in 9 | at the end, but it's very common to inherit a module that has no tests 10 | and it's good to know how to start retrofitting them in. 11 | 12 | We’re now going to retrofit in `puppet-rspec` unit tests, rspec unit 13 | tests for the type/provider, rspec unit tests for the fact, and Beaker 14 | acceptance tests to test the full workings of the module. 15 | 16 | For examples of all of the testing we’re going to do you can 17 | investigate the puppetlabs modules 18 | [on the forge](https://forge.puppetlabs.com). Modules like MySQL, 19 | Apache, RabbitMQ, PostgreSQL and Apt have all sorts of tests and you 20 | can often steal entire chunks for use in your own testing. 21 | 22 | #Manifest Testing 23 | 24 | Rspec-puppet is the framework that we use for manifest testing. 25 | Written and maintained by Tim Sharpe, better known as `rodjek`, a 26 | community member and Githubber. Rspec-puppet extends rspec by adding 27 | matchers and other infrastructure to allow it to understand Puppet 28 | catalogs, which allows you to make assertions about the state of your 29 | catalog in the presence of certain parameters, variables, and facts. 30 | 31 | For rspec-puppet testing you want to cover your “entry” classes, 32 | rather than having a spec test for the private subclasses. That means 33 | for our module we just need to test ssh::client and ssh::server. 34 | 35 | Before we start the actual manifest testing we’ll cover setting up 36 | your module to be testable: 37 | 38 | ##Setup 39 | 40 | In order to test a module we need to do several things; add a Gemfile 41 | with the testing framework dependencies, add a spec_helper.rb file, a 42 | .fixtures.yml and a Rakefile. Our Gemfile will be fairly barebones 43 | and borrowed from the puppetlabs modules: 44 | 45 | The opening section just sets the source of the gems we use, and 46 | creates a “development” group for [Bundler](http://bundler.io/) to 47 | consume later. 48 | 49 | ``` 50 | source 'https://rubygems.org' 51 | 52 | group :development, :test do 53 | ``` 54 | 55 | This is our list of gems to install. I won’t talk about all of them, 56 | but ‘Rake’ is used for a kind of Makefile that allows us to type `rake 57 | spec` to run tests, ‘puppetlabs_spec_helper’ adds some rake tests, and 58 | the last two are for code coverage and puppet-lint runs via Rake. 59 | 60 | ``` 61 | gem 'rake', :require => false 62 | gem 'rspec-puppet', :require => false 63 | gem 'puppetlabs_spec_helper', :require => false 64 | gem 'puppet-lint', :require => false 65 | gem 'simplecov', :require => false 66 | end 67 | 68 | if puppetversion = ENV['PUPPET_GEM_VERSION'] 69 | gem 'puppet', puppetversion, :require => false 70 | else 71 | gem 'puppet', :require => false 72 | end 73 | 74 | # vim:ft=ruby 75 | ``` 76 | 77 | With our Gemfile in place we can now add the Rakefile: 78 | 79 | ``` 80 | require 'puppetlabs_spec_helper/rake_tasks' 81 | ``` 82 | 83 | Then the .fixtures.yml file, which is used to download and setup any 84 | dependencies when doing testing. We need to create a symlink for our 85 | own module into the testing space, as well as any other modules that 86 | we depend on. This means you'll have to constantly update this file 87 | any time you add a Modulefile dependency. 88 | 89 | ``` 90 | fixtures: 91 | repositories: 92 | "stdlib": "git://github.com/puppetlabs/puppetlabs-stdlib" 93 | symlinks: 94 | "ssh": "#{source_dir}" 95 | ``` 96 | 97 | And lastly our spec/spec_helper.rb. We require and include simplecov 98 | as this will generate code coverage reports for types and providers. 99 | Sadly it doesn’t work for manifests. Lastly it includes 100 | puppetlabs_spec_helper which does all the rest of the work for us: 101 | 102 | ``` 103 | require 'simplecov' 104 | SimpleCov.start do 105 | add_filter "/spec/" 106 | end 107 | require 'puppetlabs_spec_helper/module_spec_helper' 108 | ``` 109 | 110 | At this point you should be able to issue `be rake spec`. If this 111 | fails then the rest of this section won’t work for you. You can grab 112 | the module with all the required files 113 | [at github](https://github.com/apenney/puppetlabs-ssh/) to ensure 114 | there’s no issues in your local copy. 115 | 116 | ``` 117 | $ be rake spec 118 | /opt/boxen/rbenv/versions/1.9.3-p448/bin/ruby -S rspec --color 119 | No examples found. 120 | 121 | 122 | Finished in 0.00009 seconds 123 | 0 examples, 0 failures 124 | ``` 125 | 126 | ##Demonstration 127 | 128 | Now we have the module ready for testing we can start by adding some 129 | simple tests for ssh::client in the file 130 | spec/classes/ssh_client_spec.rb. (All spec tests should end in 131 | _spec.rb or the rake task won’t find them). 132 | 133 | We open the file with a description of the class we want to test: 134 | 135 | ``` 136 | require 'spec_helper' 137 | 138 | describe 'ssh::client' do 139 | ``` 140 | 141 | If you want to test multiple distributions or systems in rspec-puppet 142 | you have to “mock” out the facts for that system to trick Puppet into 143 | supplying a catalog for that system. In order to do this in our tests 144 | we create a ‘shared_example’, which is basically a chunk of code you 145 | can pass parameters into elsewhere in the test. This lets us avoid 146 | repeating ourselves to test multiple distributions. 147 | 148 | Within this shared_example block we named ‘client’ we create two 149 | separate tests, one for the package, and one for the configuration 150 | file. To make sure our catalog works properly we’ll need to check 151 | that the package_name is right and that the package_path is correct. 152 | We use the variables in the shared_examples block to fill in the gaps 153 | in the .with() sections of our tests. Lastly we'll make sure the 154 | configuration file contains something that matches our defaults. 155 | 156 | ```ruby 157 | shared_examples 'client' do |osfamily, package_name, package_path| 158 | let(:facts) {{ :osfamily => osfamily }} 159 | 160 | it 'contains the package' do 161 | should contain_package('ssh').with( 162 | :ensure => 'present', 163 | :name => package_name 164 | ) 165 | end 166 | 167 | it 'contains the ssh_config file' do 168 | should contain_file('ssh_config').with( 169 | :ensure => 'present', 170 | :path => package_path 171 | ) 172 | end 173 | 174 | it 'contains default configuration' do 175 | should contain_file('ssh_config').with_content( 176 | /Host */ 177 | ) 178 | end 179 | end 180 | ``` 181 | 182 | Now we call our ‘client’ shared_examples by passing variables into a 183 | method called ‘it_behaves_like’. We pass in the name of the 184 | shared_example first and then the other variables in order. This 185 | means that osfamily in the ‘client’ example gets get to ‘RedHat’ in 186 | the first example below. 187 | 188 | ``` 189 | context 'RedHat' do 190 | it_behaves_like 'client', 'RedHat', 'openssh-clients', '/etc/ssh/ssh_config' 191 | end 192 | 193 | context 'Debian' do 194 | it_behaves_like 'client', 'Debian', 'openssh-client', '/etc/ssh/ssh_config' 195 | end 196 | ``` 197 | 198 | Here we test an unsupported operating system. We set the fact to make 199 | the osfamily something we didn’t account for in ssh::params and inside 200 | the it{} block we state that “we expect (including the class) to raise 201 | an error()” which it does, telling us that the class is unsupported. 202 | 203 | ``` 204 | context 'Unsupported' do 205 | let(:facts) {{ :osfamily => 'Unsupported' }} 206 | it { expect { should contain_class('ssh::client') }.to raise_error(Puppet::Error, /is unsupported on Unsupported/) } 207 | end 208 | 209 | end 210 | ``` 211 | 212 | Now we switch to testing the ssh::server class. These tests are 213 | almost identical to ssh::client except for the additional of a 214 | ‘contains_server()’ block in the main shared_examples. 215 | 216 | ```ruby 217 | require 'spec_helper' 218 | 219 | describe 'ssh::server' do 220 | 221 | shared_examples 'server' do |osfamily, package_name, package_path, service_name| 222 | let(:facts) {{ :osfamily => osfamily }} 223 | 224 | it 'contains the package' do 225 | should contain_package('sshd').with( 226 | :ensure => 'present', 227 | :name => package_name 228 | ) 229 | end 230 | 231 | it 'contains the ssh_config file' do 232 | should contain_file('sshd_config').with( 233 | :ensure => 'present', 234 | :path => package_path 235 | ) 236 | end 237 | 238 | it 'contains the sshd service' do 239 | should contain_service('sshd').with( 240 | :ensure => 'running', 241 | :name => service_name 242 | ) 243 | end 244 | end 245 | 246 | context 'RedHat' do 247 | it_behaves_like 'server', 'RedHat', 'openssh-server', '/etc/ssh/sshd_config', 'sshd' 248 | end 249 | 250 | context 'Debian' do 251 | it_behaves_like 'server', 'Debian', 'openssh-server', '/etc/ssh/sshd_config', 'ssh' 252 | end 253 | 254 | context 'Unsupported' do 255 | let(:facts) {{ :osfamily => 'Unsupported' }} 256 | it { expect { should contain_class('ssh::server') }.to raise_error(Puppet::Error, /is unsupported on Unsupported/) } 257 | end 258 | ``` 259 | 260 | Lastly we can check that the configuration file contains the expected parameters 261 | from the defaults for Debian. We do this by checking the configuration file 262 | for various parameters we know are set. You could condense this to a single 263 | test that checks multiple parts of file with a complex regular expression, but 264 | this is more readable. Intent and clarity is more important than saving space 265 | within tests. This makes it easy to extend by adding new tests as we add new 266 | ssh configuration to the defaults. 267 | 268 | ```ruby 269 | context 'default template contents' do 270 | let(:facts) {{ :osfamily => 'Debian' }} 271 | 272 | it { should contain_file('sshd_config').with( 273 | :content => /Port 22/ 274 | )} 275 | it { should contain_file('sshd_config').with( 276 | :content => /Protocol 2/ 277 | )} 278 | it { should contain_file('sshd_config').with( 279 | :content => /PermitRootLogin yes/ 280 | )} 281 | it { should contain_file('sshd_config').with( 282 | :content => /PasswordAuthentication yes/ 283 | )} 284 | it { should contain_file('sshd_config').with( 285 | :content => /Subsystem sftp \/usr\/lib\/openssh\/sftp-server \/usr\/lib\/openssh\/sftp-server/ 286 | )} 287 | it { should contain_file('sshd_config').with( 288 | :content => /HostKey \/etc\/ssh\/ssh_host_rsa_key/ 289 | )} 290 | end 291 | 292 | end 293 | ``` 294 | 295 | ###Results 296 | 297 | When I wrote these tests against a development copy of the SSH module they 298 | immediately failed with errors. 299 | 300 | ``` 301 | 1) ssh::client RedHat behaves like client contains the package 302 | Failure/Error: ) 303 | expected that the catalogue would contain Package[ssh] with ensure set to `"present"` but it is set to `nil` in the catalogue, and parameter name set to `"openssh-clients"` but it is set to `"ssh"` in the catalogue 304 | Shared Example Group: "client" called from ./spec/classes/ssh_client_spec.rb:24 305 | # ./spec/classes/ssh_client_spec.rb:12:in `block (3 levels) in ' 306 | ``` 307 | 308 | Upon looking into this I found that ssh::client wasn’t picking up the 309 | information from ssh::params properly and when I checked manifests/client.pp it 310 | turned out that I had forgotten to inherit from ssh::params and it was indeed 311 | broken. 312 | 313 | This kind of thing is pretty common to discover when you write manifest tests 314 | for the first time, and you'll be surprised to find how easily this can expose 315 | missing parameters, logic bugs, and other simple mistakes. 316 | 317 | #Unit Testing 318 | 319 | Now we have functioning manifest tests we switch to traditional rspec unit 320 | testing. We’re going to test the code we wrote for facts earlier. Most module 321 | writers are not traditional developers, and so we’ll build out our tests in a 322 | slightly less sophisticated way than is supported by rspec in the interests of 323 | clarity. Our module team has found that the tests for types, providers, and 324 | facts, within Puppet can be terrible examples to learn from as they assume a 325 | high level of familiarity with ruby and with testing. We hope the below 326 | demonstration will be a bit more sysadmin oriented and easier to follow. 327 | 328 | ##Demonstration 329 | 330 | We need two separate spec tests, one for each fact. We start by including 331 | `spec_helper` and `etc`. Spec_helper is always included in our unit tests, but 332 | etc is only required for access to Struct::Passwd, the 333 | [struct](http://www.ruby-doc.org/core/Struct.html) that Etc.passwd returns when 334 | called. 335 | 336 | ```ruby 337 | require 'spec_helper' 338 | require 'etc' 339 | ``` 340 | 341 | Next we need to create some objects to work with. We're going to use the let() 342 | command, with the syntax `let(:name) { thing }`. We're going to create several 343 | objects, starting with singleuser. Singleuser is a Struct::Passwd.new() 344 | object, which contains a single fake /etc/passwd entry. Struct::Passwd.new() 345 | comes from the `etc` library we required earlier. We also create a multiuser 346 | array of two entries. Finally we create `keys`, containing a fake id_rsa.pub 347 | file. 348 | 349 | ``` 350 | describe 'public_key_comments', :type => :fact do 351 | let(:singleuser) { Struct::Passwd.new('test', nil, nil, 1, 1, '/home/test') } 352 | let(:multiuser) {[ 353 | Struct::Passwd.new('test', nil, nil, 1, 1, '/home/test'), 354 | Struct::Passwd.new('test2', nil, nil, 2, 2, '/var/tmp/test') 355 | ]} 356 | let(:keys) { 357 | "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVwwxl6dz6Y7karwyS8S+za4qcu99Ra8H8N3cVHanEB+vuigtbhLOSb+bk6NjxFtC/jF+Usf5FM5fGIYd51L7RE9BbzbKiWb9giFnNqhKWclO5CY4sQTyUyYiJTQKLuVtkmiFeArV+jIuthxm6JrdOeFx8lJpcgGlZjlcBGxp27EbZNGWIlAdvW0ZXy0JqS9M/vj71NBBDfkrpyzAPC0aBa9+FmywOH6HXbyeFooHLOw+mfzP87jwDDQ2yXIehDoC1BsLYXD+j+kdnR0CNltJh1PYOFNpbKQpfnPhfdw4Oc0hZ34n+kfBPavKlbwxoVAoisBWWo4c9ZnUoe2OBRHAX test@local 358 | ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQjmi7VZln/ehWW5nDgpHKWRSAOEUd//Qft00vqBq1khyGF6o0hUmLhjU1fxXv3w7GNshgoqTTsgMRNSOP06UaNJwU8g1Gyji9ard+mVtJ7ohgv/OSR2cujL/853Q/LOVo5LIgEKRxCyA1KAPE68n44WC3RIvUO78tk5flagAHYN/i+B8k4W040aqtEvTMjKGag7377eQIzp4GNJ8Hzm1VdFeZQQewAi/hrUOyU3gWoXTpN+xaWr41b4Vugrgb5V9/esDBXb+y2zj8Wc/hX2xc33crfWLkFh7YhmsgmhMEAKE8G2mZEG3Sx3/9BHNsleBTh0oJl5CZm+cBh+BCCGq/ test@127.0.0.1" 359 | } 360 | ``` 361 | 362 | Now we’re into the actual tests. We start by describing what the test is for, 363 | in this case a single user, then create our ‘before :each do’ block. This is 364 | code that is ran before each it{} within this describe block. In our case we 365 | start using ‘stubs’ as a way of faking the return of various functions that 366 | exist in our fact. This means when the fact attempts to execute we sneak in 367 | and replace the intended returns for various functions with our fake entries. 368 | 369 | Our first stub is for Etc.passwd, and we make it yield up the singleuser struct 370 | we had before. We next stub Dir[] with the test users .ssh directory returning 371 | a single file, id_rsa.pub. Last, we stub File.read() for that file to return 372 | the fake keys content we defined earlier. 373 | 374 | Inside the it{} block we then actually run Facter.fact() to call our real ruby 375 | fact. This code is run, but when the functions we stubbed are found it just 376 | skips running them for real and returns the fake stuff we stubbed out. 377 | 378 | We then use .value.should eq ‘thing’ to assert that our fact, when run with our 379 | fake data, should find two comments. When taken together this test ensures 380 | that all the logic of the fact works if fed with pregenerated data that we can 381 | make assertions against. 382 | 383 | ``` 384 | describe 'single user' do 385 | before :each do 386 | Etc.stubs(:passwd).yields(singleuser) 387 | Dir.stubs(:[]).with("/home/test/.ssh/*.pub").returns(["id_rsa.pub"]) 388 | File.stubs(:read).with('/home/test/.ssh/id_rsa.pub').returns(keys) 389 | end 390 | 391 | it { Facter.fact(:public_key_comments).value.should eq 'test@local,test@127.0.0.1' } 392 | end 393 | ``` 394 | 395 | This test is almost identical to the previous one, except we stub out the 396 | functions that are called for both the users found in the multiuser section. 397 | 398 | ``` 399 | describe 'multiple user' do 400 | before :each do 401 | Etc.stubs(:passwd).yields(multiuser) 402 | Dir.stubs(:[]).with("/home/test/.ssh/*.pub").returns(["id_rsa.pub"]) 403 | Dir.stubs(:[]).with("/var/tmp/test/.ssh/*.pub").returns(["id_rsa.pub"]) 404 | File.stubs(:read).with('/home/test/.ssh/id_rsa.pub').returns(keys) 405 | File.stubs(:read).with('/var/tmp/test/.ssh/id_rsa.pub').returns(keys) 406 | end 407 | 408 | it { Facter.fact(:public_key_comments).value.should eq 'test@local,test@127.0.0.1' } 409 | end 410 | 411 | end 412 | ``` 413 | 414 | Our tests for the second fact are extremely similar to our other test. We 415 | define the same let()s, the same describe blocks, the only differences come in 416 | the stubbed functions. 417 | 418 | ``` 419 | require 'spec_helper' 420 | require 'etc' 421 | 422 | describe 'public_key_comments', :type => :fact do 423 | let(:singleuser) { Struct::Passwd.new('test', nil, nil, 1, 1, '/home/test') } 424 | let(:multiuser) {[ 425 | Struct::Passwd.new('test', nil, nil, 1, 1, '/home/test'), 426 | Struct::Passwd.new('test2', nil, nil, 2, 2, '/var/tmp/test') 427 | ]} 428 | let(:keys) { 429 | "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVwwxl6dz6Y7karwyS8S+za4qcu99Ra8H8N3cVHanEB+vuigtbhLOSb+bk6NjxFtC/jF+Usf5FM5fGIYd51L7RE9BbzbKiWb9giFnNqhKWclO5CY4sQTyUyYiJTQKLuVtkmiFeArV+jIuthxm6JrdOeFx8lJpcgGlZjlcBGxp27EbZNGWIlAdvW0ZXy0JqS9M/vj71NBBDfkrpyzAPC0aBa9+FmywOH6HXbyeFooHLOw+mfzP87jwDDQ2yXIehDoC1BsLYXD+j+kdnR0CNltJh1PYOFNpbKQpfnPhfdw4Oc0hZ34n+kfBPavKlbwxoVAoisBWWo4c9ZnUoe2OBRHAX test@local 430 | ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDQjmi7VZln/ehWW5nDgpHKWRSAOEUd//Qft00vqBq1khyGF6o0hUmLhjU1fxXv3w7GNshgoqTTsgMRNSOP06UaNJwU8g1Gyji9ard+mVtJ7ohgv/OSR2cujL/853Q/LOVo5LIgEKRxCyA1KAPE68n44WC3RIvUO78tk5flagAHYN/i+B8k4W040aqtEvTMjKGag7377eQIzp4GNJ8Hzm1VdFeZQQewAi/hrUOyU3gWoXTpN+xaWr41b4Vugrgb5V9/esDBXb+y2zj8Wc/hX2xc33crfWLkFh7YhmsgmhMEAKE8G2mZEG3Sx3/9BHNsleBTh0oJl5CZm+cBh+BCCGq/ test@127.0.0.1" 431 | } 432 | 433 | describe 'single user' do 434 | before :each do 435 | Etc.stubs(:passwd).yields(singleuser) 436 | Dir.stubs(:[]).with("/home/test/.ssh/*.pub").returns(["id_rsa.pub"]) 437 | File.stubs(:read).with('/home/test/.ssh/id_rsa.pub').returns(keys) 438 | end 439 | 440 | it { Facter.fact(:public_key_comments).value.should eq 'test@local,test@127.0.0.1' } 441 | end 442 | 443 | describe 'multiple user' do 444 | before :each do 445 | Etc.stubs(:passwd).yields(multiuser) 446 | Dir.stubs(:[]).with("/home/test/.ssh/*.pub").returns(["id_rsa.pub"]) 447 | Dir.stubs(:[]).with("/var/tmp/test/.ssh/*.pub").returns(["id_rsa.pub"]) 448 | File.stubs(:read).with('/home/test/.ssh/id_rsa.pub').returns(keys) 449 | File.stubs(:read).with('/var/tmp/test/.ssh/id_rsa.pub').returns(keys) 450 | end 451 | 452 | it { Facter.fact(:public_key_comments).value.should eq 'test@local,test@127.0.0.1' } 453 | end 454 | 455 | end 456 | ``` 457 | 458 | #Types and Provider Testing 459 | 460 | Testing for types and providers is similar to testing facts. We still rely on 461 | rspec, except this time we get a little more sophisticated with our stubbing 462 | and creating of types and providers within the tests for behavior assertion. 463 | 464 | ##Demonstration 465 | 466 | We start with our type test. These are relatively straightforward because they 467 | are generally about testing the validation of the class. We require ‘puppet’ 468 | this time as we’re going to be creating Puppet::Types and testing them. 469 | 470 | ```ruby 471 | require 'spec_helper' 472 | require 'puppet' 473 | 474 | describe Puppet::Type.type(:ssh_tunnel) do 475 | ``` 476 | 477 | Here we create two instances of our type, one for socks and one for a tunnel, 478 | so we can easily test without repeating ourselves. 479 | 480 | ```ruby 481 | let(:socks) { Puppet::Type.type(:ssh_tunnel).new( 482 | :name => 'test', 483 | :local_port => '8080', 484 | :socks => :true, 485 | :remote_server => 'testserver' 486 | )} 487 | let(:tunnel) { Puppet::Type.type(:ssh_tunnel).new( 488 | :name => 'test2', 489 | :forward_server => 'localhost', 490 | :forward_port => '80', 491 | :local_port => '8080', 492 | :socks => :false, 493 | :remote_server => 'testserver' 494 | )} 495 | ``` 496 | 497 | Here we simply check that all of the parameters we set above actually return 498 | the correct information. 499 | 500 | ```ruby 501 | context 'socks' do 502 | it 'should accept a name' do 503 | socks[:name].should eq 'test' 504 | end 505 | it 'should accept a local_port' do 506 | socks[:local_port].should eq '8080' 507 | end 508 | it 'should accept a socks boolean' do 509 | socks[:socks].should eq :true 510 | end 511 | it 'should accept a remote_server' do 512 | socks[:remote_server].should eq 'testserver' 513 | end 514 | end 515 | 516 | context 'tunnel' do 517 | it 'should accept a name' do 518 | tunnel[:name].should eq 'test2' 519 | end 520 | it 'should accept a local_port' do 521 | tunnel[:local_port].should eq '8080' 522 | end 523 | it 'should accept a socks boolean' do 524 | tunnel[:socks].should eq :false 525 | end 526 | it 'should accept a remote_server' do 527 | tunnel[:remote_server].should eq 'testserver' 528 | end 529 | it 'should accept a forward_server' do 530 | tunnel[:forward_server].should eq 'localhost' 531 | end 532 | it 'should accept a forward_port' do 533 | tunnel[:forward_port].should eq '80' 534 | end 535 | end 536 | ``` 537 | 538 | Now we test our validations. We create various types without parameters. We 539 | rely on rspecs expect{} functionality. It allows us to say “we expect{ a thing 540 | }.to DoSomething”, and in our case we expect it to raise an error, specifically 541 | a Puppet::ResourceError with the string we test for. 542 | 543 | ``` 544 | context 'validation' do 545 | it 'should raise an error without forward_server' do 546 | expect { Puppet::Type.type(:ssh_tunnel).new( 547 | :name => 'test2', 548 | :forward_port => '80', 549 | :local_port => '8080', 550 | :socks => :false, 551 | :remote_server => 'testserver') 552 | }.to raise_error(Puppet::ResourceError, 'Validation of Ssh_tunnel[test2] failed: forward_server is not set') 553 | end 554 | it 'should raise an error without forward_port' do 555 | expect { Puppet::Type.type(:ssh_tunnel).new( 556 | :name => 'test2', 557 | :forward_server => 'localhost', 558 | :local_port => '8080', 559 | :socks => :false, 560 | :remote_server => 'testserver') 561 | }.to raise_error(Puppet::ResourceError, 'Validation of Ssh_tunnel[test2] failed: forward_port is not set') 562 | end 563 | it 'should fail with a non-boolean socks' do 564 | expect { Puppet::Type.type(:ssh_tunnel).new( 565 | :name => 'test2', 566 | :local_port => '8080', 567 | :socks => :no, 568 | :remote_server => 'testserver') 569 | }.to raise_error(Puppet::ResourceError, 'Parameter socks failed on Ssh_tunnel[test2]: Invalid value :no. Valid values are true, false. ') 570 | end 571 | it 'should fail with a local_port out of range' do 572 | expect { Puppet::Type.type(:ssh_tunnel).new( 573 | :name => 'test2', 574 | :local_port => '99999', 575 | :socks => :false, 576 | :remote_server => 'testserver') 577 | }.to raise_error(Puppet::ResourceError, 'Parameter local_port failed on Ssh_tunnel[test2]: Port is not in the range 1-65535') 578 | end 579 | end 580 | 581 | end 582 | ``` 583 | 584 | ##Provider 585 | 586 | Providers are slightly more difficult to test. It can help to create the it{} 587 | blocks up front and then fill in the things that fail until the tests pass, 588 | when retrofitting. We’ll refer to the provider we’re testing by ‘subject’, 589 | which is the “thing between describe and do”, and expect two existing tunnels 590 | running that we’ll stub out later. In our example this might look like. 591 | 592 | ```ruby 593 | it 'tests the tunnels exist' do 594 | instances = subject.class.instances.map { |p| {:name => p.get(:name), :ensure => p.get(:ensure)} } 595 | instances[0].should == {:name => '8080:localhost:80-tunnel@remote', :ensure => :present} 596 | instances[1].should == {:name => '8080-socks@remote', :ensure => :present} 597 | end 598 | ``` 599 | 600 | That’s a pretty confusing looking it block, so I’ll try to describe what we’re 601 | doing. First we call ‘subject.class.instances’ which will get the instances 602 | that we stub out later. We then run .map on the array of instances and pull 603 | out the :name and :ensure status from each one. This is all stuffed into 604 | instances as an array of hashes. 605 | 606 | We then check each hash independently via instances[0] and instances[1] to look 607 | for the name and ensure status we expect to be present. This, then, would test 608 | that self.instances is able to run and return an appropriate list of instances 609 | given fake data. 610 | 611 | Working from the top of the file again, we fill in two type and provider 612 | instances as well as stub out the returns from ssh_processes: 613 | 614 | ```ruby 615 | require 'spec_helper' 616 | 617 | describe Puppet::Type.type(:ssh_tunnel).provider(:ssh) do 618 | let(:socks_resource) { 619 | Puppet::Type.type(:ssh_tunnel).new( 620 | :name => '8080-socks@remote', 621 | :socks => :true, 622 | :remote_server => 'socks@remote', 623 | :local_port => '8080', 624 | :provider => :ssh 625 | ) 626 | } 627 | let(:tunnel_resource) { 628 | Puppet::Type.type(:ssh_tunnel).new( 629 | :name => '8080:localhost:80-tunnel@remote', 630 | :socks => :false, 631 | :forward_server => 'localhost', 632 | :forward_port => '80', 633 | :remote_server => 'tunnel@remote', 634 | :local_port => '8080', 635 | :provider => :ssh 636 | ) 637 | } 638 | let(:socks_provider) { socks_resource.provider } 639 | let(:tunnel_provider) { tunnel_resource.provider } 640 | 641 | describe 'self.instances' do 642 | before :each do 643 | subject.class.expects(:ssh_processes).with('-fN -D').returns({'8080 socks@remote' => '10'}) 644 | subject.class.expects(:ssh_processes).with('-fN -L').returns({'8080:localhost:80 tunnel@remote' => '11'}) 645 | end 646 | 647 | it do 648 | instances = subject.class.instances.map { |p| {:name => p.get(:name), :ensure => p.get(:ensure)} } 649 | instances[0].should == {:name => '8080:localhost:80-tunnel@remote', :ensure => :present} 650 | instances[1].should == {:name => '8080-socks@remote', :ensure => :present} 651 | end 652 | end 653 | ``` 654 | 655 | Now we have a very short prefetch test that just makes sure it runs without 656 | errors: 657 | 658 | ``` 659 | describe 'self.prefetch' do 660 | it 'exists' do 661 | socks_provider.class.instances 662 | socks_provider.class.prefetch({}) 663 | end 664 | end 665 | ``` 666 | 667 | Our next real test is our ‘create’ test. We ‘expect’ the required arguments 668 | that would be sent to ssh() for a successful tunnel to be created, stub out the 669 | exists?() method return of true, and then run create() and ensure we get no 670 | errors. When the .create method is called it automatically checks the expects 671 | we assert above, making sure that it tried to call `ssh` with the appropriate 672 | arguments. It stops them actually being called for real on a machine, and just 673 | checks that they would have been. 674 | 675 | ``` 676 | describe 'create' do 677 | it 'makes a socks proxy' do 678 | socks_provider.expects(:ssh).with('-fN', '-D', '8080', 'socks@remote').returns(true) 679 | socks_provider.expects(:exists?).returns(true) 680 | socks_provider.create.should be_true 681 | end 682 | it 'makes a tunnel' do 683 | tunnel_provider.expects(:ssh).with('-fN', '-L', '8080:localhost:80', 'tunnel@remote').returns(true) 684 | tunnel_provider.expects(:exists?).returns(true) 685 | tunnel_provider.create.should be_true 686 | end 687 | end 688 | ``` 689 | 690 | Our destroy test sets the pid for each provider instance first, then stubs out 691 | the kill we sent to it as succeeding: 692 | 693 | ``` 694 | describe 'destroy' do 695 | it 'destroys socks' do 696 | socks_provider.pid = 10 697 | Process.stubs(:kill).with('SIGTERM', 10).returns(true) 698 | socks_provider.destroy.should be_true 699 | end 700 | it 'destroys tunnel' do 701 | tunnel_provider.pid = 11 702 | Process.stubs(:kill).with('SIGTERM', 11).returns(true) 703 | tunnel_provider.destroy.should be_true 704 | end 705 | end 706 | ``` 707 | 708 | For exists? we simply set the ensure parameter to :present, then call exists?() 709 | and assert that it should return true: 710 | 711 | ``` 712 | describe 'exists?' do 713 | it 'checks if socks proxy exists' do 714 | socks_provider.ensure= :present 715 | socks_provider.exists?.should be_true 716 | end 717 | it 'checks if tunnel proxy exists' do 718 | tunnel_provider.ensure= :present 719 | tunnel_provider.exists?.should be_true 720 | end 721 | end 722 | ``` 723 | 724 | # Beaker 725 | 726 | In the above testing we've focused on unit tests. These are the frontline of 727 | your tests and should cover all the lines of code where possible. However, 728 | there's a second kind of testing available to Puppet modules; acceptance 729 | testing, which runs tests against real virtual machines to ensure your 730 | manifests actually have the effects you're expecting. 731 | 732 | Internally at Puppetlabs we have built a framework for this kind of testing, 733 | which is called [Beaker](https://github.com/puppetlabs/beaker). It's a little 734 | rough and ready around the edges in terms of documentation, because it was only 735 | used internally until the module team started adopting it for modules and 736 | spreading the good word about how powerful this kind of testing can be. 737 | 738 | Beaker is a framework to automatically create and manage virtual machines on 739 | various hypervisors, then apply numerous rspec tests against those virtual 740 | machines, then delete and destroy the machines as required. 741 | 742 | In order to add beaker tests to our SSH module we start with creating the 743 | 'spec_helper_acceptance.rb' file in the specs directory. This is a central 744 | place to put setup information for Beaker, generally things like code to 745 | install Puppet or modules. 746 | 747 | First we add beaker to our Gemfile. 748 | 749 | ```ruby 750 | source 'https://rubygems.org' 751 | gem 'beaker-rspec' 752 | ``` 753 | 754 | If you use [bundler](http://bundler.io/) then you can run `bundler update` to 755 | have it pull in all the gems required for acceptance testing. 756 | 757 | Once this is done we need to create some framework for Beaker. We'll start 758 | by creating a very simple "nodeset", a yaml file that lists out a virtual 759 | machine to test against. We'll put this in spec/acceptance/nodesets/default.yml. 760 | 761 | ``` 762 | HOSTS: 763 | centos-65-x64: 764 | roles: 765 | - master 766 | platform: el-6-x86_64 767 | box : centos-65-x64-vbox436-nocm 768 | box_url : http://puppet-vagrant-boxes.puppetlabs.com/centos-65-x64-virtualbox-nocm.box 769 | hypervisor : vagrant 770 | CONFIG: 771 | type: foss 772 | ``` 773 | 774 | These can be much more complex, including multiple hosts, but we'll keep it 775 | simple. Next we create spec/spec_helper_acceptance.rb. We begin this file by 776 | requiring beaker-rspec itself. The next block checks to see if we've passed in 777 | certain environment options when running beaker (we'll talk more about these 778 | later) and only runs the provisioning code if we didn't set RS_PROVISON=no. 779 | 780 | Assuming we didn't set that, it then checks the nodeset that we pass to Beaker 781 | to determine if it should install Puppet Enterprise or plain old Puppet. 782 | 783 | ```ruby 784 | require 'beaker-rspec' 785 | 786 | unless ENV['RS_PROVISION'] == 'no' 787 | hosts.each do |host| 788 | # Install Puppet 789 | if host.is_pe? 790 | install_pe 791 | else 792 | install_puppet 793 | end 794 | end 795 | end 796 | ``` 797 | 798 | Next is some boilerplate RSpec.configure information. We're creating a 799 | proj_root variable that points to the module we're testing, as well as making 800 | the rspec output a little prettier and easier to read. 801 | 802 | ```ruby 803 | RSpec.configure do |c| 804 | # Project root 805 | proj_root = File.expand_path(File.join(File.dirname(__FILE__), '..')) 806 | 807 | # Readable test descriptions 808 | c.formatter = :documentation 809 | ``` 810 | 811 | This final block says "before you run an actual test, do the following", which 812 | then calls puppet_module_install() to install the current module into 'ssh' on 813 | the virtual machine, as well as runs two shell commands to create an empty 814 | hiera.yaml and install stdlib. Here we see :acceptable_exit_codes for the 815 | first time, one of our primary ways of asserting the exit codes we'll accept 816 | from commands. 817 | 818 | ```ruby 819 | c.before :suite do 820 | puppet_module_install(:source => proj_root, :module_name => 'ssh') 821 | hosts.each do |host| 822 | 823 | shell("/bin/touch #{default['puppetpath']}/hiera.yaml") 824 | shell('puppet module install puppetlabs-stdlib', { :acceptable_exit_codes => [0,1] }) 825 | end 826 | end 827 | end 828 | ``` 829 | 830 | With the basic helper and nodeset created we just need to make a few tests. We'll 831 | put these in a file called spec/acceptance/class_spec.rb. We include the file 832 | we just created, spec_helper_acceptance, and then describe what we're testing. 833 | 834 | First we create `pp`, a variable to hold our manifest. We use ruby's EOS 835 | functionality to make sure we don't have to backslash a bunch of stuff and 836 | can just cut and paste in manifests from elsewhere. 837 | 838 | ```ruby 839 | require 'spec_helper_acceptance' 840 | 841 | describe 'ssh class:' do 842 | it 'should run successfully' do 843 | pp = <<-EOS 844 | class { 'ssh::client': } 845 | class { 'ssh::server': } 846 | EOS 847 | ``` 848 | 849 | Now we do the actual test. apply_manifest() takes a manifest and various 850 | options about what the outcome should be. On our first run we're looking to 851 | "catch any failures" and then on our second run we're looking to "catch any 852 | changes". This allows us to be confident we're not changing the state of the 853 | machine on multiple runs if everything is set correctly the first time. We 854 | have some other choices here, including :expect_failures and :expect_changes 855 | for testing things we expect to fail, or change. 856 | 857 | ```ruby 858 | 859 | # Apply twice to ensure no errors the second time. 860 | apply_manifest(pp, :catch_failures => true) 861 | apply_manifest(pp, :catch_changes => true) 862 | end 863 | ``` 864 | 865 | Next we take advantage of our [ServerSpec](http://serverspec.org/) integration 866 | in order to check the service is running. ServerSpec understands a whole bunch 867 | of distributions and allows you to describe packages or services in an OS 868 | independent way. It'll understand that RHEL needs service x status and that 869 | Windows needs something totally different. It has a number of resources 870 | documented on the website, we'll just use service now. 871 | 872 | ```ruby 873 | 874 | describe service('sshd') do 875 | it { should be_enabled.with_level(3) } 876 | it { should be_running } 877 | end 878 | 879 | ``` 880 | 881 | Alternatively if we need to test something more complex we can rely on shell() 882 | commands. Our module is very simple so we'll reproduce the above test, but 883 | with shell commands. It's important to realize this can easily make your tests 884 | unportable, with the commands only working for a single operating system, which 885 | is why we recommend serverspec. 886 | 887 | ```ruby 888 | describe 'service' do 889 | it 'should be running' do 890 | shell('service sshd status', :acceptable_exit_codes => [0]) do |r| 891 | expect(r.stdout).to match(/openssh-daemon.*is running/) 892 | end 893 | end 894 | end 895 | 896 | end 897 | ``` 898 | 899 | 900 | Then putting all this together we just need to run a single command, if things 901 | worked, to see the output of all these tests. 902 | 903 | ``` 904 | $ rspec spec/acceptance/ 905 | Hypervisor for centos-64-x64 is vagrant 906 | Beaker::Hypervisor, found some vagrant boxes to create 907 | created Vagrantfile for VagrantHost centos-64-x64 908 | [bunch of deleted stuff about bringing up a virtual machine and setup commands] 909 | Finished in 3 minutes 15.2 seconds 910 | 4 examples, 0 failures 911 | ``` 912 | 913 | I hope this has helped explain a little bit of our acceptance testing framework 914 | and made you realize it's pretty easy to integrate into your workflow. Here at 915 | Puppetlabs we mostly test with Vagrant/Virtualbox on our laptops as we develop 916 | and then use Vsphere to test via Jenkins before we're ready to merge a PR or 917 | release the updated module. For real life examples of Beaker you can look at 918 | many of the puppetlabs modules, such as apache, mysql, postgresql, firewall, 919 | for examples of real world testing. 920 | --------------------------------------------------------------------------------