devopsmuc.slide

Puppet Testing for fun and profit
DevOpsMUC 2015, July 27, 2015
Tags: devopsmuc, puppet, testing

Sergio Jimenez
@tripledes
Teradata


* About me

Recovering sysadmin:

.image images/purple_minion.png 120 160

- Used to work in traditional sysadmin envs.
- 2012, started to be interested in development.
- 2013, met Puppet, Ruby and Unit testing.
- 2014, met Acceptance testing and Golang.
- 2015, granted to switch to development (still doing Ops ;-) ).


* Agenda

- Unit Testing with RSpec Puppet
- Acceptance Testing with RSpec Beaker


* Why testing?

- Developers have been doing it for ages...
- Puppet is code, isn't it?
- Early detection of errors/bugs.
- Integration with CI systems.

.image images/Lazy-dog-framed.jpg
.caption _Watching_the_CI_system_


* Introducing Unit Testing

Unit testing, also known as component testing, refers to tests that verify the functionality of a specific section of code, usually at the function level. In an object-oriented environment, this is usually at the class level, and the minimal unit tests include the constructors and destructors. [[https://en.wikipedia.org/wiki/Software_testing#Unit_testing][Wikipedia]]

How does that translate to Puppet?

- Whatever the catalog contains.
 - Resources.
 - Classes.
 - Defined types.
 ...

* Introducing RSpec Puppet

- What is RSpec?
 Behavior-driven development framework for Ruby.

- What is RSpec-Puppet?
 RSpec extension for Puppet testing.

- Do I need to know RSpec to write Puppet tests?
 It's not mandatory, but helps.

* RSpec example

- Code:
.code examples/rspec/lib/silly.rb

- Test:
.code examples/rspec/spec/classes/silly_spec.rb

* RSpec-puppet example

- Code:
.code examples/rspec-puppet/modules/silly/manifests/init.pp

- Test:
.code examples/rspec-puppet/modules/silly/spec/classes/init_spec.rb

* RSpec-puppet example (cont.)

- Create module directory hierarchy.
  $ mkdir -p silly/{manifests,files,templates}

- Inside module directory.
  $ rspec-puppet-init --name silly
   + spec/
   + spec/classes/
   + spec/defines/
   + spec/functions/
   + spec/hosts/
   + spec/fixtures/
   + spec/fixtures/manifests/
   + spec/fixtures/modules/
   + spec/fixtures/modules/silly/
   + spec/fixtures/manifests/site.pp
   + spec/fixtures/modules/silly/manifests
   + spec/spec_helper.rb
   + Rakefile

* Introducing PuppetLabs spec helper

*puppetlabs_spec_helper* includes lots of perks:

  - helpers
  - configuration settings for puppet
  - rake tasks...

- Replace *spec_helper.rb* contents with:
  require 'puppetlabs_spec_helper/module_spec_helper'

- Replace *Rakefile* contents with:
  require 'puppetlabs_spec_helper/rake_tasks'

* Introducing PuppetLabs spec helper (cont.)

  rake beaker            # Run beaker acceptance tests
  rake beaker_nodes      # List available beaker nodesets
  rake build             # Build puppet module package
  rake clean             # Clean a built module package
  rake coverage          # Generate code coverage information
  rake help              # Display the list of available rake tasks
  rake lint              # Run puppet-lint
  rake metadata          # Validate metadata.json file
  rake spec              # Run spec tests in a clean fixtures directory
  rake spec_clean        # Clean up the fixtures directory
  rake spec_prep         # Create the fixtures directory
  rake spec_standalone   # Run spec tests on an existing fixtures directory
  rake syntax            # Syntax check Puppet manifests and templates
  rake syntax:hiera      # Syntax check Hiera config files
  rake syntax:manifests  # Syntax check Puppet manifests
  rake syntax:templates  # Syntax check Puppet templates
  rake validate          # Check syntax of Ruby files and call :syntax and :metadata

* Fixtures example

- _.fixtures.yml_

.code examples/fixtures/fixtures.yml


* Let's do a quick live demo

.image images/homer_epic_fail.jpg 450 600

* Other useful matchers

- Catalog:
  compile[.with_all_deps()]
- Resources:
  contain_*(), have_resource_count(), have_class_count(), have_*_resource_count()
- Resource dependencies:
  .that_requires(), .that_comes_before(), .that_notifies() and .that_subscribes_to()
- Errors:
  expect{}.to raise_error()
- Functions:
  run.with_params().(and_return() or and_raise_error())

* Other useful matchers (examples)

  it { should compile.with_all_deps }
  it { should have_resource_count(2) }
  it { should have_class_count(1) }
  it { should have_logrotate__rule_resource_count(1) }
  it { should contain_file('foo').that_requires('File[bar]') }
  it { should contain_file('foo').that_comes_before('File[bar]') }
  it { should contain_file('foo').that_notifies('File[bar]') }
  it { should contain_file('foo').that_subscribes_to('File[bar]') }
  it do
    expect {
      should
    }.to raise_error(Puppet::Error, /foo must be a boolean/)
  end
  it { should run.with_params('foo').and_return('bar') }
  it { should run.with_params().and_raise_error(ArgumentError) }

* Helpful tools

.image images/rvm.png 100 100
.caption [[http://rvm.io][RVM]]
.image images/bundler.png 100 100
.caption [[http://bundler.io/][Bundler]]
.image images/guard.png 100 100
.caption [[https://github.com/guard/guard-rspec][Guard::RSpec]]

* Say hello to Beaker!

.image images/beaker_introduction.gif 367 502

* What is Beaker

- PuppetLabs' cloud acceptance test framework.
- Provision and configure hosts for Puppet PE or other Puppet projects.
- Tests are just Ruby files with Beaker DSL.

* What is Beaker-RSpec

- Bridge between Beaker and RSpec.
- Provisions and configures hosts.
- Uses Vagrant and ServerSpec under the hood.
- Tests are written in RSpec format using ServerSpec matchers and Beaker DSL.

* What is ServerSpec

- RSpec tests run on a live machine.
- Actual state is checked via SSH, WinRM, docker API and more.
- Does OS abstraction.
- It's not tight to Puppet, Chef, Ansible...but meant to be used with them.
- Resource types:
  - Networking: bond, bridge, default gw, interface...
  - OS: package, service, cgroup, cron, file, user, group...
  - Firewall: iptables, ipfilter, ipnat...
  - Security: SELinux, x509 certs...
  - Containers: docker and lxc

* Preparing all the bits for Beaker-RSpec
.code examples/rspec-puppet/modules/silly2/spec/spec_helper_acceptance.rb
.caption spec/spec_helper_acceptance.rb

* Preparing all the bits for Beaker-RSpec (cont.)
.code examples/rspec-puppet/modules/silly2/spec/acceptance/nodesets/default.yml
.caption spec/acceptance/nodesets/default.yml

* Writing Beaker-RSpec tests
.code examples/rspec-puppet/modules/silly2/spec/acceptance/silly2_spec.rb /START BLOCK1/,/END BLOCK1/
.caption spec/acceptance/silly2_spec.rb

* Writing Beaker-RSpec tests (cont.)
.code examples/rspec-puppet/modules/silly2/spec/acceptance/silly2_spec.rb /START BLOCK2/,/END BLOCK2/
.caption spec/acceptance/silly2_spec.rb

* Controlling Beaker-RSpec through environment
- *BEAKER_debug*: turn on extended debug logging

- *BEAKER_set*: set to the name of the node file to be used during testing (exclude .yml file extension, it will be added by beaker-rspec), assumed to be in module's spec/acceptance/nodesets directory

- *BEAKER_setfile*: set to the full path to a node file be used during testing (be sure to include full path and file extensions, beaker-rspec will use this path without editing/altering it in any way)

- *BEAKER_destroy*: - set to no to preserve test boxes after testing, set to onpass to destroy only if tests pass

- *BEAKER_provision*: set to no to skip provisioning boxes before testing, will then assume that boxes are already provisioned and reachable

* Let's run a complete live demo!
.image images/Beaker_experiment.gif 324 380

* Next steps?
- Test other hypervisors.
- Integration with CI systems.
- System tests (i.e. [[http://robotframework.org/][Robot Framework]])
- I'm all ears! :-)

* Useful Links
.link http://rspec.info RSpec
.link http://rspec-puppet.com RSpec-Puppet
.link https://github.com/puppetlabs/puppetlabs_spec_helper PuppetLabs Spec Helper
.link https://github.com/puppetlabs/beaker Beaker
.link http://serverspec.org ServerSpec
.link https://github.com/puppetlabs/beaker-rspec Beaker-RSpec


* Questions?
.image images/fry_questions.jpg