Reuse your tdd skills to build an application cluster with vagrant and chef: iteration 3

This is the last of the series (iteration 1, iteration 2) ‘Reuse your tdd skills to build an application cluster with vagrant and chef’.

I created, on my repository, a fully functionnal example that will serve as a support to this post. Let’s explore the cookbook:

It is divided into 4 recipes:

  • a data recipe that configures our data storage: creates a vm, installs mysql, creates a user and a schema
  • a search recipe: creates a vm, installs java and an elasticsearch instance
  • an appserver recipe: creates 2 vm, installs java and jetty
  • a proxy recipe: creates a vm, installs haproxy and configures the members

To reach our goal we have to understand Chef layout.

Understanding the layout
A cookbook follows a precise directory layout. In the last post we created that cookbook manually. The knife tool make this process less tedious.
First install knife

$ gem install knife-solo
$ knife cookbook create limber -C 'Louis Gueye' -I apachev2 -m '' -o site-cookbooks -r md
WARNING: No knife configuration file found
** Creating cookbook limber
** Creating README for cookbook: limber
** Creating CHANGELOG for cookbook: limber
** Creating metadata for cookbook: limber
$ tree site-cookbooks/limber
├── attributes
├── definitions
├── files
│   └── default
├── libraries
├── metadata.rb
├── providers
├── recipes
│   └── default.rb
├── resources
└── templates
    └── default

10 directories, 4 files

Knife just created a classic cookbook layout. You may not need everything but it saves time. For example creating a resource is clearly an advanced feature. You won’t need it before a long time. So don’t forget to delete unused dirs at the end because they mislead the cookbook users. Don’t try to understand everything upfront. Just follow the methodology: test first and you’ll soon enough be confronted to the concepts you need.

Recipes name drive most naming: let’s say you have a data recipe, then you’ll endup with the following related files:


Where do I write my tests?
To write tests for a recipe named data, we have to create create site-cookbooks/limber/files/default/data_test.rb.

require 'minitest/spec'

describe_recipe 'limber::data' do

  include MiniTest::Chef::Assertions
  include MiniTest::Chef::Resources

  MiniTest::Chef::Resources.register_resource :mysql_database, :connection

  it "selecting on database 'limber' with user 'limber' should succeed" do
    resource ='limber')
    resource.connection({:host => 'localhost',
                         :username => 'limber',
                         :password => '*mysql-limber@0')
    provider =, nil).tap(&:load_current_resource)
    row = provider.send(:db).query('select 1 from dual where 1 = 1')
    assert row


Implementation your test in your recipes
The implementation of the above test would be:

include_recipe 'limber::default'
include_recipe 'database::mysql'

mysql_connection = {:host => 'localhost',
                    :username => 'root',
                    :password => '*mysql-root@0'}

mysql_database 'limber' do
  connection mysql_connection
  action :create

mysql_database_user 'limber' do
  connection mysql_connection
  password '*mysql-limber@0'
  database_name 'limber'
  host 'localhost'
  privileges [:select, :update, :insert, :delete]
  action [:create, :grant]
mysql_database_user node['app']['db']['user'] do
  connection mysql_connection
  password '*mysql-limber@0'
  database_name 'limber'
  host '%'
  privileges [:select, :update, :insert, :delete]
  action [:grant]

include_recipe 'minitest-handler'

Refactor in attributes
The above implementation works but is not resistant to change. The obvious first refactor is to extract user attributes, database attributes, … and reuse them in both test and implementation. The attributes files were created for that exact purpose.
Below, the limber/attributes/data.rb file:

include_attribute "limber::default"
include_attribute "mysql::server"

node.set['mysql']['server_root_password'] = '*mysql-root@0'
node.set['mysql']['server_debian_password'] = '*mysql-root@0'
node.set['mysql']['server_repl_password'] = '*mysql-root@0'

node.set['app']['db']['schema'] = node['app']['name']
node.set['app']['db']['user'] = node['app']['name']
node.set['app']['db']['password'] = '*mysql-limber@0'

Use templates if needed
When needed, we can take advantage of ruby templating mechanism to replace default values with node values. This is a really powerful customization tool.
Say you want to configure elasticsearch: create a template under templates/default/elasticsearch.yml.erb <%= node['elasticsearch'][:cluster][:name] %>
bootstrap.mlockall: <%= node['elasticsearch'][:bootstrap][:mlockall] %> <%= node.set['elasticsearch'][:discovery][:zen][:ping][:multicast][:enabled] %>

Then interpolate in your recipe (recipes/search.rb) withe the template resource:

template '/etc/elasticsearch/elasticsearch.yml' do
  source 'elasticsearch.yml.erb'
  mode '0644'
  owner 'root'
  group 'root'

include_recipe 'minitest-handler'

Elascticsearch default values will be replaced
Reuse recipes
A good practice is to favor recipes reuse. For instance everyone knows java recipe is not trivial because the ecosystem is at war and the required licence agreement has made the automation process a bit tricky. So don’t try to create your own java recipe, use the provided one and contribute if it doesn’t exactly fit your needs.
Another good practice is to gather all common behaviour in the default recipe and reuse it in the other ones: in our default recipe we ask Chef to install basic packages like vim, curl, tree and htop.

include_recipe 'limber::default'
include_recipe 'java'

Use foodcritic
Foodcritic is a lint tool that reveals weaknesses and syntactic code smells. It should help you write cleaner cookbooks. I did not find a way to automatically run it from vagrant and I think it’s not the best approach. A better one would be to include vagrant provisioning in a more generic build and after vagrant completes, launch foodcritic. I guess it would be easily done with Rake.

$ sudo gem install foodcritic
$ echo "gem 'foodcritic', '>= 2.2.0'" >> Gemfile
$ bundle install
$ bundle exec foodcritic site-cookbooks/limber
FC007: Ensure recipe dependencies are reflected in cookbook metadata: site-cookbooks/limber/recipes/appserver.rb:47
FC007: Ensure recipe dependencies are reflected in cookbook metadata: site-cookbooks/limber/recipes/data.rb:38
FC007: Ensure recipe dependencies are reflected in cookbook metadata: site-cookbooks/limber/recipes/default.rb:1
FC007: Ensure recipe dependencies are reflected in cookbook metadata: site-cookbooks/limber/recipes/default.rb:2
FC007: Ensure recipe dependencies are reflected in cookbook metadata: site-cookbooks/limber/recipes/proxy.rb:3
FC007: Ensure recipe dependencies are reflected in cookbook metadata: site-cookbooks/limber/recipes/search.rb:24

Correct those warnings to make foodcritic happy
This completes our attempt to learn Chef using our tdd skills. We saw that the tool is not trivial to understand but once apprehended it’s flexible enough to build powerful recipes. The testing framework is really a plus.

We used vagrant to create fresh virtual machine with virtualbox provider and used chef-solo to provision them. This is one usage but it barely scratches the surface of configuration management and there is far more to explore:

  • use chef in server mode: our example seems quite complex just to create and provision empty machines. the server mode  added value lies in it ability to update/upgrade those machines configuration at commit.
  • use puppet instead of chef
  • use the very promising saltstack instead of chef
  • use linux containers: docker or raw lxc might be a better alternative than fat containers like virtualbox or vmware as they are really faster. vagrant-lxc and vagrant-docker are on early development stage
  • use Iaas: AWS or Digital Ocean are remote providers that also prevent you from worrying about the network configuration but you have to learn about the solution and its limitations (communication beetween machines in the same cluster, shared file system, users, etc.)

I really enjoyed (and still do) what I’ve learned so far because agility expands to operations and that is excellent news for end products because we are (some more thant us) on the right way to deliver high quality softwares with nearly zero downtime deployments. Infrastructure as code really takes it to the next level!

J Timberman blog: full of interesting material

Seth Vargo’s blog: test oriented

Shawn Dahlen: qite complete

Robert fox: useful tips

Bryan Berry: very inspiring

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s