We use many tools here at bitcube, however one has made a bigger impression than any other and we believe it gives us the edge over other companies. That tool is Puppet.

Puppet allows us to set up "recipes" on a central "puppetmaster" machine. All the servers we look after use then use these recipes to ensure they are set up correctly.

For example, one simplified recipe says:

• create /etc/postfix/main.cf using a given template
• if /etc/postfix/main.cf changes, reload postfix

Templates are very powerful, but in this example let's just pretend that the template just sets the default destination domain (the part after the "@" in an email address) depending upon the server's hostname.

We gain several benefits compared with the traditional "do it by hand" approach:

• we will never forget to reload postfix after a change since puppet does this for us
• a change in the server's hostname will automatically cause the postfix configuration to be updated
• any improvements or corrections in the template will be rolled out across all servers
• any unauthorised change to the file will be noticed
• all changes are recorded in a source control system (a CMDB as ITIL would say)
• we have the ability to rebuild the server from scratch in the event of a disaster

There are many provisioning tools out there which allow you to build servers and set them up automatically. However most of them stop there - they do not allow updates to the server throughout its lifetime. Puppet does and that makes a huge difference as otherwise the templates in the provisioning tool will become horrendously outdated.

If a problem is found that can be fixed in puppet (for example the permissions on a file are incorrect), this can be added to the puppet recipe. This not only fixes it on all servers, but more importantly fixes it forever - you will never have to fix that problem "again". This is a very substantial time saving and also a cost saving as you are pro-actively preventing future outages.

One unexpected side benefit is that as the cost of rolling out changes is now far lower and benefits far more machines, small changes that were previously "not worth" making become cost effective.

In summary, Puppet provides:

• far lower TCO (Total Cost of Ownership)
• automated server provisioning and maintenance throughout the lifetime of the server
• ability to rebuild servers in the event of an emergency
• fix a problem on one server, you fix it on all server, forever
• prevention of problems occurring in the first place