Configuration Management Tools | Puppet DevOps

Most people know the job of system administrators, which is to deploy, configure, and maintain the server machines. It’s quite challenging as the security of the whole organization is on stake, yet at the same time, it is monotonous as countless tasks are repeatedly done on a regular basis. Since those tasks are repetitive, (shell) scripting is the best bet to execute something that has to be implemented on all the computer systems that require any kind of updates or a change. But the main drawbacks are:

Drawbacks of shell scripting

1. There is no proper documentation, publishing or announcement of the scripts that are customized to solve or automate a task.

2. The scripts are modified randomly by other system administrator(s) about which they do not communicate. It is like someone goes to work and discovers that something is not working because the script originally created by him/her has been altered by somebody else which is not easily detectable.

3. The scripts do not serve the purpose if large infrastructures are there with different platforms to deploy, configure, and maintain.

There are countless configuration management tools available but most of them are costly due to intense demand. Moreover, even if someone manages to buy something useful they have to wait for the next release of the update with additional add-ons. If the source code is made available then the code can be modified by the developers themselves.

What is Puppet?

Puppet is one of the most popular configuration management tools (Infrastructure As Code) which is often used to achieve the operations side of the DevOps approach. Founded by Luke Kanies in 2005, it is a very powerful tool to deploy, configure, manage, and maintain a server machine, thereby reducing the manual labour and making everyone’s life easier. Since servers are always expected to keep running, Puppet ensures that despite the condition, the data will never be harmed or lost in any manner and the users will always get the kind of information they want. Puppet is there to be used throughout the life of a server.

Most commonly it is seen to be run on Unix-like systems as well as Windows, but it can also run on several other platforms such as Cisco switches, IBM mainframes, and Mac OS servers.

Through Puppet, the programmers can write codes, test, and launch the created applications without waiting for operations (Ops) staff to deliver the required resources.

Puppet has open source versions as well as commercial versions. It has its own language, eponymous Puppet, which is accessible to sysadmins. It is not like other programming languages like PHP, nor it is like another shell language like Windows’ PowerShell or Unix’s and Linux’s Bash shells. It is more like a declarative model-based approach which enables defining Infrastructure As Code and enforcing system configuration with programs.

Resources, classes, and nodes

The main basis in Puppet language is to declare resources. Therefore, flexibility and convenience are dependent on the way resources are declared. The groups of resources can be classified into classes that are larger units of configuration. A resource can be described as a single file or a package, whereas a class can be illustrated as an entire service or application that also includes many packages, service daemons, config files, and maintenance tasks. The smaller classes can be made larger that involves the whole custom system roles, such as “database server” or “web application worker”.

Nodes that have a different set of roles and functions have different classes in general. Node classification is the term given to the process when the task of configuring, which classes are applied to a given node, is executed. Nodes can be grouped using node definitions.

Ordering

Although Puppet language is usually based on resource declaration, there are times when several parts of the language depend on the evaluation order. For instance, variables that need to be set before being referenced.

Files

The files in Puppet language are known as manifests that have .pp file extension. The manifests must use UTF8 encoding. These can also use the line breaks in Windows (CRLF) and Unix (LF), whose format affects the literal line breaks in strings.

Puppet compiles the main manifest first which may contain a single directory comprising of several files OR just a file. Any class which is being declared in the main manifest can be automatically loaded from the manifest files in modules. Puppet also autoloads any classes that are declared by an optional external node classifier. The simplest Puppet deployment is that of a lone main manifest file with just a few numbers of resources. The complexity can progress by grouping the resources into modules and categorizing the nodes more finely.

Compilation and catalogs

The manifests can use conditional statements in order to make things done, such as nodes’ configurations all at once. But before configuring, the manifests are compiled into a catalog that is valid for a single node and that has no ambiguous logic.

The Catalogs are the static documents that contain resources and their relationships. During different stages of Puppet run, a catalog will be there in memory as an object (in Ruby), transferred as JSON, and upheld to disk as YAML. The catalog form being used by this Puppet version is not officially documented, and hence, does not have specifications.

In the standard master-agent architecture, the nodes request Catalogs from Puppet master server, which compiles and serves them to nodes as and whenever required. While running Puppet standalone with Puppet apply, the Catalogs are compiled locally and applied right away. The agent nodes cache the most recent Catalogs. If in case a Catalog is requested and the master fails to compile then the nodes reuse the cached ones. This behavior is handled by usecacheonfailure setting in puppet.conf. While testing the updated manifests, one can save time by turning it off.

In Puppet, the programming syntax is basically decoupled from applications’ syntax and underlying operating system’s syntax. It helps defining high-level concepts into the OS and software setup instructions. It encourages its users to control the complexity by writing code in such a way that it is always reusable, easy to configure, and quite easy to refactor if need be. This is mainly accomplished by Puppet roles and profiles. This separates the main code into three sub-levels:

1. Component modules: Normal modules that handle one particular technology such as Apache or Puppetlabs.
2. Profiles: Multiple component modules being used by the wrapper classes to configure a layered technology stack.
3. Roles: Wrapper classes that use multi-profiles in order to make a complete system configuration.

These divisions of the hierarchical data make the system configurations and refactoring way easier that could not be possible if not done this way.

Without roles and profiles, it is typically done by building system configurations in their node classifier or main manifest. The complex inheritance issues are managed by Hiera. The most standard approach is to form a group of similar nodes and allocate classes to it; then form child groups with extra classes belonging to nodes that have additional requirements. One may also put everything in Hiera using a massive hierarchy reflecting each variation in the infrastructure. This evidently gets extremely difficult to understand and maintain with time.

One doesn’t have to begin from scratch with Puppet. There are many available ready-to-run modules that can run as it is or be used as blueprints.

Puppet Forge is a community repository consisting of modules where one could find nearly 5000 ready-to-run modules that can be downloaded. There are some that have been downloaded millions of times so one need not worry about their working. Some modules are not directly supported by the Puppet company, yet approved by it.

As said earlier, Puppet has Open Source as well as Enterprise versions. The open source provides the basic Puppet, yet it doesn’t come up with an easy web user interface. It does not support enterprise-only OS such as IBM AIX or F5 Big-IP hardware, either. Though it is easy to install open source one for trials.

The additional features of the Enterprise version are:

1. In Role-based Access Control, one needs to assign tasks to designated admins and groups. It unifies with directory services.
2. The Puppet Server Reporting involves collecting metrics of Puppet server’s overall health and performance. It consists of execution timings, active requests, request duration, and compilation load.
3. The Event Inspection inspects about classes, node, and resources’ changes by using Puppet logs, and generates reports based on that. It gives an easy view of the overall working of the Puppet-managed servers.

If one wants much more than the open source version they should go for the Enterprise one for obvious reasons by doing proper research on what’s best based on their requirements. A trial version can be examined by using a virtual machine supported by the Puppet Enterprise.

Puppet enables continuous software delivery/releases by automating operations and deployment workflow.

It has wide industrial support at present, and so, it can work on almost every platform across the Cloud and DevOps.

The Sys Admin tasks are saved as manifests and modules. These can be reused in order to create preconfigured servers swiftly.

If you are dealing with slow servers, then it is better you resolve this first before moving on to anything further. Apachebooster plugin rescues those who want significant server boost. This cPanel plugin is a combination of Nginx and Varnish so that the issue remains resolved.