Building your servers manually is so last year. These days, the cool kids are automating their infrastructure. Chef has been popularized by big companies such as Facebook and Airbnb, but you don’t need to be a Fortune 500 company to benefit from it. In fact, you can easily start with just a few servers and use the free version of Chef!
Bridging the gap
This article is intended as an introduction to Chef automation for small-to-medium-sized enterprises (SMEs) and Startups. It’s the result of my experiences using the framework for several years on various Symfony/PHP projects.
The gap between operations and developers is decreasing as platforms like Chef enable developers to be more involved in aspects that were traditionally ‘ops’ only. So, even if you don’t plan on being fully involved in DevOps, I believe that having a better understanding of a project’s infrastructure will help you become a better coder. It will improve the overall development process by minimizing the back and forth between you and DevOps.
Do I need to learn Ruby to use Chef?
Chef is a Ruby-based framework. But don’t worry if you are not a Ruby developer. You’ll be interacting with small blocks of code – called resources – that are similar to PHP/Symfony helper functions. There are other similarities between the syntax of Ruby and PHP. In fact, the Domain Specific Language (DSL) used by Chef is a subset of Ruby, so most of the time you will be using only Chef DSL.
You’ll barely even notice you’re in a Ruby environment!
Why use Chef?
This is the real question you need to ask yourself before you learn anything new. There are so many new platforms, frameworks, and languages coming out that being selective and methodical in your evaluation is more crucial than ever before. So rather than brush over this, I’ve tried to dive into all the reasons I think you should use Chef.
Consistency between environments
If you only take one thing away from this article, let it be this: Chef gives you consistency between all your environments.
Local = Development = Staging = Production
Are you tired of hearing developers say “It worked fine on my machine”? Me too. There is nothing more frustrating. Chef can save you from that recurring conversation because you will be using consistent environments throughout your project. This way, bugs are easier to reproduce in either local or dev. Because they are found earlier, it’s less likely they will get to production.
Automation
Chef automates the process of building a server, and that automation helps you recover your data faster.
If you use AWS, you’re likely aware that instances (their term for a server) are disposable. From time to time some of them fail and you could get an “Amazon EC2 instance scheduled for retirement” email in your inbox. Or what I call the ‘AWS email of death’. It’s basically a message saying that your instance will be shut down in the future, because of a hardware failure, and it’s up to you to replace it or make it work again. Having a good backup of your data as well as a tool like Chef will help you easily rebuild your server in minutes – rather than hours or days.
Above: The infamous AWS Email of Death
Reusable infrastructure code
Having your infrastructure as code means your environment can have some of the qualities of your application. It can be versionable, testable and reusable. When your infrastructure’s code lives in the same repo as your application’s code it will help everybody on the team have a better understanding of the project as a whole.
How can I integrate Chef into a Symfony repository?
Chef can be easily integrated into your Symfony app directory structure, all you need is a directory that can be called “chef”. For example at the root of your application structure.
Above: Example of a Symfony 4 application with Chef
Basic Chef terminology
Chef comes with its own flavor (sorry) of culinary-inspired terminology. Here is a list of some of the most important terms to keep front-of-mind when working in the framework:
Chef node
A Chef node is simply a server. Chef can manage many different servers, but most of the time they will either be VMs on your computer or cloud servers.
Chef workstation
A workstation is just another term for your computer: the machine you use to manage your cookbooks and interact with the Chef server.
Chef server
A Chef server works as the control center of the Chef ecosystem. It is where your cookbooks are stored, metadata of the nodes, environment details, etc.
Resource
This is a small piece of code (in the Chef DSL) that implements a specific configuration state.
For example: creating a directory with some specific permissions. A resource is declarative, you specify ‘what’ should happen, not ‘how’. If we compare this to PHP, it can be seen as helper function that you call with some parameters. It performs a particular action, so you don’t have to worry about the actual process that drives it.
Recipe
A recipe is a group of resources used to implement a particular configuration. A recipe should be focused on implementing a particular service. For example: Apache, PHP or MySQL.
Cookbook
A cookbook consists of a number of recipes. It provides structure to the Chef code. It contains other elements like attributes, templates, etc.
Run list
A run list is a combined list of certain recipes/roles. It is applied to a given node in a certain order. The order is very important as the run list’s items are executed in the specified order
Role
Roles allow you to apply recipes directly to a node’s run list. For example, you could have a role called “web-server” and that role would include all the different recipes required to build a web server such as “PHP”, “Apache” and similar.
Attributes
Attributes are where we set values or override other default values, for the various parameters used in our cookbook. Attributes are the configuration .yaml files we use in our Symfony apps. For example, in our attributes, we can define default directory paths, versions, IP addresses, ports, etc.
Data bags
A data bag is a JSON-formatted container that includes information to be shared between nodes. For example, system users, passwords, and SSL certificates.
Knife
It is the command line tool provided by Chef to communicate with the Chef server. It is included in the Chef DK.
Berkshelf
In the PHP world, we have Composer. Chef has Berkshelf; a dependency manager that manages the implementation of community cookbooks in our projects.
Chef supermarket
This would be the equivalent of Packagist, and it is where all the community cookbooks are published so we can use them in our any Chef project.
Environments
Environments in Chef are very similar to the application environments you likely use in any programming application framework. Environments in Chef allow you to target configuration data to specific nodes.
Chef solo
Chef solo allows you to provision a machine without connecting it to a Chef Server. This is the preferred way to use Chef with virtualization software (like Vagrant). Some people even use Chef solo to provision their servers. Which could be the ideal option if you have a small infrastructure.
A Brief Introduction to Chef Workflow
Now that we are familiar with some of the terms used in the Chef world, let’s take a look at the basic workflow used to update nodes. This workflow assumes you have a Chef server in your infrastructure.
Setup your Chef Workstation
The Chef Development Kit, or ChefDK, provides you with all the tools required to interact with a Chef server and to start writing Chef code. The ChefDK supports the latest versions of all the most important operating systems: Linux, Mac OS X, and Windows. There is a ChefDK package for each OS, so they are easy to install.
Push changes to the Chef server
To be able to apply any change to your servers, you need to first push those changes to the Chef server. You can use the knife command line tool for that but typically you would use Berkshelf to upload your cookbooks and manage your cookbook dependencies. Berkshelf lets you manage your cookbook dependencies in a similar way to your application dependencies in PHP world. This way, you can define the community cookbooks and versions you will use in your Chef project and it can also be used for versioning your own cookbooks.
If you don’t use Berkshelf to upload your cookbooks to the Chef server, then you will use the knife command line tool. Knife is what you use to communicate with your Chef server for many different tasks. For example; it will help you manage your nodes, data bags, roles, and environments.
Applying changes to your nodes
After any changes are uploaded to your Chef server, the final step is to apply them to your nodes. The good news is that this is done automatically by Chef. This happens because each Chef node is running an agent called ‘Chef client’ that is frequently polling the Chef server for changes. Chef client typically does this every 15 minutes, and if it finds anything to update it will apply the change to the server. This process is called a Chef run. After a successful Chef run the node is updated according to the cookbook specifications.
Cool Things You Can Do With Chef
Continuous Deployment
We recently wrote an article about the benefits of Continuous Integration. If the idea appealed but not the methodology, then Chef may be a great alternative. Especially in development environments for small projects. This is because Chef uses a resource called ‘deploy’, based on Capistrano, which has almost the same callbacks: after_restart, before_migrate, before_restart, and before_symlink. Using these callbacks, you can attach all the different commands used in a Symfony/PHP deployment: clearing cache, install vendors, running tests, migrations and processing assets.
Provision a VM
Often applications depend on many different services, for example, web servers, email servers, databases, cache systems, javascript libraries and others. Configuring those services can take a lot of time and effort. Chef can be used to provision, not only cloud servers but also local environments. So, for example, you could provision a VM with Chef in the same way you would do it in AWS. That means you can have almost the same configuration of your production server on your computer.
Vagrant offers multiple options to provision a VM. Chef provides a tool called Chef Solo and this is perfect for this case because it allows to provision a VM using Chef without requiring a Chef server so you can run the cookbooks directly from your repo and independently of your Chef project infrastructure.
Conclusion
Let’s start by recapping some of the main benefits of using Chef:
- Reduce bugs in your applications by having consistent environments
- Increase the visibility of the project infrastructure to all members of the team
- Automate the process of creating a new environment
- Infrastructure as code makes it versionable, testable and reusable
- If you are using AWS you have to have backups and a tested strategy to rebuild your servers. Using a tool like Chef can save you time whenever your server fails (and you get that AWS email of death).
- Chef allows you to have up to 5 nodes for free which is perfect for small infrastructures.
Chef and Vagrant for your local environments is a great combination that can make you more productive as a developer. For example, if your computer dies one day and you have to buy a new computer, you don’t need to worry about it too much because you can easily rebuild your local environment with Chef without wasting days in this process. Also, this is helpful when a new developer is introduced to the project.
If you are interested in learning more about Chef, you should go first to https://learn.chef.io they have some nice tutorials with webinars, quizzes and cool badges with culinary terms. Chef official documentation is very good and they have done a great job organizing it. Check out their website here: https://docs.chef.io.
Are you searching for your next programming challenge?
Scalable Path is always on the lookout for top-notch talent. Apply today and start working with great clients from around the world!