Our Favourite Devops Tools - Check Them Now

At Netguru, we work on many applications, and we manage a lot of servers, so here we present a list of tools we use on a daily basis - tools which we like and would recommend using.

Which Devops Tools We Use at Netguru

Ansible

Ansible is an open-source configuration management tool which helps automate configuring and deploying apps to one or multiple servers. Ansible has great documentation and an expanding community. It is secure because it doesn’t deploy vulnerable agents to nodes but rather uses OpenSSH which is already heavily tested. It connects to the machine using the ssh protocol and runs all tasks from the shell. Ansible lets you manage your nodes in an inventory and configure all of them quickly and in parallel, which is great when managing a many-server infrastructure. It is easy to use and has a low learning curve since Ansible allows us to write sets of tasks called playbooks, using an easy and descriptive language based on YAML, which keeps playbooks easy to read for humans - making them pretty much self-explanatory.

A sample playbook that installs curl and writes to a file would look like:

- hosts: localhost
  user: root
  tasks:
    - name: install curl
      apt: pkg=curl state=present
    - name: write to a file
      shell: echo hello > /tmp/foo

It is indeed easy to read, and after a first quick glance, it is clear what this playbook is trying to achieve. It’s worth noting that, since all we are doing is connecting to machines over ssh, we can write scripts in languages such as Ruby or Python, then copy them over to the machine and execute them.

Docker

Docker is an open-source platform for building, shipping and running distributed applications. It gives us a common toolbox to take advantage of the distributed and networked nature of modern applications. It lets us make a specific application into a small, self-contained mini operating system. It is very similar to a virtual machine, but instead of having a completely different machine, it all happens on the same host. There is the docker daemon which handles everything. Then, there are images (a similar concept to a class in Ruby world), and finally, the containers which are the single instances of images and which run a specific application.

Docker allows us to “contain” an application into its own “box”, which can be shipped to absolutely any system with Docker installed. This deals with the common issues associated with dependency hell. Docker allows us to create a box with all the dependencies for an app inside the box without the need to install them on the host machine. This process allows the whole team to have a version of the environment.

Each image is configured using a Dockerfile which is a specific file that represents a set of “steps” that Docker executes to create a working image. At Netguru, we have a concept that every project/application has its own Docker config for each environment, and when it needs other services, such as MySQL or Redis, there is a separate container linked to the main application container, making our containers smaller.

The advantages are simple: we leave application configuration Docker-side, which means that when we are setting up the server, the effort is minimal and we leave app dependencies to the Docker configuration file. It’s easy to scape which is just a matter of running an additional app container. This makes Ruby upgrades super-easy - a matter of changing one line in Docker configuration file.

Sensu

We use the Sensu app to monitor the servers. It’s is a centralised service - there is one server, and all nodes are clients. It works as a “push service” - the server “pushes” requests for the checks to clients and then clients respond with statuses. Sensu allows us to monitor everything we want using simple configurations. It uses the nagios check syntax - 0 for OK, 1 for WARNING and 2 for CRITICAL. Sensu can have multiple notification channels - every client has its own tag, i.e. “web”, and if we have a check that includes the tag “web”, we can send a single status request for all clients with the tag “web” instead of pinging each of them individually. Sensu provides multiple methods of notification - simple email, text message or slack.

There is nice GUI for Sensu - Uchiwa. It’s a really minimalistic dashboard that shows us the checks and basic information about them.

Graphite

Graphite lets us create complex graphs easily using its own syntax instead of - say - writing PHP code when using Ganglia.

Graphite does not gather data but rather displays the data in a form that is easily read by humans. We use it to monitor the CPU usage of machines, memory usage, networking data, disk I/O operations, and processes running on machines - which we can later divide into processes that are running, stopped or zombie; we also look for data such as how many users are connected using ssh to monitor if any ssh connections are idle, and how many requests nginx receives from users.

We can easily add new metrics from our apps to Graphite, and not just the technical ones - we can also deal with business metrics such as how many unique users visit a page, or how many users sign in to the application on a daily basis.

Logmatic

Logs, logs, logs - everyone knows them, but not everyone likes to read them. There is a great tool, Logmatic, that comes to the rescue. All application and server logs are sent to the Logmatic cloud where we can access them in a nice and readable manner - with a lot of metrics and graphs so we can see the cumulated results and alerts.

Everything is aggregated in one place where you can filter the logs based on the hostname, process name or other options we can define. If you have your apps running inside Docker containers, Logmatic provides an image that gets the logs from other containers so later you can also filter logs by containers.

Using Logmatic, you can easily determine what is going on in the application or on the server. You can easily select logs from the Rails app that take longer than - say - 500 milliseconds as well as see the date and the action that was performed along with the timestamp of when it happened.

PagerDuty

A simple service that sends an SMS or makes a phone call to a person when there is something going on in a project or on the server - usually triggered by external services such as Sensu. It’s a good tool to notify people that there is a fire going on and that someone needs to fix it as soon as possible.

Do you know any other useful tools to recommend? Please let me know in the comments below.