The epitome of being on top of your infrastructure is when you know why, when and how the cardinal details of a device or any critical service changes in a manner that attracts attention. Sufficient monitoring of core resources within your organization releases the burden of fear, conjecture and panic and replaces it with a systematic, organized, and informed remediation when anything queer or catastrophic happens. This can be barely achieved without tools in the hands of the stewards and custodians of infrastructure. One such tool that can decorate the day to day support and maintenance of servers and services running within them is Icinga.
From their rich webpage, Icinga is a monitoring system that checks the availability of your network resources, notifies users of outages, and generates performance data for reporting. It has been developed with Scalability and extensibility in mind. Icinga can monitor large, complex environments across multiple locations through Distributed Monitoring. To leverage on this monitoring product, we are going to set up Icinga 2 and Icinga Web 2 on Ubuntu 20.04 (Focal Fossa).
Icinga 2 is the monitoring server and requires Icinga Web 2 on top to have a clear, convenient and comfortable view of what is happening under the hood via your browser. The configuration can be easily managed with either the Icinga Director, config management tools or plain text within the Icinga DSL. We shall begin with getting Icinga 2 up and purring.
Stage 1: Install and Configure Icinga 2 on Ubuntu 20.04
This stage will take up the onus of step-by-step installation of Icinga 2. Empecemos.
Step 1: Update and install essential applications
We shall shall start from a clean and well updated server with latest software.
sudo apt update && sudo apt upgradeInstall essential applications we shall need throughout the installation process.
sudo apt install -y vim apt-transport-https wget gnupgStep 2: Add Icinga Package Repository
We will need to add the Icinga repository to our server package management configuration as shown below.
wget -O - https://packages.icinga.com/icinga.key | sudo apt-key add -
. /etc/os-release; if [ ! -z ${UBUNTU_CODENAME x} ]; then DIST="${UBUNTU_CODENAME}"; else DIST="$(lsb_release -c| awk '{print $2}')"; fi;
echo "deb https://packages.icinga.com/ubuntu icinga-${DIST} main" | sudo tee /etc/apt/sources.list.d/${DIST}-icinga.list
echo "deb-src https://packages.icinga.com/ubuntu icinga-${DIST} main" | sudo tee -a /etc/apt/sources.list.d/${DIST}-icinga.listThen update the server to get packages from the newly added repository.
sudo apt updateStep 3: Install Icinga2 on Ubuntu 20.04
Now that we have official Icinga repository, we can install Icinga 2 by using apt package manager to get the icinga2 package as below.
sudo apt install icinga2 -yStep 4: Setting up Check Plugins
Without plugins Icinga 2 does not know how to check external services. The Monitoring Plugins Project provides an extensive set of plugins which can be used with Icinga 2 to check whether services are working properly. The recommended way of installing these standard plugins is to use Ubuntu’s package manager as shown below.
sudo apt install monitoring-plugins -yAfter that is done, start and enable icinga2
sudo systemctl enable --now icinga2And check its status
$ systemctl status icinga2
● icinga2.service - Icinga host/service/network monitoring system
Loaded: loaded (/lib/systemd/system/icinga2.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/icinga2.service.d
└─limits.conf
Active: active (running) since Wed 2020-07-15 12:33:17 EAT; 3min 53s ago
Main PID: 42856 (icinga2)
Tasks: 8
Memory: 13.3M
CGroup: /system.slice/icinga2.serviceStep 5: Installing MySQL|MariaDB and Configuring DB IDO MySQL
The Database Icinga Data Output (DB IDO) feature for Icinga 2 takes care of exporting all configuration and status information into a database. Let us install MariaDB and the Database Icinga Data Output.
Fortunately, we have a detailed guide already to get MariaDB 10.5 installed. Check out How To Install MariaDB on Ubuntu 20.04 (Focal Fossa)
After that, get DB IDO installed as below.
sudo apt install icinga2-ido-mysql -yYou will see prompts during installation, choose “” in the first.
In the second prompt, choose “” since we already have installed and configured the Database.
Next, create a database and user for Icinga2. You are free to name your database and user differently and ensure you use a safe password.
$ mysql -u root -p
MariaDB [(none)]> CREATE DATABASE icinga;
MariaDB [(none)]> GRANT SELECT, INSERT, UPDATE, DELETE, DROP, CREATE VIEW, INDEX, EXECUTE ON icinga.* TO 'icinga'@'localhost' IDENTIFIED BY 'icinga';
MariaDB [(none)]> FLUSH PRIVILEGES;
MariaDB [(none)]> quitAfter creating the database you can import the Icinga 2 IDO schema using the following command. Enter the root password into the prompt when asked and give it sometime to complete.
$ mysql -u root -p icinga < /usr/share/icinga2-ido-mysql/schema/mysql.sql
Enter password: Step 6: Enable the IDO MySQL module
The package provides a new configuration file that is installed in /etc/icinga2/features-available/ido-mysql.conf. Update the database credentials in this file.
$ sudo vim /etc/icinga2/features-available/ido-mysql.conf
/**
* The db_ido_mysql library implements IDO functionality
* for MySQL.
*/
library "db_ido_mysql"
object IdoMysqlConnection "ido-mysql" {
user = "icinga",
password = "icinga",
host = "localhost",
database = "icinga"
}Next, enable the ido-mysql feature configuration file using icinga2 feature enable as follows:
$ sudo icinga2 feature enable ido-mysql
Enabling feature ido-mysql. Make sure to restart Icinga 2 for these changes to take effectRestart Icinga 2 as adviced.
sudo systemctl restart icinga2Step 7: Setting Up Icinga 2 REST API
Icinga Web 2 and other web interfaces require the REST API to send actions (reschedule check, etc.) and query object details. Being important, we need to activate the api.
Enable the api feature using the icinga2 cli command like this:
sudo icinga2 api setupSample output when the command above is run is shown below
information/cli: Generating new CA.
information/base: Writing private key to '/var/lib/icinga2/ca//ca.key'.
information/base: Writing X509 certificate to '/var/lib/icinga2/ca//ca.crt'.
information/cli: Generating new CSR in '/var/lib/icinga2/certs//ubuntu-20.04-amd64.csr'.
information/base: Writing private key to '/var/lib/icinga2/certs//ubuntu-20.04-amd64.key'.
information/base: Writing certificate signing request to '/var/lib/icinga2/certs//ubuntu-20.04-amd64.csr'.
information/cli: Signing CSR with CA and writing certificate to '/var/lib/icinga2/certs//ubuntu-20.04-amd64.crt'.
information/pki: Writing certificate to file '/var/lib/icinga2/certs//ubuntu-20.04-amd64.crt'.
information/cli: Copying CA certificate to '/var/lib/icinga2/certs//ca.crt'.
information/cli: Adding new ApiUser 'root' in '/etc/icinga2/conf.d/api-users.conf'.
information/cli: Reading '/etc/icinga2/icinga2.conf'.
information/cli: Enabling the 'api' feature.
Enabling feature api. Make sure to restart Icinga 2 for these changes to take effect.
information/cli: Updating 'NodeName' constant in '/etc/icinga2/constants.conf'.
information/cli: Created backup file '/etc/icinga2/constants.conf.orig'.
information/cli: Updating 'ZoneName' constant in '/etc/icinga2/constants.conf'.
information/cli: Backup file '/etc/icinga2/constants.conf.orig' already exists. Skipping backup.The above command will not only enable the api feature but will set up certificates and a new API user root with an auto-generated password in the /etc/icinga2/conf.d/api-users.conf config file.
Edit the /etc/icinga2/conf.d/api-users.conf file and add a new ApiUser object below the auto-generated one and specify the permissions attribute with minimal permissions required by Icinga Web 2.
$ sudo vim /etc/icinga2/conf.d/api-users.conf
object ApiUser "icingaweb2" {
password = "Wijsn8Z9eRs5E25d"
permissions = [ "status/query", "actions/*", "objects/modify/*", "objects/query/*" ]
}Note that this API will be listening on port 5665 by default.
A screenshot of the whole configuration file is shared below for clarity
Restart Icinga 2 to activate the new configuration made.
sudo systemctl restart icinga2Stage 2: Setting up Icinga Web 2
Icinga 2 can be used with Icinga Web 2 and a variety of modules to take the mnitoring experience notches higher. This part illustrates how to set up Icinga Web 2.
Step 1: Add Package Repository
The preferred way of installing Icinga Web 2 is to use the official package repository. Note that if you are installing Icinga Web 2 on the same server as Icinga 2, there is no need of adding the repositories once again. On the other hand, if it is your intention to install Icinga Web 2 on a separate Ubuntu 20.04 server, then you will have to do Step 1 and Step 2 in Setting up Icinga 2 stage 1 above.
Step 2: Install Icinga Web 2
We shall use apt package manager to retrieve and install the icingaweb2 package. Below is the command we will run to accomplish this. You will note additional packages icingacli and libapache2-mod-php. libapache2-mod-php is cool for Ubuntu since it makes Icinga Web 2 working out-of-the-box without touching PHP FPM. Additionally, your web server should be up and running after the installation of Icinga Web 2. This is only in Ubuntu|Debian installation.
sudo apt install icingaweb2 libapache2-mod-php icingacli -yStep 3: Prepare Web Setup
To prepare for web setup, we will have to manually create a database and a database user. If you are on a separate server, you will have to install MariaDB|MySQL for this part. Kindly use How To Install MariaDB on Ubuntu 20.04 (Focal Fossa) to set it up quickly. If you are on the same server, skip the link and proceed to add the database user.
$ mysql -u root -p
MariaDB [mysql]> CREATE DATABASE icingaweb2;
MariaDB [mysql]> GRANT ALL ON icingaweb2.* TO [email protected] IDENTIFIED BY 'StrongPassword';
MariaDB [(none)]> FLUSH PRIVILEGES;
MariaDB [(none)]> quitAfter you are done we shall continue to set up Icinga Web 2 quickly and easily with the Icinga Web 2 setup wizard which is available the first time you visit Icinga Web 2 in your browser. When using the web setup you are required to authenticate using a token. In order to generate a token we shall use the icingacli tool available for us as shown below.
sudo icingacli setup token create
The newly generated setup token is: 47cd8e45837e8db0You can later display the token using the icingacli:
sudo icingacli setup token showIf you have a firewall, allow port 80 or 443 (if you are using https)
sudo ufw allow httpStep 4: Starting Web Setup
Finally visit Icinga Web 2 in your browser to access the setup wizard and complete the installation: http://[domain-name-or-ip-address]/icingaweb2/setup.
That will launch the web interface where we will complete the Icinga Web 2 setup. In the first page illustrated below, you will be asked to enter the token we created in the step just before this. Copy your token and paste it in then click “Next”
Modules
Page two prompts you to activate the other modules that the Icinga team put together. If you will find them helpful, simply activate them then click “Next” for the next step.
PHP Modules
The page that follows shows you the PHP modules that are available and the ones that are missing. Click “Next”.
Configure Authentication Type
What comes next is choosing the Authentication type you will use. If you have an LDAP implementation or would like to use Database, choose the one that will suit your needs here from the drop-down list. Hit “Next” once done.
Configure Database|LDAP
Depending on what you selected in the previous step, you will be presented with a form to fill the details Icinga Web 2 needs to authenticate users. This guide will be using Database. Enter the DB details we keyed in Step 3 Stage 2. After you are done, you can click “Validate Configuration” to cross-check whether the details you entered are correct. Click on “Next” when done.
Message displayed once validated is as illustrated below
Authentication Backend
The next page will ask you to give a name to your backend.
Add an administrative user
Next, we shall add an Administrative user that we will use to log into Icinga web 2 once the setup is done, It is through this account that we shall be able to create others as well.
Application Configuration
Here, adjust all application and logging related configuration options to fit your needs then hit “Next“
Review of configurations done
At this stage, we have configured Icinga Web 2 successfully. We only need to confirm our settings before setting it up. If all is nice and good, hit “Next“.
Configuring monitoring module
The monitoring module offers various status and reporting views with powerful filter capabilities that allow you to keep track of the most important events in your monitoring environment. Click “Next” to start configuring the monitoring module.
Configure Monitoring Backend
This will guide Icinga 2 on how it should retrieve monitoring information.
Monitoring IDO Resource
Fill out the connection details below to access the IDO database we configured in Step 5, Stage 1. Validate the settings and then click “Next“.
After proper validation, you should see a success message like so. Click “Next” to move forward.
Command Transport
This is where the API we setup in Stage 1, Step 7 comes to good use. Enter the details you set in your environment in the form. For a reminder, the configs for this guide were as shown below:
object ApiUser "icingaweb2" {
password = "Wijsn8Z9eRs5E25d"
permissions = [ "status/query", "actions/*", "objects/modify/*", "objects/query/*" ]
}Once done keying in the credentials, click on “Validate” to check if everything was set correctly.
Monitoring Security
To protect your monitoring environment against prying eyes please fill out the settings below.
Final Monitoring module Review
You can review the changes supposed to be made before finishing the setup in this step. You are advised to feel free to navigate back to make any corrections. Otherwise, if everything looks pretty, click on “Finish” below and you should get the message that “Icinga Web 2 has been successfully set up“.
Click on “Login to Icinga Web 2” to begin your monitoring journey with Icinga. The next screenshots show the remaining parts of the road.
That will land you into the arms of the “Login page”. Enter the administrative user we created earlier and we should be led in via the main door.
We have finally come to an end of the Icinga monitoring guide. It has been a long while but we are there finally. Ours is to thank you for visiting, for supporting, and for staying till the end.
Must read:
Install Sensu Monitoring Tool on Ubuntu
