Install and Configure Graylog Monitoring Server Ubuntu 20.04

Graylog is a free and open-source log management software tool that can be used to monitor log of the network systems from the central server. It uses Elasticsearch to store logs data and provide search capabilities, and MongoDB for storing meta information. It helps you to monitor, search and analyze a large amount of data into a simple readable format.

In this tutorial, we will show you how to install Graylog on Ubuntu 20.04 server.

Prerequisites

• A server running Ubuntu 20.04.with minimum 4GB of RAM
• A root password is configured on.

Getting Started

First, you will need to update your system packages to the latest version. You can update them all with the following command:

apt-get update -y

After updating all the packages, you will also need to install some dependencies in your server. You can install all of them with the following command:

apt-get install apt-transport-https gnupg2 uuid-runtime pwgen curl dirmngr -y

Once all the required dependencies are installed, you can proceed to the next step.

Install Java

Graylog requires Java to be installed in your server. If not installed, you can install it with the following command:

apt-get install openjdk-11-jre-headless -y

Once the Java is installed, you can verify the installed version of Java by running the following command:

java -version

You should get the following output:

openjdk version "11.0.8" 2020-07-14
OpenJDK Runtime Environment (build 11.0.8 10-post-Ubuntu-0ubuntu120.04)
OpenJDK 64-Bit Server VM (build 11.0.8 10-post-Ubuntu-0ubuntu120.04, mixed mode, sharing)

Once you are finished, you can proceed to the next step.

Install and Configure Elasticsearch

Graylog uses Elasticsearch to store logs coming from the external resource. So you will need to install Elasticsearch in your system.

By default, the latest version of Elasticsearch is not available in the Ubuntu default repository. So you will need to add Elasticsearch repository in your system.

First, download and add the Elasticsearch GPG key with the following command:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | apt-key add -

Next, add the Elasticsearch repository with the following command:

echo "deb https://artifacts.elastic.co/packages/oss-6.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-6.x.list

Next, update the repository and install the Elasticsearch with the following command:

apt-get update -y

apt-get install elasticsearch-oss -y

After installing Elasticsearch, you will need to edit the Elasticsearch configuration file and define the cluster name. You can do it with the following command:

nano /etc/elasticsearch/elasticsearch.yml

Define your cluster name to graylog and add the another line as shown below:

cluster.name: graylog
action.auto_create_index: false

Save and close the file when you are finished. Then, start the Elasticsearch service and enable it to start at boot with the following command:

systemctl daemon-reload

systemctl start elasticsearch

systemctl enable elasticsearch

You can also verify the status of the Elasticsearch service with the following command:

systemctl status elasticsearch

You should get the following output:

? elasticsearch.service - Elasticsearch
     Loaded: loaded (/lib/systemd/system/elasticsearch.service; disabled; vendor preset: enabled)
     Active: active (running) since Sat 2020-09-05 08:41:18 UTC; 9s ago
       Docs: http://www.elastic.co
   Main PID: 7085 (java)
      Tasks: 17 (limit: 2353)
     Memory: 1.1G
     CGroup: /system.slice/elasticsearch.service
             ??7085 /bin/java -Xms1g -Xmx1g -XX: UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX: UseCMSInitiatingOccupancyOnly ->

Sep 05 08:41:18 ubuntu2004 systemd[1]: Started Elasticsearch.

Now, verify the Elasticcsearch response with the following command:

curl -X GET http://localhost:9200

You should get the following output:

{
  "name" : "vzg8H4j",
  "cluster_name" : "graylog",
  "cluster_uuid" : "6R9SlXxNSUGe6aclcJa9VQ",
  "version" : {
    "number" : "6.8.12",
    "build_flavor" : "oss",
    "build_type" : "deb",
    "build_hash" : "7a15d2a",
    "build_date" : "2020-08-12T07:27:20.804867Z",
    "build_snapshot" : false,
    "lucene_version" : "7.7.3",
    "minimum_wire_compatibility_version" : "5.6.0",
    "minimum_index_compatibility_version" : "5.0.0"
  },
  "tagline" : "You Know, for Search"
}

Install MongoDB Server

Graylog uses MongoDB as a database. So you will need to install the MongoDB database to your server. You can install it with the following command:

apt-get install mongodb-server -y

Once the MongoDB is installed, start the MongoDB service and enable it to start at system reboot with the following command:

systemctl start mongodb

systemctl enable mongodb

Once you are finished, you can proceed to the next step.

Install and Configure Graylog

By default, Graylog package is not available in the Ubuntu default repository. So you will need to install the graylog repository to your server.Advertisement

You can download the Graylog repository package with the following command:

wget https://packages.graylog2.org/repo/packages/graylog-3.3-repository_latest.deb

Once the download is completed, install the downloaded package with the following command:

dpkg -i graylog-3.3-repository_latest.deb

Next, update the repository and install the Graylog server with the following command:

apt-get update -y

apt-get install graylog-server -y

After installing Graylog server, you will need to generate a secret to secure the user passwords. You can generate it with the following command:

pwgen -N 1 -s 96

You should see the following output:

Wv4VQWCAA9sRbL7pxPeY7tb9lSo50esEWgNXxXHypx0Og3CezMmQLdF2QzQdRSIXmNXKINjRvZpPTrvZv4k4NlJrFYTfOc3c

Next, you will also need to generate a secure password for Graylog admin user. You will need this password to login to the Graylog web interface. You can generate it with the following command:

echo -n password | sha256sum

You should see the following output:

5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8  -

Now, edit the Graylog main configuration file and define both passwords:

nano /etc/graylog/server/server.conf

Paste both password which you have generated above as shown below:

password_secret = Wv4VQWCAA9sRbL7pxPeY7tb9lSo50esEWgNXxXHypx0Og3CezMmQLdF2QzQdRSIXmNXKINjRvZpPTrvZv4k4NlJrFYTfOc3c
root_password_sha2 = 5e884898da28047151d0e56f8dc6292773603d0d6aabbdd62a11ef721d1542d8   

Next, you will also need to define your server a bind address as shown below:Advertisement

http_bind_address = 127.0.0.1:9000

Save and close the file when you are finished then start the Graylog service and enable it to start at system reboot with the following command:

systemctl daemon-reload

systemctl start graylog-server

systemctl enable graylog-server

Next, you can verify the status of the Graylog server using the following command:

systemctl status graylog-server

You should see the following output:

? graylog-server.service - Graylog server
     Loaded: loaded (/lib/systemd/system/graylog-server.service; disabled; vendor preset: enabled)
     Active: active (running) since Sat 2020-09-05 08:50:16 UTC; 15min ago
       Docs: http://docs.graylog.org/
   Main PID: 8693 (graylog-server)
      Tasks: 156 (limit: 2353)
     Memory: 865.0M
     CGroup: /system.slice/graylog-server.service
             ??8693 /bin/sh /usr/share/graylog-server/bin/graylog-server
             ??8726 /usr/bin/java -Xms1g -Xmx1g -XX:NewRatio=1 -server -XX: ResizeTLAB -XX: UseConcMarkSweepGC -XX: CMSConcurrentMTEnabled -XX>

Sep 05 08:50:16 ubuntu2004 systemd[1]: Started Graylog server.

You can also verify the Graylog server log with the following command:

tail -f /var/log/graylog-server/server.log

Once the Graylog server has been started successfully, you should get the following output:

2020-09-05T08:51:36.473Z INFO  [ServerBootstrap] Services started, startup times in ms: {InputSetupService [RUNNING]=59, JobSchedulerService [RUNNING]=105, GracefulShutdownService [RUNNING]=106, OutputSetupService [RUNNING]=110, BufferSynchronizerService [RUNNING]=111, UrlWhitelistService [RUNNING]=153, JournalReader [RUNNING]=166, KafkaJournal [RUNNING]=222, MongoDBProcessingStatusRecorderService [RUNNING]=240, ConfigurationEtagService [RUNNING]=259, EtagService [RUNNING]=302, StreamCacheService [RUNNING]=306, LookupTableService [RUNNING]=376, PeriodicalsService [RUNNING]=655, JerseyService [RUNNING]=58701}
2020-09-05T08:51:36.477Z INFO  [ServerBootstrap] Graylog server up and running.

At this point, Graylog server is started and listening on port 9000.

Configure Nginx as a Reverse Proxy for Graylog

Next, you will need to install and configure Nginx as a reverse proxy to access the Graylog server.

First, install the Nginx server with the following command:

apt-get install nginx -y

After installing the Nginx server, create a new Nginx virtual host configuration file with the following command:

nano /etc/nginx/sites-available/graylog.conf

Add the following lines:

server {
    listen 80;
    server_name graylog.example.org;

    location / {
      proxy_set_header Host $http_host;
      proxy_set_header X-Forwarded-Host $host;
      proxy_set_header X-Forwarded-Server $host;
      proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
      proxy_set_header X-Graylog-Server-URL http://$server_name/;
      proxy_pass       http://127.0.0.1:9000;
    }
}

Save and close the file when you are finished. Then, verify the Nginx for any syntax error with the following command:

nginx -t

You should get the following output:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Next, enable the Nginx virtual host configuration file with the following command:

ln -s /etc/nginx/sites-available/graylog.conf /etc/nginx/sites-enabled/

Finally, restart the Nginx service to apply the changes:

systemctl restart nginx

Next, verify the status of the Graylog with the following command:

systemctl status nginx

You should get the following output:

? nginx.service - A high performance web server and a reverse proxy server
     Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2020-09-05 09:07:50 UTC; 20s ago
       Docs: man:nginx(8)
    Process: 9408 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
    Process: 9419 ExecStart=/usr/sbin/nginx -g daemon on; master_process on; (code=exited, status=0/SUCCESS)
   Main PID: 9423 (nginx)
      Tasks: 3 (limit: 2353)
     Memory: 10.2M
     CGroup: /system.slice/nginx.service
             ??9423 nginx: master process /usr/sbin/nginx -g daemon on; master_process on;
             ??9424 nginx: worker process
             ??9425 nginx: worker process

Sep 05 09:07:50 ubuntu2004 systemd[1]: Starting A high performance web server and a reverse proxy server...
Sep 05 09:07:50 ubuntu2004 systemd[1]: Started A high performance web server and a reverse proxy server.

Access Graylog Web Interface

Now, open your web browser and type the URL http://graylog.example.com. You will be redirected to the Graylog login page as shown below:

Provide your admin username, password and click on the Login button. You should see the Graylog dashboard in the following page:

Now, click on the System >> Overview. You should see the status of the Graylog server in the following page:

Conclusion

Congratulations! you have successfully installed and configured Graylog server with Nginx as a reverse proxy on Ubuntu 20.04. You can now explore the Graylog and create a input to receive Rsyslog logs from external sources. Feel free to ask me if you have any questions.