Logs with syslog-ng

The end of this post, you should be able to perform the following tasks and understand the basic concept of logging:

• Why we use logs
• Installing syslog-ng
• Custom configuration for logs via syslog-ng

Logs are equals kind of agent in Linux that tells what is happening on the system. The Logs tracks and monitor important services, kernel, and applications that run on your machine. Regardless of whether you are Sysadmin, DevOps engineer, or Developer the Logs are a powerful instrument to inspect and understand the occurred problem which made daily life quite easier.

https://www.pexels.com/@paulita

In Linux, the Logs are typically located under /var/log directory. You might manipulate the given log's path which is not recommended for avoiding inconsistencies.

There are several logging tools such syslog, rsyslog etc. In this post, I will focus on syslog-ng.

syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, message queues, databases (SQL and NoSQL alike), and more. see

syslog-ng is basically extended the concept of syslog which also addresses known vulnerabilities of Syslog and makes better logging. For the centralization of logs , the syslog-ng is one of the powerful instruments for Sysadmins and DevOps engineers.

Let’s start with the installation process.

$ sudo apt install syslog-ng

After successful installation to syslog-ng will run as daemon service on your machine. To verify installation and running process, simply hit the terminal:

systemctl status syslog-ng

Let’s inspect the output line by line

1. The description of service: System Logger Daemon
2. The Path where syslog-ng.service file are stored and enabled or disabled status. 

If you have read my post about creating daemon service with systemd, you might ask why not unter /etc/systemd/system.

The /lib/systemd/system directory holds unit files that are provided by the system or are supplied by installed packages.

The /etc/systemd/system directory stores unit files that are user-provided.

3. Current status and since when is running
4. Documentation about syslog-ng
5. Process ID
And memory usage, CGroup and some logs

So the syslog-ng is running and from now on we can configure our logger. In Linux, the configuration files are located under /etc and further in the corresponding directory. The absolute path is /etc/syslog-ng/syslog-ng.conf

We will create our custom syslog-ng.conf file. Before we move further backup the existed syslog-ng.conf file.

# backup the existing config file
$ sudo mv /etc/syslog-ng/syslog-ng.conf syslog-ng.confBAK
#create a new config file
$ sudo touch /etc/syslog-ng/syslog-ng.conf
# Before we start the config file, let's make a directory where syslog-ng's logs will be stored and the file as well.
$ sudo mkdir -p /var/log/syslog-ng && sudo touch /var/log/syslog-ng/all-logs.txt
# Let's start to configure as we need
$ sudo vim /etc/syslog-ng/syslog-ng.conf

May you want to keep syslog-ng.conf file alive and get additional logs, then simply write your custom foo.conf file and put under /etc/syslog-ng/conf.d in this way syslog-ng.conf file will include all files with .conf extentison under conf.d directory.

Let’s Jenkins is running as a service. We aimed to collect all Jenkins related logs. Sure this can be any service or application you desire. Then our configuration file will look as below:

@version: 3.25 
@include “scl.conf”
filter myfil_f{ not match(“mehmet”); };
source s_local {
 system(); internal();
};
destination d_local {
 file(“/var/log/mehmet-log.txt”);
};
log {
 
 source(s_local); filter(myfil_f); destination(d_local);
};

The first two lines are mandatory. Then I used the filter that allows you to use regular expressions. In our example, we are excluding all log messages contains “mehmet” string.

The next part is source, so you should tell the syslong-ng where to collect logs. We will fetch logs from the system and internal messages.

Then destination, the file, or the directory that logs will be stored. Typically as we mentioned above logs are under /var/log directory. Simply create your file there and give a custom name that related your application.

The last and essential part is log scope. Here you should list your sources, filters, and the destination. Please keep an eye on the syntax each statement followed by a semicolon.

So the configuration is done now we have to reload daemon and restart the syslog-ng.

$ sudo systemctl daemon-reload
$ sudo systemctl restart syslog-ng.service

If you do not have any typo then the output of commands above be nothing. Now with the configuration above, if I check mehmet-log.txt file It should not contain any logs with “mehmet” string.

No mehmet

I just changed filter from

filter myfil_f { not match(“mehmet”); }; → filter myfil_f { match (“mehmet”); };

contains mehmet

I try to simply give you a quick start with syslog-ng. There are a lot of useful features.

References
* Syslog-ng documentation