Integrating InfluxDB and Icinga

Icinga2 has built-in support for writing monitoring data to InfluxDB. This makes Icinga quite extensible as it allows for other programs to read the gathered data from InfluxDB. For example, Icinga does not have built-in graphing support. But monitoring data can be written to InfluxDB and then consumed by a dedicated graphing tool like Grafana.

Install Packages

Let's install a few necessary packages

apt install influxdb influxdb-client ssl-cert

Generate self-signed certificate

Now generate a self-signed certificate for accessing InfluxDB over TLS

make-ssl-cert generate-default-snakeoil
usermod -aG ssl-cert influxdb

Create Database and Users

Start and enable the InfluxDB service

systemctl enable --now influxdb

Now we'll create our database and users. We'll be creating 3 users with different access rights. An admin user with full control over the database, a user with write access that Icinga will use to write data to the database, and a read-only user to allow external programs to read data from the database.

influx -ssl -unsafeSsl -execute "create database icinga2; create user admin with password 'changeme'; create user icingauser with password 'changeme'; create user readonly with password 'changeme'; grant all to admin; grant write on icinga2 to icingauser; grant read on icinga2 to readonly;"

Configuration Files

Then write InfluxDB's configuration file at /etc/influxdb/influxdb.conf

reporting-enabled = false
[meta]
  dir = "/var/lib/influxdb/meta"
[data]
  dir = "/var/lib/influxdb/data"
  wal-dir = "/var/lib/influxdb/wal"
[coordinator]
[retention]
[shard-precreation]
[monitor]
[http]
  enabled = true
  bind-address = ":8086"
  auth-enabled = true
  https-enabled = true
  https-certificate = "/etc/ssl/certs/ssl-cert-snakeoil.pem"
  https-private-key = "/etc/ssl/private/ssl-cert-snakeoil.key"
[ifql]
[logging]
[subscriber]
[[graphite]]
[[collectd]]
[[opentsdb]]
[[udp]]
[continuous_queries]
[tls]
  min-version = "tls1.2"

And restart InfluxDB to pickup the changes

systemctl restart influxdb

Then we need to configure Icinga to write data to our database. Start by enabling the influxdb feature in Icinga

icinga2 feature enable influxdb

Now we tell Icinga how to write to our database. Open the configuration file at /etc/icinga2/features-available/influxdb.conf and replace with the following

object InfluxdbWriter \"influxdb\" {
  host = "127.0.0.1"
  port = 8086
  username = "icingauser"
  password = "icinga_password"
  ssl_enable = true
  database = "icinga2"
  flush_threshold = 1024
  flush_interval = 10s
  host_template = {
  measurement = "$host.check_command$"
    tags = {
      hostname = "$host.name$"
    }
  }
  service_template = {
    measurement = "$service.check_command$"
    tags = {
      hostname = "$host.name$"
      service = "$service.name$"
    }
  }
}

And finally restart Icinga to make those changes live.

systemctl restart icinga2

Icinga2 will now be writing the data it collects to your InfluxDB instance.


Consider donating if this article was useful. [BTC]