Customizing triggers in Zabbix - Part 2

Zabbix monitoring customize triggers
Following the articles initial article on how to install Zabbix and Zabbix triggers, this article adds some more trigger customization for IMAPS, LDAPS, SMTP/submission and SSL Certificate lifetime check These aren't available by default in Zabbix 3.

1. IMAPS

As a first step, we're searching an addon to do this. First, you need to enable external scripts. Edit the zabbix server configuration:

vi /etc/zabbix/zabbix_server.conf
...
# Default:
# ExternalScripts=${datadir}/zabbix/externalscripts

ExternalScripts=/etc/zabbix/externalscripts

As for the script, we use the ssltls.check script. It's made by Simon Kowallik. fa-thumbs-up green:

https://github.com/simonkowallik/Zabbix-Addons/tree/master/ssltls.check

Download it, and copy it to the externalscripts directory.:

chown zabbix:root ssltls.check
chmod og+x ssltls.check

Try it from the commandline. If the service is up, the command will return 1, 0 if not.:

ssltls.check imapserver:993 native simple
1

The script works. Next we have to get Zabbix to use the external script. Adjust the base Template "Template App IMAP service". Go to Configuration > Templates > Template App IMAP Service > Items

Change template item

The corresponding trigger:

Change template trigger

Another possibility would probably be to use a port check. This didn't seem to work for me.:

Type: Simple check Key: net.tcp.port[,993]

2. SMTP and submission

SMTP

We want to check port 25. Create a new item or edit the existing SMTP template.:

Change SMTP template item

Create a new trigger or edit the existing one.

Change SMTP template trigger

I tried these, but that didn't seem work. If you want to try, here is the info:

Type: Simple check
Key: net.tcp.service[tcp,25]

If that doesn't work, use this key:
Key: net.tcp.port[,25]

If that doesn't work:

Type: Zabbix agent (active)
Key: net.tcp.listen[25]

Submission

Add a check for submission (port 587) Go to the SMTP item of the SMTP Template and copy it to a new item. Change the item to submission and port 587.:

Change SMTP template item for submission

Go to the triggers and create a new trigger. Use a copy of the existing trigger to start with.:

Change SMTP template trigger for submission

3. LDAPS

Again, at the time of writing, this wasn't possbile with Zabbix 3.0. To this end, I've written a python script to do a check. Calling the script from Zabbix as an external script didn't work. I wrapped it in a bash script and that worked. Since this is a small tiny script, I didn't put comments since it should be obvious what it does. The python script.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
 #!/usr/bin/env python3
 # -*- coding: utf-8 -*-
 """
 This script is meant as an external check script for Zabbix.
 It will try to connect to an ldaps server.

 Author BV @ ICT Force
 """
 import ldap
 import datetime
 import logging
 import logging.handlers
 import sys
 import os

 #============== Parameters ================
 # Set DEBUG=True if you want to first test the script
 # to see if it can connect. It's a good idea.
 DEBUG=False
 # Change your LDAP settings here
 USE_SSL=True
 LOGFILE="check_ldap.log"
 LDAP_SERVER="ldapserver.mydomain.com"
 LDAP_PORT="636"
 LDAP_USER="cn=read-user,ou=Users,dc=mydomain,dc=com"
 LDAP_PWD="my-read-only-pwd"
 #==========================================

 BASE_PTH=os.path.abspath(os.path.dirname(__file__))
 LOGFILE_PTH=os.path.join(BASE_PTH,LOGFILE)

 def setupLogger():
     log = logging.getLogger("check_ldap")

     # Rotating file
     rotating_file_handler = logging.handlers.RotatingFileHandler(
               LOGFILE_PTH, maxBytes=5000, backupCount=5)
     if DEBUG:
         log.setLevel(logging.DEBUG)
     else:
         log.setLevel(logging.INFO)
     formatter = logging.Formatter('%(asctime)s %(name)-12s: %(levelname)-8s %(message)s', datefmt='%d/%m/%Y %H:%M:%S')
     rotating_file_handler.setFormatter(formatter)
     log.addHandler(rotating_file_handler)

     # Console
     console = logging.StreamHandler()
     #console.setLevel(logging.DEBUG)
     #ormatter = logging.Formatter('%(name)-12s: %(levelname)-8s %(message)s')
     console.setFormatter(formatter)
     log.addHandler(console)

     return log

 def ldap_initialize(remote, port, user, password, use_ssl=False, timeout=None):
     prefix = 'ldap'
     if use_ssl is True:
         prefix = 'ldaps'
         # ask ldap to ignore certificate errors
         ldap.set_option(ldap.OPT_X_TLS_REQUIRE_CERT, ldap.OPT_X_TLS_NEVER)

     if timeout:
         ldap.set_option(ldap.OPT_NETWORK_TIMEOUT, timeout)

     ldap.set_option(ldap.OPT_REFERRALS, ldap.OPT_OFF)
     server = prefix + '://' + remote + ':' + '%s' % port
     l = ldap.initialize(server)
     l.simple_bind_s(user, password)

 if __name__=='__main__':
     logger = setupLogger()
     logger.debug("Trying to connect")
     try:
         ldap_initialize(LDAP_SERVER, LDAP_PORT, LDAP_USER, LDAP_PWD, use_ssl = USE_SSL)
         logger.debug("Connection succeeded.")
     except Exception as e:
         logger.debug("Can't connect to the server %s" % LDAP_SERVER)
         logger.debug(str(e))
         sys.exit(1)
     sys.exit(0)

The wrapper script.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
 #!/bin/bash
 trap '' SIGINT SIGTERM SIGQUIT SIGHUP SIGALRM
 ret=`python3 check_ldap.py`

 if [ "$ret" == "0" ]; then
     echo 0
     exit 1
 else
     echo 1
     exit 0
 fi

Now to use the wrapper script in Zabbix, edit the existing LDAP template.

Change LDAP template item

Now change the trigger:

Change LDAP template trigger

4. Check certificate lifetime

Another useful check is to check a certificates lifetime so you can renew the certificate in time. No need to write my own script as I found a script to do so. See ZBXNEXT-1147 for more information. The script is made by Peter Baumann. Thanks Peter! fa-smile green

Create the script, and copy it to the /etc/zabbix/externalscripts directory. Make it executable.:

cd /etc/zabbix/externalscripts
vi ./zabbix_certificate_check.sh
...
chmod og+x zabbix_certificate_check.sh

Test the script:

sh ./zabbix_certificate_check.sh www.myserver.com 443

Now for the script.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
 #!/bin/sh
 SERVER=$1
 PORT=${2:-443}
 TIMEOUT=25
 end_date="$(/usr/bin/timeout $TIMEOUT /usr/bin/openssl s_client \
 -host $SERVER -port $PORT -showcerts < /dev/null 2>/dev/null | \
 sed -n '/BEGIN CERTIFICATE/,/END CERT/p' | openssl x509 \
 -enddate -noout 2>/dev/null | sed -e 's/^.*\=//')"

 if [ -n "$end_date" ]; then
         end_date_seconds=$(date "+%s" --date "$end_date")
         now_seconds=$(date "+%s")
         CALC=$((($end_date_seconds-$now_seconds)/24/3600))
         echo $CALC
 else
         exit 124
 fi

In Zabbix, create a new template:

Create a new template

To ease changing the port of the ssl check, we add a macro to the template. For every host you add this template to, use the macro to change the port if it is different then 443. You could add another macro for the threshold in days for when the warning should kick in. I've set it to send a warning when we have less than 31 days.

Create a new macro

Create an item where we run the external script:

Create a new item for the SSL check

Create a new trigger:

Create a new trigger for the SSL check

Add the template to a system which has an SSL certificate and see what happens.


It's pretty cool as you can easily change the checks with external scripts. It isn't all that hard to do.