Author Archive for Andy Brist

Monitoring Apache Cassandra Database Nodes with Nagios XI

As cloud services grow in popularity, so do the networks that provide those cloud services.  Few webserver-based distributed databases are as easy to install and configure as Apache Cassandra.  Apache Cassandra is an open source distributed database management system designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure. Cassandra offers robust support for clusters spanning multiple data centers, with asynchronous master-less replication allowing low latency operations for all clients.

Cassandra relies on the Java platform, and as those of you who have tried to configure Java app monitoring most likely know, the experience can be painful.  There are a handful of plugins on the Nagios Exchange that attempt to simplify the configuration.  As these plugins rely on the Apache Cassandra utility “nodetool”, you either need to install Cassandra on the Nagios server (which is not suggested) or use an agent (like NRPE) to run the plugin script directly from the Cassandra server (which should have the nodetool utility).

The Cluster Node Check is designed to verify whether the number of live nodes is less than a specified number, and if so trigger a warning or critical alert within Nagios.

Continue reading ‘Monitoring Apache Cassandra Database Nodes with Nagios XI’

Exploring the New JSON CGIs in Nagios Core 4.0.7! (Part 1)

The JSON CGIs, from the JSON branch of core, have been officially released with Nagios Core 4.0.7!

The original design goals were:

  1. To provide all information available in current CGIs in JSON format.
  2. Place the presentation responsibility on the client to minimize network traffic and server load.
  3. Perform operations on the server side that are significantly more easily done there.
  4. Spark community developers to create new Nagios Core UI’s from the easy to work with JSON from the CGIs.

The CGIs provide an API to query object, status, and historical information through GET requests.  They use the same authentication as other CGIs.  Once queried, they return valid JSON that can be parsed into JavaScript objects for client side models and processing.  The API is very robust, providing multiple ways to limit queries – name/descriptions, host/service groups,  update/changes times, among many others.

The three new CGIs are:

  1. objectjson.cgi  (object configuration)
  2. statusjson.cgi  (status information)
  3. archivejson.cgi  (historical logs)

Additionally, a new web app is included – jsonquery.html & jsonquery.js.  This is a small UI for crafting GET requests, it can be used to trial specific parameters for GET requests, or to just explore the api.  It is also the easiest way to get acquainted with the new CGIs.

Continue reading ‘Exploring the New JSON CGIs in Nagios Core 4.0.7! (Part 1)’

Unleashing the Power of Nagios — Utilizing the Actions Component

The “Action URL” component has been quietly deprecated in favor of the newer “Actions” component. The “Actions” component has been released for some time now and has been rolled up into the base install of XI. Although it is most commonly used to add URLs to specific Nagios objects on the details pages in XI, it is by far more advanced than its predecessor “Action URL”. The “Actions” component is truly one of the more powerful components available for Nagios XI, on par with event handlers in extensibility and complexity.

The use cases for the Actions component are near endless. A simple example would consist of the streamlining of repeated tasks, like the adding of common comments to particular hosts. This can be accomplished using the Actions component and the command pipe. For the following example, we will assume that the comment to be added to a host relates to the completion of a successful security audit by the user “auditor”. This action will include the passing of the %host% macro to a script that will write to the Nagios command pipe.

First create a script named “security_audit_completed.sh” in /usr/local/nagios/libexec. The script follows:

Continue reading ‘Unleashing the Power of Nagios — Utilizing the Actions Component’

Take Control, and Automate Your Network with Event Handlers in Nagios XI

Event handlers have been a major part of Nagios Core and Nagios XI for long time and can greatly increase the efficiency of your network and decrease incident response time. From restarting downed services, forwarding emails to a ticketing systems, or updating thousands of servers, the possibilities for event handlers are limited only by your imagination. Below, you will find the introductory portion of the documentation which will walk you through the creation of your first event handler in its most basic form. If the following example isn’t enough, the full documentation includes a complete tutorial for event handler macros, and more advanced users can learn how to fully customize their Nagios environment.

Step I. Create a command in XI for the event handler

In XI, go to the CCM (Configure → Core Config Manager → Commands). Click “Add New” in the right hand pane. Give the command a name, we will use “event_handler_test” for this example.

Now we need to define the “Command Line”. “$USER1$” references the folder “/usr/local/nagios/libexec” from the resources.cfg file. This is the default path for plugins and scripts in XI. We will create a script later on to be called by this command, and for the sake of uniformity, we will name our script “event_handler_test.sh”.

As this command will not be used to check hosts nor services, but will be used as an event handler, set the “Command Type” dropdown to “misc command”.

Make sure the “Active” box is checked.

The final command definition should resemble the image to the right. Click save when finished.

Step II. Create a dummy host

In XI, go to the CCM (Configure → Core Config Manager → Monitoring → Hosts). Click “Add New”. We will set the “Host Name” field as “event_handler_test”.

As this is just a mock up, set “Address” to the localhost “127.0.0.1”.

Choose “check_dummy” for the “Check command”.

Enter “0” for $ARG1$. This forces the “check_dummy” command to always return “0” (UP).

Clicking “Test Check Command” should output “OK”, this means you have set up the command correctly.

Set the required defaults:

Under the “Check Settings” tab, enter “5” for “Check interval” and “Max retry attempts”, and “1” for “Retry interval”.

Set the “Check period” dropdown to “xi_timeperiod_24x7”

Under the “Alert Settings” tab, set the “notification period” dropdown to “xi_timeperiod_24x7”.

Step III. Add the event handler to the dummy host

Under the tab “Check Settings” for the dummy host choose “event_handler_test” in the “Event handler” dropdown box.

Enable the event handler by clicking the bullet “on” for the “Event handler enabled” option. Save out and “Apply configuration”.

Step IV. Create a script for the handler to run

Now that the XI event handler and host object are configured, we need to create a script to be run by the handler. The first iteration of the script will just output some *very* basic information to a text file in /tmp. In the proceeding sections of this document we will add functionality and complexities to the script and command of the handler (mostly macros and script logic). But before we get to the fun stuff, lets make sure the basic script works.

Open up a terminal to the XI server and navigate to plugin directory.

cd /usr/local/nagios/libexec

Now create a file named “event_handler_test.sh”, make it executable and edit it:

touch event_handler_test.sh
chmod +x event_handler_test.sh
nano event_handler_test.sh

Paste the following into the file:

#!/bin/bash
DATE=$(date) #sets the DATE variable to the output of the 'date' command
echo "The host has changed state at $DATE" > /tmp/hostinfo.txt #echos text to a file including the contents of the DATE variable.

Save out.

Step V. Test the event handler

Now that everything is in place, we need to force the dummy host into a down state to verify if the hostinfo.txt file is created in /tmp. The easiest was to do this is to submit a passive check with the “Check Result” of “DOWN”. This should trigger the event handler as it will register as a state change.

The XI web interface, go to → Home → Host Detail → Click the Host “event_handler_test” → Click the “Advanced” tab → And then Click “Submit passive check result” under the “Commands” list. Change the “Check Result” dropdown menu to “DOWN” and input “TEST” for the “Check Output” field.

Click “Commit”, and then done. You should see the host change to a down state after a moment or two. Now lets check the hostinfo.txt file from the XI server’s CLI:

cat /tmp/hostinfo.txt

You should see output resembling:

The host has changed state at Sun Jun 2 22:36:22 CDT 2013

Congratulations! You have created your first event handler. With this knowledge, you can automate nearly anything in your environment to keep it running smoothly. Don’t know bash scripting?  No problem! The full document, linked below, includes a number of use cases in bash to get you started.  From emailing a ticketing system to performing yum updates, event handlers can help you create a more self-sufficient network environment and integrate with other systems in your organization.

For the full documentation, click on the link below: Introduction to Event Handlers