Tag Archive for 'buffer'

Passing key value pairs through kernel messages with libnagios

If you caught my last post, you learned about compiling libnagios and using the Nagios library in your own code. This time we’re going to use that same Nagios library along with some borrowed NDOUtils code and build something super neat: a message passing system utilizing key-value vectors and buffers attached to the kernel messaging (IPC) utilities from NDO.

Just like last time, I’m going to assume you have a sane build environment set up (where tools like make and ./configure are working) before we go any further. If you are following along, now would be the time to get these in order.

Before we get started writing our own code, we need to borrow some files from NDOUtils. We only need two files: src/queue.c and include/queue.h. You can find the source for each at https://github.com/NagiosEnterprises/ndoutils/blob/master/src/queue.c and https://github.com/NagiosEnterprises/ndoutils/blob/master/include/queue.h, respectively. Once you have them downloaded, we’ll be ready to move on.

First, open up queue.c in your favorite editor and delete the line that includes “../include/config.h” (we’ll manually include the headers we need in a bit). You’ll want to change the path to your queue.h file to reflect wherever you downloaded it. For the purposes of this post and the examples that follow, all of my source and header files will be in the root directory of the Nagios Core source code that we downloaded while following along with the last post.

So, the top of your queue.c should have looked like this when you downloaded it:

After you’ve made the changes I mentioned, it should look like this:

Save that file and open up queue.h. We’re going to add all of the necessary header files that would have been included with config.h in here. Find the line that defines NDOUTILS_INCLUDE_QUEUE_H, and directly after it add the following include directives:

By now, the top of your queue.h file should look like this:

Perfect! Now we’re going to set up our main application. Last time I walked through the file as we were creating it, but this time it’s a tad more advanced so we’re going to look at it in its entirety and then step through the explanation. Create a file named test2.c, and fill it with the following content:

Here we’re just including our necessary header files:

This is a function that we use to “walk” through the key-value vector and perform some meaningful operation against each pair we find. *kv will be the key-value pair, and *arg is an additional argument you can pass to the kvvec_foreach function (https://github.com/NagiosEnterprises/nagioscore/blob/master/lib/kvvec.h#L136-L146).

Now we’re going to start our application and define a few parameters to be used later. KEYVAL_SEP will be the separating character used to determine key from values. PAIR_SEP will be the separating character to keep key-value pairs separate from each other. OVERALLOC will be used to determine how much extra space is allocated when we create our vector. Then we go ahead and set up a basic char * buffer with our key-value data and set keyval_len equal to how many [key-values] we have there.

Now we’re getting to the nitty gritty. It’s time to declare some vectors and a buffer. We’ll initialize one for use immediately, but save the others for later. We’re using kvvec and kvvec_buf structures to store vectors and buffers (https://github.com/NagiosEnterprises/nagioscore/blob/master/lib/kvvec.h#L25-L46) and initializing an empty kvvec with kvvec_create (https://github.com/NagiosEnterprises/nagioscore/blob/master/lib/kvvec.h#L70-L76).

Let’s cycle through our *keyval array and inject our pairs into our freshly initialized kvvec using kvvec_addkv (https://github.com/NagiosEnterprises/nagioscore/blob/master/lib/kvvec.h#L115-L134).

Remember the function we defined that was going to “walk” through the key-value vector and perform a meaningful operation against each key-value pair? We’re going to call it now, and as it walks through each key-value, it will print both elements.

Now we’re going to sort the vector using kvvec_sort (https://github.com/NagiosEnterprises/nagioscore/blob/master/lib/kvvec.h#L108-L113), which will sort the pairs alphabetically by key. Once we’ve done that, we’re going to call the walker function again which will print the key-value pairs in their new order.

The magic and beauty of libnagios’s kvvec function really shines through with the buffer/vector conversion functions. We’re going to use kvvec2buf (https://github.com/NagiosEnterprises/nagioscore/blob/master/lib/kvvec.h#L163-L178) to convert our vector into a single buffer using the parameters we set up when we began our application.

Now, we finally get to the borrowed NDOUtils code. This next block is responsible for initializing a message queue utilizing ndo2db_queue_init (https://github.com/NagiosEnterprises/ndoutils/blob/master/include/queue.h#L35-L36). It takes only one argument: an integer ID to be used as identification for the message queue. We’re passing our process id.

Next we’re creating a struct ndo2db_queue_msg (https://github.com/NagiosEnterprises/ndoutils/blob/master/include/queue.h#L30-L33) variable and copying our key-value buffer.

Finally, we’ll send our message to the queue using ndo2db_queue_send (https://github.com/NagiosEnterprises/ndoutils/blob/master/include/queue.h#L41-L42). This function takes two arguments: an ndo2db_queue_msg and a size_t the size of the message to send.

Our example here is very basic. Ideally, it is at this point in the application that something would happen to this buffer. It would be picked up by some child process or thread or a different process entirely and processed there. In order to maintain some brevity, we’ll simply be pulling it right back out of the queue! In order to do that, we’re using pop_from_queue (https://github.com/NagiosEnterprises/ndoutils/blob/master/include/queue.h#L44-L45). This simply returns a char pointer containing your message text (which happens to be the buffer text). Once we’ve done that, we’re going to use buf2kvvec (https://github.com/NagiosEnterprises/nagioscore/blob/master/lib/kvvec.h#L180-L191) to convert our message passed buffer back into a key-value vector.

After that conversion takes place, we’ll issue a call to the walker function yet again just so we can make sure that our keys and values are right where we expect them to be.

Whew. How exciting! All that’s left now is to clean up the mess we made. In order to do that, we need to call ndo2db_queue_free (https://github.com/NagiosEnterprises/ndoutils/blob/master/include/queue.h#L38-L39) so that we release the queue’s system resources. After that, we’re freeing some buffers and then destroying the key-value vectors using kvvec_destroy (https://github.com/NagiosEnterprises/nagioscore/blob/master/lib/kvvec.h#L148-L154). We’re passing the KVVEC_FREE_ALL flag here, but you could decide to only KVVEC_FREE_KEYS or KVVEC_FREE_VALUES.

Just in case you missed it last time, we’ll need to have libnagios compiled and installed if we want any of this to work. If you haven’t already, you can do this with the following commands:

Now that we have libnagios all set up and usable, we can compile our test2 application. You can do that with the following command:

Now lets run our application and see the output!

Neat! Just as a side note, when you’ve pushed your messages onto the kernel message queue, you can actually watch the queue with the command ipcs -q.

I hope you’ve learned a little more about using the Nagios library to extend your own code.