WebFaction
Community site: login faq

We have around 20 log files of varying formats, generated by several different applications. We would like to receive notifications (preferably text messages, but email can be made to work in lieu of that) should certain conditions be met in the contents of those files.

For example, for one log file we might like to be notified if a line containing the word "EXCEPTION" is logged. In another, we would like to be notified if anything is logged at all. And still other log files should be ignored altogether.

Has anyone a suggestion on a decent way of accomplishing this? We were really hoping to find a solid, existing solution rather than rolling our own monitoring scripts, but we're not afraid to get our hands dirty if that turns out to be the best answer.

Cheers!

asked 17 Feb '16, 13:22

kirkb
33
accept rate: 0%


If I were doing this, I'd opt for a simple script running for each log.

For example, if I wanted to be pinged when the word "EXCEPTION" showed up in my app's error log:

#!/bin/bash

LOGFILE=$HOME/logs/user/error_myapp.log
SEARCHFOR="EXCEPTION"
SUBJECT="Exception in my app"
MAILTO="me@domain.com"

tail -Fn0 $LOGFILE | \
while read line ; do
  echo "$line" | grep -q "$SEARCHFOR" && ( echo "$line" | /bin/mail -s "$SUBJECT" $MAILTO )
done

This will mail the log line containing EXCEPTION to me@domain.com whenever it occurs.

You can change the variables at the start of the script to adapt to whatever it is you're monitoring, then run the scripts continuously, perhaps backgrounded in a screen/tmux session.

Hope that helps!

permanent link

answered 17 Feb '16, 20:21

seanf
12.2k41836
accept rate: 37%

edited 17 Feb '16, 20:23

1

This is great. We will play around with it a bit and see if we can get it working the way we want and post the final solution back here.

There are a few other things we need to work out, for example, if we hit a rocky patch, it would be nice if we could queue up a few errors and send them all at once rather than sending one email for each error.

Also, we'll want these "listeners" running the the background so we'll need to monitor them for uptime using crontab or maybe supervisor.

We'll also need to figure out what happens to these "listeners" when log rollovers occur.

Anyhow, lots of stuff to work out (some of which involves us hashing out our requirements).

Thank you!

(18 Feb '16, 15:25) kirkb
1

So we ended up going a different way with this. We added the ability to view log files from our application. I'm going to go ahead and mark this as the accepted answer, though, in case it helps someone else in the future.

Thanks for your help!

(22 Feb '16, 23:20) kirkb
Your answer
toggle preview

Follow this question

By Email:

Once you sign in you will be able to subscribe for any updates here

By RSS:

Answers

Answers and Comments

Markdown Basics

  • *italic* or _italic_
  • **bold** or __bold__
  • link:[text](http://url.com/ "title")
  • image?![alt text](/path/img.jpg "title")
  • numbered list: 1. Foo 2. Bar
  • to add a line break simply add two spaces to where you would like the new line to be.
  • basic HTML tags are also supported

Question tags:

×28
×15
×10
×9
×1

question asked: 17 Feb '16, 13:22

question was seen: 1,730 times

last updated: 22 Feb '16, 23:20

WEBFACTION
REACH US
SUPPORT
AFFILIATE PROGRAM
LEGAL
© COPYRIGHT 2003-2019 SWARMA LIMITED - WEBFACTION IS A SERVICE OF SWARMA LIMITED
REGISTERED IN ENGLAND AND WALES 5729350 - VAT REGISTRATION NUMBER 877397162
5TH FLOOR, THE OLD VINYL FACTORY, HAYES, UB3 1HA, UNITED KINGDOM