Email notifications via supervisor on abrupt termination of process

supervisor has some awesome plugins. One among them is superlance. It allows you to notify via email in case of certain event occurrence. For ex: you can send an email to the ops guys if a process terminates abruptly.

Here's a sample python application that will terminate abruptly every 30 seconds:

import time

i = 10
while True:
	print "I am still working"
	time.sleep(3)
	j=  1000 / i
	i -= 1

To have supervisor and superlance setup (for Ubuntu) you need to:

pip install supervisor
pip install superlance
sudo apt-get install sendmail
```

Next is `superlance` config file. I am providing the `program` and  `email` config part below. For full file you can look at my github project here:

```language-bash
[program:simple_app]
command=python simple_app.py              							
process_name=%(program_name)s 										
numprocs=1                    										
directory=/home/ravi/dev/projects/python/supervisor-play         	
autostart=true                										
startretries=3                										
stdout_logfile=~/dev/projects/python/supervisor-play/s.out
stderr_logfile=/home/ravi/dev/projects/python/supervisor-play/s.err 

[eventlistener:crashmail] 
command=/home/ravi/.virtualenvs/super-awesome/bin/crashmail -a -m james.bond@gmail.com
events=PROCESS_STATE
```

To start up your program simply do:  `supervisord -c supervisord.conf -n`

And that's it! *You have got mail!!! ;-)*

**Gotchas**:

1. I did not have `sendmail` installed and supervisor was failing silently.
1. You  need to fail abruptly and apparently sending process kill signal is not abrupt. I have to fail with some unexpected program error.

> Written with [StackEdit](https://stackedit.io/).