Locking in bash
To ensure that there is not more than one running instance of your bash script you need to implement locking. If your platform has flock utility this is pretty straightforward:
Using this approach you need to remember, that any forked processes will inherit file handles from this script. I have a script, that is run by cron. This script starts ssh-agent if it is not running and runs commands via ssh on several hosts. The problem is that with approach above ssh-agent will hold the lock, so this script will run only once until ssh-agen will exit. To avoid this you need to close explicitly file handles while starting processes that will fork. For ssh-agent example this will look like:
If for some reason you can’t use flock locking is still possible. Here is how you can implement it:
How this works:
- Pids are tracked via pid list. We go through all pids and check if any of them are alive.
- Dead pids are ignored.
- If we encounter alive pid, that is not current pid - some other process was started earlier, we report failure.
- For alive pid that is this process we truncate pid list to just current pid (mv is atomic) and report success.
- If we exited pid check loop we append current pid to the pid list and try again. Append is atomic operation (more details here)
How reliable is this?
For testing I used following command:
Success criteria was no unexpected pids.
For final testing I used following command:
I tried to run this test on my laptop with 4 cores i7, 2 core vm and 24 core server. There were no failures. However I’m not sure I can forsee all possible scenarios and suspect this code may fail. You shouldn’t use this code to control nuclear reactor, but it is good enough to ensure that your cron job will run as a singleton.