Stream any log file to Slack using curl

When was the last time you had to execute a long-running script and wished you could monitor the progress remotely without setting up a plethora of systems? For us, it was yesterday. And doing that was so simple, we had to share it.

At Postman, we use Slack for a lot of stuff, having it report the progress of an arbitrary script was simply natural. The following piece of code tails a file and for every new line added to the file, streams it to a slack incoming webhook using curl.

To use this script, save it as an executable script and simply pass the path to the log file and a webhook url to this script.

If you want this script to keep running even after you have logged out of your SSH terminal, use nohup command.

You can even make this script send the line to slack only when a particular keyword is found by sending the keyword as a third parameter.

The above example tails an nginx access log and streams the line to slack when it has a 404 response. How simple and cool is that!

  • genius – thanks for sharing!

  • Phil GeLinas

    This is awesome…question – Is it possible to capture and stream more than one type of error (e.g., 403, 404, 500)?

    • shamasis

      Sure. Check the syntax of grep command. You could do "\ 404 \ |\ 302\ " or something similar.

  • This post is amazing.

    I'd like a recommendation: how would you make this job persist across system reboots? Should I set it in a cron job for @reboot or do you have another suggestion? I'm currently running as:

    @reboot ./ "/var/log/nginx/error.log" "…"

    • cron would be a bit unreliable and messy. Try using upstart. Assuming you're on Ubuntu.

      • Thanks. Apparently Ubuntu 16.04 moved to systemd as default, just as I was getting confident with upstart. Any tips on how this could work from systemd?

        • shamasis

          I haven't spent much time with systemd to comment on it off hand. It should be similar in theory though.

  • superman

    The script work as designed. Thank you.

    I'm late to this show and just recently discovered #slack.

    The script is missing some customization. For example: the script is missing the options to:

    *override the username (payload={"username": "robot")
    *channel (payload={"channel": "off-topic")
    *icon (payload={"icon_url": "")

    Was wondering if you could provide some guidance.

    • shamasis

      In the script, simply replace the JSON text I have set with a customised variant of your own and it should work just fine.

  • Razvan Rosca

    This. Is. Awesome!!!

    Any way to have an ignore list, rather than a "whitelist"? I would like to parse just *some* information from an error_log, not all.

    Thank you so much!

    • shamasis

      You could try piping the tail through a grep before sending it to the loop.

    • shamasis

      You could invert grep. Replace the grep in my code using grep -v or grep -L. Try running grep --help | grep invert to see more on how inversion switches work.