# Parsing logs in sed efficiently

I have an application which log file has most lines start with a time/date, followed by a debugging level and the actual log message. Often I want to watch the output live and colors make it a lot better readable. This is what I use at the moment for piping the logs:

while read line; do
echo "$line" | sed -e "s,^.* DEBUG,$(tput setaf 2)&$(tput sgr0)," \ -e "s,^.* INFO,$(tput setaf 2)&$(tput sgr0)," \ -e "s,^.* WARN,$(tput setaf 3)&$(tput sgr0)," \ -e "s,^.* ERROR,$(tput setaf 1)&$(tput sgr0)," \ -e "s,^.* SEVERE,$(tput setaf 1)$(tput bold)&$(tput sgr0),"
done


The call on the command line looks like this:

tail -fn 200 /path/server.log | ./logparser


This all works as expected but when many lines are printed at once the parser lags behind. I think this is because I run sed once for every single line. Is there a better solution?

• You're wasting time having the shell read input and repeat it to sed. Why are you doing that? sed would just read from standard input anyway? The logparser script could just be one line or you could put the commands in an input file and throw away the logparser script. – itsbruce Mar 17 '15 at 11:16
• tail -fn 200 /path/server.log | sed -f /path/to/sedcommands – itsbruce Mar 17 '15 at 11:18
• How would I use tput in a sed script? I really prefer tput over the arcane obscure escape key notation for terminal colors. – Hubert Grzeskowiak Mar 17 '15 at 11:25
• @itsbruce feel free to post a real answer and you'll get some points :-) – Hubert Grzeskowiak Mar 17 '15 at 11:30
• Oh, well, I would but I wasn't so much reviewing your shell script as talking unix admin basics. When you have it working the way you want, post your own answer and say what worked for you. Perfectly acceptable StackExchange behaviour. – itsbruce Mar 17 '15 at 11:32

Thanks to itsbruce this runs much faster:

sed -e "s,^.* DEBUG,$(tput setaf 2)&$(tput sgr0)," \
-e "s,^.* INFO,$(tput setaf 2)&$(tput sgr0)," \
-e "s,^.* WARN,$(tput setaf 3)&$(tput sgr0)," \
-e "s,^.* ERROR,$(tput setaf 1)&$(tput sgr0)," \
-e "s,^.* SEVERE,$(tput setaf 1)$(tput bold)&\$(tput sgr0),"


Note: first two and the last line are removed.

Since sed takes the STDIN, this basically starts one sed instance for whole stream in contrast to one per line.