8
\$\begingroup\$

Trying to write a utility script to get downtime measurements from ping logs which generally look like this:

Mar 25 00:07:13 64 bytes from 10.11.10.12: icmp_seq=2499 ttl=64 time=0.320 ms
Mar 25 00:07:14 64 bytes from 10.11.10.12: icmp_seq=2500 ttl=64 time=0.324 ms
Mar 25 00:07:15 64 bytes from 10.11.10.12: icmp_seq=2501 ttl=64 time=0.368 ms
Mar 25 00:07:16 64 bytes from 10.11.10.12: icmp_seq=2502 ttl=64 time=0.310 ms
Mar 25 00:07:17 64 bytes from 10.11.10.12: icmp_seq=2503 ttl=64 time=0.351 ms
Mar 25 00:07:18 64 bytes from 10.11.10.12: icmp_seq=2504 ttl=64 time=0.299 ms
Mar 25 00:07:19 64 bytes from 10.11.10.12: icmp_seq=2505 ttl=64 time=0.341 ms
Mar 25 00:07:20 64 bytes from 10.11.10.12: icmp_seq=2506 ttl=64 time=0.290 ms
Mar 25 00:07:21 64 bytes from 10.11.10.12: icmp_seq=2507 ttl=64 time=0.363 ms
Mar 25 00:07:22 64 bytes from 10.11.10.12: icmp_seq=2508 ttl=64 time=0.299 ms
Mar 25 00:07:23 64 bytes from 10.11.10.12: icmp_seq=2509 ttl=64 time=0.336 ms
Mar 25 00:07:24 64 bytes from 10.11.10.12: icmp_seq=2510 ttl=64 time=0.285 ms
Mar 25 00:07:25 64 bytes from 10.11.10.12: icmp_seq=2511 ttl=64 time=0.350 ms
Mar 25 00:07:26 64 bytes from 10.11.10.12: icmp_seq=2512 ttl=64 time=0.319 ms
Mar 25 00:07:27 64 bytes from 10.11.10.12: icmp_seq=2513 ttl=64 time=0.377 ms
Mar 25 00:07:28 64 bytes from 10.11.10.12: icmp_seq=2514 ttl=64 time=0.260 ms
Mar 25 00:08:07 From 10.11.10.1 icmp_seq=2550 Destination Host Unreachable
Mar 25 00:08:07 From 10.11.10.1 icmp_seq=2551 Destination Host Unreachable
Mar 25 00:08:07 From 10.11.10.1 icmp_seq=2552 Destination Host Unreachable
Mar 25 00:08:10 From 10.11.10.1 icmp_seq=2553 Destination Host Unreachable
Mar 25 00:08:10 From 10.11.10.1 icmp_seq=2554 Destination Host Unreachable
Mar 25 00:08:10 From 10.11.10.1 icmp_seq=2555 Destination Host Unreachable
Mar 25 00:08:13 From 10.11.10.1 icmp_seq=2556 Destination Host Unreachable
Mar 25 00:08:13 From 10.11.10.1 icmp_seq=2557 Destination Host Unreachable
Mar 25 00:08:13 From 10.11.10.1 icmp_seq=2558 Destination Host Unreachable
Mar 25 00:08:16 From 10.11.10.1 icmp_seq=2559 Destination Host Unreachable
Mar 25 00:08:16 From 10.11.10.1 icmp_seq=2560 Destination Host Unreachable
Mar 25 00:08:16 From 10.11.10.1 icmp_seq=2561 Destination Host Unreachable
Mar 25 00:08:19 From 10.11.10.1 icmp_seq=2562 Destination Host Unreachable
Mar 25 00:08:19 From 10.11.10.1 icmp_seq=2563 Destination Host Unreachable
Mar 25 00:08:19 From 10.11.10.1 icmp_seq=2564 Destination Host Unreachable
Mar 25 00:08:22 From 10.11.10.1 icmp_seq=2565 Destination Host Unreachable
Mar 25 00:08:22 From 10.11.10.1 icmp_seq=2566 Destination Host Unreachable
Mar 25 00:08:22 From 10.11.10.1 icmp_seq=2567 Destination Host Unreachable
Mar 25 00:08:25 From 10.11.10.1 icmp_seq=2568 Destination Host Unreachable
Mar 25 00:08:25 From 10.11.10.1 icmp_seq=2569 Destination Host Unreachable
Mar 25 00:08:25 From 10.11.10.1 icmp_seq=2570 Destination Host Unreachable
Mar 25 00:08:29 From 10.11.10.1 icmp_seq=2571 Destination Host Unreachable
Mar 25 00:08:29 From 10.11.10.1 icmp_seq=2572 Destination Host Unreachable
Mar 25 00:08:29 From 10.11.10.1 icmp_seq=2573 Destination Host Unreachable
Mar 25 00:08:32 From 10.11.10.1 icmp_seq=2574 Destination Host Unreachable
Mar 25 00:08:32 From 10.11.10.1 icmp_seq=2575 Destination Host Unreachable
Mar 25 00:08:32 From 10.11.10.1 icmp_seq=2576 Destination Host Unreachable
Mar 25 00:08:35 From 10.11.10.1 icmp_seq=2577 Destination Host Unreachable
Mar 25 00:08:38 From 10.11.10.1 icmp_seq=2581 Destination Host Unreachable
Mar 25 00:08:38 From 10.11.10.1 icmp_seq=2582 Destination Host Unreachable
Mar 25 00:08:39 64 bytes from 10.11.10.12: icmp_seq=2583 ttl=64 time=1001 ms
Mar 25 00:08:39 64 bytes from 10.11.10.12: icmp_seq=2584 ttl=64 time=1.34 ms
Mar 25 00:08:40 64 bytes from 10.11.10.12: icmp_seq=2585 ttl=64 time=0.275 ms
Mar 25 00:08:41 64 bytes from 10.11.10.12: icmp_seq=2586 ttl=64 time=0.256 ms
Mar 25 00:08:42 64 bytes from 10.11.10.12: icmp_seq=2587 ttl=64 time=0.263 ms
Mar 25 00:08:43 64 bytes from 10.11.10.12: icmp_seq=2588 ttl=64 time=0.302 ms
Mar 25 00:08:44 64 bytes from 10.11.10.12: icmp_seq=2589 ttl=64 time=0.394 ms
Mar 25 00:08:45 64 bytes from 10.11.10.12: icmp_seq=2590 ttl=64 time=0.355 ms
Mar 25 00:08:46 64 bytes from 10.11.10.12: icmp_seq=2591 ttl=64 time=0.293 ms
Mar 25 00:08:47 64 bytes from 10.11.10.12: icmp_seq=2592 ttl=64 time=0.384 ms
Mar 25 00:08:48 64 bytes from 10.11.10.12: icmp_seq=2593 ttl=64 time=0.319 ms
Mar 25 00:08:49 64 bytes from 10.11.10.12: icmp_seq=2594 ttl=64 time=0.228 ms
Mar 25 00:08:50 64 bytes from 10.11.10.12: icmp_seq=2595 ttl=64 time=0.316 ms
Mar 25 00:08:51 64 bytes from 10.11.10.12: icmp_seq=2596 ttl=64 time=0.281 ms
Mar 25 00:08:52 64 bytes from 10.11.10.12: icmp_seq=2597 ttl=64 time=0.301 ms
Mar 25 00:08:53 64 bytes from 10.11.10.12: icmp_seq=2598 ttl=64 time=0.283 ms
Mar 25 00:08:54 64 bytes from 10.11.10.12: icmp_seq=2599 ttl=64 time=0.232 ms
Mar 25 00:08:55 64 bytes from 10.11.10.12: icmp_seq=2600 ttl=64 time=0.349 ms
Mar 25 00:08:56 64 bytes from 10.11.10.12: icmp_seq=2601 ttl=64 time=0.259 ms
Mar 25 00:08:57 64 bytes from 10.11.10.12: icmp_seq=2602 ttl=64 time=0.297 ms
Mar 25 00:08:58 64 bytes from 10.11.10.12: icmp_seq=2603 ttl=64 time=0.257 ms
Mar 25 00:08:59 64 bytes from 10.11.10.12: icmp_seq=2604 ttl=64 time=0.242 ms
Mar 25 00:09:00 64 bytes from 10.11.10.12: icmp_seq=2605 ttl=64 time=0.332 ms
Mar 25 00:09:01 64 bytes from 10.11.10.12: icmp_seq=2606 ttl=64 time=0.369 ms
Mar 25 00:09:02 64 bytes from 10.11.10.12: icmp_seq=2607 ttl=64 time=0.276 ms
Mar 25 00:09:03 64 bytes from 10.11.10.12: icmp_seq=2608 ttl=64 time=0.370 ms
Mar 25 00:09:04 64 bytes from 10.11.10.12: icmp_seq=2609 ttl=64 time=0.301 ms
Mar 25 00:09:05 64 bytes from 10.11.10.12: icmp_seq=2610 ttl=64 time=0.392 ms
Mar 25 00:09:06 64 bytes from 10.11.10.12: icmp_seq=2611 ttl=64 time=0.313 ms
Mar 25 00:09:07 64 bytes from 10.11.10.12: icmp_seq=2612 ttl=64 time=0.679 ms

but sometimes they don't contain the unreachability lines:

Mar 24 23:58:31 64 bytes from 10.11.10.12: icmp_seq=1977 ttl=64 time=0.244 ms
Mar 24 23:58:32 64 bytes from 10.11.10.12: icmp_seq=1978 ttl=64 time=0.282 ms
Mar 24 23:58:33 64 bytes from 10.11.10.12: icmp_seq=1979 ttl=64 time=0.269 ms
Mar 24 23:58:34 64 bytes from 10.11.10.12: icmp_seq=1980 ttl=64 time=0.342 ms
Mar 24 23:58:35 64 bytes from 10.11.10.12: icmp_seq=1981 ttl=64 time=0.261 ms
Mar 24 23:58:36 64 bytes from 10.11.10.12: icmp_seq=1982 ttl=64 time=0.292 ms
Mar 24 23:58:37 64 bytes from 10.11.10.12: icmp_seq=1983 ttl=64 time=0.274 ms
Mar 24 23:58:38 64 bytes from 10.11.10.12: icmp_seq=1984 ttl=64 time=0.394 ms
Mar 24 23:58:39 64 bytes from 10.11.10.12: icmp_seq=1985 ttl=64 time=0.338 ms
Mar 24 23:58:40 64 bytes from 10.11.10.12: icmp_seq=1986 ttl=64 time=0.304 ms
Mar 24 23:58:41 64 bytes from 10.11.10.12: icmp_seq=1987 ttl=64 time=0.338 ms
Mar 24 23:58:42 64 bytes from 10.11.10.12: icmp_seq=1988 ttl=64 time=0.302 ms
Mar 24 23:58:43 64 bytes from 10.11.10.12: icmp_seq=1989 ttl=64 time=0.302 ms
Mar 24 23:58:44 64 bytes from 10.11.10.12: icmp_seq=1990 ttl=64 time=0.328 ms
Mar 24 23:58:45 64 bytes from 10.11.10.12: icmp_seq=1991 ttl=64 time=0.220 ms
Mar 24 23:59:24 64 bytes from 10.11.10.12: icmp_seq=2030 ttl=64 time=1.00 ms
Mar 24 23:59:25 64 bytes from 10.11.10.12: icmp_seq=2031 ttl=64 time=0.333 ms
Mar 24 23:59:26 64 bytes from 10.11.10.12: icmp_seq=2032 ttl=64 time=0.279 ms
Mar 24 23:59:27 64 bytes from 10.11.10.12: icmp_seq=2033 ttl=64 time=0.272 ms
Mar 24 23:59:28 64 bytes from 10.11.10.12: icmp_seq=2034 ttl=64 time=0.282 ms
Mar 24 23:59:29 64 bytes from 10.11.10.12: icmp_seq=2035 ttl=64 time=0.430 ms
Mar 24 23:59:30 64 bytes from 10.11.10.12: icmp_seq=2036 ttl=64 time=0.364 ms
Mar 24 23:59:31 64 bytes from 10.11.10.12: icmp_seq=2037 ttl=64 time=0.381 ms
Mar 24 23:59:32 64 bytes from 10.11.10.12: icmp_seq=2038 ttl=64 time=0.414 ms
Mar 24 23:59:33 64 bytes from 10.11.10.12: icmp_seq=2039 ttl=64 time=0.330 ms
Mar 24 23:59:34 64 bytes from 10.11.10.12: icmp_seq=2040 ttl=64 time=0.354 ms
Mar 24 23:59:35 64 bytes from 10.11.10.12: icmp_seq=2041 ttl=64 time=0.301 ms
Mar 24 23:59:36 64 bytes from 10.11.10.12: icmp_seq=2042 ttl=64 time=0.305 ms
Mar 24 23:59:37 64 bytes from 10.11.10.12: icmp_seq=2043 ttl=64 time=0.274 ms
Mar 24 23:59:38 64 bytes from 10.11.10.12: icmp_seq=2044 ttl=64 time=0.404 ms
Mar 24 23:59:39 64 bytes from 10.11.10.12: icmp_seq=2045 ttl=64 time=0.377 ms
Mar 24 23:59:40 64 bytes from 10.11.10.12: icmp_seq=2046 ttl=64 time=0.392 ms
Mar 24 23:59:41 64 bytes from 10.11.10.12: icmp_seq=2047 ttl=64 time=0.313 ms
Mar 24 23:59:42 64 bytes from 10.11.10.12: icmp_seq=2048 ttl=64 time=0.336 ms
Mar 24 23:59:43 64 bytes from 10.11.10.12: icmp_seq=2049 ttl=64 time=0.313 ms
Mar 24 23:59:44 64 bytes from 10.11.10.12: icmp_seq=2050 ttl=64 time=0.311 ms
Mar 24 23:59:45 64 bytes from 10.11.10.12: icmp_seq=2051 ttl=64 time=0.223 ms
Mar 24 23:59:46 64 bytes from 10.11.10.12: icmp_seq=2052 ttl=64 time=0.226 ms
Mar 24 23:59:47 64 bytes from 10.11.10.12: icmp_seq=2053 ttl=64 time=0.241 ms
Mar 24 23:59:48 64 bytes from 10.11.10.12: icmp_seq=2054 ttl=64 time=0.389 ms
Mar 24 23:59:49 64 bytes from 10.11.10.12: icmp_seq=2055 ttl=64 time=0.213 ms
Mar 24 23:59:50 64 bytes from 10.11.10.12: icmp_seq=2056 ttl=64 time=0.233 ms
Mar 24 23:59:51 64 bytes from 10.11.10.12: icmp_seq=2057 ttl=64 time=0.553 ms
Mar 24 23:59:52 64 bytes from 10.11.10.12: icmp_seq=2058 ttl=64 time=0.309 ms
Mar 24 23:59:53 64 bytes from 10.11.10.12: icmp_seq=2059 ttl=64 time=0.273 ms
Mar 24 23:59:54 64 bytes from 10.11.10.12: icmp_seq=2060 ttl=64 time=0.453 ms
Mar 24 23:59:55 64 bytes from 10.11.10.12: icmp_seq=2061 ttl=64 time=0.208 ms
Mar 24 23:59:56 64 bytes from 10.11.10.12: icmp_seq=2062 ttl=64 time=0.314 ms
Mar 24 23:59:57 64 bytes from 10.11.10.12: icmp_seq=2063 ttl=64 time=0.267 ms
Mar 24 23:59:58 64 bytes from 10.11.10.12: icmp_seq=2064 ttl=64 time=0.384 ms
Mar 24 23:59:59 64 bytes from 10.11.10.12: icmp_seq=2065 ttl=64 time=0.302 ms
Mar 25 00:00:00 64 bytes from 10.11.10.12: icmp_seq=2066 ttl=64 time=0.320 ms
Mar 25 00:00:01 64 bytes from 10.11.10.12: icmp_seq=2067 ttl=64 time=0.292 ms
Mar 25 00:00:02 64 bytes from 10.11.10.12: icmp_seq=2068 ttl=64 time=0.274 ms
Mar 25 00:00:03 64 bytes from 10.11.10.12: icmp_seq=2069 ttl=64 time=0.192 ms
Mar 25 00:00:04 64 bytes from 10.11.10.12: icmp_seq=2070 ttl=64 time=0.329 ms
Mar 25 00:00:05 64 bytes from 10.11.10.12: icmp_seq=2071 ttl=64 time=0.210 ms
Mar 25 00:00:06 64 bytes from 10.11.10.12: icmp_seq=2072 ttl=64 time=0.199 ms
Mar 25 00:00:07 64 bytes from 10.11.10.12: icmp_seq=2073 ttl=64 time=0.197 ms
Mar 25 00:00:08 64 bytes from 10.11.10.12: icmp_seq=2074 ttl=64 time=0.276 ms
Mar 25 00:00:09 64 bytes from 10.11.10.12: icmp_seq=2075 ttl=64 time=0.189 ms
Mar 25 00:00:10 64 bytes from 10.11.10.12: icmp_seq=2076 ttl=64 time=0.258 ms
Mar 25 00:00:11 64 bytes from 10.11.10.12: icmp_seq=2077 ttl=64 time=0.392 ms
Mar 25 00:00:12 64 bytes from 10.11.10.12: icmp_seq=2078 ttl=64 time=0.238 ms
Mar 25 00:00:13 64 bytes from 10.11.10.12: icmp_seq=2079 ttl=64 time=0.178 ms
Mar 25 00:00:14 64 bytes from 10.11.10.12: icmp_seq=2080 ttl=64 time=0.183 ms
Mar 25 00:00:15 64 bytes from 10.11.10.12: icmp_seq=2081 ttl=64 time=0.237 ms
Mar 25 00:00:16 64 bytes from 10.11.10.12: icmp_seq=2082 ttl=64 time=0.370 ms
Mar 25 00:00:17 64 bytes from 10.11.10.12: icmp_seq=2083 ttl=64 time=0.211 ms
Mar 25 00:00:18 64 bytes from 10.11.10.12: icmp_seq=2084 ttl=64 time=0.354 ms
Mar 25 00:00:19 64 bytes from 10.11.10.12: icmp_seq=2085 ttl=64 time=0.181 ms
Mar 25 00:00:20 64 bytes from 10.11.10.12: icmp_seq=2086 ttl=64 time=0.306 ms
Mar 25 00:00:21 64 bytes from 10.11.10.12: icmp_seq=2087 ttl=64 time=0.217 ms
Mar 25 00:00:22 64 bytes from 10.11.10.12: icmp_seq=2088 ttl=64 time=0.417 ms
Mar 25 00:00:23 64 bytes from 10.11.10.12: icmp_seq=2089 ttl=64 time=0.293 ms
Mar 25 00:00:24 64 bytes from 10.11.10.12: icmp_seq=2090 ttl=64 time=0.313 ms
Mar 25 00:00:25 64 bytes from 10.11.10.12: icmp_seq=2091 ttl=64 time=0.222 ms
Mar 25 00:00:26 64 bytes from 10.11.10.12: icmp_seq=2092 ttl=64 time=0.404 ms
Mar 25 00:00:27 64 bytes from 10.11.10.12: icmp_seq=2093 ttl=64 time=0.293 ms
Mar 25 00:00:28 64 bytes from 10.11.10.12: icmp_seq=2094 ttl=64 time=0.396 ms

but there is a gap between two entries which also means the VM was unreachable:

Mar 24 23:58:45 64 bytes from 10.11.10.12: icmp_seq=1991 ttl=64 time=0.220 ms
Mar 24 23:59:24 64 bytes from 10.11.10.12: icmp_seq=2030 ttl=64 time=1.00 ms

This is what I have come up with:

import csv,sys
from datetime import datetime as dt

def main(logfile):
        gap=0
        down=False
        first=True
        st_et=[]
        dt_pairs=[]
        lt=-1
        with open(logfile,'r') as lf:
                for line in lf:
                        tokens=line.split()
                        t=dt.strptime(' '.join(tokens[0:3]), '%b %d %H:%M:%S')
                        if first:
                                first=False
                                st_et.append(t)
                        else:
                                diff=(t-lt).total_seconds()
                                if diff>4:
                                        gap+=diff
                                        print t,lt, gap
                        if "Destination" in line and not down:
                                #record downtime
                                down=True
                                dt_pairs.append(t)
                        if "bytes" in line and down:
                                # ping is back up
                                down=False
                                dt_pairs.append(lt)
                        lt=t

                st_et.append(lt)

#       print st_et
        print dt_pairs
        downtime=0
        for i in range(0,len(dt_pairs),2):
                downtime+=(dt_pairs[i+1]-dt_pairs[i]).total_seconds()
        print "Downtime: %d (unreachable) + %d (gap) = %d"%(downtime,gap,downtime+gap)

main(sys.argv[1])

It does work but I haven't put much thought into it and I arbitrarily chose 4 as the lower bound for the gap. I am wondering if there is an easier/ more efficient / more robust solution for doing this?

\$\endgroup\$
2
\$\begingroup\$

This looks like a useful code. I'm going to keep this bookmarked in case I need it.


Review

import csv,sys
  • You are not using csv get rid of that import.
from datetime import datetime as dt
  • Why are you shortening datetime to dt? Not really needed. Even if you are using vim to write code you can use autocomplete.
st_et=[]
  • This is not really used?
print dt_pairs
  • Get rid of debug prints.

Other things I noticed

  • Make it runnable.
  • Follow PEP8.
  • You can easily make this Python 3 compatible.
  • Lot of magic numbers.

After changes are applied.

#!/usr/bin/env python

"""
Identify downtime from ping log
"""

from __future__ import print_function

import sys
from datetime import datetime

THRESHOLD_SECONDS = 4
ERROR_EXIT = -1
PING_LOG_MONTH_COL = 0
PING_LOG_BYTES_COL = 3


def main():
    if len(sys.argv) != 2:
        print("Usage: {0} log.txt".format(__file__))
        sys.exit(ERROR_EXIT)

    display_downtime(sys.argv[1])


def display_downtime(logfile):
    gap = 0
    down = False
    first = True
    unreachable_time_pairs = []
    previous_timestamp = None

    with open(logfile, "r") as logfile_handle:
        for line in logfile_handle:

            tokens = line.split()
            timestamp = datetime.strptime(
                " ".join(tokens[PING_LOG_MONTH_COL:PING_LOG_BYTES_COL]),
                "%b %d %H:%M:%S",
            )

            if first:
                first = False
            else:
                diff = (timestamp - previous_timestamp).total_seconds()
                if diff > THRESHOLD_SECONDS:
                    gap += diff

            if "Destination" in line and not down:
                down = True
                unreachable_time_pairs.append(timestamp)
            if "bytes" in line and down:
                # ping is back up, so add previous time for unreachable_time_pairs
                down = False
                unreachable_time_pairs.append(previous_timestamp)

            previous_timestamp = timestamp

    unreachable = 0
    for i in range(0, len(unreachable_time_pairs), 2):
        unreachable += (
            unreachable_time_pairs[i + 1] - unreachable_time_pairs[i]
        ).total_seconds()

    print(
        "Downtime: {0}s (unreachable) + {1}s (gap) = {2}s".format(
            unreachable, gap, unreachable + gap
        )
    )


if __name__ == "__main__":
    main()

Output

~ python3 unreachability.py logfile.txt
Downtime: 31.0s (unreachable) + 444.0s (gap) = 475.0s
~ python2 unreachability.py logfile.txt
Downtime: 31.0s (unreachable) + 444.0s (gap) = 475.0s
~ ./unreachability.py logfile.txt
Downtime: 31.0s (unreachable) + 444.0s (gap) = 475.0s
~ ./unreachability.py logfile.txt a b c
Usage: ./unreachability.py log.txt
\$\endgroup\$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.