6
\$\begingroup\$

I have a simple script that will take a list of hosts and ping each host (there's about 200) a single time before moving on. This is not the most effecient method I am sure, as it is very linear. And it takes a few minutes to complete. I would ideally like to run this script every minute (so each IP address is checked every minute (the actual running of the script is controlled externally).

I was looking for some pointers on potentially making the script more effecient/dynamic.

import sys
import os
import platform
import subprocess

plat = platform.system()
scriptDir = sys.path[0]
hosts = os.path.join(scriptDir, 'hosts.txt')
hostsFile = open(hosts, "r")
lines = hostsFile.readlines()
if plat == "Windows":
    for line in lines:
        line = line.strip( )
        ping = subprocess.Popen(
            ["ping", "-n", "1", "-l", "1", "-w", "100", line],
            stdout = subprocess.PIPE,
            stderr = subprocess.PIPE
        )
        out, error = ping.communicate()
        print out
        print error

if plat == "Linux":
    for line in lines:
        line = line.strip( )
        ping = subprocess.Popen(
            ["ping", "-c", "1", "-l", "1", "-s", "1", "-W", "1", line],
            stdout = subprocess.PIPE,
            stderr = subprocess.PIPE
        )
        out, error = ping.communicate()
        print out
        print error

hostsFile.close()
\$\endgroup\$

migrated from stackoverflow.com Jul 15 '12 at 13:12

This question came from our site for professional and enthusiast programmers.

  • \$\begingroup\$ I think it may be the way I'm reading a file. \$\endgroup\$ – MHibbin Jul 15 '12 at 11:41
  • \$\begingroup\$ Might want to take a look at Nagios's scheduler code to get some ideas. \$\endgroup\$ – Kev Jul 15 '12 at 13:14
5
\$\begingroup\$

Here's a solution using threads:

import sys
import os
import platform
import subprocess
import Queue
import threading

def worker_func(pingArgs, pending, done):
    try:
        while True:
            # Get the next address to ping.
            address = pending.get_nowait()

            ping = subprocess.Popen(ping_args + [address],
                stdout = subprocess.PIPE,
                stderr = subprocess.PIPE
            )
            out, error = ping.communicate()

            # Output the result to the 'done' queue.
            done.put((out, error))
    except Queue.Empty:
        # No more addresses.
        pass
    finally:
        # Tell the main thread that a worker is about to terminate.
        done.put(None)

# The number of workers.
NUM_WORKERS = 4

plat = platform.system()
scriptDir = sys.path[0]
hosts = os.path.join(scriptDir, 'hosts.txt')

# The arguments for the 'ping', excluding the address.
if plat == "Windows":
    pingArgs = ["ping", "-n", "1", "-l", "1", "-w", "100"]
elif plat == "Linux":
    pingArgs = ["ping", "-c", "1", "-l", "1", "-s", "1", "-W", "1"]
else:
    raise ValueError("Unknown platform")

# The queue of addresses to ping.
pending = Queue.Queue()

# The queue of results.
done = Queue.Queue()

# Create all the workers.
workers = []
for _ in range(NUM_WORKERS):
    workers.append(threading.Thread(target=worker_func, args=(pingArgs, pending, done)))

# Put all the addresses into the 'pending' queue.
with open(hosts, "r") as hostsFile:
    for line in hostsFile:
        pending.put(line.strip())

# Start all the workers.
for w in workers:
    w.daemon = True
    w.start()

# Print out the results as they arrive.
numTerminated = 0
while numTerminated < NUM_WORKERS:
    result = done.get()
    if result is None:
        # A worker is about to terminate.
        numTerminated += 1
    else:
        print result[0] # out
        print result[1] # error

# Wait for all the workers to terminate.
for w in workers:
    w.join()
\$\endgroup\$
  • \$\begingroup\$ sorry for the delayed response this part of my project has to take a massive "side-step". Couple of questions... NUM_WORKERS... I assume this is the amount of pings done in one cycle, for exaample you have "4" this would mean 4 hosts would be pinged in one go. Also am I right in assuming I would feed my ip addresses in as pending? \$\endgroup\$ – MHibbin Oct 2 '12 at 9:52
  • \$\begingroup\$ Each worker gets a host from the pending queue, sends a ping, and puts the reply in the done queue, repeating until the pending queue is empty. There are NUM_WORKERS workers, so there are at most concurrent pings. Note that it fills the pending queue before starting the workers. Another way of doing it is to replace "pending.get_nowait()" with "pending.get()" and put a sentinel such as None in the pending queue at the end to say that there'll be no more hosts; when a worker sees that sentinel, it puts it back into the pending queue for the other workers to see, and then breaks out of its loop. \$\endgroup\$ – MRAB Oct 2 '12 at 18:01
  • \$\begingroup\$ Ha! I didn't actually scroll down on the code! \$\endgroup\$ – MHibbin Oct 3 '12 at 7:42
  • \$\begingroup\$ This is amazing! How many threads is too many? Is there a any calculations that I can perform to (i.e. using CPU and memory, etc) work out what the optimal number of threads is? - I have tried a few, and got the script to finish the list in about 13 seconds (it was in minutes before). \$\endgroup\$ – MHibbin Oct 3 '12 at 8:40
  • \$\begingroup\$ The optimal number of threads depends on several factors, including the ping response time, so you'll just need to test it with differing numbers of threads. I expect that increasing the number of threads will decrease the run time, but with diminishing returns, so just pick a (non-excessive) number of threads which gives you a reasonable run time. \$\endgroup\$ – MRAB Oct 4 '12 at 13:46
4
\$\begingroup\$

It's not a complete answer, but your code would be more clear (and easy to maintain) if you won't duplicate code:

import sys
import os
import platform
import subprocess

plat = platform.system()
scriptDir = sys.path[0]
hosts = os.path.join(scriptDir, 'hosts.txt')
hostsFile = open(hosts, "r")
lines = hostsFile.readlines()
for line in lines:
    line = line.strip( )
    if plat == "Windows":
        args = ["ping", "-n", "1", "-l", "1", "-w", "100", line]

    elif plat == "Linux":
        args = ["ping", "-c", "1", "-l", "1", "-s", "1", "-W", "1", line]

    ping = subprocess.Popen(
        args,
        stdout = subprocess.PIPE,
        stderr = subprocess.PIPE
    )
    out, error = ping.communicate()
    print out
    print error

hostsFile.close()
\$\endgroup\$
0
\$\begingroup\$

Why not using threads ? You could run your pings simultaneously.

This works quite well :

import sys
import os
import platform
import subprocess
import threading

plat = platform.system()
scriptDir = sys.path[0]
hosts = os.path.join(scriptDir, 'hosts.txt')
hostsFile = open(hosts, "r")
lines = hostsFile.readlines()

def ping(ip):
    if plat == "Windows":
        ping = subprocess.Popen(
            ["ping", "-n", "1", "-l", "1", "-w", "100", ip],
            stdout = subprocess.PIPE,
            stderr = subprocess.PIPE
        )

    if plat == "Linux":
        ping = subprocess.Popen(
            ["ping", "-c", "1", "-l", "1", "-s", "1", "-W", "1", ip],
            stdout = subprocess.PIPE,
            stderr = subprocess.PIPE
        )

    out, error = ping.communicate()
    print out
    print error

for ip in lines:
    threading.Thread(target=ping, args=(ip,)).run()
\$\endgroup\$
  • \$\begingroup\$ When I run this, I get the following error... OSError: [Errno 24] Too many open files \$\endgroup\$ – MHibbin Jul 15 '12 at 12:07
  • \$\begingroup\$ seems like you exceeded your file-quota. either you raise your limit or limit the amount of threads. \$\endgroup\$ – Don Question Jul 15 '12 at 13:16
  • \$\begingroup\$ How would I do that then? \$\endgroup\$ – MHibbin Jul 15 '12 at 15:13
  • \$\begingroup\$ In the above code the file is being open, but never close that's why you are getting this error.. \$\endgroup\$ – Mak May 8 '17 at 9:57
0
\$\begingroup\$

i would look into the python-nmap package.

why reinvent the wheel if there is already one free to use? You will be much more productive and flexible this way.

to check for hosts active in an ipv4 network you would usually do a:

nmap -sP 192.168.100.0/24 

which basically does a ping and takes in most cicumstances 2-5 sec.

nmap is also avaiable for windows.

\$\endgroup\$
  • \$\begingroup\$ nmap is frowned upon in the network :/ . Plus we only want to check specific hosts not network ranges. \$\endgroup\$ – MHibbin Jul 15 '12 at 12:10
  • \$\begingroup\$ frowned? why? thats like a chirurg using a scalpel being frowned on not using a dull spoon. \$\endgroup\$ – Don Question Jul 15 '12 at 12:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.