Python internet speed testing

Question

I understand there are a few similar tools available for testing speed but that's not what I'm looking for. I'm looking at seeing how long it takes to request a site's HTML. However, I'm getting slightly odd results.

The strange results are incredibly slow internet speeds so I'm confused about whether I'm doing something wrong or something isn't working.

It's reading at roughly 30 kilobytes a second which is quite odd.

Important Info

• OS X (would like it to work the same on Windows as well)
• Running using Sublime build, have also tried Python Launcher, but appears to be the same.

---

def getInternetSpeed():
    path = sys.argv[0]
    path = list(path)
    while True: # Just cuts back from say C:\User\Example\File.txt to
        if path[len(path)-1] == chr(92) or path[len(path)-1] == '/':
            path = ('').join(path)
            break
        else:
            path.pop(len(path)-1)
    testInternet();
    start = str(datetime.datetime.now())
    x = urllib2.urlopen('https://google.com')
    end = str(datetime.datetime.now())
    x = x.read()
    start = list(start)
    end = list(end)
    for i in range(17):
        start.pop(0)
        end.pop(0)
    start = ('').join(start)
    end = ('').join(end)
    start = float(start)
    end = float(end)
    diff = end - start
    f = open(path+'TEMP.txt','w')
    f.write(x)
    f.close()
    size = os.path.getsize(path+'TEMP.txt')
    os.remove(path+'TEMP.txt')
    print 'It took ' + str(diff) + ' second(s)'
    print 'to get a file that was ' + str(size / 1000) + ' kilobytes large!'

    scale = 1 / diff
    size = size * scale
    size = size / 1000

    print 'Download speed of ' + str(int(size)) + ' kilobytes a second'

Answer 1

Concept

Internet "speed" actually consists of multiple factors, including bandwidth, latency, and packet loss. What exactly are you trying to measure? If it's bandwidth (data transfer speed in bits per second), then you should also measure transfers of larger payloads, because the time to complete small transfers will tend to be dominated by latency effects. If it's latency (milliseconds for a round trip) that you are want to measure, then a tiny packet (typically an ICMP echo request, commonly called a "ping") would be more appropriate.

In addition, there are confounding issues:

• Upload vs. download speed: Residential Internet connections tend to significantly favour downloads, since that's what most consumers care about.
• Traffic shaping: Some ISPs will throttle the bandwidth to penalize very large transfers, so that smaller responses feel more responsive.
• Server performance: It does take a few milliseconds for Google's server to formulate the response payload; that delay would be a more significant fraction of the measured round-trip time for a short response than for a long response.
• HTTP and HTTPS overhead: You're considering only the size of the HTTP response payload. Since you're using HTTPS, there are also multiple TCP round trips needed to perform the TLS handshake, and a few kilobytes to transfer the certificates. The HTTP headers could add another kilobyte or so. All of this is significant when the payload is only ~10 kB short.
• DNS overhead: Resolving the google.com hostname to an IP address might involve a DNS lookup, which would add a few milliseconds. The time for that DNS lookup is accounted for in your benchmark, but not the traffic.
• Proximity: The "Internet" is not one entity. Connectivity to various parts of the Internet will have different performance characteristics.

Time calculation bug

When you do str(datetime.datetime.now()), you are converting the timestamp from a datetime object that is suitable for arithmetic into a string, which is a horrible format for calculations. You then try to salvage the information by dropping the first 17 characters of the string and converting the seconds portion back into a float.

What happens if the start and end times straddle a minute boundary? You'll get an elapsed time of approximately -60 seconds!

A better way would be:

start = datetime.datetime.now()
…     # Do some work here…
end = datetime.datetime.now()
diff = (end - start).total_seconds()

Inefficiencies

The code expects a path as sys.argv[0]. What is it for? It seems to be used only to derive a directory to use for a temporary file, and the temporary file is used just to determine the length of the response payload.

All you had to do to obtain the length was len(urllib2.urlopen('https://google.com')). No filesystem access is needed!

Lint

• By PEP 8, getInternetSpeed() should be get_internet_speed(). The function should probably accept a parameter for the URL to test. Furthermore, if the function is named "getSomething", I would expect it to return a value rather than print its result.
• For filename manipulation, use the os.path module.

• Avoid redefining variables, especially in a way that changes their type:

  x = urllib2.urlopen('https://google.com')
  …
  x = x.read()
  

  x is a very poor variable name, by the way. I suggest req.

• open() should almost always be used in the context of a with block, so that the closure will be handled correctly for you:

   with open(os.path.join(temp_dir, 'TEMP.txt'), 'w') as f:
       f.write(x)
  

• To generate temporary files, use the tempfile library to guarantee that you will not overwrite an existing file.