2
\$\begingroup\$

Here is my code;

import socket
def read_line(sock):
    "read a line from a socket"
    chars = []
    while True:
        a = sock.recv(1)
        chars.append(a)     
        if a == "\n" or a == "":
            return "".join(chars)

def read_headers(sock):
    "read HTTP headers from an internet socket"
    lines = []
    while True:
        line = read_line(sock)
        if line == "\n" or line == "\r\n" or line == "":
            return "".join(lines)
        lines.append(line)

def parse_headers(headerstr):
    "Return a dict, corresponding each HTTP header"
    headers = {}
    for line in headerstr.splitlines():
        k, _, v = line.partition(":")
        headers[k] = v.strip()
    return headers

http_request = """GET / HTTP/1.1
Host: {}

"""

site = "www.google.com.tr"
conn = socket.create_connection((site,80))
conn.sendall(http_request.format(site))

headersstr = read_headers(conn)
headersdict = parse_headers(headersstr)

if "Transfer-Encoding" in headersdict and headersdict["Transfer-Encoding"] == "chunked":
    content = ""
else:
    content = conn.recv(headersdict["Content-Length"])

print content
conn.close()

Can you please comment on how I am using sockets. Is it correct practive to read characters one by one from socket as I did in read_line function? Are there any other mistakes I am doing?

\$\endgroup\$
3
  • \$\begingroup\$ Code Review strictly improves working code, we aren't here to help debug your code.You can ask on Stackoverflow, but you should narrow down your question first. \$\endgroup\$ Commented Aug 25, 2012 at 15:50
  • \$\begingroup\$ @WinstonEwert I edited my question and stripped buggy part. Is it ok now? If so, can you reopen? \$\endgroup\$
    – yasar
    Commented Aug 25, 2012 at 17:50
  • \$\begingroup\$ reopened your question \$\endgroup\$ Commented Aug 25, 2012 at 19:35

1 Answer 1

3
\$\begingroup\$

Reading one byte at a time is bad idea, because it won't buffer the input. Instead, you'll probably get a system call for each byte which will be really inefficient.

Easiest solution is to use makefile

myfile = sock.makefile('r') 

Then you can treat myfile like a file, including using methods like readline. That'll be more efficient then your version.

It seems a bit of waste to have readheaders() convert all the headers into one long string, only to have parseheaders() split it apart again.

if "Transfer-Encoding" in headersdict and headersdict["Transfer-Encoding"] == "chunked":

I'd use if headersdict.get('Transfer-Encoding') == 'chunked':, if the key is not present, get will return None, and you'll have the same effect as before.

Of course, for serious stuff, you'd want to use the builtin python libraries for this sort of thing. But as long as you are doing it to learn, all well and good.

\$\endgroup\$
2
  • \$\begingroup\$ Thanks for the answer. Does it matter when I use makefile. Should I use it after or before I make HTTP request? \$\endgroup\$
    – yasar
    Commented Aug 25, 2012 at 21:01
  • \$\begingroup\$ @yasar11732, shouldn't matter. \$\endgroup\$ Commented Aug 25, 2012 at 21:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.