Following is the code I wrote to download the information of different items in a page.

I have one main website which has links to different items. I parse this main page to get the list. This is handled by the Items class.

I also parse these each of the links in the list using the Item class.

I have implemented a Handler class which is the base class for both of these classes.

class Handler:

    def __init__(self, url):
        self.url = url
        self.property = {}
        self.homeDir = os.path.dirname(__file__)
        self.parser = self.getParser()
        self.name = self.getTitle()

    def updateName(self, name):
        self.name = name

    def setupFolder(self):
        dataDir = os.path.join(self.homeDir, self.name)
        if not os.path.exists(dataDir):

    def getTitle(self):
        return "".join(char 
            for char in self.parser.title.string 
            if char.isalnum() or char == " ")

    def getFilePath(self):
        return os.path.join(self.homeDir, self.name)

    def getRequest(self, url):
        return urllib.urlopen(url).read()

    def getParser(self):
        parser = BeautifulSoup(self.getRequest(self.url))
        return parser

    def saveProperty(self, key, value):
        self.property[key] = value

    def writeProperty(self):
        fileName = self.getTitle() + ".property"
        with open(fileName, 'w') as f:
                key + ":" + self.property[key] 
                for key in self.property))

class Items(Handler, object):

    def __init__(self, url, category) :

        super(Items, self).__init__(url)
        self.category = category

    def extractContents(self):
        self.parser = self.getParser()
        contents = self.parser.find("ul",{"class" : "galerie"}).findAll('li')
        print len(contents)
        return contents

    def downloadContents(self):
        for content in  self.extractContents():
            for elements in content.find('a'):
                for element in elements.findAll('dl'):
                    print element.text # I am just printing the contents
            print "==============================="

class Item(Handler, object):

    def __init__(self, url):

        super(Item, self).__init__(url)
        self.url = url
        self.homeDir = os.path.dirname(__file__)
        self.parser = self.getParser()
        self.name = self.getTitle()

    def write(self):

I'd appreciate any and all comments - correctness, style, best practices, logging, error handling.

  • \$\begingroup\$ As far as styling is concerned, I would suggest you go through. google-styleguide.googlecode.com/svn/trunk/…. function_name are usually snake cased in python. Then maybe it's a bit Java-ish with get and set methods. For the rest I which this would be in github to have that cool feature of commenting directly in the code. \$\endgroup\$ – jackdbernier Nov 13 '14 at 7:59
  • \$\begingroup\$ @jackdbernier Python has an official style guide, PEP8. \$\endgroup\$ – Ethan Bierlein Jul 7 '15 at 0:46
  • \$\begingroup\$ Thanks @EthanBierlein. I know about PEP8 and use it. Most of (if not all) the content on google-styleguide for python is from PEP8. The only reason I usually fallback to it is because they have stye guides for almost every languages. Also google-styleguide involve a lot of good common sense and is according to me less strict. Because really who likes to be PEP8'ed in a code review? And the very first python rule of the google-styleguide is to use pylint. Because code reviews need to go beyond PEP8 and let the style be judged by a machine. \$\endgroup\$ – jackdbernier Jul 8 '15 at 17:05

The general style for naming in Python is snake_case for functions and variables, and PascalCase for classes. You should also have two blank lines between top-level functions/classes/code blocks, not, an arbitrary amount. You have a few other style violations. To fix these, visit PEP8, Python's official style guide.

Secondly, it looks like you're using Python 2.x. If you're using Python 2.x, you need to have classes explicitly inherit from object. For example: class MyClass(object):, not class MyClass:. If you are using Python 3.x, then you can use the second example.

Finally, you can use string multiplication to print many characters. For example, the line print "===============================" can be shortened to `print "=" * 31.

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.