Return to Answer

However since you're inheriting from SitemapSpider you also need to run its __init__ function in yours. You need to call it so that your base class is initialised before you run your particular __init__ code. There's a good explanation in this Stack Overflow answer

However since you're inheriting from SitemapSpider you also need to run its __init__ function in yours. You need to call it so that your base class is initialised before you run your particular __init__ code. There's a good explanation in this Stack Overflow answer

class OxygenSpider(SitemapSpider):

    def __init__(self):
        print 'MY SPIDER, IS ALIVE'
        self.NAME = "oxygen"
        self.ALLOWED_DOMAINS = ("oxygenboutique.com")
        self.SITEMAP_URLS = ('http://www.oxygenboutique.com/sitemap.xml')
        self.sitemap_rules = generate_sitemap_rules()
        self.ex_rates = get_exchange_rates()

However since you're inheriting from SitemapSpider you also need to run its __init__ function in yours. You need to call it so that your base class is initialised before you run your particular __init__ code. There's a good explanation in this Stack Overflow answer

class OxygenSpider(SitemapSpider):

    def __init__(self):
        super(SitemapSpider, self).__init__()
        print 'MY SPIDER, IS ALIVE'
        self.NAME = "oxygen"
        self.ALLOWED_DOMAINS = ("oxygenboutique.com")
        self.SITEMAP_URLS = ('http://www.oxygenboutique.com/sitemap.xml')
        self.sitemap_rules = generate_sitemap_rules()
        self.ex_rates = get_exchange_rates()

Well to start with you have bad practices in your imports. It's recommended to stay away from using from module import * because doing that imports things without explicitly declaring their names. Without realising it, you could be overwriting other functions, including builtins in the module was made carelessly. Instead use just import module or from module import func1, func2, CONST. Especially though, don't do this:

import oxygendemo.utilities
from oxygendemo.utilities import *

It's totally redundant to have the first line since you're then ignoring it to import everything. In case you don't know, you can still alias plain imports:

import oxygendemo.utilities as util

So you don't even need to worry about the name being too long.

Also OxygenSpider is not laid out properly. You have loose code that should probably be in an __init__ function. Let me show you how this works in the interpreter:

>>> class A:
    print "Printing class A"

    
Printing class A

So what happened there? The print command was run when the class was created. I haven't created any object yet, so what happens when I create an object:

>>> A()
<__main__.A instance at 0x0000000002CA5588>
>>> b = A()
>>> 

Nothing. It's not printing the command that you intended to appear when creating an OxygenSpider object. If you were to wrap it in __init__ though, it would. __init__ is a special function that runs when a new object is created, like so:

>>> class A:
    def __init__(self):
        print "Printing this object"

>>> A()
Printing this object
<__main__.A instance at 0x0000000002113488>
>>> b = A()
Printing this object

You see now? Nothing happens after the class is created but when actual objects are created __init__ gets run. You should be putting the whole opening block to OxygenSpider in a function like that. Also the variables should be assigned as self.var, and the constants should be in UPPER_SNAKE_CASE and constant lists should be tuples instead. Tuples are made with () and are basically like lists except they cannot be changed.

class OxygenSpider(SitemapSpider):

    def __init__(self):
        print 'MY SPIDER, IS ALIVE'
        self.NAME = "oxygen"
        ALLOWED_DOMAINS = ("oxygenboutique.com")
        SITEMAP_URLS = ('http://www.oxygenboutique.com/sitemap.xml')
        sitemap_rules = generate_sitemap_rules()
        ex_rates = get_exchange_rates()

Also printing when creating an object just to say it's created isn't very nice anyway, you should remove that.

Well to start with you have bad practices in your imports. It's recommended to stay away from using from module import * because doing that imports things without explicitly declaring their names. Without realising it, you could be overwriting other functions, including builtins in the module was made carelessly. Instead use just import module or from module import func1, func2, CONST. Especially though, don't do this:

import oxygendemo.utilities
from oxygendemo.utilities import *

It's totally redundant to have the first line since you're then ignoring it to import everything. In case you don't know, you can still alias plain imports:

import oxygendemo.utilities as util

So you don't even need to worry about the name being too long.

Also OxygenSpider is not laid out properly. You have loose code that should probably be in an __init__ function. Let me show you how this works in the interpreter:

>>> class A:
    print "Printing class A"

    
Printing class A

So what happened there? The print command was run when the class was created. I haven't created any object yet, so what happens when I create an object:

>>> A()
<__main__.A instance at 0x0000000002CA5588>
>>> b = A()
>>> 

Nothing. It's not printing the command that you intended to appear when creating an OxygenSpider object. If you were to wrap it in __init__ though, it would. __init__ is a special function that runs when a new object is created, like so:

>>> class A:
    def __init__(self):
        print "Printing this object"

>>> A()
Printing this object
<__main__.A instance at 0x0000000002113488>
>>> b = A()
Printing this object

You see now? Nothing happens after the class is created but when actual objects are created __init__ gets run. You should be putting the whole opening block to OxygenSpider in a function like that. Also the variables should be assigned as self.var, and the constants should be in UPPER_SNAKE_CASE and constant lists should be tuples instead. Tuples are made with () and are basically like lists except they cannot be changed.

class OxygenSpider(SitemapSpider):

    def __init__(self):
        print 'MY SPIDER, IS ALIVE'
        self.NAME = "oxygen"
        self.ALLOWED_DOMAINS = ("oxygenboutique.com")
        self.SITEMAP_URLS = ('http://www.oxygenboutique.com/sitemap.xml')
        self.sitemap_rules = generate_sitemap_rules()
        self.ex_rates = get_exchange_rates()

Also printing when creating an object just to say it's created isn't very nice anyway, you should remove that.