# Best way of converting a dynamic number of list items into variables in Python [closed]

I have a simple scrapy spider that crawls a page and returns H1 on the pages. Since, each page is unique, one cannot know how many h1's will be on the page. Since the scrapy spider returns a list, I need to convert the list of lists into variables that I can then insert.

The output of H1 could look like

['some text h1','second h1', 'third h1']


I have a working code that looks like the following

    def _h1(self, page, response) :
if isinstance(response, HtmlResponse):
h1 = response.xpath("//h1/text()").getall()
length_h1 = (len(h1))
page['h1_count'] = length_h1
if length_h1 >= 4:
page["h1"] = h1[0]
page["h11"] = h1[1]
page["h12"] = h1[2]
page["h13"] = h1[3]
elif length_h1 == 3:
page["h1"] = h1[0]
page["h11"] = h1[1]
page["h12"] = h1[2]
elif length_h1 == 2:
page["h1"] = h1[0]
page["h11"] = h1[1]
elif length_h1 == 1:
page["h1"] = h1[0]
else :
page["h1"] = "---"


Now I am only accounting for 5 cases but sometimes the page may have as many as 15 h1's.

I have considered a for loop but not sure if that is a more memory efficient way or a better way exists in Python at all ? Please consider me a beginner and go gentle.

• The provided code doesn't work with 15 h1's, and so this code does not work the way you intend. – Peilonrayz May 19 at 11:48
• @Peilonrayz - You are being pedantic and infact closing the question based on code that works and requires review for scalability. Good way to tel a new user that you are not welcome here ! Thank you. – Sam May 20 at 23:46
• No. Everyone has to follow the rules. If you have a problem with how you've been treated you can raise it on Code Review Meta. – Peilonrayz May 20 at 23:51

## 1 Answer

Why don't use a 2-dimesion array? It's more simple and efficent, you don't need to waste time in condition statemento or loop.

def _h1(self, page, response) :
if isinstance(response, HtmlResponse):
page["h1"] = response.xpath("//h1/text()").getall()


If you need the number of H1 in page simply use len(page["h1"]) or if you need second result of your search use page["h1"][1] and so on.

• This makes much more sense. This creates another problem though that I am then inserting this into a db and a sqlite will usually not insert a list but that another problem altogether. Thank you for taking the time to respond. – Sam May 19 at 9:24
• @Sam When you save in db use un loop, something like this for r in page["h1"]: db.insert(col=r) – n1k9 May 19 at 10:00