This currently starts out with the speed of 2000 pages/min but shortly after starting it becomes very slow with a speed of about 200 pages/min. Why is this happening? How can I improve this scraper?

import scrapy
from scrapy.crawler import CrawlerProcess
from time import time

def isAscii(s):
return all(ord(c) < 128 for c in s)

region = input('Region? ')
start = int(input('Start page? '))
end = int(input('End page? '))

startTime = time()

class LolSpider(scrapy.Spider):
name = 'lolspider'
start_urls = ['http://lolprofile.net/leaderboards/{}/{}'.format(region, page) for page in range(start, end)]
def parse(self, response):
for name in response.xpath('//span[not(@class)]/text()').extract():
if (' ' not in name) and (name != 'LoL') and isAscii(name):
f.write(name+'\n')

process = CrawlerProcess({})

process.crawl(LolSpider)
process.start()

endTime = time()
print('Completed in {:.2f} seconds'.format(endTime-startTime))


My main goal is to be as fast as possible and that's why I used scrapy. If anyone knows of a faster alternative I'd love to hear it.

The main problem is that you are writing/appending to the file inside the spider's parse() method - this is not how this should be done in Scrapy - there is a special place - Item Pipelines. Check this answer to see how to include a custom pipeline when running Scrapy from a script.
Note that in the pipeline you should not be appending to a file, but instead collecting the data in the process_item() method and dumping it when the spider is finished in spider_closed() method. You may also do the writing in chunks to avoid potential memory problems on large output data sizes.