My program opens a website and downloads a text file. The text file is a simple file with one word per line. I save the file to local disk and then create a list to hold each line of the text file for later processing. I would like to know if I am doing these first steps in a way that would be considered idiomatic Python and have I made any big mistakes that will hamper my efforts to expand on it later.
This is similar to an exercise in Think Python by Allen Downey. He suggests using a browser to download the text file but I wanted to do it with Python.
import requests def get_webpage(uri): return requests.get(uri) def save_webpagecontent(r, filename): """ This function saves the page retrieved by get_webpage. r is the response from the call to requests.get and filename is where we want to save the file to in the filesystem.""" chunk_size = 8388608 # number of bytes to write to disk in each chunk with open(filename, 'wb') as fd: for chunk in r.iter_content(chunk_size): fd.write(chunk) def make_wordlist(filename): wordlist =  with open(filename) as fd: wordlist = fd.readlines() return wordlist def get_mylist(wordlist, num_lines=10): if len(wordlist) <= num_lines: return wordlist return wordlist[:num_lines] def print_mylist(mylist): for word in mylist: print(word.strip()) return None """List of words collected and contributed to the public domain by Grady Ward as part of the Moby lexicon project. See https://en.wikipedia.org/wiki/Moby_Project """ uri = 'https://ia802308.us.archive.org/7/items/mobywordlists03201gut/CROSSWD.TXT' filename = 'wordlist.txt' r = get_webpage(uri) save_webpagecontent(r, filename) wordlist = make_wordlist(filename) mylist = get_mylist(wordlist) print_mylist(mylist)
My program works as I expect it to. I have basically found how to do each individual piece by reading this forum and but I would like to know if I'm putting all the pieces together correctly. By correctly I mean something that not only functions as expected but also will be easy to build larger programs and modules from.
I hope it isn't wrong of me to post this much code. I wasn't sure how I could trim it down and still show what I am doing. Please let me know if I need to change the format of my question.