The first thing you should do is make this a "proper" script - functions, and an
if __name__ == '__main__' block. This allows it to be executed as a standalone script, or imported as needed.
Next, you need to make sure to close your files. The easiest way to do this is with context managers.
Next you'll want to make it a but more generalized - pass more things as parameters instead of hardcoding them. This includes filenames, the number of words, etc.
Next, store your
stop_words in a
frozenset for more efficient lookup.
Next, use the
Counter more effectively by using it within the
word_map function and using a list comprehension as well. Write a function to filter for readability.
Lastly, use better names and combine your two helper functions - they're simple enough that you don't need both of them.
from collections import Counter
def find_n_top_words(source_filename, stopwords_filename, num_top_words):
with open(source_filename, 'r') as source_file, \
open(stopwords_filename, 'r') as stop_word_file:
stop_words = frozenset(stop_word_file.read().split())
words = [filter(word.isalnum).lower().strip() for word in string.split()]
return word and word not in stopwords
filtered_words = filter(filter_func, words)
word_dict = Counter(filtered_Words)
top_words = word_dict.most_common(num_top_words)
for word, count in top_words:
print '%s : %10d' % (word, count)
if __name__ == '__main__':
find_n_top_words('story.txt', 'stopwords.txt', 5)