3
\$\begingroup\$

So I have a file containing a huge list of sentences, some containing keywords, and some not, so in order to specifically focus on the ones with keywords, I used this method. It works, but is there another way to this without having to create a new file?

keyW = ["love", "like", "best", "hate", "lol", "better", "worst", "good", "happy", "haha", "please", "great", "bad", "save", "saved", "pretty", "greatest", 'excited', 'tired', 'thanks', 'amazing', 'glad', 'ruined', 'negative', 'loving', 'sorry', 'hurt', 'alone', 'sad', 'positive', 'regrets', 'God']
with open('tweets.txt') as oldfile, open('newfile.txt', 'w') as newfile:
    for line in oldfile:
        if any(word in line for word in keyW):
         newfile.write(line)

because with these specific tweets, I'm going to use them when doing another function

for line in open('tweets.txt'):
    line = line.split(" ")
    lat = float(line[0][1:-1]) #Stripping the [ and the ,
    long = float(line[1][:-1])  #Stripping the ]
    if eastern.contains(lat, long):
        eastScore += score(line)
    elif central.contains(lat, long):
        centralScore += score(line)
    elif mountain.contains(lat, long):
        mountainScore += score(line)
    elif pacific.contains(lat, long):
        pacificScore += score(line)
    else:
        continue

Ultimately, this is what my code looks like.

from collections import Counter
try:
    keyW_Path = input("Enter file named keywords: ")
    keyFile = open(keyW_Path, "r")
except IOError:
    print("Error: file not found.")
    exit()
# Read the keywords into a list
keywords = {}
wordFile = open('keywords.txt', 'r')
for line in wordFile.readlines():
    word = line.replace('\n', '')
    if not(word in keywords.keys()): #Checks that the word doesn't already exist.
        keywords[word] = 0 # Adds the word to the DB.
wordFile.close()
# Read the file name from the user and open the file.
try:
    tweet_path = input("Enter file named tweets: ")
    tweetFile = open(tweet_path, "r")
except IOError:
    print("Error: file not found.")
    exit()
#Calculating Sentiment Values
with open('keywords.txt') as f:
    sentiments = {word: int(value) for word, value in (line.split(",") for line in f)}

with open('tweets.txt') as f:
    for line in f:
        values = Counter(word for word in line.split() if word in sentiments)
        if not values:
            continue
keyW = ["love", "like", "best", "hate", "lol", "better", "worst", "good", "happy", "haha", "please", "great", "bad", "save", "saved", "pretty", "greatest", 'excited', 'tired', 'thanks', 'amazing', 'glad', 'ruined', 'negative', 'loving', 'sorry', 'hurt', 'alone', 'sad', 'positive', 'regrets', 'God']
with open('tweets.txt') as oldfile, open('newfile.txt', 'w') as newfile:
    for line in oldfile:
        if any(word in line for word in keyW):
            newfile.write(line)
def score(tweet):
    total = 0
    for word in tweet:
        if word in sentiments:
            total += 1
    return total
def total(score):
    sum = 0
    for number in score:
        if number in values:
            sum += 1
#Classifying the regions
class Region:
    def __init__(self, lat_range, long_range):
        self.lat_range = lat_range
        self.long_range = long_range
    def contains(self, lat, long):
        return self.lat_range[0] <= lat and lat < self.lat_range[1] and\
               self.long_range[0] <= long and long < self.long_range[1]
eastern = Region((24.660845, 49.189787), (-87.518395, -67.444574))
central = Region((24.660845, 49.189787), (-101.998892, -87.518395))
mountain = Region((24.660845, 49.189787), (-115.236428, -101.998892))
pacific = Region((24.660845, 49.189787), (-125.242264, -115.236428))

eastScore = 0
centralScore = 0
pacificScore = 0
mountainScore = 0
happyScoreE = 0

for line in open('newfile.txt'):
    line = line.split(" ")
    lat = float(line[0][1:-1]) #Stripping the [ and the ,
    long = float(line[1][:-1])  #Stripping the ]
    if eastern.contains(lat, long):
        eastScore += score(line)
    elif central.contains(lat, long):
        centralScore += score(line)
    elif mountain.contains(lat, long):
        mountainScore += score(line)
    elif pacific.contains(lat, long):
        pacificScore += score(line)
    else:
        continue



print(keywords)
print("The number of tweets in the Pacific region is:", pacificScore)
print("The number of tweets in the Montain region is:", mountainScore)
print("The number of tweets in the Central region is:", centralScore)
print("The number of tweets in the Eastern region is:", eastScore)
\$\endgroup\$
4
  • \$\begingroup\$ What does keywords.txt contain? Your first use of the file suggests that it contains one word per line only, while your second use of it suggests it is a 2-column CSV file. \$\endgroup\$ Commented Nov 17, 2016 at 15:00
  • \$\begingroup\$ @RichardNeumann love,10 like,5 best,10 hate,1 lol,10 better,10 worst,1 good,5 happy,10 haha,5 please,5 great,5 bad,1 save,5 saved,5 pretty,5 greatest,10 excited,10 tired,1 thank,5 amazing,10 glad,10 ruined,1 negative,1 loving,10 sorry,1 hurt,1 alone,1 sad,1 positive,5 regrets,1 God,5 \$\endgroup\$ Commented Nov 17, 2016 at 15:06
  • \$\begingroup\$ @RichardNeumann key words and assoicated values \$\endgroup\$ Commented Nov 17, 2016 at 15:07
  • \$\begingroup\$ What ist the purpose of the dictionary keywords? You just assemble it and print it at the end of the program, but you don't use it for any processing or calculations. \$\endgroup\$ Commented Nov 17, 2016 at 15:35

1 Answer 1

3
\$\begingroup\$

Your code in general

You should definitely familiarize yourself with PEP8 which specifies Python coding style guidelines. Those include joint lower case names for variables an functions as well as empty lines around function and class definitions and a maximum line length of 79 characters.

Consistency

Whatever you do, be consistent. You mix single and double quotes for string literals and you do not use the file's build-in context management and iteration capability on wordFile.

Divide and conquer

Your code seems to do some complex data processing in several steps. Isolate these different tasks and outsource them into several functions, each specialized for one certain process. You already did this with the functions score() and total() and your Regions() class. Let's have more of those.

Use the script's __name__

Put the part of your code that should run when you execute your script inside an

if __name__ == '__main__':
    <your code here>

block. This prevents it to run on the import of the script as a module, if you or some other user one day decide to re-use its members, i.e. functions and classes, in other programs.

Comments

Though you commented parts of your code, those comments are a counterexample of their kind. You state the obvious by commenting on checking the membership of items in a dictionary, which can obviously be read from the code itself. On the other hand it is not clear, why you store comma seperated values as a key in this very dictionary, giving each key the value of 0 without using the dictionary any further apart from printing it out.

Filtering lines

Regarding your first question, filtering lines of a file by certain keywords is a common example for coroutines. You can use a method like

def grep(keywords):
    """Yields lines containing keywords"""

    file = yield

    for line in file:
        if any(keyword in line for keyword in keywords):
            yield line

and invoke it with

    with open(tweets_file) as tweets:
        fltr = grep(words)
        next(fltr)
        fltr.send(tweets)

        for line in fltr:
            print(line)

assuming words is your keyword list.

\$\endgroup\$
2
  • \$\begingroup\$ file = yield – is that correct? \$\endgroup\$
    – mkrieger1
    Commented Nov 17, 2016 at 18:11
  • 1
    \$\begingroup\$ Oh, I see. Awesome! \$\endgroup\$
    – mkrieger1
    Commented Nov 17, 2016 at 18:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.