# Define the scope of negation with the Dependency Parser of spaCy

Sentiment words behave very differently when under the semantic scope of negation. I want to use a slightly modified version of Das and Chen (2001) They detect words such as no, not, and never and then append a "neg"-suffix to every word appearing between a negation and a clause-level punctuation mark. This method is very static and I want to create something a little bit more dynamic with the help of dependency parsing from spaCy.

This is an example of a tweet that I will process:

RT @trader $AAPL 2012 is ooopen to Talk about patents with GOOG definitely not the treatment Samsung got heh someURL  See here the visualized dependency parser in action on my string. I am able to identify not as negation modifier of got with the following: negation_tokens = [tok for tok in doc if tok.dep_ == 'neg']  Next, I want define the scope of the negation: negation_head_tokens = [token.head for token in negation_tokens] for token in negation_head_tokens: end = token.i start = token.head.i + 1 negated_tokens = doc[start:end] print(negated_tokens) # ooopen to Talk about patents with GOOG definitely not the treatment Samsung  Now I have defined the scope, I want to add "not" to certain words conditional on their POS-tag: list = ['ADJ', 'ADV', 'AUX', 'VERB'] for token in negated_tokens: for i in list: if token.pos_ == i: print('not'+token.text) # notooopen, notTalk, notdefinitely, notnot  Logically, I want to exclude notnot from the string. Full script import spacy nlp = spacy.load("en_core_web_sm") doc = nlp(u'RT @trader$AAPL 2012 is ooopen to Talk about patents with GOOG definitely not the treatment Samsung got heh someURL', disable=['ner'])

def negation(doc):
negation_tokens = [tok for tok in doc if tok.dep_ == 'neg']
negation_cue = [token.text for token in negation_tokens]

if not negation_tokens:   # no negation token(s) present in the string
return doc
else:
new_doc = []

end = token.i
negated_tokens = doc[start:end]
for token in doc:
if token.text not in negation_cue:
if token in negated_tokens:
if token.pos_ in pos_list:
new_doc.append('not'+token.text)
continue
else:
pass
new_doc.append(token.text)
return new_doc

negated_doc = negation(doc)
print(negated_doc)


Output:

['RT', '@trader', '\$', 'AAPL', '2012', 'is', 'notooopen', 'to', 'notTalk', 'about', 'patents', 'with', 'GOOG', 'notdefinitely', 'the', 'treatment', 'Samsung', 'got', 'heh', 'someURL']


Questions

Q1: Is it possible to use a direct route instead of the creating a list with

negation_cue = [token.text for token in negation_tokens]


and then check

if token.text not in negation_cue:


Q2: Is it maybe faster to remove the negation_cue token at the end of the script from the returned doc?

Q3 / Most important question: Do you see any speed-improvements in my script