Questions tagged [natural-language-processing]

The field of natural language processing covers attempts to make sense of text in a human language using computers

19 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
5
votes
0answers
73 views

Optimally splitting a text into strings from a set

I am trying to write an algorithm that starts with a corpus of texts (e.g., a Wikipedia dump). It first builds an array of individual characters (e.g., "a", ...
3
votes
0answers
40 views

Django project for events of the day, grouped by keywords in common

I have made a website where people write about their day and see it analyzed. In particular, the website takes event titles and groups them together if they have any common words (after stemming the ...
3
votes
0answers
46 views

Download and analyze PDFs of Congressional records

This is built with Python 2.7.15. The goal of this script is to count the number of words spoken by each Senator on the floor of Congress between given dates. It pulls from the Congressional Record, ...
3
votes
0answers
57 views

Time calculation for NLTK tagging

I am trying to calculate the time required to tag one sentence/file by one trained NLTK HMM Tagger. To do this I am writing the following code, please suggest if I need to revise anything here. ...
3
votes
0answers
112 views

Converting a sentence string to a number in Scheme

I wrote a scheme procedure that converts strings such as "Two hundred and fifty" into a number like "250" that I can use in math calculations (It's for some Natural Language Processing project). Is ...
3
votes
0answers
229 views

Naive implementation of Naive Bayes in Haskell

A simple implementation of Naive Bayes, as I'm a Haskell beginner I've tried to put an emphasis on clarity and documentation. I've tried to incorporate the feedback from a similar question. This is ...
3
votes
0answers
138 views

Read in file, Filter out records, output results

I have been working on a project for some time and have let one Python file grow into a huge file that tries to do everything. The structure is pretty flat and I don't like it. I've used other ...
3
votes
0answers
89 views

Generation and parsing of English numerals (cardinal and ordinal)

I need to use the English numerals (American) in Haskell, so I looked for a library that did it. I have not found one, therefore I developed the program that I am presenting here. There are actually ...
2
votes
0answers
85 views

Define the scope of negation with the Dependency Parser of spaCy

Sentiment words behave very differently when under the semantic scope of negation. I want to use a slightly modified version of Das and Chen (2001) They detect words such as no, not, and never and ...
2
votes
0answers
43 views

Fetching and modifying CNN headlines

This is a project I came up just as an exercise with to familiarize myself with Python syntax and data types. I am just learning how to code and would like to avoid establishing bad habits. On the ...
2
votes
0answers
40 views

Classes to help transform “How hard is it to …?” questions into replies like “It's really hard to …”

I've written a Twitter bot, @answering_yelp, which responds to another Twitter bot, @hard_to_yelp. They scrape yelp reviews for sentences which start "How hard is it to..." and my bot answer with some ...
1
vote
0answers
70 views

Language-detection heuristic (English, French or German) based on Unigram and Bigram models

Given a string, for example "I hate AI", I need to find out if the sentence is in English, German or French. Unigram Model makes the prediction on the basis of each character frequency in a training ...
1
vote
0answers
35 views

Function for root matching between two paragraphs

I created an algorithm that match roots of two texts, a question and a paragraph made of sentences. I aim at predicting in which sentence it exists the answer of a question. Yet It seems that I really ...
1
vote
0answers
165 views

Analysis of the most common words in a text

I have been trying to gain more understanding and experience in the NPL area and to get some more practice. I decided to attempt to create a simple high-level ...
1
vote
0answers
34 views

NER and its F Measure Calculation

I am trying to write one Name Entity Recognition in Hindi. I have primarily used NLTK of Python. I have used HMM Module with its supervised training. The data is annotated and saved in .pos files ...
1
vote
0answers
129 views

Naïve Bayes classifier to group questions by intent

I am trying to train a question-answer system, where I am trying to group similar questions, and identify the most apt response. The program should identify the intent/focus. To do it, I have tagged ...
1
vote
0answers
97 views

Using polymorphic objects to represent parts of a deconstructed sentence

I am trying to make a chatbot using javascript by deconstructing a sentence into its intent by separating the Noun's Verb's, Adjectives, Durations's, Numbers etc... to construct a data query to try to ...
1
vote
0answers
63 views

Linear SVM(in R) giving suspiciously high CV accuracy

Here is my code that reads a text dataset, featurizes the text column and then does CV using Linear SVM. Could I be leaking labels from somewhere as the accuracy I get is suspiciously high and way ...
1
vote
0answers
1k views

Extracting names from a body of text

I am trying to extract names from a body of text to use as stopwords. I tried a few different approaches to identifying names (or proper nouns in general) below. The second approach is much faster ...