I'm iterating through a number of text files, trying to locate all carriage-returns, and individually save the text between carriage-returns. I get the index numbers of all carriage-returns, but I haven't got the slightest about saving the text.
Basically I want to save every string between two carriage-returns into an individual variable. The next step is to save all the words within a string as an individual hash.
Here is my code so far (edited based on the help by Tin Man and Screenmutt) to get every single paragraph of a file into an array:
# script start
# outputfile
output_text = File.open("output.txt", 'w')
# directory with files
Dir.chdir("nkamp")
#count lines
lines = File.readlines("first.txt")
line_count = lines.size
text = lines.join
paragraph_count = text.split("\.\r").length
puts "#{paragraph_count} paragraphs."
#array of paragraphs
paragraphs = Array.new
contents = []
File.foreach("first.txt", "\.\r") do |paragraph|
puts paragraph.chomp
puts '-' * 40
contents << paragraph.chomp
paragraphs << paragraph.chomp
end
puts paragraphs[10]
This code gives me an array with all the paragraphs. I am using ".\r" instead of "\n\n" because the texts are copied from PDF files, and have lost the normal page layout structures.
The next step is to save an array of the words in a paragraph into the array instead of simply a string of text:
words_in_each_paragraph = Array.new
File.foreach("Ann Reg Sci (2).txt", "\.\r") do |paragraph|
word_hash = {}
paragraph.split(/\W+/).each_with_object(word_hash) { |w, h|
h[w] = []
}
words_in_each_paragraph << word_hash
end
puts words_in_each_paragraph[8]
Which gives the following output:
{""=>[], "The"=>[], "above"=>[], "contributions"=>[], "highlight"=>[], "the"=>[], "importance"=>[], "of"=>[], "sophisticated"=>[], "modeling"=>[], "work"=>[], "for"=>[], "a"=>[], "better"=>[], "understanding"=>[], "complexity"=>[], "entrepreneurial"=>[], "space"=>[], "economy"=>[]}
Now the next step is loop through every file, and create a dynamic hash that gives me
a. a number for the article. b. a number for the paragraph. c. the list of words as seen above.
For this I need to learn how to dynamically create hashes. This is where it goes wrong:
lines = File.readlines("test.txt")
line_count = lines.size
text = lines.join
paragraph_count = text.split("\.\r").length
puts "#{paragraph_count} paragraphs."
testArray = Array.new(paragraph_count.to_i, Hash.new)
for i in 0...paragraph_count.to_i do
testArray[i] = Hash.new
puts "testArray #{i} has been made"
end
words_in_each_paragraph = Array.new
File.foreach("test.txt", "\.\r") do |paragraph|
word_hash = {}
paragraph.split(/\W+/).each_with_object(word_hash) { |w, h|
h[w] = []
}
words_in_each_paragraph << word_hash
testArray[i][:value] = word_hash
puts testArray[i] # IT WORKS HERE #
end
puts testArray[1] # AND IT DOESN'T WORK HERE #