Markov chain-based word salad generator

Question

I like Markov chains. Last time I used one, I made one that generates words. This time, I made one that generates sentences, given a set of words and valid connections. This time, it's not weighted and it's reading from a JSON file, so the code is a bit simpler:

require 'json'

data = JSON[ARGV[0]]
CONNECTIONS = data['connections']
WORDS = data['words']

sentence = []
current_pos = 'start'
until current_pos == 'end'
  current_pos = CONNECTIONS[current_pos].sample
  sentence << WORDS[current_pos].sample
end
puts sentence.join(' ')

If we use this as the input data:

{
  "connections": {
    "start": ["article"],
    "article": ["adjective", "noun"],
    "adjective": ["adjective", "noun"],
    "noun": ["verb", "end"],
    "verb": ["article"],
    "end": []
  },
  "words": {
    "article": ["the"],
    "noun": ["dog", "cat", "penny", "truck"],
    "verb": ["ate", "stole", "helped", "cuddled"],
    "adjective": ["red", "big", "fluffy", "honorable"],
    "end": [""]
  }
}

which gives some fantastic word salad like:

the fluffy penny ate the truck stole the penny cuddled the cat stole the big cat 
the red red dog helped the honorable honorable fluffy big big red big cat stole the truck 

And, my personal favorite so far:

the fluffy honorable cat stole the dog 

I'm looking for tips on:

• Efficiency -- I think this is about as good as it'll get, but any suggestions are great.
• Usability -- I like word salad. I want to share it with everyone. To that end, I want to make this as easy and simple-to-use as possible.
• Readbility -- Because I like word salad (it's very nutritious) I want people to easily understand how to make it. Again, I think this is about as it's gonna get, but any suggestions are great.

Answer 1

There are various methods to generate a random n-word text. Markov chain is one of them.

Basic idea how to do this is described in Centennial of Markov Chains (by Oleksandr Pavlyk) under "Text Generation" - you generate n-grams carry info, encode transitions and define a function to walk the graph (in a non-brute force manner).

Result will heavily depend on word corpus and their connections rather then algorithm implementation itself.

---

The real problem is actually to generate the word corpus and connections from random text. In order to do this well you need to work out the grammatical structure of sentences. Because, you do not want to be doing this manually.

For example, consider the text:

The strongest rain ever recorded in India shut down the financial hub of Mumbai, snapped communication lines, closed airports and forced thousands of people to sleep in their offices or walk home during the night, officials said today.

You want to tag word grammatical structure

The/DT strongest/JJS rain/NN ever/RB recorded/VBN in/IN India/NNP shut/VBD down/RP the/DT financial/JJ hub/NN of/IN Mumbai/NNP ,/, snapped/VBD communication/NN lines/NNS ,/, closed/VBD airports/NNS and/CC forced/VBD thousands/NNS of/IN people/NNS to/TO sleep/VB in/IN their/PRP$ offices/NNS or/CC walk/VB home/NN during/IN the/DT night/NN ,/, officials/NNS said/VBD today/NN ./.

To do this you can use The Stanford Parser (ruby wrapper).