1
\$\begingroup\$

I'm working on a simple parser that is supposed to extract hash maps representing vinyl records from web pages (online stores) and XML files. The goal is to have a parser that I can easily run with multiple threads over a ~8 MB XML and a ~2 MB HTML file to read products into a relational database.

I know the relational database will be its own nightmare to marshal things into and to update, but learning how I can marry Clojure to a traditional database is part of my fun.

The initial parser seems to work reasonably well, but my grasp of idiomatic Clojure and my Clojure vocabulary are rudimentary at best. Do you have any suggestions?

Source on GitHub

The result of the doseq there is:

{:artist Lantlôs, :title Melting Sun, :price 12.99}
{:artist Alcest, :title Shelter, :price 12.99}
{:artist Alcest, :title Opale, :price 8.99}
{:artist Duncan Evans, :title Lodestone, :price 14.99}
{:artist Empyrium, :title Dead Winter Ways, :price 9.99}
{:artist Falkenbach, :title Asa, :price 14.99}
{:artist Negura Bunget, :title Gînd a-prins, :price 8.99}
{:artist Sol Invictus, :title Mr Cruel, :price 8.99}
{:artist LowCityRain, :title LowCityRain, :price 13.99}
{:artist Vàli, :title Forlatt, :price 14.99}
{:artist Vàli, :title Skogslandskap, :price 14.99}
{:artist Nhor, :title Within The Darkness Between The Starlight, :price 24.99}

I'm only working on one single parser (the one for Prophecy Productions in that core.clj file) right now. Once I have multiple parsers ready, I'll have to find a good way to split the project into multiple files and have some sort of centralized reader that takes the hash-maps and marshals them into a database. But I'll bug people here about that when I get there.

\$\endgroup\$
  • \$\begingroup\$ (Even if it wasn't in the review queue) welcome to Code Review! Usually the guideline here is to include the full code in question, not just a link to the source. \$\endgroup\$ – ferada Dec 8 '14 at 13:55
  • \$\begingroup\$ Oh, sorry, I'll read up about the rules. \$\endgroup\$ – Ramón Dec 8 '14 at 17:04
1
\$\begingroup\$

On top of my head, you could convert (string/trim (html/text (first (html/select ... to a function as it's used twice already.

In product-hash the function extract-artist-album is called twice, so the return value should've been reused instead.

A minor style thing is the nth, personally I'd rather use first, second, ..., unless the indexes get larger, but YMMV.

Also, the last two forms should be in a separate function so the file can be loaded without them being executed.

| improve this answer | |
\$\endgroup\$
  • \$\begingroup\$ Thanks! Yes, those forms will go away once I split everything into the proper files and namespaces. I'll implement your suggestions and will tie them together with some sort of centralized crawler. \$\endgroup\$ – Ramón Dec 8 '14 at 17:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.