I decided to sidetrack and create a XKCD viewer. For certain functionality, I needed to be able to find the ID of the last comic published. This was my attempt. I'm using Enlive here to parse the page itself.
I struggled with trying to find a CSS selector to get the text node, then finally gave up and decided to do some manual parsing. It got long, and ugly, but it works! The problem is that the only place I can concretely find page IDs is as a note at the bottom of the page:
Permanent link to this comic: https://xkcd.com/1988/
To parse that ID at the end of the link out, I need to find the text node, then parse the String. The latter was easy. The former took me a little under an hour due mostly to inexperience with CSS selectors.
What I'm looking for:
- Is there a way to get the text node directly via Enlive CSS-like selectors?
- Anything else that may simplify this. It's quite a series of transformations. I obviously could separate it down into a few function, but I can't see ever needing the functionality anywhere else, and it's fairly simple to test as is. Any recommendations here?
Use as of posting this:
(find-last-id) => 1988
(ns xkcd-viewer.mcve (:require [net.cgrand.enlive-html :as e]) (:import (java.net URL))) (def base-url "https://xkcd.com/") ; I actually use this a couple time in the real code. It doensn't seem as useful here though. (defn parse-id? "Returns the str-n parsed as a long, or nil if it's unparsable." [str-n] (try (Long/parseLong str-n) (catch NumberFormatException _ nil))) (defn find-last-id  (let [digit? #(Character/isDigit ^Character %) id-container (-> (e/html-resource (URL. base-url)) (e/select [:#middleContainer]) (first) (:content)) raw-id (->> id-container ; The text node to find is surrounded by <br>s, so (drop-while #(not= (:tag %) :br)) ; get rid of everything before the first br, (drop 1) ; then the br itself, (first) ; then get the text node, then (drop-while (comp not digit?)) (take-while digit?) (apply str))] ; then turn the digits into a string to be parsed. (if-let [parsed (parse-id? raw-id)] parsed (throw (RuntimeException. (str "Parser broken! Did XKCD change their site?\nFound ID: " raw-id))))))