`nov.el` function to search the entire `epub` ebook

Question

nov.el does not provide an easy function to search the whole of the ebook content. Every search performed on buffer only applies to the "document" or chapter of the book on display at that time.

I wrote my own search function with reference to this answer below:

(defun nov-search (pattern)
  "Search whole ebook in nov."
  (interactive "sEnter search pattern: ")
  (let ((doc-size (length nov-documents))
        (version nov-epub-version)
        (doc-index 1)
        result)

    (while (< doc-index (1- doc-size))
      (seq-let (id &rest path) (aref nov-documents doc-index)

        ;; prepare temp buffer for search
        (with-temp-buffer
          (cond
           ((and (version< version "3.0")
                 (eq id nov-toc-id))
            (insert (nov-ncx-to-html path)))
           (t
            (insert (nov-slurp path))))

          (nov-render-html)
          (goto-char (point-min))

          ;; search function
          (while (search-forward pattern nil t)
            (push (list (concat (file-name-base path) ": " (replace-regexp-in-string "\n" " " (thing-at-point 'sentence)))
                        doc-index
                        (point))
                  result))
          ;; iterate to next document
          (setq doc-index (1+ doc-index)))))

    (seq-let (doc pos) (alist-get (completing-read "Jump to: " (reverse result)) result nil nil #'string=)
      (nov-goto-document doc)
      (goto-char pos)
      )))

The function generally does what is expected, but response is relatively slow - for a 10MB ebook, a simple search term takes more than 10 seconds to generate results.

I would like to seek help in this review to improve the efficiency of the search, possibly (1) by the implementation of some form of indexing beforehand, or (2) through a refactoring of the above search function to make use of existing libraries more suited for the task and (3) point out algorithmic design issues that affect response speed in the code above, if there is any.

Answer 1

One thing you could do to speed up the search, is to search for a match first in the raw html, and if you find a match, only then render it using (nov-render-html) to obtain the correct positions.

The replace newline looks redundant when using sentence in thing-at-point. In order to get the correct position of the match in the buffer, you have to use nov-render-html first, otherwise you would search in the raw html file and the position would be off (test it by commenting out that line). Therefore, I guess you can not use any other tool, and this will be the fastest solution (except that you are not skipping rendering images, as in my original answer).

You could try how much the following version speeds up things:

(defun nov-search (pattern)
  (interactive "sEnter search pattern: ")
  (let ((version nov-epub-version)
        (index 1)
        results)
    (while (< index (1- (length nov-documents)))
      (seq-let (id &rest path) (aref nov-documents index)
        (let (;; HACK: this should be looked up in the manifest
              (imagep (seq-find (lambda (item) (string-match-p (car item) path))
                                image-type-file-name-regexps))
              ;; NOTE: allows resolving image references correctly
              (default-directory (file-name-directory path)))
          (unless imagep
            (with-temp-buffer
              (if (and (version< version "3.0") (eq id nov-toc-id))
                  (insert (nov-ncx-to-html path))
                (insert (nov-slurp path)))
              (goto-char (point-min))
              (when (search-forward pattern nil t)
                (nov-render-html)
                (goto-char (point-min))
                (while (search-forward pattern nil t)
                  (push (list (format "%d %s" index
                                      (replace-regexp-in-string "\n" " "
                                                                (thing-at-point 'line)))
                              index (point))
                        results)))))
          (setq index (1+ index)))))
    ;; (print results)))
    (seq-let (index point) (alist-get (completing-read "Jump to: " (reverse results)) results
                                      nil nil #'string=)
      (nov-goto-document index)
      (goto-char point))))