3
\$\begingroup\$

I'm starting with Kotlin recently and am hoping to improve.

I wrote a small app to parse the list of free Springer books and download the books into your chosen local folder.

Comments around obvious mistakes, unidiomatic Kotlin, any other points of improvements will be greatly appreciated. Thank you.

Gradle dependencies:

    implementation("org.apache.poi:poi:4.1.2")
    implementation("org.apache.poi:poi-ooxml:4.1.2")
    implementation("org.jsoup:jsoup:1.13.1")

Kotlin code:

package dev.rayfdj.kotlinutils.springer

import org.apache.poi.ss.usermodel.WorkbookFactory
import org.jsoup.Jsoup
import java.io.File
import java.io.FileOutputStream
import java.net.URL
import java.nio.channels.Channels
import java.nio.file.Files
import java.nio.file.Path
import java.nio.file.Paths

data class Book(val title: String, val author: String, val edition: String, val year: String,
                val category: String, val url: String) {
    fun suggestedFileName(): String { return "$title, $edition - $author.pdf" }
}

fun extractBooksFromExcelFile(xlsxFile: File): List<Book> {
    // drop(1): skip the first row because it contains the headers
    return WorkbookFactory.create(xlsxFile).getSheetAt(0).drop(1).map {
        Book(it.getCell(0).stringCellValue, it.getCell(1).stringCellValue,
                it.getCell(2).stringCellValue, it.getCell(4).numericCellValue.toString(),
                it.getCell(11).stringCellValue, it.getCell(18).stringCellValue)
    }
}

fun deriveFullLocalPathForBook(downloadFolder: String, book: Book): Path {
    val fullLocalFileName = arrayOf(
            downloadFolder,
            book.category,
            book.suggestedFileName()).joinToString(separator = File.separator)
    return Paths.get(fullLocalFileName)
}

fun createDirectoriesAndFile(fullLocalFilePath: Path) {
    Files.createDirectories(fullLocalFilePath.parent)
    if(!Files.exists(fullLocalFilePath)) { Files.createFile(fullLocalFilePath) }
}

fun getBookDownloadURL(book: Book): URL {
    val bookPage = Jsoup.connect(book.url).get()
    val bookCanonicalURL = bookPage.select("link[rel=canonical]").attr("href")

    val bookCanonicalPage = Jsoup.connect(bookCanonicalURL).get()
    val bookPDFRelativeURL = bookCanonicalPage.select("a[href^=\"/content/pdf/\"]").attr("href")

    return URL("https://link.springer.com${bookPDFRelativeURL}")
}

fun downloadAndSaveBook(bookDownloadURL: URL, fullLocalFilePath: Path) {
    Channels.newChannel(bookDownloadURL.openStream()).use { inChannel ->
        FileOutputStream(fullLocalFilePath.toFile()).channel.use { outChannel ->
            print("Saving $bookDownloadURL to $fullLocalFilePath... ")
            outChannel.transferFrom(inChannel, 0, Long.MAX_VALUE)
            println("DONE.")
        }
    }
}

fun main(args: Array<String>) {
    if(args.size != 2) {
        println("Please pass <full_path_to_springer_excel_file> and <full_path_to_download_folder")
        kotlin.system.exitProcess(-1)
    }
    val (excelFile, downloadFolder) = args

    val books = extractBooksFromExcelFile(File(excelFile))

    books.forEach { book ->
        val fullLocalFilePath = deriveFullLocalPathForBook(downloadFolder, book)
        createDirectoriesAndFile(fullLocalFilePath)
        val bookDownloadURL = getBookDownloadURL(book)
        downloadAndSaveBook(bookDownloadURL, fullLocalFilePath)
    }
}
\$\endgroup\$
0

1 Answer 1

4
\$\begingroup\$

It's not exact translation of your code, but still :) I've just tried!

Unfortunately, it's not a github project, so I don't have an access to the excel sheet, so I can't really run and test it. But, anyways, ideas are:

  1. Utilization of toString() java method
  2. Using of lazy properties allows to 'cache' the URL easily. It's also possible because all of the Books properties are immutable.
  3. Extension method download() doesn't really belong to a book, but, instead, can easily download any URL
  4. I used require(), though, it's not really correct, since it throws an exception, instead of gracefully exiting the app. I used it, just to demonstrate it, nothing more.

And, probably, something else. Please, tell me what you're thinking!

data class Book(
    val title: String,
    val author: String,
    val edition: String,
    val year: String,
    val category: String,
    val url: String
) {
    constructor(row: Row) : this(
        row.getCell(0).stringCellValue, row.getCell(1).stringCellValue,
        row.getCell(2).stringCellValue, row.getCell(4).numericCellValue.toString(),
        row.getCell(11).stringCellValue, row.getCell(18).stringCellValue
    )

    override fun toString() = "$title, $edition - $author"

    val downloadURL by lazy {
        val canonicalURL = Jsoup.connect(url).get().select("link[rel=canonical]").attr("href")
        val pdfRelativeURL =
            Jsoup.connect(canonicalURL).get().select("""a[href^="/content/pdf/"]""").attr("href")
        URL("https://link.springer.com${pdfRelativeURL}")
    }
}

fun URL.download(to: Path): Path {
    Files.createDirectories(to.parent)

    Channels.newChannel(openStream()).use { inChannel ->
        FileOutputStream(to.toFile()).channel.use { outChannel ->
            print("Saving $this to $to... ")
            outChannel.transferFrom(inChannel, 0, Long.MAX_VALUE)
            println("DONE.")
        }
    }

    return to
}

fun main(args: Array<String>) {
    require(args.size == 2) { "Please pass <full_path_to_springer_excel_file> and <full_path_to_download_folder" }

    val (excelPath, downloadFolder) = args
    val excelFile = WorkbookFactory.create(File(excelPath))

    // drop(1): skip the first row because it contains the headers
    val books = excelFile.getSheetAt(0).drop(1).map { Book(it) }

    books.map { book ->
        val path = Paths.get(downloadFolder, book.category, "$book.pdf")
        book.downloadURL.download(to = path)
    }
}
```
\$\endgroup\$
4
  • 1
    \$\begingroup\$ Thank you very much! I learned a few things more about Kotlin from your answer :) The code is in github, here. Will keep this in mind next time I'm posting. In case you're interested in getting the books yourself, the Excel is available from Springer themselves (link in my original post, if you scroll down a bit and look for "Free English textbook titles (all disciplines)"). \$\endgroup\$
    – user224321
    Commented May 16, 2020 at 3:25
  • 1
    \$\begingroup\$ (I thought it funny they offer different editions of the same title.) \$\endgroup\$
    – greybeard
    Commented May 16, 2020 at 6:00
  • 1
    \$\begingroup\$ For this one: books.map { book -> val path = Paths.get(downloadFolder, book.category, "$book.pdf") book.downloadURL.download(to = path) } Wouldn't forEach represent the intent more closely? Because strictly speaking we're not "map"-ping the books to anything, are we? \$\endgroup\$
    – user224321
    Commented May 16, 2020 at 7:09
  • 1
    \$\begingroup\$ Yeah :) Actually, you're exactly right. I was thinking about editing this code, but was already done with this example at that time. I believe, here forEach is better AND it would also be nice to move printing from URL.download to this loop, since it's not related to the downloading, but, more, to the script scenario itself. \$\endgroup\$
    – Slava
    Commented May 16, 2020 at 8:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.