# Optimize an algorithm for preparing a dataset for machine learning

I'm learning how to use R coming from a python background. I'm following Andrej Karpathy's zero-to-hero course, reimplementing it in R.

We start with a list of 32033 names. These names have to be broken into a format digestible by the network. For example, if the first name in the dataset is emma, we would represent it as such:

X   |  Y
________
... | e
..e | m
.em | m
emm | a
mma | .
... | etc


Although each character is represented as a number so they can later be stored as a tensor. I've written the following alogrithm to do so:

  XS <- vector("list",length(data)*block_size*2)
YS <- vector("numeric",length(data)+1)
stoi <- setNames(0:26,c('.',letters))
i = 1

for (item in data) {
context <- rep(0,block_size)

for (ch in strsplit(item,"")[[1]]) {
ch <- stoi[ch]
XS[[i]] <- context
YS[[i]] <- ch
context <- c(context[-1],ch)
i <- i + 1
}
}
return(list(XS,YS))
}


I would really appreciate any feedback on how to write more performant and idiomatic code in R that can accomplish this better than my implementation. Thank you.

Right now your code basically loops through words to process. For each word, it first uses strsplit to split it into characters, and then it loops through each character, maintaining a running vector of the last block_size characters it has encountered. It iteratively stores the running vectors into a list, which it eventually returns.

A few thoughts on this code:

1. Since each of your blocks is of the same size (as determined by your block_size variable), it would make more sense to me to actually build a matrix instead of a list. Probably you will find the matrix easier and faster to work with.
2. In R we strive to identify pre-implemented, vectorized code that does what we are working to accomplish. Usually this involves some amount of searching around to find the correct function. In your case, it turns out that there is a built-in function called embed that does basically what you're asking. For instance, here is the output for your "emma" example, with block size 3, where we separately compute the contributions to X and Y as defined in your question:

word <- "emma"
block_size <- 3
(wordLetters <- c(rep(".", block_size), strsplit(word, "")[[1]]))
# [1] "." "." "." "e" "m" "m" "a"
(Xpart <- embed(wordLetters, block_size)[,block_size:1])
#      [,1] [,2] [,3]
# [1,] "."  "."  "."
# [2,] "."  "."  "e"
# [3,] "."  "e"  "m"
# [4,] "e"  "m"  "m"
# [5,] "m"  "m"  "a"
(Ypart <- c(strsplit(word, "")[[1]], "."))
# [1] "e" "m" "m" "a" "."


Once we have things working for one word, we can simply loop to combine them together across all the words we need to process. I'll also include the actual conversion to numbers (via stoi) as you defined in your code:

data <- c("emma", "hello")
block_size <- 3
stoi <- setNames(0:26,c('.',letters))
(XS <- do.call(rbind, lapply(data, function(word) {
wordLetters <- stoi[c(rep(".", block_size), strsplit(word, "")[[1]])]
embed(wordLetters, block_size)[,block_size:1]
})))
#       [,1] [,2] [,3]
#  [1,]    0    0    0
#  [2,]    0    0    5
#  [3,]    0    5   13
#  [4,]    5   13   13
#  [5,]   13   13    1
#  [6,]    0    0    0
#  [7,]    0    0    8
#  [8,]    0    8    5
#  [9,]    8    5   12
# [10,]    5   12   12
# [11,]   12   12   15
(YS <- unlist(lapply(data, function(word) {
unname(stoi[c(strsplit(word, "")[[1]], ".")])
})))
#  [1]  5 13 13  1  0  8  5 12 12 15  0