Plot daily time spent on codereview.stackexchange.com

Question

This draws an xyplot from csv data obtained from RescueTime API for activity time for "http://codereview.stackexchange.com".

# -------------------------------------------------------
# PLOT TIME SPENT ON CODEREVIEW SE
# USING RESCUE TIME API AND R
# -------------------------------------------------------

# -------------------------------------------------------
# LIBRARIES
if(!require("RCurl")){
  install.packages("RCurl")
}
if(!require("lattice")){
  install.packages("lattice")
}
require("RCurl")
require("lattice")
# -------------------------------------------------------

print("Initializing...")

# -------------------------------------------------------
# RESCUE TIME API ACCESS

# api key 
# get from https://www.rescuetime.com/anapi/manage
api_key <- "..."

# query from date
from_date <- "2014-01-01"

# query to date
to_date <- format(Sys.Date(), "%Y-%m-%d")

# activity to get details of
activity <- "codereview.stackexchange.com"


# --api description--
# for more details see https://www.rescuetime.com/anapi/setup/documentation
# key - API key
# format - Output format
# rb - Restrict Start Date ("%Y-%m-%d")
# re - Restrict End Date ("%Y-%m-%d")
# rk - Restrict Kind (in this case it's activity)
# pv - Perspective (in this case it's interval)
# rs - Resolution Time (if it is day it's a daily report)
# rt - Restrict Thingy (Restrict Kind of Type)
url_format  <- "https://www.rescuetime.com/anapi/data?key=%s&format=csv&rb=%s&re=%s&rk=activity&pv=interval&rs=day&rt=%s"

# create url
api_url  <- sprintf(url_format,api_key,from_date,to_date,activity)
# -------------------------------------------------------

# -------------------------------------------------------
# PLOT

# get csv as a text
csv_text_data <- textConnection(getURL(api_url))

# parse csv
activity_data <- read.csv(csv_text_data,header = TRUE)


print(sprintf("Mean Time Spent on '%s':%f (seconds)",activity,mean(activity_data$Time))) 

activity_time_spent <- activity_data$Time
activity_dates <- activity_data$Date

# create an index for dates
activity_date_index  <- 1:length(activity_dates)

# plot
print(xyplot(activity_time_spent ~ activity_date_index, type="b", xlab = "Date Index", ylab = "Time (seconds)" ))
# -------------------------------------------------------

# -------------------------------------------------------
# CLEAN UP
detach("package:RCurl", unload = TRUE)
detach("package:lattice", unload = TRUE)
rm(list = ls())
# -------------------------------------------------------

Does this follow R conventions?

Execute by loading it to R console using source("./codereview.R")

Download

CSV and R file

Output

Loading required package: RCurl 
Loading required package: bitops
Loading required package: lattice 
[1] "Initializing..." 
[1] "Mean Time Spent on 'codereview.stackexchange.com':2764.559322 (seconds)" 

Answer 1

Here is how I would rewrite your code. I will add my comments after:

plotActivity <- function(
   activity  = 'codereview.stackexchange.com',
   api_key   = getOption('rescuetime_api_key'),
   from_date = NULL,
   to_date   = NULL) {

## This function plots time spent on a given URL
## using the rescue time API
##
## Arguments:
##    - activity:  the url you want to check for usage
##    - api_key:   user API key, you can get yours at
##                 https://www.rescuetime.com/anapi/manage
##    - from_date: a Date or string in "YYYY-MM-DD" format,
##                 defaults to January 1st of the same year as to_date
##    - to_date:   a Date or string in "YYYY-MM-DD" format,
##                 defaults to today
##    - verbose:   adds verbose, defaults to FALSE

   library("RCurl")
   library("ggplot2")

   # arguments parsing and default settings
   if (is.null(api_key)) stop("please set your rescue time API key...")

   if (is.null(to_date)) to_date <- Sys.Date()
   to_date <- as.Date(to_date)
   if (is.na(to_date)) stop("error parsing to_date...")

   if (is.null(from_date)) from_date <- format(to_date, "%Y-01-01")
   from_date <- as.Date(from_date)
   if (is.na(from_date)) stop("error parsing from_date...")

   # build the API request url
   # for more details see https://www.rescuetime.com/anapi/setup/documentation

   base_url  <- "https://www.rescuetime.com/anapi/data"
   arguments <- c(key = api_key,   # API key
                  format = 'csv',  # Output format
                  rb = from_date,  # Restrict Start Date
                  re = end_date,   # Restrict End Date
                  rk = 'activity', # Restrict Kind
                  pv = 'interval', # Perspective
                  rs = 'day',      # Resolution Time
                  rt = activity)   # Restrict Thingy (Restrict Kind of Type)

   arg_str <- paste(names(arguments), arguments, sep = '=', collapse = '&')
   api_url <- sprintf("%s?%s", base_url, arg_str)

   # read data
   csv_text_data <- textConnection(getURL(api_url))
   activity_data <- read.csv(csv_text_data, header = TRUE)

   # print diagnostic and plot
   cat(sprintf("Total Time Spent between %s and %s on '%s':%f (seconds)",
               activity, from_date, end_date, sum(activity_data$Time)))

   activity_data$Date <- as.Date(activity_data$Date)
   ggplot(activity_data, aes(x=Date, y=Time.Spent..seconds.)) +
      geom_bar(stat="identity") +
      xlab("Date") +
      ylab("Time (seconds)")
}

So here are a few ideas:

1. I went from a script to a function. This way it is a lot easier to use and share. I have abstracted what I believe were all the right inputs and chosen sensible defaults. Of interest is the use of getOption for the api_key so the user can set it once for all by running options(rescuetime_api_key = "xyz123abc"). So now, all you have to do is source the file where this function will be stored, then call the function. A function, when executed, runs in its own environment so all the variables that are created at runtime are deleted as you exit the function. Your global environment is never polluted, which removes the need for your "clean up" section (which was otherwise pretty harmful as janos pointed out.)

2. Like rm(list=ls()), forcing the install of packages via install.packages() is a bit harmful. No, you just want to use library and it will die right there, leaving the user with the decision of installing the package. There is a nice blog about why some people -me included- prefer library over require: http://yihui.name/en/2014/07/library-vs-require

3. I made use of R's Date object wherever it made sense.

4. I rewrote the way the api url is built, making use of a named vector for the arguments. With inline comments. I hope you'll agree the code is a bit more readable and easier to maintain, something you should always aim for.

5. I replaced print with cat. This way you don't get the [1] prefix that comes with printing a vector.

6. This has less to do with code reviewing but I thought that it would be more useful to see the total time spent on the website. Average time spent per day can be a little confusing: the user might be wondering if it is including all days or only those when the website was visited (you chose the latter.) While total time removes that ambiguity. Also, I thought it would be more useful to plot the data as a barplot with a real timescale, i.e. including holes for periods when the website was not visited. I used ggplot2 for that, see the picture below. The x-axis might look weird but you did spend six seconds on the website on July 1st so that's why it starts on that day, although you won't really notice there is a data point there.

1. My rewrite got rid of activity_date_index <- 1:length(activity_dates) but had I needed it, I would have used activity_date_index <- seq_along(activity_dates). Functions like seq_len or seq_along are better practice than 1:n because they properly handle the special case when n is zero. They return an empty vector (c()) while 1:n will return the vector c(1, 0) and create havoc.

Disclaimer: I don't have an account to that rescuetime website so I was not able to fully test my code. If you find a mistake, please feel free to edit my post, I won't mind.

Thanks for sharing your code, I like R and particularly enjoy seeing people use it to connect and analyze online databases like you did here. I hope my comments will help you.

Answer 2

Nice initiative! And R is perfect for this kind of stuff.

Conditional logic

if(!require("lattice")){
  install.packages("lattice")
}
require("lattice")

If the first require was successful, then no need to call it again. You can move the second inside the if:

if(!require("lattice")){
  install.packages("lattice")
  require("lattice")
}

Coding style

For coding style, you might find Google's R Style Guide interesting.

If that's too much reading, note that many general recommendations in other languages make sense for R too.

Example #1: put a space after commas in parameter lists:

# not so good
api_url  <- sprintf(url_format,api_key,from_date,to_date,activity)

# better
api_url <- sprintf(url_format, api_key, from_date, to_date, activity)

Example #2: put a space before opening paren and closing paren in if statements:

# not so good
if(!require("RCurl")){

# better
if (!require("RCurl")) {

And so on.

Cleaning up

I'm sure you had the best intentions with the cleaning up at the end. But it can be harmful.

What if you already had the RCurl and lattice libraries loaded before running your script, and still want them loaded after?

This can be outright harmful:

rm(list = ls())

This removes all objects in the global namespace. If you had something important in it before running this script, it will be lost.

The only use I see for the cleanup code is if after running the script, you want to continue working in a clean session. But then, it would be so much easier to just exit and restart.

Future code reviews

The CSV file included in the download link contains 59 lines. That's not so bad, you could have just included it verbatim in the question. I didn't test with the real API, there's not much to test in the way you retrieve a URL anyway. I just changed the code a bit like this:

# csv_text_data <- textConnection(getURL(api_url))
csv_text_data <- '/tmp/codereview_raw.csv'
activity_data <- read.csv(csv_text_data, header=TRUE)