Time series data in R

Handling time series data in R

In this blog post I want to write some thoughts about handling time series data in R. In contrast to cross-sectional data, in time series applications each observation has an additional component besides it’s value: the point of time. This requires some additional efforts, for example:

  • x-axis has to be labeled with dates instead of numbers
  • vectors are better sliced with respect to dates instead of indices, as this is more natural to humans
  • concatenation of column vectors requires chronological ordering with respect to dates

There are two formats that seem to me most practical: zoo objects and data frames. Zoo objects manage to store respective date specifications separately from the core data, while still providing a very convenient API for slicing, merging or ad-hoc plots. Data frames, however, are first choice for high-quality ggplots, while dates usually are stored in a separate column among the other data as well. This way, data can be easily reshaped with melt() into the pattern required by ggplot.

Let’s look at some examples.

First, set up some R session, respective packages loaded.

rm(list=ls())
library(zoo)
library(reshape)
library(ggplot2)

Now, we want to familiarize ourselves with the way R handles dates. Internally, R has its own (numeric) calendar, with beginning set to

as.Date(1)
[1] "1970-01-02"

When we want to create dates associated with financial time series, we have to account for the fact that stock markets are closed on weekends. This can easily be done by relying on the internal numeric calendar of R.

Get weekday of calendar origin:

weekdays(as.Date(1))
[1] "Friday"

Some transformation in other direction:

as.numeric(as.Date("2013-01-01"))
[1] 15706

Hence, all numeric dates with value 2 or 3 modulo 7 are weekends:

weekdays(as.Date(2:3))
weekdays(as.Date(9:10))
[1] "Saturday" "Sunday"

[1] "Saturday" "Sunday"

An easy function to check for weekends is

is.weekend <- function(x) ((as.numeric(x)-2) %% 7) < 2

Create sequence of dates

dates <- seq(as.Date("2013-01-01"),as.Date("2013-01-31"), 
             by = "1 day")
weekdays(dates[4:7])
[1] "Friday"   "Saturday" "Sunday"   "Monday"

Delete dates of weekends

business_days <- dates[!is.weekend(dates)]
weekdays(business_days[4:7])
[1] "Friday"    "Monday"    "Tuesday"   "Wednesday"

Or, with just one line

business_days2 <- dates[!(weekdays(dates) %in% c('Saturday','Sunday'))]
weekdays(business_days2[4:7])
[1] "Friday"    "Monday"    "Tuesday"   "Wednesday"

Let’s now create some artificial time series data, which we will use to show some main features.

nAss <- 800
nObs <- length(business_days)
log_ret <- rnorm(nObs*nAss, mean=0.01, sd=1.2)
log_ret <- matrix(log_ret, ncol=nAss, nrow=nObs, byrow = T)

## cumulate log returns to get log prices: 2 indicates columnwise
log_p <- apply(log_ret, 2, cumsum)

## create zoo object
data_zoo <- zoo( log_p, order.by=as.Date(business_days))
names(data_zoo) <- 1:nAss

## plot zoo object: with transparent colors
## colors <- topo.colors(nAss, alpha = 0.1)
## plot(data_zoo, plot.type = "single", col = colors)

## plot zoo object with slightly transparent black color
colr <- rgb(0.1,0.1,0.1,0.2, names = NULL, maxColorValue = 1)
plot(data_zoo, plot.type = "single",  col=colr, pch=16)

file:///home/chris/research/cfm/blog_posts/src_results/1.png

As can be seen, plotting of zoo objects will include weekends, too. This can heavily distort the graphics through inclusion of artificial patterns, especially in the case of relatively short time series. An alternative way of plotting also based on R’s standard and fast plotting capabilities is:

colr <- rgb(0.1,0.1,0.1,0.2, names = NULL, maxColorValue = 1)
matplot(data_zoo, type="l", xaxt="n", col=colr, lty
        = 1, lwd = 1, ylab = "prices")
timelabels<-format(index(data_zoo))
axis(1,at=1:23,labels=timelabels)

file:///home/chris/research/cfm/blog_posts/src_results/2.png

As long as we just want to take a first look at the patterns of our data, we usually want to rely on fast and on-the-fly visualization techniques like with the code examples given above. However, once pictures have to be made ready for publishing, where certain aspects of the data need to be emphasized in the most appealing way, the standard graphics routines can become insufficient, and more extensive packages like ggplot2 could come into focus. The advanced features, however, do come with a downside: creation of graphics will require significantly more computational time.

ggplot2 requires data given in data frames. Hence, in order to plot a zoo object with ggplot2, we first have to convert it.

df1 <- as.data.frame(data_zoo, time=index(data_zoo))

df2 <- data.frame(time=time(data_zoo), data_zoo)

These two conversion have one major difference: they differ in their dimensions.

dim(df1)
dim(df2)
[1]  23 800

[1]  23 801

The reason for this is that the second call additionally includes the time dimension as a separate column.

df1[1:2,1:4]
df2[1:2,1:4]
                  1          2          3         4
2013-01-01 2.015704  1.1656187 -1.1083786 -1.149100
2013-01-02 2.980257 -0.3672188  0.8720595 -1.124556
                 time       X1         X2         X3
2013-01-01 2013-01-01 2.015704  1.1656187 -1.1083786
2013-01-02 2013-01-02 2.980257 -0.3672188  0.8720595

This way, dates are already better accessible for ggplot2 than when they are stored as row.names only. It now will be easier to convert the data into “long format”, so that ggplot2 can make use of its capabilities of faceting the data with respect to different dimensions.

df3 <- melt(df2, id.vars = "time")
df3[1:4,]
        time variable    value
1 2013-01-01       X1 2.015704
2 2013-01-02       X1 2.980257
3 2013-01-03       X1 3.556476
4 2013-01-04       X1 3.200586

The fastest way to achieve long format is to directly manipulate the data during conversion from zoo object to data frame.

df4 <- data.frame(time = time(data_zoo),
                  variable = rep(names(data_zoo), each = nrow(data_zoo)),
                  value = as.vector(data_zoo))
df4[1:4,]
        time variable    value
1 2013-01-01        1 2.015704
2 2013-01-02        1 2.980257
3 2013-01-03        1 3.556476
4 2013-01-04        1 3.200586

For example, it is now where easy to group the data with respect to either point of time or stock.

p <- ggplot(df4, aes(time,value,group=variable)) + 
  geom_line(alpha = 0.2)
p

file:///home/chris/research/cfm/blog_posts/src_results/3.png

p <- ggplot(df4, aes(time,value,group=time)) + geom_boxplot()
p

file:///home/chris/research/cfm/blog_posts/src_results/4.png

However, we again should get rid of weekends first.

p <- ggplot(df4, aes(ordered(format(df4$time, format = '%d%b')),value,group=time)) +
  geom_boxplot()
p + opts(axis.text.x=theme_text(angle=-90))

file:///home/chris/research/cfm/blog_posts/src_results/5.png

Advertisements

Posted on 2013/04/22, in R, visualization and tagged , , , , . Bookmark the permalink. Leave a comment.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: