Support vector machines (SVM)

Support vector machines (SVM) can also be used to predict a binary class. SVM projects the data into a higher dimensional space so that hyperplanes can be used to separate the classifiers. SVMs can be very accurate but difficult to interpret and computationally expensive. They are a classic example of a low bias algorithm.

Here is a simple example of using an SVM to predict whether a person is satisfied based upon the day of the week and whether or not it is a payday. (The vector element is marked as 1 in the payday vector, which can be interpreted as Friday if you start counting from Sunday.)

library(e1071) 
satisfied = factor(c(F,F,F,F,F,T,F)) 
day = c(1,2,3,4,5,6,7) 
payday = c(0,0,0,0,0,1,0) 
satisfaction.df = data.frame(day=day, payday=payday, satisfied=satisfied) 
model.svm = svm(satisfied ~ day + payday, data = satisfaction.df) 
plot(model.svm,satisfaction.df,main="",col=c('grey','black')) 

As you can see from the following plot, the decision line is a curve near the upper-right part of the quadrant, and the area where payday is close to 1 (yes), and day of week is close to 4, 5, 6, 7 (later part of the week). So we can interpret that as saying people are satisfied on payday and at the end of the week. However, this is a very low dimension example (rendered in two axes) that we used to illustrate the concept. Higher dimensional examples are not that easy to interpret: