R package Blotter
How many times have you been disappointed by nice trading system, because neither trading cost or slippage or bid/ask spread were included into back-test results? Did you find difficult to back-test a portfolio in R or many portfolios with different stocks? Blotter package is supposed to solve these problems.
In really – it is complicated. I spent a couple of days to start using it. There was one bug in the demo example, but the rest was in my code. You should remember:
1. Time zone must be specified:
Sys.setenv(TZ="GMT") |
2. xts object must be created with explicit index class:
getSymbols('^GSPC',from='2000-01-01',<strong>index.class=c("POSIXt","POSIXct")</strong>) |
3. When the test is conducted all blotter related values are written into .blotter environment. If you want repeat the same test, then you need to get rid of all .blotter values. So, you have to run something like this:
You can find the author’s comment about this issue here: http://n4.nabble.com/Blotter-package-problem-with-example-tp1018634p1572911.html
4. It is slow. If you want to find the fittest rule from a bunch of them – try avoid using blotter, instead, use pure cumulative return and later on, include blotter.
Let’s test this rule with blotter as an example:
short SPY if today’s low is higher than yesterday’s close and close next day.
require(xts) require(quantmod) Sys.setenv(TZ="GMT") initDate='2000-01-01' getSymbols('SPY',from=initDate,index.class=c("POSIXt","POSIXct")) SPY<-adjustOHLC(SPY,use.Adjusted=T) #yesterday's price tmp<-lag(SPY,1) # if today's low is higher than yesterday's close 1, else 0 signal<-ifelse(Lo(SPY)>Cl(tmp),1,0) signal[1]<-0 #let's plot cumulitative return to make sure, that we are on the right path tmp<-lag(Delt(Cl(SPY)),-1) tmp[length(tmp)]<-0 plot(cumprod(signal*tmp+1)) SPY<-Cl(SPY) symbols<-c('SPY') rm(list=ls(envir=.blotter),envir=.blotter) initPortf(ltportfolio,symbols, initDate=initDate) initAcct(ltaccount,portfolios=c(ltportfolio), initDate=initDate,initEq=initEq) signal<-signal['2000-02-01::'] for(i in 1:length(signal)) { currentDate= time(signal)[i] equity = getEndEq(ltaccount, currentDate) position = getPosQty(ltportfolio, Symbol=symbols[1], Date=currentDate) if(position==0) { #open a new position if signal is >0 if(signal[i]>0) { closePrice<-as.double(Cl(SPY[currentDate])) unitSize = as.numeric(trunc((equity/closePrice))) # 5$ per transaction addTxn(ltportfolio, Symbol=symbols[1], TxnDate=currentDate, TxnPrice=closePrice, TxnQty = -unitSize , TxnFees=5, verbose=T) } } else { #position is open. If signal is 0 - close it. if(as.double(signal[i])==0 && position<0) { position = getPosQty(ltportfolio, Symbol=symbols[1], Date=currentDate) closePrice<-as.double((Cl(SPY[currentDate])))#as.double(get(symbols[1])[i+100]) # 5$ per transaction addTxn(ltportfolio, Symbol=symbols[1], TxnDate=currentDate, TxnPrice=closePrice, TxnQty = -position , TxnFees=5, verbose=T) } } updatePortf(ltportfolio, Dates = currentDate) updateAcct(ltaccount, Dates = currentDate) updateEndEq(ltaccount, Dates = currentDate) equity = getEndEq(ltaccount, currentDate) } plot(getAccount(ltaccount)[["TOTAL"]]$End.Eq) |
The result (it will take time to get it…):
Brian G. Peterson said,
October 11, 2010 @ 13:41
Note that newer versions of blotter have vectorized all calculations possible, and are now ~400x+ faster, per our testing.
For developing trading strategies, package quantstrat should hide much of the complexity of the trading P&L calculations, especially over large numbers of instruments.
Regards,
– Brian
none said,
October 23, 2010 @ 11:47
for those who experimented execution error with above script, please find a working version below…
require(xts)
require(quantmod)
require(blotter)
rm(list=ls(envir=.blotter),envir=.blotter)
try(rm(“account.longtrend”,”portfolio.longtrend”,pos=.blotter),silent=TRUE)
try(rm(“ltaccount”,”ltportfolio”,”ClosePrice”,”CurrentDate”,”equity”,”GSPC”,”i”,”initDate”,”initEq”,”Posn”,”UnitSize”,”verbose”),silent=TRUE)
Sys.setenv(TZ=”GMT”)
initDate=’2000-01-01′
initEq=100000
ltportfolio=’longtrend’
ltaccount=’longtrend’
symbols = c(“SPY”)
for(symbol in symbols){ # establish tradable instruments
stock(symbol, currency=”USD”,multiplier=1)
}
getSymbols(symbols,from=initDate,index.class=c(“POSIXt”,”POSIXct”))
SPY<-adjustOHLC(SPY,use.Adjusted=T)
#yesterday's price
tmp<-lag(SPY,1)
# if today's low is higher than yesterday's close 1, else 0
signalCl(tmp),1,0)
signal[1]<-0
#let's plot cumulitative return to make sure, that we are on the right path
tmp<-lag(Delt(Cl(SPY)),-1)
tmp[length(tmp)]<-0
X11()
plot(cumprod(signal*tmp+1))
SPY<-Cl(SPY)
initPortf(ltportfolio,symbols, initDate=initDate)
initAcct(ltaccount,portfolios=c(ltportfolio), initDate=initDate,initEq=initEq)
signal0
if(signal[i]>0)
{
closePrice<-as.double(Cl(SPY[currentDate]))
unitSize = as.numeric(trunc((equity/closePrice)))
# 5$ per transaction
addTxn(ltportfolio, Symbol=symbols[1], TxnDate=currentDate, TxnPrice=closePrice, TxnQty = -unitSize , TxnFees=5, verbose=F)
updatePortf(ltportfolio, Dates = currentDate)
updateAcct(ltaccount, Dates = currentDate)
updateEndEq(ltaccount, Dates = currentDate)
equity = getEndEq(ltaccount, currentDate)
}
}
else
{
#position is open. If signal is 0 – close it.
if(as.double(signal[i])==0 & position<0)
{
position = getPosQty(ltportfolio, Symbol=symbols[1], Date=currentDate)
closePrice<-as.double((Cl(SPY[currentDate])))#as.double(get(symbols[1])[i+100])
# 5$ per transaction
addTxn(ltportfolio, Symbol=symbols[1], TxnDate=currentDate, TxnPrice=closePrice, TxnQty = -position , TxnFees=5, verbose=F)
updatePortf(ltportfolio, Dates = currentDate)
updateAcct(ltaccount, Dates = currentDate)
updateEndEq(ltaccount, Dates = currentDate)
equity = getEndEq(ltaccount, currentDate)
}
}
}
X11()
plot(getAccount(ltaccount)$summary$End.Eq)
none said,
October 23, 2010 @ 12:13
sorry i forgot ONE LINE : currency(“USD”)
Please find a new version below :
require(blotter)
rm(list=ls(envir=.blotter),envir=.blotter)
try(rm(“account.longtrend”,”portfolio.longtrend”,pos=.blotter),silent=TRUE)
try(rm(“ltaccount”,”ltportfolio”,”ClosePrice”,”CurrentDate”,”equity”,”GSPC”,”i”,”initDate”,”initEq”,”Posn”,”UnitSize”,”verbose”),silent=TRUE)
Sys.setenv(TZ=”GMT”)
initDate=’2000-01-01′
initEq=100000
ltportfolio=’longtrend’
ltaccount=’longtrend’
currency(“USD”)
symbols = c(“SPY”)
for(symbol in symbols){ # establish tradable instruments
stock(symbol, currency=”USD”,multiplier=1)
}
getSymbols(symbols,from=initDate,index.class=c(“POSIXt”,”POSIXct”))
SPY<-adjustOHLC(SPY,use.Adjusted=T)
#yesterday's price
tmp<-lag(SPY,1)
# if today's low is higher than yesterday's close 1, else 0
signalCl(tmp),1,0)
signal[1]<-0
#let's plot cumulitative return to make sure, that we are on the right path
tmp<-lag(Delt(Cl(SPY)),-1)
tmp[length(tmp)]<-0
X11()
plot(cumprod(signal*tmp+1))
SPY<-Cl(SPY)
initPortf(ltportfolio,symbols, initDate=initDate)
initAcct(ltaccount,portfolios=c(ltportfolio), initDate=initDate,initEq=initEq)
signal0
if(signal[i]>0)
{
closePrice<-as.double(Cl(SPY[currentDate]))
unitSize = as.numeric(trunc((equity/closePrice)))
# 5$ per transaction
addTxn(ltportfolio, Symbol=symbols[1], TxnDate=currentDate, TxnPrice=closePrice, TxnQty = -unitSize , TxnFees=5, verbose=F)
updatePortf(ltportfolio, Dates = currentDate)
updateAcct(ltaccount, Dates = currentDate)
updateEndEq(ltaccount, Dates = currentDate)
equity = getEndEq(ltaccount, currentDate)
}
}
else
{
#position is open. If signal is 0 – close it.
if(as.double(signal[i])==0 & position<0)
{
position = getPosQty(ltportfolio, Symbol=symbols[1], Date=currentDate)
closePrice<-as.double((Cl(SPY[currentDate])))#as.double(get(symbols[1])[i+100])
# 5$ per transaction
addTxn(ltportfolio, Symbol=symbols[1], TxnDate=currentDate, TxnPrice=closePrice, TxnQty = -position , TxnFees=5, verbose=F)
updatePortf(ltportfolio, Dates = currentDate)
updateAcct(ltaccount, Dates = currentDate)
updateEndEq(ltaccount, Dates = currentDate)
equity = getEndEq(ltaccount, currentDate)
}
}
# updatePortf(ltportfolio, Dates = currentDate)
# updateAcct(ltaccount, Dates = currentDate)
# updateEndEq(ltaccount, Dates = currentDate)
# equity = getEndEq(ltaccount, currentDate)
}
X11()
plot(getAccount(ltaccount)$summary$End.Eq)
A Few Posts - Quantitative Finance - said,
December 8, 2010 @ 19:00
[...] and bid/ask spread cost into your trading strategy backtesting performance by R Blotter package, http://www.investuotojas.eu/?p=365.Keep warm. Tags – blog … reading the full post… You may also interested into other posts: [...]
Alexey said,
December 13, 2010 @ 23:26
Do you know any approaches to simulating slipperage realistically? For example, adding slippage more during high volatility? Or maybe by collecting some statistics from a specific broker?
And thanks for great posts. Keep it up!
kafka said,
December 15, 2010 @ 10:34
Dear Alexey,
I didn’t spend a lot of time tweaking slippage problem, so I’m not right person to ask such question. I think the problem differs from one instrument to another. For example, I don’t think slippage is the problem for equity indexes, like ES, NQ – these are liquid instruments (up to some size of course). In this case simple volatility incorporation could be right solution.
But slippage can be issue for stocks and forex. Some stocks have liquidity issues, so in that case, for testing purpose I would recommend at least to check market depth and
The different story is for Forex, where slippage problem differs from one broker to another.
I would recommend to check ssrn.com or other academic repositories – maybe there are some studies concerning slippage problem.
Brian G. Peterson said,
December 15, 2010 @ 14:15
Kafka is correct that true, nuanced, modeling of slippage needs to model queue position. Doing this requires a level of data management that I have only rarely found to be useful unless you are working in an environment of true market makeing where you’re trying to make thousands of round trips per instrument per day, and be very very close ot the top of the book at all times.
There are two less nuanced methods that we’ve chosen to model instead.
The first, and simplest, is the ‘delay’ parameter modeled in quantstrat. This allows you to combine things like decision delay, wirespeed delay, and queue position into one parameter that delays the time between when you see a market state, generate a signal, and enter an order. This delay effectively introduces slippage if you are using any order type, as the market state will have changed before the order is ‘live’ and could be turned into a transaction. Explicit modeling of these delays, and the separation between orders and transactions, in practice leads to much more accurate backtests. It also deals explicitly with the backtesting problem of assuming perfect execution, which is all too common in ad-hoc, published, and commercial backtests.
The second is to model slippage using some random or deterministic chance that your order will not turn into a transaction. This type of model is widely used in commercial backtesting systems, but has several problems. If the slippage is random, it requires that you run the same backtest many times to get a distribution of potential outcomes. It may not correctly model the notion that if the market passes through your order, you *would* get filled. If the slippage is deterministic, it could be more easily modeled by a haircut applied after the simulation.
In practice, I think modeling delay plus some haircut or scepticism decrease is sufficient for almost all strategies. If you have the problem of true high frequency market making activities, then you probably have a more nuanced view of this, but for most things, simple is better.
Kafka is also correct that there are academic papers on measuring slippage. These may form the basis for a haircut approach as I mention above.