This strategy is taken from Example 3.

3 in Ernie Chan's book, Algorithmic Tradin

g: Winning Strategies and Their Rationale. It's been posted here previously but
that used constants for the linear regression coefficients.
In this case, a Kalman filter is used to dynamically update the linear regressio
n coefficients between the EWA and EWC ETFs. It performs well up to 2009 but aft
er that the performance degrades.

import numpy as np
import pytz
def initialize(context):
context.ewa = sid(14516)
context.ewc = sid(14517)
context.delta = 0.0001
context.Vw = context.delta / (1 - context.delta) * np.eye(2)
context.Ve = 0.001
context.beta = np.zeros(2)
context.P = np.zeros((2, 2))
context.R = None
context.pos = None
context.day = None
set_slippage(slippage.FixedSlippage(spread=0))
set_commission(commission.PerShare(cost=0))
def handle_data(context, data):
exchange_time = get_datetime().astimezone(pytz.timezone('US/Eastern'))
# update Kalman filter and exectue a trade during the last 5 mins of trading
each day
if exchange_time.hour == 15 and exchange_time.minute >= 55:
# only execute this once per day
if context.day is not None and context.day == exchange_time.day:
return
context.day = exchange_time.day
x = np.asarray([data[context.ewa].price, 1.0]).reshape((1, 2))
y = data[context.ewc].price
# update Kalman
if context.R is
context.R =
else:
context.R =

filter with latest price

not None:
context.P + context.Vw
np.zeros((2, 2))

yhat = x.dot(context.beta)
Q = x.dot(context.R).dot(x.T) + context.Ve
sqrt_Q = np.sqrt(Q)
e = y - yhat
K = context.R.dot(x.T) / Q

context.beta = context.beta + K.flatten() * e

context.P = context.R - K * x.dot(context.R)
record(beta=context.beta[0], alpha=context.beta[1])
if e < 5:
record(spread=float(e), Q_upper=float(sqrt_Q), Q_lower=float(-sqrt_Q
))
if context.pos is not None:
if context.pos == 'long' and e > -sqrt_Q:
#log.info('closing long')
order_target(context.ewa, 0)
order_target(context.ewc, 0)
context.pos = None
elif context.pos == 'short' and e < sqrt_Q:
#log.info('closing short')
order_target(context.ewa, 0)
order_target(context.ewc, 0)
context.pos = None
if context.pos is None:
if e < -sqrt_Q:
#log.info('opening long')
order(context.ewc, 1000)
order(context.ewa, -1000 * context.beta[0])
context.pos = 'long'
elif e > sqrt_Q:
#log.info('opening short')
order(context.ewc, -1000)
order(context.ewa, 1000 * context.beta[0])
context.pos = 'short'

@Simon, it's easy to extend to n-dimensional linear regression by increasing the

dimensions of the observation matrix for the Kalman filter (variable x in the a
lgo). This backtest does a regression between SPY and the sector ETF's XLE, XLF
and XLI.
mport numpy as np
import pytz
def initialize(context):
context.y = sid(8554)
context.x = [sid(19655), sid(19656), sid(19657)]
context.n_dim_state = len(context.x) + 1
context.delta = 0.0001
context.Vw = context.delta / (1 - context.delta) * np.eye(context.n_dim_stat
e)
context.Ve = 0.001
context.beta = np.zeros(context.n_dim_state)
context.P = np.zeros((context.n_dim_state, context.n_dim_state))
context.R = None
context.pos = None
context.day = None

set_slippage(slippage.FixedSlippage(spread=0))
set_commission(commission.PerShare(cost=0))
def handle_data(context, data):
exchange_time = get_datetime().astimezone(pytz.timezone('US/Eastern'))
# update Kalman filter and exectue a trade during the last 5 mins of trading
each day
if exchange_time.hour == 15 and exchange_time.minute >= 55:
# only execute this once per day
if context.day is not None and context.day == exchange_time.day:
return
context.day = exchange_time.day
x = np.hstack([[data[sec].price for sec in context.x], 1.0]).reshape((1,
context.n_dim_state))
y = data[context.y].price
# update Kalman
if context.R is
context.R =
else:
context.R =

filter with latest price

not None:
context.P + context.Vw
np.zeros((context.n_dim_state, context.n_dim_state))

yhat = x.dot(context.beta)
Q = x.dot(context.R).dot(x.T) + context.Ve
sqrt_Q = np.sqrt(Q)
e = y - yhat
K = context.R.dot(x.T) / Q
context.beta = context.beta + K.flatten() * e
context.P = context.R - K * x.dot(context.R)
record(beta1=context.beta[0], beta2=context.beta[1], beta3=context.beta[
2], alpha=context.beta[-1])
#if e < 5:
#
record(spread=float(e), Q_upper=float(sqrt_Q), Q_lower=float(-sqrt_
Q))
if context.pos is not None:
if context.pos == 'long' and e > -sqrt_Q:
#log.info('closing long')
order_target(context.y, 0)
for sec in context.x:
order_target(sec, 0)
context.pos = None
elif context.pos == 'short' and e < sqrt_Q:
#log.info('closing short')
order_target(context.y, 0)
for sec in context.x:
order_target(sec, 0)
context.pos = None
if context.pos is None:
if e < -sqrt_Q:
#log.info('opening long')
order(context.y, 1000)
for i in xrange(len(context.x)):
order(context.x[i], -1000 * context.beta[i])

context.pos = 'long'
elif e > sqrt_Q:
#log.info('opening short')
order(context.y, -1000)
for i in xrange(len(context.x)):
order(context.x[i], 1000 * context.beta[i])
context.pos = 'short'

Aidan O'Mahony posted Mar 5, 2014

@Eric and @Jason, there is reasoning behind using a Kalman filter for this appli
cation!
First, let's look at what we are trying to achieve. We want to construct a stati
onary time-series (spread) from two individually non-stationary time-series (EWA
and EWC). Let's assume there exists a parameter (beta or cointegration coeffici
ent) such that the residual from the linear combination of the non-stationary ti
me-series is stationary and we can use the resulting stationary series to genera
te trading signals. In a perfect world, beta would be constant and our job would
be done.
Our problem now is that beta is not constant and changes gradually (assuming the
series stay cointegrated) over time. To use a Kalman filter, we need to formula
te an observation and transition equation. The observation equation is the linea
r combination of the non-stationary time-series plus Gaussian white noise. This
is really just a linear regression of EWA and EWC. The transition equation assum
es that beta is equal the previous beta plus Gaussian white noise. This is a ran
dom walk.
Using more complex non-linear models probably won't improve things, as Ernie Cha
n said on non-linear models "experiences showed me that nonlinear models have mo
stly been unmitigated disasters in terms of trading profits" and "One is almost
certain to overfit a nonlinear model to non-recurring noise".
Using the current setup of the algo, it trades frequently, almost every day, and
the profit from each trade is not sufficient to cover transaction costs. It tra
des frequently because the Kalman filter responds quickly to changes in the EWA/
EWC spread. Reducing the sensitivity of the Kalman filter will decrease trading
frequency and only trade larger deviations in the spread, which will increase th
e expected payoff. To answer your question, it only makes sense to use this stra
tegy, or any other strategy, when the expected payoff of a trade is greater than
transaction costs.