I want to introduce to all of you a statistical measure which has come to have a significant influence on the trading systems I am developing. Ralph Vince describes the measure very well in his book Portfolio Management Formulas. This measure is called the Z-Score, or the Runs Test.
I want to skip most of the statistical jargon and get right to the meat of the issue, but I will be happy to answer specifics in the comment section.
To understand why the Z Score or Runs Test is important, we need to digest Rob Hanna’s statement from his recent post How to Trade the Choppiest Environment in 50 Years. Rob writes, “As you can see, buying after strong days and selling after weak ones worked well for 40 years. In 2000 that changed, and the last year and a half is the worst it has ever been with regards to follow through. This would suggest that strategies that may have worked well for forty years or more could be suffering greatly now.”
Also, it is important to read Dr. Brett Steenbarger’s recent post Short-Term Reversal Patterns Among Global Equity Indexes.
Both authors conclude that short-term trend following is not working very well. We can test their conclusion by applying the Z-Score to the data from the indices. I should mention that both Bhh from IBDIndex and Damian from Skill Analytics have been instrumental in helping me flesh out the rest of the ideas presented below.
The Z-Score can determine whether wins or losses are dependent on previous wins or losses. Think of dependency in this way: Do wins begat more wins? Do losses begat losses? If so, this relationship would be described as a positive dependency. What if wins begat losers, and losers begat wins? This would be a negative dependency.
While Z-Score has traditionally been used to analyze the win and loss streaks of a trading systems, it seems that another application for the measure may be to analyze the win and loss streaks of the indices in order to determine whether there is any dependency. Are the sequences of wins and losses containing more or less streaks (of wins and losses) than would be expected in a truly random sequence? When digesting this, consider the fair coin, where one flip is equally as likely to be heads as it is tails. We want to determine if the indexes are trading as a fair coin, or one that is biased to heads or tails, or both.
Below are the Z-Scores for the S&P 500 (SPX), using all data available from yahoo, which goes back to 1950. In January of 1993, the S&P 500 SPDRs was introduced. I will quit using SPX data and use SPY data from 1993 forward.
All Data, 1950 to Present: Z-Score -9.359014
This negative Z-Score implies a positive dependency at a confidence level of much higher than 99.73%. In short, a positive close on the S&P 500 generally begat more positive closes, and losing days generally begat more losing days, over this broad time span.
1960 to Present: -7.196132
1970 to Present: -3.795268
Note that the positive dependency is decreasing, yet from 1970-Present, the confidence level is still above 99.73%.
1980 to Present: Z-Score .3465443
1990 to Present: Z-Score 1.692151
1993 to Present (With SPY Data): Z-Score 2.115444
Note that there has been a switch. The positive Z-Score implies a negative dependency, where buying begats selling, and selling begats buying. Be careful though with this data, as the score must be above 1.64 to have a confidence level of greater than 90%.
2000 to Present: Z-Score 1.3608623
2003 to Present: Z-Score 1.1696094
2006 to Present: Z-Score .4530397
2007 to Present: Z-Score 1.3030246
October 2007 to Present: Z-Score .3994188
Note that from 2000 on, the Z-Scores move lower. The highest score from this period, 1.303, gives a little better than an 80% confidence level. I interpret this data to mean that the S&P 500 is basically moving through a random walk, although the confidence level is not high enough to draw any firm conclusions.
January 2008 to Present: Z-Score -0.628093
February 2008 to Present: Z-Score -0.400456
March 2008 to Present: Z-Score -0.246511
April 2008 to Present: Z-Score -0.403907
May 2008 to Present: Z-Score -0.288564
June 2008 to Present: Z-Score 0.0022265
July 2008 to Present: Z-Score -0.09631
From January 2008 to the present, we begin to once again see negative Z-Scores. A negative score implies a positive dependency, where selling begats selling and buying begats buying. The scores are not significant enough to exhibit a high level of confidence.
Conclusions
The recent data show no definitive dependency, either positive or negative. This means that buying because the market has closed up or selling because it has gone down has not been working as well as in the past. Also, buying weakness or selling strength, in order to catch a reversal, has not been working as well either.
Right now, betting on the market, as represented through the SPY, is similar to betting on the flip of a fair coin. This data, while it may not prove the conclusion of Hanna and Steenbarger, certainly does not disprove it.
Implications for Further Research
It seems to me that keeping shorter and intermediate time frame of Z-Scores, update daily across the indices, could give the trader a head’s up that market conditions may be changing to be more favorable to trend-following or contrarian strategies.
This article was tits, although I am “disappointed” at the lack of solid conclusion. Thats the data’s fault though obviously not yours.
So, strength in the indices follows through at a low confidence level, but what about individual stocks breaking out? Can you just plop em in your excel(?) sheet? That would be interesting. I suspect I already know the answer to that one though.
Does the use of a z-score imply a normal dist. in the stock market?
Danny, the Z-score does imply a normal distribution. Let me give you Vince’s quote about that.
“The first is that futures trading systems trades (i.e., the profits and losses) do not necessarily conform to a Normal Probability Distribution. Rather, they conform pretty much to whatever the distribution is that futures prices conform to, which is as yet undetermined. Since the runs test assume a Normal Probability Distribution, the tuns test is only as accurate as the degree to which the system trade P&Ls conform to the Normal Probability Distribution.”
Wow Danny, I am impressed at how quickly you headed “there” 😉
Nice post Wood.
Woodshedder,
I think I remember having read something by Vince and hating it :), but I like looking at serial correlation and related issues. However, I am wary of using the assumptions that linear statistics need, like normal distribution. This is why I used the approach given in my reply to Brett’s recent post, to filter for “significant moves”.
(I do have a quite extensive statistical education, and I would say it has helped me a lot in my trading NOT to employ most of that.)
Even though I think the concept is weak, I think there still is informational value in it. I would, however, rather use it to compare between different markets / parts of the market than as a look on the market as a whole: Wouldn’t it be interesting to see which _areas_ of the market are more positively correlated than the broad market?
If this has some stability over time(!), it surely would lend to exploitable edges.
Nico
Im surprised at the positive score through Oct. 07.
Would you mind posting these once a month Shed?
I would be reluctant to lean towards a random walk conclusion unless I saw a Z score > 2.57
Wood,
I enjoyed this post tremendously, it made me think.
Having thought about it overnight, I’ve crafted a response that I’ve posted due to the length of the reply.
Excellent post.
jog on
duc
Duc, you are truly a first class link whore. Why not just post your carefully crafted 2 paragraph response (that took you all night to think about) here?
p.s. Get rid of the gay snoopy picture, unless that’s the point.
Woodshedder,
Nice to see some additional research on the subject.
Without getting into a lengthy debate or independently running the numbers, I would suggest this is an instance where the z-score testing is failing to reflect reality. No doubt part of this is a difference in testing procedures and another part is the fact that z-score tests only look at direction and not magnitude of a move.
The high z-score reading from 1993-present suggesting a high negative correlation over the time period is somewhat misleading. During 1993-2000 trendiness prevailed greatly.
Your 2000-present z-score is lower then ’93-present, yet the market rewarded chop over trend by a large amount during that period.
This can be seen by examining the chart in my original post.
In 2008 you show a negative z-score, suggesting positive dependency. So far this year, buying strength and holding through the next down day, and shorting weakness and holding through the next up day would have resulted in under 28% winning trades. This is the 2nd worst year since 1960. The worst was 2007.
I’ll be discussing and expanding on some of the stats from the study later this week.
In addition to the z-score not considering magnitude, you may be looking at streaks differently. For instance, three days up that get wiped out in 1 day would result in a losing trade in my system. In yours I believe it would be 2 wins and 1 loss (buys after day 1 and 2 are profitable – buy after day 3 is a loser). Apples and oranges when you apply a z-score.
Best,
Rob Hanna
Technical analysis is like coin flipping:
It works perfectly. Until the day it doesn’t.
Great post, Wood.
Great comments on this thread. Except for antons, which was gibberish.
danni, not “getting it,” apparently in any aspect of your trailer park life, is what you seem to do best.
If you want useless “gibberish,” read that stain you call a blog.
You are a joke. And a poor one.
Why the hostility Anton?
Rob, glad to see you over here.
I agree with the majority of your points, but I want to add a few additional thoughts.
First, it is my suspicion (anecdotal evidence only) that a high negative correlation can still lead to trends. If traders adapt well to buying leading to selling and selling leading to buying, a strong trend can still emerge. I see confusion being the biggest reason for lack of trending, and if traders feel they can understand the markets actions and reactions, there is still plenty of reason to buy and sell in hopes of following a trend.
The 2000 to present Z-score seems to me to suggest chop over trend. The low Z-Score offer no confidence that there is any follow through, only chop.
The positive dependency that is suggested in 2008 offers very little in the way of confidence as the Z-Score is so low. Again, I read it as suggesting that a change is underway, but it is in no way statistically significant in terms of the Z-Score.
Z-Score does look at streaks differently. For example, 4 winning days in a row, or WWWW (Vince uses ++++) would be counted as 1 streak. a WLWL (+-+-) would be 4 streaks.
Anton, I was “just kidding,” but, since you spelled my name like a fag and have way to thin of skin for the internet, fuck you.
Insulting my blog is akin to calling Fly an idiot, because he hired me. People seem to forget that I am the supreme weekend janitor of all time. Shed tried to carry on in my stead and failed. That is definitely not a knock against him, just historical record from before the iBC era.
Re my blog, I know it’s awesome on all fronts and I don’t need the validation from you.
A joke? A Poor one? ooooh ouch. Yeah. Good one. Those insults were weak and drunk on pussyhol, so I won’t reply in kind.
Guaranteed I have more money than half the people here, but that is a semi-retarded thing to brag about. Why would that matter?
Finally lets address your statement:
TA works until is doesn’t.
That is not true, and nonsensical, which was why I poked fun in the first place.
“The sun rises, until it doesn’t”
Additionally, one might argue the POINT of TA is to reduce the coin flip aspect.
Fortunately for me, I just work here, you come here, so don’t mind me if I spend more time bugging you since I know how easy it is.
Take my advice Anton: Grow up.
I could care less, it’s up to you if you carry on a meaningless grudge.
Interesting post.
Not to be too picky but I wrote about Z-Scores on March 30-08
http://www.ibankcoin.com/peanut_gallery/index.php/2008/03/30/dissecting-dinosaur-trader-information-theory-part-0/
and again on April 4-08 when I advocated a similar method:
http://www.ibankcoin.com/peanut_gallery/index.php/2008/04/04/dissecting-the-dow-jones-indu/
whereby I attempted to sketch out a means of using the Z-Score as a technique to swing trade.
Nice to know I wasn’t the first to think it up.
danny, migraine here. re-reading it, i’m amazed at the vileness of my own post. your retarded jibe just pissed me off today. normally i’d just ignore you. i stand by the “not getting it” stuff. the bit about ta “working until it doesn’t” is not my observation, but an old saying about ta. and, in this context, a joke re coin flipping in wood’s headline. all the bile spewed, however, is beneath me and should not have been hurled at you and i do apologize. but talk about thin skin. dissing you is not the same as calling the fly an idiot, either. i’m not the one who sent the neg karma to your post above, btw.
I didn’t know we had moderated forums:
This is from a comment I posted a few minutes ago
“cuervoslaugh Says: Your comment is awaiting moderation.”
dsb, sorry for that. totally my bad.
wellll…. I like both Anton and Danny, and IMO you both have an extra shot of testosterone. Guys will fight, no doubt about it (testosterone explosion!). Fortunately there were no casualties and no arrests, so hopefully it’s calmed down enough we can go forward and Shed can get back to talking ’bout z-score patterns.
Seriously, there must be something wrong with me, I’m fond of you all, my online dysfunctional buds.
Yeah Anton, I really meant nothing by it.
The problem with internet is that it’s difficult (impossible at times) to detect some of the nuances of language or intonation that would identify a comment as made in jest.
Obviously Fly isn’t an idiot, and it was stupid to insinuate that someone who doesn’t like my blog thinks so. That was probably an unintentionally baiting remark.
Anton, I respect your non-emotional response, I’ll buy you a white rhino the next time I bump into you at the bar.
Cuervo, I have my blog set up to require moderation of any comment with 2 or more links embedded. Keeps the spam out.
I went back an re-read your posts about z scores. I left you a comment on one of the posts. I’m confused exactly what you are doing and what is triggering what, and after a trigger occurs, what is supposed to be happening.
Danny, Re-re-reading the above makes me feel even more of an ass. Nuanced humor, bludgeoning wit and finding new and different ways to bank coin are the reasons I keep coming back here. No excuse, but when these things come on my vision is skewed. So it’s a reading comprehension problem, not a nuanced humor transmission problem. But if there is blame to be placed, I think we can both agree that it should be placed on Boca. My testosterone is always elevated when she’s around. Garcon! White rhinos and high quality pain meds for everyone! Roses for the lady.
Woodshedder,
We’re nearly on the same page, but not quite. Mr. Vince’s ++–++ stuff refers to streaks of winning and losing TRADES – not winning and losing days. So, as I wrote my study, the trade is closed and reversed when there is a move in the opposite direction. For instance 4 up days is not a winning trade. It is an open trade. If day 5 closes down that will close out the trade. If day 5 is down slightly, you’ve just closed a winning trade. If day 5 wipes out all the gains of the previous streak (4 days), you’ve just closed a losing trade. In either case you’ve also just opened a short position.
You can debate whether my approach to measuring chop was appropriate. There are certainly other, and perhaps better ways to do it. It depends how you would like to measure the market. Look at the last few days in the S&P. Today wiped out the gains of the last two days. The trend following system would have entered on the close of the 20th when the market turned up. If you booked profits each night and then re-bought you could say you made money on 8/21 and 8/22 and then lost today. For me it would count as 1 trade that ended badly today. Either way, the hit to the wallet is the same (aside from commissions and slippage).
It should also be noted this was not designed as a tradable system (and its an unlikely candidate to achieve a high z-score). It was a study of market activity to see whether the market has followed through in a way that would have allowed a trader to capture profits, or whether they were more likely to be punished when chasing moves. The results showed that chasing the move was the way to go for a long time. Over the last several years it has ceased to work. Betting against the follow-through would have provided higher profits.
Regards,
Rob
Wood: the original idea had been that when the standard deviation moved past 2D or there abouts then one should expect a move of smaller advancement/decline or (in many cases) a complete reversal.
For example, the market has been making moves of 20,50, -40,-10 and suddenly there is a day or two of 250+ moves. At that point, one should expect (much as your RSI(2) pattern would show) a reversal or at least several movements less than 250.
The method of calculation is to treat the difference between today’s value and yesterdays value as an “earning” and then taking the normal distribution of the Z-Score of the Standard Deviation of the point difference one arrives at a score between 0-.99. At any point in time where the value was more than .7 it was a ‘bear flag’ meaning the index would reverse. When it was less than .3 – that would be a ‘bull flag’.
Duc will probably come by and point out that the market doesn’t work as a Gaussian distribution (i.e. the bell curve) but, using the Z-Score of the Standard deviation of the previous seven dates worked at the time of writing.
My guess is that while Duc and Taleb are right on the ball about the market following a lognormal distribution, it would appear that the Standard deviation falls within a normal distribution.
In fact, I am fairly certain of it considering the results of some of my researching the market with regards to linear correlations and extreme value theory.
Is that any more clear?
Check Rob’s reply Wood.
I think he and I are using different words to say the same thing.
Rob is saying that Duc’s comment http://www.ibankcoin.com/peanut_gallery/index.php/2008/03/30/dissecting-dinosaur-trader-information-theory-part-0/#comment-7535
was wrong.
While the market is non-linear, the earnings tend to follow a normal distribution.
I quote Rob’s comment:
Look at the last few days in the S&P. Today wiped out the gains of the last two days.
Rob, I’m with you completely as to the differences between your study and mine. I thought it would be interesting to apply the Z-Score, which has typically been applied to sequences of winning and losing trades, to the market as a whole, where each day is treated as a winning or losing trade.
It sounds like you might think I disagree with your results and method of testing, and I absolutely do not. Again, I was looking for another way to describe mathematically the changing market behavior.
Finally, my tests confirm that there has been little follow-through, and that prices over the last 8 years have been showing decreasing levels of dependence, both positive and negative. If there is no dependence, what we have is chop, as each day is as likely to be up as it is to be down.
So, to be clear, I think the majority of the Z-Score results confirm your results about choppiness.
Cuervo- You wrote: “While the market is non-linear, the earnings tend to follow a normal distribution.”
Are you saying that the standard deviations will fall within a normal distribution, when the market is non-linear?
Very interesting. I’m not sure I understand, but I’m trying.
Anton, I will happily take any assigned blame for the aforementioned testosterone explosion. Cheers all.
(Shed, pardon my quick intrusion into the z-score discussion.)
Anton, you’re right, it’s definitely boca’s fault.
Wood: That’s it in a nutshell.
Boca, you are never an intrusion!
Cuervo, that is very very cool. I have to spend some time digesting that.
Let me try one last time because there was a lack of clarity last night:
Now if you do the normal distribution of the z-score on the Dow Jones closing figures, you’ll get gibberish. But, when you take the normal distribution of z-score of the movement differential ( 200,-400) you find that the ‘earnings’ seem to fall in a normal distribution over a period of five to seven trading days.
Daily movement of DJI = lognormal
Differential distance over seven periods = normal?
Two things seem to confirm this:
1. The reversal to mean tendency of the market means that when the differential distance over seven days reaches 3SD then predictably one will see a weakening of the movements (not reaching same climb) or probably a reversal.
2. Extreme Value Theory points to lognormal distributions having more than one distribution which over lap as exemplified by Peak over Threshold technique (EVT) which treats the outliers as if they were members of a separate distribution.
I did some initial testing on the S&P and while similar results were found it was interesting to note that this technique didn’t backtest very well past 2005 for the S&P.
Thinking on it now, I would suggest avoiding the index related equities themselves and try to find equities with the following:
1. High Correlation to the index (more than .8 or less than -.8)
2. High beta values
Just some morning thoughts.
Cuervo,
very nice statistical details. I love to read those “boring distribution details”, because you need to look that closely IMHO if at all costs one wants to apply linear statistics to markets.
Woodshedder,
one more thought: z-score is a measure of the statistical significance. By comparing z-scores of different intervals, you change the denominator: a VERY small significance may get a high z-score if measured over 20 years, while the same dependency might render no significance (z near 0) for something like 2 years. I think it is important to keep that in mind.
(Maybe such inaccuracies were the reason why I hated Vince when I read it way back. Don’t remember.)
Are any of you guys on Twitter? I’m under w0nk0.
W0nk0
Trend Following [types of] always make much more money in the long run. Less wins than overbought/oversold but more profit on winners. You can have an 80% win rate doing overbought/oversold and still not make money. You need to trade market flow with the right system to come out on top.
Markets only trend 20 to 30 percent of the time. So, you can either be content with long-term trend following only, or you can trade different strategies that work in different types of markets.
Trend followers make money. Swing traders make money. Day traders make money. It all works.
Bottom line, find what is right for you then stick to it.
Anton, Danny, glad to see everybody getting along again. Now, let’s get back to making fun of Tom Brown and Ducati – as it should be.
Dogwood;
You are correct. I was just talking about % of income for trade using trading systems over the long term. You need to build trading systems using complex ratios to switch between different strategies. The ones’s that work are not for sale!
Lots of good info.
Armed with it, I’m not sure it would have a snow cones chance in hell of changing my approach to the markets right now, but we all have to be open to new information.