Before I give some explanation of how to interpret the above numbers, I want to give a huge thanks to BHH from IBD Index. Due to his help and expertise, my understanding of system trading and backtesting has increased tremendously. He has been a co-developer of this current system (and others), and is responsible for all of the very cool Excel reports and charts that I will be posting. His blog has not recently had any of his fantastic reports on system testing and backtesting (probably due to how busy we’ve been working on systems), but the archives are must-reads.
The above report shows the statistics for trading this system using variable stop levels and variable percent risk, with commission and slippage included. (Actually, I didn’t add in any slippage). The dates of the testing run from July 15, 2006 to October 3rd, 2008.
To be clear, here is how the position sizing works for this system. Assume a starting equity of $100,000.
4% Stops
- Maximum of 4 positions means each position is allotted 25% of equity. The first positions would then start at 25K
- Maximum of 2 new entries per day.
- This ensures only 1%Â per position is at risk as $25,000*.04 = $1,000
6% Stops
- Maximum of 6 positions means each position is allotted 16.66% of equity. The first positions would then start at 16.6K
- Maximum of 3 new entries per day.
- This ensures only 1%Â per position is at risk as $16,666*.06 = $1,000
8% Stops
- Maximum of 8 positions means each position is allotted 12.5% of equity. The first positions would then start at 12.5K
- Maximum of 4 new entries per day.
- This ensures only 1%Â per position is at risk as $12,500*.08 = $1,000
Position sizing / money management will have a huge effect on any system and should be the top priority for testing, after the entry and exits. Viewing the system in these terms allows the trader to decide the optimum level of risk so that he may meet his goals while still being able to adhere rigidly to the rules. A system with huge profits and huge drawdowns may be just as bad as a system that does not generate high enough returns.
Z-Score: Read a primer on Z-Score here: Profiting from the Z Score, and here: Z Score
As this system has a Z-Score ranging between -2.615 and -4.730, we can assume with a high degree of certainty (at a confidence level of greater than 99%) that streaks longer than would be expected in a random distribution may occur while trading this system. This includes winning and losing streaks, but the Win% shows that the winning streaks are longer than the losers. To be clear, once this system starts hitting winning trades, it will continue to win. Similarly, once a losing trade or two happens, we know that it is more likely to continue losing than it is to start a new winning streak. This is why proper position sizing is of the utmost importance. I like 1% risk because that allows me to weather 10 losing trades in a row with only a ~10% drawdown.
Peak Drawdown: The peak drawdown on this system is currently very small.
CAGR: Compound Annual Growth Rate on this system is promising. The system stayed near flat for more than the first year of testing (equity curve to follow in another post). It should be noted that the YTD CAGR on the 4% stop system is currently running at ~95%.
Now let’s assume we want to trade this system more aggressively, using 1.5% risk ($1,500) per trade. Keep in mind that if 10 trades stop out in a row, the draw down will now be at least 15%.
Here are the results:
Note that I added .10% of slippage per trade, as well as increasing the percent risked per trade. CAGR is now a very respectable 45.13% on the 4% system.
Also note that the peak drawdown is now -14.75% (and since the future is often very different from the past, we should assume the next drawdown will be even larger).
A Final Note about Adapting the System to Market Conditions
If we know the system is prone to long winning and losing streaks, what if percent risk were increased with each winning trade, and then reduced on each successive loser? Sure, the first losing trade after a long streak of winners would be for a large loss, but that might be mitigated by the pyramid of profits before that loser hit. And then, each loser would get successively smaller, until another winner occurs.
We have done some preliminary studies on this topic, but have not yet developed a robust method for applying this strategy of adjustable percent-at-risk.
Up Next:
More really cool graphs, including equity curves of each stop level, trade distributions, and a chart of the drawdowns.
Hi Woodshedder,
Thank you for your super informative posts! I’m fairly new to iBC (and short-term trading) and hadn’t read much about technical analysis before that. Your posts have been really helpful, and technical analysis (albeit my novice understanding of it) has been an important new addition to my trading strategy.
I had a question: Since your sell signals don’t come till after the market closes, how do you think the system would be affected if a 4% trailing stop was used instead? Also, in the case of a huge upward spike mid-day (i.e., before your nightly sell signal) followed by a drop, is there some kind of best-case sell threshold indicator that could be used rather than always having to wait till after the market has closed?
Thanks!
Michael, great questions.
I am going from memory here with the trailing stop issue…I don’t recall that we ever tested a 4% trailing stop. It might work well. My personal opinion on the issue is this: I am a very short-term trader. The shorter I’m in a trade, the better. I’m not interested in riding the trade as long as possible, in order to rack up huge gains. I want to get in, get my 3% or so, and be out.
As for mid-day signals…You could certainly set a limit order to exit at RSI2 of 80. I know that both Tradestation and Amibroker will calculate where the price will be for a particular RSI2 setting. BHH did test using limit orders for exits, which would include intra-day exits. The results showed that market orders, on the open, the morning after the signal was generated out performed limit orders. Here is the reason why…Many times, these trades get closed out at an RSI2 of much greater than 80. Sometimes, 85, sometimes 90, sometimes higher. If you are closing them out religiously at 80, you will be giving away some gains.
It turns out, as we continue to tweak and test variables in this New ETF Trading System that an RSI2 exit of 80 or greater may not be the best exit. I’ll be posting more on that later.
We can always optimize and develop a best-case sell threshold indicator, but I still believe that having it trigger intraday will cause it to underperform an End-of-Day exit.
I absolutely can’t trade intra-day, and therefore, the systems I design will always be tweaked to perform better using end-of-day signals.
Important note: BHH reminded me that if I raise the percent risk above 1%, I am technically going to run out of money or have to go on margin. If I am trading at 1.5%, I am only going to be able to afford 3 max positions. I will still see an improvement in total equity but I won’t be able to take the same number of trades.
Wood,
I have some questions regarding your useage of Z-scores.
First off I agree that stock prices, or any prices in financial markets are not a “random walk” viz. independent observations. Thus are trends accounted for.
This does not seem to be what you are stating here however…..
“To be clear, once this system starts hitting winning trades, it will continue to win. Similarly, once a losing trade or two happens, we know that it is more likely to continue losing than it is to start a new winning streak. This is why proper position sizing is of the utmost importance. I like 1% risk because that allows me to weather 10 losing trades in a row with only a ~10% drawdown.”
I disagree that the Z-score is an appropriate measure in this context [winning trades]
Second, Z-scores are calculated as part of a Gaussian distribution. As stock prices are non-Gaussian, again, is this an appropriate statistic?
Third point.
As this system is an overnight hold, you are assuming liquidity is unchanged. This is a dangerous assumption, and confuses liquidity with marketability.
The system is using ETF’s many of which have a very short history for data.
The current crisis in the credit markets has been in part a liquidity crisis….viz liquidity was a myth, all that existed was limited marketability.
Thus your slippage allowances look a tad low. If you changed them, would that alter the tested results?
jog on
duc
Wood, what are you doing man? You can’t just give up one of the best blogs out there. Damn it.
While I am more of an intermediate to longer term trader, that site helped tune my game. I never questioned my entry, exit, position sizing, holding periods before I got turned on to that site. It’s amazing how you can see by playing with variables how you can improve your system or disprove common myths that some authors put out about their work.
The archives are gold. Just go and read. Then go play with the data.
Good trading.
Hey Wood. I’m beginning to buy into your research. Here’s my question to you. I’m particularly interested in the RSI(2) System. I’ve been doing some research on it’s utility for individual stocks as well. Do you use a screener to find those stocks that have closed with an RSI(2) below 10, or do you have a watchlist that you just monitor for EOD data to catch the trigger? Thanks man.
Z-score is a perfectly appropriate measure of whether strings of wins and losses are what can be expected in a random distribution. What Z-score will determine is whether the distribution is non-random.
While stock prices do not conform to a Gaussian distribution, I am not running a Z-score on stock prices. The Z-score is being run on whether a trade is a winner, or a loser. Think coin flip. The high negative correlation of greater than a 99% confidence level shows that winners begat winners, and losers begat losers.
For example, flip a coin 132 times (the number of trades tested for the 8% stop) and tell me how many times within that 132 flips that you get 14 heads in a row, and then later, 15 heads in a row (heads being winners). Sure, if you repeat the experiment enough times, eventually within that 132 flips, you’ll be able to replicate 14 heads in a row, and then another streak of 15 heads in a row. The point is, such long streaks are not random.
I have been trading these ETFs almost exclusively for a year. Scottrade, almost without exception, will give the opening price. Since the system has been tested using opening prices, I do not see slippage being a huge issue, unless the opening price is skewed higher by a large volume of buy orders from traders all trying to trade this system.
Even slippage of .5% on every trade, which has never been observed in real-time, yields a CAGR of 23%. Again, let me re-emphasize that the opening price is being used for testing. Even if the price closed higher the day before, and gapped down the morning of our entry or exit, that type of movement is already factored into the equation. We expect that a lot of exits will have been triggered with an RSI2 over 80, but in actuality, will be recorded with an RSI2 of under 80, due to a gap down. There is a rather large zone of profitability with RSI2. Yes, 80 is optimum for some systems, but 70 will work too, as will 90.
Many of these ETFs are trading well over 5 million shares a day. I really do not see how liquidity is an issue.
Stock Speculator, his blog is pure gold. Sorry to open it up to the leeches, but I had to give him props, some how. He probably would have objected had I instead provided the address of his home, and his phone number 😉
Junk Spread, you can do it either way: screen over a watchlist or portfolio, or screen over the entire market.
I use Stockfetcher for both, and I only use EOD data.
I made an error. There were actually 196 trades tested on the 8% stop. I was looking at only one page. STill the 14 winners and 15 winners in a row is accurate.
Wood. I posted the same question on your Covester blog. Please disregard.
Wood. I didn’t see what program you use to do your backtesting?
Junk, a combination of Stockfetcher, Trader’s Studio, Tradestation, and Excel.
Great stuff Woodshedder, great stuff. This Double Trouble system is great reading! Have you profiled the average losing streak to see on average how many losses you have before it turns around? Maybe one option to scaling up the winners would be to sit out once a losing streak starts and “miss” the first winner on a turn-around? Or as a losing streak starts maybe tighten up the stop/greatly reduce the position size? So many options to try, so little free time.
Too bad you couldn’t trade intraday, it’d be interesting to see how this does on 60/30 minute timeframes.
Icha, I just scanned through the trade-by-trade data for the system. The average losing streak is around 3-4 trades, but the longer streaks are running around 5-6. There are several times when it loses 5 times, has a winner or two, and then has 5 more losers.
Your thinking is very similar to mine and BHH’s. We have been looking at trying to implement exactly what you describe.
Here is the problem. Lets say you have the max of 4 positions on, and 1 of the four loses. Now what do you do with the rest? Now 2 of the four are losers. Should you sell off some shares of the remaining 2 positions to reduce risk?
I’m thinking the best way is to reduce position size once a loser occurs. Any existing positions would be maintained at their current risk level, but new positions would be taken with smaller position sizes.
Another way to handle it would be to ease the exit criteria. For example, if RSI2>80 was your exit, if a loser or two happens, easing the exit down to RSI2>70 may also help, until winners begin again.
Yeah that’s pretty tough. My unscientific bias leans toward leaving the current positions actual size alone too. It would be interesting to see when a losing streak starts, how many positions are on and how many turn into losers that are already on. Maybe if once 2 losers hit the other current positions usually fail, if they are currently profitable maybe do what you suggest and also raise the stop to break-even.
I know your entry criteria is top secret, but do you have any links you recommend that talk more about different entry signals? I’ve been toying with Ameritrades StrategyDesk a bit, lots to learn that’s for sure. I need to put the time in to read BHH’s blog too, thanks for all of the links!
Thanks for the reply! I’m looking forward to your next post.