Clicking on the Tradestation Performance Summary will enlarge it.
As I have determined that the RSI exits and Stops and Targets seem to perform the best, I optimized those variables using exhaustive optimization. The results of the system run with the optimizations is above.
The system generates roughly four trades a day with the average trade earning around $35.00. The largest weekly drawdown during the time tested was -2.22%.
Although the annual rate of return is huge at 340.57% and the system doubled its starting equity in 50 days time, I’m still not sure how robust it will be going forward.
For the final post, I will run the system (as it is currently optimized) on out-of-sample data, from 2/16/09 to 4/28/09. We will then find out if the system might be over-optimized. We will also get a sense of how robust Lazy Man’s system, with my modifications, might be going forward.
Summary of Parameters
Dates Tested: 12/1/08-2/13/09
Starting Equity: $10,000
Contracts Per Trade: 1
Commissions Included: Yes, $2.50 per trade
Exits: RSI Long Exit, 85; RSI Short Exit, 10
Profit Target: $700.00
Stop: $300
340% annual? Will you be releasing this on autotrade code? I am determined to both trade and eat a sandwich at the same time
lol..the code is simple mana… do you want it?
seems pretty good. what is its current stance and how has it done in 2008
Wood,
Best thing to do would be to call out live entry signals (and charge for the service).
Tell them the exit is all theirs but almost every trade will work at some point…
Hah!
Will be curious to see how the last data set plays out. I did well with it through that time period but, you know my exits are all based on the “itchy mouse finger indicator” (and I traded other markets as well).
This has been very cool.
I’d pay coin for that
Woodshedder,
For those of us that did not catch all parts of this system, can you confirm the latest criteria for entering / exiting a position?
I tried to back read but got a bit lost.
PS. Great blog with lots of useful info and learning material.
Thanks in advance
Hang on folks, you might want to see the out-of-sample results that I will post tonight before deciding that this system is for you.
I have no plans to charge anything for the signals, nor do I have the infrastructure in place to be able to do such a thing as I would be at work. Although I guess I could get the wife on board 😉 since she is at home.
Magpie-
Entry Criteria = if 1 bar ago high was beneath EMA(20) and current bar low is above the EMA(20) then buy the close of the bar, or buy open of the next bar.
Opposite criteria for shorting.
Exits- RSI exits as specified in the post above.
Profit targets and stops as specified in the post above.
The system will stop and reverse, meaning if a long position is on and the short criteria is met, it closes the long and goes short.
If you haven’t run this through a Monte Carlo Simulation yet, you can’t trust the numbers your system is showing you so far. You have to throw real-world randomness at your system to see if it really works.
You also should try to build three systems: Market breadth/trend, stock trend, and stock entry/exit — that all work together. You should test all independently, and together.
Why no “K” ratio? That’s the most important number to look at.
Just to summarize if you are trying to look back.
This is a quick system that I came up with last fall to take advantage of recent volatility.
I’ve managed to make it work (and work well) in three markets (E-Mini S&P, Euro futures, 30 year bond futures).
By nature, the system catches every trend – big, small, & otherwise.
The system also makes some calls that end up being chop and, while you could have made money on the signal, you will end up with a loss if you are more inclined to wait out a larger profit.
As the trend lengths are so varied, there is no perfect exit (at least not one that will probably work across multiple time periods).
The end result (to me) is that profit comes down to trader discretion.
THAT is why I said, “charge for the entry signal, let them figure out the rest.” The entry is the EASY part.
And system trades for today…
So far there have been 3 trades today – profit is quoted per contract traded (I’m trading 2-4 today)
1 bond trade – short – 125’050
drawdown was 2 ticks (-$37)
best exit was 124’185 (+$486)
I got out at 124’280 (+$229)
1 e-mini S&P trade – long – 849
drawdown was 2 ticks (-$25)
best exit was 858.50 (+$950)
I got out at 853 (+$200)
1 Euro futures trade – long – 1.3071
drawdown was 4 pips (+$50)
best exit was 1.3086 (+$187)
I got out at 1.3070 ($-12) – I moved by stop to BE
BTW Wood,
Down limits on lean hog & belly futures. I wanted so badly to be in this but Think or Swim does not support trades on those symbols…
Should have held the bond trade – ended up being down all day – would have been $1200 per contract so far with the same $37 drawdown right after entry.
Did get chopped about on 2 Euro tradesbut I just got 58 pips on the last one – more than made up…
Woodshedder and Lazy,
How much would you guys be willing to sacrifice your W:L ratio for Average % Gain vs Average % Lose?
Redshark,
What’s your idea? I’m open to anything.
The nature of the system does not really allow a big loss on any trade UNLESS the market is moving sideways but ever-so-slightly against you.
Lately, I’ve been doing better actually with break-even stop losses after I see it pull away by 2 points or so (ES) or 20 pips or so with 6E (Euro). I’m trying really hard to allow winners to run but cut down on trades that reverse against me within the first few minutes.
Lazy,
Wood really took the ball and ran with it from your request huh? great info and great work.
i think after all of the testing, for me at least, the best exit would be to sell half a position at a set target (what should that be?), move stop on remaining to entry, let the rest run ’til your original exit (50% over 20EMA) is hit.
have you made any changes to your trading since Wood compiled all of this info?
j.M
Well right now, I have a system written that uses a TrendlineMA for 100 bars. Because the MA is backed out so far at 100 Bars, your entry does not take *as* much chop out, but it still does take out a fair amount. Currently the system averages winning about 45% of the time, but the winners are about 3x as big as the losers.
You are right about the tight stops. My trades either seem to take off, or they are not worth much.
When I tightened the MA to around 50 bars, my results were closer to Woodshedders, some better, some worse. Also, as it pertains to letting the system run, the TrendlineMA 50 and 100 do a nice job of that, take you out of your trades very near the top.
I am running some optimizations on slope data at the moment. If I find anything meaningful, I will probably post it in the PG, or link you the data.
I was just curious how far down in W:L ratios you would be willing to go trading a system. We can theorycraft ideas all day, but you have to be willing to trade something to make the data meaningful.
j.M.
Yes he did! I really can’t believe the effort he has put in. I’ve said thank you but, man, THANK YOU.
I do trade multiple contracts so I do scale out. I tend to watch the outer Bollinger band for my exits – if we hit it and ride it, I usually get out after it backs off. If it takes too long to hit it, it concerns me and I will usually look to dump the whole trade with some profit – hopefully.
I can’t say my trades have changed drastically other than I do feel more confident in the fact that I realize more than ever that trader discretion is the profit. Anybody (well maybe not anybody) can get into a good trade – greed and fear usually screw it up for 95%.
Redshark,
You are right about the tight stops. My trades either seem to take off, or they are not worth much.
1000% CORRECT
With that system, are you using time bars or tick bars? I prefer the tick bars with a faster MA than a slower MA with time bars.
That said, I do want to see what you are working on.
I am using $1 Range bars. I like it because it has the same benefit of the tick bars you are using, except it is 100% price driven.
Redshark,
What platform do you use – as far as I know TOS does not have that as an option.
Investor/RT with IB
Why don’t you test your entry and exit methodologies as independent variables and then use a 3D optimization chart to graph the results and identify distinct patterns of worthiness? If there are too many variables to test (due to the crunching time, which can take days or weeks), then use genetic algorithms to speed things up… once again, you’ll need to use MCS (Monte Carlo) in the final testing to determine if the system holds up under real-world randomness (and which will also obliterate any chance of curve fitting, which seems very likely given the strategies I have read, above). The Amibroker long-timers can tell you how to do all of this that I describe.
Bman
I don’t see how any of the above comments relate to curve fitting.
There have been a lot of good comments, but alas, I was at work. I’ll try to answer/respond as best and as briefly as I can in order to get caught up.
B-man, I could run a Monte Carlo. No big deal. Sure, if I were going to market this system, or trade it myself, I would run a Monte Carlo. I may anyway, just for fun. Keep in mind that this blogging thing I do is mainly for my own enjoyment and to spark creativity and discussion. As I am always pressed for time, I will rarely write/post/discuss everything that I would had I 24hours a day to blog. All that being said, I think that optimizing on in-sample and running another test on out-of-sample does well to include real-world randomness.
More B-man- Tradestation does not compute K-ratio. Ami-Broker does but I do not have futures data for AB.
@Lazy, I hear you on the pork belly futures…
@Red, here is how I look at win%. If I am trading a short term system, I expect a high win% of 60% or more. However, if I am trading a trend-following system, then I would expect a much lower winning percentage. As Lazy’s system is sort of a hybrid, in that it is a trend following system, but is also short term, I’m not sure how to feel. Since the entry is more of a trend following entry, and since trend following exits work, I think I would be willing to accept a lower win%.
@Lazy, I’ve really enjoyed working on this, and I hope some folks have gotten some good ideas out of it. Thank you for helping me flesh it all out and for keeping us up to date with how you actually trade it. I’ve learned more about futures trading in the past 3 weeks than I’ve learned over the last 3 years.
@B-man, in early posts on this system, I posted the results of optimizations. The data is the same, regardless if it is presented in 3D or not. I do not have futures data for AB, but can say that without a doubt, the 3D opti charts that AB generates are really really cool. The time spent testing independent variables is usually devoted to the systems my partner and I are developing. I would like to delve into that aspect of testing, but that was not the goal of this series. And again, any curve fitting will be at least partially revealed in out-of-sample tests, which will probably be delayed until tomorrow, considering how much time I’ve spent writing this comment.
Wood: I understand where you’re coming from. I am just throwing ideas to help out, not criticize.
MCS is how you avoid curve-fitting, ultimately, because out-of-sample tests are very limited: you can only select certain time frames that makes sense to use, from the available samples. Plus, are you using random-sampling or are you hand selecting the test samples? I would do both, and make sure to test the system using data from the worst market conditions possible, to see how it holds up.
MSC does the same thing, but it does it 10,000 times or whatever — not just a couple of times. If you don’t use MCS, your chances of curve-fitting are quite high, in my experience. You know just by checking the results of both tests. If MCS says your system sucks, when your prior testing says it was ok, you know you should be using MCS.
If you hand-pick test data, here are some ideas (for equities) —
a. COMPLETE SINGLE BULL-BEAR CYCLES
i. In Sample = 4/1/2002 – 2/1/2004
ii. In Sample = 11/1/1998 – 10/1/2001
b. ULTRA-BEAR CYCLES
i. In Sample = 10/1/2000 – 10/1/2002
c. ULTRA-BULL CYCLES
i. In Sample = 1/1/1996 – 4/1/2000
ii. In Sample = 4/1/2003 – 2/1/2006
Use what you can and throw the rest away.
Bman, it is cool. I wasn’t sure at first if you were trying to help, but I know now, and I really appreciate all the advice. Keep it coming.
There is 1 major problem with testing Lazy’s system and that is that Tradestation only has tick data for futures going back about 6 months. That is not near enough data to make any significant conclusions.
The problem with running Monte Carlo over a small data set is that if the data set is favorable, the MC runs will also be favorable.
I ran the MC over Lazy’s system, as optimized in the post above, and the results are STELLAR.
However, the MC are meaningless. How do I know? Because in the out-of-sample tests, the system, as I have it optimized, does not do well (sorry for the spoiler before the post is up.)
I’ll try and address these issues as well as post the MC results in the near future.
I completely agree with your cycle testing that you listed above, but I just don’t have enough data to run tests like that with this system.
I’ve been lurking here for the past few weeks and wanted to finally say how much I’ve enjoyed the blog recently (yours too Prospectus). Since the beginning of the year I’ve gotten heavily into strategy development and testing with TradeStation and find the ideas & comments exchanged here very enlightening. I will try to contribute more in the future.
I have a question regarding the MC tests — do you use the results of *each day’s trading* within your timeframe, then randomly shuffle those day totals and recompute the equity curve? I assume you then quantify the mean and scatter (and max drawdown etc) of the X number of realizations? Or should it be done on a trade-by-trade basis, and resample smaller timeframes than the original, mixing trades from different days? I suppose that would be the most random way to do it.
I’ve recently developed a long-only strategy that, after initial optimization and testing, works very well on GOOG 2min between 1/07 and today. Trading 200 shares a time and including commissions it would have generated ~$46k. The largest drawdowns are ~8k (just two of them though, otherwise the curve rises steadily for ~2 years), but I can’t risk taking this live and experiencing an 8k drawdown (or worse) right off the bat. A take it that an MC test is what I need here to know if I should actually take this live? Granted the parameters are optimized for this timeframe, _but_ given how long the timeframe is, how confident should I be that these values will continue doing well (that is, I would guess that the past 2.4 years encompass such a wide range of intraday activity that my results shouldn’t be too sensitive to either curve fitting or changing market conditions, at least to the level we’ve seen over the past 2.4 years). Thanks for any input here, I’m itching to use this code live (I’m only running it in my simulated account during the day for now, to be sure that everything runs fine real-time), but will only do so once I’m totally confident I could handle an initial bad spell. At what point do you finally feel good enough with a backtested strategy to take it live??
A comment on the limited TRAD tick backtesting — if you are willing to use an intraday timeframe, and as long as you are dealing with a liquid stock or contract, you don’t really need to use the look-inside-bar backtesting (I think– as long as you have active stop and limit orders out during each bar, and you only take signals at the close of bars), which should allow you to backtest a much much longer timeframe. You could first compare the difference on a short timeframe covered by tick data to be sure that it isn’t a big deal, but again you would have to be willing to use an intraday timeframe.
Sky, some excellent questions. I’ll try to answer all of them as best as I can, but it may take me the course of several days or posts to cover them.
Let me start with Monte Carlo.
http://www.tickquest.com/product/equitymonaco.html
Go to the site above and download their free monte carlo product. It works well with tradestation. Once you have it downloaded, just pull up Equity Monaco into the strategy you are working on and it will output the necessary data into a text file. The software will open the text file and generate the analysis for you.
SkyTrader,
Always count on a perfectly tested system to test you when you apply money to it…
Knowing nothing about the system, my first question would be; does the system only work with GOOG?
In other words, did you curve fit it? Are your sim trades working out like they should? Was most (or a lot) of the profit only from a couple of trades or is it pretty consistent?
Beyond that, the issue (for me) with not having a few different stocks or markets to trade is that many traders get impatient and start to take questionable trades rather than waiting for A+ perfect entries. If you are confident that you are not that guy, then go for it. I don’t have that type of self-control and need 5 or 6 entries per day
Hey, thanks for the link, the documentation answered my questions about MC. Unfortunately I have TRAD running on Vista (bleh), so I don’t think their program will work for me — does anyone know if it does run on Vista?. It sounds pretty easy to code so I may just do it myself. The TRAD interface would be nice though.
No worries if you don’t get to my other questions, I know how time can be tight. Thanks for the great blog!
I’m running it right now on Vista. You shouldn’t have any problems.
I’m working on a new post right now, but let me jot down a few more thoughts.
A robust system should have some success on other stocks besides GOOG. You might try just running it on other random stocks from the Nasdaq 100. I feel comfortable saying that if it only works on GOOG, it is curve-fit.
My only concern is how the Monte Carlo was implemented in the application. It seems to be closed source and that bothers me greatly because I would need to feel comfortable with their RNG.
My concern boils down to something that most would find fairly esoteric but please bear with me.
To wit: My understanding is that it takes the returns and jumbles them up and then creates multiple sets of “runs” based on a group of selections from the initial data.
Now that sounds all well and good except if the RNG is using a Gaussian formula – I am afraid there would be results that would mimic a lognormal distribution (which is what conventional wisdom indicates the market has) but is essentially, Gaussian (which could be really bad – see any of Taleb’s commentary on that subject)
Most of the MC theory I’ve read indicates that it’s a potentially useful tool when working with a limited data set but, if the distribution curves are different between the original data and the simulated trades then you’re about to make account decisions based on an apples to oranges comparison while you think you’re dealing with an apples to apples simulation.
lazy man,
The strategy does seem to work on other stocks too, although the optimized values (the stop and target percentages) come out different. I expect those to vary though, depending on the short-term volatility (i.e. how volume and momentum-heavy they are on the 2min chart, the default for this strategy – dealing with tick chart strategies in TRAD is more of a pain). I’m not yet sure how robust the strategy is to random fluctuations in short-term volatility, I hope that a MC simulation will provide some insight there. I haven’t played with many other stocks yet, GOOG jumped out at me a while back and I’ve mostly been playing with this, although I have gotten FSLR to work out alright too.
Have a look at the GOOG equity curves: http://twitpic.com/photos/SkyTrader
The increase is pretty consistent, and over a long timeframe (the strategy typically takes a few trades per day — I chose the strategy route to help remove my personal discretion from my trading, but like you I like at least a few trades per day — I prefer to watch the action during the day though, rather than take trades myself). A bonus of this strategy is that for now it only runs in the morning, so I can still follow my real job. In the long run my goal is to have a few different strategies running on a basket of stocks, ETFs, and futures.
Granted the strategy is optimized to this timeframe, but given how long the timeframe is, I’m not sure how worried I should be about having over-fit. The only variables I have are the risk/reward percentages, which should be mainly set by the short-term volatility of the selected chart. If my timeframe samples a large enough range of this intraday volatility, would you think this is then ‘ok’ to use live, pending the results of an MC test?
Thanks in advance for anyone’s input.
Sky, wow, your comment and my new post share some similar themes, and I was even able to comment on Cuervos comment before ever reading it. Good times.
Wood — thanks for letting me know! I’ll try it out sometime soon.
Cuervos — I didn’t follow that, do you mean that if the actual equity curve isn’t contained within the range of simulated equity curves you shouldn’t trust the MC results? Can you explain the Gaussian/lognormal argument in more detail?
Okay system geeks-
Take a second to stop and smell the roses…
The “Run for the Roses” that is…. On the Most Important Manner of the Kentucky Derby.
____
@Woodshedder “want the code?”
Thanks, I coded it already but didn’t feed it more than daily data.. Bummer on those out of sample results 🙁