On Friday’s open, the system closed all remaining open positions. There were no new Power Dip signals for Monday’s open, and so the system will sit in cash until the appropriate dip-buying opportunity presents itself. I have decided to begin posting any new buy/sell signals at 9:00 a.m. EST. As soon as the PD gets another signal, I will post it 30 minutes before the open.
For the week, the Power Dip system closed down slightly, losing $282.00, for a percentage loss of 0.27% from last Friday’s close. I find these results to be acceptable, even though the SPY closed down slightly less for the week, losing only 0.22%.
Highlights:
The win % is slowly climbing up to the backtested historical average. I love how often this system wins.
Also, many of the losing trades (but not stopped out) were closed for small losses.
Concerns:
The current Avg. % Trade is of concern me. It will need to triple to get near the backtested historical average. Even a system that wins consistently 70% of the time will not be very successful with a low Avg. % Trade, unless perhaps it trades very very frequently. Anyway, this system does not trade all that frequently and so we rely on an Average % Trade of much higher than 0.35% in order to outperform the broader indices. Imagine how the current results would look if the average % gain on every trade made so far was 0.60% higher.We can run a few quick and rough calculations to approximate what the current results would be had the system performed near the backtested historical norm.
- 40 trades * $35.00 = $1,400 (at current expectancy of $35 gain on the average trade)
- 40 trades * $95.00 = $3,800 (at backtested historical expectancy of $95 gain on the average trade)
The historical average trade would have the system up about 3.8% since inception. That is the type of future performance I expect from this system, and if it continues to perform at only 1/3rd of what I expect, I will have to dig around under the hood and figure out whether this is just a normal phase of underperformance or whether there is something more nefarious afoot.
The other concern is with commissions. Commissions are a cost of doing business, and like all costs of doing business, they should be minimized to the greatest extent possible. At the present, commissions have eaten away better than 1/3rd of the total profit. I am using Tradestation’s commission structure which is $0.01/share with a $1.00 minimum up to 500 shares, with every share over 500 costing $0.006. In the future I may add to the spreadsheet the cost of shares purchased with a fixed commission structure. As the system keeps making money, it will eventually benefit from a fixed commission structure, especially if the system continues to favor low-priced stocks. Traders using Interactive Brokers’ commissions of .005/share can perform some quick calculations to learn how much money they would have saved.
Summary:
While not outperforming, the system is making money. Once the average % trade begins to rise, this system will outperform the broader indices.
Anyone who wants to read more on the Power Dip system, please visit this link which provides access to every post ever written on the Power Dip.
It seems that high percent winner strategies eek out small profits as a result. Many of the breakout systems have a low percent winner ratio, but have nicer profits. I think Richard Dennis’s Turtle system had a 90% loser rate.
Have you considered walking forward your system to determine what might be a reasonable expectation of future returns. First backtest, then optimize, then walk forward. From the walk forward profile, you can bootstrap or use Monte Carlo to determine what is an expected equity curve and expected drawdown. This is about the only place in trading you can employ Gaussian distribution to arrive at a reasonable expectation of what drawdowns and/or equity curve returns should be.
The equity curve derived from historical simulation is based purely on luck. That’s why it’s important to walk forward your system and then randomize the trades to take out the element of luck.
This is about the only place in trading you can employ Gaussian distribution to arrive at a reasonable expectation of what drawdowns and/or equity curve returns should be.
That’s the funniest thing I’ve seen all weekend.
It’s that kind of belief in Gaussian distributions that helped the quants blow themselves up.
Cuervos Laugh:
I don’t think you have any idea about what I’m talking about wrt Gaussian distribution.
Did you gather I was referring to price action?
Do you know what the bootstrapping method is?
Are you aware that the equity curve generated in a backtest is a product of luck, dependent on the sequence of trades?
If you re-arrange the sequence of trades, you take out the element of luck wrt the equity curve. From this data (1,000 data points) you can make a reasonable statistical inference. If the drawdown in real time exceeds 98% of what you’ve calculated, you shut down the system.
You don’t have to use 2 standard deviations as your threshold, but it helps to have an idea of what to expect.
If there are any quants using Gaussian distribution in their system models, I doubt they’re still employed after 2008. There were 12 days that price action exceeded 3 standard deviations, four fours, four fives and four sixes; and in both directions. The chances of that happening in a Gaussian world is 1 in a gazillion to the gazillion power (or something like that).
Clearly, the classic bell curve does not apply to market/price action. The only people who still believe that are those using probability of expiring arguments with options.
Milk, I’m guessing here, but I think that Cuervo’s argument will be that it is all a derivative of price, even the sequence of trades.
As price does not conform to a Gaussian distribution, the derivatives of it will not either.
Still, I find Monte Carlo simulations to be very valuable.
Milk, do you happen to know where I can find a decent MC simulator, preferably for the price of free?
Upon further reflection, I realize my use of the term Gaussian was ill-advised.
There is a distribution of drawdowns derived from Monte Carlo, but it is rare to see one that is normally distributed.
Market action has fat tail distribution, and Monte Carlo has kurtotic distributions, usually to the worse case side of the curve.
My point, which could have been stated more clearly, is that if you run 1,000 Monte Carlo sequences and 950 of them have a drawdown of less than $100,000, then when you experience a drawdown of $100,000 in real-time trading, it’s time to stop trading the system.
Wood,
First of all, good transparent review. Pretty refreshing on the internets to see an honest review of ones trading system.
How long has the system been trading to arrive at these results? I am curious about the market trend during this period. I think that many systems are highly correlated to the market trend. This system which I think if I understand it correctly is long only so if the market is strong it probably does better. In your backtesting/optimization did you do any correlations of trade performance vs. the background market trend?
Unfortunately I think many systems that are backtested and optimized look better than they will perform because the backtesting was done in periods of stong bullish markets. In those conditions even a random system will perform if you know what I mean. Back when I spent alot of time doing system testing, I came up with a way of correcting my returns for the market bias to see if the system was truly good or not. And it is not just by comparing to the market average, but I had a correction scheme for the % winners.
I can explain this in more detail if you are interested.
Caveman, the spreadsheet has the dates starting when I began publishing the entry and exit signals. However, the system has been in existence long before then. Honestly, I have been doubtful of the success of a long-only dip buying system in the middle of a big-ass bear market, and so I have not written much about it, until recently.
Backtesting shows that the system tends to underperform during periods of low volatility. One of the worst years for the system, since 1998, was 2006. It did better in 2001 and 2002 I believe due to the volatility. So yes, if the market is strong and volatile, it does well. If it is strong and not volatile, it will underperform.
As most (if not all mean reversion systems) it did not due well prior to the late 80s, early 90s.
One of the reasons I have chosen to present this system as a candidate for subscription is because it has done very well during the bear market of 2000-2002 and during the current bear.
I would love to have you explain in more detail your method for correcting returns for market bias. That sounds right up my alley.
By the way, just to be clear, I was not implying that your Power Dip system was not a good system. I was just curious about the backtest period and the recent performance period that this post is summarizing because I thought that may explain some of the reason why maybe it has not been performing as desired.
No no, I did not take your comments in any negative connotation.
My suspicion is that the last 6 months of market history, (or really, last 1 year) has been unprecedented. Therefore, I cannot expect the system to perform as normal during a period of unprecedented behavior. There is really no way to test this system on a similar period since I only have data going back to 1985.
Furthermore, if I were to go back to the 60s-70s, it would not be helpful anyway as we know mean reversion did not work very well as that was a more momentum driven period.
Bottom line, if the market is not undergoing a regime change, it will eventually become somewhat normal again, whereby I can expect the system to outperform.
Milk, I have backtested this system on differing time frames, for example, 1998-2001, 2003-2005, etc. with slightly different settings, and it always performs the same. There has been very little optimization of the variables. In fact, the bulk of the variables have never been optimized.
This is a system that was developed after watching hundreds if not thousands of stocks pullback. I basically just described what I was seeing in code, and voila, the Power Dip was born. It is very simple and was probably the first code I ever wrote.
My partner has put just as much time into this system as I have. We have a couple of measures which comprise our “System Health.” It is simple, as is most that we do. It plots the ratio of N day win % : total win %. We do the same with avg. % trade. What we have found about most systems is that they often go through periods of under-performance. Sometimes they will even go through a period of negative expectancy. When they hit these periods of under-performance or hit a negative expectancy, you simple stop trading the system, but keep paper trading it. Once expectancy turns positive, the system turns back on, and it usually roars back to life.
I have not spent a great deal of time with walk-forward testing, on this system. The main reason is that the code has existed for more than a year, and I have been taking some of the signals / paper trading it during that time. It continues to outperform. I did not trade it a whole lot during the second half of 08 due to the extraordinary volatility. However, I did track it, and I would have done better in 08 had I followed it by the book.
I have a problem with the Monte Carlo of the out-of-sample results. This is just another snapshot of system performance. Like any other subject/snapshot, it will have good and not-so-good periods. And as a snapshot, it only covers one period in time. I would rather have a method for measuring system health, and stop trading the system when it looks unhealthy, relative to the backtested historical norms.
All that being said, I do not disagree with what you state in regards to walk-forward analysis and monte carlo testing. Its just that this system is very simple, and when it quits working (it will quit working) it will not be due to over-optimization, curve-fitting, etc. It will quit working because the markets will have undergone a regime change. My system health measure will have turned the system off many moons before we actually can quantify the regime change.
Putting observable market action into code is one of the most basic building blocks of system development that is often ignored by many so-called system developers. There needs to be a logic behind the system instead of pattern chasing. I’m leary of systems with too many parameters because I suspect their results are a product of Poisson Distribution.
Optimization does not create curve-fitting, Only the incorrect execution of optimization does that. Besides finding best parameters, optimization sheds light onto how robust the system is. If you have peak parameters that have no neighbors, it’s likely a product of chance. A smooth space of optimal parameters gives one more confidence, no?
Paper trading a system for a year is essentially walk forward testing. It’s deploying your system on data it has never seen, and observing how it performs. It’s also an opportunity to re-optimize parameters.
As far as measuring system health, I’m sure there are many valid methods. The most interesting method I’ve seen (but haven’t used yet) is bootstrapping. It is re-arranging the trade sequence from a backtest period to arrive at a drawdown profile.
Milk, I agree about optimization, smooth space giving confidence.
I am not aware of any Monte Carlo that will work seamlessly with AmiBroker. I have one I run with Tradestation, but TS will not handle portfolio level testing so it does me no good with this system. I think Prospectus may have a Monte Carlo sim for excel that may work. Once I find one, I run the Monte Carlo and then get back to you.
Should be fun.
TradersStudio has a Monte Carlo macro, but haven’t used it seriously yet.
I’ve heard TS does not support portfolio level testing and not sure why not. TradersStudio does support it.
Let me run a quick MC on the 50/200 and blog it. Tell me what you think it tells us.
Cool, let me know when it is up.
Me and the family are heading out to Busch Gardens. Will be back this evening.
http://is.gd/1iC4l for a Monte Carlo simulation run through of the Woodshedder Cross over 25 years data.
Interesting how the max drawdown in historic simulation is below average in Monte Carlo.
Wood,
On my method of correcting for market bias, I will try to post some on it in the future on the PeeG. It is part of a charting back testing system I made using VBA code in excel a few years back. It was a pretty cool project that really ended up going nowhere due to me not really knowing what to do with it. But it may eventually have a place here on IBC should Fly ever be interested. But I will have to think about it. Jake has seen it in detail so he could vouch for it.
Anyway, I will post some stuff on it in the future maybe.
Caveman – if Fly isn’t interested in that code, I would be.
fwiw
Cuervo,
I will keep that in mind. It has been a few years since I played with it and I never got it finished and fully cleaned up but if anything it was fun. It was essentially a TA/charting “testing module” that allowed you to test charting ideas and test how good you are at calling different plays. The difference from other testing schemes out there is that it did not have to be coded. It was not for testing mechanical systems, but instead for testing ones ability to call the direction a stock would go given everything in the chart, including gut, etc.