Inevitably, the active trader will find him or herself in the following predicament: More stock picks than he or she has cash to purchase. When this happens, release the biases! If the trader does not have a quantifiable method of ranking the stocks so that he or she spends the cash on the best picks, there exists the distinct possibility that the stocks that are selected are chosen due to the bias of the trader, and not because they are the best setups.
Now certainly there are some traders among us who are more gifted than others. I do not count myself in the gifted group. The vast majority of traders are not able to look at a particular chart or setup and not introduce biases. Instead of selecting the best stock for the particular trade he or she wants to execute, the stock he or she selects is the one he or she likes the most. There is a difference. Instead of seeing the stock or setup for what it is, he or she sees AMZN and remembers how frustrating it was when the book was delivered two weeks late. Or, they see XOM and recall something about an oil spill in the Gulf of Mexico.
The best way to handle this dilemma is to quantify the setup and then rank the picks. Of course there must be a way to backtest the ranking method to ensure that it is better than random selection, and when backtesting, you want to be careful not to curve fit the ranking method.
And so after a year of real-time trading of PDS, it is time to examine the ranking mechanism to see how it has performed vs. random selection and an alternate method of ranking.
First I will examine the ranking mechanism by testing it from 1/1/2001 – 11/19/10. Then I will look at how the ranking mechanism has performed since inception. From these results I will draw some conclusions.
It should be noted that PDS is a positive expectancy system, meaning that any given pick, no matter how low it is ranked, can be expected to generate a positive return. Thus, random selection doesn’t necessarily mean the system will fail. In fact, if it is truly a positive expectancy system, we would not expect the system to fail under most circumstances–we would predict that random selection will mean that it will not perform as well as when using a ranking mechanism.
Ranking Tests:
(all tests include .01/share commissions)
Reading the Report-(Thanks to Frank at Engineering Returns for the code that snips these metrics to the clipboard, making for easy recording of multiple tests).
- RAR = Risk Adjusted Return (CAR/Exposure);
- CAR = Compound Annual Return;
- DVR = R² of the Equity Curve * Sharpe Ratio (as popularized by the great folks at CSS Analytics)
- SR = Sharpe Ratio
- R2 = A statistical measure of how close an equity curve is to a straight line plotted on a logarithmic graph. A value of 0 indicates a jagged line and a value of 1.0 represents a straight line. A fixed percentage investment that compounded and paid daily would have a straight line equity curve and an R Squared value of 1.0.
- %W = Percentage of Winning Trades
- %Avg = The Average Trade in Percentage Terms
- MaxDD = The Maximum Drawdown
- TT = Total Trades
About the Ranking Mechanism:
- PrimaryRank = The Default Ranking Mechanism used by PDS.
- Inv_Primary = The Default Ranking Mechanism in reverse. High ranked picks become low ranked and vice versa.
- Alt.Rank = An Alternative Ranking Mechanism
- Inv_Alt.Rank = The Alternative Ranking Mechanism in reverse. High ranked picks become low ranked and vice versa.
- Random = Picks are Selected at Random with no Ranking
Interpreting the Results:
- The PrimaryRank and Alt.Rank generate the best returns, with the PrimaryRank easily beating a random selection. This is positive and demonstrates the power of ranking stock picks. The really interesting result is that the inverses of the ranking mechanisms also tend to beat a routine of random selection.
- The random selection method also generated good returns. The caveat is that I only ran ten tests with random selection. I would feel better if I ran 30 or so tests, but these things take time. There are two reasons why it is hard for the system to show significance degradation of performance, even using a random selection routine.
- Some of the time, there are only a few picks, and the system has enough cash that it takes all of the picks. In these cases, ranking doesn’t matter as all picks will be bought.
- The system has a high percentage of winners, so it would take a very unlucky series of randomly selected stocks for it hit a long losing streak. Because there are over 2,000 trades represented, the system has plenty of time to hit a losing streak and then recover. Over a shorter time frame, we expect that a random selection routine will show more variation. (In fact I think we’ll see this when I run these tests only since inception).
Summary:
The ranking mechanism beats a method of random selection. The inverse of the PrimaryRank generally performed worse than a random selection. From this we can extrapolate that the ranking method is better than a random method. (Previous caveat about sample size applies).
Perhaps the most important implication of these tests is that even when subscribers to PDS do not religiously trade ONLY the top-ranked picks, they may still expect great performance from the system. Yes, it is possible for someone to have either very strong biases or very bad luck and perform worse than a random selection routine. This selection bias may be magnified over a short time frame. The next post will take a look at the ranking method over a short time frame. I will be examining the past year of real-time results from PDS.
Another interesting possibility, and you and I have had some back-and-forths on this, would be ranking based upon some kind of liquidity measure. And of course, it would be interesting to see how the inverse faired as you have done with your other measures.
Not that this would be a good measure in and of itself, but it might add a practicality to a trading system by further reducing slippage. And it could offer insight in to how some highly liquid stocks fair in a trading system versus their less liquid counterparts. My hypothesis would be that from a backtest the less liquid stocks would fair better, but it would be harder to replicate these returns in real life trading.
Red, you read my mind. I will run tests that rank with liquidity. By the way, I use our previous iteration of liquidity for this, not the LLV version. I will definitely run the inverse as well.
With the CBT and the de-listed data, these tests are taking quite a while to run. Hence, I cannot compile a lot at one time.
Good stuff woodshedder. I’ve always been kind of curious how the ranking system actually goes about ranking. Of course, I understand if you’d rather keep it general (just from looking at the picks, I have a few ideas about how it works anyway). I guess the important thing is that it makes the money, and as the post shows, it does seem to do the trick.
Data, ranking is exceptionally simple. In general, think volatility.
Great stuff, Wood – looking forward to the next post!
Wood,
Can you direct to me the link where you discussed how you size positions and establish stops? I thought it was in the last month or so, but I can’t find it anywhere now. Thanks if you can help.
Fast, start looking here. I think I have a better one somewhere. I’ll find it and post it too.
http://ibankcoin.com/woodshedderblog/category/mgmt-postion-sizing/
If you have anything specific you are wondering about, let me know.