Interpreting Studies

Apologies to readers as the blog has suffered from the summer doldrums a bit lately. I thought today I would address a few comments I got regarding a post from Tuesday.

In Tuesday’s post I showed a setup that suggested the market was likely to close higher on Wednesday than it did on Monday. This statement is based on the fact that all previous 16 instances did close higher 2 days later. I showed the results table for days 1-5 (Tues through next Mon). As sometimes happens when you deal in probabilities (and not certainties) the market did not adhere to historical norms and instead sold off. After it was clear that the SPX was not likely to finish higher on Wednesday I got a few comments from blog readers about the study.

The first commenter, Daniel, noted that “the odds and ratios up to 5 days out are still appealing from the bull side – even more so now that some short-term overbought conditions have been neutralized.”

The 2nd commenter, 24/5 Trader, remarked, “Isn’t this study showing a bearish edge at the third day?” He noted that while there was a perfect record through 2 days, day 3 showed a decent number of losers as well as a drop in total profits.

Let me first say that there is no “right” answer to the question of how to best evaluate a study or consider it useful. Both of these comments showed logical interpretations of the data. Now I’ll share my interpretation.

When I look at a study like this I generally only track it as long as the results remain consistent. While I could have looked to follow it further, the real value in this study in my eyes was what occurred in the first 2 days. So regardless of whether it works out over day 1 and 2, after that I’m on to looking at my next group of studies. In this case it didn’t work out. Does this tell me anything? Assuming I don’t have a data mining issue with this study (unlikely in this case) then all I’ve really found out for sure is that the market is not reacting as it normally does to that particular setup. Again, let’s assume the setup does provide an edge and it will continue to be useful in the future. (Not always the case but an assumption that need to be made for this discussion.)

Does this abnormal reaction affect my outlook? It was supposed to go up. It went down. Does that mean it will continue to go down? Does that mean it is now oversold and will likely bounce in the next couple of days? Taken as 1 individual study it is impossible to determine.

My studies tend to look at one or two specific aspects of the market. In this case it was strong consistent breadth. While I found this study to be the most interesting and “blogworthy” Tuesday morning, there was also a short-term bearish study that triggered Monday night which was published in the Subscriber Letter. Additionally, there were 3 other short-term bearish studies from the Sunday night letter that were also in effect. So it wasn’t terribly surprising that the market struggled Tuesday and Wednesday. And while the strong breadth was saying we were going higher, these other studies that looked at things such as the overbought price action and weak volume suggested a pullback was in order.

So in this particular case, when the market didn’t react in the “normal” way to the setup studied in Tuesday’s blog, it didn’t concern me. On the other hand, if there were a large number of bullish studies, and few or no bearish studies and the market traded lower, then I might get concerned. If this abnormal behavior persisted then I would view risk as elevated and would consider adjusting my position size and trade management accordingly.

My database looks at an awful lot of information. I try and share a few tidbits on the blog each week, which I hope are often useful. But that is really all they are – useful tidbits. They are not market calls and when the market reacts the way it normally does, it doesn’t me I was right. And when the market moves opposite a study it doesn’t mean I was wrong. I do suggest trade ideas in my Subscriber Letter and I have found studies such as those I often post to the blog to be very useful in determining my bias and in making index trades. My primary tool for interpreting all the studies is the Quantifiable Edges Aggregator. The Aggregator basically takes all of my currently “active” studies and uses them to make a projection for the next few days. If you would like to learn more about the Aggregator, then below are two links.

The first one is a detailed explanation of the tool from about 2 years ago.

The 2nd link is to a webinar I held for subscribers about a week and a half ago. It includes a detailed overview of the Aggregator as well as a question and answer period from subscribers that posed some interesting questions. The video is about 40 minutes, so it takes some time to watch. I should also apologize for the lack of editing. I just started holding these webinars and I need to get up to speed with all the tools. It takes about 40 seconds before the presentation really gets going.

The QE Aggregator – July 19, 2010