I've been building trading systems for a number of years, and of course, this has had a big influence on the design and development of RightEdge. One of the major goals is to make a RightEdge simulation as close to real life as possible. One thing that sets us apart from competitors is running simulations bar based and not symbol based. Let me give you an example. Let's say you'd like to run a simulation against all 30 components in the Dow Jones 30. Other packages will run the simulation for component 1, then component 2, then component 3, etc. Where RightEdge will run for all of the components at the same time. This is more inline with reality.
Why do I bring this up? I was reading an article about how the new fundamental indexes aren't performing all that well for 2007, or at least they're not outperforming everything else. I'm reminded of all of the "backtesting" that was done to prove that this way of indexing is/was superior. As you might guess, I was skeptical.
I was skeptical because I've built some stellar systems. I can make a system turn 10K into a million in just a couple of years. The problem, is that it doesn't work in reality. With curve fitting, market issues (slippage, partial fills, etc), psychology (can you handle a 70+% drawdown), hand picked symbols, and so on, it's easy to make a system look as good or as bad as you want it to. I'm not saying that fundamental indexing is not good or bad, I think the jury is still out, however, be skeptical when someone or a company shows you an equity curve from a simulation. If the ETF or fund goes live, track the error between live trades and the simulated equity curve.
This is why I always stress exercising great caution when transitiong from a simulation to live trading. It may not be entirely what you expect.