Having worked through the processes to either develop and test your own trading system or perform detailed due diligence on a commercially available trading system using the steps discussed over the past weeks, this week we will look at what constitutes a winning trading system. A winning trading system will give you a profitable ‘edge’ over the market and result in a profitable outcome over time. To ensure your system has an edge, there are a few more questions that need to be answered and some more data that will need to be analysed. These questions include:-

• How many completed historical trades are required to demonstrate that an edge has a high probability of    working in the future?
• How will an edge perform with limited capital when only a subset of trades are traded out of the full available    list?
• What metrics can be used to determine which set of trading system research results is better than the next?

When you run your research program against the historical data, a list of trades will be generated with data recorded for each trade. Look for typical data like:

• entry and exit date
• entry and exit price
• percentage profit
• hold period for the trade
• MAE & MFE (Maximum Adverse and Maximum Favourable Excursion)

To start with, always test your edge with the same unit position sizes. For testing and analytical purposes don’t use any money management until you have established that you have a statistical edge. Use a position size that will allow at least one share of the highest priced instrument in your ‘in sample’ historical data. Typically, a position size of $10,000 should do the trick for researching stocks. Index research for CFDs typically requires a larger minimum unit. For example, the DJIA has had values above and below 10,000 as has the Nikkei and ASX 200 Energy Index; therefore a position size of $10,000 would not be large enough to trap the trades when the index was above 10,000 because it is not large enough to trade one CFD. Hence, valid trades would be omitted from the trade’s database.

The research results are then measured statistically across the entire database of historically generated trades according to the rules of the system that you have coded. More data for analysis of the results is calculated from this database of trades:

• total number of trades
• number of winners and losers and hence the percentage winning rate
• the average percentage size of the profit trades and loss trades—will be the same as absolute because a unit    position size has been used for all trades
• the average percentage move per trade across all trades—immediately tells you whether you have an edge
• the average hold period for winners, losers and per trade across all trades

The diagram below shows a solid black line that depicts a breakeven edge. Note that the final edge that you trade should include brokerage, commissions and slippage.

The breakeven edge is computed from the percentage of winners and the profit ratio, which is the relative size of the average winner to the average loser. A profit ratio of 2.0 means that the average winner is twice the size of the average loser.

The leftmost dot on the breakeven line is positioned at a winning rate of 20%. A profit ratio of four will be required to break even, net of brokerage and slippage. The rightmost dot on the breakeven line is positioned at 50% where a profit ratio of one will be required to break even, net of brokerage and slippage.

Any system with an edge that falls below the breakeven line has a negative mathematical expectation and hence virtually no probability of being profitable over a large sample of trades. Through luck, over a small sample of trades you may make money, but the more trades you complete with a negative mathematical expectation system, the higher the probability that you will lose money. Absolutely no type of money management rules can save such a system over a large sample of trades.

Any system positioned above the breakeven line can make money. The further away from that line, the higher the probability that not only will it make money but also that it will be profitable from a smaller sample of trades. Also, the further away from the breakeven line, the less volatile the system’s equity curve should be and the steeper the equity curve should be. This means lower drawdown’s while making money faster.

A good system does not need to be that far away from the breakeven line to make good money as the power of compounding is on your side when you have a positive mathematical expectation working for you in the market. I use the word ‘can’ rather than ‘will’ because poor money management rules will undo an edge with a high positive mathematical expectation as will poor trading psychology.

In reality, generating a system that is much further away than the ‘Excellent System’ will be difficult to achieve. If you do, there is a high probability that the system is either overly optimised, and hence curve fitted to a narrow price action dataset, or has too many criteria and filters which means the same thing. It is overly optimized by constraining its degrees of freedom (covered in next week's blog on optimisation). This may also occur with a system that trades infrequently, i.e. the historical testing generated a small sample of trades. Again, this usually results from too many criteria and over optimising.

Other metrics that should be established to compare one system to the next include:
• expectancy
• pessimistic rate of return
• average hold period per trade
• average profit per trade
• average size of profit and loss trades
• maximum consecutive profit and loss trades
• Sharpe Ratio
• Recovery Factor, Ulcer Index, etc.

Your research tool should include these metrics and explanations as standard features, or you can program them yourself, or use Excel. The definitions of these can also be found in books by Perry Kaufman, Van Tharp, Joe Krutsinger, Thomas Stridsman and others.

The steps of choosing a concept, programming it, establishing a testing data set, measuring the research results, modifying the concept or choosing another are an ongoing iterative loop that will continue until you have met the original objectives that you stated for your system.

An important point needs to be made regarding the ‘best system’. Simply stated, there is no ‘best’ system. Most people are led to the markets in search of larger returns than are on offer elsewhere, which creates a paradigm of evaluating systems from a single metric—those with the ‘best’ returns. Are the returns measured as raw percentage return on capital? Or return per daily/weekly/quarterly bar? Or return per trade?

A system with the ‘best’ raw percentage returns will probably not be able to be traded by the great majority of people because it would likely require many hours of effort every day: trading a leveraged instrument, having a volatile equity curve, having an average hold period of less than a day and requiring a trading mindset only achieved by a select few who engage the markets. You see, the more often you watch price action, the more negative feedback you will get from the market, which psychologically makes it far more difficult to trade successfully. Randomness takes care of this. The less often you watch price action, the more positive feedback you will get from the market.

A system with the ‘best’ return per trade will probably have an average hold period of many months or even years. This shows that using a paradigm of measuring a system by its returns will determine the timeframe in which you trade and how much time you will require to trade. This is the tail wagging the dog. You need to start at the top and specify how you want to engage the market and then find/build a system that fits your lifestyle requirements such as time availability, ability to handle equity curve volatility, absolute returns on capital requirement, etc.

As you can now appreciate, the process of designing and developing a professional system that works and delivers a positive edge to the trader is a complex process. It is not one to be taken lightly as the number of variables that need to be considered and examined is large. Coupled with that are the psychological issues and paradigms that also need to be observed and confronted by those wanting to develop a system that has no personal bias associated with it.

In next week’s blog I will look at the traps of over optimisation and curve fitting that can create a false sense of security and unobtainable profit objectives for the unwary developer.

Having researched and designed your trading system by working through the steps we have discussed so far it is now time to back-test your ideas to see if the system ‘works. Research is conducted via back-testing concepts against available historical data. This is where your programming skills are required, unless you wish to conduct your research manually or visually.

A word of caution regarding visual back-testing: your bias of wanting the rules that you are testing to work will block out instances of price action that result in loss trades, missing trends by exiting too early. These are called ‘scotomas’ or ‘blind spots’. This is also referred to as ‘perceptual blindness’ where price action that contravenes what you believe should happen (or wish to happen) according to your chosen rules is simply not seen on the chart. This occurs in live trading as well as visual back-testing. When a consistent mindset is achieved the researcher becomes far more objective – objectivity is a subset of consistency – and perceptual blindness is mostly overcome when doing research or in live trading.

Perceptual blindness when doing research will result in a skew towards omitting losing trades, including trades that don’t actually meet the rules and overstating winning trades, which will make the research results look far better than they really are. There is a high probability of disaster taking such a set of rules into live trading.

Computer based research will always be far more objective than visual back-testing. Merely learning a programming language is not enough. Programming techniques also need to be learned which will be covered in any half decent programming course.

Choose a concept or combination of concepts from the list discussed in last week's blog, or elsewhere, including the internet, magazines such as Stocks & Commodities and books that contain ‘base’ trading systems. Then code into your chosen research tool (e.g. Wealth-Lab, TradeStation, Trade Navigator) the criteria to filter price action and/or volume and to determine entry and exit points. Remember that the chosen concept needs to match your stated requirements for the system you are researching. For example, using a momentum indicator on daily charts simply will not produce a profitable, let alone robust, system that has average hold periods of one to five days.

Typically, a filter should be used to improve the win rate of your system and control the opportunity to trade that is required in your stated objectives. A filter could be:
• a moving average
• comparative relative strength against an Index
• inter-market analysis
• fundamental analysis
• Elliot Wave
• Fibonacci retracements
• a combination of the above, or others

Once you have programmed your research concept, and removed all the bugs from the code, the next step is to test your rules against a set of ‘in sample’ historical data. Ensure that you use clean data that closely represents price action in historical live trading. I suggest that you spend some time researching this topic, especially if you are working with futures markets where historical data mostly needs to be normalised for research purposes. Futures contracts expire at regular intervals and if the data is not ‘back adjusted’ to reflect changes in the values of contract months the data will be virtually irrelevant for back testing purposes..

‘In sample’ testing data should be historical data that has suitable information to generate sufficient trades to provide a large enough statistical sample for the timeframe you are trading. For example, if your trading timeframe is intra-day using five minute bars, then one to two months’ data will be sufficient. If your trading timeframe is medium to long term using daily and weekly bars, then five years of historical data should be used and > 10 years for monthly charts.

As a rule of thumb, the ‘in sample’ data set should cover at least two cycles each of markets moving up, down and sideways. This usually requires a subjective ‘eye-ball’ call of the overall market. Again, be careful not to succumb to your personal biases when using your subjectivity. Basic psychology books will inform you of the generic biases that we all have. These biases may assist us in other parts of our lives such as surviving in business or society but hinder us when it comes to trading.

‘Out of sample’ testing data should then be used to test the system rules on a different set of price action data. You are looking for similar results between the two sets of sample data or, as a minimum, one set not being drastically different to the other.

●------------------------------------------●-------------------------●-----

In sample data Out of sample data

If testing a single market, such as an index or a currency, then using ‘in sample’ and ‘out of sample’ is very important. However, if testing an equities system, then testing price action across, say, 200 to 500 stocks over many years, will provide a wide variety of price action as a test bed. It then becomes more important to test the system rules in different types of all market action, i.e. rising, sideways and falling markets. Using an overall market index like the All-Ords to determine whether the overall market was a bull or a bear should then be brought into the sample test data and tested with trading rules. Such rules might fall under the heading of risk management rules.

Another issue that requires mention is liquidity, which is more applicable to equity trading systems than single market trading systems. Ensure that your research code includes criteria for excluding price action where there is insufficient liquidity to trade. For example, researching price action where an average of $10,000 a day is traded for a particular instrument will be too illiquid for a portfolio that will trade average position sizes of $10,000. Remember, price action that was “illiquid” 10 years ago may be very liquid now. In my experience, trapping illiquid trades that you simply could not have traded historically artificially improves an edge.

In next week's journal posting I will discuss the welding together of the concepts we have been discussing over the past few weeks to arrive at a winning trading system.

Having set out on the path of constructing your own trading system, it is now time to consider the aims and objectives of your system. In other words, what you are really doing, and why you are doing it.

If you don’t specify what you want your edge to deliver, how will you know when to stop developing and start trading? This is a major problem for many analysts and system designers. They get so caught up in the application of analysis techniques and trying to develop a ‘perfect’ system, that they forget that they are supposed to be developing a system that can be applied to the markets in order to make a profit. They continue developing for years on end with some never actually getting to the implementation stage because they haven’t specified what it is they are trying to achieve!

You need to be very specific with the stated objectives of your edge. Listed below are some of the questions you need to consider when defining the objectives of your trading system or edge.

• What is your time availability to trade an active investment edge? I see this as the most important decision that any trader or systems designer has to make. There is little point in setting out to design a short term trading system for example if you have a demanding full time job, or run a business that requires your presence for long days every day.

• The time you have available to trade will determine the timeframe in which you trade: intra-day, short, medium, long or ultra-long term trading. Intra-day is largely day trading. Short term might be defined as average hold periods of 2—10 days, medium term 6—12 weeks, long term 6—12 months and ultra-long term as 1—2 years. Greater than two years might be termed buy and hold. The shorter the timeframe traded, the more trades that will be completed, the more time it will take to trade but the higher the potential percentage profits that can be made through leverage and compounding of re-invested profits.

• How many opportunities to trade will you require your system to generate for you? Is it on average 1—2 trades a month, 8—12 trades a month, or even 50— 70 trades a month? This is closely linked to your time availability which will determine your timeframe for trading which, in turn, will determine your opportunity to trade; these three factors essentially go hand-in-hand Opportunity to trade will also determine how effectively you use your capital, e.g. having $100,000 to actively invest and only having sufficient signals to ever have $50,000 of it in the market is not a high probability method of generating good returns on your investment capital.

• Will the trading system be totally mechanical or will there be a component of discretion applied? If so, where and when will the discretion be applied? Specify how this will be researched as discretion can be difficult to research with a computer, especially over a large sample of trades!

• Will the system be a directional strategy that signals trends? Will it be a directional strategy that trades in range bound markets—long and/or short? Will it be a non-directional strategy that trades multi-legged options?

• What exchanges will it be designed to trade: ASX, SFE, LSE, USA? What instruments will it be designed to trade: equities, CFDs, stock futures, equity indices, ETFs (Exchange Traded Funds), bonds, Forex (currencies), commodity futures, a single market such as a single equity index, single futures market or single currency, or others? The timeframe traded will play a major role in deciding what instrument to trade.

• What is an acceptable maximum drawdown for you when running a portfolio with and without money management? When money management rules are applied the maximum drawdown percentage should increase as position sizes are increased.

• What average loss per trade are you prepared to accept? While specifying what portfolio maximum drawdown you are prepared to accept is more important, the risk per individual trade is still important. Be aware that the smaller the average loss that you are prepared to accept the shorter the timeframe that you will be forced to trade. If you want long hold periods because you only have time to turn over 1-2 trades per month then you will have to accept wider stops to allow trades to run which will, almost as an unwritten law of the markets, lead to larger average loss trades. This is just the way it is.

• What is the minimum winning rate that you are prepared to accept over a large sample of trades? Is it 35%, 40%, 50% or even as high as 65%? Trying to even specify this without a good dose of trading experience is difficult; it’s even harder to achieve it in research.

• What profit ratio (or reward to risk ratio or payoff ratio) are you prepared to accept? This should be a minimum of 1.5. (i.e. the average size of profit trades are 1.5x the average size of loss trades). This is closely linked to the winning rate. For your system to have a ghost of a chance of being profitable requires knowledge of what determines a breakeven edge.

• What returns are you looking for on your invested capital? Is it >50% pa or around 10% pa? This will determine what timeframe you trade in and what instruments you trade. For example, there is a low probability that you will generate >50% pa returns trading equities in the largest 100 market capitalisation stocks with no leverage.

• Do you need to trade leveraged instruments to achieve your performance objectives? Leverage can assist in generating greater percentage returns but can also generate greater percentage losses. The more capital that you have to invest, the less you need to include leverage in your strategy because larger absolute returns can be made with larger capital bases. Certain instruments are leveraged by default such as commodity futures, CFDs and FX. Also, if you intend using your Super Fund (applicable in Australia only – roughly equivalent to a 401K in the United States) to trade your methodology then leverage may not be able to be used. (You should research what the best entity is for you to trade through).

These are all questions that need to be answered BEFORE you step into the system design process. You cannot design and build a successful trading system with an edge over the market if you are unable to answer these questions.

In next week's posting we will take a look at entry and exit signal concepts.

In last week’s blog we looked at the mindset required to begin designing and building your own trading system. This week we take a look at the tools and resources required to undertake the journey.

To design your own system with an ‘edge’ over the market certain basic resources will be required. These resources include tools, skills, people, time and money:

• A minimum of two to three years’ technical analysis knowledge is essential. You need this in order to be aware of technical concepts to research. You will need to start using and understanding charting software; do technical analysis courses to familiarise yourself with all forms of technical analysis and to discover what resonates with your personality and trading style; read everything you can on technical analysis; books, journals, magazines, on line articles, etc, etc.

• A research tool such as TradeStation, Trade Navigator, Wealth-Lab Developer or AmiBroker. (MetaStock on its own may not be sufficient for system development but there are complimentary tools such as TradeSim to assist MetaStock). Note that your research and system design software or program need not be the same technical analysis software program that you use to trade.

• Computer programming capability or access to someone who can program your ideas.

• Mathematics and statistics knowledge.

• Knowledge of system robustness metrics.

• Creativity and some knowledge or experience with the creative process (right brained capability). In essence, an ability to think ‘outside the square’.

• Logical thought patterns (left brained capability and hence in contrast to the above point). These allow you to keep the concepts and ideas ‘useable’ within the confines of the trading environment.

• Problem solving capability.

• Accurate historical data.

• Plenty of time. Allow a minimum of 2000 hours (= 1 person year of effort) but more like 4000 hours to specify, create, test and complete a system that is complete with entry and exit rules, risk management and money management.

• Mentors who understand the system design process so that you can ‘bounce’ results and ideas off them. And who you are prepared to listen to for advice and constructive criticism.

• Risk capital to test the system in a live trading environment to ensure that the system is practicable and that slippage is within the bounds of the tested parameters. Live testing will also help ensure that you haven’t ‘curved fitted’ the system design ideas to suit the historical data.

• Lastly and perhaps most importantly, the unending drive and desire to continue the process when the going gets tough (it’s tough throughout the entire process!) and you are at your wits’ end.

Besides the existence of the system design paradox (as discussed last week) when you look at the necessary resources required to design a system it is no wonder that so few complete the process

If you have no prior experience with computer programming or with the creative process add more time. If you don’t have easy access to accurate data or to mentors during the process, again add more time.

You might be thinking that these are serious resource requirements. You are right; they are. This is not a trivial process. Think of it as the preparation for a business career. What business career can you prepare for in less than a few years?

It is important to realise, too, that trying to design a system while holding down a job in a non-associated industry will prove very difficult because a degree of continuity is necessary as you plough through your research. The 2000 hours should, therefore, be as contiguous as possible.

As you improve as a system designer and roll your 2nd, 3rd and more systems off the production line, each system will incrementally build on previous experience and therefore will require less time to complete.

Do not attempt to make money out of incomplete research. Most start the process of system design and are not that serious about it or find it too tough to complete. Some may start the process merely because there are available inexpensive tools or their mate had a bash and recommended they try it—usually to justify the mate’s half-hearted attempt.

Either persevere to completion or seek another path such as acquisition of a system with an ‘edge’. Don’t trade your current work-in-progress research. Only trade completed systems and don’t let your work-in-progress research slip into your live system until the research is complete.

This is extremely important because when you trade an edge you need to execute it with consistency and a disciplined approach. Achieving consistency requires self trust, confidence and objectivity. At the time of execution you must be committed to the trade as signalled according to the rules of your system. If you attempt to trade a half completed edge, any deviation from the edge will leave you in a quandary—is this ‘normal’ deviation or is it a result of the incomplete system?

This will lead to doubt, lack of trust in the system and lack of confidence in your ability to execute. That in turn will lead to inconsistency and system development ‘on the go’ manifested through subjectively letting trades go (that typically soar), doing trades that are not signalled by the system (that typically plummet), exiting early from profitable trades (that typically rocket to record highs), or exiting early from trades in loss territory (that typically turn around into handsomely profitable trades). The financial and psychological effects of this will be disastrous.

Next week we will look at how to specify the objectives of the ‘edge’ you are designing into your trading system.