This series of articles serves to illustrate the development process used for my algorithmic (or automated) trading strategies. There are two main development workflows, depending on your choice of software. The first uses Metatrader 4’s strategy tester module, where we build a strategy from the ground up by incrementally adding entries, exits and finally filters. The second workflow uses StrategyQuant, where we first run a genetic algorithm to automatically generate a basket of profitable strategies, which will then be subjected to several rounds of selection and robustness testing. After developing a number of strategies, our focus then shifts to portfolio composition and monitoring techniques for live trading.
This process is meant to be used as a reference or starting point; feel free to customize it to suit your particular trading style or market. Since I primarily trade the spot forex market, most of the examples will be geared towards currencies. As a primer for the development process, this article will address the following three areas:
- Understanding the general concepts of algorithmic strategy development
- Establishing your trading preferences and goals
- Obtaining the required programming and trading software
2. General Strategy Development Concepts
2.1. What Is an Algorithm?
An algorithm is a computer program that converts an input to an output using a quantitative set of rules. In the case of trading, the input would be the market prices from your broker, while the output would be a certain trading action. Trading rules usually take the form of IF/THEN statements. For example, IF the price exceeds the 20-day moving average, THEN enter the market. Figure 1 illustrates the concept.
Figure 1: An algorithm uses your trading rules to convert incoming prices to trading actions
Your challenge as an algorithmic trader is to determine a profitable set of rules. Evaluating your profitability requires backtesting the algorithm using historical price data.
2.2. Robustness
Robustness is a central theme in systematic trading. Simply put, robustness is the ability for a trading strategy to perform well over many markets. Because markets are always changing, I consider robustness to be the single most important quality in a strategy.
Robustness is typically achieved by having a simple set of trading rules. As the saying goes, loose pants fit all. Most authors in the trading literature seem to agree that a robust strategy should contain no more than four trading rules. Adding too many rules will constrain your algorithm to very specific market conditions. Your backtest will undoubtedly look great, but your live trading performance will suffer once the market deviates from your backtest conditions.
2.3. Optimization and Parameter Stability
Once you have determined your trading rules, you will need to decide whether you wish to optimize its parameters. In Figure 1 above, the lookback period of the moving average may be optimized. A 50-day moving average may perform better than a 20-day moving average, especially if your market is often choppy. If you plot your strategy performance as a function of your lookback period, you will likely get an optimization chart that resembles that in Figure 2.
Figure 2: Hypothetical graph of strategy performance vs. lookback period
When selecting your parameter value, look for high plateaus on the graph. The parameter should be in a stable region, such that minor changes in parameter value will not drastically affect strategy performance. In the graph above, a lookback period of 73 would be a more prudent option than 54. Although trading performance appears to be weaker, the strategy will be far more resilient to changing market conditions.
Most trading strategies contain more than one parameter. For example, in addition to the price > moving average rule in Figure 1, suppose you wish to restrict your market entry to certain hours of the day. You may be wondering whether you should optimize your trading hours as well. If you compare your optimization charts obtained before and after optimization of the second parameter, you may get either of the two graphs shown in Figure 3 below.
Figure 3: Two possible ways that optimizing additional parameters can affect strategy performance
The graph on the left shows that optimization of the second parameter has increased performance over a broad range of values, and is thus a robust improvement, whereas the graph on the right shows an increase in peak performance at the expense of robustness. For the second scenario, it is best to assign a constant value to the additional parameter, or omit the parameter entirely.
If you do decide to have several optimizable parameters, try not to optimize more than one parameter simultaneously. Apart from a serious increase in computation time, simultaneous optimization increases the risk of overfitting your trading rules to historical data, reducing your strategy’s robustness.
For a more in-depth look at robustness in trading strategy development, I recommend the excellent book ‘Trading Systems: A new approach to system development and portfolio optimization’ by Emilio Tomasini and Urban Jaekle.
2.4. Backtest Precision
As a trading strategy developer, you will be running tons of backtests. Regardless of software, you will have to balance the conflicting requirements of test precision and computation speed. Due to the countless variations of trading strategies that exist, I recommend a less precise backtest during the early stages of development. A high precision backtest will only be done on strategies that have passed all the selection criteria specified in the MT4/StrategyQuant workflow sections. The vast majority of strategies I have developed tend to perform equally well or slightly worse (up to 10% poorer performance) when moving to the high precision backtest. There is a small possibility that you will miss a great strategy by testing it with a basic model in the early stages, but the overall time savings will more than compensate for it.
3. Trading Strategy Preferences and Goals
To achieve long-term success, you should trade a strategy that suits your trading style and risk appetite. The following are some factors you may consider when determining your preferences.
3.1. Trend Following vs. Countertrend
Trend following (or momentum) strategies are reactive in nature, entering the market in the direction of an emerging trend. Sustained trends occasionally appear in the markets, usually caused by fundamental economic factors and subsequently reinforced by market participants’ herd mentality. The average winning trade is typically much larger than the average losing trade. Unfortunately, markets do not trend often, and most trend following strategies have a win rate of only 40-50%. Equity curves of such strategies mostly consist of flat periods punctuated by a few large increases. Figure 4 shows a 16-year backtest of a basic trend following strategy on GBPJPY.
Figure 4: Typical equity curve of a trend following strategy
Countertrend strategies bet on the reversal of a temporary price extension. This means buying when the market is oversold, and selling when it is overbought. Their characteristics are typically the opposite of trend following strategies: win rates are high, but losers are much larger than winners. Figure 5 shows a 12-year backtest on GBPCAD. The strategy had an 82% win rate, but losing trades were 3X as large as winning trades on average.
Figure 5: Typical equity curve of a countertrend strategy
To exploit the benefits of both types of strategies, it is best to trade a multi-strategy portfolio. Portfolio composition is addressed here.
3.2. Trading Timeframe
With a fully automated trading strategy, part-time retail traders are no longer restricted to the higher timeframes. Some factors to consider when picking your trading timeframe:
Strategy style
Market noise is random price movement that is unrelated to any underlying trend. Higher timeframes contain less noise, making them more suitable for trend following strategies. Conversely, countertrend strategies usually perform better on the lower timeframes.
Time spent in market
Trading a higher timeframe likely means you will holding positions for longer periods, exposing you to more risk. Markets can abruptly reverse for a multitude of reasons, ranging from poor economic reports to political upheavals. If you trade an illiquid market and hold a position overnight or over the weekend, be prepared for price gaps when the market reopens.
Backtest reliability
The more trades you have in your backtest, the more reliable the results. Trading frequency will be lower on the higher timeframes, making it difficult to get a sufficient sample size for your backtest. If you have only a few years of historical data, an hourly timeframe or lower is recommended.
3.3. Trading Goals
Setting goals encourages you to evaluate your trading preferences and helps keep track of progress. Goals usually address the following aspects of trading: returns, risks, and time commitment. Examples for each are given below:
Returns
- % return per month/year
- % winning rate, or % of profitable months
- Number of pips/month
Risks
- Maximum % drawdown
- Longest stagnation period
- % of time in market
Time commitment
- 10 hours/week for strategy development
- 2 hours/week for literature reviews and journaling
4. Strategy Development Software
Here I will list the software I use to develop and trade algorithmic strategies for the forex markets.
4.1. Trading Platform
I use the ubiquitous Metatrader 4 (MT4) platform, which predominantly focuses on the forex markets. Most brokers offer MT4 as a free download. Its proprietary programming language (MQL4) allows traders to develop trading algorithms (called expert advisors in MT4) and custom indicators for the platform. The Strategy Tester module can run backtests up to tick-precision, and it even includes a visual mode that shows the algorithm running over the price data in real time. I have fixed many programming bugs with this feature. Figure 6 shows the visual mode with a Bollinger Band reversal strategy being backtested.
For a step-by-step guide on how to use MT4’s Strategy Tester to debug and backtest your strategy, refer to my blog post here.
To learn how to optimize your strategy parameters in MT4, check out my guide here.
Figure 6: Visually testing a Bollinger Band reversal strategy. Blue/red dashed lines indicate long/short trades executed.
Metatrader 5 is also available. Apart from having better inclusion of the stocks and futures markets, MT5 is multi-threaded and thus runs backtests more efficiently than MT4. Nonetheless, as of 2020, MT4 is still the most popular forex trading platform.
4.2. Programming
Converting your trading idea into an executable program is the cornerstone of algorithmic trading. Metatrader 4 comes bundled with its MQL4 language editor that allows you to code your algorithms from scratch. Learning a new programming language is a daunting task for many though. Fortunately there is software that can quickly and easily convert human-readable pseudocode into MQL4. I use AlgoWizard, which is included in StrategyQuant. Figure 7 shows a basic Bollinger Band breakout strategy being programmed in AlgoWizard. Entry and exit rules are easily coded with a few clicks.
Figure 7: Using AlgoWizard to program entry and exit rules
A few websites let you code your own algorithm for free, though I have not used any.
4.3. Price Data
Accurate price data is essential for reliable backtesting. A good dataset covers a long historical period and contains minimal gaps. As of 2020, you should have little trouble obtaining at least 10 years of data for liquid forex markets. Many forex brokers provide free historical data at 1-minute precision, which can be downloaded through MT4. This means the OHLC prices for every minute in your historical period can be modeled in your backtest. I have found Alpari data to be among the best. For the forex majors, data since 1999 is available.
A guide to downloading free historical data can be found here.
Tick-precision data allows every price movement to be modeled in your backtest. You may require such precision if your algorithm takes advantage of small price movements, or often executes actions in the middle of a bar. Tickstory and Tick Data Suite are two popular providers of tick data. I have used Tickstory with success; the MT4 backtests in Figures 4 and 5 were obtained using Tickstory data.
To learn more about whether you require a tick backtest, and some sources of quality tick data, refer to my article on tick backtesting.
Note that if you are trading a decentralized market such as forex, data will vary slightly from source to source.
4.4. Strategy Generation & Analysis
Nowadays, StrategyQuant has superseded MT4 for the vast majority of my strategy development. It is a comprehensive platform that can fulfill every step of my development process. Your strategy can be easily programmed using AlgoWizard, or you can harness the efficiency of StrategyQuant’s genetic algorithm to automatically generate profitable strategies. These strategies can then be put though a series of inbuilt robustness tests, such as walk-forward optimizations and Monte Carlo simulations. The inclusion of these key functionalities into a single platform makes it both easier and faster to produce profitable and robust strategies. StrategyQuant’s strategy generation and robustness testing capabilities will be explained in more detail as we progress through the development process.
StrategyQuant also provides a backtest analysis tool called QuantAnalyzer. With the free version, you can obtain detailed analyses of your MT4 backtests, merge strategies to create a portfolio, and experiment with various trading filters and money management rules. Figure 8 shows an example of the analysis you can do – the profits/losses of individual trades are plotted as a function of the holding time. QuantAnalyzer will be referenced at various points throughout the development process, for example during incubation and portfolio composition.
Figure 8: Trade profits/losses vs. holding time. This is typical of a trend following strategy that captures large trends.
4.5. Virtual Private Server (VPS)
While not technically a software, a VPS can be used to run your trading platforms 24 hours a day. Using your personal computer is possible of course, but I prefer to rent a reliable server specifically for this purpose. Discussions on whether you need a VPS, and some factors to consider when selecting your provider, are covered in the Live Trading article. I currently use FXVM and am pleased with their performance.
5. Conclusion
This article has outlined some general concepts in the development and testing of algorithmic trading strategies. Your strategy’s robustness should always be prioritized; robustness can be improved by implementing simple trading rules, and selecting stable parameter values. Adhering to these concepts, and establishing your trading preferences beforehand, will accelerate your progress towards your trading goals. Finally, several popular trading and development software packages were introduced. MT4 is available as a free download, and successful strategy development is entirely possible using MT4 alone; the MT4 workflow section addresses this. Alternatively, StrategyQuant offers vast improvements in terms of development speed and robustness testing. With a free 14-day trial, it is certainly worth your consideration.
With numerous markets available at your fingertips, deciding which to trade can be a daunting task. It pays to quantify the characteristics of your markets, and focus on markets that are aligned with your trading preferences. The next article on Market Research will discuss this.