Developing and backtesting systematic trading strategies

Quantitative Trading Strategies: The WHAT, WHY and HOW. Link to heading

![[289175171_696414891451337_8400780716761382367_n.jpg]]

This is a summary of my notes from a paper by Brian G. Peterson on developing quantitative trading strategies, as well as from a youtube video on What is a QUANTITATIVE TRADING STRATEGY from Hudson & Thames.

References:

  1. Best Practices in Research for Quantitative Equity Strategies by Joseph A. Carniglia, Frank J. Fabozzi, and Peter N. Kolm
  2. Developing & Backtesting Systematic Trading Strategies by Brian G. Peterson.

Keywords: Strategy, Indicators, Models, Hypotheses

A quantitative strategy is a systematic, data- and model-based approach to making investment decisions. It is a scientific approach to investments.

Characteristics. Link to heading

The characteristics of scientific approach to quantitative strategies:

  • Development of a thoughtful hypothesis to be evaluated.
  • Use of empirical work to put precision around investment decisions and economic reasoning.
  • Reliance on high standards of economic rigor
  • Use of sensitivity analysis to challenge assumptions and the context in which the strategy was developed.
  • Incorporation of adjustments to the strategy based on judgement.
  • Ability to explicitly measure results, and outcomes.
  • Incorporations of revisions/updates to the models as new information becomes available. The steps to [[Scientific design of quantitative strategies]].

Key steps Link to heading

Constraints, Benchmarks and Objectives Link to heading

Constraints Link to heading

Clearly define hat you want to achieve with your investment strategy, and understand what you are tying to achieve. This helps when carefully evaluating backtests, and ensures you do not just accept any backtest that is not in line with your investment goals.

Some investment constraints include:

  • Execution platform for the trades you want to make.
  • Accessto the markets and the products you can trade.
  • Data availability and management.
  • Capital availability.
  • Transaction costs.

Benchmarks Link to heading

The choice of a benchmark is important as it informs you on how to better understand the performance of you strategy, avoid the risks of bechmark chasing and overfitting. A choice of benchmark should reflect your investment objectives and constraints, and should be a good way to measure performance over time.

Examples of benchmarks:

  • When comparing two or more indicators, you could use one as a benchmark. A momentum strategy could use a Golden-Death crossover strategy.
  • Indices provide a goo benchmark. They give a good idea about how a certain sector is performing, a certain market’s overall performance etc. However one should keep in mind that most indice are the average of many instrunments, and thus their performance is not usually exceptional, thus beating the benchmark should just be a bare minimum in this case.
  • Custom tracking portfolios also provide a good benchmark e.g. a market-cap weighted portfolio which is rebalanced annually/quarterly. Synthetic portfolios can be constructed as benchmarks such as including assets which are competitors to your portfolio’s constituents. In choosing benchmarks, ensur you understand the limitations of the benchmarks chosen, which could include:
  • benchmarks composed of assets from a different investable universe.
  • Any specific representative features of the benchmarks.
  • The weighting criteria(i.e. market-cap weighted, price-weighted, equally-weighted), and rebalancing criteria used for the benchmarks These limitations will help you evaluate you choice for the benchmarks chosen.

Several ways to measure performance against benchmarks exist and include, but are not limited to: Alpha, Beta, Beta+, Beta-, Tracking error, Information Ratio, Active Premium, Up-capture, Down-capture, Outperformance probability, Co-skewness, Co-Kurtosis etc.

Objectives Link to heading

Your business objectives state the type of returns you require from your capital, your risk objectives, amount of leverage you are willing to take, and your drawdown constraints (closely tied to leverage)

These business objectives might be specified as ranges such as:

  • Minimum acceptables: For instance when setting the Minimum Acceptable Return (This is important since there will always be a returns objective below which a stategy is not worth doing.)
  • Maximum acceptables: This is important when setting the drawdown limit, such as a Maximum Drawdown of x%.
  • Plausible targets etc. The strategy objectives need to be formulated i such a way that they are easily used to programatically evaluate the strategy.

Common business objectives that are mostly used in portfolio optimimzation include: Maximize some measure of return subject to: Minimize some measure of risk The common measures of return include: Sharpe ratio, Sortino ratio, Information ratio, profit factor, Calmar ratio, return/ETL etc.

Other important factors to consider which might not be in the business objectives are metrics such as:

  • Correlation to perfect profit
  • Equity curve slope
  • Linearity of equity curve
  • Other trade statistics

Hypothesis Driven Development Link to heading

Since the process of coming up with investment strategies is expensive in terms of the time put in research, as well as the Capital necessary to deploy the strategy in a live market, a Hypothesis driven development approach is necessary in ensuring:

  • Confirmation of ideas quickly (in line with the objectives, and constraints).
  • Cheap rejection of failures.
  • Avoiding the danger of over fitting.

Any trading idea has to be testable, and the following steps are helpful:

  • Formulate the idea as a declarative conjecture. A conjectire can simply be a statement about the nature of the markets, or the instrunments you want to trade.
  • Ensure the conjecture is predictive. A good conjecture also makes a prediction, which must specify either cause, or an observable state that preced the oredicted outcome.
  • Define the expected outcome of the conjecture and means of testing (verifying) the expected outcome. A good hypothesis must specify its own tests. A good hypothesis always specifies how it can be verified. It is important to test statistically whether the returns of the strategy are differentiable from other measurable effects, or from noise, or from factor interactions.

A complete hypothesis statement will have the following:

  • What is being analyzed (the subject)
  • The dependent variable, which forms the output, result, or prediction
  • The independent variables which are the inputs into the model
  • The anticipated possible outcomes.
  • How each hypothesis will be validated/refutes which forms the tests

The process of coming up with hypotheses should be set clearly in order to avoid HARKing (Hypothesizing After the Results are Known). True scientific inference ensures that the hypothese are developed a priori

Defining a strategy Link to heading

1. Filters Link to heading

A FILTER helps in selecting the instrunments to trade, for instance, an analyst may decide to use a screener to filter stocks based on certain fundamental or technical metrics. This simply helps in narrowing down the investable universe. An example of the filtering is: only selecting top 10 stocks by Market Cap, selecting stocks from a certain sector only etc.

2. Indicators Link to heading

These are quantitative values derived from market data, or any other alternative data. Example of indicators include: Moving average, Volatility, Bands, Pricing models etc. An indicator on its own is not a strategy, it simply is a function of data. In backtesting, indicators are usually calculated in advance using market data, but in deployment, they are computed in a streaming fashion.

3. Signals Link to heading

The SIGNALS describe the interaction between filters, market data and indicators. They describe the desire for an action but the strategy may choose not to act on the signals. Signals include classic things like:

  • Thresholds: A LONG/SHORT/EXIT signal may be initiated when the price/spread (say) is above 0.3, or any fixed value
  • Crossovers: A LONG/SHORT/EXIT signal may be specified, when the price(indicator) crosses above(below) a given indicator(price)
  • Comparison: A LONG/SHORT/EXIT signal may be issued when the price(indicator) is always above a given threshold/indicator(price). Signals may often interact with other signals, especially when a strategy is based on multiple indicators, and thus a strategy can decide to actually OPEN/CLOSE positions when e.g.
  • Either one of the signals is ON (OR)
  • If both signals are ON (AND)

Like indicators, during backtests, signals are calculated in advance on market data, but in deployment, they are calculated in a streaming fashion. In most strategies, signals and actions are actually the same, however a seperation of the two is important in certain other strategies. The seperation serves to make clear what analysis can be done to support the decision, and what needs to be postponed until a path dependent decision is warranted.

4. Rules Link to heading

Rules are simply objects which take input from market data, indicators and signal and actually take action. Rules include:

  • Entry: Opening LONG/SHORT positions
  • Exit: Exiting from LONG/SHORT positions
  • Risk: Setting Stop-losses and Take-profits
  • Rebalancing: Either time-based or event-based rebalancing.

Strategy specification document Link to heading

Before writing the code for the strategy, the strategy needs to be documented clearly in a strategy specification document, where the strategy is completely specified, and all components relating to the strategy are laid out. The strategy specification doc needs to describe details such as initial parameter choices, as well as data requirements.

This document needs to be written in such a manner as to easily translate it into working code. The aim of writing this strategy specification document before coding is to provide a curb against over fitting. Changes to be made to the strategy specification document need to be documented, in a deliberate manner while indicating that something was incomplete in the initial specification.

Evaluating the strategy Link to heading

In this stage we want to evaluate the backtest against our hypothesis and business objectives. Common errors in backtesting include:

  1. Look ahead bias: this introduced when an analysis uses knoweldge of future events. To correct for this, most indicators should be fed with lagged values. Look ahed bias is simply using data which you couldnt possibly have.
  2. Data mining: This is simply testing multiple confgigurations and parameters without carefully controlling for the introduction of bias, simply caused by brute force search over the parameter space without a driving hypothesis for why those parameters would be a good candidate. To counter this walk forward analysis is a good candidate.
  3. Data snooping: This is the process by which knoweldge of the dataset before hand contaminates the backtest, because you have already looked at the data. This can be confused with market data, and an analyst should be careful not to make changes to the strategy to overcome specific deficiencies he/she sees in the test data.

Evaluation of the strategy should be done per component as opposed to the whole strategy at once. Methods for cross-validation and testing for the effective number of parameters in machine learning literature are valid methods and can be applied to every component of strategy design, rather than on the whole strategy, as found in [[FEATURE ENGINEERING HAND BOOK]]. The different components of the strategy include the filters, indicators, signals, and the rules. Testing the components individually ensures that every component is working well in validating our hypothesis, before moving onto other components.

Benefits for Testing individually Link to heading

  1. Testing individually guards against over-fitting, and especially the risk of data-snooping, rejecting an indicator, signal or any strategy component as quickly in the process guards aagainst trying to over fit the models on data.
  2. Allows construction of tests which are specific to the technique.
  3. Saves time. Since analysis, and strategy formulation and creation is a time consuming process, testing individually saves time, since by testing individual parts independently allows us reject badly-formed specifications earlier in our analysis.

Evaluating Indicators Link to heading

An indicator could be evaluated in the ML-fashion approach, where we costruct training datasets, validation and test datasets. We then proceed to label data in the training dataset, where we aim to capture a specific feature of the price movements e.g. For a turning point indicator, we could label the training data, where all turning points occured, on the price series or the smoothed price series, using features learnt from the training data such as volatility etc. We could then train a model to learn turning points on the traininig set, and then tune it over the validation set, and finally deploy it on the testing set to measure performance.

The danger of this approach is over-fitting, and this is best guarded by the analyst being keen on whatever he/she is doing.