  |
Best viewed with MS Internet Explorer 5 and above |
Today:
- Page last modified:
December 08, 2007
Copyright © ForeTrade Technologies 21st century and thereafter
1 Some results seem to good to be true ! Well, they may just be as good as they look. However, you should make sure your patterns still behave well in out-of-sample data, or at least exhibit good statistical stability. The GPF allows for such comprehensive testing. 2 You may want to know more the 'internals' of the G.P.F. You are right. Even though genetic algorithms are complex to understand, this is not meant to be a 'black box'. You are entitled to know what's in there, and we have spent much time describing here how the G.P.F. actually works. We also recommend acquiring Ward Systems' GeneHunter which technology is used in the GPF. 3 You want to better tune the G.P.F. Yes, to make it versatile, we had to give the user full control over many parameters, and sorry to say that, but this may also mean some home work on your side down the line. Ultimately, this will guarantee that the patterns you detect do suit your trading style. We advise you to use default parameters and then start exploring! You'll soon realize how to guid the genetic process wherever you want it to.
Having said that, you may just feel that there are a number of issues regarding optimizations which may need further clarifications. (This page is however not a tutorial on genetic algorithms: please click here if you wish to know more on that topic) Preliminary caveatThe first question we would like to answer is : Is the G.P.F suitable for every trader? We would like to think it can be used by all, but one should not be fooled by its apparent ease of use. It is more a toolbox than a turnkey system, and thus still requires some work to be used efficiently, work that only advanced traders may be willing to do. The whole point of the G.P.F is to SEARCH for patterns. So we deliberately give the product maximum flexibility to make sure users could reach that objective. At the end of day, see for yourself. The G.P.F. is free for you to test for 30 days, and will then still run as a comprehensive working demo. Please
click here to read a special note to
|
| Date | Close | P&L (1 day momentum) | Signal | P&L x Signal | Accumulated |
|---|---|---|---|---|---|
| n | 100 | 3 | 1 | 3 | 3 |
| n+1 | 103 | -5 | 1 | -5 | -2 |
| n+2 | 98 | -7 | 0 | 0 | -2 |
| n+3 | 91 | 8 | 1 | 8 | 6 |
| n+4 | 99 | N/A | N/A | N/A | 6 |
In a nutshell, the genetic process will search for patterns which will generate signals in such a way that the "Accumulated" column (last on the right) yields the highest possible figure.
The G.P.F. also offers among other things, the possibility of cutting losses, filtering small profits, and ignoring large infrequent gains (outliers), making it a very versatile tool.
There are 8 basic fitness functions offered in the G.P.F. for 8 different objectives as listed below. Alternatively, you can use a ready made column in your spreadsheet (called Profit Benchmark) which will contain the profit potential you are looking for. Some users do try to find how some patterns in a share (e.g. AMAT) can affect a profit potential in another share (e.g. INTC).
| Fitness | Momentum or Rate of Change | Long / Short * | Close on Close or Profit Target** |
| 1 | M | L | CC |
| 2 | M | L | PT |
| 3 | RC | L | CC |
| 4 | RC | L | PT |
| 5 | M | S | CC |
| 6 | M | S | PT |
| 7 | RC | S | CC |
| 8 | RC | S | PT |
* Short can obviously mean an exit signal for those trading long only.
** There is an option to generate signals on the next open instead of the current close. Price Target objectives should be used with caution. They inherently cannot return negative values, therefore could mislead the user.
Not necessarily. As said above, fitness, i.e. the optimization objective is calculated as a sum of potential absolute or relative gains. Adding up absolute profits gives a correct figure, while adding relative gains is mathematically incorrect.
U.S. shares have fluctuated widely in a ratio from 1 to 3 for the S&P 500, from 1 to 5 (tech stocks) or even up to 1 to 10 (internet stocks) in the last couple of years. Using absolute objectives is of course likely to favor recent gains versus older gains, and will affect the shape of Equity Curve. It is up to the user to look for stability using linear regression for instance, out of sample testing, or update patterns on a regular basis.
All signals are by default detected day after day (bar after bar to be precise) regardless of previous signals. The Boolean signal can be used to confirm a position, close a position or accumulate in the position. The G.P.F. is not a fully fledged trading system by itself. While optimizing, it only searches up to the number of 'Bars Ahead' to calculate potential profits. Unless you wish to trade short term and stay in position up to the number of 'Bars Ahead', you should want to use your own position management technique (trailing stops, profit targets, types of entries...).
Enabling pyramiding allows to ignore patterns if already "in position', and it may be sometimes advisable to do so. Allowing signals to be detected on all bars may generate a lot of mediocre signals, which once summed up can still return an apparent good profit. You may like to trade lots of small positive signals, and the direction of trades may be statistically often correct. You may therefore find such signals interesting. This is more a 'by-product' than a good use of the G.P.F. optimization capabilities, but here again, there are so many trading styles, that we left that to you.
If you decide to use pyramiding, it is recommended to filter out all signals where the profit potential is no significant, for instance those lower than the gain average (as displayed in the status bar at the bottom of the screen after data is loaded in memory)).
Additionally, from a conceptual point of view, separation of functions is often desirable in trading system analysis. Such 'raw' signals have a very specific meaning and for instance can make excellent inputs to a neural net, or any other logical decision making system. It is pretty easy to wrap any technique into a trading system, using a built-in trading technique, which first may not be yours, and secondly could conceal poorer entry signals. Again, the G.P.F. is focused, and does its job well.
As you can see, it is up to you to tune the G.P.F. according to what you wish to get out of it. We personally believe pyramiding should be allowed (with or without filtering) if you wish to use the G.P.F. signal in a more complex system, such as an input in a neural net. If you wish to trade on the G.P.F. signal, rather disable pyramiding, add some trailing stops and stop losses, and there you go !
No. Strictly speaking, the Equity Curve is only valid on "Close on Close" momentums signals (see table above, signals 1 and 5).
In 4 other cases (Profit Target signals 2,4,6, and 8 ), the optimization technique is more aggressive, as G.P.F. attempts to only recognize the best patterns, i.e. giving the best returns. Lower or negative returns are excluded from the optimization. For instance, with signal type 2, the objective is calculated Close to HighestHigh, which is almost always a positive value. The equity curve is therefore bound to be upward going. Even though it is not a true equity curve per se, the shape of the curve (slope, volatility) already give good indications on the quality the signal.
Using threshold can also help filtering patterns. There is however a risk of distorting the statistical stability of such signals, in other words, "replicability" of those patterns may be questioned. The GPF does provide in-sample and out-of-sample statistics to alleviate such problem.
In the last 2 cases (Rate of Change signals 3 and 7), the curve shape is correct, and can be used to spot draw-downs. It is however not an Equity Curve as such. It would have been possible indeed to compound returns to reflect a correct equity curve. However, this would have biased the optimization toward reaching higher returns first. By adding returns, the statistical distribution of returns is quasi-normal, i.e. random.
All optimization processes may be stuck in a local optimum, instead of improving toward the best optimum solution. You may then consider increasing the Fitness Peaks parameter. A value between 25 and 50 is suitable in most cases, and the overall process then takes a little over 100 generations to reach an optimum. By experience, all other process stops (time, fitness plateau, number of evolutions) may be more suspicious. You should always increase those parameters until the process stops on a fitness peak. Regardless, genetic algorithms are undoubtedly the best technique available for such optimization, and should therefore give you entire satisfaction. We often find that the second or third best pattern does actually show better statistical stability
As said before, it is not meant to be a panacea, a magic box, therefore your critical mind is the judge in the last resort. It may indeed be possible that there is no good pattern to trade with on a particular stock. We believe that finding this out is already an information per se, ... and an invitation to move on to other stocks.
The stochastic indicator used is a simple Fast Stochastic K. It made more sense to use a "raw" overbought / oversold indicator unaltered by the smoothing / lagging effect of a moving average. The length of the indicator as well as the optimum level are detected through the same genetic optimization process. The length cannot exceed the number of bars back set in the search space setting.
While optimizing, only signals which fall within the same StoK range will be considered. For instance, the genetic process may find that the highest profit comes from signals occuring on bars where StoK(5) is betwen 60 and 80.
This is a difficult question to answer. Filtering will undoubtedly improve the quality of signals. It will however also to an extent increase the chances of over-fitting.
The G.P.F. may detect a large number of poor positive signals. It may be interesting to use the threshold parameter to make the G.P.F. ignore poor signals. Large infrequent signals (outliers) may also be filtered.
Stock market returns do not follow exactly the bell shaped curve of random returns (fat tail distribution). It is up to you to aim for returns that fit the close fitting bell curve, or aim for the rare extra-ordinary returns. The G.P.F. gives you all latitude to select exactly the type of signal you wish to detect using the right measure
It is likely that the registry settings have been tampered somehow. Any direct modification into the registry can have unexpected consequences on the behaviour of the G.P.F. It is recommended to delete saved settings ("Other Settings" tab on the main form), and save your favorite settings again.
If this happens after a number of generations (usually 15 to 50), this means that your parameter settings are too constraining, and no patterns can be found. Try and relax some of them a little.
Yes, absolutely. Two methods are here possible:
1. The supplied SP500.XLS file contains 2 data sheets. While the initial optimization is carried out on the larger data file from January 1991 to April 1998, the other data from the second data sheet is unknown to the G.P.F.
When the process is completed, and one or several solutions are listed on the Analysis form, you may then go back to the Main form ("Data Source" tab) and load the data contained in the second data sheet in memory. Instead of pressing "Start G.P.F.", just go to Analysis. The same patterns appear listed as before, but clicking on a listed pattern will calculate new statistics based on the new data. You may then compare with the original data or any other data, just by loading new data in memory.
If the test data is not available on the same spreadsheet, just press "Hide" instead of "Close", and open the spreadsheet containing the test data. Pressing "Close" resets all variables in memory.
This functionality goes beyond in-sample / out-of-sample. As long as the pattern remains in memory, you may go often go back and forth from the Analysis form to the Main form, change a setting and see what how it affects profitability. You can even apply a pattern to another file. It has been suggested that patterns found on sector leaders (MSFT, INTC, IBM, ...) may have an incidence on other shares in the same sector.
For simplicity and convenience, it is recommended that to keep In-Sample and Out-of-Sample data in the same Excel workbook.
2. You can hold out a portion of your data for out-of-sample analysis. It is simpler to use, and does not require file and data manipulation.
The G.P.F. detects a winning pattern and translates it into a Boolean
rule (i.e. Yes/no or True/False), which can
be interpreted as an indicator or as a signal. That Boolean output can be easily turned
into a neural net input. However Booleans do generally not constitute good
patterns in feed-forward neural nets (like back-props) why try to fit a continuous
functions through the training set data. You should therefore use them with
caution. We generally use many concurrent patterns to alleviate that
problem, but there again you may over-fit your data.
Boolean inputs can probably be best used in classification nets, or in nets themselves
predicting a Boolean value. In other words, the neural net should be trained to
confirm GPF signals (ensemble net).
It would be interesting to generate up to 10 G.P.F. signals over several bars in the future, for instance 1,2 and 5, using several lengths back (search space), for instance 5, 10 and 15 days (or bars) back, and with 2 or 3 components. You must be careful to avoid patterns to be too much correlated to one another. Detecting the best patterns for each day of the week can also be very interesting, as they may be less correlated.
Careful analysis of each individual signals generated is needed. Using a stop loss, outlier filtering, or a genetic stochastic K may be of interest, but more often that not, this will distort the signal and affect its stability. You should always check your patterns on out-of-sample data.
Please see our new neural net page.
We would like to warn against using the Highest High / Lowest Low type of patterns as "regular" G.P.F. patterns. They are not. Such patterns do always look good, and so may easily be misleading. At any point in time, the Highest H ahead, is almost always higher than current close or next open, so the fitness fitness function which sums up the profit potentials over the entire data set, will always be positive or null (See the paragraph on the Equity Curve for more details). The point of attempting to detect such type of patterns is that a plain close-to-close pattern may not see a pattern formation like a doji or hanging man candlesticks simply because the high of the bar is not considered. I would therefore say that this kind of pattern detection is very specific and should only be used to complement an existing set of GPF patterns.
Such signals should therefore be regarded as 'secondary' signals, simply because their statistical distribution is very skewed. They should be used along with filtering to detect few great price movements which could have been overlooked by other types of optimizations. Should you still wish to trade on such signals only, you should be cautious and use a sound money management technique.
Having said that, they do provide great contribution to profits in consensus based systems.
This is one of the advanced features of the G.P.F., and does indeed require some clarification.
Pyramiding is a parameter from the "Patterns and Trading" tab on the main form. If 'Pyramiding' is ticked off, then during the optimization process, the signal is triggered every time the current pattern is detected. It does not check whether the previous bar was "in the market". This will generate many more signals. The signal is only reset when the pattern condition is no longer met. On the other hand, if "Pyramiding' is disallowed, a signal can only be activated for next bar if the current signal position is off. Being a genetic parameter, this does impact dramatically the optimization process. It is NOT a post-processing filter.
Using this parameter depends mainly on how you interpret signals. The G.P.F. does not pretend to be a system generator as such, and position management should be left to you usual charting program. In short, the 'pyramiding' parameter allows to enter into a position while already in position. It is disallowed by default.
Leaving pyramiding out also allows the G.P.F. to focus on the best entries, while using it will concentrate on building up profits.
Example:
| Current signal | Pyramiding allowed | Pyramiding not allowed | |
| Day n | 1 | 1 | 1 |
| Day n+1 | 1 | 1 | 0 |
| Day n+2 | 0 | 0 | 0 |
| Day n+3 | 1 | 1 | 1 |
| Day n+4 | 0 | 0 | 0 |
This all happens during the actual optimization. The consequence is that resetting a signal to zero impacts the fitness function, and may then lead to a totally different pattern being detected. Again, this is NOT a post-processing filter.
Advanced only: Should you wish too, you can test the same pattern with and without pyramiding. When a pattern search is completed, you can always click on 'Analysis', display and eventually record stats and charts, then come back to the main form, change the 'pyramiding' parameter and click 'Analysis' to recalc your pattern. It probably won't return an optimal solution though. Any post-processing changes will undoubtedly affect the results.
Signal always occurs on day 0. The search space extends from current day (Day 0) to the number of days (or bars) back as set in the program. Within that search space, all OHLC quotes will be considered during the optimization process. If you choose to use the Day of Week selection feature, you will see the Day 0 or signal day, which is the current day if you trade on current close, or the day before entry if you trade on next open. This is why you may see WD=Friday on the analysis form to express a up day on the following Monday.
If you trade on close, Day 0 will also be the day of trade. One may argue that trading on close is not possible, however the pattern may trigger a signal irrespective of the current close level, and slippage can be built in ! With Instinet, or Globex, what does Close now mean anyway?
The number of days ahead set before loading the data in memory serves to calculate the profit potential in the short, medium or long run. You may actually decide to exit the market within or after that same number of days (or bars). Alternatively, you will use your trade management in a way that is consistent with that setting. You may also use a subsequent signal to confirm a signal or trigger an early exit.
There is no big difference if you trade on next day's open. You should however understand that the same number of days ahead is used from date of entry, which actually means one more day from current day.
We can only be sure of one thing, namely, providing settings are right, the G.P.F. will ALWAYS find optimal or near optimal patterns. This does always mean there is always a good pattern to trade with. In particular, a share that has been trending for a long while, may give you obvious patterns in the direction of the trend, which could be inappropriate when the trend stops. Volatile shares, or range-bound shares may sometimes be easier to trade with.
As a rule, if there are two many good patterns, it may be advisable to use some filtering. If on the other hand, there are no stable pattern over the period of the analysis, the market configuration may have changed, and the optimization may be improved on a shorter, more recent data sample. At the end of the day, stock-picking is still left to your intuition and/or your fundamental analysis (beta, volatility, other correlations,...).
User validation has been implemented in the final release. While it does not pretend to be as secure as security blocks placed on the computer parallel port, it is a reminder that the program is not meant to be installed freely. All users genuinely interested in getting the most out the G.P.F. won't waste time and effort on hacking and cracking the software, and there is a pretty good chance it will only block the G.P.F. permanently.
The G.P.F. may be blocked if trial period is over, or if users tamper with the Window Registry. Of course, the registry may be corrupted. Registered users who incidently block their program must re-enter their validation codes. In the unlikely event the G.P.F. would still be blocked, please contact our ForeTrade Support. We will then need to send you a separate validation utility program.
At this stage, trial users can only be unblocked by entering their registration code. Versions 1.5+ require new codes. Please contact us before upgrading.
A special note: deleting registry settings does
NOT revert
to the original state where the program could be downloaded again.
(but don't take our word for it... try it! :) )
It is possible that settings recorded in the registry need to be reset. Please delete them and save them again ("Other Settings" tab on the main form).
This occasional problem is likely to occur when updating/upgrading your program with the latest version.
Some parameters are accessible to all users, some are reserved for GeneHunter registered users. Population Size is one of them. However distribution rights include the possibility of a calculated parameter. The G.P.F. therefore automatically adapts the size of the population to the complexity of the problem, i.e. the size of the search space. In order to give users a little more flexibility, three choices are offered. There is to an extent a compromise to be found between accuracy and performance. The first two choices are good enough in most cases. The largest population may be used to exhaust all search possibilities.
All patterns listed in the Analysis form can be written back into the workbook. At the end of the optimization, pressing the "Write Signal Data to File" does recalculate all signals over the data series and then write them in separate columns on the data workbook. This helps tracking signals historically.
The potential P&L for each bar is calculated, sorted into classes and then listed in the first column, followed by signal data. Columns can be deleted manually. This new command button always adds new columns to the right hand side of the worksheet regardless of previous optimizations. In order to remain consistent, you may want to save your file under a new name, if you change of optimization objective.
You can then copy the whole file to your neural net software and train a net with these inputs. Additional inputs will probably be needed as data is likely too sparse to bring about a consistent neural net. We believe that the best use here is to process this file through a combinatory system, fuzzy logic, or any scoring system to decide from you batch of patterns the best composite pattern.
The P&L scoring of course contains positive and negative figures. It is therefore recommended to run on or several consistent long objectives optimizations, followed by their equivalent short counterparts. This will better balance your set of inputs.
Please contact us if you have any difficulty with this feature.
On earlier versions, stats used to be displayed on the Analysis form for testing, but were not recorded in your workbook. When you optimize on sample data, and then test on out-of-sample data, you may want to save both sets of statistics. Saving stats always adds a new column on the right hand side of existing data in a separate sheet. This will help you also compare stats from different patterns. Columns must be deleted manually.
Since GPF V1.2, the chromosomes values are now also listed. These values must be entered in the NeuroShell Trader and Omega Research interfaces.
The system is a pseudo-signal written in a Omega Research EasyLanguage syntax. For simplicity, not all GPF settings are restituted though.
Please contact us if you have any difficulty with this feature.
Older versions could only take up to 32500 quotes (2^16 -1 to be exact). Version 1.52 and above should handle full length worksheet, but it is not advisable to use so much data anyhow.
The GPF fails with IntraDay data (time value)
When using IntraDay data, the GPF assumes the time value is a text value, otherwise, it will convert automatically to time format, i.e. a decimal value from 0 (midnight) to 1 (midnight next day). The easiest way to overcome this is to use the TEXT worksheet function =TEXT(timevalue,"hh:mm: ), where timevalue is the cell you wish to convert. A sample spreadsheet is available.
Last edited: Friday, 03. March 2006 09:50 AM Central European Time
Today:
- Page last modified:
December 08, 2007 |