Would be nice if the Define Network Pane would sort items alphabetically when you SaveAs a new NN. Now they are only sorted after you close and open the Neuro-Lab window.

Also would be nice if a runtime error would terminate the NN Apply Training Set, instead of continuing to 100%. I am seeing an Index Out of Range Error probably due to incomplete input data would be easier if it just stopped at the first error...

James,

Which version and build of Wealth-Lab are you using?

Hi Eugene,

I am using 6.4.52.0 and Neuro-Lab 1.0.2.1.

This is a trial, I am a new user, have lots of programming experience but not much C# .Net.

When I clicked on Apply Training Set to a Yahoo dataset of 200 stocks, I get a message "Input Script contained errors" with Progress around 8%. Then Neuro-Lab keeps going behind the scenes until progress reaches 100% and no output is produced in the Statistics pane. For a large training set this can take some time. I would prefer if it stopped completely at the first error.

If I go back to the Input Script pane I see the message Runtime error: Index was out of range. Must be non-negative and less than the size of the collection.

Now I found by clicking on individual stocks that the error occurs only on some stocks, maybe a data completeness problem with Yahoo Finance ASX data.

I tried inserting PrintDebug messages into the Neuro-Lab script but didn't see any output in the debug window.
What is the best way to write a log from inside the Neuro-Lab script?

Would I be better off testing my code in a strategy first and then put it into Neuro-Lab inputs?

Hi James,

QUOTE:
I would prefer if it stopped completely at the first error.

Sure, just click on "Input and Output Scripts for Errors" before running.

Found your repeat trial TraderJim63. Since the penultimate trial was in 2011, this one's OK with us.

Sorry, I started to do a trial a while back and got distracted by other jobs.

I have more time now and am doing a thorough test with 18 years of ASX history.

Neuro-Lab:

It's one of the main things I am interested in. As I said in other post I am trying to work through Bruce V's book as a learning exercise.

I'm trying to understand his methodology, NN inputs, portfolio simulation and monte carlo testing etc...

I'm just getting up to speed on the best way to debug inside a scripting environment with a programming language I don't know all that well.

As you said in your other post, incomplete data can cause errors. I found the free Yahoo data provider is a bit inconsistent with ASX data. Some symbols end up with no data.

I have purchased some better quality data and getting the hang of the scripting environment better...

Here are two relatively simple NeuroLab Enhancements to improve the training process. They are my order of my priority. These are not my original ideas, but features of other ANN software I use.
1. During training, graph the out-sample error (from the same portfolio) in addition to the training error, in a different color. Up to the point of over-training, this will draw parallel lines. When overtraining starts, the lines begin to diverge. Since this option adds processing, it may be prohibitive for large datasets, so also add a checkbox or radio button to turn this feature on and off. This feature takes out a lot of the guesswork.

2. Add another stopping option, to stop when the error reaches a user-specified value. Consider: I try a set of inputs and after training the minimum error of my dependent variable is, say, 10. Trying to improve that, I change the input script. On my next training run, I can enter that value(10) and have training automatically stop when that threshold is reached. This has the added benefit of always stopping training on an "Error Rate Low".

Hi Leonard,

Your suggestions are much appreciated. I'll pass them on to our NL wizard but frankly, we can't promise anything now.

NeuroLab - Detecting erroneous Neural Network values

Background:
When a Neural Network(NN) is used, but inputs at a bar exceed the boundaries of the data that was used for training, the resulting output value is not reliable. If used to signal a trade, results are unpredictable. It would be a good idea if a strategy could detect these boundary violations.

I can see in the NeuroLab NN XML that the necessary information is available. The Mins and Maxs used for training are captured in tags <MinInput> and, <MaxInput>.

Request:
I would suggest a new Indicator be created, perhaps called "NNMaxExceedPct", which returns a DataSeries having the maximum percentage of the range (0 to n) that any input at each bar was outside of its training range, always as a positive number. For example, if an input's range is -5 to 5, a range of 10, a value of -6 at that input, 1 below the range, would return 10, 10% out of range. Return zero where all inputs are in range, the usual case.

A strategy could use this indicator to detect potentially unreliable NN values, possibly accepting small violations.

An alternate design might be to add another signature to NeuroLab's NNIndicator.Series having a third parameter, an output DataSeries to return this data. Advantages? This data is easily gathered as the NN calculation is run. Simplifies documentation, too.

Usage:

CODE:
Please log in to see this code.

Could you give response to #9, please? Do you see the issue? Internally, the neural network calulation's first step is to normalize the data to values between 0 and 1. When input data is out of range, those first values are not as expected. This then gets multiplied through the network and into the answer.

BTW, Regarding the alternate solution, I recognize the bad programming practice of returning values (the DataSeries) in the paramlist of a function. It should be a procedure call.

Len

Hi Len,
Thank you for the thoughtful post. So that your idea won't get lost, please create a support ticket so it could be added to our list of potential enhancements.