Out of memory - Wealth Lab Forum

traderray

#1

12/6/2010 9:55 PM

I have the following strategy.

CODE:
Please log in to see this code.

This strategy was working great and then I added the loop inside the new loop comments along with the associated processing for the new dataseries. The original strategy would work on a large number of datasets. With the new logic, I get an out of memory error on the 2nd or 3rd dataset.

Can someone see why I am getting this error.

I am also new to C# so if anyone sees programming improvements that I can make to the strategy that would help me learn I would appreciate the help.

Thanks.

Size:

Color:

I have the following strategy.

[CODE]using System;
using System.Collections;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using WealthLab;
using WealthLab.Indicators;
using System.IO;
using Community.Components; // GetDataSetName here
namespace WealthLab.Strategies
{
	public class MyStrategy : WealthScript
	{
	    private string Level;
		protected override void Execute()
		{
			DataSourceManager dm = new WealthLab.DataSourceManager();
			dm.RootPath = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData) + 
				@"\Fidelity Investments\WealthLabPro\1.0.0.0\Data\";
			DataSeries Index = Close - Close;
			DataSeries StdDev = Close - Close;
			DataSeries WorkSeries = Close - Close;
			DataSeries MomentumAvg = Close - Close;
			DataSeries x = Close - Close;
			int SymbolCnt = 0;					   
			foreach(DataSource ds in dm.DataSources)
			{
				// if (ds.Name.StartsWith("Indust Office"))  // test with small data set				
				if ((ds.Name.StartsWith ("IndGrp")) ||
				 	(ds.Name.StartsWith ("Indust")) ||
					(ds.Name.StartsWith ("ISector")))
				{
					PrintDebug("Processing DataSet " + ds.Name);
					PrintStatusBar("Processing DataSet " + ds.Name);					
					if (ds.Name.StartsWith("ISector"))
					    Level = "Sector";
					if (ds.Name.StartsWith("IndGrp")) Level = "IndGroup";
					if (ds.Name.StartsWith("Indust")) Level = "Industry";						 					
						foreach(string symbol2 in ds.Symbols)  // calc average
						{ 
						SymbolCnt = SymbolCnt + 1;	
						SetContext( symbol2, true );   
							x = (Bars.Close);
							Index = Index + x;						
						    MomentumAvg = MomentumAvg + MomentumPct.Series(Bars.Close, 250);    
							RestoreContext();
						}
					Index = Index / SymbolCnt;
					MomentumAvg = MomentumAvg / SymbolCnt;
					// ******************************** new loop **************************
					    foreach(string symbol2 in ds.Symbols)  // prepare to calc std dev
					{ 
						SetContext( symbol2, true );
						WorkSeries = MomentumPct.Series(Bars.Close, 250) - MomentumAvg;
						WorkSeries = WorkSeries * WorkSeries;
						StdDev = StdDev + WorkSeries;
						RestoreContext();
					}
					// ******************************* end new loop ************************
					StdDev = StdDev / SymbolCnt;
					for(int bar = 0; bar < Bars.Count; bar++)
						StdDev[bar] = Math.Sqrt(StdDev[bar]);
						// Now write out the index
						Utility u = new Utility( this );			
						PrintDebug( ds.Name );
						string dsn = ds.Name;
					string path = @"C:/DataBus/WLP5 ASCII Files/Index-Lab " + Level + "/" + dsn + " Index.csv";
						StreamWriter csv = new StreamWriter( path);
					for(int bar = 0; bar < Bars.Count; bar++)
						{
						if( Bars.BarInterval == 0 )
								csv.WriteLine( Date[bar].ToShortDateString() + "," + Index[bar] + ", " + MomentumAvg[bar] + ",  " +  StdDev[bar]);
						}
					// zero out
					Index = Index - Index;
					StdDev = Index;
					SymbolCnt = 0;
					MomentumAvg = Index;
					csv.Close();
					csv = null;
					ClearExternalSymbols();
					Bars.Cache.Clear();   
					}
			}
			PrintDebug("All Finished");
			PrintStatusBar("All Finished");
		}
	}
}
[/CODE]

This strategy was working great and then I added the loop inside the new loop comments along with the associated processing for the new dataseries.  The original strategy would work on a large number of datasets.  With the new logic, I get an out of memory error on the 2nd or 3rd dataset.

Can someone see why I am getting this error.

I am also new to C# so if anyone sees programming improvements that I can make to the strategy that would help me learn I would appreciate the help.

Thanks.

Eugene

#2

12/7/2010 4:00 AM

Here's your bug:

CODE:
Please log in to see this code.

Which creates Close-Close
Close-Close*Close-Close
Close-Close*Close-Close*Close-Close*Close-Close
Close-Close*Close-Close*Close-Close*Close-Close*Close-Close*Close-Close
Close-Close*Close-Close*Close-Close*Close-Close*Close-Close*Close-Close*Close-Close*Close-Close
...
Many thousands of times. I guess you get the picture. Here's what I think your code intended to do originally:

CODE:
Please log in to see this code.

traderray

#3

12/7/2010 8:28 AM

Eugene,
I modified the part inside the comments to this:

CODE:
Please log in to see this code.

and I am still getting the out of memory condition. Do you see something else wrong?

I do not understand why "WorkSeries = WorkSeries * WorkSeries;" gets expanded as you show. Can you explain?

Eugene

#4

12/7/2010 9:06 AM

Simple: you're creating WorkSeries * WorkSeries times series. For each symbol in the loop. Try applying the approach to rest of the code with similar series manipulation (addition, subtraction etc.)

traderray

#5

12/7/2010 1:19 PM

Eugene,

I made all of the series calculation changes that you suggested and it still did not work. I then tried adding a statement "DataSeries ZeroSeries = Close - Close". I made each of the statements in the beginning that zeroed a series = to ZeroSeries and then did the same with the statements at the end. Suprisingly, after making the simple changes to zero the series, it ran ok.

Since it did not work until I put in the zero series changes, I went back to the original script above and just made the zero series changes. It did not work. It took all of the changes mentioned above to get it working.

Thanks for your help.

traderray

#6

12/7/2010 2:34 PM

Eugene,

The plot thickens. I believe there is a bug in some of the DataSeries logic in WLP as this strategy illustrates

CODE:
Please log in to see this code.

Look at the value of a in the debug window. I believe there are also other combinations of logic that will give the wrong answer for b and c.

Eugene

#7

12/7/2010 3:57 PM

I would not call that a bug. You declared a series, created two more, and assigned to them a reference to the first series. Even though the value has changed, the data within the data series it used to refer to hasn't. Please consider taking a moment to read some good source illustrating the concept of reference and value types in C#.

What you did was not a "zero series". To properly initialize a DataSeries, create a new zero-filled DataSeries: