Hi!

This is my first experiment with parallel series computing. And due to that i have some problems.
Unfortunately all maetrials on C# concern mostly on windows forms interface.
Could you take a look and may be give and advice.
Thanks in advance.

If it will work i hope to improve many times series computing.
My idea is simple if i ll start 8-12 parallel threads in my 8-10 working strategies (4 of them are multisymbols strategies, 2-4 symbols) - theoretically cpu ll be better loaded. And i hope it ll give advantage in real time and in optimization. Even if computing time wont be less than 20-30% - it wil be great improvement.

Sometimes wealthlab is computing them, sometimes gives me error - there s no objects or smthg like this.

Here s class:

CODE:
Please log in to see this code.

Creating object and starting thread:

CODE:
Please log in to see this code.

Empty cycle till computing s finished:

CODE:
Please log in to see this code.

finishing thread:

CODE:
Please log in to see this code.

---

Let me suggest you wait until version 6.4 is released (shortly). There, in compiled Strategy libraries (support for C# 3.0/4.0 features in the built-in Editor may only be coming in 6.5) it will be possible to use asynchronous Tasks that are easier to manage than threads in ThreadPool.

i ve found little error in empty cycle - must be || not &&...
Now i see 100% load of 8 cores )), but 15 minutes of not started WealthLab. I ve shut it down;

I use external dll compiled in 3.5 FrameWork environement in Visual Studio 12.
Seems other methods worked before.
Do You think there s no any variant?

I'm not saying it's impossible, I just don't share your desire to fully load the CPU (expressed in every other post), and, to do it using suboptimal code patterns (when .NET4 Tasks are underway).

If you want it right now, I suggest you employ this design pattern:

C# Tutorial - Using The ThreadPool
How to: Use a Thread Pool

It's quite intuitive and easy enough.

Thanks, i just have read MSDN article concerning multitasks in 4.0 - you are right, i think better use 4 wheel drive environment.
But if you dont load cpu you wont get best results.
I ve changed my mind after many optimization cycles. If it lasts 15-20 minutes that's fine, but unfortunately i had to wait many of them more than 10 hours of GA before see valuable results. And cores were unloaded.
And 2nd reason - my intraday strategies execute in 800-1500 ms
I cant reduce Series range - they need 8-10 days, smallest range - 4 days.
I ve done many experiments and trics, Cache.Clear of unused intermediate series, excluded .Value calls and replaced by direct calls of time window in cycles etc etc, if strategy doest use multitimeframe data - additional series are not loaded and not computed etc. Each step gave me additional part of memory and less computing time by 1-3 %.
Now i think i ve done all i could with your advices. And only one think can be improved - parallel threading.

Thank you very much for this example - others i ve read in manuals and books were not good enough.

QUOTE:
I ve done many experiments and trics,

In .NET4, you may be able to parallelize your long-running calculation tasks with new platform/language features like Parallel.For, PLINQ etc. Chances are it comes to a drop-in replacement for existing code.

However, consider optimizing your algorithms themselves in the first place - and only resort to "tricks" after making sure that the algorithm implementation is efficient.

So i implented this threads pool but but ve got another surprise.
Here is extraction from my volatility class. This method i put in thread pool later.

CODE:
Please log in to see this code.

This method is called in my main working method

CODE:
Please log in to see this code.

After some hours of collective debugging - seems there are problems with cache when strategy calls Cache.Clear and problems with SetScaleCompressed and RestoreScale in thread. I mean - other methods in other threads have some problems in access and computing their series. Couldnt preview such surprise.
Is there only one way - to lock this part of code during compressed bars computing series?
As for cache - same way?
In this case async method seems is not needed.

There is no guarantee that every WealthScript method or an internally used method is thread-safe.

Seems i realised these threads but wealthlab crushes often when i m trying to run optimizations, when i load more than 6 charts etc with strange errors.

When i load only one chart - everything is fine and but speed seems same as without parallel threads - i trid in single symbol mode, pairs.

There is an idea - equity calculation takes big time.
Can i switch if off from code inside strategy? in real time i dont need it at all. All trades etc my strategy computes itself if needed of opt f etc.

You can't switch something internal off without risking to ruin something. However, the Equity tab can be disabled (Preferences > Visualizers).

Alexey, you're doing these 'hackish' experiments on your own. I can only wish you good luck on your venture.

i cant do anything more
seems still have to wait new wlb edition.

Yes, you can. Optimize your algorithms. Believe me: what we do ourselves is almost always the bottleneck. ;)

Knut if i remember right was writing - every working code can be always improved.
Already have an achievement - due to thread pool code requirements now i have only one time calling of SetScaleCompressed and RestoreScale. Seems before they were called 2 times for each symbol.
My reoptimization of 8 pair strategies 7 parameters on 50 days 1 minute timeframe took only 3 hour 10 minute!! Previuos version could take 5-10 hours. 2 parameters strategies and one parameter is timeframe - 12 minutes of 7 parallel strategies.
Real improvement.

Can you tell me - KAMA, SetscaleCompressed etc have optimal code?

So i started these 14 parallel optimizations - seems parallel threading is more effective
Optimization is really going quickly.
But had many problems in loading - 5 crashes of WealthLab

I was wrong - i was talking about optimized strategies without thread pool.

But i really started strategies with thread pool. But effect is funny. 6 showed result.
8 - black chart screen, i press escape - it stops. I press run - strategy seems waiting something.
Does WealthLab limit quantity of threads?

At the risk of repeating (10/16/2012 2:26 AM), no promise is being made that every internal method is thread-safe.

Now it works. It was mistake of threads counters that were declared as static and became common to all strategies. That explains why only one was working fine.

I can load 10, 20 and more strategies and they work good - results are same that in sync mode.
But there are problems in optimization - i even had 3 parallel optimizations. But now no genetic at all, and only - exhausted or montecarlo.
If wealthlab shuts down due to error - i restart it with empty default workspace, reload workspace with needed strategies- everything s fine with strategies or it may shutdown. In this case i recompile my dll with sync methods, restart wealthlab, reload workspace etc. In this case 100% everything is fine. IF restart wealthlab and load my strategies with dll compiled as async calculations - wealthlab executes strategies without any problem and maybe even little be faster as desired. Bit if i try to run optimization - only one strategy can be optimized. Or for example if i start wealthlab with empty or deleted default workspace, load 2 strategies - 2 parallel exhaustive optimizattions may run.
-
If everything fine with execution means optimizations have to work too. What can be wrong?
There s a bug i have to report throught ticket, or you could give me and advice?
Or let say 6.4 will be so different and may have own requirements and better ro relax and wait?

Seems better not to use threading in dataseries computing.
I found improvement only in parallel synchronizations of dataseries.
Technical article on hyper threading s telling that better not to use threading for float point computing.
) But I would prefere u had told me that.

By the way, you can utilize C# 4.0 features like Tasks in the Strategy Editor right now if you follow this simple workaround: I'm using Wealth-Lab 6.4 but can't use C# 3.0/4.0 language features such as LINQ, Tasks etc. What's wrong?