Hi

I notice that if I run with this code, my scan is a lot slower than without it... I probably just generated this code from rule or copied from this forum some years ago. What's the fastest code to just get next earnings date ?

CODE:
Please log in to see this code.

When EarningsDate.GetNext... can not find the next earnings date in its cache file, it may make up to 3 requests to external website(s). Off the top of my head, this is the most likely reason. Usually it's a delisted symbol or a symbol without a known earnings date.

Update C.Components to the latest version (I committed a related change lately) and run a PrintDebug() on your DataSet to discover the problematic symbol.

Eugene

what is Update C.Components to the latest version ? spell it out a bit more for me pls, that's the WL component ? which one oh community component ?

Oh it's the slang for Community Components library.

In the Extension Manager, switch to "Other" (from "Fidelity supported"), then "Addins" and make sure that it's at version 2019.11 or higher.

My fault again.. It was this code making it slow... Is there anything way to speed this up ? I copied and pasted from someone in this forum

CODE:
Please log in to see this code.

QUOTE:
EarningsDate.GetNextUseCache() [gets slow when it] cannot find the next earnings date..., it may make up to 3 [different] requests to external websites. ... [this happens when a symbol is] without a known earnings date.

In this case, it would be nice if its earnings date cache assigned a "null" value for that quarter (or month) so it's not checking for an unknown earnings date for the rest of the quarter (or month). That would speed things up when the earnings date is missing.

Other than that, EarningsDate.GetNextUseCache() works well.

---
For the lines below from http://www2.wealth-lab.com/WL5Wiki/MorningstarDataExtractor.ashx:

CODE:
Please log in to see this code.

The string assignment in the last line would not be the slow step; therefore, the slow step has to be the first line. My "guess" is that the Morningstar data is not locally cached (like the EarningDate data is); therefore, I would only call it in an open Chart window and at no other time. And frankly, I do the same thing with the EarningDate class as well. See the example below:

CODE:
Please log in to see this code.

If you're working with US stocks, I would use the GICS directory queries instead of the Morningstar lookup to get the sector and industry of a stock. It's built into WLPro, so it's much faster. See the WL WealthScript Programming Guide >> Fundamental Analysis >> GICS Data. For non-US stocks, you'll have to used the slower Morningstar resource.

All valid points. Thanks Mark.

QUOTE:
My "guess" is that the Morningstar data is not locally cached (like the EarningDate data is);

You hit the nail on the head. MorningstarSymbolDataExtractor does not support caching to file. However, caching its requests in memory can be achieved without much effort if you follow this design pattern:

CODE:
Please log in to see this code.

For the rest of Wealth-Lab session, subsequent requests will use memory i.e. be instant. Hope topic starter finds it useful. I've updated the Wiki sample code also.

I like the way the Post# 7 solution employs the global cache to save the Morningstar data. Now is there an "easy way" to import this cache from disk with the strategy constructor [e.g. MyStrategy() block] and export this cache data to disk with the strategy destructor [e.g. ~MyStrategy() block]?

Perhaps doing the import/export with an existing WL startup constructor/destructor class would be better than doing it in a strategy. Put what needs to be nonvolatile in a special cache and let WL manage the nonvolatile cache internally. I make this suggestion because I'm concerned WL could hang on startup if the end user handles this wrong.

I'm also wondering if Morningstar will quit supporting this data access if everyone hits their servers each time WL starts up rather than using local disk data like GICS queries do. (Another reason to use GICS queries if you need this sector/industry data for US stocks.)

yeah but I only request this data once a day.. oh let me write it to a file and load it locally back to the hashtable... yes yes

QUOTE:
I'm also wondering if Morningstar will quit supporting this data access if everyone hits their servers each time WL starts up rather than using local disk data like GICS queries do.

I doubt these calls create a server load too high to notice.

QUOTE:
(Another reason to use GICS queries if you need this sector/industry data for US stocks.)

GICS is not available in Wealth-Lab Developer. This was the rationale for development of this class.

for this GICS queries I tried yahoo before (is that even the same thing i think it is ??) but lots of symbols are not included... I am fetching it and writing to my local disk now.... I figured that I just have to update it once every 3 months or so and it will run so much faster

The best way of doing the lookup is through a .NET Dictionary datatype. What you want is a Dictionary<TKey,TArray> generic type, but the .NET framework doesn't have one--bummer. It does have a Dictionary<Tkey,object> generic type, but that implementation is too long and complicated to code. The last resort is their Dictionary<TKey,TValue> generic type, and that you can make work easily if you let TValue be a concatenated string containing the sector, industry, and description. You can then use string.split() to convert that concatenated TValue string into a string[] array. TKey would be the stock symbol string itself, which is indexed for fast lookup.

So define your cache as:

CODE:
Please log in to see this code.

And you can get some examples using it here: https://docs.microsoft.com/en-us/dotnet/api/system.collections.generic.dictionary-2?view=netframework-4.8

You might ask the Stack Overflow programmers if someone has implemented a Dictionary<string,string[]> datatype, where object=string[]. That would be perfect. If they have, I would be interested in that code myself. But I don't see it "natively" in the Microsoft .NET framework.

For easy serialization of delimited records like MorningstarSymbolData I'd recommend the open source FileHelpers library. I'm happy about its application in my private projects. Here goes Quick Start guide. It'd take little time to integrate it into a scan but IMHO the only challenge may be to choose suitable delimiter character to separate entries (obviously it cannot be a character frequently found in GICS descriptions like ",")

Re: Dictionary lookup, I'll stop here saying that these days lookup can be performed elegantly on Tuple<> or custom data types like MorningstarSymbolData using LINQ queries.

I did this to write to a file on my PC and figured that I will update this once every qtr

CODE:
Please log in to see this code.

Then I load it back to a hashtable

CODE:
Please log in to see this code.

Then fetch it when I am in the loop

CODE:
Please log in to see this code.

Now speed is back to normal.

Thanks for the link to FileHelpers in Post# 13. Gee, it even supports asynchronous I/O. There's also a "multi-record" engine for writing records with different schemas (field organizations) all into the same linear stream--interesting feature. It supports Excel *.csv files, but that's the only Excel format supported. I employ Excel Reader for reading Excel binaries (*.xls) created by the Fidelity website stock/ETF screener, which works fast.

FileHelpers would write a more compact "formatted" file than what the XML serializer in the .NET framework would produce. The XML serializer would still be preferred for writing a deeply nested C# object, though, such as the Gics.xml file in WL.

Glad you like it but let's avoid offtopic conversations here please.

I don't have Gics.xml file in WL cau I have fidelity's version or it is there i just don't know where to find it?

No you're not alone with this problem for which there is no workaround right now:

GICS Structure not updated