WLP crashes periodically during morning hours
Author: maximgl
Creation Date: 8/25/2019 4:35 PM
profile picture

maximgl

#1
I have a VM on which I keep WLP running all the time. It is running automatic FidelityStatic provider updates and generates daily alerts in the evening (some of which I place through "Place Order" option) however, it is periodically becoming irresponsive/crashes during morning hours when no activity takes place. I do not have streaming enabled and not doing any trade operations during the day. WLP.log file does not show anything close to the time of the crash (or maybe log is not flushed fast enough). The last message that I see in the log is about 15-20 minutes before the crash:

2019-08-23 06:32:37,511 [Runner1] ERROR Fidelity.Components.RequestManager.FidelityWebService [(null)] - Exception parsing XML Response
System.Exception: Response XML does not contain any account information.
at Fidelity.Components.RequestManager.Balances.ParseResponse(XmlDocument xml)

I only keep Orders and Strategy Monitor windows on my default workspace, no charts or account windows. I have a separate autoIt script to automatically type in credentials in login window.

Streaming is disabled. I've started to use "knas Restarter" app that is monitoring WLP is active and restarts it automatically when it is crashed, so I have a workaround for this problem, but I would like to understand why it is happening 2-3 times a week.

Thanks
profile picture

Eugene

#2
Thank you for the report. It's a known issue, and the developer is working on getting it fixed in 6.9.21:

WLP 6.9.20.7 quits when running Strategy Monitor
profile picture

maximgl

#3
Eugene, thank you for confirming. I read through the post you linked and I have 2 updates:
1. I've experienced these issues even before 6.9.20 - I saw them at 6.9.19 as well.
2. I've checked the Event Log and I see following 2 errors there happening around crash times:

CODE:
Please log in to see this code.
profile picture

Eugene

#4
Thank you Maxim, I've forwarded this to the developers.
profile picture

Cone

#5
We're hearing that there's a good chance this was related to xml parsing errors that will be fixed in the upcoming build. We'll be testing it this week, so my fingers are crossed!
profile picture

superticker

#6
I've too experienced two WL 6.9.20.7 ABENDs since I installed it. That's about one ABEND every two weeks. I did not have this problem in WL 6.9.19. I was streaming at the time, but I wasn't doing anything in particular during the crash. And I don't do real-time trading. The crash maybe because of some background function. Today, my crash occurred around 8:50am CDT, so you can ignore all the other ERROR events that occurred way before 8:50am in the logger annotation below. The US market opens at 8:30am CDT here.

CODE:
Please log in to see this code.

Update: Here's the Windows log
CODE:
Please log in to see this code.
profile picture

Eugene

#7
From what I heard, the "(500) Internal Server Error" (with what it causes) that is all over your log should be fixed in 6.9.21.x.
profile picture

superticker

#8
QUOTE:
the "(500) Internal Server Error" (with what it causes) that is all over your log should be fixed in 6.9.21.x.
I didn't think that was a problem with the WL client itself, but if you're "fixing it" in the WL client, then I must be wrong.

Thanks for prioritizing fixing the bugs over new features.
profile picture

superticker

#9
Perhaps this is obvious but the ABEND reported in Post# 3
CODE:
Please log in to see this code.

and the ABEND reported in Post# 6
CODE:
Please log in to see this code.

are totally different and unrelated. Since neither problem can be reproduced reliability, debugging has to be done with a trace dump to the error logger. *If* you don't know what's causing both errors today, you might consider writing an error handler for both error conditions and have that handler perform a trace dump to the error logger in WL 6.9.21.X now. Then you might have enough information to catch both problems for 6.9.22.X.

The error in Post# 6 only started happening in WL 6.9.20.7. That one is new. I have never experienced the one in Post# 3.
profile picture

Cone

#10
We're getting there. We still don't know what causes that 500 server error, but it's handled in build 21. So if this occurs when placing a trade, the order will now Error out instead of stuck "Submitted". We haven't observed that error with a normal use case, but we've forced the error and the solution worked.

Likewise, we think that other errors while parsing xml response were causing crashes and these will be handled too. In these cases, you may see see a dialog pop up with an error - but no crash.
profile picture

superticker

#11
QUOTE:
We still don't know what causes that 500 server error, but it's handled in build 21. So if this occurs when placing a trade, the order will now Error out instead of stuck "Submitted".
This is great news. If I start getting dialog boxes saying to put the error logger into WARNing mode, I'm happy to comply.
profile picture

superticker

#12
There's another annoying ABEND that occurs in Wealth-Lab 6.9.X.X, and this one started right after we switched from WL 6.8.X.X to 6.9.X.X. It's very easy to reproduce. Just don't refresh the WL login password when it expires. The crash stack trace is below.

Hmmm. This crash stack trace looks identical to the stack trace in Post# 3. Well,... now we know how to reproduce the crash in Post# 3. Apparently that WL user didn't re-enter his WL password fast enough after it expired. Solved that mystery. The ABEND in Post# 6 is more important to fix because it occurs in the middle of the trading day. The password not-refreshed ABEND (Post# 3 & 12) only occurs when the US markets are closed.

CODE:
Please log in to see this code.
profile picture

Eugene

#13
Thanks for reporting. Yes, the crash signature is identical to the one in post #3 by @maximgl.
profile picture

superticker

#14
The crashing in Post# 3 & 12 should be easy to fix since you already know the problem is the failure of WL to forcibly logoff the Fidelity user if he doesn't re-enter his expiring password in time before WL tries to access something that's password protected. WL 6.8.X.X did this right and dropped secure requests that required an "active" Fidelity password. The developer simply needs to resurrect that 6.8.X.X code behavior.

The crashing in Post# 6 (which started in WL 6.9.20.X) is more complicated to find. But the stack dump provides the clue that problem is in the WealthLabPro.ChartForm.s() routine. Since that routine doesn't have any parameters, we "might" be able to assume one of its field (I call them "state") variables remains NULL without being properly initialized. But that means the problem may have occurred several minutes earlier. The actual problem is with the process that's responsible for setting those field variables in WealthLabPro.ChartForm.s() so it is ready for execution. --Happy debugging
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).