3.1.18 Hanging on download when minimized

[dkerber]

I have been having issues with BYE occasionally hanging while trying to download a picture from my T3i. I haven't fully finished isolating when this happens, but right now, it seems to be most frequent when BYE is minimized. It has done so twice tonight; both times it was minimized, and when I brought it back up, it had been stuck for 10 min or so (based on the time stamp of the last frame successfully downloaded).  Right now, I have it running on top with the focus, and I'll see if it hangs.

[astroman133]

I tried to re-create your issue with BYE 3.1.18 RC11 and my T5i. I had no problem downloading images when BYE was minimized. This is not surprising. All that Windows does when you minimize a program is to move its screen location to coordinates that do not display. I believe that it is off the screen by 10,000 pixels in both the x and y directions. 

I would consider a faulty cable or saturated USB hub in your PC as the culprits.

[admin]

A minimized window can not be the cause.

How long has your computer been left unattended?  I suspect something may be going to sleep.  The harddrive, the usb port, the laptop itself.

Can you please elaborate more on what you are doing when it happens?

 

[dkerber]

I know this may be a tough one. It's not entirely reproducible for me, but it has happened often enough that I think there is something to it. As I said, this had happened twice earlier in the night, but on my last attempt, I left BYE maximized on top, and it got through my full imaging run (120 shots of M45) with no issues. The laptop is a quad-core I7 with 8 GB RAM, 512 GB SSD, and is set to never sleep when AC power is connected. USB is kept busy by PHD guiding, and it's an SSD, so no drive sleeping either. I suspect (with little evidence so far) that it may be resource related somehow, but Task Manager indicates that I've got plenty of headroom in CPU usage, memory usage, and HD bandwidth, so I don't know what the issue would be. It might be due to a resource usage spike during an image download or something, where BYE doesn't get quite enough resources unless it has the focus (the laptop performance option is set to give resource preference to the foreground application). It might be a race condition of some kind that only pops up once in a while.

My normal operation is to connect to the laptop remotely via remote desktop over my house's WiFi connection. I have CPWI managing the mount connection, PHD guiding, and use APT to do an autofocus run. Then I disconnect the camera from APT, and connect it in BYE, usually leaving APT running in the background but not doing anything. In BYE, I take a few shots to verify my exposure, then configure the sequence and start it. Then when I had issues, I minimized BYE and may have disconnected the RD session (I don't remember for sure whether I disconnected or just minimized the RD session). This happened twice, after a relatively small number of shots (I haven't gone back and figured out exactly how many). The symptoms are that when I go back to BYE, it's stuck on (I think) "Downloading", and when I look at the progress log window, it had been 10 minutes since the last image was downloaded (they were 20-second images). Clicking on Abort or Suspend did nothing; I had to kill BYE completely. On restarting, it started working normally again, and the last time, I left it maximized with the focus, and it got through 120 shots in 45 min or so with no trouble.

If you can think of anything else to look at, let me know!

[admin]

The next time this occurs send me that log file.

 

[dkerber]

1 hour ago, admin said:

The next time this occurs send me that log file.

 

Log file sent.

[admin]

I got the log file and there is no evidence of any freezing.... in fact there are several log entries per minutes throughout the entire session... the app was responsive. Are you sure you sent a log file when you had the observed behavior?

The only oddity is this entry... and that was after a 10+ minute of inactivity. This is an action initiated by the camera, it fired a disconnect event and BYE responded properly be disconnecting the camera from the app.

020-03-07 21:01:03,759 [Main] INFO  - Camera action fired: 'MaestroDisconnect'
2020-03-07 21:01:03,763 [Main] INFO  - CameraPropertyChangedArgs fired: AeMode               = ''
2020-03-07 21:01:03,763 [Main] INFO  - CameraPropertyChangedArgs fired: Artist               = ''
2020-03-07 21:01:03,763 [Main] INFO  - CameraPropertyChangedArgs fired: Av                   = ''

...

2020-03-07 21:01:03,814 [Main] INFO  - Running (log): EDSDK.EdsCloseSession(CameraRef)
2020-03-07 21:01:04,039 [Main] INFO  - Success: EDSDK.EdsCloseSession(CameraRef)
2020-03-07 21:01:04,039 [Main] INFO  - Running (log): EDSDK.EdsRelease(CameraRef)
2020-03-07 21:01:04,043 [Main] INFO  - WARNING EDS_ERR_UNIMPLEMENTED : EDSDK.EdsRelease(CameraRef) 
2020-03-07 21:01:04,044 [Main] INFO  - CAMERA IS DISCONNECTED!
2020-03-07 21:01:04,044 [Main] INFO  - Camera state changed: 'Disconnected'
2020-03-07 21:01:04,069 [Main] INFO  - Camera SHUTDOWN!
 

According to the log file you have an AC power supply to the camera so it can't be the battery. Do you have the camera to shutdown after 10'ish minutes of inactivity by any chance?

Regards,

 

 

 

 

 

[dkerber]

The camera may be set to shut down after 10 min  (I don't remember the exact setting value), but the problem is that it shouldn't have stopped for 10 minutes at that point; it was only a few frames into what should have been either a 60 or 120 image sequence.  you look at time 20:51:02, it starts repeatedly querying the weather station, and never breaks out until I catch it and it click Abort. The same pattern appears in the earlier hang - it stops shooting pics and appears to get stuck in ReadWeatherCenter.

[admin]

Okay, I see what you mean.. a 10+ minute gap between the last download and the abort button.

It's not stuck on the weather provider, this is unrelated and it runs on a separate thread.

The camera fired a disconnect event, that is a fact. The odd thing is that there is a 10+ minute gap (ignoring the weather provider log entries) between the last download and the camera disconnect event.

I'd be interested to see other log files where you see this behavior and see if they show the same 10+ minute gap.

 

[admin]

One thing you should try is go to Settings -> Advance Settings and set the processor affinity option ON and see if this makes difference.

Keep us posted.

[dkerber]

2 hours ago, admin said:

Okay, I see what you mean.. a 10+ minute gap between the last download and the abort button.

It's not stuck on the weather provider, this is unrelated and it runs on a separate thread.

The camera fired a disconnect event, that is a fact. The odd thing is that there is a 10+ minute gap (ignoring the weather provider log entries) between the last download and the camera disconnect event.

I'd be interested to see other log files where you see this behavior and see if they show the same 10+ minute gap.

 

They do show a similar gap. In this attached log file, go down to time 20:23:50,453, and you'll see pretty much the same symptoms, but it took longer for me to notice it, so it's 14 minutes or so of just ReadWeatherStation calls. This was the previous hang the same night. It looks to me like the image capture thread either died or hung, but at least some of the other threads continued processing. As I said in a previous message, the UI was non-responsive when I tried clicking on various items (Abort, Suspend, Disconnect, etc) on it, but the Close button (X) worked.

I'll try the processor affinity setting to see if that helps.

dkerber_logfile-[20200307-19h11m41s016]-[5536]-2020-03-07.zip

[astroman133]

Keep in mind that there are hundreds of other users who do not have this symptom. That means that it is most likely due to some anomaly in your setup.

[dkerber]

33 minutes ago, astroman133 said:

Keep in mind that there are hundreds of other users who do not have this symptom. That means that it is most likely due to some anomaly in your setup.

Oh, I"m well aware of that, though I can reproduce it consistently enough that I would think someone else could do so too, though it may well depend on some details of the hardware. See my next post for some more controlled testing.

[dkerber]

I did some more controlled testing so see what kind of timing I saw when it locked up on me.

As before, I was connected to the laptop via remote desktop over my home WiFi.

I attached a log file from tonight, and some .png files of the UI while it's locked up. You'll also see come lines that I inserted in the log after closing BYE to indicate when I did various things. I was working on filling out my Darks library, and set up a sequence. Then I minimized it and opened up the log file in Notepad++. Having Notepad++ open on the log file doesn't seem to have affected anything, because I was seeing the exact same thing tonight as I did last night: after shooting several shots, when I go back into BYE, the UI is now not responding to user input, even though it still shows the clock time at the top of the UI, and the progress wheel is continuing to spin. However, the progress of the current shot does not change. On the screen shots,you can see the clock time, which should correspond to what is in the log files.

 

logfile-[20200308-19h47m47s353]-[9508]-2020-03-08-edited.txt

[dkerber]

I guess it won't let me upload the other screen shots I took, but basically they show that the screen doesn't change even after a couple of minutes, when the exposures were at most 30 sec. I can e-mail them if you think they would be useful. I think the log file will have most of what you need. When I ran BYE again but left it maximized, it continued to run fine even when it didn't have the focus. Looks more and more like a resource issue or a race condition  to me. I did have processor affinity set (as you can see in the log file in the previous msg), and it didn't seem to change anything.

 

[admin]

Okay, I looked at your new log file and there is a bit more in the log file and the camera has shut down again.

2020-03-08 20:24:27,162 [OnStateEventHandler] INFO  - StateEvent_WillSoonShutDown Fired! (inParameter=30)
2020-03-08 20:24:27,162 [OnStateEventHandler] INFO  - Running (log): EDSDK.EdsSendCommand(CameraRef, EDSDK.CameraCommand_ExtendShutDownTimer, 0)
2020-03-08 20:24:27,194 [OnStateEventHandler] INFO  - Success: EDSDK.EdsSendCommand(CameraRef, EDSDK.CameraCommand_ExtendShutDownTimer, 0)
2020-03-08 20:24:27,397 [OnStateEventHandler] DEBUG - OnStateEventHandler Fired! (inEvent=772), inParameter=0)
2020-03-08 20:24:30,569 [ReadWeatherCenter(Normal)] DEBUG - No weather provider configured.
2020-03-08 20:24:56,662 [Main] INFO  - Camera action fired: 'MaestroDisconnect'

 

These events are initiated by the camera, I'm 100% sure about this.  At this point I'm almost ready to call it.  I'm 99% + certain this is caused by a power starvation.  I suspect your camera AC adapter is causing this.

Do you have another AC adapter... or a fully charged battery you can try as a test?

Regards.

 

[dkerber]

Umm, 20:24 is 14 minutes after the capture plan stopped responding. 20:10,529 was the last picture successfully captured. From 20:10:20,778 on, the UI did nothing, and I just watched it. You can see in the log where I said I clicked on 'Abort'. Then I watched the UI for another minute or so, and again nothing happened, even though the log acknowledged the Abort. Then I clicked on 'Suspend', and the UI said 'Busy", but again did nothing else. The 'StateEvent_WillSoonShutDown' was after I clicked the "X" in the upper right of BYE to shut it down, 14 minutes after the last successful image.

After sending this log to you, I reran the exact same capture plan with BYE maximized, and it went through all 40 images successfully.

Just to rule things out, I'll try it with a battery in the camera, but I'm pretty certain that's not going to change anything. The camera is powered by the same 60 Amp 12V power supply that runs the laptop, mount, and dew heaters. It is connected through one of the 12V pass-through jacks on the DewMaster.

I noticed that there is another Logging option checkbox in the Advanced settings box; would that give any more detailed tracing?

[dkerber]

I did some more tests tonight, and found the same results, including after replacing the AC adapter with the battery. It still ran fine while I had BYE maximized (25 shots or so), and froze with the exact same symptoms within a few minutes after minimizing BYE. It should have been a 40-shot sequence, and when I let it run with BYE maximized, it finished all 40 shots just fine.

All in all, it still looks like a resource issue of some kind, leading to (maybe) a race condition. It seems like there needs to be some more logging on the details of the imaging thread, and/or the UI thread. I can post the logs if you wish, but they show pretty much the same things as the previous ones I've posted.

Tomorrow I'll give the 3.2 beta a try and see if that acts differently.

[astroman133]

I tried to duplicate your issue with BYE 3.1.18 RC11 and my T5i. Twice I started a capture plan with 40 10-second exposures and minimized BYE while the first exposure was exposing. Both sequences completed without error.

I tend to agree that you system has some resource limitation. I do not believe that what you are seeing is a bug in BYE.

[admin]

Do you have another computer you can try with same camera / USB cable / AC adapter? I'd be curious to see if it would show the same behavior.

 

[s3igell]

Check in the Win10 Control Panel Advanced Power Settings to make sure that the USB Port isn't set to Power-Down under any conditions that may be triggered by minimizing the Program.

[dkerber]

1 hour ago, s3igell said:

Check in the Win10 Control Panel Advanced Power Settings to make sure that the USB Port isn't set to Power-Down under any conditions that may be triggered by minimizing the Program.

The USB port is kept busy by guiding, so it won't shut down just because I minimize BYE.

[dkerber]

9 hours ago, astroman133 said:

I tried to duplicate your issue with BYE 3.1.18 RC11 and my T5i. Twice I started a capture plan with 40 10-second exposures and minimized BYE while the first exposure was exposing. Both sequences completed without error.

I tend to agree that you system has some resource limitation. I do not believe that what you are seeing is a bug in BYE.

Thanks for trying it.

I should be able to test this with the same camera on my desktop computer. It won't have the same USB cable, but should otherwise be equivalent. Since I see the same issue when running on battery, the AC adapter can be ruled out as a contributer. I can also try it with the astro laptop and a different cable, both direct and through the hub to see if those variations make a difference.

[dkerber]

Well, I doubt you expected this result, but 3.2.0 RC2 appears to have fixed it.

I ran different variations with 3.18.1 with different settings, cables, etc, and the result was always the same as before; sometimes more successful shots, sometimes less, but it always eventually hung before completion of the full session after being minimized. Then I tried 3.2.0 RC2, and voila`, it worked with no issues! The full 40-shot imaging session ran to completion twice in a row. Pending further real-world usage (I don't plan to do any more formal "testing"), I'll consider it fixed.

Thanks for your patience!

[admin]

Well, you're right... I was not expecting this. There is nothing in 3.2.0 RC2 that would address the behavior you are experiencing... but if it works, use it.

[astroman133]

Does the RC have a new version of the SDK? Perhaps Canon fixed something.

[admin]

No, but is has a few underlying technology upgrades; namely .Net 4.5, StructureMap, and a new installer guid, forcing Windows to see 3.2 as a different app from 3.1.x.  Nothing in there AFAIK would play any role is solving this behavior

[dkerber]

No refactoring of the background processes, or moving stuff between background and foreground threads? I did notice that there is a lot more logging going on than in the previous version.

My theory is still that when the app was minimized, the system re-allocated some resources in such a way that one process or another took a different amount of time to complete, triggering a latent race condition. Pretty much any change in how the background threads are structured could change this, IME. The .Net version change would be another candidate for causing slightly changed behavior, too...

[MAXIMUSMIKE]

I had the same issue, I had 3 hookups to my usb hub. camera, guide camera, mount. my solution was to run the camera using bye on its own. it did not fix the issue, but it allowed me to continue imaging without the camera freezing up, or bye freezing up. Now what was responsible for the freezing, i'm not sure, I have no background in technical stuff. but I do know that my issue with the freezing happened, and it came down to either the hub or the cable could not handle the traffic. this works for me. I just want it to work, don't want to play the why it won't work game, my time is short enough imaging, with weather and all else that limits my imaging sessions. I let the people who know more about this stuff, and are into solving problem do that, I would screwup more things than I would fix. Just my fix to keep going. CLEAR SKYS ALL

[astroman133]

Interesting...My initial reply to David's post, back on March 7th, indicated a saturated USB hub as a possible cause for a momentary disconnect which could cause BYE to hang. This is also borne out by the fact that the issue occurs when a completed image is being downloaded from the camera to the PC when BYE locks up.

Another thing to try, for testing purposes, is to save the image only to the camera's memory card. Then operate the camera both with BYE restored and minimized.

This is most likely an issue on your Computer since no other users are reporting anything remotely (no pun intended) like it.

One other thing is that Windows 10 has a USB Selective Suspend setting that could be causing your issue, even if you think that you have disabled the power saving features. Following is a link with instructions on how to disable it:

https://www.windowscentral.com/how-prevent-windows-10-turning-usb-devices

[dkerber]

I'm glad to hear I'm not the only one with this issue, though it's obviously not common. I didn't think to try saving the image only to the camera's card, but really don't think USB saturation was the issue. It's a nearly new laptop with USB-3, a Startech USB-3 hub, and good (read: not cheap) cables, kept as short as possible. And this is supported by the fact that the new version of BYE does not have this issue.