Planetary captures sometimes don't finish

[gfunkernator]

I am using Backyard EOS 3.1.18.  When capturing 2500 frames into an AVI it will sample the fps, begin the capture, then when 2500 frames are captured, in that same area where it showed the fps and frames captured, it shows the word "memory" with a number counting down from about 900.  When all works properly, when the counter reaches 0 I see the queue in the camera panel blinking "1".  Then eventually I hear the tone indicating that the capture is complete.  I check to see if the AVI was created and it was.  When it does not work, the memory counter never reaches zero, will stop anywhere from 200-500, then nothing happens.  No tone, no queue indicator, no AVI file.  This started happening in version 3.1.18 for me.  I am using a Canon EOS Ra in "M" mode.  Also no errors were shown in the log.

 

Note that this has happened only when I try to capture more than 2000 frames.  I am using a Surface Pro 6 with Windows 10 8GB RAM.  The RAM usage never goes above 4 or 5GB while capturing AVI's through BYE.

[gfunkernator]

I wanted to confirm that enabling the Background Worker Process did the trick!  I haven't tried capturing over 2000 frames but each 2000 frame capture completed successfully.  Thanks!

[admin]

This is 100% related to memory/disk IO.

1. When recording a planetary session, BYE tries to capture as many frames as possible into memory first... then slowly saves them to disk while given top priority to the capture of frames itself.
2. The number you see when is says memory are the number of frames yet to be saved to disk.
3. When that is done, if you selected AVI, you'll get the job counter "1" blinking... indicating there there is one background job being worked on...that would be creating the AVI.

Your issue occurs in #2, correct?

This is a disk IO intensive process.... and you may be hitting the limit of your system and it's ability to manage memory swap files, I suspect this is where the contention is.  If this is case there is not much that can be done, Windows is fighting for disk IO in all direction.

Are you using the background worker and/or processor affinity?  See advance settings in BYE.

[astroman133]

You may have available RAM, but Windows only allows 32-bit applications to use a total of 2 GB. BYE must be a 32-bit app, because it uses the Canon SDK which is a 32-bit API.

So can you do multiple smaller captures, back-to-back, keep them as JPGs? You may be able to use dcraw, or some other program, to combine into an AVI them outside of BYE.

[gfunkernator]

I don't think it is the page file.  I was watching my RAM usage while capturing, it never exceeded 5GB, total system RAM is 8GB with like 7.8GB available to Windows.  Then again this not a workstation I am using, just a Windows tablet so yeah I may be pushing it too much.  I remember one session where the sampled framerate was 50fps!  Never again though, its been sampling at 30fps.  I don't know how that happened but need to find out.  Time is critical in the morning when the sun is chasing the planets, so getting as many frames as possible in the shortest time is key.

 

I can do JPG captures no problem.  I noticed that they got saved to the BackyardTEMP folder not the standard PLANETARY folder in BackyardEOS, but I moved them before closing BYE fearing they'd get deleted.  Can I get the same quality level from using jpg vs avi?  I would think not unless there's a way to do it without much loss.  

[admin]

9 hours ago, gfunkernator said:

I can do JPG captures no problem.  I noticed that they got saved to the BackyardTEMP folder not the standard PLANETARY folder in BackyardEOS, but I moved them before closing BYE fearing they'd get deleted.  Can I get the same quality level from using jpg vs avi?  I would think not unless there's a way to do it without much loss.  

The JPG gets copied only after the AVI is complete.

AVI and JPG are 100% the same quality.  Think about it for a second... the JPG is what is downloaded on the computer form live view....then the AVI is created from those images... there is no way creating an AVI (a 30 year old technology) would increase the quality of the images being given at it .  They are identical.

Hope this helps.

 

[astroman133]

The JPG images are the individual LiveView frames that BYE gets from the camera. The JPG files are combined, after the fact, to make AVI. This means that the quality is the same.

Initially the downloaded JPG planetary images are saved in the BackyardTEMP folder. Once all the requested frames have been collected in the BackyardTEMP folder, a background thread is used to move them to the Planetary folder, rename them, and potentially combine them into the AVI. You need to let this process happen without intervention. By you worrying that BYE is not doing what it is supposed to and moving the files manually, you are interfering with that process. BYE will warn you if you try to quit the program while you have unprocessed data. You can tell that it is finished when the Queue entry (in the Camera Information Center, to the right of the battery indicator) disappears.

I suggest that you play around with BYE and your tablet, connected to the camera, at the kitchen table, so you are familiar with how to use it and are comfortable with how it behaves.

I have an i7 desktop PC with Windows 10 Pro and a T5i camera. I get about 14 fps in Planetary mode. I created two 3000 frame AVI files, back-to-back (by setting the Loop count to 2), using BYE 3.2.0 RC4. I had no problems. The frames for the first video were being processed while the data for the second video was being collected. BYE's memory usage did not come close to the 2GB limit, staying below 160MB and its CPU usage was always under 10%.  It took about 3.5 minutes to capture the 3000 frames and less than 2 minutes to create the AVI, so the number of queued tasks never got above 1.

If you are trying to capture Jupiter, you need to keep your video short. I have heard that less than 70 seconds is good. If you go longer than that you will start to see blurring in the stacked image due to the rotation of the planet. Of course there is software that can de-rotate the images to let you go longer.

[gfunkernator]

Ok thanks@adminfor clearing that up for me.  I thought that they were less quality because looking at the jpg frames, they were blurrier than some of the frames in the avi file.  A JPG is a compressed image afterall, but I didnt know that BYE takes x jpg frames and compiles them into an uncompressed avi.  

 

@astroman133 Are you using a de-rotater that you mentioned?  All of my planetary images are blurry, even after making sure focus is on point.  I use Jupiter's moons to coarse focus, then fine tune once the shutter speed is up and I can see Jupiter's details.  I read that 60sec and below is good for Jupiter, 45sec preferable.  I have to find the site that had a chart showing the optimal video length for each planet based on period of rotation.  My videos are 1min 14sec long.  The first video will capture at 30fps, then while that is compiling, the second video will capture at 15fps.  Are lower framerates better?  

[astroman133]

WinJUPOS is what is typically used to de-rotate and combine images of Jupiter and Saturn.

The AVI file contains the JPG images which are downloaded from the camera. The AVI is not uncompressed because the JPGs are not uncompressed.

I am not sure why you get such variation in the frame rate. AFAIK, lower frame rates are not better, or worse either. The rate may be lower because the PC is busy processing the first image. In any case you cannot control the frame rate. BYE gets the LiveView frames as quickly as it can.

 

[gfunkernator]

That one time I got 50fps I had the same applications running as when I get 30fps.  I'm even using a USB 3.0 cable from the camera to a powered USB 3.0 Hub which then goes to an active USB 3.0 cable to my Surface which only has one USB 3.0 port.

[admin]

If you got 50fps I can guarantee that this number was inflated due to possible dropped frames.  Most Canon average about 25 FPS +/- 5 so your 30 fps is most likely the best throughput you'll achieve.

[gfunkernator]

Bumping.  Last night I was attempting to do some planetary imaging.  I took two 1min videos, only one rendered.  This never happened in previous versions of BYE.  There was no queue indicator for the second one.  There needs to be more status info for the video being rendered.  I wasn't aware that the first video was rendered until after I heard the notification sound of a complete capture and I actually checked the planetary folder, but the second capture never chimed or showed up.

[astroman133]

Do you have the individual JPG images from the second capture in the BackyardTEMP\Download folder? If not, then perhaps it was pilot error and the capture was never started. The log file would have the info to determine what was going on.

Did you set the Loop value to 2, or click the Record button a second time?

It is also possible that the camera shutdown LiveView to prevent the camera from overheating. I believe that there is a 30 minute limit to a continuous LiveView session. Did the camera shutdown due to a low battery?

As has been previously reported, BYE collects the LiveView frames as individual JPG images in a subfolder of the BackyardTEMP\Download folder. Once all the files have been downloaded they are assembled into an AVI file, if requested, and moved into the PLANETARY folder. If requested, the folder with the JPG image files is also moved into the PLANETARY folder, otherwise once the AVI has been completely processed, the JPG folder is deleted.

I tried to duplicate your issue with BYE 3.2.0 RC6 and my T5i. I created a 1000 frame planetary capture plan using both SaveTo AVI and SaveTo AVI+JPG. With my camera and PC it took 64 seconds to capture the 1000 frames.  I executed the plan multiple times in close succession and also with the Loop value being set to 2. The correct files were always created in the correct folder. It did take about a minute for the AVI file and the JPG folder to appear in the PLANETARY folder after BYE was done downloading the last frame.

Can you try to re-create the issue? The target is unimportant so you can operate the PC and camera at the kitchen table.

[gfunkernator]

I did hit record a second time.  My EOS Ra is on continuous power and the live view never shuts down.  I had only Avi set, not the Avi+jpg.  What version of windows does your PC have?  What build number?  I have windows 10 1909.  I didn't check the logs.  I'll test again tonight.  Are there previous versions of BYE available to download?

[admin]

Goto advance settings and enable the backgroundworker process.  This is generally not needed but it may help the AVI encoding.

[astroman133]

I am running Windows 10 x64 Version 1909 (OS Build 18363.900).

AFAIK, LiveView shutdown is not to save power, so it should not matter whether is on battery or A/C power.

I googled "Canon LiveView shutdown 30 minutes" and came up with lots of support for the 30 minute LiveView shutdown limit. One of the most interesting was that allowing LiveView for more than 30 minutes could put the camera into a higher tariff category in some countries.

All the previous versions of BYE are available on the Downloads page.

If you want to provide a log file, please do not post the file to this forum. Only provide a single file that has information about a session that has a problem. Keep the file as small as possible.

Follow these instructions and email it to the admins:

Thanks!

[admin]

This limit is different for different models.... and it is to protect your camera electronics. Most models will shutdown between 20 and 30 minutes.  This is not a BYE issue, it's a camera protection feature.  

[gfunkernator]

Ok I'll try to get the logs from that session.  If the live view auto shuts down, then I would know right?  Like the image would freeze or go blank?  I'm considering building a PC just for my yard since that's where I do most of my imaging.  Once in a while I'll travel to a dark site and that's where I'd have to use my surface or I could always bring my gaming laptop, but only to the observatory that's 1.5hrs away, they AC power outlets.  

[gfunkernator]

I just checked the BackyardTEMP/Download folder and found three subfolders.  One for Jupiter and the other two were for the moon.  The Jupiter folder has 2000 jpg's.  The first moon folder has 2000 jpg's and the second has 869 jpg's.  Does this mean the avi capture process didnt finish?  I never use avi+jpg, always select just AVI. 

[admin]

10 minutes ago, gfunkernator said:

I just checked the BackyardTEMP/Download folder and found three subfolders.  One for Jupiter and the other two were for the moon.  The Jupiter folder has 2000 jpg's.  The first moon folder has 2000 jpg's and the second has 869 jpg's.  Does this mean the avi capture process didnt finish?  I never use avi+jpg, always select just AVI. 

There is no such thing as an AVI capture.  Liveview frames are always jpg.  AVI is a post process when all frames are downloaded.

In your case you are missing frames in your last capture, which probably means the camera stop providing images... this does look like live view was terminated by the camera as a safety measure to save the electronics.

Regards,

[gfunkernator]

39 minutes ago, admin said:

There is no such thing as an AVI capture.  Liveview frames are always jpg.  AVI is a post process when all frames are downloaded.

In your case you are missing frames in your last capture, which probably means the camera stop providing images... this does look like live view was terminated by the camera as a safety measure to save the electronics.

Regards,

If live view was terminated, then how was it still active?  I should have seen the feed freeze or go blank correct?  

[admin]

Live view is still active... the mirror is still up... but the internal camera electronics have shut down.  You won't get a blank image....it's just that the camera won't provide live view frames anymore.  Not sure about a freeze.

[astroman133]

It all depends on if the SDK signals back to BYE that LiveView has shutdown. If not, BYE will not know. Although it may be possible to make it  smart enough that if it is expecting the camera to provide several frames a second, but the camera has stopped making frames available, that BYE could time out,  abort the capture, and process what it has received. That is speaking as a programmer, but not a BYE programmer. That is up to Guylain.

gfunkernator, you could create a post in the Feature Suggestion Box Forum to suggest this behavior, if it makes sense to you.

 

[admin]

At this point all camera electronics have shutdown.  The only way to recover is to power cycle the camera once it has cool down.  It's been a while since I tested this but I do not think there is a way to recover from this from software.  

[gfunkernator]

Definitely never happened to me then if all the electronics shut down because the live view has never stopped.  The EOS Ra is mirror less btw.  

[admin]

22 hours ago, admin said:

Goto advance settings and enable the backgroundworker process.  This is generally not needed but it may help the AVI encoding.

Did you try the backgroundworker as suggested earlier?

[astroman133]

There is an easy way for you to test this. Just create a planetary capture plan that will take say 35 minutes, and when it does not complete, look at the time difference between the first JPG and the last JPG.

[gfunkernator]

2 hours ago, admin said:

Did you try the backgroundworker as suggested earlier?

Not yet.  Should I try capturing the 2000 frames as with the same shutter and iso speed as if I'm trying to capture Jupiter?  

[admin]

19 minutes ago, gfunkernator said:

Not yet.  Should I try capturing the 2000 frames as with the same shutter and iso speed as if I'm trying to capture Jupiter?  

Don't change your workflow, do what you would do normally.

[astroman133]

I duplicated your issue...almost.

I am using BYE 3.2.0 RC 6 with a Canon T5i. I went into Planetary mode at 10:49:42 and started capture at 10:51:20. Capture proceeded as expected for 23 full captures of 1000 images and the last capture of 644 images. LiveView terminated with the final image being downloaded at 11:58:18. All times are US MST.

The elapsed time was 29 minutes and 16 seconds, from the time that I went into Planetary mode to the time the last JPG was downloaded.

LiveView did NOT continue, however, it is likely that the last image continued to be displayed. 

After shutting down BYE and disconnecting the camera from the PC the camera was non-responsive until I cycled power on it.

[gfunkernator]

21 minutes ago, astroman133 said:

I duplicated your issue...almost.

I am using BYE 3.2.0 RC 6 with a Canon T5i. I went into Planetary mode at 10:49:42 and started capture at 10:51:20. Capture proceeded as expected for 23 full captures of 1000 images and the last capture of 644 images. LiveView terminated with the final image being downloaded at 11:58:18. All times are US MST.

The elapsed time was 29 minutes and 16 seconds, from the time that I went into Planetary mode to the time the last JPG was downloaded.

LiveView did NOT continue, however, it is likely that the last image continued to be displayed. 

After shutting down BYE and disconnecting the camera from the PC the camera was non-responsive until I cycled power on it.

Right so the last frame from live view was displayed before it shut down.  That's what I meant earlier when I said "freeze".  The thing is I never have live view on that long.