Jump to content

Canada's top-tier Telescopes & Accessories
Be as specific as possible when reporting issues and *ALWAYS* include the full version number of the application you are using and your exact *CAMERA MODEL*
NEVER POST YOUR KEY IN ANY PUBLIC FORUM, INCLUDING THE O'TELESCOPE SUPPORT FORUM ::: IF YOU DO YOUR KEY WILL BE DEACTIVATED WITHOUT NOTICE!
  • 0

Error in Planetary mode


aamiet
 Share

Question

Hi there, I'm starting to run into problems when using BackyardEOS 3.1.17.27056 on my Windows 10 computer with my Canon 700D when using planetary mode. I have found that after recording around 30,000 to 40,000 frames BYE appears to have memory problems and I'm losing data. Sometimes this is recoverable from the temp folder, sometimes not. Restarting BYE fixes the problem, but I need to set up my imaging parameters again, not a huge issue, but a bit annoying.

My workflow consists of taking 2500 frames at a time in 4 loops to give me my 10,000 frames. I save the data as jpgs as it takes too long to save to avi. The first 10,000 run smoothly, the 2nd is usually good also, but by the 3rd or 4th time through I am starting to get the error below and collection stops.

Collection was modified after the enumerator was instantiated.

I realise I'm running BYE pretty hard, but my images are coming out well with this  workflow. Is there anything I'm doing wrong here? Any way to flush the memory (or whatever) to fix this issue? I have attached the log file if you need it.

Thanks,

Andrew

logfile-[20190621-19h32m57s651]-[22904]-2019-06-21.txt

Link to comment
Share on other sites

  • Answers 23
  • Created
  • Last Reply

Top Posters For This Question

23 answers to this question

Recommended Posts

  • 0

This is repeating. It happens quite often, I have been reducing the throttle setting recently because I thought I could increase the fps rate by setting a lower value, but it's currently on 50 ms. It would be good if you had more detail in the User Guide about what all the settings do.
 

Thanks, Andrew.

Link to comment
Share on other sites

  • 0

There just seems to be a lot of extra disk activity involved with saving the instantaneous jpgs to the temp folder, then transferring them to the "download" folder with new names while downloading the next set of files for the new loop to the temp folder again. 

Would it not be possible to simply save the jpgs directly to the "download" folder and not worry about the temp folder when saving jpgs only? This would certainly improve throughput and put less work on to the hard drive.

Just a thought.

Andrew

Link to comment
Share on other sites

  • 0

Moving files to a different folder on the same drive is a very low overhead operation. Only the directory entries are changed; the files are not copied. Are your BackyardTEMP folder and your Download folder on the same drive?

Can you watch system memory/BYE memory usage while this is happening? BYE is a 32-bit .NET app and so cannot exceed 2 GB of allocated memory.

Just curious...why are you capturing soooo many frames?

Link to comment
Share on other sites

  • 0

Hi there, moving the files might be low overhead in terms of cpu and memory, but my hard disk runs at 100% during the download and transfer process and might struggle. I can understand why you use the temp folder for the avi creation, but not for jpg only. The temp folder is on the same partition as the download folder.

Why 10,000? From experience, this is what you need for good imaging. People with dedicated planetary cameras use many more. One of my examples is attached.

Andrew

104B3280-B4D4-4C70-8D8F-F688EC5AC06F.png

Link to comment
Share on other sites

  • 0

I just checked - BackgroundWorker Enabled is not checked by default - I have just checked it on now.

I'm taking some dummy data now and watching Task Manager. BYEOS is not using more than about 200 MB while running, and it appears the BackgroundWorker is transferring the files. With the BackgroundWorker on it's taking longer to transfer the files, but it seems to be doing the job well, I've taken 40,000 frames with no problems. Why is the default option to have this off?

The user manual does not discuss most (if any) of the options in the Advanced Settings area, could this be enhanced to include a description of what these options do? For instance, what does Processor Affinity mean?

It would also be great if the Planetary mode had a histogram feature, so I could check that I was collecting images at the correct brightness. It doesn't need to be enabled during capture, but just to check at the beginning would be nice.

It would also be great if you could keep the same date and time for the original temp file (ie 000001.jpg) as for the transferred file (test_Tv115s_200iso_1024x688_20190622-13h00m56s-loop03_000001,jpg). That way I would know for sure what the actual fps was from the date/time for the last frame compared to the first. I would also like to be able to change the file name in planetary mode, you can do it for normal imaging, just not for planetary.

Sounds like I'm complaining a lot, but I do think that BYEOS is a great program, and recommend it when I can to other astronomers. If the BackgroundWorker feature fixes my current issues with downloading, that would go a long way to keep me happy :).

Thanks, Andrew

BYEOS task manager.JPG

Link to comment
Share on other sites

  • 0

It's turned off by default because it there a few side effects using it and these non-intrusive side effect where causing the true beginners / non computer savvy some issues from time to time.  One of these side effect is that the background worker process will only use 1 CPU core and therefore takes a bit more time to do tasks that benefits from multi-threading computing but in your case it seems like the side effects bring you a great disk i/o advantage.  Another non-intrusive side effect is that the background worker will keep running for a few minutes after you close the app to make sure it is done processing it's task buffer.

Regards,

Link to comment
Share on other sites

  • 0

OK, I understand, thanks for this.

It does appear that the files are being copied, not moved, as the File Creation time is different for the raw files kept in the Temp folder to the ones transferred to the download folder. See the images below, I had a few files left over in the temp folder that didn't get removed after being transferred to the download folder. Same "File modified" time, different "File Creation" time. Simply moving them over should keep the same creation time, copying them changes that.

raw file from temp.JPG

transfer file from download folder.JPG

Link to comment
Share on other sites

  • 0

Ok, I looked at code and this is not a straight move, the files are being renamed in the process and a move command does no do rename in the process. so the files are being copied.  Perhaps I should rename the file first in the temp folder and then move the files; I'll look into.

 

Link to comment
Share on other sites

  • 0

Or better yet, not rename the file and do a straight move.  The files would be named "000001.jpg" to "999999.jpg" in the download folder instead of "Planetary_1600iso_1056x704_20190622-14h57m50s_000001.jpg" but that may be just fine because the download folder name where the images are being moved to is already called "Planetary_1600iso_1056x704_20190622-14h57m50s" so renameing the files is more or less a duplicate process, meant only for consistency... but it is by no mean necessary to rename them.

I've already fixed this; it will be in the next release.

Link to comment
Share on other sites

  • 0

Thanks for this, appreciate the quick turnaround on this one.

Two questions - why use the temp folder at all, saving directly to the save folder means no extra work on the hard drive ?

Second, why not save the file with the long name in the first place? Is there a reason why you just make it a number?

Thanks again,

Andrew

Link to comment
Share on other sites

  • 0
26 minutes ago, aamiet said:

why use the temp folder at all, saving directly to the save folder means no extra work on the hard drive ?

The temp folder is a hard requirement and by design.   

TEMP folder MUST be on a local drive = guaranteed availability

DOWNLOAD folder can be anywhere... including and external drive or even a network drive = no guaranteed availability.

BYE needs guaranteed folder to store items at run time.

 

26 minutes ago, aamiet said:

Second, why not save the file with the long name in the first place? Is there a reason why you just make it a number?

 

It's all about speed in planetary recording.  Less code = faster code.

The images are dumped/saved with as little of code execution as possible between the image capture and the file dump.  

This is a fair question though, I guess I would have to run some test and the difference will probably be negligible that it would work too.  However, I have learned over the years that changing any code that is working for another version of code is also apparently working may not work in some unforeseen scenarios and this leads to new bugs :(

Regards,

 

Link to comment
Share on other sites

  • 0
10 minutes ago, admin said:

The temp folder is a hard requirement and by design.   

TEMP folder MUST be on a local drive = guaranteed availability

DOWNLOAD folder can be anywhere... including and external drive or even a network drive = no guaranteed availability.

BYE needs guaranteed folder to store items at run time.

 

 

It's all about speed in planetary recording.  Less code = faster code.

The images are dumped/saved with as little of code execution as possible between the image capture and the file dump.  

This is a fair question though, I guess I would have to run some test and the difference will probably be negligible that it would work too.  However, I have learned over the years that changing any code that is working for another version of code is also apparently working may not work in some unforeseen scenarios and this leads to new bugs :(

Regards,

 

OK, I understand about the temp folder. I also do some coding (in Visual Studio) and I understand why you need a guaranteed folder to store data.  I also understand making changes to a program can cause unintended consequences in other locations. Kudos for getting the Backgroundworker running on a separate thread (I assume), doing that with a 32 bit application is not easy at all (I've tried, and failed).

However, I don't think that saving a file with a long filename should take any longer than saving with a short file name. Most of the filename should be known from the start of the capture (the program already spend 2-3 seconds determining frame rate so there is ample time to work out the file names), camera settings are known prior, it's just the actual start time needs to be added just before capture and that currently doesn't change for the whole capture. Of course it would be nicer if the time incremented in the individual file names during capture, but that takes (a little) more processing to ask the computer for the time. Obviously the counter increases from frame to frame, but this increases anyway. Computers these days are fast, as far as I can see the main bottleneck to the whole process is getting the throughput from the camera above 20 fps :)

Thanks again for your help, this program has made my imaging session much easier and I wouldn't be without it.

Andrew

Link to comment
Share on other sites

  • 0
On 6/23/2019 at 5:01 AM, admin said:

Or better yet, not rename the file and do a straight move.  The files would be named "000001.jpg" to "999999.jpg" in the download folder instead of "Planetary_1600iso_1056x704_20190622-14h57m50s_000001.jpg" but that may be just fine because the download folder name where the images are being moved to is already called "Planetary_1600iso_1056x704_20190622-14h57m50s" so renameing the files is more or less a duplicate process, meant only for consistency... but it is by no mean necessary to rename them.

I've already fixed this; it will be in the next release.

Hi there, is there any timeframe on this next release? I'm keen to get this issue sorted, as I had the same issue again a few nights ago, even with the BackgroundWorker on.

Thanks,

Andrew

Link to comment
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Answer this question...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

 Share


×
×
  • Create New...

Important Information

This site uses cookies to offer your a better browsing experience. You can adjust your cookie settings. By closing this banner, scrolling this page, clicking a link or continuing to browse otherwise, you agree to the use of cookies, our Privacy Policy, and our Terms of Use