bug reports

[Alexey Popov]

I have a few problems. Currently I use last version (3.1.9), but problems have appeared early. 

1. If I click on the thumbnail bar to look on a different pictures too often, after some time (minutes, tens on minutes) Backyard just stop show the pictures. Solution - close application and open it. This is an old problem.  Sometime it start to work after shooting, sometimes not. 

2. Very inconvenient bug. It appear about half of year ago. Time to time Backyard  does not catch last preview picture, recently downloaded from camera. It simply has not appeared in thumbnail bar. At all. But I could find a file on a disk. Currently I "lost" 1 picture from 4-5. Sometime a sequence. Solution - just make a new preview. And again. And again.  Very inconvenient when you should do something fast.  

3. I have no idea how, but when Backyard started, I have some problems with wireless communication (I use laptop): website opens slowly, ping very unstable. As soon as I close Backyard - I have fast internet and no problems with ping and etc. Can`t understand how it possible... 

Backyard 3.1.9, Windows 8.1 Pro 64

 

 

[astroman133]

I wonder if the issues that you are reporting are symptoms of an "out of memory" situation. In Windows, 32-bit applications have a maximum address space of 2 GB. It could be that managing all those images is causing the app to run out of memory...But that is just a guess, based on what I saw above.

[Alexey Popov]

A lot of errors, and not out of memory only:

2017-04-25 23:40:47,465 [Main] INFO  - Loading image file : C:\Users\Alexey\Pictures\BackyardEOS\2017-04-24\PREVIEW_20170424-23h54m36s400ms.JPG
2017-04-25 23:40:47,864 [Main] INFO  - GdPicture PREVIEW_20170424-23h54m36s400ms.JPG
2017-04-25 23:40:47,864 [Main] ERROR - Parameter is not valid.
2017-04-25 23:40:47,864 [Main] ERROR -    at System.Drawing.Bitmap..ctor(Int32 width, Int32 height, PixelFormat format)
   at System.Drawing.Bitmap..ctor(Image original, Int32 width, Int32 height)
   at System.Drawing.Bitmap..ctor(Image original)
   at BinaryRivers.Common.ImageLoader.GdJpgImageProvider.LoadImage(FileInfo file)
2017-04-25 23:40:48,421 [Main] DEBUG - Loading JPG image try 1/3 in progress...
2017-04-25 23:40:48,421 [Main] ERROR - JpgProvider.LoadJpgImageFile()
2017-04-25 23:40:48,421 [Main] ERROR - Out of memory.
2017-04-25 23:40:48,421 [Main] ERROR -    at System.Drawing.Image.FromStream(Stream stream, Boolean useEmbeddedColorManagement, Boolean validateImageData)
   at BinaryRivers.Common.ImageLoader.JpgProvider.LoadJpgImageFile(FileInfo fileinfo, Boolean thumbnail)
2017-04-25 23:40:48,427 [Main] ERROR - CaptureMode.<ThumbnailClickedOnMessageRecieved>b__0
2017-04-25 23:40:48,427 [Main] ERROR - Unable to load image file error
2017-04-25 23:40:48,427 [Main] ERROR -    at BinaryRivers.Common.ImageLoader.ImageLoader.Load(String fullpath, Boolean limitsize)
   at BinaryRivers.BackyardEOS.Modes.CaptureMode.<>c__DisplayClass30_0.<ThumbnailClickedOnMessageRecieved>b__0()
2017-04-25 23:40:48,427 [Main] ERROR - PREVIEW_20170424-23h54m36s400ms.JPG
2017-04-25 23:40:48,427 [Main] ERROR - 
2017-04-25 23:40:50,715 [Main] INFO  - Loading image file : C:\Users\Alexey\Pictures\BackyardEOS\2017-04-24\PREVIEW_20170425-00h05m35s807ms.JPG
2017-04-25 23:40:51,110 [Main] INFO  - GdPicture PREVIEW_20170425-00h05m35s807ms.JPG
2017-04-25 23:40:51,110 [Main] ERROR - Parameter is not valid.
2017-04-25 23:40:51,110 [Main] ERROR -    at System.Drawing.Bitmap..ctor(Int32 width, Int32 height, PixelFormat format)
   at System.Drawing.Bitmap..ctor(Image original, Int32 width, Int32 height)
   at System.Drawing.Bitmap..ctor(Image original)
   at BinaryRivers.Common.ImageLoader.GdJpgImageProvider.LoadImage(FileInfo file)
2017-04-25 23:40:51,666 [Main] DEBUG - Loading JPG image try 1/3 in progress...
2017-04-25 23:40:51,666 [Main] ERROR - JpgProvider.LoadJpgImageFile()
2017-04-25 23:40:51,666 [Main] ERROR - Out of memory.
2017-04-25 23:40:51,666 [Main] ERROR -    at System.Drawing.Image.FromStream(Stream stream, Boolean useEmbeddedColorManagement, Boolean validateImageData)
   at BinaryRivers.Common.ImageLoader.JpgProvider.LoadJpgImageFile(FileInfo fileinfo, Boolean thumbnail)
2017-04-25 23:40:51,671 [Main] ERROR - CaptureMode.<ThumbnailClickedOnMessageRecieved>b__0
2017-04-25 23:40:51,671 [Main] ERROR - Unable to load image file error
2017-04-25 23:40:51,671 [Main] ERROR -    at BinaryRivers.Common.ImageLoader.ImageLoader.Load(String fullpath, Boolean limitsize)
   at BinaryRivers.BackyardEOS.Modes.CaptureMode.<>c__DisplayClass30_0.<ThumbnailClickedOnMessageRecieved>b__0()
2017-04-25 23:40:51,671 [Main] ERROR - PREVIEW_20170425-00h05m35s807ms.JPG
2017-04-25 23:40:51,671 [Main] ERROR - 
2017-04-25 23:40:52,916 [Main] INFO  - Loading image file : C:\Users\Alexey\Pictures\BackyardEOS\2017-04-24\PREVIEW_20170425-00h00m19s868ms.JPG
2017-04-25 23:40:53,319 [Main] INFO  - GdPicture PREVIEW_20170425-00h00m19s868ms.JPG
2017-04-25 23:40:53,357 [Main] ERROR - Out of memory.
2017-04-25 23:40:53,357 [Main] ERROR -    at System.Drawing.Graphics.CheckErrorStatus(Int32 status)
   at System.Drawing.Graphics.DrawImage(Image image, Int32 x, Int32 y, Int32 width, Int32 height)
   at System.Drawing.Bitmap..ctor(Image original, Int32 width, Int32 height)
   at System.Drawing.Bitmap..ctor(Image original)
   at BinaryRivers.Common.ImageLoader.GdJpgImageProvider.LoadImage(FileInfo file)
2017-04-25 23:40:53,914 [Main] DEBUG - Loading JPG image try 1/3 in progress...
2017-04-25 23:40:53,914 [Main] ERROR - JpgProvider.LoadJpgImageFile()
2017-04-25 23:40:53,914 [Main] ERROR - Out of memory.
2017-04-25 23:40:53,914 [Main] ERROR -    at System.Drawing.Image.FromStream(Stream stream, Boolean useEmbeddedColorManagement, Boolean validateImageData)
   at BinaryRivers.Common.ImageLoader.JpgProvider.LoadJpgImageFile(FileInfo fileinfo, Boolean thumbnail)
2017-04-25 23:40:53,921 [Main] ERROR - CaptureMode.<ThumbnailClickedOnMessageRecieved>b__0
2017-04-25 23:40:53,921 [Main] ERROR - Unable to load image file error
2017-04-25 23:40:53,921 [Main] ERROR -    at BinaryRivers.Common.ImageLoader.ImageLoader.Load(String fullpath, Boolean limitsize)
   at BinaryRivers.BackyardEOS.Modes.CaptureMode.<>c__DisplayClass30_0.<ThumbnailClickedOnMessageRecieved>b__0()
2017-04-25 23:40:53,921 [Main] ERROR - PREVIEW_20170425-00h00m19s868ms.JPG
 

The full log in the message above. 

[Alexey Popov]

I do not think that Backyard has memory leaks. In this case all applications would have problems and I would get it from different software. And I would see it in the resource monitor. I have to check, but I do not remember that. 

The problem could be in memory fragmentation. Free memory is enough, but its not possible to get really big piece of memory. But its a week hypothesis too. 

[astroman133]

No, this is not a physical memory limitation or a memory leak in BYE or other process. It is a limit on the maximum memory allocation for 32-bit process.

[admin]

This out of memory is a .net error.  This has been discussed many times here and unfortunately not much I can do.  Let me explain,

When an image is loaded .NET will reserve the amount of memory needed to load the image into based on the file meta data.  If not enough memory is reserved .NET will throw an out-of-memory error.

You did not run out of memory per say, .net simply did not reserve enough memory to load the image into.

This is a .net issue out of my control  as a single command to load the image is send to .net with the image file path... and it fails to load it

Regards.

[Alexey Popov]

4 hours ago, astroman133 said:

No, this is not a physical memory limitation or a memory leak in BYE or other process. It is a limit on the maximum memory allocation for 32-bit process.

Uh, yes.. 32-bit application.. Yes, you are right. 

[Alexey Popov]

5 hours ago, admin said:

This out of memory is a .net error.  This has been discussed many times here and unfortunately not much I can do.  Let me explain,

When an image is loaded .NET will reserve the amount of memory needed to load the image into based on the file meta data.  If not enough memory is reserved .NET will throw an out-of-memory error.

You did not run out of memory per say, .net simply did not reserve enough memory to load the image into.

This is a .net issue out of my control  as a single command to load the image is send to .net with the image file path... and it fails to load it

:(( Really bad.

So, this mean that everybody can be faced with this problem simply after Windows upfdate? 

In case it depends on the image meta data, may be its possible to write something into the file to force .Net reserve more memory? Or simply use 3-d party library to load images into the memory? There are a few possibilities to find way around usual. 

As you can see in the video, after 5 minutes its simply not possible to work with BYE. I do nothing special. This could bring you to the very bad situation when many users will be disappointed. And unfortunately, your sentence "this is a .net issue out of my control" would not be excuse.

To be honest, I understand you very well. Couple of months ago my team was faced we the same situation. We have commercial b2b product, suddenly crashes appeared. 3 months we was looking for the root of the problem and find it in a .Net update. We made special test software to make diagnostic and let our users test their systems and remove harmful update. And unfortunately this is quite often situation (according with my 15 years experience). Usually, we are trying to find way around. The another problem we were faced in the same time was with delay during file opens. Time to time file open operation takes a lot of time under Windows. But our product has some real time functions.. We had to write special file cache system. Another issue - sometimes Windows lost notifications that file was changed.. And so on, so on... If we say "this out of our control" - we lost customer. Very simple rule... 

[admin]

Oddly enough you may try another computer and never, ever have the issue, even with the same images.

astroman133 sent me a link the other day, it may contain a solution but I'm not sure.  I'll look at it a keep you posted.

Regards,

[admin]

12 hours ago, Alexey Popov said:

:(( Really bad.

So, this mean that everybody can be faced with this problem simply after Windows upfdate? 

In case it depends on the image meta data, may be its possible to write something into the file to force .Net reserve more memory? Or simply use 3-d party library to load images into the memory? There are a few possibilities to find way around usual. 

As you can see in the video, after 5 minutes its simply not possible to work with BYE. I do nothing special. This could bring you to the very bad situation when many users will be disappointed. And unfortunately, your sentence "this is a .net issue out of my control" would not be excuse.

To be honest, I understand you very well. Couple of months ago my team was faced we the same situation. We have commercial b2b product, suddenly crashes appeared. 3 months we was looking for the root of the problem and find it in a .Net update. We made special test software to make diagnostic and let our users test their systems and remove harmful update. And unfortunately this is quite often situation (according with my 15 years experience). Usually, we are trying to find way around. The another problem we were faced in the same time was with delay during file opens. Time to time file open operation takes a lot of time under Windows. But our product has some real time functions.. We had to write special file cache system. Another issue - sometimes Windows lost notifications that file was changed.. And so on, so on... If we say "this out of our control" - we lost customer. Very simple rule... 

 

There are already 3 different method built into BYE to load images.  .Net, a 3rd party commercial library called GDpictures, and DCRaw.  If one fails it goes to the next and so.  After 3 attempts it stops.

Is this bad, sure.  Is this generalized... no.  I I have had about 2o reported case.  Lets assure for each case there a 5 other case.  This is about 100 cases out of 30,000 copies in circulation.

It does not means this can not be solved.  It means because it is random it is difficult to solve and even more difficult to reproduce in a controlled envionment.

Recent tests that I have ran today demonstrate a clear reduction in this issue when you do 2 things in Advance settings

1) Enable the Background server (Important)

2) Use processor affinity (less important)

Without these 2 settings I got the occasional out of memory error when dragging 50 images to be loaded in BYE

With these 2 settings all 50 images were loaded and I got 0 out of memory errors.

Try then and see if it makes a difference.

Regards,

 

[astroman133]

When you load many images, can you display all of them? What I see is that for some images, clicking on the thumbnail does not cause the image to be displayed, even though the thumbnail is correctly displayed.

Selecting to use the BackgroundWorker and Processor Affinity eliminates this issue, although it is much slower to load 15 images via Drag and Drop.

[Alexey Popov]

50 minutes ago, admin said:

 

There are already 3 different method built into BYE to load images.  .Net, a 3rd party commercial library called GDpictures, and DCRaw.  If one fails it goes to the next and so.  After 3 attempts it stops.

Is this bad, sure.  Is this generalized... no.  I I have had about 2o reported case.  Lets assure for each case there a 5 other case.  This is about 100 cases out of 30,000 copies in circulation.

It does not means this can not be solved.  It means because it is random it is difficult to solve and even more difficult to reproduce in a controlled envionment.

Recent tests that I have ran today demonstrate a clear reduction in this issue when you do 2 things in Advance settings

1) Enable the Background server (Important)

2) Use processor affinity (less important)

Without these 2 settings I got the occasional out of memory error when dragging 50 images to be loaded in BYE

With these 2 settings all 50 images were loaded and I got 0 out of memory errors.

Try then and see if it makes a difference.

Regards,

 

Thanks for recommendation! I`ll make test today slightly later and let you know about the result. The sly is clear, I`ll take some pictures. 

Its strange that 3 different library has similar problem... Are all these libraries .Net-based? 

Hm... What I see right now is very interesting (with old settings).. In the resource monitor I see that when I click on the pictures on the thumbnail bar BYE eat memory really fast. It was 500 mb a few seconds ago, and after a few clicks - 1.2 Gb and even more. Its very close to the limit. And whats really fantastic - the next successful picture display I have when BYE fall down to 500-600 Mb! Practically, win32 application can get something like 1,6-1.8. GB. Therefore:

1. Its really could be a out memory problem because .Net does release memory fast enough. Its not a bug. 

2. But much more likely its fragmentation problem. Its not possible to get a big enough piece of memory because 3-4 big piece of memory was given already and brake a 1.8 Gb to a few small parts.  

May be solution is to do something with garbage collector? Or make a cache and read files to memory on the lower level. And disable async  pictures operations as much as possible. 

Ok, I`ll change settings and try again. 

 

And about 20 issues from 30000.  Let say that 100 your users have such problems and request money back or will be just disappointed (really this quantity can be much more). Problem is not in this hundred. Problem in other 500-1000 and more potential users who listen to these 100 disappointed and decided to buy something else. This is a problem. But definitely cost of detractor and promoter should be calculated in each practical case, and may be that in your case its not so dangerous. Ok, lets close this point, you know what does it mean and what to do.

[Alexey Popov]

38 minutes ago, astroman133 said:

When you load many images, can you display all of them? What I see is that for some images, clicking on the thumbnail does not cause the image to be displayed, even though the thumbnail is correctly displayed.

Yes, this is a problem 1. If wait some time - this picture will be displayed on the next lucky click. 

[Alexey Popov]

Ok, tested with both parameters switched on. Drag and drop works! 17 from 17! During this operation BYE works with 500-600 Mb. But, when I click on the picture in teh thumbnail BYE display picture successfully but take memory up to 0.9 - 1.2 Gb and that`s all. The next click fails. After sometime (from a few seconds up to at least tens of seconds) BYE free some memory (falls down to 500-700 Mb) and the next successful click is possible. 

So the solution is to find a way how to release memory faster (garbage collector?) and to avoid simultaneous picture opening. Because in any cases its not possible tp load a few pictures. 

Or to make image cache and handle pictures manually. So instead of release and take memory again just re-use the same.  

[Alexey Popov]

Hope setting at least help me not to lose recently taken pictures. Continue to collect information... 

[Alexey Popov]

:(( It does not help. BYE lost images during preview shooting. 

[Alexey Popov]

Ok, made different tests.

If BYE already took more then 800 Mb - with very high probability (~95%) it fails with picture display by the click on thumbnails. More then 1 Gb - no chance. If I make preview shooting when BYE already took more then 900 Mb - the new picture does not appears in thumbnail at all. And if I wait a few minutes (print this post e.g.) BYE release memory down to 300-400 Mb and all functions start works properly.  

[Alexey Popov]

Whats interesting else it that even during failed attempt to display picture BYE took a memory and does not release it.  

[Alexey Popov]

There are a lot of articles about .Net GC, memory fragmentation and so on... And a lot of problems around that. 

If you have fragmented heap you do not need 3 libraries to load a picture. It fails, all 3 together. May be second or third will b more lucky and in the moment of third attempt memory will be released. But in this case you need just to way a bit or make more attempts with first function. I think the best way - use .Net and try to find a way hour to dispose picture`s memory ASAP. 

By the way, may you keep old pictures too long somewhere? Time to time I see that BYE keep more then 1 gb for a few minutes... 

Some useful links...

https://blogs.msdn.microsoft.com/mariohewardt/2013/06/26/no-more-memory-fragmentation-on-the-net-large-object-heap/

http://www.informit.com/articles/article.aspx?p=1409801&seqNum=4

http://library.softwareverify.com/memory-fragmentation-your-worst-nightmare/

https://www.simple-talk.com/dotnet/net-framework/large-object-heap-compaction-should-you-use-it/

https://blog.maartenballiauw.be/post/2016/10/19/making-net-code-less-allocatey-garbage-collector.html

https://blogs.msdn.microsoft.com/dotnet/2011/10/03/large-object-heap-improvements-in-net-4-5/

 

[admin]

Okay, using the background server solved the out of memory issue.

Only thumbnail not showing sometimes is the remaining issue.

Regards,

[Alexey Popov]

1 minute ago, admin said:

Okay, using the background server solved the out of memory issue.

Only thumbnail not showing sometimes is the remaining issue.

Regards,

No, it does not. It helps only during Drag and drop. During clicking and shooting - nothing changed. Sorry for the mess of posts

[admin]

Forget about what process manager reports in memory used.  This is a false reading because the images are disposed of and the process manager will still show the memory as being used event though it is not.  Once the garbage collector passes it will get recycle.    

[admin]

Just now, Alexey Popov said:

No, it does not. It helps only during Drag and drop. During clicking and shooting - nothing changed. Sorry for the mess of posts

What do you mean clicking?

[Alexey Popov]

Just now, admin said:

What do you mean clicking?

Clicking on the thumbnails like in the yesterday video - absolutely the same result. 

[admin]

Forget about clicking... that is thumbnail related and you already said this was still an issue.  Your are trying to troubleshoot an issue and you keep performing multiple tasks.

With the background server enabled, if you take 20 images and do nothing else while it is taking those images, do you get 20 images loaded and 20 images in the thumbnail?

 

[Alexey Popov]

2 minutes ago, admin said:

Forget about what process manager reports in memory used.  This is a false reading because the images are disposed of and the process manager will still show the memory as being used event though it is not.  Once the garbage collector passes it will get recycle.    

What I see is the 100% correlation between BYE memory usage in the task manager resource monitor and the result of clicks on thumbnails and of shooting. If BYE use more then 750-800 Mb - I have a problems. Are you sure that task manager lie more then GC? Dispose does not guarantee immediate memory releasing as far as I know. Try to get more information tomorrow... Good night.

[Alexey Popov]

9 minutes ago, admin said:

Forget about clicking... that is thumbnail related and you already said this was still an issue.  Your are trying to troubleshoot an issue and you keep performing multiple tasks.

With the background server enabled, if you take 20 images and do nothing else while it is taking those images, do you get 20 images loaded and 20 images in the thumbnail?

 

1. I droped 17 pictures and I got 17. 

2. Clicking still have a problem if BYE use more the 750 Mb

3. Shooting still have a problems (lose picture, it does not appears in thumbnails) if BYE use more then 800-900 Mb. 

[admin]

Okay, thank you for your hard work running those tests.  I really appreciate it.

Regards,

EDIT:  I just ran session plan with 100 pictures and it never got over the 450MB... then drops to about 300MB... goes up, then drops again.  Weird that yours goes close to 800-900