Filename time stamp discrepency

[johnastro.info@gmail.com]

Problem: The filename timestamp lags the shutter open time by about half a second

At some point, this time stamp was changed to reflect the shutter open time rather than the file write time. In the latest versions of both BackyardEOS and BackyardNikon, the time stamp can be anything from 0.3 seconds to 2.0 seconds out, depending on the speed of the PC.

We would be very grateful if this could be corrected. We need accurate timing for the project we are currently involved in - monitoring space junk (old satellites, rocket bodies etc)

Thanks, John

[astroman133]

You should add your post to the Suggestion Box Forum.

[tcgainey]

According to the release notes archive, from BYEOS 3.1.3 the timestamp was changed from the time the file was saved to the time the shutter was instructed to open. The actual image start time should be slightly after this due to shutter lag.

I am also using BYEOS 3.1.16 and seeing variable timestamps which are from 400ms to 1000ms AFTER the shutter opens. (Using BYEOS to photograph the PC clock displayed in ms).

It would appear that the software has reverted to recording the file save time again rather than the shutter open command time as otherwise I cannot see how I have a negative shutter lag.

Trevor

[astroman133]

Trevor,

My testing indicates that your are incorrect when you say that the software appears to be using the file save time in the file name. This is not what my investigation shows.

I took a 10 second exposure with BYE 3.1.17 RC2 and my T5i. I used a digital watch for the timings and recorded the minute and second that was displayed when the event was recorded.

01:10 - Click Start Capture
01:12 - Shutter opens
01:22 - Shutter closes
01:26 - Image file created (BYE completion tone)
01:12 - Time in timestamp portion of the file name
01:26 - Time from BackyardCaptureDate value in EXIF data

According to what I am seeing, it appears that BYE is behaving as expected. That said, it may not be as accurate or as repeatable as John needs. It may be good for Guylain to look to see if it is possible record the start time closer to when BYE calls the SDK command to open the shutter. There will likely still be some delay.

[admin]

49 minutes ago, astroman133 said:

Trevor,

My testing indicates that your are incorrect when you say that the software appears to be using the file save time in the file name. This is not what my investigation shows.

I took a 10 second exposure with BYE 3.1.17 RC2 and my T5i. I used a digital watch for the timings and recorded the minute and second that was displayed when the event was recorded.

01:10 - Click Start Capture
01:12 - Shutter opens
01:22 - Shutter closes
01:26 - Image file created (BYE completion tone)
01:12 - Time in timestamp portion of the file name
01:26 - Time from BackyardCaptureDate value in EXIF data

According to what I am seeing, it appears that BYE is behaving as expected. That said, it may not be as accurate or as repeatable as John needs. It may be good for Guylain to look to see if it is possible record the start time closer to when BYE calls the SDK command to open the shutter. There will likely still be some delay.

This is the intended behavior.  If you see something else please be a specific as possible in order to understand the issue; it wasn't 100% clear what your observed behavior is.

[tcgainey]

We really appreciate the rapid responses.

John and I are part of a citizen science project in collaboration with the UK government to assist them in tracking space debris. Specifically they have asked us to track the Remove Debris satellite that was launched this week from ISS.

We have been asked to provide wide field photographs of about 1-3 sec exposure to track the satellite trails which can then be plate solved to accurately determine the orbits and decay paths. In order to do this they have asked us to have accurate timing of the photos to about 100ms or better. We are all training our system time to UTC via ntp servers or GPS timing modules to get ms accuracy and this seems to be effective. Your timestamp is to 1ms resolution so seems ideal. We realize that the accuracy of the timestamp will not be perfect so in a series of calibration exercises we took images of ISS and submitted them for analysis with widely varying results. There are about 8 of us in our society using BYE and one with BYN, all using the latest version 3.1.16.

To determine the root cause of the discrepancies John developed a Java app that displays the system clock in ms. I took a series of 10 frames at 1/000s exposure using BYE and my 500D of the PC running BYE and compared the system time shown in the photo to the timestamp. From the description in your release notes we understood that BYE creates the timestamp from the system clock at the time the shutter is told to open. We expected that there would be some shutter lag whilst the mirror moves etc so thought we would see perhaps 50-100ms delay due to shutter lag and the screen displayed time in the photo would be after the timestamp. If this was consistent we could calibrate this out of our results and provide an accurate time of observation. (actually we can provide two positions from one plate as we measure the position and time of each end of the trail).

A few of us have completed this test with varying results but all show the same issue: The system time captured in the images is before the timestamp, which we do not understand. In order to check that the Java app that displays the system time was not the cause I also downloaded an Android app (Atomic Clock) and synced this with the same ntp server as I am using to train my system clock. I included this in the photograph when I repeated the test so I can compare Timestamp from BYE against system time as shown on the PC screen by the Java app and also UTC as shown on the Android Tablet leaning against the PC screen.

I can provide actual results tables and screenshots if required but in summary:

The PC Screen time and Android app derived time agree to about 45ms so we think that the Java app displaying our PC system time is OK.

The timestamp on BYE images differs from Java displayed system time by an average of 545ms but as stated above the timestamp is after the captured system time. The worrying part is that in a series of 10 images taken in a single sequence this negative shutter delay varied from 406ms to 904ms so we currently cannot calibrate this out of our results. (In a previous similar experiment I go an average of 503ms difference with a range from 390ms to 1016ms).

We have seen differences with each setup due to differing cameras, PCs operating systems etc and are trying to assemble this data for better analysis but we all see the same issue to a greater or lesser extent. (The worst case errors seem to be on BYN which showed an average negative shutter lag of over 2 seconds.

Since you derive the BYE frame timestamp from the same system clock we are photographing on screen can you please take a look at when in your code you are capturing the timestamp compared to when the shutter open command is provided to the camera?

Thanks again

Trevor

[johnastro.info@gmail.com]

Thanks for looking into this. As requested, more details:

(1) You are correct - the time stamp is closer to the shutter open time than the file write time

(2) Provided the computer is not too old and slow, the time stamp is less than a second from the shutter open time. Typically the discrepancy is between 0.3 and 0.5 seconds. Unfortunately our project needs the accuracy to be greater than 0.1 seconds.

(3) I have written a Java program that displays the seconds and milliseconds components of the PC system time. Photographing this clock with a 1/1000 second exposure will reveal the accuracy. To run the program, unpack the zip file and double click on the clock.exe

https://drive.google.com/file/d/1ZbdxBuLQIGoz6pQw6GwsYCYGfKjH9V_R/view?usp=sharing

All the required Java run time files are supplied in the correct relative directories. To uninstall after using, simply delete the folders you unpacked. This was built for Windows 10.

(4) Alternatively, if you have the Java JRE installed on your computer, simply save the attached Clock.jar file and double click on it. This should work on any OS.

If you are interested in the clocks source code, you can find it here:

https://drive.google.com/open?id=1L0YzUtUT0z8gIOOd1Ntg7bjmTRSsioa6

Your help on this would be most appreciated.

Thanks, John

Clock.jar

[astroman133]

Trevor,

Your project sounds interesting and worthwhile.

You mentioned that the format of the timestamp in the file name is a high-resolution value. My guess would be that this format is used in order to ensure the uniqueness of the file name rather than to imply any measure of accuracy of the timestamp itself. Actually getting the repeatability that you need may prove difficult with a general purpose multi-tasking operating system like Windows where it is not possible for applications to control context switching to an absolute extent..

I am sure that Guylain will do what he can to help. Speaking as a retired developer who used to write real-time applications, it may be possible to get some improvement in repeatability, but applications really have little control over when Windows performs context switches. One obvious approach is to structure BYE so that it records the image start time and starts the exposure from a high-priority thread. The fact that BYE is going through the Canon SDK to actually talk to the camera makes me wonder if this approach will be effective.

Thinking back to a year ago, I was learning how to use Eclipse Orchestrator to shoot images of the upcoming solar eclipse in the US. The author of Eclipse Orchestrator recommended using a remote shutter cable like the Shoestring Astromony DSUSB cable. BYE supports this cable, as does BYN, for some camera models. His claim was that it gave a faster response than going through the SDK. Since this is important to you it may be worth investigating.

I hope this is helpful.

[tcgainey]

We have received further feedback from our agency regarding the errors in our timing data when calibrating our equipment. In my case I was using a 1 second exposure and the agency plate solved both the start and end of the satellite trail. Looking a the timing errors determined from the very well known orbit of the ISS we can determine the timing errors of the shutter opening. John and I have bot noticed that the tiing error for the start position and end position are not consistent. In my case I have an average timing error when the shutter opens of 0.26s but and average error when the shutter closes of 0.68s so the average exposure appears to be 1.42s when it was supposed to be 1.0s for all frames.

We had the camera controlled via USB and used B mode and set the exposure time to 1. As an alternative I guess we could have used M mode and selected 1s from the drop-down menu.

When used in B mode does the camera needs a Shutter open command and a Shutter close command and could there be additional delays to both giving us a random start delay and an extended exposure time?

If so would using the camera in M mode and picking 1s exposure from the drop-down menu use the camera timer for the exposure length and therefore eliminate one cause of error?

Thanks

Trevor

p.s. We looked at the cable solution but it would be quite expensive and we would probably overload the supplier with orders as we all need a solution in the next couple of weeks.

[astroman133]

You are correct Bulb mode exposures (both the start and the end) are controlled by BYE. A Tv exposure (what you called M mode) is started by BYE, but timed by the camera. I would expect that Tv exposures would have a more repeatable duration. As a matter of course Tv should be used whenever the exposure is 30 seconds or less.

You should summarize your need, along with your use case, in a post in the Feature Suggestion Box Forum. This is where the authors go to find suggestions for enhancements. Your request certainly seems to qualifiy, although I am not sure how much improvement is possible.

[tcgainey]

I took another look at the User Manual with respect to using a serial cable in case it gives us any better timing accuracy. It says that any home brew cable should work if correctly wired, whatever that means.

I am looking at using a USB-to-relay module from Amazon and connecting this to cable with a 2.5m jack plug. The device indicates the coding required to make the com port fire: e.g.

USB switch the default communication baud rate: 9600 BPS
USB switch communication protocol
Data (1) - startup logo (the default is 0 xA0)
Data (2) - switch address code (the default is 0 x01, identifies the first switch)
Data (3) - operation data (0 x00 to "off", 0 x01 to "on")
Data (4) - check code
For example:
Open the USB switch:A0 01 01 A2

Is this the same coding format that BYEOS uses to fire a serial port?

Thanks

Trevor

[astroman133]

I believe that the way the DSUSB cable works is that BYE opens the serial port which causes the shutter to open then closing the serial port closes the shutter.

Guylain will need to confirm, however.

 

UPDATE: Oops, I forgot about recent support for Relay devices. Guylain WILL need to answer how they work.