The support forum

Destination snapshot not reliable over long time?

chris2 :

Jul 20, 2019

Up to now I used the "destination snapshot" option as the backup destination (network drive) is only touched by the Bvckup tool. The backup is running every 3 hours.

I now changed the settings to "rescan every time" and the outcome was, that at the end around 1300 changes were detected. It is very unlikely that in the last three hours so many changes in the personal data folders were done.

Could there be any scenario where the "destination snapshot" option is not working properly? (Expect for manual changes in the destination).

I don't mind if the backup takes some more time, but I would like to avoid that the fans of the laptop speed up due to the big backup job (200 GB?). Therefore I already set the option "run at low system and disk consumption".

Alex Pankratov :

Jul 20, 2019



I am not aware of any issues with destination snapshot getting out of sync on its own. Before the program updates any per-file information in the snapshot, it re-queries it from scratch. So the info that is saved is the actual reported data *as it existed immediately after the file was processed*.

In your case, I'm guessing, there's some sort of post-processing that's done by the device itself or by some other program that happens shortly after bvckup2 finishes processing a file. This changes file timestamps, which is exactly the kind of thing that's assumed to be NOT happening for the snapshots to work correctly.

Do this - in the backup log, pick any file from those 1300 changes, right-click on its entry, select "Copy block to clipboard" and let me take a look at it. I bet that timestamps between source and backup copies are quite a bit off.

chris2 :

Jul 22, 2019

Thank you much for your fast answer and this very good software tool.

As the 1st backup with complete re-scan was some time ago, I could not clearly identify the logs for that. But searching in the logs a bit and after starting some test backup jobs I indeed found, that the file change reason was quite often "time stamp", as you assumed:

"What: timestamps (modified, accessed)"

This relates to data files from special programs which seem to make these changes on the data files when working with the program.

It is not a problem for me if these files are updated in each backup because they were touched by the special program.

Just to be sure if I understood it correct regarding the backup process itself: As long as nobody changes anything in the destination location, using the "snapshot" option over months and years produces the same exact copy as if the destination is rescanned each time. Re-scanning is not more accurate. Correct?

Froggie :

Jul 22, 2019

Chris, I'm sure Alex will expound more extensively on this topic, but my experience is that if your DESTINATION storage device is not "touched" in any way by an external process (and that is definitely not the case with certain NASes and some cloud storage devices), it will always represent the true status of the stored files.

Many external "intelligent" storage devices (NASes in particular) have a problem in this area due to how their local OS manages time stamps.  Alex has developed many workarounds for these anomalies but in many cases they still exist.

Alex Pankratov :

Jul 22, 2019

As long as nobody changes anything in the destination location, using the "snapshot" option over months and years produces the same exact copy as if the destination is rescanned each time. Re-scanning is not more accurate. Correct?


Yes, that's correct.

It's still not a bad idea to run a backup with full rescan (you can do that by launching it manually via Ctrl-Go), because some backup copies may end up being changed through basic oversight. Opening a photo in a viewer, for example, may touch EXIF data or rotate the image or what have you.

chris2 :

Jul 23, 2019

OK, thank you again for this explanation. And let me emphasize again, that using your software and experiencing your support provides a really good feeling regarding a reliable backup.

The destination is indeed a Synology NAS drive with newest firmware. If the Synology OS messes up with the time stamp this is not a "real" problem as the file content at the destination is safe anyhow.

You can either decide
1) to re-scan and maybe do some unnecessary copies (and consume some time for re-scanning)
2) to use the destination snapshot which means you shut your eyes regarding possible changes at the destination.

I tested again and made some backups with some minutes distance of my personal data folder. And each time there are some changes indicated (currently 10) unless definetely no change was made at the source manually or a controlled end user programm.

1. Sample 1: .htaccess Text file in Homepage Archive

-----------------------------

2019.07.23 09:12:39.977 (UTC+1) 2 2         1. Updating file XXX\Homepages\XXX\.htaccess
2019.07.23 09:12:39.977 (UTC+1) 3 3             Details
2019.07.23 09:12:39.977 (UTC+1) 3 4                 Source: 75 bytes, created 2015.11.13 17:05:04.292, modified 2019.07.21 22:01:35.268, archive
2019.07.23 09:12:39.977 (UTC+1) 3 5                     Raw: 75 / 130919007042922553 / 132082128952683722 / 00000020
2019.07.23 09:12:39.977 (UTC+1) 3 4                 Backup: 75 bytes, created 2019.07.23 09:01:07.311, modified 2019.07.21 22:01:35.268, archive
2019.07.23 09:12:39.977 (UTC+1) 3 5                     Raw: 75 / 132083388673118825 / 132082128952683722 / 00000020
2019.07.23 09:12:40.018 (UTC+1) 3 3             Completed in 41 ms, copied in full

2. .exe in Software Tools Archive

-----------------------------

2019.07.23 09:15:03.199 (UTC+1) 2 2         1. Updating file XXX\EOS615DUO3Plus\__MACOSX\._EOS_FW_UP_24.exe
2019.07.23 09:15:03.199 (UTC+1) 3 3             Details
2019.07.23 09:15:03.199 (UTC+1) 3 4                 Source: 240 bytes, created 2013.04.30 10:36:50.370, modified 2012.01.18 08:07:54.000, (no attributes) / normal
2019.07.23 09:15:03.199 (UTC+1) 3 5                     Raw: 240 / 130117846103705865 / 129713404740000000 / 00000080
2019.07.23 09:15:03.199 (UTC+1) 3 4                 Backup: 240 bytes, created 2013.04.30 10:36:50.370, modified 2012.01.18 08:07:54.000, hidden
2019.07.23 09:15:03.199 (UTC+1) 3 5                     Raw: 240 / 130117846103705865 / 129713404740000000 / 00000002
2019.07.23 09:15:03.202 (UTC+1) 3 3             Updating: last-modified timestamp, attributes

Executing this backup again and again, the number of changes reduced a bit but at the end remained at the number of 10. Looking at the copy log of the sample file 2 from above, a later log looks exactly the same:

2019.07.23 09:22:39.349 (UTC+1) 2 2         1. Updating file XXX\EOS615DUO3Plus\__MACOSX\._EOS_FW_UP_24.exe
2019.07.23 09:22:39.349 (UTC+1) 3 3             Details
2019.07.23 09:22:39.349 (UTC+1) 3 4                 Source: 240 bytes, created 2013.04.30 10:36:50.370, modified 2012.01.18 08:07:54.000, (no attributes) / normal
2019.07.23 09:22:39.349 (UTC+1) 3 5                     Raw: 240 / 130117846103705865 / 129713404740000000 / 00000080
2019.07.23 09:22:39.349 (UTC+1) 3 4                 Backup: 240 bytes, created 2013.04.30 10:36:50.370, modified 2012.01.18 08:07:54.000, hidden
2019.07.23 09:22:39.349 (UTC+1) 3 5                     Raw: 240 / 130117846103705865 / 129713404740000000 / 00000002
2019.07.23 09:22:39.351 (UTC+1) 3 3             Updating: last-modified timestamp, attributes

The only difference between source and destination each time seems to be the 3rd number "00000080" vs. "00000002".

Maybe this indeed the Synology timestamp problem as also discussed here: https://bvckup2.com/support/forum/topic/762

Alex Pankratov :

Jul 23, 2019

.htaccess


It was updated because "created" timestamp was off on the backup copy. When cloning timestamps, bvckup2 will first test if "created" timestamps are supported in general (and with Synology they are), then clone them and finally read them back and verify that they were set correctly.

In your case all of that worked, but then the timestamp somehow got reverted to "now". I can only guess it's one of Synology many quirks.

You can work around this by telling the program to NOT look at "created" timestamps when deciding if a file was modified or not. See here for details - https://bvckup2.com/support/forum/topic/139

._EOS_FW_UP_24.exe


Synology repeatedly sets a Hidden attribute on this file for whatever reason. Probably due to seeing it in __MACOSX folder, but your guess is as good as mine. In either case it shouldn't be doing that, but it does, so bvckup2 notices that and attempts to reset the attributes to match the source. It doesn't re-copy the file, so it's a smaller issue, but it's an issue still.

*This* you can work around by adding the following override to the job's configuration:

         conf.attr_comp_mask     23

This will force the program to look only at ReadOnly (1), Hidden (2) and Archive (20)    =>    1 + 2 + 20 = 23

For how to add an override see here - https://bvckup2.com/support/forum/topic/1140

New topic

Create
Made by IO Bureau in Switzerland
Support

Updates
Blog / RSS
Follow Twitter
Reddit
Miscellanea Press kit
Testimonials
Company Imprint

Legal Terms
Privacy