The support forum

Delta copy not working for file which I think should

Filecollector :

Feb 24, 2019

Hello!

For my Backup strategy I am using a combination of the software Macrium Reflect and Bvckup 2. With Macrium I sheduled a system backup which is a full backup (ca. 300 GB file) plus a daily incremental of only the changed block since the full backup (an extra file which is a few GB). Every day the oldest incremental backup file gets merged into the full backup file, which according to the Macrium documentation happens "in place". I believe that this only changes a small part of the big "full backup file" when merged every day, which is plausible because the merge takes only about one minute.

Now, I use Bvckup 2 to backup the backup destination folder of Macrium Reflect onto an external USB3 HDD. I was expecting that Bvckup 2 would detect the delta in the big "full backup file" and would only have to copy a few gigabytes each day. However, I am a bit dissapointed to observe, that each time the *entire* ca. 300 GB file is copied over again, making this very inefficient.

What do you think is the explanaition for this? Does Macrium perhaps indeed modify the entire full backup file at each merge, or is Bvckup 2 unable to correctly detect the delta?

I have observed the delta copying behaviour of Bvckup 2 with a ca. 30 GB Virtual Machine Disk file. It works... but shouldn't it also apply to my above situation? For a long time I had been searching for a delta copying capability under windows for exactly this purpose (to backup the backup of my system), and Bvckup 2 looked like it would fullfil my expecation, based on it's features description.

Thanks for any advice / help!
Nick

Froggie :

Feb 24, 2019

Nick, I can only add some Macrium insight here.  If you are running Forever Incrementals, once all the incrementals are in place, the merge forward of the FULL is significant in its changes... thay are scattered all over the image and include changes not just to disk blocks (512b) but to disk clusters (4K bytes).

Filecollector :

Feb 24, 2019

Hmm, Froggie, are you saying that Macrium modifies the *entire* backup file when merging the incremental file into it?

Bvckup 2 does not detect any unchanged parts of the file (the entire progress bar is one color).

Froggie :

Feb 24, 2019

I can't say whether it's the entire file, I know it's a lot of it. Alex may be able to shed more light on this as far as the progress bar is concerned or exactly how much of the file is being dealt with.

Froggie :

Feb 24, 2019

Nick, I see the exact same thing in reference to just merging Incrementals, not using FOREVER mode.  I'm sure this is a Macrium Reflect image formatting issue rather than a delta copying issue.  The Reflect image is in tact (as far as Reflect is concerned), it's just changed its positional reference as far as delta copying is concerned.  There may be something Reflect writes towards the beginning of the file that causes everything to shift a bit.

Froggie :

Feb 25, 2019

From Macrium's Dev team (it looks like the DELTA COPY algorithm is making the decision to do the whole file)...

"Consolidation is non destructive. So, for the very first consolidation changed blocks are appended to the end of the data blocks in the 'From' file. This means that in the event of a failure the original blocks are still there and the index can be re-appended to reconstruct the original file.  The next and subsequent consolidations will place changed blocks in the gaps created by the previous consolidation(s). If there is insufficient space then blocks are added to the end. Again, this is non-destructive in that the original blocks are unmodified and can be recovered.  For a synthetic full this means that the consolidated Full will grow in size, but only by the size of the largest consolidated incremental.

The contents of the file is never 'shifted' in any way so it would appear that the delta copy routine that is being used is unable to handle the data pattern change that occurs in the file."

Alex Pankratov :

Feb 25, 2019

What do you think is the explanaition for this?


Based on what you described, the delta copying should work. It is after all is dead simple in what it does - it looks at each file block, checks if it was modified since the last run and copies it over if it was.

So in your case it's either that each and every 64K block of the file is touched (which clearly shouldn't be happening) or that the _backup_ copy of this file is touched between the backup runs. When this happens (ie when the timestamps/size of a backup copy change between backup runs), you will have a message to that effect logged and the file will be re-copied in full.

What do you see in log exactly?

Let me see the full log excerpt for an update like you describe. Right-click in the "Updating file..." line and select "Copy block to clipboard" to pick the whole thing up. Then post it here.

Froggie :

Feb 25, 2019

Alex, don't know whether this helps or not but the process used to create the "merged" file in question is as follows...

You have (2) Incremental images (#1 & #2) and # 3 is created.  During the merging process described above, #2 & #3 are merged backwards into #1.  #s 2&3 are deleted and #1 is RENAMEd to # 3, thereby eliminating #s 1&2 and bringing their System changes into #3.  The constant recreation of #<n> is what happens in the above described process.  My guess is the file triplet is definitely changing for the merged file each time its recreated even though its content is changing in a much, much smaller way.

Alex Pankratov :

Feb 25, 2019

This should be fine. Delta copying is reset only of the triplet on the *backup* copy of a file changes.

New topic

Create
Made by IO Bureau in Switzerland
Support

Updates
Blog / RSS
Follow Twitter
Reddit
Miscellanea Press kit
Testimonials
Company Imprint

Legal Terms
Privacy