The support forum

Aging and discarding delta copying state

Sep 07, 2020

Overview


As of release 81.7 it is possible to automatically discard delta copying state for a file after it's been used a certain amount of times or when it reaches a certain age.

When this happens, it forces the file to be re-copied in full and its delta state recreated from scratch.

This can be used as an ad-hoc data scrubbing mechanism for the backup media [1] or as a hedge against backup copies acquiring data rot in some other way.

Usage threshold


The usage threshold is configured by adding the following override [2] to the backup configuration:

        conf.copying.ultra.delta.max_use_cap   <number>

This will cause file's delta state to be used "number" times and reset on the next copy. When this happens, the following message is logged -

        https://bvckup2.com/wip/r81-7-delta-state-caps.png

The number _can_ be set to 1, causing the delta state to be reset on every copy. This is supported solely for testing purposes and has no practical use.

Age threshold


The age threshold is configured by adding the following override [2] to the backup configuration:

        conf.copying.ultra.delta.max_age_cap   <time interval>

whereby "time interval" is in the "<count> <unit>" format, e.g. "10 minutes", "24 hours", "2 days", etc.

Effective thresholds


Effective threshold values are reported in "Preparing" > "Configuring Details" > "How To Copy" > "Delta Copier" section of the backup log.

        https://bvckup2.com/wip/r81-7-delta-state-caps-config.png

---

[1]  Data scrubbing - https://en.wikipedia.org/wiki/Data_scrubbing
[2]  Adding an override - https://bvckup2.com/support/forum/topic/1140

Homer712 :

Sep 15, 2020

So the override for the usage threshold would look as follows:

conf.copying.ultra.delta.max_use_cap30.ini

And it would be put in the same folder as any other override. Correct?

Alex Pankratov :

Sep 16, 2020

No, not like that.

You need to create override.ini file as per [2] above and put the following line in it -

         conf.copying.ultra.delta.max_use_cap   30

No need for preserve leading spaces, but it won't hurt if they are there.

Save override.ini next and reload the job via Ctrl-right-click menu (or just restart the program).

Homer712 :

Sep 16, 2020

Sorry, you follow directions here, do things, and then a few months later draw a complete blank. I'm blaming it on "seasoned citizen" status.

I had already created one override which was as follows: conf.archive_modified      1

That one is still there in the folder (even though the instruction say that it can be deleted once active). I'll delete that one, and add the new: conf.copying.ultra.delta.max_use_cap   30

I'll need to make a sticky note somewhere to remind me of what changes I've made because if this instance is any indication, I'll forget what I did here by tomorrow. Thanks for a truly GREAT backup program!

Alex Pankratov :

Sep 16, 2020

You can have more than one line in override.ini. That is, if you already have one in place, you can just append another line there with a new override.

Glad you like the program, thanks!

lawlorz :

Sep 17, 2020

Hi Alex, got no issues with the instructions, got it working just fine. My question is do you have any recommendations as to what these values should be set at? I understand thats hugely variable depending on whats being backed up. But is there any expectation of files being corrupted after a certain number of delta copies? Ive initally set my backups at 24 copies or time of 1 month. I am not sure if this is ideal as everything I am backing up is backed up a 2nd time to the internet. Lots of potential extra traffic.

Alex Pankratov :

Sep 17, 2020

But is there any expectation of files being corrupted after a certain number of delta copies?


No, not due to delta copying itself. Files may get corrupted in transfer (due to bad RAM somewhere along the line, bug in the drive firmware, etc.) or they may get corrupted due to media failure (in which case a file chunk will become unreadable).

The ageing of delta state is an exotic feature that is not needed in the vast majority of setups, if any at all. We just had an one-off request for it, it was easy to add and it seemed to fit well with the rest of the features, so here it is. But, in my opinion, it's not really needed in practice.

If you are concerned about backup data integrity, it's best to wait for the backup verification support, which is planned for the next release (R82).

New topic

Create
Made by IO Bureau in Switzerland
Support

Updates
Blog / RSS
Follow Twitter
Reddit
Miscellanea Press kit
Testimonials
Company Imprint

Legal Terms
Privacy