R81 pre-release / Parallel execution
Nov 04, 2019
The central new feature in Release 81 is its support for parallel backup execution - an ability to execute multiple backup steps at once.
Prior to R81, the file system scan and file ID retrieval were already done
in a parallel fashion, using several threads.
Release 81 extends this behavior to all phases of backup, allowing the program to execute all suitable operations in parallel with each other.
If you are familiar with the /mt switch of robocopy, it's a similar idea, but with automatic thread count management and several other important improvements.
In practical terms, parallel execution results in a *very* noticeable speed increase in the following cases:
⦁ Backups with a lot of small- to medium-sized files
⦁ Backups that require creating a large number of folder
⦁ Backups that require deleting a large number of folders or files
⦁ Backups that "touch" a lot of folders
For a bit of anecdotal data see here - https://bvckup2.com/wip/03102019
Parallel execution is enabled by default with the thread count set to the number of CPUs for local backups and to the double of that for network backups.
To set the thread count to some fixed custom value use the following override -
for how to do this.
Pre-release 18.104.22.168 and newer.
Nov 11, 2019
⦁ Fixed an issue with handling failures when _retrying_ a backup of a
More specifically, if a new file appears at source, the backup is run and
it fails with a retryable error mid-copying. For example, a network
connection goes away. This will cause the program to retry the copying
after a pause. If this retry would also fail with an error, then bvckup2
would pop up the "something is wrong" message and shut down.
Nov 13, 2019
⦁ Fixed an issue with the retrying mechanism panicking when the last
attempt fails. There was an invalid self-consistency check (an assert),
which is as ironic as it gets. A bug in a bug checking code.
Nov 14, 2019
⦁ Fixed a crashing issue with the archive trimming code that surfaced
every time an archive scan would identify and remove expired items.
Dec 03, 2019
⦁ Reworked retrying logic that is used to re-execute failed steps in a
presence of transient errors (e.g. network disconnects). This also
fixed an issue that caused self-inflicted crashes in cases when
copying of a _new_ file succeeded after several attempts.
⦁ Reworked logging module to report step info as it's being executed
IF this step happens to be the only one active and no other steps are
being scheduled until it completes.
This happens when bvckup2 is backing up a very large file, at which
point it will finish all other active steps, but won't queue any new ones.
This is done to allow the bulk copying to proceed in peace if you will.
Prior to 99.5 all step-specific info was recorded into a backup log only
after the step finished. However if there's exactly one step in progress
we can dump its progress into log as we go - which is exactly what 99.5
Dec 05, 2019
⦁ Fixed an invalid self-consistency check in the logging module that
caused 99.5 to shut down with "Something happened" message.
This was caused by change #2 in 99.5.
Topic is locked.