May 08, 2020
Faster backups via fully parallelized execution.
Release at a glance
⦁ Support for executing multiple backup steps in parallel (!)
⦁ Support for waking up computer to run scheduled backups
⦁ Support for password-protecting UI access in service mode
⦁ Support for replicating case changes in file and folder names
⦁ Improved delta copying performance with mid-sized files
⦁ Improved IO configuration defaults for bulk copying
⦁ Improved progress reporting for longer copies
⦁ Improved log indexing performance
May 08, 2020
Big, new and very important feature in Release 81 is an option for executing multiple backup steps at once.
When enabled, it helps speeding up backups, with the effect especially pronounced when backing up over network and when working with smaller files.
First prototyped in 2016, then revised a number of times and finally arriving at stable beta in last November, it's been a long time coming.
This is a Pro feature. It is enabled by default for all backups and it is configurable in Backup Settings -
There is a new accompanying KB article that discusses this feature in more detail and explains the context of its advanced options.
The feature is roughly similar to the "robocopy /mt" option, but with a more nuanced implementation that provides for better performance and run-time profile.
See the KB article for details.
May 08, 2020
Waking up computer to run scheduled backups
When enabled, this option will cause Bvckup 2 to ask Windows to wake up computer around the time of the next scheduled backup run. If no user interaction is registered while the backup is running, Windows will put it back to sleep once the backup is done.
It's a program-wide option that can also be set on per-backup basis with an override.
For a bit more detailed description see the following blog post -
May 08, 2020
Password-protecting UI access in service mode
With Release 81 it's now possible to have the UI prompt for the password when it is being started up in service mode. This option is primarily meant for preventing non-admin local users on the host machine from opening the UI and messing with the setup.
See the above link for details and caveats.
May 08, 2020
Replicating case changes in file and folder names
As of Release 81 the backup planner will now check the case of file and folder names and correct any differences as required.
It will check if the file system on the backup side supports case preservation to begin with and suppress renames if it doesn't.
As per usual, this behavior can be suppressed with an override if needed.
May 08, 2020
⦁ Improved delta copying performance with mid-sized files - this has to do with how block hashing is distributed between the CPUs, which helps completing hashing faster for smaller files.
⦁ Improved IO configuration defaults for bulk copying - revised default IO buffer counts and sizes based on things learned in past several months.
⦁ Improved progress reporting for longer copies - this has to do with the UI feature of displaying additional decimal digits in %-age counter when the copy is going slow. Previously, the decimal precision was not reset when the copy was completed, so if the next copy was fast, it ended up showing a %-age counter that showed a frenzy of activity. No more of this.
⦁ Improved log indexing performance - this has to do with how the logging system works in the program. The backup engine is a log writer and the UI is a log reader. Logs can grow big, so the UI builds a separate index in order to be able to display them quickly. Index building involves parsing logs, line by line. This release reworks how this parsing is done, speeding it up by a factor of 300.
See here for technical details -
May 09, 2020
⦁ Resolved an issue with the planning module not being able to produce correct backup plan when multiple files from different levels of a folder were moved to a new directory and the folder itself was deleted. In certain cases the resulting plan would have two steps in the wrong order, which was caught during the plan validation phase, which in turn triggered an abort.
May 15, 2020
⦁ Resolved an issue with delta copying zero-sized files. More specifically - files that show up as large during the scan but then end up being empty when the program gets to actually backing them up.
⦁ Added an Easter egg... o_O
Jun 05, 2020
⦁ Reworked retrying logic to act on errors that occur when processing item attributes. That is, the program will now retry folder creation, file/folder move and file copying steps if they fail with a transient error when cloning item's meta data (attributes, timestamps, security info, etc.)
⦁ Reworked retrying logic to accommodate steps failing with 2+ errors.
⦁ Reworked backup summary wording to be more precise in presence of errors and retries. See the above link for details.
⦁ Reworked logging to preserve original IDs of steps when they are retried. This is a purely cosmetic fix. Previously, when a backup step was retried, it would appear in the log under a new step ID. Now the ID is preserved, so it's easier to find all retries of the same operation.
⦁ Reworked name case changing logic to handle certain case better.
More specifically, if both file's and its parent folder's names need a case change (on the backup side), then in some cases these two changes will be made _exactly_ in parallel, resulting in some rather obscure errors (e.g. "the file in use" reported for the folder rename). The fix was to sequence these operations, so that one is executed after the other.
Kudos to Mike for reporting this.
⦁ Reworked auto-update option to not require a production license. Previously, when the program is set to auto-update itself, it wouldn't do that if there was no valid license. For the life of me I can't recall why it was done this way, but regardless of that this restriction is now removed.
⦁ Fixed an issue on Windows XP with handling backup cancellation. This had to do with certain Windows function being not available on XP and re-implemented using another, XP-specific API, which in turn appears to have been documented incorrectly, so things weren't working as designed.
And, yes, someone did actually run into this issue, so they ARE still on XP.
This is 81.3 with a small patch for an one-off issue with the former. If you run into 81.3 reporting "Something went wrong", go Help > Menu > [Check for updates] and update to this release.
Jun 17, 2020
⦁ Reworked code in several places for clarity and speed. This is a purely internal rework. It has to do with rolling out new IPC (inter-process communication) message packing  and a revised version of list/tree containers .
⦁ Fixed an issue with not recognizing some USB disks as removable. Disks that Windows reported as "fixed" but with a "surprise" removal policy weren't treated as removable, and backups going to/from these disks did NOT get device tracking automatically enabled for them.
Jul 20, 2020
⦁ Fixed an issue with monthly backup scheduling. One part of the backup scheduling code was erroneously using an _average_ month length (of 30 days) when calculating the date of a last missed backup run. In particular, the issue was triggered by a July-August combo, because both months are 31 days in length. This led to the date being one day off and that tripped an internal consistency check further down the road.
⦁ Added an option for suppressing the "live file" warning. This has to do with a warning added to R81 series, whereby the file copying module will check if the source file size changed during the copying and log a message to that effect and some suggestions. This release includes an override for suppressing this warning, because there are setups where backing up "live" files is a routine.
Aug 07, 2020
⦁ Fixed handling of corrupted files in the copying module.
This covers a particularly exotic case when a backup copy gets corrupted (at rest!) in a way that still allows extending it, but prevents it to be trimmed to the exact size. Basically, we would append, say, 1 MB to such file and then ask it to be trimmed to exactly that new size - and the latter request will fail. Copying module didn't handle it terribly well because of an overly strict internal check. Removed the check, solved the problem. The end.
⦁ Fixed handling of certain exclude/include rule combos - this has to do with https://bvckup2.com/support/forum/topic/499/7211
Kudos to @highend for the report.
⦁ Fixed an issue with the file tree browser.
When the Backup Settings > Backup What window is opened, the file/folder selection widget starts populating the tree. It's a process that may take a while and it runs in the background. The issue was that if the window is OK-closed while the tree wasn't fully populated, any existing excl/incl rules that applied to not-yet-discovered items were discarded.
Kudos to Eden for the report.
⦁ Improved file tree browser rendering performance - tangentially related to the previous item, basically improved the rendering speed of the widget by at least a factor of 2. Beware of the temptation to use WM_SETREDRAW, use with utmost care and only after due testing.
Also added an explicit indicator of whether the background scan is still in progress or not - https://bvckup2.com/wip/r81-6-scanning-spinner.gif
⦁ Revised symlink verification logic in the scanning module.
In later 79 releases the scanning module got a new check, whereby it would query the attributes of a location that a symlink is pointing at and would complain and ignore the symlink if the query fails. Now, as you can probably guess, in some cases this check misfired. More specifically, it will fail when backing up from a remote share and the symlink points at a location that is local to the remote machine.
Kudos to Milan for the report.
Same as 81.6 + a small patch for an issue when switching between "Start with an empty list" and "Include everything" options in "Backup What" window for _existing_ backups.
Sep 07, 2020
⦁ Added an option for discarding delta state based on its age or usage. It is now possible to have files re-copied in full after they were delta-copied either a certain number of times or for a certain amount of time.
details and https://bvckup2.com/wip/06092020
⦁ Fixed an issue with bi-monthly backup scheduling.
⦁ Fixed a couple of typos in backup log messages.
Sep 22, 2020
⦁ Revised and improved the command-line control interface. It's now possible to specify backup jobs using the name of their queue, and it's now possible to apply additional commands (simulate, delete, unload, reload) to several jobs at once.
⦁ Added an option for refreshing Personal and Basic licenses. This is to support the case when a Pro license upgrade is initiated through the web interface rather than from within the program.
Oct 16, 2020
⦁ Added preliminary support for Blake3 and xxHash3 hashers - these can now be optionally used with delta copying, and they will become the new defaults with R82. Current defaults are the original xxHash and Blake2b , which are already fast, but still several times slower than the new variants.
⦁ Fixed an issue with delta copying stats reporting - in some cases the program would erroneously report in the log that a file was copied in full whereby it was delta-updated.
⦁ Fixed an issue with delta state saving in presence of errors - the program would trigger a self-consistency check and shut down when it would run into a certain type of error while attempting to start delta copying of a file. This is an exotic case.
⦁ Fixed an issue with the mapped drive monitoring - the module that periodically checks the status of all mapped drives (with the goal of proactively detecting them going offline) wasn't starting up correctly in some cases.
Same as 81.9 + a patch for the new CPU detection code that used xgetbv instruction without first checking that the CPU supports it.
Nov 24, 2020
⦁ Internal rework of the code responsible for text string manipulation, self-checks (asserts), generic data containers and CPU feature detection.
As a side effect of this, finally got around to counting the number of self-checks that ship with every release. There are 2184 of them in 81.10, spread over 204,176 lines of code that comprise the full source of the program.
⦁ Added detection of ECC vs non-ECC RAM on the host machine. This information is meant to assist with analyzing and confirming crashes, caused by spontaneous memory bit flips.
⦁ Cosmetic cleanup of the logs, including NT API error reporting.
Jan 11, 2021
⦁ Fixed an issue with "staged" file copying - this has to do with copying files _without_ the use of the delta copier, in full.
When a file are copied in full, the program first copies it into a temporary file on the backup side and then renames this temp file to match the source file. This is referred to as "staged" copying and it helps preserving an older backup copy in case of abortive mid-backup failures.
The problem was that the program was looking at source and backup copies to decide which meta data (attributes, timestamps, etc.) should be cloned after the contents are updated. So if there _was_ an existing backup copy and its "created" timestamp matched that of the source, the program, being a pedantic little construct that it is, would skip updating this timestamp.
However with staged copying the backup copy is always created from scratch, so it needed ALL meta data to be cloned regardless of what the existing backup copy has.
So what ended up happening is that if a file was modified at source, then the next backup run would create a temp file, copy the contents and "last modified" timestamp, but the "created" time on that temp file would remain set to "now". On the next run, the program would notice that "created" timestamps are now out of sync and would re-update the file. This time it would sync this timestamp and put everything in a proper order.
That is, when files were copied in full, modified files were needlessly backed up for second time.
Kudos to Tom for tracking this down and not giving up on the issue.
⦁ Fixed an issue with service/desktop mode switching - in some cases if the program ran into a problem switching the mode, it would report it as a "cancelled" rather than "failed", further confusing the situation that weren't already going that well.
⦁ Revised 'created' timestamp replication logic - for the cases with a read-only source volume the program will now try and copy "created" timestamps.
This has to do with the fact that not all file systems support these timestamps and the program relies on active tests to determine what the case is. These tests can't be run if the volume is read-only, so previously this caused the program to assume that "created" timestamps are unsupported by the volume.
This nonsense is no more. From now on, if we can't determine if X is supported, then it is "unknown" and not "unsupported" => the new and improved handling of the timestamp replication.
⦁ Humanized error 433 - a smaller change to log a better message when running into this error, which is ERROR_NO_SUCH_DEVICE.
⦁ Extended list of default exclusion rules - added ".dropbox.cache" and "Microsoft\\Edge\\User Data\\*\\*\\*Cache*" to the list of default exclude rules.
⦁ Another round of internal code rework. It is even more beautiful than before, which is always a nice thing to have.
Mar 09, 2021
⦁ Fixed an issue with the archival of modified files.
If the archival of modified files was enabled and a file was both touched and had its name _case_ changed, then the program would sometimes abort with a failed internal check.
For a change, it wasn't actually a problem with the check itself (as it might or might have not happened on some prior occasions), but a genuine flaw in the planning module logic. Resolved now.
⦁ Fixed an issue with the exclusion/inclusion rules dialog.
This has to do with the "Backup What" dialog. More specifically, with the widget that allows selecting files/folders to be included in (or excluded from) the backup. This happens to be one particularly complicated piece code due to how it populates itself based on the results of a background file system scan.
The problem was that if you were to make changes to the selection and OK them _while the scan in still in progress_, then, in certain cases, this confused the scanner and led to the program panic. The good news is that not only the program doesn't panic anymore, the issue has been properly resolved too. Imagine that.
⦁ Added support for auto-pruning obsolete exclusion/inclusion rules.
Starting with this release, when the "Backup What" window is closed with OK, the program will check if any of the previously made exclusions or inclusions are no longer valid. That is, if they now refer to files/folders that no longer exists. When this happens, the UI will pop up a notification like so to celebrate the occasion -
All further celebration of the same kind can be suppressed with a handy "Don't ask again" option.
⦁ Added support for a new variation of %RUN...% variable.
This is an extension of the "%RUN%" variable used for dynamically-adjusted backup paths. It is set to the current backup run number, starting with 1 and incrementing with every successful run.
It can also be made to cycle through a predefined range using "%RUN#123" notation, whereby 123 is the range length. In this example the variable will evaluate to 1, 2, 3, ... 123, 1, 2, 3, ... etc.
This release adds a new notation - "%RUN|123%" - with the pipe - which produces 0-padded values - 001, 002, 003, ... 123, 001, ... This makes for folder names that have a better natural order.
⦁ Added support for selecting all idle backups in one click.
It is now possible to select all _idle_ backups by Ctrl-Shift-Left-clicking on the empty space to the right of the Go/Stop buttons. This extends existing behavior, where Ctrl-Left-click selects all jobs and Left-click clears the selection.
⦁ Speeded up character case conversion code.
This is a change to one of the core libraries. It improves the speed of conversion between upper and lower cases of Unicode characters, a fairly basic operation that is very frequently used when doing
case-insensitive comparison, searching, keying, etc.
May not sound like much, but it's a very nice little rabbit hole that happens to be closely related to the Traveling Salesman Problem, one of fundamental algorithmic problems in computer science. So if you need something to read over your next coffee break, here, skim through this -
A patch for the case when an attempt to set up real-time folder monitoring fails in a certain way. If you run into this problem, just restart the program and it should pop up an update prompt.
Mar 15, 2021
⦁ Fixed an issue with variables not getting expanded in email alerts.
The issue was introduced in 81.12.
Topic is locked.