The support forum

Release 74

Jun 05, 2015

Run-time memory use is now capped


The amount of memory the app now uses to run a backup does NOT depend on the size of said backup.

Needless to say that this is extremely important change that means that Bvckup 2 is now formally capable of handling backups of arbitrary size - anything from a handful of files to millions of files.

Behind the scenes the app uses a form of a swap file to keep things off RAM when it's not actively using them. For the details, pretty graphs and diagrams see here - https://bvckup2.com/wip/27052015

  --  The Fine Print  --

This change is linked to a major rework of several parts of the app that spanned nearly 3 months. In practical terms this means that there's an increased risk of edge cases, bugs and wrinkles slipping through the testing and ending up in the release. The risk is low, but still please keep an eye on anything odd and report to the support if spotted.

Delta copying improvements


Previously, delta copying was used for files of 2MB and larger.

With this release the threshold is raised to 32MB, but the delta copying is also used for files between 2 and 32MB that were modified less than 30 days ago. The idea, basically, is that delta copying now used for files that are either large or that change frequently.

The net effect of this change that the backups complete faster *and* that there's less state date stored in the backup configuration.

For the details have a look here - https://bvckup2.com/wip/04062015

"Export configuration" option


Just what it says on the label - an option to save app's entire configuration in a single .zip file, accessible from Menu / Help.

https://bvckup2.com/wip/r74-export-configuration.png

Restoring the configuration is a simple matter of extracting the contents of the zip file into %LocalAppData%\Bvckup2 folder, but just in case the exact instructions are included in the .zip file itself.

For details see here - https://bvckup2.com/wip/03062015

Alex Pankratov :

Jun 05, 2015


Smaller changes


1. Source or destination location disappearing during the scan now correctly aborts the backup

2. It's now possible to suppress "Backup run was missed because the app wasn't running" notifications. This is done by setting "conf.report_missed_runs" to 0 in the backup's settings.ini. See FAQ for how to make this change properly.

**  As of Release 76 this setting is deprecated in favor of
**  
**      conf.missed_run.register_on_launch    0
**      conf.missed_run.register_on_wakeup  0
**  
**  which will force the engine to ignore missed runs on program's
**  launch and computer's waking up from the sleep respectively.

3. Tentative workaround for log index files getting locked by the antivirus when the log is rotated.

4. It's now possible to force the app ignore specific drive letters (just as if they have no devices behind them). This is controlled by "drives_to_ignore" variable in bvckup2-engine.ini file, which is a bit mask of drives (bit 0 - drive A, bit 1 - drive B, etc.).

5. \Windows\System32\Microsoft\Protect\Recovery is now excluded by default

6. UI-to-service connection timeout is now configurable and the default was raised from 5 to 7 seconds. Because reasons.

soylent :

Jun 05, 2015

When will it be available? The check for updates is still not showing it.

Alex Pankratov :

Jun 05, 2015

@soylent - I was getting to that :)

We are doing a soft roll-out, so at the moment the release is available via https:\\bvckup2.com\update and you'll need to download and run it manually.

It will also start trickling out through the update server and it should be visible to everyone by Monday.

soylent :

Jun 05, 2015

Sorry! I didn't know it was a multi-part post :)

Alex Pankratov :

Jun 09, 2015

New revision is out - R74.3 - please update.

Several issues have been identified and resolved.
  --
One was with the coding error with modified objects [1] in the cache sometimes not getting tagged as such, causing the changes to get lost when an object was swapped out and then loaded back into the memory.
  --
Another was the scalability issue with the move/rename detection, whereby it would take a lot of time to process the changes if there were a lot of new *and* some deleted files.

Deipotent :

Jun 09, 2015

Run-time memory use is now capped

Kudos on all the effort that went into this, and the results, but I'm curious why you didn't use an embedded database like SQLite or Firebird ?

Alex Pankratov :

Jun 10, 2015


Mostly because of the performance and the code size considerations.

Both SQLite and Firebird are generic storage facilities and as such they come with lots of features that won't be used. They can be trimmed off, of course, but then it becomes a question of whether we should spend time on learning/customizing/optimizing 3rd party code or if we should roll out an in-house replacement that we know inside-out.

I looked at SQLite and several other key-value storage libraries. I also looked at using memory-mapped files with custom allocators, and even at explicit virtual memory management. Ultimately, the choice was between the convenience of writing the code vs. the cost of the scaffolding that delivers this convenience. For example, with mem-mapped files and custom allocators the scaffolding is cheap, but the convenience is really quite bad. With SQLite - it's the reverse. So the custom solution gives you not only the full control over the code, it also allows you to pick and choose this balance. What's there not to like :)

Alex Pankratov :

Jun 11, 2015

New revision is out - R74.5 - please update.

This resolves an issue with so-called "staged" (or two-step) moves. It's an exotic case when we have a file (say, "foo") and a folder (say, "bar"), then we move "bar" into "foo" *and* also rename foo itself into "bar". In any case, there was a technical issue with reference counting and the app tripped one of the self-consistency checks. No more.

Further reduced memory usage during the prep stage. This won't be noticeable for backups smaller than few million items, but for those - it does make a difference.

Also, for real-time backups going *from* network drives - the app now monitors for another type of error that might be indicative of the connectivity issues with the drive and, upon detection, it will now re-check the drive and re-establish the monitoring.

Alex Pankratov :

Jun 12, 2015

New revision is out - R74.6 - please update.

Resolved an issue with archival of empty folders. Two of the issues actually. One was that top-level empty folders weren't moved to the archive, but rather deleted. And another was that deleting a folder with an empty sub-folder wasn't correctly handled by the new planning module.

Also added a couple of UI hints when the app fails to make a volume snapshot for the shadow copying and it happens because either the "vss" or "swprv" service is disabled. So now there's a hint logged that suggests enabling respective service.

Alex Pankratov :

Jun 15, 2015

New revision is out - R74.7 - please update.

Resolved an issue with wrong time zone getting written into the Date field in email alerts. As of this release, the Date is always expressed in UTC, so the time zone offset is simply 0.

Resolved an issue with the archive trimming logic - prior to this release *empty* archived folders might've been trimmed from the archive prematurely (that is, before the end of the archive grace period).

Alex Pankratov :

Jun 18, 2015

New revision is out - R74.8 - updating is optional.

This update fixes an obscure issue with how Windows file stack does the Unicode file encoding.

Details are here - https://bvckup2.com/wip/18062015

Alex Pankratov :

Jun 20, 2015

New revision is out - R74.10 - update is recommended.

This update addresses several issues with handling of massively populated directories (in tune of 100,000 files per folder).

Alex Pankratov :

Jul 07, 2015

New revision is out - R74.11 - please update.

A handful of assorted tweaks, fixes and smaller changes:

1. Added "Help/Feedback" option to the menu. Feedback is great. Can't never have too much of it.

2. Malformed exclusion/inclusion filters are now reported in the backup log

3. "Wake-up delay" is now observed for resuming, in addition to the reboots and de-hibernation.

4. "Missed a run" warning is now suppressed if it's because the app wasn't running.

5. Reworked messaging around the case of launching app more than once in rapid succession (i.e. if it was linked from both user's and system's Startup folders). The original warning was very cryptic and not exactly correct.

6. Added support for NFR ("not for resale" / demo) distributor licenses.

7. Fixed an issue with self-backup option. bvckup2.ini is now backed up if the backup copy doesn't exist.

8. Archive location (conf.arc_path) can be an absolute path. Previously it was always relative to the destination folder.

9. Fixed a smaller issue with the log rotation.

10. "Mozilla\Firefox\Profiles\*\Cache*" is a new default exclusion filter

Alex Pankratov :

Jul 17, 2015

New revision is out - R74.12 - please update.

Two fixes for the issues with backup archive trimming  -

1. Some folders that had a . in their name weren't picked for trimming.
2. In certain cases the trimming wasn't picking up any items at all.

Two new things -

1. It's now possible to tell the app to adjust its process priority. This is done by adding "proc_priority" setting to %LocalAppData%\Bvckup2\bvckup2.ini file, e.g.

        proc_priority    -2

will lower process priority to the IDLE level. Similarly, "-1" will set it to BELOW_NORMAL, "1" to ABOVE_NORMAL and "2" to HIGH. Values below "-2" are treated as "-2" and above "2" - as "2".

Needless to say, make sure the app is not running when changing the .ini

2. A backup job can now be set to ignore empty folders at source location. This is enabled by setting "conf.scanning.skip_empty_folders" line in job's settings.ini to "1", i.e.

        conf.scanning.skip_empty_folders        1

When this feature is enabled, the engine first scans source location as usual (applying configured exclusion/inclusion filters) and then it traverses resulting tree snapshot and trims all folders that are empty. This works recursively, starting with the deepest folders first, so for a setup like this -

            \not-quite-empty\
                    \empty\                                     <- empty
                    \sort-of-empty\
                          \foo\                                     <- empty
                          \bar\                                     <- empty

all five folders will end up getting trimmed.

Again, if you decide to make use of this feature, make sure the app's not running when you change settings.ini (or use alternative way - through override.ini). See FAQ for details.

Alex Pankratov :

Aug 10, 2015

New revision is out - R74.13 - update's optional.

1. Resolved an issue with symbolic link processing - in certain convoluted cases when multiple level of symlinked locations are present, the app wasn't able to process them properly. As of this release it will log additional details about such cases and just complain loudly in the log.

2. Added an option of matching files and folders based on their timestamps. Also reworked the filtering rule format that's used in settings.ini so that it can be further expanded with other match criteria if needed. As of now item can be matched on their name, type (file/folder), attributes (both set and cleared), size, created and modified timestamps (both in absolute "2015-08-10 12:34:56" and relative "now - 3 h" formats).

Full details are here - https://bvckup2.com/support/forum/topic/632

3. The following four locations are now excluded by default (this applies only to *new app installations*) -

     ProgramData\Microsoft\Diagnosis
     Microsoft\Internet Explorer\Recovery
     Microsoft\Windows\INetCache
     Microsoft\Windows\Explorer\IconCacheToDelete

Alex Pankratov :

Aug 14, 2015

New revision is out - R74.14 - update's optional.

1. Resolved an issue with "created" timestamp handling - prior to this release if created timestamps weren't supported at *source*, the app would still copy and compare them, and in certain cases this led to files getting re-copied on every run.

Note that the lack of "created" timestamp support at *destination* has been around since the early betas.

2. Added BVCKUP_type and BVCKUP_start variables to the environment set up for pre/post-backup commands. Former indicates the backup type and it's set to "Manual", "Periodic" or "Real-time". Latter is set to either "Manual" or "Automatic" and it indicates if the job was run manually or if it was triggered automatically.

3. Added W2003-specific warning to the installer to identify Microsoft hotfix that needs to be installed if the machine lacks support for SHA2.

Alex Pankratov :

Aug 18, 2015

New revision is out - R74.15 - update's optional.

1. Restyled delta copying progress bar for a bit more contrast -
https://bvckup2.com/wip/r74.15-progress-bar.png

2. File system parameters (such as timestamp resolution) are now re-queried when the volume serial number changes as it typically indicates a drive reformat.

3. Deprecated "staged moving" operation in backup planner - this one of more exotic planner arrangements when a file had its name changed to a name of a folder that got deleted, whereby file's own folder too was getting deleted. To wrap this into a move operation the planner would set up a temporary "staging" folder, move file there, delete folders and then finally un-stage the file to where it should be.

Ultimately this was way to clever with a bit too much complexity added to the code and without that much of benefit. So - no more.

4. Switched to "Extended validation" code signing certificate for all executable signing needs. This should be entirely transparent.

Alex Pankratov :

Sep 01, 2015

New version is out - R74.16 - update's recommended.

1. Files are now read in "backup semantics" mode, if possible - as per https://bvckup2.com/support/forum/topic/655

2. Added support for "sparse reparse point" file system entries - as per
https://bvckup2.com/support/forum/topic/652

3. Maximized window state is now correctly restored in all cases - in particular, when "start minimized" was enabled and the app exited with its window maximized, then upon launch the app will not re-maximize window when it needed to be shown. Fixed now.

4, "Start at logon" in admin mode now corrects process priority - in this case the app is launched using an at-logon task in Task Scheduler. It turns out that TS launches tasks at "lower" process priority by default. This release corrects this by explicitly setting the priority for the task, and it also respects the priority override if there's one configured (as per https://www.bvckup2.com/support/forum/topic/598/3537 above).

If you are running in the admin mode and using "start at logon", then you will need to go into Preferences, uncheck the option, OK the changes, go into Preferences again, check the option back and again OK the changes. This will re-create the TS task with proper priority written into it.

5. Resolved an issue with DNS resolver in email alerts module - a pro-active fix, just something picked up through the code analysis.

6. Resolved an issue with an update module - same here.

Alex Pankratov :

Sep 10, 2015

New version is out - R74.17 - update's optional.

1. Resolved an issue with "backup semantics" mode - as per https://www.bvckup2.com/support/forum/topic/655/3702

2. Slightly reduced per-file copying overhead - when copying file attributes and other meta information, the app now does it in fewer system calls than before.

3. Tweaked how "Go" works with multiple job selection - previously, clicking on Go with multiple jobs selected could yield different commands applied to the jobs if those were in different states. E.g. if one job was Paused and another Queued, then Go would resume former and force-start latter.

As of this release each click on Go executes exactly one kind of command applied to relevant jobs only. The first choice is Enable, then Start/Queue, then Resume, then Force-Start.

In the above example, first click on Go would resume first job, but do nothing to the queued one, which will be force-started with the second Go click.

This is now also consistent with how Stop works - it first takes care of Pause, then Cancel, then Disable.

Alex Pankratov :

Sep 15, 2015

New version is out - R74.18 - update's optional.

1. Resolved an issue with updating folder timestamps on QNAP NAS - the actual change was for the app to request FILE_READ/WRITE_ATTRIBUTES permissions when opening folders instead of GENERIC_READ/WRITE.

As they say - in theory there's no different between theory and practice, in practice through there is. So this is an example of that.

Original thread - https://bvckup2.com/support/forum/topic/660

2. Resolved an issue with querying of *destination* file system information - prior to this release the app would look at the cache of the *source* filesys info when deciding if the destination one needs a refresh. That was a bit of apples to oranges logic to put it mildly.

Original thread - https://bvckup2.com/support/forum/topic/658

3. Added an option for using so-called write-through disk writing mode.  This a tentative workaround for this issue - https://bvckup2.com/support/forum/topic/661

Alex Pankratov :

Sep 17, 2015

New version is out - R74.19 - update's optional.

A single very small change - when cloning security information (owner/group/DACL) of folders or *moved* files, the app wasn't requesting enough access rights from Windows. Oddly enough, Windows was OK with that for locations under its own control. but it did yield "access denied" when doing the same against a Synology NAS. No more, proper rights are requested now.

In other words, if you are (1) copying security info (2) *from* a Synology NAS, then this update is for you. Otherwise it's as optional as it gets, so it will take few days for show up as an in-app update notification.

berra.lindstrom :

Oct 01, 2015

I got this (R74.19 ):

Updating: 'modified' time, 'accessed' time
Copying file details...
Error: access denied
Function: CreateFile
Bwana2\MSOCache\All Users\{90140000-0019-041D-0000-0000000FF1CE}-C 00000100

Alex Pankratov :

Oct 01, 2015

Drop me an email at support@pipemetrics.com. You can get this in certain legitimate cases, e.g. if you copied file owner information across the machine boundary.

Alex Pankratov :

Oct 05, 2015

New version is out - R74.20 - update's optional.

1. Added an option for suppressing systray alerts -- it's now possible to suppress failed backup run from triggering a systray alert. This change is provisional, more like a temporary kludge for certain cases. If you need to use it *now*, drop me an email. Otherwise, it will be done properly in R75.

2. Added "is-idle" query to the command-line control, and also added support for *  as a job specifier for all other commands. Latter allows launching all idle backup jobs at once (with --command start * ), and former allows checking if the app is busy backing up or if it's idle.

I'll be updating primary topic in a moment - https://bvckup2.com/support/forum/topic/411

3. Resolved an issue with attribute copying in the planner -- previously if item attributes didn't match between source and destination, the planning module would always schedule an attribute update *even if these attributes weren't supported at destination*. No more, fixed.

4. Resolved a cosmetic issue with the backup logging -- there was an extraneous line-break in shadow copying related log entries.

Alex Pankratov :

Oct 20, 2015

New version is out - R74.21 - update is recommended.

1. Resolved an issue with 12-month backups set for December - a fix for https://bvckup2.com/support/forum/topic/436/3807

2. Resolved an issue with .ini overrides for archive tags - this has to do with manually blanking conf.archive_deleted_tag not being sticky. That is when you set it to "", it would revert back to the default on the next backup configuration change.

3. Extended environment variable set to cover pre/post commands - 8 additional env. variables are now set up when pre/post commands are spawned and when email alerts are formatted.  In particular,

    BVCKUP_pre_command   --   the actual pre command
    BVCKUP_pre_command_status  --  "executing", "executed", "timed out"
    BVCKUP_pre_command_exit_code  --  the exit code once executed
    BVCKUP_pre_command_elapsed  --  time taken to execute

Post command has the same set with BVCKUP_post_...

Alex Pankratov :

Nov 04, 2015

New version is out - R74.22 - update is recommended.

1. Additional details are now logged for files being updated - previously the app logged timestamps and the file size, now it also logs the attributes as a change in them may also trigger a file update. The logging is now also more compact.

    https://bvckup2.com/wip/r74.22-file-details.png

2. Resolved an issue with inaccessible src/backup locations - previously the app would check if source and destination locations *exist* and abort the backup if they weren't. With this release the app now also checks if these locations are *accessible*, and it will abort the backup if they aren't.

Admittedly, this is a pretty marginal case, but it still managed to surface - https://bvckup2.com/support/forum/topic/684 - so here's a fix for it.

3. Resolved an issue with how backup settings are stored - this is a minor issue with how conf.copy_extras were stored in- and read back from backup's settings.ini. Saving an empty string (meaning "don't copy anything but the data and modified timestamp") would result the value to go back to its full default on app's restart.

4. Resolved two smaller issues with env. variables setup - BVCKUP_app_mode is now correctly set to "Service" when the app is in the service mode; BVCKUP_app_version now includes the revision number, e.g. "Release 74.22" rather than just "Release 74".

Logician :

Mar 18, 2016

Hi Alex,

"proc_priority    -2" setting is a gem! May I suggest making it more prominent in help docs etc.

I'm running my backsup very frequently (once an hour) and it was causing bad performance issues when they were running (eg. mail program freeze etc.).  This setting cured all my issues.

Alex Pankratov :

Mar 19, 2016

Thanks for the idea. I'll add it to the preferences in the next release.

New topic

Create
Made by IO Bureau in Switzerland
Support

Updates
Blog / RSS
Follow Twitter
Reddit
Miscellanea Press kit
Testimonials
Company Imprint

Legal Terms
Privacy