The support forum

Including/excluding files and folders

Aug 29, 2017

Overview of the backup process


Every backup run consists of 3 main phases - scanning, planning and execution.

In the scanning phase the program looks at the source and backup locations and builds two lists of files and folders. These are referred to as source and backup "file trees".

In the planning phase the program compares these two trees and compiles a list of simple steps that, when executed in sequence, bring backup location in sync with the source. The steps include basic file system operations like "create directory", "copy file", "move file", etc. The resulting list is referred to as a "backup plan".

Finally, in the execution phase Bvckup 2 merely goes through the backup plan and carries out operations as prescribed by each step.

Exclusion/inclusion filters


Exclusion/inclusion filters (or "rules") are configured in the "Backup What" section of the backup settings and they are used to decide which files or folders should be included in the file tree.

        https://bvckup2.com/support/data/backup-what-window.png

For each item (file or folder) the program first checks if it is explicitly included or excluded in the middle pane. If there's no match, then it goes through the rules from the bottom pane and checks them as well.

If there is NO match, then the item is treated the same way as its parent folder - if the folder is included, the item is in; if the folder is excluded, the item is out.

If there is a match, then the item is included/excluded as prescribed by the rule.

-- And that's it --

The option at the very top of the window merely controls the action assigned to the top location. With "Include everything" it's "include", with "start with an empty list" it's "exclude".

For the details on how *name* matching works in filtering rules see this topic - https://bvckup2.com/support/forum/topic/499/2946

Aug 29, 2017

"Final" rules


There are cases when we exclude/include a folder and we want it to be in/out with all its files and subfolders, all the way down and with no exceptions.

That is, we don't want another rule to override our exclusion, because it just happens to match some file in this folder. For example, we exclude /foo/bar/, but also have an "Include *.jpg" rule that will cause any jpegs in /foo/bar/ to be included.

Enter the "final" rule attribute.

When set to Yes, it suppresses all further filtering of folder contents and causes the folder to be included or excluded in full.

A rule may have this attribute set Yes or No, or not have it set at all. In the latter case, the filtering module uses one of 4 presets, based on what type of rule it is.

        Rules from the middle pane are referred to as "verbose" rules.
        Rules from the bottom pane are referred to as "generic" rules.

The 4 "final" presets are (verbose include), (verbose exclude), (generic include) and (generic exclude) rule types. All of these default to No.

However, there's a special case.

Aug 29, 2017

Filtering behavior in older releases


There are 2 differences to how filtering worked in pre-R78 versions:

1. In the middle pane it was not possible to include items in excluded folders and vice verse (see https://bvckup2.com/wip/12072017).

2. The filtering logic in "Include everything" and "Start with a blank list" modes was slightly different.

In particular, in "Include everything" mode all folder exclusions were final, meaning that the program didn't even bother descending into them during the scan.

And this brings us to...

"Upgrade notice - Legacy mode"


When you upgrade from a pre-R78 version, the program may change the global "final" presets so that the filtering logic remains the same as before.

When this happens, you will see the following in the backup log:

        Upgrade notice -
        Legacy mode is enabled for processing of excluded folders.

This means is that the program changed the default for (verbose excluded) and (generic excluded) rules to be "final".

Aug 29, 2017

Finally


In configuration files the "final" attribute is defined by adding ! for Yes and ? for No to the rule's action (+ or -), like so:

        conf.filters.src.verbose.folder        +!  .\AdvancedInstallers
        conf.filters.src.generic.rule             -?  match \Recycler

The global "final" presets are controlled by:

        conf.filters.src.verbose.include_is_final
        conf.filters.src.verbose.exclude_is_final
        conf.filters.src.generic.include_is_final
        conf.filters.src.generic.exclude_is_final

entries for the source filters and respective keys for the destination filters.

MWorthington :

Sep 09, 2017

Hi Alex,

Nice update!

However, when viewing a selection in "What to Backup", the small tick used on a selected-subfolder (when the parent folder is not selected) is quite hard to see. I'm not sure what to suggest .... maybe a grey, big tick against the parent folder? Or make that small tick red, or blue ...?

Regards,

Mark

MWorthington :

Sep 11, 2017

Alex,

I think it's worth highlighting a significant improvement (I'm not sure if I've missed it in your notes) ....

Up until R78, I could exclude any number of sub-folders for a top-level folder, However, any newly created folders would, by default, be included in a new backup.

With the latest version, I'm in better control. I exclude the top-level and only include the sub-folders I want.

In my case, for the backup strategy I use, that's a BIG improvement, thanks!

Mark

Alex Pankratov :

Sep 11, 2017

the small tick ... is quite hard to see


It's a secondary piece of information, so it doesn't really need to be highly visible. If someone's looking for this information (as in "I wonder if I have any folders excluded below the surface level"), they should be able to find it.

I had another option for this (the one on the right), but I think it takes an effort to notice it and to understand how it works. Though once you know that, it's pretty obvious - https://bvckup2.com/wip/r78-filter-icons.png

With the latest version, I'm in better control. I exclude the top-level and only include the sub-folders I want.


Yep. The new filtering framework is more flexible *and* more uniform at the same time. My main concern with it is that in some cases people might want to exclude a folder, period, and in other cases they may want to exclude everything in a folder by default and then reverse this with wildcard "include" rules. The default is the latter, but we'll have to wait and see if it's a good default or not.


New topic

Create
Made by Pipemetrics in Switzerland
Support


Follow
Twitter
Dev blog
Miscellanea Press resources
Testimonials
On robocopy
Company
Imprint

Legal Terms
Privacy