Including/excluding files and folders
Aug 29, 2017
Overview of the backup process
Every backup run consists of 3 main phases - scanning, planning and execution.
In the scanning phase the program looks at the source and backup locations and builds two lists of files and folders. These are referred to as source and backup "file trees".
In the planning phase the program compares these two trees and compiles a list of simple steps that, when executed in sequence, bring backup location in sync with the source. The steps include basic file system operations like "create directory", "copy file", "move file", etc. The resulting list is referred to as a "backup plan".
Finally, in the execution phase Bvckup 2 merely goes through the backup plan and carries out operations as prescribed by each step.
Exclusion/inclusion filters
Exclusion/inclusion filters (or "rules") are configured in the "Backup What" section of the backup settings and they are used to decide which files or folders should be included in the file tree.
https://bvckup2.com/support/data/backup-what-window.png
For each item (file or folder) the program first checks if it is explicitly included or excluded in the middle pane. If there's no match, then it goes through the rules from the bottom pane and checks them as well.
If there is NO match, then the item is treated the same way as its parent folder - if the folder is included, the item is in; if the folder is excluded, the item is out.
If there is a match, then the item is included/excluded as prescribed by the rule.
-- And that's it --
The option at the very top of the window merely controls the action assigned to the top location. With "Include everything" it's "include", with "start with an empty list" it's "exclude".
For the details on how *name* matching works in filtering rules see this topic -
https://bvckup2.com/support/forum/topic/499/2946
1
Aug 29, 2017
"Final" rules
There are cases when we exclude/include a folder and we want it to be in/out with all its files and subfolders, all the way down and with no exceptions.
That is, we don't want another rule to override our exclusion, because it just happens to match some file in this folder. For example, we exclude /foo/bar/, but also have an "Include *.jpg" rule that will cause any jpegs in /foo/bar/ to be included.
Enter the "final" rule attribute.
When set to Yes, it suppresses all further filtering of folder contents and causes the folder to be included or excluded in full.
A rule may have this attribute set Yes or No, or not have it set at all. In the latter case, the filtering module uses one of 4 presets, based on what type of rule it is.
Rules from the middle pane are referred to as "verbose" rules.
Rules from the bottom pane are referred to as "generic" rules.
The 4 "final" presets are (verbose include), (verbose exclude), (generic include) and (generic exclude) rule types. All of these default to No.
However, there's a special case.
2
Aug 29, 2017
Filtering behavior in older releases
There are 2 differences to how filtering worked in pre-R78 versions:
1. In the middle pane it was not possible to include items in excluded folders and vice verse (see
https://bvckup2.com/wip/12072017).
2. The filtering logic in "Include everything" and "Start with a blank list" modes was slightly different.
In particular, in "Include everything" mode all folder exclusions were final, meaning that the program didn't even bother descending into them during the scan.
And this brings us to...
"Upgrade notice - Legacy mode"
When you upgrade from a pre-R78 version, the program may change the global "final" presets so that the filtering logic remains the same as before.
When this happens, you will see the following in the backup log:
Upgrade notice -
Legacy mode is enabled for processing of excluded folders.
This means is that the program changed the default for (verbose excluded) and (generic excluded) rules to be "final".
3
Aug 29, 2017
Finally
In configuration files the "final" attribute is defined by adding ! for Yes and ? for No to the rule's action (+ or -), like so:
conf.filters.src.verbose.folder +! .\AdvancedInstallers
conf.filters.src.generic.rule -? match \Recycler
The global "final" presets are controlled by:
conf.filters.src.verbose.include_is_final
conf.filters.src.verbose.exclude_is_final
conf.filters.src.generic.include_is_final
conf.filters.src.generic.exclude_is_final
entries for the source filters and respective keys for the destination filters.
4
MWorthington :
Sep 09, 2017
Hi Alex,
Nice update!
However, when viewing a selection in "What to Backup", the small tick used on a selected-subfolder (when the parent folder is not selected) is quite hard to see. I'm not sure what to suggest .... maybe a grey, big tick against the parent folder? Or make that small tick red, or blue ...?
Regards,
Mark
5
MWorthington :
Sep 11, 2017
Alex,
I think it's worth highlighting a significant improvement (I'm not sure if I've missed it in your notes) ....
Up until R78, I could exclude any number of sub-folders for a top-level folder, However, any newly created folders would, by default, be included in a new backup.
With the latest version, I'm in better control. I exclude the top-level and only include the sub-folders I want.
In my case, for the backup strategy I use, that's a BIG improvement, thanks!
Mark
6
Alex Pankratov :
Sep 11, 2017
the small tick ... is quite hard to see
It's a secondary piece of information, so it doesn't really need to be highly visible. If someone's looking for this information (as in "I wonder if I have any folders excluded below the surface level"), they should be able to find it.
I had another option for this (the one on the right), but I think it takes an effort to notice it and to understand how it works. Though once you know that, it's pretty obvious -
https://bvckup2.com/wip/r78-filter-icons.png
With the latest version, I'm in better control. I exclude the top-level and only include the sub-folders I want.
Yep. The new filtering framework is more flexible *and* more uniform at the same time. My main concern with it is that in some cases people might want to exclude a folder, period, and in other cases they may want to exclude everything in a folder by default and then reverse this with wildcard "include" rules. The default is the latter, but we'll have to wait and see if it's a good default or not.
7
daniel.filipowicz :
Dec 14, 2017
Previous releases (i.e. 76.14) offer an option to exclude folders in both source and also destination which causes Bvckup to ignore/omit them fully. I could edit destination exclusions via Ctrl-click the Details button next to What to backup. How do I achieve the same behavior in the current release since Ctrl-click on Details doesn't work anymore
8
Alex Pankratov :
Dec 14, 2017
Daniel,
This is still very well supported. You just need to enable this in the UI settings:
1. Exit the program
2. Add the following line to %LocalAppData%\Bvckup2\ui\bvckup2-ui.ini
edit_dst_filters 1
3. Start it back up. Then Ctrl-Click on Details will work.
https://bvckup2.com/support/forum/topic/502/29599
daniel.filipowicz :
Dec 15, 2017
Hi Alex, Thank you for your prompt response.
Problem solved - nice and easy! Thank you again for your help!