Matching items by size or timestamps
Aug 10, 2015
Backup job can be set up with rules to match files/folders by their type, attributes and names. This is configured from the UI in the Backup What section of the backup configuration -
https://www.bvckup2.com/img/r8/screenshot-2-backup-settings-5.png
In additional to these, it's possible to also match items based on their timestamps and to match files on their size. This requires editing settings.ini file for the backup job, but this shouldn't really scare anyone who's in need of these matching options, should it? :)
--
To get to settings.ini - right-click on the job in the UI, select "Open Folder" > "Configuration and Logging".
To edit settings.ini - exit the app, wait for it to disappear from the Processes tab in Task Manager and then open the ini in Notepad.
--
Each filter is stored as an entry under "conf.filters.2.manual" key, e.g.
conf.filters.2.manual 0 00000010 00000000 \hiberfil.sys
Simple format
There are two formats used to store the rules - simple (as above) and extended (described below). Simple format is used for rules that are created through the UI -
Leading 0 or 1 is "exclude" or "include"
Followed by attribute mask in hex format
Followed by attribute values in hex format
Followed by a pattern for the name match
Attribute matching works by taking file/folder attribute bitfield, AND'ing it with the "mask" and matching to the "value". Bits in a bitfield come straight from Windows API and they are documented here in [1]
Extended format
Extended format is ... well ... extended. Here's an example of the rule using it:
+ attr (00000010, 00000000) size (100, 200) ctime ( 1991-01-01 00:00:00, now - 1 w ) match *.xyz
This will match _files_ (the "attr" part) that are between 100 and 200 bytes in _size_ and that were _created_ between Jan 1st, 1991 and a week before now, and with a name that matches "*.xyz"
Basically the format is
+ or -
<match-target> ( <match-details> )
<match-target> ( <match-details> )
match <pattern>
<match-target> can be one of the following:
attr (<mask>, <value>) -- attribute match as per above
size (<min-bytes>, <max-bytes>) -- size match, files only
ctime (<when>, <when>) -- created timestamp match
mtime (<when>, <when>) -- modified timestamp match
<when> can be in absolute format:
1970-01-02 03:04:05
or in relative format:
now
now - <interval>
now + <interval>
whereby <interval> is a number followed by 's' for second, 'm' for minute, 'h' for hour, 'd' for day, 'w' for week. For example,
now - 1 w -- a week ago
now + 123 s -- 123 seconds in the future
--
If anything's unclear, ask below.
--
[1]
https://msdn.microsoft.com/en-us/library/windows/desktop/gg258117%28v=vs.85%29.aspx1
soundsfishy :
Nov 29, 2015
Thanks for this detailed inormation Alex... What would I need to type to have all files from today ( November 29th 2015 ) onwards be included in the back up ?
Will the rules I create in the UI populate the bitfield section and I can just add on the extended part to set the date range ?
2
Alex Pankratov :
Nov 30, 2015
I'd do it in reverse - I'd add a blanket rule at the top of the list to exclude all older files:
- attr (00000010, 00000000) mtime (1970-01-01 00:00:00, 2015-11-29 00:00:00)
Re: second question - rules that are specified through UI are stored is a simpler format, e.g.
0 00000010 00000000 *.erm
whereby leading 0/1 is exclude/include, followed by the attribute mask/value (same as "attr" above), followed by a name pattern. But it's easy enough to convert to the extended format.
* Just make sure to exit the app before messing with settings.ini
3
soundsfishy :
Nov 30, 2015
Thanks Alex... I don't really understand the bitfield 00's and 1's ... does that series of numbers change according to the field I am filtering on.. ie would it be different if i was excluding files based on folder location for example ?
4
Alex Pankratov :
Nov 30, 2015
Don't mind those, these are attribute matching rules, e.g. "hidden", "system", "read-only", etc. Incidentally, there's also a "directory" attribute that tells us whether an item is a file or a folder - that's why you see these attribute clauses in the rules.
To keep it simple(r), just remember that -
00000010, 00000000
will match FILES and
00000010, 00000010
will match FOLDERS. If you want to match both, then you can either omit "attr (...)" altogether, or set it to
00000000, 00000000
Makes sense?
5
soundsfishy :
Dec 01, 2015
Great - thank you !
6
soundsfishy :
Dec 01, 2015
Sorry.. one more question.. do I add the "conf.filters.2.manual " above all the other entries in that secion, plus the code you gave me ... or just the code on its own ?
7
soundsfishy :
Dec 02, 2015
Hi again.. tried to add the code to the ini file after exiting the app and checking it is gone from task manager. But it doesn't seem to work. And when I open the ini file again, the entry i added is not there ?
8
Alex Pankratov :
Dec 02, 2015
1. Show me the exact entry you are using.
2. Make sure you run a recent version of the app. Support for this sort of filters was added not long ago, in Release 73.
Also, you can have a look in %LocalAppData%\Bvckup2\bvckup2.log - there should be an entry that starts with "Invalid ini key".
9
soundsfishy :
Dec 04, 2015
Hi Alex.. I am running v74.22.
Nothing with the word invalid in the log file you pointed to.
I am entering the following above the list of all the other conf.filters.2.manua entries ....
conf.filters.2.manual - attr (00000010, 00000000) mtime (1970-01-01 00:00:00, 2015-11-29 00:00:00)
10
Alex Pankratov :
Dec 04, 2015
Oh, my bad! Every rule must end with the "match ..." clause, so the above rule should be:
conf.filters.2.manual - attr (00000010, 00000000) mtime (1970-01-01 00:00:00, 2015-11-29 00:00:00) match *
And it needs to be to go above all other conf.filters.2.manual rules in the INI file.
11
soundsfishy :
Dec 05, 2015
trying it now !!
12
soundsfishy :
Dec 05, 2015
That works great.. except it copies all the folders (and subfolders) from the source drive, even though the majority do not have any files in them (as they do not match the rule we added). Is there a way to delete the empty folders from the BACK UP (destination) drive, or prevent a folder being created if no files from that folder will be copied ?
13
Alex Pankratov :
Dec 05, 2015
Yes, there is. There's another option for that :)
conf.scanning.skip_empty_folders 1
This will cause the app to go through the source tree it gets from the scanner and trim all empty folders from it. That is, before it feeds this tree into the backup planner along with the destination tree. The net effect is as if these empty folders weren't present at the source.
14
soundsfishy :
Dec 05, 2015
Will any of these features (date selection & skip empty folders) be available in the UI soon ?
15
soundsfishy :
Dec 05, 2015
(thank you by the way... all working great now !)
16
Alex Pankratov :
Dec 06, 2015
Can't promise "soon", but the UI will get a facility for editing all available options. Similar to how uTorrent & Co. do this -
https://bvckup2.com/support/forum/topic/379
