R80 (before) | 17 sec |
R81 (after) | 5.7 sec |
Block size | 32KB |
Block hashes | MD5 + a variant of CRC32 |
Full-file hash | SHA-1 |
HDD | Seagate ST9500325AS | 70.5 MB/s |
HDD | WDC WD5000LPVX | 118.8 MB/s |
SHA-1 | Intel i3 @ 2.53GHz | 332.4 MB/s |
SHA-1 | Xeon E5 v4 @ 2.10GHz | 380.1 MB/s |
SSD | Samsung T3, USB 3.0 | 429.7 MB/s |
SHA-1 | Intel i5 @ 2.70GHz | 520.9 MB/s |
SSD | Samsung 860 Pro | 534.8 MB/s |
NVMe | Samsung MZVPV512HDGL | 2157.9 MB/s |
Block size | 64KB |
Block hashes | Blake2b + xxHash + Spooky |
Full-file hash | - |
MD5 | 595.4 MB/s |
SHA-1 | 561.9 MB/s |
SHA-256 | 247.4 MB/s |
SHA-512 | 391.4 MB/s |
Blake2b | 762.3 MB/s |
Release 78 | Release 79 | |
Creating | 262.8 MB/s | 547.2 MB/s |
Updating | 272.5 MB/s | 2026.6 MB/s |
Bonus material
If you are to think about all this for as looong as I have, you might realize that the Include everything / Start with an empty list selector at the top, this one -
First up is a new and important feature - the write verification.
This process helps to safeguard against two types of problems.
1. Data corruption on the way to the device
This may happen due to the software and firmware bugs, memory bit flips (both in the machine's own RAM and in the controller/device memory) and transport errors if going over the network.The relative risk of such corruption is exceedingly small, but the backups tend to push a lot of data around, so the absolute risk may grow with time to the non-negligible levels.
2. Latent storage media problems
Reading data back effectively acts as a form of scrubbing and helps flushing any pending issues with the storage media.If the media is OK, the read-back will just complete nominally.
If the media is dying, i.e. if the data can still be read, but only after retrying or via the sector ECC recovery, then we will get our data back and trigger the sector relocation, avoiding potential data loss if the media were to degrade further.
If the media is dead, then we'll won't have our data back, but we'll learn about the problem now rather than later.
How it works
The somewhat less obvious aspect of write verification is that it needs to bypass the file system cache and to try and read the data from actual the device.The program arranges for this by opening the target file for the second time and requesting a so-called unbuffered read access to it.
This doesn't prevent writes from being cached, but ensures that reads are served from the disk. Going around the cache automatically means that the copying becomes slower.
The degradation is in part mitigated by the asynchronous nature of the ultra copier and it's further reduced by the delta copying if it's in effect, but there is a slow down. No miracle there.
How it looks
Once enabled, the write verification setting will show up in the Configuration section:This is by design to keep the issue on the surface and to force investigating it. In prerelease builds the write verification can be enabled for a job via the right-click menu as shown in the opening screenshot.
In production builds it's likely to be the same, except the Preview submenu will have a different name and it will be visible only if the Ctrl held down. You can get a prerelease build by switching your installation to the "preview" update channel as described here.