Hello /g/entoomen,
In Dropbox, if I created a zip archive of 100MB, it 10mo backup. If I add 10 MB file in the archive, it will add 10 mo and send these 10mo there, it will not save 110MB again.
What do we call this technology?
copy /b 100mb.zip + 10mb.zip 110mb.zip
>>55296396
Dropbox uses a binary diff algorithm to break down all files into blocks, and only upload blocks that it doesn't already have in the cloud. All of this is done locally on your computer.
Dropbox doesn't just use your files that you have already uploaded, it aggregates everyone's files into one database of blocks, and checks each local block hash against that database.
This means that if someone else has uploaded the same file as yourself (say for example, the latest Ubuntu ISO), then the upload will seem instant as there is nothing to upload, but if you are updating a file that changes regularly, like your backup file, then only the changes are uploaded. If you upload a totally unique file, then you have to wait for it all to upload.
More:
http://serverfault.com/questions/52861/how-does-dropbox-version-upload-large-files
https://blogs.dropbox.com/business/2014/05/what-is-file-sync/
>>55296493
Ok i see...
And this technologie are used by some self hosted cloud solutions like Pydio or OwnCloud ?
>>55296511
Google it up yourself fgt. It's called Incremental sync in owncloud.
>>55296396
This sounds interesting, is it possible to add this feature for a custom FTP server? I upload a backup of some local files from time to time, but as my archives grow larger it needs a lot of time consuming my 2 MBit connection. This could really help.
>>55296396
It's called block level sync, and so far only dropbox do it correctly.
>>55296904
If the underlying file system has deduplication, this will happen, i think zfs and brtfs may have a deduplication option.
