stevepedwards.com/DebianAdmin linux mint IT admin tips info

Cool Cmd #4 Unison

I'm really big on backups and redundancy in all its forms, whether storage, duplicate server functionality or communications paths, due to seeing the mess people get into without them when something fails and is lost forever.

I've known about and used rsync for years, but only found out about another equally good tool tonight that does an equivalent job in two way cloning but in one pass, instead of reversing the to-from folders or using && in one command line such as:

rsync -vr /from /to && rsync -vr /to /from

so that both directories end up with the exact same files inside.

Unison does also use the clever rsync algorithm for various tasks if required - read its man page.

man unison

NAME

       unison-2.40.102 - a file-synchronization tool for Unix and Windows  

Unison  is  a  file-synchronization  tool  for  Unix and Windows. It

       allows two replicas of a collection of files and directories  to  be

       stored  on  different  hosts  (or different disks on the same host),

       modified separately, and then brought up to date by propagating  the

       changes in each replica to the other.

Unison  offers several advantages over various synchronization meth‐

       ods such as CVS, Coda, rsync, Intellisync, etc. Unison  can  run  on

       and  synchronize  between  Windows  and  many UNIX platforms. Unison

       requires no root privileges, system  access  or  kernel  changes  to

       function. Unison can synchronize changes to files and directories in

       both directions, on the same machine, or across a network using  ssh

       or a direct socket connection.

       Transfers  are optimised using a version of the rsync protocol, mak‐

       ing it ideal for slower links. Unison has a clear and precise speci‐

       fication, and is resilient to failure due to its careful handling of

       the replicas and its private structures.

       The two roots can be specified using an URI or a path. The URI  must

       follow  the  convention: protocol://[user@][host][:port][/path]. The

 

       protocol part can be file, socket, ssh or rsh.

 

It's a bit odd at first go, as you get a little Q+A and use explanation:

unison /home/stevee/Documents/ /home/stevee/Downloads/Dropbox/

Contacting server...

Looking for changes

Warning: No archive files were found for these roots, whose canonical names are:

/home/stevee/Documents

/home/stevee/Downloads/Dropbox

This can happen either

because this is the first time you have synchronized these roots, 

or because you have upgraded Unison to a new version with a different

archive format.  

 

Update detection may take a while on this run if the replicas are 

large.

 

Unison will assume that the 'last synchronized state' of both replicas

was completely empty.  This means that any files that are different

will be reported as conflicts, and any files that exist only on one

replica will be judged as new and propagated to the other replica.

If the two replicas are identical, then no changes will be reported.

 

If you see this message repeatedly, it may be because one of your machines

is getting its address from DHCP, which is causing its host name to change

between synchronizations.  See the documentation for the UNISONLOCALHOSTNAME

environment variable for advice on how to correct this.

 

Donations to the Unison project are gratefully accepted: 

http://www.cis.upenn.edu/~bcpierce/unison

 

Reconciling changestinue.[<spc>] \ systest_script.sh

 

Documents      Dropbox            

file     <-?-> file       googledrive-sync.sh  [] y

Unrecognized command 'y': try again  [type '?' for help]

file     <-?-> file       googledrive-sync.sh  [] ?

Commands:

  f                     follow unison's recommendation (if any)

  I                     ignore this path permanently

  E                     permanently ignore files with this extension

  N                     permanently ignore paths ending with this name

  m                     merge the versions

  d                     show differences

  x                     show details

  L                     list all suggested changes tersely

  l                     list all suggested changes with details

  p or b                go back to previous item

  g                     proceed immediately to propagating changes

  q                     exit unison without propagating any changes

  /                     skip

  > or .                propagate from left to right

  < or ,                propagate from right to left

file     <-?-> file       googledrive-sync.sh  [] 

 you finally get a Yes option once you give a list option letter when it meets a file type it wants to know how to handle:

 

Once copied, both directories will be identical if you check them with diff:

diff /home/stevee/Documents/ /home/stevee/Downloads/Dropbox/

Except where I messed up the answer to the first file question of course before ignoring the transfer of those two slightly different files...

MintAspire-5630 stevee # diff /home/stevee/Documents/ /home/stevee/Downloads/Dropbox/

diff /home/stevee/Documents/googledrive-sync.sh /home/stevee/Downloads/Dropbox/googledrive-sync.sh

8c8

< BACKUPDIR=/home/stevee/Downloads/Gdrive

---

> BACKUPDIR=/home/stevee/Downloads/Dropbox

14c14

< TARGETDIR=/backups/

---

> TARGETDIR=/backups

As it states, if you want that file handled differently in future you edit the file relating to it, else when unison is run again it will show no changes between directories as expected.

MintAspire-5630 stevee # unison /home/stevee/Documents/ /home/stevee/Downloads/Dropbox/

Contacting server...

Looking for changes

Reconciling changes       

 

Nothing to do: replicas have not changed since last sync.

If I remove one of the different files, then unison still shows no change - you have to edit /root/.unison/default.prf bt deleting the ignore command I set above:

# Unison preferences file

ignore = Path {googledrive-sync.sh}

Once deleted and unison run, I get a Q+A again for the change, but only one identical copy of the file in each folder once done:

MintAspire-5630 stevee # unison /home/stevee/Documents/ /home/stevee/Downloads/Dropbox/

Contacting server...

Looking for changes

Reconciling changes  

Documents      Dropbox            

         <---- new file   googledrive-sync.sh  [f] f

Proceed with propagating updates? [] y

Propagating updates

UNISON 2.40.102 started propagating changes at 01:55:58.49 on 22 Apr 2014

[BGN] Copying googledrive-sync.sh from /home/stevee/Downloads/Dropbox to /home/stevee/Documents

[END] Copying googledrive-sync.sh

UNISON 2.40.102 finished propagating changes at 01:55:58.49 on 22 Apr 2014

Saving synchronizer state

 

Synchronization complete at 01:55:58  (1 item transferred, 0 skipped, 0 failed)

Now both folders are identical for diff:

MintAspire-5630 stevee # diff /home/stevee/Documents/ /home/stevee/Downloads/Dropbox/

MintAspire-5630 stevee #

OK, so it seems more work than using && in rsync after all, but its other advantages stated in the man page may be worth it depending on your situation. It can be annoying when rsync or Win Syncback fails to copy hundreds of files from one backup drive to another say, due to permission issues, and worse - just copies all the folder names first, so you think its all ok at a glance, then find them empty days later!

I will test unison for that claim and let you know...

15/11/14

I'm still struggling with this program - I can't seem to get it to do what I want or expect.

I wanted to clear and sync my web folders on one Mint PC and my backup drive so had another go with this command and struggled to get it to do what I wanted at first. The backup drive is under /SB

The idea was to get the contents of my Mint PC /www folder how I wanted it by deleting a lot of unwanted and recursively copied crap first, then propagate those changes to the backup drive using Unison i.e. Yes, you COULD just delete the whole /SB/www folder and straight copy the new /www folder in its place, but then you are not learning anything, and future changes would not be so quick if you have a very large folder to copy - which is the whole point of using Rsync and Unison in the first place!

contents of /www ---------> /SB/www

The command you need to stop being asked about every single file transfer is :

unison  /www/ /SB/www/ -nocreation /www/

This is because the default and only option in the main menu (using ?) is to copy any files that DONT exist in one folder to the other - NOT what I want. I want deletion of the remaining crap I just cleared from one folder to be deleted from the backup also. So what happens? You answer the first question to get options with ?

IomegaMint / # unison /www/ /SB/www/ -nocreation /www/
Contacting server...
Looking for changes
Reconciling changes

/www SB/www
props <-?-> props / [] ?

This gives the menu, so now all you need is to continue propagating changes with g, because the

-nocreation /www/

prevents files being written both ways to /www, only one way to /SB/www. the propagate left/right using > or < is useless here as it's a per file option only.

Commands:
f follow unison's recommendation (if any)
I ignore this path permanently
E permanently ignore files with this extension
N permanently ignore paths ending with this name
m merge the versions
d show differences
x show details
L list all suggested changes tersely
l list all suggested changes with details
p or b go back to previous item
g proceed immediately to propagating changes
q exit unison without propagating any changes
/ skip
> or . propagate from left to right
< or , propagate from right to left
props <-?-> props / [] g

....skipped: www2/www
skipped: wwwjpgs.zip
skipped: zippedpics.zip
skipped: ~$NA Crib.docx
skipped: ~$NACrib.htm
skipped: ~$and_Java_Info.doc
skipped: ~$dYRFdScAssignments.docx
skipped: ~$plications of Probability Theory Assignment 30.htm
failed:.....

If you now diff the 2 directories, they should be the same:

diff /www /SB/www

Hmm...they're not I'll have to check this out....

 

 

Comments are closed.

Post Navigation