BwFileStorage rsync
Using rsync from UNIX client via the login-cluster
Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon. It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amoun of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.
Rsync finds files that need to be transferred using a "quick check" algorithm (by default) that looks for files that have changed in size or in last-modified time. Any changes in the other preserved attributes (as requested by options) are made on the desti-nation file directly when the quick check indicates that the fileâs data does not need to be updated.
Example:
>rsync -e 'ssh -c arcfour -p 22' --timeout=180 –av /user path/ ka_xy1234@bwfilestorage-login.lsdf.kit.edu:~/bwfs path/ 1>/tmp/ka_xy1234.msg 2>/tmp/ka_xy1234.err ka_xy1234@bwfilestorage-login.lsdf.kit.edu's password:
ka=prefix for KIT user : fileset name per organization; prefix for login name and group name
xy1234=loginname
We recommend the arcfour- or aes128-cbc-algorithm for the encryption of the transmission.
As during all transfers errors might occur, we recommend controlling the correct transfer by checking the return value and rerun the transfer if it is not zero.
> echo $?
If you continue to work in bwgrid and delete data there you can use the option ‚--delete‘. With this your copy in bwFileStorage will be kept up to date. The option ‚--delete‘ deletes files in the target directory if they were deleted in the source directory.
