LFTP

From Lsdf
Revision as of 14:51, 17 August 2017 by Ressmann (talk | contribs)
Jump to navigationJump to search

Intro

LFTP is an enhanced sftp client that allows faster parallel down and uploads as well as a more convenient interface

Documentation

LFTP Documentation can be found here http://lftp.yar.ru/

Getting and installing

Most current Linux distributions offer lftp through their package management. However they carry mostly rather old versions which have annoying bugs.

We suggest to use at least lftp versions above 4.7.x. The LFTP web site offers source packages that enable you to compile the most recent version. Check the README and the INSTALL files in the documentation of lftp.

To install lftp using your distribution package management tool

 yum install lftp
 sudo apt-get install lftp

Using

Log in with your archive account

 lftp -u <abc> sftp://archive-sftp.lsdf.kit.edu
 Password: <password>

Usage examples

Upload a local file (lfile) to the archive:

 lftp>put lfile

Upload, reverse mirror (-R), a local directory (ldir) to a remote directory (rdir) using 4 processes. Use the --continue (-c) option to allow the mirror operation to continue where it left off in case something happens (usefull if you archive thousands of files and directories).

 lftp>mirror -c -R --parallel=4 ldir rdir

Download a remote directory using 4 streams for every single file. (This improves download speeds for large i.e. more than 100 MB, files):

 lftp>mirror -c --use-pget-n=4 rdir ldir

From the command line to copy a whole directory towards the archive and redirection of the output:

 lftp -u <username>,<password> -e "mirror --reverse -v <source directory> ./private/<destination>/;quit" sftp://archive-sftp.lsdf.kit.edu 2> error_output.txt 1> stdoout.txt

The advantage of the last command is that the verbose output. The error output and the standard out is in seperate files, as such the error messages are clear identified. The standout still allows to see each single file which has been transfered. Please be aware that the password is written in clear text on the command line and can be found in the history, this is just an example and should be avoided.

Many examples with explanations can be found here: LFTP use examples

Throughput measurements