Difference between revisions of "LSDF Online Storage: Secure Shell"

From Lsdf
(Using SSHFS on Linux or Mac OS)
(Add migration note)
(Tag: Replaced)
 
(5 intermediate revisions by one other user not shown)
Line 1: Line 1:
  +
This page has been updated and migrated to the new docs site:
Secure Shell or SSH refers to both a cryptographic network protocol as well a number of applications that can be used to establish a secure connect with a service provider like the LSDF Online Storage. SSH is typically used to login interactively to a remote machine and execute commands, but it can also be used to transfer file using the protocols SSH File Transfer Protocol (SFTP) or Secure Copy (SCP). The SSH Filesystem (SSHFS) is a file system client based on the SSH protocol which allows to locally mount and interact with directories and files located on a remote server.
 
   
  +
https://www.lsdf.kit.edu/docs/ssh/
SSH access to the LSDF Online Storage is provided via a cluster of login servers reachable at the address '''os-login.lsdf.kit.edu'''.
 
 
__FORCETOC__
 
 
== Using SSH on Linux or Mac OS ==
 
 
To interactively login to the LSDF Online Storage login cluster enter the following command in a linux shell or Mac OS terminal
 
<pre>
 
$ ssh <USERNAME>@os-login.lsdf.kit.edu
 
</pre>
 
where ''<USERNAME>'' needs to be replaced with your actual user name, e.g. xy1234. The command above will ask for a password to grant access to the login server.
 
 
To avoid being asked for a password on every login, password-less access can be configured by distributing a public key to the login server. To create a new set of keys issue the following command:
 
<pre>
 
$ ssh-keygen
 
</pre>
 
The newly generated key should be password protected.
 
 
To copy the public part of your key pair to the login server issue the following command:
 
<pre>
 
$ ssh-copy-id -i ~/.ssh/id_rsa.pub <USERNAME>@os-login.lsdf.kit.edu
 
</pre>
 
Subsequent logins will use the key instead of a password to authenticate to the login server.
 
 
Further details on the usage of the SFTP command line client can be found on the [https://man.openbsd.org/ssh.1 man page].
 
 
== Using SCP on Linux or Mac OS ==
 
 
SCP is a protocol that allows secure data transfer between a local and a remote computer or between two remote computers. To transfer the data to or from the LSDF
 
Online Storage, you can execute the following commands:
 
<pre>
 
$ scp <LOCAL PATH>/<FILES(S)> <USERNAME>@os-login.lsdf.kit.edu:<REMOTE PATH>/
 
or
 
$ scp <USERNAME>@os-login.lsdf.kit.edu:<REMOTE PATH> <LOCAL PATH>/<FILES(S)>
 
</pre>
 
Please be aware that symbolic links will usually not be copied as links.
 
 
'''Examples:''' <br>
 
<pre>
 
>scp -c arcfour ~/filename xy1234@os-login.lsdf.kit.edu:~/
 
or
 
>scp -c arcfour xy1234@os-login.lsdf.kit.edu:~/filename ~/
 
</pre>
 
To reach maximum bandwidth we recommend ssh ciphers '''arcfour''', '''aes128-gcm@openssh.com''' or '''aes128-cbc''' algorithm for the encryption of the transmission.
 
 
Further details on the usage of the SCP command line client can be found on the [https://man.openbsd.org/scp man page].
 
 
== Using SFTP on Linux or Mac OS ==
 
 
The following example illustrates the usage of SFTP in a linux shell:
 
<pre>
 
$ sftp xy1234@os-login.lsdf.kit.edu
 
Connecting to os-login.lsdf.kit.edu<br>
 
xy1234@os-login.lsdf.kit.edu's password:
 
 
sftp> ls
 
snapshots
 
temp test
 
 
sftp> help
 
...
 
 
sftp> put myfile
 
 
sftp> get myfile
 
</pre>
 
Further details on the usage of the SFTP command line client can be found on the [https://man.openbsd.org/sftp.1 man page].
 
 
 
== Using rsync on Linux or Mac OS ==
 
 
rsync is a fast and versatile file copying tool. It can copy locally, to or from another host over any remote shell, or to or from a remote rsync daemon. It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. '''rsync''' is widely used for backups and mirroring and as an improved copy command for everyday use.
 
 
Rsync finds files that need to be transferred using a "quick check" algorithm by (default) that looks for files that have changed in size or in last-modified time. Any changes in the other preserved attributes (as requested by options) are made on the destination file directly when the quick check indicates that the files data does not need to be updated.
 
 
The following example synchronizes the local directory /user_path/user_dir/ with the LSDF directory /lsdf_path/target_dir/ and sets the correct owning group in the LSDF:
 
<pre>
 
In the LSDF:
 
$ chmod -g+s /lsdf_path/
 
 
On the client host:
 
$ rsync -uva --no-g --chmod=Dg+s /user_path/user_dir/ xy1234@os-login.lsdf.kit.edu:/lsdf_path/target_dir/ 1>/tmp/xy1234.msg 2>/tmp/xy1234.err
 
 
xy1234@os-login.lsdf.kit.edu's password:
 
</pre>
 
 
As during all transfers errors might occur, we recommend controlling the correct transfer by checking the return value and rerun the transfer if it is not zero.
 
<pre>
 
$ echo $?
 
</pre>
 
 
<br>
 
If you continue to work and delete data you can use the option ''--delete''. The option ''--delete'' deletes files in the target directory if they were deleted in the source directory.
 
 
 
 
== Using SSHFS on Linux or Mac OS ==
 
 
SSHFS is a file system implementation allowing users to mount and interact with directories and files located on a remote server over a normal ssh connection. Using SSHFS a mounted remote file system behaves similar to other local volumes or files systems. In situations where higher performance and / or simultaneous access to the LSDF Online Storage by many users are required other protocols like NFS, CIFS or WebDAV might be better suited.
 
 
=== Installation ===
 
 
SSHFS is available on many linux distribution via the standard software distribution channels.
 
 
On Debian/Ubuntu like systems:
 
<pre>
 
$ apt-get install sshfs
 
</pre>
 
 
On RedHat/CentOS like systems:
 
<pre>
 
$ yum install fuse-sshfs
 
</pre>
 
 
=== Mounting a Remote File System Interactively (as normal user) ===
 
 
To mount a remote file system via command line:
 
<pre>
 
$ mkdir mountpoint
 
$ sshfs <USERNAME>@os-login.lsdf.kit.edu:/lsdf/kit/inst/projects/ ./mointpoint
 
</pre>
 
 
=== Mounting a Remote File System with ''/etc/fstab'' (as root) ===
 
 
Add (as root) a line to your ''/etc/fstab'' describing how to mount the remote folder:
 
<pre>
 
sshfs <USERNAME>@os-login.lsdf.kit.edu:/lsdf/kit/<INSTITUTE>/projects/ <MOUNTPOINT> fuse uid=<UID>,gid=<GID>,umask=0,allow_other,defaults,auto 0 0
 
</pre>
 
where ''<UID'' is the numerical user id of the user the remote mount will be mapped to, ''<GID>'' is the numerical group id of the
 
user the remote mount will be mapped to, ''<USERNAME>'' is the user name user for the SSH login, ''<MOUNTPOINT>'' is the local mount point (directory).
 
 
'''For example:'''
 
<pre>
 
sshfs xy1234@os-login.lsdf.kit.edu:/lsdf/kit/inst/projects/ /mnt/mountpoint fuse uid=7777,gid=12345,umask=0,allow_other,defaults,auto 0 0
 
</pre>
 
 
In order to make the above example work, password-less ssh login with keys needs to be properly configured. Mount the remote file system by issuing the following command (as root):
 
<pre>
 
$ mount /mnt/mountpoint
 
</pre>
 
 
== Using Secure Shell with Windows ==
 
 
Please find below a list of Secure Shell clients for Windows and MacOS (without any claim to completeness):
 
 
{| class="wikitable"
 
| '''MobaXterm'''
 
| https://mobaxterm.mobatek.net/
 
|-
 
| '''PuTTY'''
 
| https://www.chiark.greenend.org.uk/~sgtatham/putty/latest.html
 
|-
 
| '''WinSCP'''
 
| http://winscp.net/eng/download.php
 
|-
 
| '''FileZilla'''
 
| http://sourceforge.net/projects/filezilla/
 
|-
 
| '''WebDrive'''
 
| https://southrivertech.com/products/webdrive/download/
 
|-
 
| '''Cygwin'''
 
| http://cygwin.com/install.html
 
|-
 
| '''Windows 10 subsystem for Linux'''
 
| https://msdn.microsoft.com/de-de/commandline/wsl/install_guide
 
|}
 
 
The following software packages provide network drive functionality based on SFTP:
 
 
{| class="wikitable"
 
| '''WebDrive''' (for Windows, Mac, iOS, Android)
 
| http://www.southrivertechnologies.com/download/downloadwd.html
 
|-
 
| '''SFTPNetDrive''' (for Windows)
 
| http://www.sftpnetdrive.com/
 
|-
 
| '''NetDrive''' (for Windows)
 
| http://www.netdrive.net/
 
|-
 
| '''ExpanDrive''' (for Windows and Mac)
 
| http://www.expandrive.com/expandrive
 
|-
 
| '''MountainDuck''' (for Windows and Mac)
 
| https://mountainduck.io/
 
|}
 
 
 
 
== External Links ==
 
 
* [http://www.openssh.com/manual.html OpenSSH Manual Pages]
 
* [https://en.wikipedia.org/wiki/Secure_Shell Secure Shell on Wikipedia]
 
* [https://en.wikipedia.org/wiki/SSH_File_Transfer_Protocol SSH File Transfer Protocol on Wikipedia]
 
* [https://en.wikipedia.org/wiki/SSHFS Secure Shell Filesystem on Wikipedia]
 
 
[[Category:LSDF_Online_Storage|Secure Shell]]
 

Latest revision as of 18:50, 21 February 2023

This page has been updated and migrated to the new docs site:

https://www.lsdf.kit.edu/docs/ssh/