LSDF Online Storage: WebDAV

WebDAV or Web-based Distributed Authoring and Versioning is an extension of the Hypertext Transfer Protocol (HTTP) allowing to access files over the internet. WebDAV access to the LSDF Online Storage is provided via a cluster of servers reachable at the URL https://os-webdav.lsdf.kit.edu or davs://os-webdav.lsdf.kit.edu depending on the client used.

Prerequisites

Please note the following requirements and restrictions when using the LSDF WebDAV service:

The WebDAV protocol in general does suport the listing of special files like symbolic links, i.e. symbolic links present in the LSDF file space will not be listed with most of the tools described on this page. Nevertheless it is still possible to access the content behind symbolic links. Therefore, it is recommended to access project directories directly, e.g. when mounting them in the file system or mapping them as network drives.

Using WebDAV from a Web Browser

To access the starting page enter the following URL in your web browser

https://os-webdav.lsdf.kit.edu/

Project directories are accessible at the usual location:

https://os-webdav.lsdf.kit.edu/kit/<OE>/projects/<PROJECTNAME>

where <OE> is the name of your organizational unit, e.g. scc, and <PROJECTNAME> ist the name of the project. Your home directory is accessible at the following URL:

https://os-webdav.lsdf.kit.edu/kit/inst/<OE>/<USERNAME>/

where <USERNAME> is you KIT username.

Using WebDAV on Linux

Using DaviX in the Command Line

The DaviX project provides a library and a set of client tools for file management over HTTP-based protocols. The command line client tools can be either installed as a regular package in you linux distribution, e.g. in RedHat based distributions with

$ yum install -y epel-release
$ yum install davix

or from the home page of the DaviX project.

Listing a Directory

The content of a directory or individual files can be listed with the following command:

$ davix-ls -l davs://os-webdav.lsdf.kit.edu/kit/scc/xy1234
Basic authentication - server is asking for username and password:
Login: xy1234
Password: 
-rwxrwxrwx 0     10485760   2019-05-20 09:29:18 a_small_file

Copying a File

To copy a file from the WebDAV share to you local directory:

$ davix-get davs://os-webdav.lsdf.kit.edu/kit/scc/xy1234/testfile testfile
Basic authentication - server is asking for username and password:
Login: xy1234
Password: 
Performing Read operation on: davs://os-webdav.lsdf.kit.edu/kit/scc/xy1234/testfile
[==================================] 100%     10MiB/10MiB         0B/s                     

Recursively Uploading Files

Uploading files recursively with 6 transfers in parallel:

$ davix-put -r6  a_directory davs://os-webdav.lsdf.kit.edu/kit/scc/xy1234/a_new_directory
Basic authentication - server is asking for username and password:
Login: xy1234
Password: 
Populating task queue for a_directory/      Files processed:          0                     

Recursively downloading files

Files and directories can be downloaded recursively

$ davix-get -r6 davs://os-webdav.lsdf.kit.edu/kit/scc/xy1234/a_new_directory downloaded_dir
Basic authentication - server is asking for username and password:
Login: xy1234
Password: 
Crawling davs://os-webdav.lsdf.kit.edu/kit/scc/xy1234/a_new_directory/      Files processed:          1    

Mounting a WebDAV Share

davfs2 is a file system driver that allows mounting a WebDAV server as a local file system, like a disk drive. This way, applications can access resources on a WebDAV server without knowing anything about HTTP or WebDAV. davfs2 does extensive caching to make the file system responsive, to avoid unnecessary network traffic and to prevent data loss, and to cope for slow or unreliable connections.

Installation

On CentOS or RedHat like systems install the package davfs2 which is available in the epel repository:

$ yum install -y epel-release
$ yum install davfs2

Mounting a Share (as root)

To mount an LSDF WebDAV share issue the following command as root:

$ mount.davfs https://os-webdav.lsdf.kit.edu/ /mnt/webdav >/dev/null 2>&1
Please enter the username to authenticate with server
https://os-webdav.lsdf.kit.edu/ or hit enter for none.
 Username: xy1234
Please enter the password to authenticate user fg9068 with server
https://os-webdav.lsdf.kit.edu/ or hit enter for none.
 Password:  

or

$ mount.davfs -o username=xy1234,uid=xxx,gid=yyy,rw https://os-webdav.lsdf.kit.edu/inst/projects /mnt/webdav

Mount WebDAV share using fstab

>cat /etc/fstab | grep webdav
https://os-webdav.lsdf.kit.edu/kit/inst/projects /mnt/webdav davfs rw,noauto,user,uid=xy1234 0 0

>mkdir /mnt/webdav 

>cat /etc/davfs2/secrets | grep os-webdav
https://os-webdav.lsdf.kit.edu/kit/inst/projects    xy1234     ***

>mount /mnt/webdav >/dev/null 2>&1

WebDAV umount:

>umount.davfs /mnt/webdav

Mount WebDAV share as user

davfs2 makes mounting by unprivileged users as easy and secure as possible.

User must be a member of the davfs2 group:

>usermod -a -G davfs2 xy1234
>grep davfs2 /etc/group

Example:

>cat /etc/fstab | grep webdav
https://os-webdav.lsdf.kit.edu/kit/inst/projects /mnt/webdav davfs rw,noauto,user,uid=xy1234 0 0

>chmod 600 /lsdf/kit/inst/xy1234/.davfs2/secrets

>cat /lsdf/kit/inst/xy1234/.davfs2/secrets | grep os-webdav
https://os-webdav.lsdf.kit.edu/kit/inst/projects  xy1234   ***

>mount /mnt/webdav >/dev/null 2>&1

>umount /mnt/webdav
umount: /mnt/webdav: umount failed: Operation not permitted

You can mount the above-mentioned WebDAV share using sudo utility.

WebDAV client cadaver

cadaver is a command-line WebDAV client, with support for file upload, download, on-screen display, in-place editing, namespace operations (move/copy), collection creation and deletion, property manipulation, and resource locking.

Installation:

yum install cadaver

Example:

cadaver https://os-webdav.lsdf.kit.edu/kit/inst/xy1234
dav:/kit/inst/xy1234/> help
Available commands: 
Available commands: 
 ls         cd         pwd        put        get        mget       mput       
 edit       less       mkcol      cat        delete     rmcol      copy       
 move       lock       unlock     discover   steal      showlocks  version    
 checkin    checkout   uncheckout history    label      propnames  chexec     
 propget    propdel    propset    search     set        open       close      
 echo       quit       unset      lcd        lls        lpwd       logout     
 help       describe   about      
Aliases: rm=delete, mkdir=mkcol, mv=move, cp=copy, more=less, quit=exit=bye

Connection via File Manager

• Nautilus (Gnome)
• Kubuntu (Dolphin/Konqueror) (KDE)
• Xubuntu (Thunar) (Xfce)
• Lubuntu (PCManFM)

Using WebDAV on Windows

To mount a LSDF WebDAV share (e.g. a project directory) and map it to a drive in windows follow the steps outlined below:

1. Open a File Explorer and select This PC on the left hand pane.
2. In the Menu select Computer => Map Network Drive.
3. Click Connect to a Web site that you can use to store your documents and pictures.
4. Click Next
5. Click Choose a custom network location and afterwards click Next.
6. Enter the URL to you project folder https://os-webdav.lsdf.kit.edu/kit/<OE>/projects/<PROJECT>, where <OE> is you organizational unit and <PROJECT> is the name of your project directory, e.g. https://os-webdav.lsdf.kit.edu/kit/scc/projects/myproject
7. Click Next
8. Enter a name for the network location and click Next
9. Click Finish