Difference between revisions of "LSDF Online Storage: HPC"

From Lsdf
(14 intermediate revisions by 2 users not shown)
Line 5: Line 5:
 
Both for interactive and for batch use, the environment variables '''$LSDF''', '''$LSDFHOME''' and '''$LSDFPROJECTS''' are set.
 
Both for interactive and for batch use, the environment variables '''$LSDF''', '''$LSDFHOME''' and '''$LSDFPROJECTS''' are set.
   
  +
<!--
 
<H2>rdata</H2>
 
<H2>rdata</H2>
   
Users of the filesystem LSDF Online Storage can furthermore transfer data to '''bwUniCluster''' and '''ForHLR''' via the tool '''rdata'''. Therefore the environment variables '''$LSDF''', '''$LSDFHOME''' and '''$LSDFPROJECTS''' are set.
+
Users of the filesystem LSDF Online Storage can furthermore transfer data to '''bwUniCluster''' and '''ForHLR''' via the tool '''rdata'''. T
 
 
 
The command rdata executes the filesystem operations on special "data mover" nodes and distributes the load.
 
The command rdata executes the filesystem operations on special "data mover" nodes and distributes the load.
Line 81: Line 82:
 
remote-fs-utils April 2013
 
remote-fs-utils April 2013
 
</pre>
 
</pre>
  +
-->
   
<H2>Mounton the login nodes</H2>
+
<H2>Mount on the login nodes</H2>
   
  +
'''Environment variables:'''<br>
Cpu time limit: 3600s
 
  +
LSDFPROJECTS=/lsdf/kit/inst/projects <br>
Virtual memory limit: 8GB
 
  +
LSDFHOME=/lsdf/kit/inst/xy1234 <br>
  +
LSDF=/lsdf <br>
   
  +
'''Limits:'''<br>
<H2>Batch Jobs</H2>
 
  +
Cpu time limit: 3600s <br>
  +
Virtual memory limit: 8-12GB
   
  +
<H2>Batch Jobs</H2>
* [https://wiki.scc.kit.edu/hpc/index.php/ForHLR_-_SLURM_Batch_Jobs#LSDF_Online_Storage Slurm HPC Workload Manager]
 
   
  +
On the compute nodes, the LSDF usage is indicated to the batch system via "'''#SBATCH --constraint=LSDF'''".
  +
* [https://wiki.bwhpc.de/e/BwUniCluster_2.0_Slurm_common_Features#LSDF_Online_Storage bwUniCluster]
  +
* [https://wiki.scc.kit.edu/hpc/index.php?title=ForHLR_-_SLURM_Batch_Jobs#LSDF_Online_Storage ForHLR]
 
<hr>
 
<hr>
 
<br>
 
<br>

Revision as of 12:54, 15 December 2020

LSDF_Online_Storage for HPC Cluster

The LSDF storage is permanently mounted on the login nodes and the data movers.

Both for interactive and for batch use, the environment variables $LSDF, $LSDFHOME and $LSDFPROJECTS are set.


Mount on the login nodes

Environment variables:
LSDFPROJECTS=/lsdf/kit/inst/projects
LSDFHOME=/lsdf/kit/inst/xy1234
LSDF=/lsdf

Limits:
Cpu time limit: 3600s
Virtual memory limit: 8-12GB

Batch Jobs

On the compute nodes, the LSDF usage is indicated to the batch system via "#SBATCH --constraint=LSDF".