LSDF Online Storage: HPC: Difference between revisions

From Lsdf
Jump to navigationJump to search
No edit summary
No edit summary
(13 intermediate revisions by 2 users not shown)
Line 5: Line 5:
Both for interactive and for batch use, the environment variables '''$LSDF''', '''$LSDFHOME''' and '''$LSDFPROJECTS''' are set.
Both for interactive and for batch use, the environment variables '''$LSDF''', '''$LSDFHOME''' and '''$LSDFPROJECTS''' are set.


<!--
<H2>rdata</H2>
<H2>rdata</H2>


Users of the filesystem LSDF Online Storage can furthermore transfer data to '''bwUniCluster''' and '''ForHLR''' via the tool '''rdata'''. Therefore the environment variables '''$LSDF''', '''$LSDFHOME''' and '''$LSDFPROJECTS''' are set.
Users of the filesystem LSDF Online Storage can furthermore transfer data to '''bwUniCluster''' and '''ForHLR''' via the tool '''rdata'''. T
The command rdata executes the filesystem operations on special "data mover" nodes and distributes the load.
The command rdata executes the filesystem operations on special "data mover" nodes and distributes the load.
Line 81: Line 82:
remote-fs-utils April 2013
remote-fs-utils April 2013
</pre>
</pre>
-->


<H2>Mount on the login nodes</H2> (only [http://www.scc.kit.edu/dienste/forhlr.php ForHLR])
<H2>Mount on the login nodes</H2>


'''Environment variables:'''<br>
⚫
Limits:
LSDFPROJECTS=/lsdf/kit/inst/projects <br>
⚫
Cpu time limit: 3600s
LSDFHOME=/lsdf/kit/inst/xy1234 <br>
⚫
Virtual memory limit: 8GB
LSDF=/lsdf <br>


⚫
'''Limits:'''<br>
<H2>Batch Jobs</H2> (only [http://www.scc.kit.edu/dienste/forhlr.php ForHLR])
⚫
Cpu time limit: 3600s <br>
⚫
Virtual memory limit: 8-12GB


<H2>Batch Jobs</H2>
⚫
* [https://wiki.scc.kit.edu/hpc/index.php/ForHLR_-_SLURM_Batch_Jobs#LSDF_Online_Storage Slurm HPC Workload Manager]


On the compute nodes, the LSDF usage is indicated to the batch system via "'''#SBATCH --constraint=LSDF'''".
* [https://wiki.bwhpc.de/e/BwUniCluster_2.0_Slurm_common_Features#LSDF_Online_Storage bwUniCluster]
⚫
* [https://wiki.scc.kit.edu/hpc/index.php?title=ForHLR_-_SLURM_Batch_Jobs#LSDF_Online_Storage ForHLR]
<hr>
<hr>
<br>
<br>

Revision as of 12:54, 15 December 2020

LSDF_Online_Storage for HPC Cluster

The LSDF storage is permanently mounted on the login nodes and the data movers.

Both for interactive and for batch use, the environment variables $LSDF, $LSDFHOME and $LSDFPROJECTS are set.


Mount on the login nodes

Environment variables:
LSDFPROJECTS=/lsdf/kit/inst/projects
LSDFHOME=/lsdf/kit/inst/xy1234
LSDF=/lsdf

Limits:
Cpu time limit: 3600s
Virtual memory limit: 8-12GB

Batch Jobs

On the compute nodes, the LSDF usage is indicated to the batch system via "#SBATCH --constraint=LSDF".





Retrieved from "https://wiki.scc.kit.edu/lsdf/index.php?title=LSDF_Online_Storage:_HPC&oldid=5710"