Difference between revisions of "LSDF Online Storage: HPC"
From Lsdf
| (12 intermediate revisions by 2 users not shown) | |||
| Line 5: | Line 5: | ||
Both for interactive and for batch use, the environment variables '''$LSDF''', '''$LSDFHOME''' and '''$LSDFPROJECTS''' are set. |
Both for interactive and for batch use, the environment variables '''$LSDF''', '''$LSDFHOME''' and '''$LSDFPROJECTS''' are set. |
||
| + | <!-- |
||
<H2>rdata</H2> |
<H2>rdata</H2> |
||
| − | Users of the filesystem LSDF Online Storage can furthermore transfer data to '''bwUniCluster''' and '''ForHLR''' via the tool '''rdata'''. |
+ | Users of the filesystem LSDF Online Storage can furthermore transfer data to '''bwUniCluster''' and '''ForHLR''' via the tool '''rdata'''. T |
The command rdata executes the filesystem operations on special "data mover" nodes and distributes the load. |
The command rdata executes the filesystem operations on special "data mover" nodes and distributes the load. |
||
| Line 81: | Line 82: | ||
remote-fs-utils April 2013 |
remote-fs-utils April 2013 |
||
</pre> |
</pre> |
||
| + | --> |
||
| − | <H2>Mount on the login nodes |
+ | <H2>Mount on the login nodes</H2> |
| + | '''Environment variables:'''<br> |
||
| − | Limits: |
||
| + | LSDFPROJECTS=/lsdf/kit/inst/projects <br> |
||
| − | Cpu time limit: 3600s |
||
| + | LSDFHOME=/lsdf/kit/inst/xy1234 <br> |
||
| − | Virtual memory limit: 8GB |
||
| + | LSDF=/lsdf <br> |
||
| + | '''Limits:'''<br> |
||
| − | <H2>Batch Jobs (only [http://www.scc.kit.edu/dienste/forhlr.php ForHLR])</H2> |
||
| + | Cpu time limit: 3600s <br> |
||
| + | Virtual memory limit: 8-12GB |
||
| + | <H2>Batch Jobs</H2> |
||
| − | On the compute nodes, the LSDF usage is indicated to the batch system via "'''#SBATCH --constraint=LSDF'''". |
||
| − | * [https://wiki.scc.kit.edu/hpc/index.php/ForHLR_-_SLURM_Batch_Jobs#LSDF_Online_Storage Slurm HPC Workload Manager] |
||
| + | On the compute nodes, the LSDF usage is indicated to the batch system via "'''#SBATCH --constraint=LSDF'''". |
||
| + | * [https://wiki.bwhpc.de/e/BwUniCluster_2.0_Slurm_common_Features#LSDF_Online_Storage bwUniCluster] |
||
| + | * [https://wiki.scc.kit.edu/hpc/index.php?title=ForHLR_-_SLURM_Batch_Jobs#LSDF_Online_Storage ForHLR] |
||
<hr> |
<hr> |
||
<br> |
<br> |
||
Latest revision as of 12:54, 15 December 2020
LSDF_Online_Storage for HPC Cluster
The LSDF storage is permanently mounted on the login nodes and the data movers.
Both for interactive and for batch use, the environment variables $LSDF, $LSDFHOME and $LSDFPROJECTS are set.
Mount on the login nodes
Environment variables:
LSDFPROJECTS=/lsdf/kit/inst/projects
LSDFHOME=/lsdf/kit/inst/xy1234
LSDF=/lsdf
Limits:
Cpu time limit: 3600s
Virtual memory limit: 8-12GB
Batch Jobs
On the compute nodes, the LSDF usage is indicated to the batch system via "#SBATCH --constraint=LSDF".
