Quickstart guide for gsatellite: Difference between revisions
(Link zu bwDAHub oben) |
(→Job creation: Kommentarzeilen schwarz) |
||
| Line 7: | Line 7: | ||
For the gtransfer job creation first use gtransfer's bash completion to construct the transfer command as usual but don't actually start it. When finished with the source and destination URLs and your desired [https://github.com/fr4nk5ch31n3r/gtransfer/blob/master/share/doc/gtransfer.1.md#options gtransfer options] copy the gtransfer command line to a text file and add the following header to it: |
For the gtransfer job creation first use gtransfer's bash completion to construct the transfer command as usual but don't actually start it. When finished with the source and destination URLs and your desired [https://github.com/fr4nk5ch31n3r/gtransfer/blob/master/share/doc/gtransfer.1.md#options gtransfer options] copy the gtransfer command line to a text file and add the following header to it: |
||
<pre> |
|||
<pre style="color: grey"> |
|||
#!/bin/bash |
#!/bin/bash |
||
#GSAT -T gtransfer |
#GSAT -T gtransfer |
||
| Line 14: | Line 14: | ||
Example gtransfer job file contents: |
Example gtransfer job file contents: |
||
<pre> |
|||
<pre style="color: grey"> |
|||
#!/bin/bash |
#!/bin/bash |
||
#GSAT -T gtransfer |
#GSAT -T gtransfer |
||
Revision as of 13:44, 20 July 2016
BWDAHub<br\> Preparing access to the BWDAHub<br\> Quickstart guide for gtransfer<br\> Quickstart guide for gsatellite
Job creation
For the gtransfer job creation first use gtransfer's bash completion to construct the transfer command as usual but don't actually start it. When finished with the source and destination URLs and your desired gtransfer options copy the gtransfer command line to a text file and add the following header to it:
#!/bin/bash #GSAT -T gtransfer
Example gtransfer job file contents:
#!/bin/bash #GSAT -T gtransfer gt -s my-source-host:/path/to/files/* \ -d my-dest-archive:/~/public/ \ -v
Start the job scheduler
To start the gsatellite scheduler - if it is not already running - issue the following command (the $ character just marks the shell prompt!):
$ gsatlcd --start
You can check that the job scheduler is running by using the --status option of gsatlcd. This should issue a message similar to the following, when it is running and nothing if it is not running.
$ gsatlcd --status gsatlc hostname="bwdahub.lsdf.kit.edu" gsatlc PID="7160"
Enabling email notifications
After starting the job scheduler but before submitting the gtransfer job, you can configure your email address to enable email notification for job start and job termination events. Create a file named myEmailAddress in the directory .gsatellite in your home directory and enter your email address.
Example contents:
john.doe@mail.domain.tld
After configuration you will receive an email after a job was started and after it terminated execution.
Job submission
For job submission you can use the gqsub command. Simply point gqsub to the gtransfer job created a few steps earlier and hit Enter.
$ gqsub gsat-jobs/gt.job 00002
Upon successful submission gqsub will return the job id of the submitted job.
Check job status
You can check the status of the job with the gqstat command. Without any options this gives a summary table of all jobs that ran or are still running:
$ gqstat job.state job.id job.execHost job.name ------------ ------ ------------------------ ------------------------ finished 00000 bwdahub.lsdf.kit.edu succeed.job.example failed 00001 bwdahub.lsdf.kit.edu fail.job.example running 00002 bwdahub.lsdf.kit.edu gt.job
With the opion -f you will get a *full* listing of all job attributes. You can limit the output to the job of interest by adding the job id:
$ gqstat -f 00002 job.id="00002" job.name="gt.job" job.dir="/home/user/.gsatellite/gschedule/jobs/00002.d" job.state="running" job.execHost="bwdahub.lsdf.kit.edu" job.pid="7491" job.start="1435048776"
Follow job execution
To follow job execution in near real-time, you can use the tail tool and the job output files available in the job directory. E.g. to follow the standard output of the running gt job you can use the following command:
$ tail -f /home/user/.gsatellite/gschedule/jobs/00002.d/job.stdout gt [.]: Transfer step: 0 gt [.]: tgftp transfer command: tgftp --log-filename "tgftp_transfer_7499__step_0.log" --force-log-o verwrite --post-command "if [[ ! -s /home/user/.gtransfer/tmp/ b3229ce5e1c76098d133ad29b5390120bc8077f1/3e9b70048804cf87ab18be4e23a aa7483b75c02f.dumpfile ]]; then rm /home/user/.gtransfer/tmp/ b3229ce5e1c76098d133ad29b5390120bc8077f1/3e9b70048804cf87ab18be4e23a aa7483b75c02f.dumpfile; fi" -- -dbg -cc 16 -tcp-bs 4M -p 16 -fast -restart -rst-retries 1 -dumpfile /home/user/.gtransfer/tmp/ b3229ce5e1c76098d133ad29b5390120bc8077f1/3e9b70048804cf87ab18be4e23a aa7483b75c02f.dumpfile -stall-timeout 30 -sync -sync-level 1 -f /home/user/.gtransfer/tmp/ b3229ce5e1c76098d133ad29b5390120bc8077f1/3e9b70048804cf87ab18be4e23a aa7483b75c02f.transferList .................................................................... .................................................................... .................................................................... .................................................................... .................................................................... .................................................................... .................................................................... .................................................................... ............................................................
You can exit by hitting Ctrl + C.
