IO optimizations and data access: Difference between revisions
(Created page with "== Technical specification/requirements == {| class="wikitable" width="85%" style="border-collapse: separate; border-spacing: 0; border-width: 1px; border-style: solid; border-c…") |
|||
| (13 intermediate revisions by 3 users not shown) | |||
| Line 1: | Line 1: | ||
== |
===[[Internals:IO|Technical specification/requirements]]=== |
||
Every participant will have a private node to run the exercises. It is '''not possible''' nor needed '''to run the exercises on a personal laptop''' because results are not comparable. |
|||
{| class="wikitable" width="85%" style="border-collapse: separate; border-spacing: 0; border-width: 1px; border-style: solid; border-color: #000; padding: 0" |
|||
! colspan="6" style="background:#ABE" | IO optimizations and data access |
|||
The exercises will be provided on a dynamic web page with hints and solutions. The link will be published on Tuesday. |
|||
|- |
|||
| colspan="6" style="background:#ffffcc" | Andreas-Joachim Peters (CERN) (in progress) |
|||
|- |
|||
== Exercise Overview == |
|||
! style="width: 5%; border-style: solid; border-width: 1px 1px 0 0" | # of nodes |
|||
! style="width: 20%; border-style: solid; border-width: 1px 1px 0 0" | Hardware |
|||
! style="width: 25%; border-style: solid; border-width: 1px 1px 0 0" | Software/Utilities |
|||
The tutorial will start with an introduction to tools used in Linux environments to measure and analyze IO performance. Based on few examples participants will be able to measure individual IO bottlenecks present in hard- and software. The second part will demonstrate optimizations in various parts of the local storage architecture like write-caching, read-ahead, compound reads etc. The third part will be an exercise illustrating the power and potential of Cloud storage where students will apply the technology of distributed hash tables to build a private cloud storage system. |
|||
! style="width: 30%; border-style: solid; border-width: 1px 1px 0 0" | Miscellaneous |
|||
! style="width: 10%; border-style: solid; border-width: 1px 1px 0 0" | Status (ToDo/DONE) |
|||
[[ File:IO-Exercises.pdf]] |
|||
! style="width: 10%; border-style: solid; border-width: 1px 0 0 0" | Tested (YES/NO) |
|||
[[ File:IO-Exercises+Hint.pdf ]] |
|||
|- |
|||
[[ File:IO-Exercises+Hint+Solution.pdf]] |
|||
| style="border-style: solid; border-width: 1px 1px 0 0" align="center" | 16 |
|||
[[ File:DataTechnologies-Exercises-3.pdf]] |
|||
| style="border-style: solid; border-width: 1px 1px 0 0" | |
|||
[[ File:cloud.sh]] |
|||
* HDD: ??GB |
|||
* RAM: ??GB |
|||
== Practical Information== |
|||
* ... |
|||
==== Assigned Exercise Machines ==== |
|||
| style="border-style: solid; border-width: 1px 1px 0 0" | |
|||
<pre> |
|||
* vmstat, dstat, iostat, wget, curl, ROOT, gcc compiler |
|||
gks-126 Markus Alex |
|||
| style="border-style: solid; border-width: 1px 1px 0 0" | |
|||
gks-127 Alexander Ditter |
|||
* open at least one port on each machine where other nodes can connect |
|||
gks-128 Bogdan Dumitru |
|||
* port number does not matter, could be 8080 |
|||
gks-129 Alexander Voigt |
|||
* if 16 nodes not available, two people sit together i.e. 8/9 nodes |
|||
gks-130 Alexander Yasnogor |
|||
| style="border-style: solid; border-color: #000; border-width: 1px 1px 0 0; color: red;" align="center" | ToDo |
|||
gks-131 Michael Bontenackels |
|||
| style="border-style: solid; border-color: #000; border-width: 1px 0 0 0; color: red;" align="center" | NO |
|||
gks-132 Herbert Cornelius |
|||
|} |
|||
gks-133 Daniel Funke |
|||
gks-134 Christian Gumpert |
|||
gks-135 Dominik Haitz |
|||
gks-136 Silke Halstenberg |
|||
gks-137 Christopher Jung |
|||
gks-138 Barbara Krasovec |
|||
gks-139 Roland Laifer |
|||
gks-140 Patrick Martin |
|||
gks-141 Maritn Spoo |
|||
gks-142 Sang Oh Park |
|||
gks-143 Stephan Pfister |
|||
gks-145 Angela Poschlad |
|||
gks-147 Christian Pulvermacher |
|||
gks-037 Doris Ressmann |
|||
gks-038 Martin Spoo |
|||
gks-039 Alexander Voigts |
|||
gks-042 Andreas Wirooks |
|||
gks-043 Manuel Zeise |
|||
</pre> |
|||
If possible create an SSH tunnel from your laptop to your assigned exercise machine like |
|||
<pre> |
|||
ssh -p 24 -L 8000:gks-017:80 gks115@gks-144 # => username@exercisemachine |
|||
</pre> |
|||
Open the follwing address in your browser: [http://localhost:8000/dm http://localhost:8000/dm] |
|||
If you are not able to do that, open this page to see the exercise to read the exercises online: [http://gks-017/dm http://gks-017/dm] |
|||
<br> |
|||
<br> |
|||
Open atleast three shell windows |
|||
<pre> |
|||
ssh -p 24 gks115@gks-144 |
|||
</pre> |
|||
Latest revision as of 16:26, 28 August 2012
Technical specification/requirements
Every participant will have a private node to run the exercises. It is not possible nor needed to run the exercises on a personal laptop because results are not comparable.
The exercises will be provided on a dynamic web page with hints and solutions. The link will be published on Tuesday.
Exercise Overview
The tutorial will start with an introduction to tools used in Linux environments to measure and analyze IO performance. Based on few examples participants will be able to measure individual IO bottlenecks present in hard- and software. The second part will demonstrate optimizations in various parts of the local storage architecture like write-caching, read-ahead, compound reads etc. The third part will be an exercise illustrating the power and potential of Cloud storage where students will apply the technology of distributed hash tables to build a private cloud storage system.
File:IO-Exercises.pdf File:IO-Exercises+Hint.pdf File:IO-Exercises+Hint+Solution.pdf File:DataTechnologies-Exercises-3.pdf File:Cloud.sh
Practical Information
Assigned Exercise Machines
gks-126 Markus Alex gks-127 Alexander Ditter gks-128 Bogdan Dumitru gks-129 Alexander Voigt gks-130 Alexander Yasnogor gks-131 Michael Bontenackels gks-132 Herbert Cornelius gks-133 Daniel Funke gks-134 Christian Gumpert gks-135 Dominik Haitz gks-136 Silke Halstenberg gks-137 Christopher Jung gks-138 Barbara Krasovec gks-139 Roland Laifer gks-140 Patrick Martin gks-141 Maritn Spoo gks-142 Sang Oh Park gks-143 Stephan Pfister gks-145 Angela Poschlad gks-147 Christian Pulvermacher gks-037 Doris Ressmann gks-038 Martin Spoo gks-039 Alexander Voigts gks-042 Andreas Wirooks gks-043 Manuel Zeise
If possible create an SSH tunnel from your laptop to your assigned exercise machine like
ssh -p 24 -L 8000:gks-017:80 gks115@gks-144 # => username@exercisemachine
Open the follwing address in your browser: http://localhost:8000/dm
If you are not able to do that, open this page to see the exercise to read the exercises online: http://gks-017/dm
Open atleast three shell windows
ssh -p 24 gks115@gks-144
