MongoDB as an In-memory Sharded Database
The Data Life-Cycle Lab Earth and Environment at KIT manages data from Earth-observing climate satellites such as Envisat MIPAS, which is stored in the NoSQL database MongoDB. The size of data in question makes it necessary for it to be distributed across a large database cluster, using sharding - a using horizontal-partitioning solution available in MongoDB.
A typical MongoDB instance makes heavy use of available system memory but ultimately relies on underlying persistent storage. At the same time, our database cluster offers only low-performance persistent storage which is unsuitable for sustained load. It is therefore required to have MongoDB on our cluster operate as an *in-memory database*. Your task will be to to research optimal configuration of MongoDB for in-memory operation, develop tools for the initial population of cluster nodes as well as periodic commits of their data to persistent storage, and finally to evaluate the performance of the system.
- basic administration of Linux/Unix systems
- familiarity with MongoDB and/or sharding would be a plus
For more information, please contact Marek Szuba.