Datareduction/Downsampling in InfluxDB: Difference between revisions

From Lsdf
Jump to navigationJump to search
(Created page with "= Description = InfluxDB [0] is used extensively for monitoring of metric of several infrastructures operated by SCC like the GridKa Tier-1 Center and the Large Scale Data Fa...")
 
No edit summary
 
Line 12: Line 12:
* basic understanding of time series databases
* basic understanding of time series databases
* good knowledge of Python
* good knowledge of Python

= Supervisors =
Andreas Petzold petzold@kit.edu


= References =
= References =

Latest revision as of 10:30, 11 April 2019

Description

InfluxDB [0] is used extensively for monitoring of metric of several infrastructures operated by SCC like the GridKa Tier-1 Center and the Large Scale Data Facility (LSDF). Grafana is used to display the metrics from InfluxDB [2]. In contrast to other time series DBs, InfluxDB does not automatically downsample data to a coarse time resolution after a while. Since storage space is limited, data has to be downsampled or removed from the database eventually. The goal of the project is to prepare an overview of the possibilities for data reduction/downsampling in InfluxDB, test the downsampling, and convert existing Grafana dashboards to use the downsampled metrics.

Tasks

  • familiarization with InfluxDB, Kapacitor [3] and Grafana
  • investigation of possibilities to develop helper tools to facilitate downsampling of generic metrics
  • implementation of data reduction strategies in a test environment
  • implementation of conversion scripts for existing Grafana dashboards

Requirements

  • basic understanding of time series databases
  • good knowledge of Python

Supervisors

Andreas Petzold petzold@kit.edu

References

[0] InfluxDB
[1] Downsampling
[2] Grafana
[3] Kapacitor