Wednesday, 18 June 2014

RRD TOOL (Round Robin Database)


In order to fetch more information like the temperature of the CPU and etcetera , we make use of RRD TOOL .
$ apt-get install rrdtool
$ apt-get install lm-sensors
RRD tool works at both the front end and the back end . At the back  end it stores the data and at the front end , it displays its graph .
In RRD , the data is stored not only in its original form but also it is consolidated and stored , this helps us to analyze the usage of a particular parameter (eg : Ram usage)
for a duration (like months/years/per day).
DATA CONSOLIDATION :
Collection of data from various sources and then storing it in one single place .
RRD stores values in a round fashion , It is more like a circle where the data is stored on the perimeter and when it reaches the start point , we re-write the data .
PARAMETERS used in RRD :
1.variable that holds the data values
2.Archives to store the consolidated values.
3.Time related parameters
Steps to create a RRD :
Install the rrdtool .
$ rrdtool   create   test.rrd \
               --start    920804700 \
               --step     300 \
                 DS :  speed : COUNTER : 600  : 0  :  671744 \
                 RRA  : AVERAGE : 0.5 : 1 : 24
                 RRA  : AVERAGE  : 0.5 : 6 : 10


This creates a round robin database called target(target.rrd) that stores one data source(DS) named speed that represents a counter from the start time :920804700 which in terms of date is : 30,March 1973 18:28:45 GMT.
Refer : www.onlineconversion.com/unix_time.htm .
This counter is ready after every 300 seconds (5 mins) .In the same data base , two round robin archives (RRA) are kept , one averages the data every time it is read ie. it takes 1 value at an interval of 5 minutes (300 sec) and create 24  rows . Hence we get an archive . The other archive is storing averaged values of 6 PDP's that makes up one CDP. And this CDP is stored in the second archive . It creates 10 such rows .
RRDtool works with the special time stamps coming from the UNIX world.This time stamp is the number of seconds that passed since January 1st 1970 UTC. 
Refer : www.epochconverter.com 
For the conversion of Dates into seconds or vice versa.
DS  : It is a data source (or) variable that stores the values .Its syntax is :
DS  : variable_name :  DST :heartbeat :min : max
The syntax for creating a round robin database can be viewed as :

$ rrdtool  create  filename  \
  --start  start_time \
  --step  step_time \
 DS :ds_name : DST : heartbeat :MAX : MIN
 RRA : CF : xff : step : rows
 Explaination :
  • filename : the name of the file that will store the values  .
  • start_time : time in seconds , from which we have to take the values .
  • step_time : time in seconds , that tells the interval time , ie , the values are stored at what interval . (In the above example , the step time was 300 seconds, hence PDP's are obtained after every 300 seconds).
  • DS : Data source ., the value that we have to store , here , it is the speed .It denotes the parameter , whose values are to be noted at specific intervals.There can be many DSs in  a database as needed .After every step interval , a new value of DS is supplied to update the database.This value is called as PDP(Primary data point).
  • After It has different data types :
     COUNTER , DERIVE , ABSOLUTE , GAUGE .
  • COUNTER : It stores the rate of change of value over a step period . This assumes that the value is always increasing and hence the difference between the current and the previous value is always greater than 0.
  • DERIVE      :  It is the same as counter type but it allows negative values as well .
  • ABSOLUTE : It saves the rate of change  , but it assumes that the previous value is set to 0. The difference between the current  and  the  previous value  is always equal to the current value .
  • GAUGE : It takes the direct value and stores the actual values.
Example :
Values : 300, 600 , 900 , 1200 
Step   : 300 seconds
COUNTER DS : 1,1,1,1
DERIVE DS : 1,1,1,1
ABSOLUTE DS : 1,2,3,4
GAUGE DS : 300, 600, 900 , 1200
Heartbeat : It is 600 seconds in the above example .If the database does not get a new PDP within 300 seconds , it will wait for 300 seconds (total 600 seconds) .If it doesnt receive any PDP within 600 seconds , it will save an UNKNOWN (representes as UN/KN/U) value into the data base .
MAX and MIN : If the variable to be stored has predictable maximum and minimum values then it should be specified here.
RRA : CF : xff : step : rows
RRA is the keyword to declare RRAs.
CF : The consolidation function can be AVERAGE, MINIMUM, MAXIMUM and LAST. The RRAs hold CDPs. If it is AVERAGE, then the average of specified number of PDPs forms a CDP and this CDP is stored in the archive. Similarly, if the function is MINIMUM, the minimum value from the specified number of PDPs will be chosen as a CDP and this is stored in the archive.


RRD UPDATE
SYNTAX:  
rrdtool update filename time : value
rrdtool update test.rrd 920804700:12345 920805000:12357 920805300:12363
rrdtool update test.rrd 920805600:12363 920805900:12363 920806200:12373
rrdtool update test.rrd 920806500:12383 920806800:12393 920807100:12399
rrdtool update test.rrd 920807400:12405 920807700:12411 920808000:12415
rrdtool update test.rrd 920808300:12420 920808600:12422 920808900:12423

We can now retrieve the data from our database using "rrdtool fetch":

rrdtool fetch filename AVERAGE --start_time --end_time

$ rrdtool fetch test.rrd AVERAGE --start 920804400 --end 920809200
$  rrdtool graph speed.png                                 \

        --start 920804400 --end 920808000               \

        DEF:  myspeed = test.rrd  :  speed  :  AVERAGE              \

        LINE2: myspeed#FF0000
$ eog speed.png
A description...
This will create speed.png which starts at 12:00 and ends at 13:00. There is a definition of a variable called myspeed, using the data from RRA "speed" out of database "test.rrd". The line drawn is 2 pixels high and represents the variable myspeed. The color is red .The colors are built up from red, green and blue. For each of the components, you specify how much to use in hexadecimal where 00 means not included and FF means fully included. The "color" white is a mixture of red, green and blue: FFFFFF The "color" black is all colors off: 000000.
  red     #FF0000
  green   #00FF00
  blue    #0000FF
  magenta #FF00FF     (mixed red with blue)
  gray    #555555     (one third of all components)
The  next command
$ rrdtool graph speed2.png                           \

     --start 920804400 --end 920808000               \

     --vertical-label m/s                            \

     DEF:myspeed=test.rrd:speed:AVERAGE              \

     CDEF:realspeed=myspeed,1000,\*                  \

     LINE2:realspeed#FF0000
A description...
$ rrdtool graph speed3.png                             \

    --start 920804400 --end 920808000               \

     --vertical-label km/h                           \

     DEF:myspeed=test.rrd:speed:AVERAGE              \

     "CDEF:kmh=myspeed,3600,*"                       \

     CDEF:fast=kmh,100,GT,kmh,0,IF                   \

     CDEF:good=kmh,100,GT,0,kmh,IF                   \

     HRULE:100#0000FF:"Maximum allowed"              \

     AREA:good#00FF00:"Good speed"                   \

     AREA:fast#FF0000:"Too fast"
A description...
$ rrdtool graph speed4.png                           \

     --start 920804400 --end 920808000               \

     --vertical-label km/h                           \

     DEF:myspeed=test.rrd:speed:AVERAGE              \

     CDEF:nonans=myspeed,UN,0,myspeed,IF             \

     CDEF:kmh=nonans,3600,*                          \

     CDEF:fast=kmh,100,GT,100,0,IF                   \

     CDEF:over=kmh,100,GT,kmh,100,-,0,IF             \

     CDEF:good=kmh,100,GT,0,kmh,IF                   \

     HRULE:100#0000FF:"Maximum allowed"              \

     AREA:good#00FF00:"Good speed"                   \

     AREA:fast#550000:"Too fast"                     \

     STACK:over#FF0000:"Over speed"
A description...
Now , we integrate snmpd and rrdtool and try to make the graphs for the data obtained from the other hosts .

No comments:

Post a Comment