Wednesday 7 August 2013

NagiosGraph – Graphs in Nagios on CentOS/RHEL 6.3

Nagiosgraph is a Nagios Addon that parses output and performance data from Nagios plugins and stores the data in RRD files. Nagiosgraph creates graphs and generates HTML pages with graphic reports from the data. This tutorial is tested on CentOS/RHEL 6.3, and have following Nagios core and plugin package:
   Nagios Core: nagios-4.0.0
   Nagios Plugin: nagios-plugins-1.4.16
To install Nagios please see my Nagios Installation Docs.

Installation Prerequisite

  • Nagios server in working condition.
  • We require following packages as well:
   yum -y install perl-GD php-gd rrdtool-perl rrdtool-php rrdtool perl-CGI perl-Time-HiRes

Installation

*) Create a directory where you would download nagios software:
   mkdir /usr/local/src
   cd /usr/local/src
*) Download nagiosgraph package/Untar it and navigate it to that directory
   wget http://downloads.sourceforge.net/project/nagiosgraph/nagiosgraph/1.4.4/nagiosgraph-1.4.4.tar.gz
   tar -xzvf nagiosgraph-1.4.4.tar.gz
   cd nagiosgraph-1.4.4
*) Check for any other Nagiosgraph dependency using "install.pl" script with "check-prereq" options:
   ./install.pl --check-prereq
   checking required PERL modules
     Carp...1.11
     CGI...3.51
     Data::Dumper...2.124
     File::Basename...2.77
     File::Find...1.14
     MIME::Base64...3.08
     POSIX...1.17
     RRDs...1.3008
     Time::HiRes...1.9721
   checking optional PERL modules
     GD...2.44
   checking nagios installation
     found nagios at /usr/local/nagios/bin/nagios
   checking web server installation
     found apache at /usr/sbin/httpd
Note: *) In case any dependency is missing we need to install it using cpan or yum command. *) You need to be root to run this script.

There are two ways to install nagiosgraph either automatically or manually.

1) Automated way which install file and set permission automatically
   ./install.pl --layout overlay --prefix /usr/local/nagios
   checking required PERL modules
     Carp...1.11
     CGI...3.51
     Data::Dumper...2.124
     File::Basename...2.77
     File::Find...1.14
     MIME::Base64...3.08
     POSIX...1.17
     RRDs...1.3008
     Time::HiRes... ***FAIL***
   checking optional PERL modules
     GD...2.44
   checking nagios installation
     found nagios at /usr/local/nagios/bin/nagios
   checking web server installation
     found apache at /usr/sbin/httpd
   Destination directory (prefix)? [/usr/local/nagios] 
   Location of configuration files (etc-dir)? [/usr/local/nagios/etc/nagiosgraph] 
   Location of executables? [/usr/local/nagios/libexec] 
   Location of CGI scripts? [/usr/local/nagios/sbin] 
   Location of documentation (doc-dir)? [/usr/local/nagios/docs/nagiosgraph] 
   Location of examples? [/usr/local/nagios/docs/nagiosgraph/examples] 
   Location of CSS and JavaScript files? [/usr/local/nagios/share] 
   Location of utilities? [/usr/local/nagios/docs/nagiosgraph/util] 
   Location of state files (var-dir)? [/var/nagios] 
   Location of RRD files? [/var/nagios/rrd] 
   Location of log files (log-dir)? [/var/nagios] 
   Path of log file? [/var/nagios/nagiosgraph.log] 
   Path of CGI log file? [/var/nagios/nagiosgraph-cgi.log] 
   URL of CGI scripts? [/nagios/cgi-bin] 
   URL of CSS file? [/nagios/nagiosgraph.css] 
   URL of JavaScript file? [/nagios/nagiosgraph.js] 
   Path of Nagios performance data file? [/tmp/perfdata.log] 
   URL of Nagios CGI scripts? [/nagios/cgi-bin] 
   username or userid of Nagios user? [nagios] 
   username or userid of web server user? [apache] 
   Modify the Nagios configuration? [n] 
   Modify the Apache configuration? [n] 
   configuration:
     ng_layout            overlay
     ng_prefix            /usr/local/nagios
     ng_etc_dir           /usr/local/nagios/etc/nagiosgraph
     ng_bin_dir           /usr/local/nagios/libexec
     ng_cgi_dir           /usr/local/nagios/sbin
     ng_doc_dir           /usr/local/nagios/docs/nagiosgraph
     ng_examples_dir      /usr/local/nagios/docs/nagiosgraph/examples
     ng_www_dir           /usr/local/nagios/share
     ng_util_dir          /usr/local/nagios/docs/nagiosgraph/util
     ng_var_dir           /var/nagios
     ng_rrd_dir           /var/nagios/rrd
     ng_log_dir           /var/nagios
     ng_log_file          /var/nagios/nagiosgraph.log
     ng_cgilog_file       /var/nagios/nagiosgraph-cgi.log
     ng_url               /nagios
     ng_cgi_url           /nagios/cgi-bin
     ng_css_url           /nagios/nagiosgraph.css
     ng_js_url            /nagios/nagiosgraph.js
     nagios_cgi_url       /nagios/cgi-bin
     nagios_perfdata_file /tmp/perfdata.log
     nagios_user          nagios
     www_user             apache
     modify_nagios_config n
     nagios_config_file   /usr/local/nagios/etc/nagios.cfg
     nagios_commands_file /usr/local/nagios/var/rw/nagios.cmd 
     modify_apache_config n
     apache_config_dir    /etc/httpd/conf.d
     apache_config_file   
   Continue with this configuration? [y] 
OR
2) Manual steps to install Nagiosgraph.
2.1) Create NagiosGraph directory and copy its configuration file in it:
   mkdir /usr/local/nagios/etc/nagiosgraph
   cp -vr etc /usr/local/nagios/etc/nagiosgraph/
2.2) Change NagiosGraph path in cgi and perl scripts:
   sed -i "s#/opt/nagiosgraph/etc#/usr/local/nagios/etc/nagiosgraph/#g" cgi/*cgi 
   sed -i "s#/opt/nagiosgraph/etc#/usr/local/nagios/etc/nagiosgraph/#g" lib/insert.pl
2.3) Copy cgi scripts to nagios appropriate directory that will be served by the web-server:
   cp cgi/*.cgi /usr/local/nagios/sbin
2.4) Copy insert.pl to nagios appropriate directory from which it can be executed:
   cp lib/insert.pl /usr/local/nagios/libexec
2.5) Copy CSS and JavaScript files to a nagios directory that would be served by the web server:
   cp share/nagiosgraph.css /usr/local/nagios/share
   cp share/nagiosgraph.js /usr/local/nagios/share
2.6) In NagiosGraph SSI file, set the URL for nagiosgraph.js,copy it to its appropriate location:
   vi share/nagiosgraph.ssi
   src="/nagiosgraph/nagiosgraph.js"   to    src="/nagios/nagiosgraph.js"

   cp share/nagiosgraph.ssi /usr/local/nagios/share/ssi/common-header.ssi
2.7) Replace the Nagios action icon with the nagiosgraph graph icon:
   cp share/graph.gif /usr/local/nagios/share/images/action.gif

Configuration

Edit Nagiosgraph configuration file and set at least the following parameters:
   vi /usr/local/nagios/etc/nagiosgraph/nagiosgraph.conf
    perflog           = /usr/local/nagios/var/perfdata.log
    rrddir            = /usr/local/nagios/var/rrd
    mapfile           = /usr/local/nagios/etc/nagiosgraph/map
    nagiosgraphcgiurl = /nagios/cgi-bin
    javascript        = /nagios/nagiosgraph.js
    stylesheet        = /nagios/nagiosgraph.css
    logfile           = /usr/local/nagios/var/nagiosgraph.log
    cgilogfile        = /usr/local/nagios/var/nagiosgraph-cgi.log

Creating necessary directory and fixing the permissions

Changing the ownership of conf and other necessary file that we have copyied earlier
   chown nagios:nagios /usr/local/nagios/etc/nagiosgraph
   chown nagios:nagios /usr/local/nagios/sbin
   chown nagios:nagios /usr/local/nagios/libexec/insert.pl
   chown nagios:nagios /usr/local/nagios/share/nagiosgraph.css  /usr/local/nagios/share/nagiosgraph.js
   chown nagios:nagios /usr/local/nagios/share/ssi/common-header.ssi
Created RRD directory and set permission of "rrddir" to be writable by *nagios* user and read by *apache* (webserver user).
   mkdir /usr/local/nagios/var/rrd
   chmod nagios:apache /usr/local/nagios/var/rrd
   chmod 755 /usr/local/nagios/var/rrd
Set writeable permissions to user *nagios* for “logfile” config directive:
   touch /usr/local/nagios/var/nagiosgraph.log
   chown nagios /usr/local/nagios/var/nagiosgraph.log
   chmod 644 /usr/local/nagios/var/nagiosgraph.log
Set writeable permissions to user *apache* for “cgilogfile” config directive:
   touch /usr/local/nagios/var/nagiosgraph-cgi.log
   chown apache /usr/local/nagios/var/nagiosgraph-cgi.log
   chmod 644 /usr/local/nagios/var/nagiosgraph-cgi.log
Make sure that the *nagios* user can create and delete perfdata files:

Nagios Configuration

Add the following line at the end of Nagios configuration file:
   vi /usr/local/nagios/etc/nagios.cfg

   # begin nagiosgraph configuration
   # process nagios performance data using nagiosgraph
   process_performance_data=1
   service_perfdata_file=/tmp/perfdata.log
   service_perfdata_file_template=$LASTSERVICECHECK$||$HOSTNAME$||$SERVICEDESC$||$SERVICEOUTPUT$||$SERVICEPERFDATA$
   service_perfdata_file_mode=a
   service_perfdata_file_processing_interval=30
   service_perfdata_file_processing_command=process-service-perfdata-for-nagiosgraph
   # end nagiosgraph configuration
In the Nagios commands file (commands.cfg) define the process-service-perfdata command at the end:
   vi /usr/local/nagios/etc/objects/commands.cfg
   # command to process nagios performance data for nagiosgraph
   define command {
      command_name process-service-perfdata-for-nagiosgraph
      command_line /usr/local/nagios/libexec/insert.pl
   }
Modify the Nagios sidebar (e.g. /usr/local/nagios/share/side.php) by inserting bullets under the 'Trends' heading:
<li><a href="<?php echo $cfg["cgi_base_url"];?>/trends.cgi" target="<?php echo $link_target;?>">Trends</a>
<ul>
<li><a href="<?php echo $cfg["cgi_base_url"];?>/show.cgi" target="<?php echo $link_target;?>">Graphs</a></li>

<li><a href="<?php echo $cfg["cgi_base_url"];?>/showhost.cgi" target="<?php echo $link_target;?>">Graphs by Host</a></li>
<li><a href="<?php echo $cfg["cgi_base_url"];?>/showservice.cgi" target="<?php echo $link_target;?>">Graphs by Service</a></li>

<li><a href="<?php echo $cfg["cgi_base_url"];?>/showgroup.cgi" target="<?php echo $link_target;?>">Graphs by Group</a></li>
</ul>
</li>

Validation and testing.

Verify Nagios configuration and restart it once every thing is ok
   /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
   /etc/init.d/nagios restart
Verify that nagiosgraph configuration is working by running showconfig.cgi:
   http://localhost/nagios/cgi-bin/showconfig.cgi
    OR 
http://Server-IP-Address/nagios/cgi-bin/showconfig.cgi
If everything is working up to this point, try graphing some data by running show.cgi:
   http://localhost/nagios/cgi-bin/show.cgi
    OR 
http://Server-IP-Address/nagios/cgi-bin/show.cgi
Note: If you get a 500 internal error and the issue is with Apache. If you get a “no rrd data found” error, then the configuration of nagiosgraph is incorrect.
To test if you configuration is working you can try to run the script directly:
   cd /usr/local/nagios/libexec
   ./insert.pl
To verify that there is some data in the RRD directory:
   ls /var/nagios/rrd/
   web001    web002 app001 app002 db001 db002

Adding Service to NagiosGraph

Now we need to define an action URL for the services that we want to graph, so that you can click to go directly to the graphs for that service from the Nagios Core web interface.
   define service {
      name  graphed-service
      action_url /nagiosgraph/cgi-bin/show.cgi?host=$HOSTNAME$&service=$SERVICEDESC$' onMouseOver='showGraphPopup(this)' onMouseOut='hideGraphPopup()' rel='/nagiosgraph/cgi-bin/showgraph.cgi?host=$HOSTNAME$&service=$SERVICEDESC$&period=week&rrdopts=-w+450+-j
    }
Then, you can have the services you want graphed inherit from it, as well as from any other templates they use, by adding "graphed-service" to the value for the use directive:
   define service {
 use generic-service,graphed-service
 host_name web001
 service_description PING
 check_command check_ping!100,10%!200,20%
}
You should do this for all the services for which you want graphing.
Validate the configuration file and reload the Nagios Core server once again:
   /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
   /etc/init.d/nagios reload
With this done, visiting the Service section of the web interface should include action icons after each graphed service.

Tuesday 6 August 2013

Installing Nagios Server (4.0.0) on CentOS/RHEL 6.3

Nagios is a powerful monitoring system that enables organizations to identify and resolve IT infrastructure problems before they affect critical business processes. Nagios monitors your entire IT infrastructure to ensure systems, applications, services, and business processes are functioning properly. In the event of a failure, Nagios can alert technical staff of the problem, allowing them to begin remediation processes before outages affect business processes, end-users, or customers.
This tutorial shows how to install Nagios 4.0.0-beta on CentOS/RHEL 6.3.

1) Prerequisites

After installing your CentOS/RHEL server, you have to install dependent packages to make nagios work.
   yum install httpd php gcc glibc glibc-common gd gd-devel openssl-devel 

2) Create nagios user and group by which we will be installing nagios

   useradd -m nagios
   passwd nagios
   groupadd nagcmd
   usermod -a -G nagcmd nagios
   usermod -a -G nagcmd apache

3) Installation

a) Create a directory where you would download nagios software:
   mkdir /opt/nagios
   cd /opt/nagios
b) Download nagios-core package/Untar/Compile and Install all necessary files for Nagios:
   wget http://sourceforge.net/projects/nagios/files/nagios-4.x/nagios-4.0.0/nagios-4.0.0-beta1.tar.gz
   tar -xzvf nagios-4.0.0-beta1.tar.gz
   cd nagios
   ./configure --with-command-group=nagcmd --with-httpd-conf=/etc/httpd/conf.d --with-init-dir=/etc/init.d/
   make all
   make install
   make install-init
   make install-commandmode
   make install-config
   make install-webconf
   make install-exfoliation

Note: By default nagios install itself in /usr/local/nagios

c) Create a password to log into the web interface:
   htpasswd -c /usr/local/nagios/etc/htpasswd.users nagiosadmin
d) Restart the httpd server:
   service httpd restart
e) Now we have to make nagios service to start automatically on boot using chkconfig:
   chkconfig --add nagios
   chkconfig nagios on
f) Verify if you have a good config of nagios with the command:
   /usr/local/nagios/bin/nagios -v /usr/local/nagios/etc/nagios.cfg
g) Check if there are no errors displayed; then start nagios with command:
   service nagios start
h) Now, we have to Download/Untar/Compile and Install all necessary files for Nagios-Plugin:
   cd /opt/nagios
   wget http://prdownloads.sourceforge.net/sourceforge/nagiosplug/nagios-plugins-1.4.16.tar.gz
   cd nagios-plugins-1.4.16
   ./configure --with-nagios-user=nagios --with-nagios-group=nagios
   make
   make install
i) You try to open Web Interface page of Nagios.
   http://localhost/nagios
   OR  
   http://Server-IP-Address/nagios

Note: If page does not opens then in most cases there must be two reasons:

a) IPtables is blocking it so either disable iptables or modify rule to allow 80 port.
   service iptables stop
b) SELinux is not allowing to open the page, so use following command:
   chcon -R -t httpd_sys_content_t /usr/local/nagios/sbin/
   chcon -R -t httpd_sys_content_t /usr/local/nagios/share/

Thursday 1 August 2013

Installing and Configuring 3G DataCard on CentOS


Installation steps mentioned in the Documentation are tested on CentOS 6

Follow the steps mentioned in the following order.

1) Install CentOS 6.3.

2) Following are the prerequisite package for connecting 3G USB Dongle
   wvdial
   usb_modeswitch-data
   usb_modeswitch
You can download the above package from following site on your Pen Drive or other media.
   http://mirror.centos.org/centos/6/os/x86_64/Packages/wvdial
   http://pkgs.repoforge.org/usb_modeswitch-data
   http://pkgs.repoforge.org/usb_modeswitch
3) Connect the USB 3G Dongle on the system and check the device is detected using lsusb command or in /var/log/messages.
   lsusb
   Bus 001 Device 003: ID 12d1:140c Huawei Technologies Co., Ltd.
OR
   tail -f /var/log/messages

   Aug 1 11:09:04 localhost kernel: usb 1-7: Product: HUAWEI Mobile
   Aug 1 11:09:04 localhost kernel: usb 1-7: Manufacturer: HUAWEI Technology
4) Check the device type by which modem is connected using dmesg.
   dmesg | grep tty

   usb 1-8: GSM modem (1-port) converter now attached to ttyUSB0
   usb 1-8: GSM modem (1-port) converter now attached to ttyUSB1
   usb 1-8: GSM modem (1-port) converter now attached to ttyUSB2
   usb 1-8: GSM modem (1-port) converter now attached to ttyUSB3
5) Once USB is connected and detected, now is the time to configure wvdial.
   vi /etc/wvdial.conf 
   [Dialer Defaults]
   Init1 = ATZ
   Init2 = ATQ0 V1 E1 S0=0 &C1 &D2
   Init3 = AT+CGDCONT=1,"IP","airtelgprs.com"
   stupid mode = 1
   Modem Type = USB Modem
   Baud = 460800
   New PPPD = yes
   Modem = /dev/ttyUSB0
   ISDN = 0
   Phone = *99#
   Carrier Check = no
   Password = airtel
   Username = airtel
   FlowControl=Hardware(CRTSCTS)
6) Once we have prepared the config file now is the time to connect wvdial.
   wvdial
   --> WvDial: Internet dialer version 1.60
   --> Cannot get information for serial port.
   --> Initializing modem.
   --> Sending: ATZ
   OK
   --> Sending: ATQ0 V1 E1 S0=0 &C1 &D2
   ATQ0 V1 E1 S0=0 &C1 &D2
   OK
   --> Sending: AT+CGDCONT=1,"IP","airtelgprs.com"
   AT+CGDCONT=1,"IP","airtelgprs.com"
   OK
   --> Modem initialized.
   --> Sending: ATDT*99#
   --> Waiting for carrier.
   ATDT*99#
   CONNECT
   --> Carrier detected.  Starting PPP immediately.
   --> Starting pppd at Mon Aug  5 18:55:58 2013
   --> Pid of pppd: 3070
   --> Using interface ppp0
   --> pppd: ?
   --> pppd: ?
   --> pppd: ?
   --> pppd: ?
   --> pppd: ?
   --> local  IP address 223.228.201.19
   --> pppd: ?
   --> remote IP address 10.64.64.64
   --> pppd: ?
   --> primary   DNS address 125.22.47.102
   --> pppd: ?
   --> secondary DNS address 125.22.47.103
   --> pppd: ?
6) Check the logs (/var/log/messages) and try to make entry of Primary and Secondary DNS server in /etc/resolv.conf, if it is not updated.
   Aug 1 14:35:40 localhost pppd[3794]: pppd 2.4.5 started by root, uid 0
   Aug 1 14:35:40 localhost pppd[3794]: Using interface ppp0
   Aug 1 14:35:40 localhost pppd[3794]: Connect: ppp0 <--> /dev/ttyUSB0
   Aug 1 14:35:40 localhost pppd[3794]: CHAP authentication succeeded
   Aug 1 14:35:40 localhost pppd[3794]: CHAP authentication succeeded
   Aug 1 14:35:43 localhost pppd[3794]: Could not determine remote IP address: defaulting to 10.64.64.64
   Aug 1 14:35:43 localhost pppd[3794]: local IP address 106.200.21.246
   Aug 1 14:35:43 localhost pppd[3794]: remote IP address 10.64.64.64
   Aug 1 14:35:43 localhost pppd[3794]: primary DNS address 125.22.47.102
   Aug 1 14:35:43 localhost pppd[3794]: secondary DNS address 125.22.47.103
   Aug 1 14:35:45 localhost ntpd[2903]: Listening on interface #5 ppp0, 106.200.21.246#123 Enabled
Resolv file will look like
   vi /etc/resolv.conf 
   nameserver 125.22.47.102
   nameserver 125.22.47.103
7) Congratulation now you are connected, grap a glass of bear and enjoy surfing the Net. Debugging steps *) Check device is in Storage or Modem Mode.
**) Use usb-devices or lsusb command to check.
   usb-devices
   T: Bus=01 Lev=01 Prnt=01 Port=07 Cnt=01 Dev#= 9 Spd=480 MxCh= 0
   D: Ver= 2.00 Cls=00(>ifc ) Sub=00 Prot=00 MxPS=64 #Cfgs= 1
   P: Vendor=12d1 ProdID=1446 Rev=00.00
   S: Manufacturer=HUAWEI Technology
   S: Product=HUAWEI Mobile
   C: #Ifs= 2 Cfg#= 1 Atr=e0 MxPwr=500mA
   I: If#= 0 Alt= 0 #EPs= 2 Cls=08(stor.) Sub=06 Prot=50 Driver=usb-storage
   I: If#= 1 Alt= 0 #EPs= 2 Cls=08(stor.) Sub=06 Prot=50 Driver=usb-storage
OR
   lsusb
   Bus 001 Device 003: ID 12d1:1446 Huawei Technologies Co., Ltd. E1552/E1800/E173 (HSPA modem)
"1446" is the product id for the file mode of this dongle. (It is in Storage mode).
   lsusb
   Bus 001 Device 003: ID 12d1:140c Huawei Technologies Co., Ltd.
"140c" is the product id for the file mode of this dongle. (It is in USB mode). **) Load the usbserial kernel module using following command.
   modprobe usbserial vendor=0x12d1 product=0x140c
**) usb_modeswitch package is used to switch the USB dongle from CDROM file system to Modem mode (→ flip flop). Create the following file.
   cat /etc/usb_modeswitch.conf
   DefaultVendor= 0x12d1 
   DefaultProduct=0x1446 

   TargetVendor= 0x12d1 
   TargetProductList="1001,1406,140b,140c,141b,14ac" 

   CheckSuccess=20 MessageContent="55534243123456780000000000000011060000000000000000000000000000" 
**) Now run the following command to switch from storage to modem mode.
   usb_modeswitch -c /etc/usb_modeswitch.conf
   Check the /var/log/message it will show something like
   Aug 1 14:35:08 localhost kernel: usb 1-8: configuration #1 chosen from 1 choice
   Aug 1 14:35:08 localhost kernel: option 1-8:1.0: GSM modem (1-port) converter detected
   Aug 1 14:35:08 localhost kernel: usb 1-8: GSM modem (1-port) converter now attached to ttyUSB0
   Aug 1 14:35:08 localhost kernel: option 1-8:1.1: GSM modem (1-port) converter detected
   Aug 1 14:35:08 localhost kernel: usb 1-8: GSM modem (1-port) converter now attached to ttyUSB1
   Aug 1 14:35:08 localhost kernel: option 1-8:1.2: GSM modem (1-port) converter detected
   Aug 1 14:35:08 localhost kernel: usb 1-8: GSM modem (1-port) converter now attached to ttyUSB2
   Aug 1 14:35:08 localhost kernel: option 1-8:1.3: GSM modem (1-port) converter detected
   Aug 1 14:35:08 localhost kernel: usb 1-8: GSM modem (1-port) converter now attached to ttyUSB3
   Aug 1 14:35:08 localhost kernel: scsi23 : SCSI emulation for USB Mass Storage devices
   Aug 1 14:35:08 localhost kernel: scsi24 : SCSI emulation for USB Mass Storage devices
   Aug 1 14:35:09 localhost modem-manager: (ttyUSB0) opening serial device...
**) Now try the steps mentioned from Steps 5 onwards. *) Also try to remove any other mounted usb storage media and try the steps mentioned above once again.
   Reference for further reading
   *) https://wiki.archlinux.org/index.php/3G_and_GPRS_modems_with_pppd
   *) https://wiki.archlinux.org/index.php/USB_3G_Modem