Sachin Sharma Blog: 2014

Tuesday, 7 January 2014

Part 3: Understanding Chef Cookbook/Recipe

This article will help you in installing packages on system via chef recipe.

Please refer my previous article on chef:

Part 1	Install/Setup and configure Chef Server/Workstation/Node on CentOS/RHEL 6.4
Part 2	Understanding Chef Cookbook/Recipe.

I) Extending existing Cookbook (cookbook-test) and configure a recipe for deploying Packages.

Login to Workstation node which have knife configured.
Lets say you have not yet install sendmail on your system, check it out using following command:
```
   # rpm -qa | grep -v grep | grep -i sendmail
```
Note: In case if sendmail is installed and you want to delete it please refer to my rpm page.

Now lets create a recipe for a sendmail package.

   # vim /usr/local/src/chef/cookbooks/cookbook-test/recipes/sendmail.rb
     package 'sendmail' do
        action :install
     end

To upload a cookbook (cookbook-test) use following command:
```
   # knife upload cookbooks cookbook-test
```

Once we have upload the cookbook, now is the time to associate it with a Node using “run_list” Knife option:

   # knife node run_list add node1.example.com recipe[cookbook-test::sendmail]
     node1.example.com:
       run_list:
         recipe[cookbook-test]
         recipe[cookbook-test::sendmail]

Now login to machine “node1.example.com” and run the following command:

   # chef-client
     Starting Chef Client, version 11.6.2
     [2013-10-25T22:05:20-07:00] INFO: *** Chef 11.6.2 ***
     [2013-10-25T22:05:22-07:00] INFO: Run List is [recipe[cookbook-test], recipe[cookbook-test::sendmail]]
     [2013-10-25T22:05:22-07:00] INFO: Run List expands to [cookbook-test, cookbook-test::sendmail]
     [2013-10-25T22:05:22-07:00] INFO: Starting Chef Run for node1.example.com
     [2013-10-25T22:05:22-07:00] INFO: Running start handlers
     [2013-10-25T22:05:22-07:00] INFO: Start handlers complete.
     resolving cookbooks for run list: ["cookbook-test", "cookbook-test::sendmail"]
     [2013-10-25T22:05:22-07:00] INFO: Loading cookbooks [cookbook-test]
     Synchronizing Cookbooks:
     [2013-10-25T22:05:22-07:00] INFO: Storing updated cookbooks/cookbook-test/recipes/default.rb in the cache.
       - cookbook-test
     Compiling Cookbooks...
     Converging 3 resources
     Recipe: cookbook-test::default
       * group[system-admins] action create[2013-10-25T22:05:22-07:00] INFO: Processing group[system-admins] action create (cookbook-test::default line 9)
      (up to date)
       * user[sachin] action create[2013-10-25T22:05:22-07:00] INFO: Processing user[sachin] action create (cookbook-test::default line 13)
      (up to date)
    Recipe: cookbook-test::sendmail
       * package[sendmail] action install[2013-10-25T22:05:22-07:00] INFO: Processing package[sendmail] action install (cookbook-test::sendmail line 1)
     [2013-10-25T22:06:14-07:00] INFO: package[sendmail] installing sendmail-8.14.4-8.el6 from base repository

         - install version 8.14.4-8.el6 of package sendmail

     [2013-10-25T22:06:28-07:00] INFO: Chef Run complete in 66.492800835 seconds
     [2013-10-25T22:06:28-07:00] INFO: Running report handlers
     [2013-10-25T22:06:28-07:00] INFO: Report handlers complete
     Chef Client finished, 1 resources updated

Check the sendmail rpm is installed on the system.

   # rpm -qa | grep -i send
     sendmail-8.14.4-8.el6.x86_64

II) Now update the recipe to start the service after installing the sendmail package.

Edit the recipe file as per following rule:

   # vim /usr/local/src/chef/cookbooks/cookbook-test/recipes/sendmail.rb
     package 'sendmail' do
        action :install
     end

     service 'sendmail' do
        action [ :enable,:start ]
     end

To upload a cookbook (cookbook-test) use following command:
```
   # knife upload cookbooks cookbook-test
```

Now login to machine “node1.example.com” and run the following command and check the output of this command:

   # chef-client
     Recipe: system-users::sendmail
       * package[sendmail] action install[2013-10-28T04:12:05-07:00] INFO: Processing package[sendmail] action install (system-users::sendmail line 1)
      (up to date)
       * service[sendmail] action enable[2013-10-28T04:12:10-07:00] INFO: Processing service[sendmail] action enable (system-users::sendmail line 5)
      (up to date)
       * service[sendmail] action start[2013-10-28T04:12:11-07:00] INFO: Processing service[sendmail] action start (system-users::sendmail line 5)
     [2013-10-28T04:12:11-07:00] INFO: service[sendmail] started

         - start service service[sendmail]

NOTE: Please check the comment marked in red.

III) Now update the recipe to delete the sendmail package.

Edit the recipe file as per following rule:

   # vim /usr/local/src/chef/cookbooks/cookbook-test/recipes/sendmail.rb
     package 'sendmail' do
        action :remove
     end

To upload a cookbook (cookbook-test) use following command:
```
   # knife upload cookbooks cookbook-test
```

Now login to machine “node1.example.com” and run the following command and check the output of this command:

   # chef-client
     Recipe: system-users::sendmail
       * package[sendmail] action remove[2013-10-28T04:16:59-07:00] INFO: Processing package[sendmail] action remove (system-users::sendmail line 1)
     [2013-10-28T04:17:09-07:00] INFO: package[sendmail] removed

         - remove  package sendmail

Part 2: Understanding Chef Cookbook/Recipe.

This article will guide you through the creation of Chef Cookbook/Recipe and how to deploy it on CentOS/RHEL 6.4.

The procedure mentioned in this tutorial is tested on:

OS	CentOS 6.4
Chef Server	11.0.8
Knife	11.6.0

What is a Cookbook?
A cookbook is the fundamental unit of configuration and policy distribution. Each cookbook defines a scenario, such as everything needed to install and configure MySQL, and then it contains all of the components that are required to support that scenario.

What is a Recipe?
Recipe files are Ruby applications that define everything that is required to configure a system, including creating and configuring folders, installing and configuring packages, starting services, and so on. A recipe is a subset or "piece" of a cookbook

What is a Attribute?
Attributes files contain a set of attributes that represent values to be used by the recipes and templates. For example, the built-in cookbook for the Rails App Server layer includes an attributes file with values for the Rails version, the application server stack, and so on.

What is Template?
Template files are templates that recipes use to create other files, such as configuration files. Template files typically let you modify the configuration file by overriding attributes—which can be done without touching the cookbook—instead of rewriting a configuration file. The standard practice is that whenever you expect to change a configuration file on an instance even slightly, you should use a template file.

What is Databags?
A data bag is a global variable that is stored as JSON data and is accessible from a server. A data bag is indexed for searching and can be loaded by a recipe or accessed during a search. The contents of a data bag can vary, but they often include sensitive information (such as database passwords).

What is knife?
Knife is a command-line tool that provides an interface between a local chef-repo and the server. Knife helps users to manage nodes, cookbook, recipes, roles etc.

Steps to create a Sample Cookbook and configure a recipe

Login to Workstation node which have knife configured.
Adding the following line to create cookbook repo that can be uploaded to git (for version control).
```
   # vi /root/.chef/knife.rb
      cookbook_path [ '/usr/local/src/chef/cookbooks' ]
```

Create the cookbook directory.

   # mkdir -p /usr/local/src/chef/cookbooks

Now lets create sample cookbook to push users to Chef Nodes:
```
   # knife cookbook create cookbook-test
```

Navigate to cookbook directory and you will see the following structure got created.

   # cd /usr/local/src/chef/cookbooks
   # tree cookbook-test
 cookbook-test/
 ├── attributes
 ├── CHANGELOG.md
 ├── definitions
 ├── files
 │   └── default
 ├── libraries
 ├── metadata.rb
 ├── providers
 ├── README.md
 ├── recipes
 │   └── default.rb
 ├── resources
 └── templates
  └── default

Before creating the recipe lets generate the password for the new user using the following commands.
```
   # openssl passwd -1 "theplaintextpassword"
```

Now lets create a recipe for a new group (system-admins) and user by the name "sanjay".

 # cat /var/chef/cookbooks/cookbook-test/recipes/default.rb
 #
 # Cookbook Name:: cookbook-test
 # Recipe:: default
 #
 # Copyright 2013, YOUR_COMPANY_NAME
 #
 # All rights reserved - Do Not Redistribute
 #
 group "system-admins" do
  gid 1001
 end
 user "sanjay" do
  comment "Sanjay User"
  shell "/bin/bash"
  home "/home/sanjay"
  gid "system-admins"
  uid 1002
  supports :manage_home => true
  password "$1$QwuUa80Z$KZkYq8CqICVyIsK1tHZ7s0"
 end

Note: Please check the Group resource and User resource page for more info.

To upload the cookbooks/directory to the server, browse to the top level of the chef-repo and enter:
```
   # knife upload cookbooks
```
Note: This will upload all the cookbook.
To upload a single cookbook use following command:
```
   # knife upload cookbooks cookbook-test
```

Once we have upload the cookbook, now is the time to associate it with a Node using "run_list" Knife option:

   # knife node list
     node1.example.com
     node2.example.com
     node3.example.com

   #  knife node run_list add node1.example.com cookbook-test
      node1.example.com:
        run_list: recipe[cookbook-test]

Now login to machine "node1.example.com" and run the following command:

   # chef-client
     [2013-10-25T04:47:36-07:00] INFO: Forking chef instance to converge...
     Starting Chef Client, version 11.6.2
     [2013-10-25T04:47:36-07:00] INFO: *** Chef 11.6.2 ***
     [2013-10-25T04:47:37-07:00] INFO: Run List is [recipe[cookbook-test]]
     [2013-10-25T04:47:37-07:00] INFO: Run List expands to [cookbook-test]
     [2013-10-25T04:47:37-07:00] INFO: Starting Chef Run for node1.example.com
     [2013-10-25T04:47:37-07:00] INFO: Running start handlers
     [2013-10-25T04:47:37-07:00] INFO: Start handlers complete.
     resolving cookbooks for run list: ["cookbook-test"]
     [2013-10-25T04:47:37-07:00] INFO: Loading cookbooks [cookbook-test]
     Synchronizing Cookbooks:
     [2013-10-25T04:47:37-07:00] INFO: Storing updated cookbooks/cookbook-test/recipes/default.rb in the cache.
     [2013-10-25T04:47:37-07:00] INFO: Storing updated cookbooks/cookbook-test/metadata.rb in the cache.
     [2013-10-25T04:47:37-07:00] INFO: Storing updated cookbooks/cookbook-test/README.md in the cache.
     [2013-10-25T04:47:37-07:00] INFO: Storing updated cookbooks/cookbook-test/CHANGELOG.md in the cache.
       - cookbook-test
     Compiling Cookbooks...
     Converging 1 resources
     Recipe: cookbook-test::default
    * group[system-admins] action create[2013-10-25T22:23:38-07:00] INFO: Processing group[system-admins] action create (cookbook-test::default line 9)
       (up to date)
       * user[sanjay] action create[2013-10-25T04:47:37-07:00] INFO: Processing user[sanjay] action create (cookbook-test::default line 9)
       (up to date)
     [2013-10-25T04:47:37-07:00] INFO: Chef Run complete in 0.48225768 seconds
     [2013-10-25T04:47:37-07:00] INFO: Running report handlers
     [2013-10-25T04:47:37-07:00] INFO: Report handlers complete
     Chef Client finished, 0 resources updated

Note: Please check the Knife node run_list page for more info.

Try to check the user got created using following command:

   # su - sanjay
   $ id
     uid=1002(sanjay) gid=1001(system-admins) groups=1001(system-admins) context=unconfined_u:unconfined_r:unconfined_t:s0-s0:c0.c1023
   $ whoami
     sanjay
   $ pwd
     /home/sanjay

Part 1: Install/Setup and configure Chef Server/Workstation/Node on CentOS/RHEL 6.4

This article will guide you through the installation and configuration steps of Chef Server/Workstation/Node on CentOS/RHEL 6.4.

The procedure mentioned in this tutorial is tested on:

OS	CentOS 6.4
Chef Server	11.0.8
Knife	11.6.0

What is Chef?

Chef is a Ruby-based configuration management engine. It acts as a hub, ensuring that the right cookbooks are used, that the right policies are applied, that all of the node objects are up-to-date, and that all of the nodes that will be maintained are registered and known to the Chef Server. The Chef Server distributes configuration details (such as recipes, templates, and file distributions) to every node within the organization. Chef then does as much of the configuration work as possible on the nodes themselves (and not on the Chef Server). This scalable approach distributes the configuration effort throughout the organization.

Chef Server:

The server acts as a hub for configuration data. The server stores cookbooks, the policies that are applied to nodes, and metadata that describes each registered node that is being managed by the chef-client. Nodes use the chef-client to ask the server for configuration details, such as recipes, templates, and file distributions. Starting with the release of Chef 11.x, the front-end for the server is written using Erlang.

Workstations:

A workstation is a computer that is configured to run Knife, to synchronize with the chef-repo, and interact with a single server. The workstation is the location from which most users will do most of their work, including:

Developing cookbooks and recipes (and authoring them using Ruby).
Keeping the chef-repo synchronized with version source control.
Using Knife to upload items from the chef-repo to the server.
Configuring organizational policy, including defining roles and environments and ensuring that critical data is stored in data bags.
Interacting with nodes, as (or when) required, such as performing a bootstrap operation.

Node:

A node is any server or virtual server that is configured to be maintained by a chef-client. A node can be any physical, virtual, or cloud machine that can run the chef-client. A chef-client is an agent that runs locally on every node that is registered with the server. When a chef-client is run, it will perform all of the steps that are required to bring the node into the expected state, including:

Registering and authenticating the node with the server.
Building the node object.
Synchronizing cookbooks.
Compiling the resource collection by loading each of the required cookbooks, including recipes, attributes, and all other dependencies.
Taking the appropriate and required actions to configure the node.
Looking for exceptions and notifications, handling each as required.

RSA public key-pairs are used to authenticate the chef-client with the server every time a chef-client needs access to data that is stored on the server. This prevents any node from accessing data that it shouldn’t and it ensures that only nodes that are properly registered with the server can be managed.

I) Prerequisite

Host should have fully configured hostname.
Should have DNS entry in place.
Following package are required.

   # yum install wget curl -y

II) Chef Server Installation

Go to http://www.opscode.com/chef/install.
Click the Chef Server tab.
Select the operating system, version, and architecture.
Select the version of Chef Server 11.x to download, and then click the link that appears to download the package.
Install the downloaded package using the correct method for the operating system on which Chef Server 11.x will be installed.
```
   # rpm -ivh https://opscode-omnibus-packages.s3.amazonaws.com/el/6/x86_64/chef-server-11.0.8-1.el6.x86_64.rpm
```
Configure Chef Server 11.x by running the following command:
```
   # chef-server-ctl reconfigure
```
The *chef-server-ctl* command will set up all of the required components, including Erchef, RabbitMQ, PostgreSQL, and all of the cookbooks that are used by chef to maintain Chef Server 11.x.
Verify the the hostname for the server by running the *hostname* command. The hostname for the server must be a FQDN.
```
   # hostname
```
Verify the installation of Chef Server 11.x by running the following command:
```
   # chef-server-ctl test
```
Note: Try to stop apache before running this test.
You can explore the Chef Server URL using your favorite browser:
```
   # https://FQDN-OR-IP-OF-CHEF-SERVER
```
Note: Default UserName/Password is admin/p@ssw0rd1
The *chef-server-ctl* command is used on the Chef Server system for management. It has built-in help (-h) that will display the various sub-commands.

II) Chef WorkStation Installation

Run the following command that appears (for UNIX and Linux environments):

   # curl -L https://www.opscode.com/chef/install.sh | bash
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                 Dload  Upload   Total   Spent    Left  Speed
      101  6790  101  6790    0     0   3826      0  0:00:01  0:00:01 --:--:-- 12190
      Downloading Chef  for el...
      Installing Chef
      warning: /tmp/tmp.KnyQTnqz/chef-.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
      Preparing...                ########################################### [100%]
      1:chef                   ########################################### [100%]
      Thank you for installing Chef!

When the installation is finished enter the *chef-client* command to verify that the chef-client was installed:
```
   # chef-client -v
      Chef: 11.6.0
```

Create the ".chef" directory The .chef directory is used to store three files:

knife.rb
ORGANIZATION-validator.pem
USER.pem

a) Copy Cert Keys from Chef Server to your Workstation User Folder:

   $ mkdir ~/.chef
   $ scp root@chef-server:/etc/chef-server/admin.pem ~/.chef
   $ scp root@chef-server:/etc/chef-server/chef-validator.pem ~/.chef

b) Now we will configure the Client setting using *knife* command.

   $ knife configure -i
      Overwrite /root/.chef/knife.rb? (Y/N) y
      Please enter the chef server URL: [https://test.example.com:443] https://chef-server.example.com:443/
      Please enter a name for the new user: [root] knife-user1
      Please enter the existing admin name: [admin] Enter
      Please enter the location of the existing admin's private key: [/etc/chef-server/admin.pem] ~/.chef/admin.pem
      Please enter the validation clientname: [chef-validator]
      Please enter the location of the validation key: [/etc/chef-server/chef-validator.pem] ~/.chef/chef-validator.pem
      Please enter the path to a chef repository (or leave blank):
      Creating initial API user...
      Please enter a password for the new user:
      Created user[knife-user1]
      Configuration file written to /root/.chef/knife.rb

c) Your Knife config file (knife.rb) will look like:

   $ cat ~/.chef/knife.rb
      log_level                :info
      log_location             STDOUT
      node_name                'knife-user1'
      client_key               '/root/.chef/knife-user1.pem'
      validation_client_name   'chef-validator'
      validation_key           '/root/.chef/admin.pem'
      chef_server_url          'https://chef-server.example.com:443/'
      syntax_check_cache_path  '/root/.chef/syntax_check_cache'

d) Verify the install by running the following commands to ensure that every chef-client and user was registered correctly.

   $ knife client list
      chef-validator
      chef-webui

   $ knife user list
      admin
      knife-user1

III) Chef Node Installation

Run the following command that appears (for UNIX and Linux environments):

   # curl -L https://www.opscode.com/chef/install.sh | bash
      % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                 Dload  Upload   Total   Spent    Left  Speed
      101  6790  101  6790    0     0   3826      0  0:00:01  0:00:01 --:--:-- 12190
      Downloading Chef  for el...
      Installing Chef
      warning: /tmp/tmp.KnyQTnqz/chef-.x86_64.rpm: Header V4 DSA/SHA1 Signature, key ID 83ef826a: NOKEY
      Preparing...                ########################################### [100%]
      1:chef                   ########################################### [100%]
      Thank you for installing Chef!

Create the Chef Directory.
```
   # mkdir /etc/chef
```
Copy Chef Server Validation Cert Keys from Chef Server to your Node at "/etc/chef":
```
   # scp root@chef-server:/etc/chef-server/chef-validator.pem /etc/chef
```
Log in to Chef client and run the following command in order for a client to register itself with Chef Server:
```
   # chef-client -S https://FQDN-OR-IP-OF-CHEF-SERVER -K /etc/chef/chef-validator.pem
```

Once the client is verified, we need to create a "client.rb" file inside "/etc/chef".

   # vi /etc/chef/client.rb
     log_level        :info
         log_location     STDOUT
         chef_server_url  'https://FQDN-OR-IP-OF-CHEF-SERVER'

Verify the Node is successfully registered with Chef Server using:
a) From Workstation Machine:
```
   # knife node list
```
b) From Chef Server Web UI (Node List):
```
   # https://FQDN-OR-IP-OF-CHEF-SERVER
```
Run the Chef Client to check if the respective cookbook (recipe's) are pushed to that node:
```
   # chef-client
   # chef-client -l debug (In case if you want to debug)
```
Starts the chef-client which will poll the chef-server every 3600 seconds for changes.
```
   # chef-client -i 3600
```

Monday, 6 January 2014

Recover/Change/ Reset MySQL root password

This article will guide you through the steps required to Recover/Change/Reset MySQL root password.

The procedure mentioned in this tutorial is tested on:

OS	Ubuntu 12.04
MySQL Server	5.5.32
MySQL Client	5.5.32

1) Stop the MySQL demon process using command:

   # /etc/init.d/mysql stop

     Rather than invoking init scripts through /etc/init.d, use the service(8)
     utility, e.g. service mysql stop

     Since the script you are attempting to invoke has been converted to an
     Upstart job, you may also use the stop(8) utility, e.g. stop mysql
     mysql stop/waiting

2) Start the MySQL demon process using "--skip-grant-tables"option so that it will not prompt for password and "--skip-networking"option to disable mysql networking functionality.

   # /usr/sbin/mysqld --skip-grant-tables --skip-networking &
     [1] 27946

3) Start the MySQL client process on another terminal using this command:

   # mysql -u root
     Welcome to the MySQL monitor.  Commands end with ; or \g.
     Your MySQL connection id is 1
     Server version: 5.5.32-0ubuntu0.12.04.1 (Ubuntu)

     Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

     Oracle is a registered trademark of Oracle Corporation and/or its
     affiliates. Other names may be trademarks of their respective
     owners.

     Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

     mysql>

4) Setup new MySQL root account password:

   # mysql> use mysql
     Reading table information for completion of table and column names
     You can turn off this feature to get a quicker startup with -A

     Database changed
   # mysql> update user set password=PASSWORD("NEW-PASSWORD") where User='root';
     Query OK, 4 rows affected (0.00 sec)
     Rows matched: 4  Changed: 4  Warnings: 0

   # mysql> flush privileges;
     Query OK, 0 rows affected (0.00 sec)

   # mysql> quit
     Bye

Note: You have to type the sql statement specified in RED color. 5) Stop/kill the MySQL Server process:

   # ps -ef | grep -i mysql
     mysql    27946 27863  0 10:10 pts/1    00:00:00 /usr/sbin/mysqld --skip-grant-tables --skip-networking
   # kill -9 27946

6) Exit and restart the MySQL server daemon.

   # /etc/init.d/mysql start
     Rather than invoking init scripts through /etc/init.d, use the service(8)
     utility, e.g. service mysql start

     Since the script you are attempting to invoke has been converted to an
     Upstart job, you may also use the start(8) utility, e.g. start mysql
     mysql start/running, process 28231

7) Verify that the root password is changed using following command:

   # mysql -u root -p
     Enter password:
     Welcome to the MySQL monitor.  Commands end with ; or \g.
     Your MySQL connection id is 71
     Server version: 5.5.32-0ubuntu0.12.04.1 (Ubuntu)

     Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

     Oracle is a registered trademark of Oracle Corporation and/or its
     affiliates. Other names may be trademarks of their respective
     owners.

     Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

     mysql> quit
     Bye