Recent Posts

Hadoop Admin Handbook - APDaga

https://drive.google.com/folderview?id=0B5sp6lHFChXeeTRkMzVtSU9pSmc


1. Procedure to get open Internet:


Contact IS team, give them IP of the system and ask them to provide open internet. They will provide similar kind of proxy IP as shown below:
proxy=https://172.29.6.70:8080

Now, Go to following path:
sudo vim /etc/yum.conf

and add following line:
proxy=https://172.29.6.70:8080





Now go to:
sudo vim /etc/wgetrc

and make following changes:
use_proxy = on

http_proxy = http://172.29.6.70:8080/
#https_proxy = http://proxy.yoyodyne.com:18023/
#http_proxy = http://proxy.yoyodyne.com:18023/
#ftp_proxy = http://proxy.yoyodyne.com:18023/

Save and Close above files and try following command:
wget google.com

If it works, you got open internet successfully.




2. How to Turn ON/OFF Cluster:


Go to Master Node (10.29.9.88) and start the server:
sudo service cloudera-scm-server status / start / stop / restart

If it shows "cloudera-scm-server is dead and pid file exists", delete the following file and retry:
/var/run/cloudera-scm-server.pid



Example:
[abimadmin@01HW524743 ~]$ ssh abimadmin@10.29.9.88
abimadmin@10.29.9.88's password: 
Last login: Tue Mar 28 18:11:58 2017 from 10.29.9.48
[abimadmin@hadoop01 ~]$ sudo service cloudera-scm-server status
cloudera-scm-server (pid 22596) is running...

then start the agent:
sudo service cloudera-scm-agent status / start / stop / restart

If it shows "cloudera-scm-agent is dead and pid file exists", delete the following file and retry:
/var/run/cloudera-scm-agent/cloudera-scm-agent.pid



Example:
[abimadmin@hadoop01 ~]$ sudo service cloudera-scm-agent status
cloudera-scm-agent (pid  23599) is running...

Go to every other node on the cluster and start the agent in similar way:
sudo service cloudera-scm-agent status / start / stop / restart





3. How to solve clock offset issue?


Go to the node where this problem occurred and rum following commands:
sudo /etc/init.d/ntpd start
sudo /etc/init.d/ntpd stop

OR


sudo service nptd startsudo service nptd stop

If this doesn't work, try it other way as follows:
sudo ntpdate -v hadoop03IPecho $?

(where, hadoop03IP is the IP of the master node where Clock is server exist.)

if output is 0, then problem is solved.

else,
sudo pkill ntpd
sudo ntpdate -v hadoop03IP
echo $?





4. Change a server's hostname in CentOS


Authored by: Rackspace Support

By default, your server is started with the server’s given name as the hostname. Some software, such as cPanel, requires a valid Fully Qualified Domain Name (FQDN) for the hostname to be used during their licensing verification system. This article describes how to change a server hostname in CentOS.

Change a server’s hostname
  1. Using a text editor, open the server’s /etc/sysconfig/network file.
    # sudo nano /etc/sysconfig/network
  2. Modify the HOSTNAME= value to match your FQDN hostname.
    HOSTNAME=myserver.domain.com
  3. For internal networking, change the host that is associated with the main IP address for your server (found at /etc/hosts).
    127.0.0.1 localhost localhost.localdomain
    123.45.67.89 hostname.domain.com hostname
    ~
    ~
    ~
    ~
    -- INSERT -- 2,43-57 ALL
  4. Run the hostname command. This command lets you change the hostname on the server that the command line remembers, but it does not actively update all programs that are running under the old hostname.
    [root@defiant ~]# hostname hostname.domain.com
    [root@defiant ~]# hostname
    hostname.domain.com
    [root@defiant ~]#
  5. Restart networking on your server to ensure that changes will persist on restart..
    # /etc/init.d/network restart

No comments