0798-7.1.2-如何在Redhat7.8中安装CDP DC

文档编写目的

CDP DC是Cloudera与Hortonworks合并后，第一个融合CDH和HDP所有组件的on-premise版本，CDP Data Center主要由Cloudera Runtime构成，Cloudera Runtime由超过35个开源项目组成，当然CDP Data Center还包括其它功能如管理功能Cloudera Manager，Key Management，专业支持等，如下图所示：

Cloudera Runtime的主要组件版本如下图所示：

本文档Fayson主要描述如何在Redhat7.8安装CDP DC7.1.2。CDP DC7与CDH5/6的安装步骤一致，主要包括以下四部分：

1.安全前置准备，包括安装操作系统、关闭防火墙、同步服务器时钟等；

2.外部数据库如MySQL安装

3.安装Cloudera Manager；

4.安装CDH集群；

• 测试环境

1.CM和CDH版本为7.1.2

2.Redhat7.8

3.OpenJDK1.8

4.MariaDB-5.5.64

5.root用户安装

前置准备

2.1 hostname及hosts配置

集群中各个节点之间能互相通信使用静态IP地址。IP地址和主机名通过/etc/hosts配置，主机名通过/etc/hostname进行配置。

以cm节点（172.24.27.124）为例：

1.hostname配置，建议你将hostname配置为有意义的字母或单词组合，比如cdp01，请不要使用localhost作为主机名，也不要使用下划线或者其它特殊字符作为主机名。/etc/hostname文件示例如下：

[root@fayson0001 ~]# cat /etc/hostname
fayson0001

2.hosts配置需要将集群的所有主机都加入进来，包括客户端节点，管理节点和工作节点。/etc/hosts文件示例如下:

[root@fayson0001 ~]# cat /etc/hosts
::1     localhost       localhost.localdomain   localhost6      localhost6.localdomain6
127.0.0.1       localhost       localhost.localdomain   localhost4      localhost4.localdomain4


172.24.27.124 fayson0001
172.24.27.121 fayson0002
172.24.27.123 fayson0003
172.24.27.122 fayson0004

以上两步操作，在集群中其它节点做相同配置。确认CM的hosts文件已经同步到集群所有节点：

2.2禁用SELinux

在所有节点执行setenforce 0 命令，此处使用批处理shell执行：

[root@fayson0001 shell]# sh ssh_do_all.sh node.list "setenforce 0"

集群所有节点修改/etc/selinux/config文件如下：

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected. 
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

2.3关闭防火墙

集群所有节点执行 systemctl stop命令，此处通过shell批量执行命令如下：

[root@fayson0001 shell]# sh ssh_do_all.sh node.list "systemctl stop firewalld"
[root@fayson0001 shell]# sh ssh_do_all.sh node.list "systemctl disable firewalld"
[root@fayson0001 shell]# sh ssh_do_all.sh node.list "systemctl status firewalld"

2.4集群时钟同步

在Redhat7.x的操作系统上，已经默认的安装了chrony，我们这里先卸载chrony，然后安装ntp。使用ntp来配置各台机器的时钟同步，将cm（172.24.27.124）服务作为本地ntp服务器，其它3台服务器与其保持同步。

1.所有机器卸载chrony

[root@fayson0001 shell]# sh ssh_do_all.sh node.list "yum -y remove chrony"

2.所有机器安装ntp

[root@fayson0001 shell]# sh ssh_do_all.sh node.list "yum -y install ntp"

3.cm机器配置时钟与自己同步

[root@ fayson0001 shell]# vim /etc/ntp.conf
#server 0.rhel.pool.ntp.org iburst
#server 1.rhel.pool.ntp.org iburst
#server 2.rhel.pool.ntp.org iburst
#server 3.rhel.pool.ntp.org iburst
server  127.127.1.0     # local clock
fudge   127.127.1.0 stratum 10

4.集群其它节点，配置找cm机器去同步

[root@ip-172-31-11-232 ~]# vim /etc/ntp.conf
#server 0.rhel.pool.ntp.org iburst
#server 1.rhel.pool.ntp.org iburst
#server 2.rhel.pool.ntp.org iburst
#server 3.rhel.pool.ntp.org iburst
server 172.24.27.124

5.重启所有机器的ntp服务

[root@ fayson0001 shell]# sh ssh_do_all.sh node.list "systemctl restart ntpd"
[root@ fayson0001 shell]# sh ssh_do_all.sh node.list "systemctl enable ntpd"
[root@ fayson0001 shell]# sh ssh_do_all.sh node.list "systemctl status ntpd"

6.验证始终同步，在所有节点执行ntpq -p命令，如下使用脚本批量执行

[root@fayson0001 shell]# sh ssh_do_all.sh node.list "ntpq -p"

左边出现*号表示同步成功。

2.5设置swap

1.在所有机器执行以下命令以临时设置swap为1，并即时生效

sh ssh_do_all.sh node.list "sysctl -a | grep vm.swappiness"
sh ssh_do_all.sh node.list "echo 1 > /proc/sys/vm/swappiness"
sh ssh_do_all.sh node.list "sysctl -a | grep vm.swappiness"

确保最后swap打印应该都为1

2.为所有机器永久设置swap为1，修改/etc/sysctl.conf中vm.swappiness为1，没有则新增。

[root@fayson0001 ~]# vim /etc/sysctl.conf
vm.swappiness = 1

将/etc/sysctl.conf文件同步到集群所有机器

2.6设置透明大页面

1.所有节点执行以下命令关闭透明大页面，并即时生效

sh ssh_do_all.sh node.list "echo never > /sys/kernel/mm/transparent_hugepage/defrag"
sh ssh_do_all.sh node.list  "echo never > /sys/kernel/mm/transparent_hugepage/enabled"
sh ssh_do_all.sh node.list  "cat /sys/kernel/mm/transparent_hugepage/enabled"
sh ssh_do_all.sh node.list "cat /sys/kernel/mm/transparent_hugepage/defrag"

2.修改所有节点的/etc/rc.d/rc.local文件的权限以实现开机执行

[root@fayson0001 shell]# sh ssh_do_all.sh node.list "chmod +x /etc/rc.d/rc.local"
[root@fayson0001 shell]# sh ssh_do_all.sh node.list "ls -l /etc/rc.d/rc.local"

3.在所有节点的/etc/rc.d/rc.local文件中新增如下内容，以实现开机自动关闭透明大页面。

if test -f /sys/kernel/mm/transparent_hugepage/enabled; then echo never > /sys/kernel/mm/transparent_hugepage/enabled fi if test -f /sys/kernel/mm/transparent_hugepage/defrag; then echo never > /sys/kernel/mm/transparent_hugepage/defrag fi

将该文件同步到所有节点

2.7配置操作系统repo

Fayson用的是AWS的环境，这步是可以省略的，放在这里供物理机部署的兄弟们参考。

1.挂载操作系统iso文件

[root@fayson0001 ~]$ sudo mkdir /media/DVD1
[root@fayson0001 ~]$ sudo mount -o loop 
CentOS-7-x86_64-DVD-1611.iso /media/DVD1/

2.配置操作系统repo

[root@fayson0001 ~]$ sudo vim /etc/yum.repos.d/local_os.repo
[local_iso]    
name=CentOS-$releasever - Media
baseurl=file:///media/DVD1
gpgcheck=0
enabled=1
[root@fayson0001 ~]$ sudo yum repolist

2.8安装httpd服务

1.安装httpd服务

[root@fayson0001 shell]# yum -y install httpd

2.启动httpd服务

[root@fayson0001 shell]# systemctl start httpd

3.安装完httpd后，重新制作操作系统repo，换成http的方式方便其它服务器也可以访问

[root@fayson0001 ~]$ mkdir /var/www/html/iso
[root@fayson0001 ~]$ scp -r /media/DVD1/* /var/www/html/iso/
[root@fayson0001 ~]$ vim /etc/yum.repos.d/os.repo 
[osrepo]
name=os_repo
baseurl=http:// 172.31.2.24/iso/
enabled=true
gpgcheck=false
[root@fayson0001 ~]$ yum repolist

4.修改/etc/httpd/conf/httpd.conf配置文件，在<IfModule mime_module>中修改以下内容

AddType application/x-gzip .gz .tgz .parcel

5.保存httpd.conf的修改，并重启httpd服务

[root@fayson0001 java]# systemctl restart httpd

2.9安装MariaDB

1.安装MariaDB

[root@fayson0001 ~]# yum -y install mariadb
[root@fayson0001 ~]# yum -y install mariadb-server

2.启动并配置MariaDB

[root@fayson0001 ~]# systemctl start mariadb
[root@fayson0001 ~]# systemctl enable mariadb
Created symlink from /etc/systemd/system/multi-user.target.wants/mariadb.service to /usr/lib/systemd/system/mariadb.service.
[root@fayson0001 ~]# /usr/bin/mysql_secure_installation

NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MariaDB
      SERVERS IN PRODUCTION USE!  PLEASE READ EACH STEP CAREFULLY!

In order to log into MariaDB to secure it, we'll need the current
password for the root user.  If you've just installed MariaDB, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.

Enter current password for root (enter for none):
OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MariaDB
root user without the proper authorisation.

Set root password? [Y/n] Y
New password:
Re-enter new password:
Password updated successfully!
Reloading privilege tables..
 ... Success!


By default, a MariaDB installation has an anonymous user, allowing anyone
to log into MariaDB without having to have a user account created for
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] Y
 ... Success!

Normally, root should only be allowed to connect from 'localhost'.  This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] n
 ... skipping.

By default, MariaDB comes with a database named 'test' that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n] Y
 - Dropping test database...
 ... Success!
 - Removing privileges on test database...
 ... Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n] Y
 ... Success!

Cleaning up...

All done!  If you've completed all of the above steps, your MariaDB
installation should now be secure.

Thanks for using MariaDB!

3.建立CM，Hive等需要的表

[root@fayson0001 shell]# mysql -u root -p
Enter password:
Welcome to the MariaDB monitor.  Commands end with ; or \g.
Your MariaDB connection id is 9
Server version: 5.5.64-MariaDB MariaDB Server

Copyright (c) 2000, 2018, Oracle, MariaDB Corporation Ab and others.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

MariaDB [(none)]> create database metastore default character set utf8;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> CREATE USER 'hive'@'%' IDENTIFIED BY 'password';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON metastore. * TO 'hive'@'%';
Query OK, 0 rows affected (0.01 sec)

MariaDB [(none)]> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> create database cm default character set utf8;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> CREATE USER 'cm'@'%' IDENTIFIED BY 'password';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON cm. * TO 'cm'@'%';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> create database rm default character set utf8;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> CREATE USER 'rm'@'%' IDENTIFIED BY 'password';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON rm. * TO 'rm'@'%';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> create database hue default character set utf8;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> CREATE USER 'hue'@'%' IDENTIFIED BY 'password';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON hue. * TO 'hue'@'%';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> create database ranger default character set utf8;
Query OK, 1 row affected (0.00 sec)

MariaDB [(none)]> CREATE USER 'rangeradmin'@'%' IDENTIFIED BY 'password';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> GRANT ALL PRIVILEGES ON ranger. * TO 'rangeradmin'@'%';
Query OK, 0 rows affected (0.00 sec)

MariaDB [(none)]> FLUSH PRIVILEGES;
Query OK, 0 rows affected (0.00 sec)

4.安装JDBC驱动

[root@fayson0001 shell]# mkdir -p /usr/share/java/
[root@fayson0001 shell]# cd
[root@fayson0001 ~]# mv mysql-connector-java-5.1.34.jar /usr/share/java/
[root@fayson0001 ~]# cd /usr/share/java
[root@fayson0001 java]# chmod 777 mysql-connector-java-5.1.34.jar
[root@fayson0001 java]# ln -s mysql-connector-java-5.1.34.jar mysql-connector-java.jar
[root@fayson0001 java]# ll
total 940
-rwxrwxrwx 1 root root 960372 Aug 10 23:09 mysql-connector-java-5.1.34.jar
lrwxrwxrwx 1 root root     31 Aug 10 23:10 mysql-connector-java.jar -> mysql-connector-java-5.1.34.jar

Cloudera Manager安装

3.1配置本地repo源

1.下载CM7.1.2的安装包，地址为：

https://archive.cloudera.com/cm7/7.1.1/redhat7/yum/RPMS/x86_64/cloudera-manager-agent-7.1.1-3274282.el7.x86_64.rpm
https://archive.cloudera.com/cm7/7.1.1/redhat7/yum/RPMS/x86_64/cloudera-manager-daemons-7.1.1-3274282.el7.x86_64.rpm
https://archive.cloudera.com/cm7/7.1.1/redhat7/yum/RPMS/x86_64/cloudera-manager-server-7.1.1-3274282.el7.x86_64.rpm
https://archive.cloudera.com/cm7/7.1.1/redhat7/yum/RPMS/x86_64/cloudera-manager-server-db-2-7.1.1-3274282.el7.x86_64.rpm
https://archive.cloudera.com/cm7/7.1.1/redhat7/yum/RPMS/x86_64/enterprise-debuginfo-7.1.1-3274282.el7.x86_64.rpm
https://archive.cloudera.com/cm7/7.1.1/redhat7/yum/RPMS/x86_64/openjdk8-8.0+232_9-cloudera.x86_64.rpm
https://archive.cloudera.com/cm7/7.1.1/allkeys.asc

2.下载CDH7.1.2的安装包，地址为：

https://archive.cloudera.com/cdh7/7.1.1/parcels/CDH-7.1.1-1.cdh7.1.1.p0.3266817-el7.parcel
https://archive.cloudera.com/cdh7/7.1.1/parcels/CDH-7.1.1-1.cdh7.1.1.p0.3266817-el7.parcel.sha256
https://archive.cloudera.com/cdh7/7.1.1/parcels/manifest.json

3.将Cloudera Manager安装需要的6个rpm包以及一个asc文件下载到本地，放在同一目录，执行createrepo命令生成rpm元数据。

[root@fayson0001 cm7.1.2]# createrepo .

4.配置Web服务器

将上述cdp7.1.2/cm7.1.2目录移动到/var/www/html目录下, 使得用户可以通过HTTP访问这些rpm包。

[root@fayson0001 ~]# mv cm7.1.2/ cdp7.1.2/ /var/www/html/
[root@fayson0001 ~]# cd /var/www/html/
[root@fayson0001 html]# ll  *7.1.2

验证浏览器能否正常访问

5.制作Cloudera Manager的repo源

[root@fayson0001 yum.repos.d]# vim cm.repo
[cmrepo]
name = cm_repo
baseurl = http://172.24.27.124/cm7.1.2
enable = true
gpgcheck = false
[root@fayson0001 yum.repos.d]# yum repolist

6.验证安装JDK

[root@fayson0001 yum.repos.d]# yum -y install openjdk8-8.0+232_9-cloudera.x86_64

3.2安装Cloudera Manager Server

1.通过yum安装Cloudera Manager Server

[root@fayson0001 java]# yum -y install cloudera-manager-server

2.初始化数据库

[root@fayson0001 java]# /opt/cloudera/cm/schema/scm_prepare_database.sh mysql cm cm password

3.启动Cloudera Manager Server

[root@fayson0001 ~]# systemctl start cloudera-scm-server
[root@fayson0001 ~]# systemctl status cloudera-scm-server

4.检查端口是否监听

[root@fayson0001 ~]# netstat -lnpt | grep 7180
tcp        0      0 0.0.0.0:7180            0.0.0.0:*               LISTEN      22048/java

5.通过http://cm_ip:7180/cmf/login访问CM

注意：完全启动成功需要几分钟时间。

CDH安装

4.1CDH集群安装向导

1.admin/admin登录到CM，选择60天试用，点击继续

2.点击继续

3.点击“继续”，输入集群名称，可以使用默认的“Cluster 1”。

4.输入主机IP或者名称，点击搜索找到主机后点击继续

5.选择自定义存储库，输入cm的http地址

“CDH and other software”中选择“Parcel Repositories & Network Settings”，点击“ 更多选项”，点击“-”删除其它所有地址，输入http://172.31.2.24/cdp7.1.2，点击“保存更改”

点击Save & Verify Configuration，验证通过，点击关闭

点击“继续”

6.选择Install a Cloudera-provided version of OpenJDK，点击继续

7.进入下一步配置ssh账号密码，确保所有机器的root密码都一致，然后输入root的密码即可。

8.点击“继续”，进入下一步，安装Cloudera Manager相关到各个节点

等待Agent安装完毕后，自动跳转到下一步开始分发Parcel

9.点击“继续”，进入下一步安装cdh到各个节点

10.点击Inspect Network Performance和Inspect Hosts，检查主机

如果有错误或者黄色警告，查看“显示检查器结果”，并逐项解决，然后“重新运行”检查，直到所有的检查都通过，否则没办法点击继续下一步。

4.2集群设置安装向导

1.选择需要安装的服务，根据需要选择，这里选择自定义服务

2.点击“继续”，进入集群角色分配，一台机器作为管理节点，另外三台机器作为DataNode

注意：Activity Monitor和Telemetry Publisher不用选择任何主机，留空，即不安装，因为用不到。

3.点击“继续”，进入下一步，测试数据库连接 

4.测试成功，点击继续。设置Range相关参数。

以下4个密码，都需要包含至少一个字母与一个数字，为了安装方便都输入admin@123

点击继续

5.进入目录设置，此处使用默认目录，根据实际情况进行目录修改.

6.点击“继续”，进入各个服务启动

7.安装成功后进入home管理界面

4.3组件版本检查

可以看到Hadoop3.1.1，HBase2.2，Hive3.1，Spark2.4，Hue4.5，Impala3.2，Kafka2.4，Kudu1.12，Oozie5.1，Solr8.4，Sqoop1.4.7，Zookeeper3.5.5等。

总结

1.从安装方式上来看，CDP DC7.1.2与CDH6变化不大，这也方便了CDH的用户可以较为快速的迁移到CDP DC7，以及适应CDP DC7的安装与使用。

2.安装条件前置没有任何变化，包括防火墙，SELinux关闭，ntp同步等等。可以参考Fayson之前的文章《​​CDH安装前置准备​​》

3.Cloudera Manager自带的JDK直接提供的是OpenJDK1.8，而不是Oracle JDK，同样CDP DC也不再支持JDK1.7。

4.与安装CDH6一样，CM的安装除了下载rpm包以外，还要下载allkeys.asc文件，否则安装agent的时候会报错。

5.对于离线安装CDH6.x，分发Parcel出现hash校验失败的问题，是因为在CM6中修复了一个bug，让它不再忽略由http服务器发送的“Content-Encoding”的header信息，但是我们在Redhat中安装的httpd服务，当它传输parcel文件时，默认会错误的设置“Content-Encoding”。于是CM server会错误的认为parcel文件已经被httpd压缩并尝试解压缩。所以会导致失败。解决办法是参考2.8章节的，设置httpd的conf文件，AddType application/x-gzip .gz .tgz .parcel，然后重启httpd服务和CM服务。这个问题在beta的时候就已经存在了，具体请参考《​​0350-Redhat7.4安装CDH6.0_beta1时分发Parcel异常分析​​》

6.在创建元数据库时，不需要再创建Navigator和Sentry相关的元数据库，但需要创建Ranger服务相关的元数据库。安装Hue时Oozie服务也不是必须的，所以也可以不创建Oozie数据库。

7.第一次启动Ranger服务时，创建x_portal_user表时会报以下错误：

com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes
SQLException : SQL state: 42000 com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Specified key was too long; max key length is 767 bytes ErrorCode: 1071
2019-11-24 01:12:06,672  [E] ranger_core_db_mysql.sql file import failed!

这是因为索引建立的时候，字符长度超过255，由于utf-8字符最长只能为255，超长之后会报错，所以需要对MySQL做如下配置：

SET GLOBAL innodb_file_per_table = ON,
           innodb_file_format = Barracuda,
           innodb_large_prefix = ON;

参考：

https://stackoverflow.com/questions/29676724/1071-specified-key-was-too-long-max-key-length-is-767-bytes