首页 > 其他分享 >部署Ambari2.7.5 + HDP3.1.5安装

部署Ambari2.7.5 + HDP3.1.5安装

时间：2023-06-01 13:33:49浏览次数：46  
标签：ambari HDP Ambari2.7 HDP3.1 server 部署 yum mysql rpm

 

java安装 
1.java解压安装
cd /opt
tar -zxvf jdk1.8.0_181.tar.gz
2.编辑环境变量配置
vim /etc/profile 
export JAVA_HOME=/opt/jdk1.8.0_231
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
3.生效环境变量配置
source /etc/profile
4.检验JDK配置结果
jave -version



关闭selinux（需重启）
vim /etc/sysconfig/selinux
修改SELINUX=disabled
查看状态：getenforce



关闭防火墙（所有节点）
暂时关闭：service iptables stop
永久关闭：chkconfig iptables off
关闭防火墙：systemctl stop firewalld
查看防火墙的状态：systemctl status firewalld
设置开机不启动：systemctl disable firewalld
查看防火墙服务是否设置开机启动：systemctl is-enabled firewalld



设置最大文件打开数最大进程数（所有节点）
# 编辑配置文件：vim /etc/security/limits.conf，添加如下配置：
* soft nproc 204800
* hard nproc 204800
* soft nofile 655350
* hard nofile 655350



修改Linux系统内核内存参数（所有节点）
# 编辑配置文件：vim /etc/sysctl.conf，添加如下配置：
vm.swappiness=1
vm.min_free_kbytes=4194304
vm.zone_reclaim_mode=0
# 使修改生效：sysctl -p



关闭透明大页THP（需重启）（所有节点）
# 查看透明大页情况：
cat /sys/kernel/mm/transparent_hugepage/defrag
cat /sys/kernel/mm/transparent_hugepage/enabled
# 关闭透明大页：vim /etc/rc.d/rc.local，添加如下配置：
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
  echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
  echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi
# 保存退出，然后赋予rc.local文件执行权限：chmod +x /etc/rc.d/rc.local



#安装时间同步工具（所有节点）
#ntp或者chrony



配置hosts（所有节点）
# vim /etc/hosts
10.99.67.79 hadoop79
10.99.67.83 hadoop83
10.99.67.84 hadoop84
10.99.67.85 hadoop85



配置SSH无密码登陆（主或者节点）
ssh-keygen -t rsa
ssh-copy-id hadoop79
ssh-copy-id hadoop83
ssh-copy-id hadoop84
ssh-copy-id hadoop85


安装一些系统所有必要的工具（所有节点）
yum -y install gcc gcc-c++ kernel-devel libtirpc-devel
yum -y install rpm-build
yum -y install net-tools
yum -y install openssl openssl-devel patch
yum -y install wget curl
yum -y install bash-completion
yum -y install unzip zip
yum -y install libtirpc libtirpc-devel （如果他安装没有，可以是提供的rpm来安装）


安装mysql
tar -xvf mysql-5.7.20-1.el7.x86_64.rpm-bundle.tar
rpm -qa |grep mysql
rpm -qa |grep mariadb
如果找到按下面命令删除对应包
rpm -e mariadb-libs --nodeps
先yum安装部分依赖包
yum -y install perl perl-devel autoconf libaio libaio-devel perl-Time-HiRes perl-JSON perl-Data-Dumper net-tools numactl
安装rpm包
rpm -ivh mysql-community-common-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-libs-compat-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-embedded-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-client-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-devel-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-embedded-compat-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-embedded-devel-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-server-5.7.20-1.el7.x86_64.rpm
rpm -ivh mysql-community-test-5.7.20-1.el7.x86_64.rpm
查看状态并启动mysqld
# 查看mysql服务
service mysqld status
#systemctl status mysqld.service
# 启动mysql服务
service mysqld start
#systemctl start mysqld.service
# 查看密码
cat /var/log/mysqld.log | grep password

# 修改root密码，密码登录：mysql -u root
# 首次登录必须先修改密码（强密码）
alter user 'root'@'localhost' identified by 'root__123AbCdHjKl';

SHOW variables LIKE 'validate_password%';
set global validate_password_policy=0;
set global validate_password_length=4;
# 可以修改为弱密码啦
#ALTER USER 'root'@'localhost' IDENTIFIED BY "bigdata123";
#flush privileges; # 更新权限

use mysql;
update user set host = '%' where user ='root'; # 配置可远程登录
flush privileges; # 更新权限
exit; # 退出

# 查看mysql服务的编码
#show variables like '%char%';
# 修改MySQL配置文件：vim /etc/my.cnf
# 在[mysqld]下添加
collation_server=utf8_general_ci
character_set_server=utf8
default-storage-engine=INNODB
# 在[client]下添加（如果没有[client]，则创建）
default_character-set=utf8

[mysqld]
character-set-client-handshake = FALSE
character-set-server = utf8mb4
collation-server = utf8mb4_unicode_ci
default-storage-engine=INNODB
init_connect='SET NAMES utf8mb4'
[client] 
default-character-set = utf8mb4 
[mysql] 
default-character-set = utf8mb4

# 启动mysql服务
systemctl restart mysqld

# 设置开机启动
systemctl enable mysqld

 

 

 

yum install yum-utils createrepo yum-plugin-priorities

yum -y install httpd
# 安装服务
yum install httpd -y
# 启动httpd服务：systemctl start httpd
# 查看httpd状态：systemctl status httpd
# 设置开启启动：systemctl enable httpd
# 查看是否开机启动：systemctl is-enabled httpd



# 解压Ambari压缩包
tar -zxvf /root/ambari-2.7.5.0-centos7.tar.gz -C /var/www/html/
# 解压HDP压缩包
tar -zxvf /root/HDP-3.1.5.0-centos7-rpm.tar.gz -C /var/www/html/
# 解压HDP-UTILS压缩包
tar -zxvf /root/HDP-UTILS-1.1.0.22-centos7.tar.gz -C /var/www/html/
# 解压HDP-GPL压缩包
tar -zxvf /root/HDP-GPL-3.1.5.0-centos7-gpl.tar.gz -C /var/www/html/

# ambari库源
# cat ambari.repo 
[ambari-2.7.5.0-72]
name=ambari-2.7.5.0-72
baseurl=http://10.99.67.79/ambari/centos7/2.7.5.0-72/
path=/
enabled=1
gpgkey= http://10.99.67.79/ambari/centos7/2.7.5.0-72/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
gpgcheck=1
priority=1
# HDP库源
# cat HDP.repo 
[HDP-3.1.5]
name=HDP-3.1.5
baseurl=http://10.99.67.79/HDP/centos7/3.1.5.0-152/
path=/
enabled=1
gpgkey=http://10.99.67.70/HDP/centos7/3.1.5.0-152/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
gpgcheck=1
priority=1
# HDP-UTILS库源
# cat HDP-UTILS.repo 
[HDP-UTILS-1.1.0.22]
name=HDP-UTILS-1.1.0.22
baseurl=http://10.99.67.79/HDP-UTILS/centos7/1.1.0.22/
path=/
enabled=1
gpgkey=http://10.99.67.79/HDP-UTILS/centos7/1.1.0.22/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
gpgcheck=1
priority=1
# HDP-GPL库源
# cat HDP-GPL.repo 
[HDP-3.1.5-GPL]
name=HDP-3.1.5-GPL
baseurl=http://10.99.67.79/HDP-GPL/centos7/3.1.5.0-152/
path=/
enabled=1
gpgkey=http://10.99.67.79/HDP-GPL/centos7/3.1.5.0-152/RPM-GPG-KEY/RPM-GPG-KEY-Jenkins
gpgcheck=1
priority=1



scp ambari.repo root@hadoop79:/etc/yum.repos.d/
scp HDP.repo root@hadoop79:/etc/yum.repos.d/
scp HDP-UTILS.repo root@hadoop79:/etc/yum.repos.d/
scp HDP-GPL.repo root@hadoop79:/etc/yum.repos.d/

scp ambari.repo root@hadoop83:/etc/yum.repos.d/
scp HDP.repo root@hadoop83:/etc/yum.repos.d/
scp HDP-UTILS.repo root@hadoop83:/etc/yum.repos.d/
scp HDP-GPL.repo root@hadoop83:/etc/yum.repos.d/

scp ambari.repo root@hadoop84:/etc/yum.repos.d/
scp HDP.repo root@hadoop84:/etc/yum.repos.d/
scp HDP-UTILS.repo root@hadoop84:/etc/yum.repos.d/
scp HDP-GPL.repo root@hadoop84:/etc/yum.repos.d/

scp ambari.repo root@hadoop85:/etc/yum.repos.d/
scp HDP.repo root@hadoop85:/etc/yum.repos.d/
scp HDP-UTILS.repo root@hadoop85:/etc/yum.repos.d/
scp HDP-GPL.repo root@hadoop85:/etc/yum.repos.d/

# yum clean all
# yum repolist
# yum makecache



安装ambari-server
yum -y install ambari-server

复制mysql jdbc驱动到/var/lib/ambari-server/resources/
mkdir /usr/share/java
cp mysql-connector-java-5.1.47.jar /usr/share/java/mysql-connector-java.jar
cp /usr/share/java/mysql-connector-java.jar /var/lib/ambari-server/resources/

配置/etc/ambari-server/conf/ambari.properties，添加如下行
server.jdbc.driver.path=/usr/share/java/mysql-connector-java.jar

执行
ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar

初始化ambari-server
ambari-server setup





# ambari-server setup --jdbc-db=mysql --jdbc-driver=/usr/share/java/mysql-connector-java.jar
Using python  /usr/bin/python
Setup ambari-server
Copying /usr/share/java/mysql-connector-java.jar to /var/lib/ambari-server/resources/mysql-connector-java.jar
If you are updating existing jdbc driver jar for mysql with mysql-connector-java.jar. Please remove the old driver jar, from all hosts. Restarting services that need the driver, will automatically copy the new jar to the hosts.
JDBC driver was successfully initialized.
Ambari Server 'setup' completed successfully.


# ambari-server setup
Using python  /usr/bin/python
Setup ambari-server
Checking SELinux...
SELinux status is 'disabled'
Customize user account for ambari-server daemon [y/n] (n)? y
Enter user account for ambari-server daemon (root):
Adjusting ambari-server permissions and ownership...
Checking firewall status...
Checking JDK...
[1] Oracle JDK 1.8 + Java Cryptography Extension (JCE) Policy Files 8
[2] Custom JDK
==============================================================================
Enter choice (1): 2
WARNING: JDK must be installed on all hosts and JAVA_HOME must be valid on all hosts.
WARNING: JCE Policy files are required for configuring Kerberos security. If you plan to use Kerberos,please make sure JCE Unlimited Strength Jurisdiction Policy Files are valid on all hosts.
Path to JAVA_HOME: /opt/jdk1.8.0_231
Validating JDK on Ambari Server...done.
Check JDK version for Ambari Server...
JDK version found: 8
Minimum JDK version is 8 for Ambari. Skipping to setup different JDK for Ambari Server.
Checking GPL software agreement...
GPL License for LZO: https://www.gnu.org/licenses/old-licenses/gpl-2.0.en.html
Enable Ambari Server to download and install GPL Licensed LZO packages [y/n] (y)? 
Completing setup...
Configuring database...
Enter advanced database configuration [y/n] (n)? y
Configuring database...
==============================================================================
Choose one of the following options:
[1] - PostgreSQL (Embedded)
[2] - Oracle
[3] - MySQL / MariaDB
[4] - PostgreSQL
[5] - Microsoft SQL Server (Tech Preview)
[6] - SQL Anywhere
[7] - BDB
==============================================================================
Enter choice (1): 3
Hostname (localhost): 
Port (3306): 
Database name (ambari): 
Username (ambari): 
Enter Database Password (bigdata): ambari123
Re-enter password: 
Configuring ambari database...
Configuring remote database connection properties...
WARNING: Before starting Ambari Server, you must run the following DDL directly from the database shell to create the schema: /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql
Proceed with configuring remote database connection properties [y/n] (y)? y
Extracting system views...
ambari-admin-2.7.5.0.72.jar
....
Ambari repo file doesn't contain latest json url, skipping repoinfos modification
Adjusting ambari-server permissions and ownership...
Ambari Server 'setup' completed successfully.





-- 创建ambari数据库
CREATE DATABASE ambari; 
-- 创建ambari用户
CREATE USER 'ambari'@'%' IDENTIFIED BY 'ambari123'; 
-- 给创建的ambari赋权
GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'%'; 

#CREATE USER 'ambari'@'localhost' IDENTIFIED BY 'ambari123'; 
#GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'localhost'; 
#CREATE USER 'ambari'@'hadoop79' IDENTIFIED BY 'ambari123'; 
#GRANT ALL PRIVILEGES ON *.* TO 'ambari'@'hadoop79';
-- 刷新权限
flush privileges;

-- 导入ambari的数据库
use ambari;
source /var/lib/ambari-server/resources/Ambari-DDL-MySQL-CREATE.sql

-- 验证数据表
show tables;
-- 验证用户
use mysql; 
select host,user from user where user='ambari';

-- #创建其它大数据组件的库
CREATE DATABASE hive; 
use hive; 
CREATE USER 'hive'@'%' IDENTIFIED BY 'hive123'; 
GRANT ALL PRIVILEGES ON *.* TO 'hive'@'%'; 
#CREATE USER 'hive'@'localhost' IDENTIFIED BY 'hive123'; 
#GRANT ALL PRIVILEGES ON *.* TO 'hive'@'localhost'; 
#CREATE USER 'hive'@'hadoop79' IDENTIFIED BY 'hive123'; 
#GRANT ALL PRIVILEGES ON *.* TO 'hive'@'hadoop79'; 
FLUSH PRIVILEGES;

CREATE DATABASE oozie; 
use oozie; 
CREATE USER 'oozie'@'%' IDENTIFIED BY 'oozie123'; 
GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'%'; 
#CREATE USER 'oozie'@'localhost' IDENTIFIED BY 'oozie123'; 
#GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'localhost'; 
#CREATE USER 'oozie'@'hadoop79' IDENTIFIED BY 'oozie123'; 
#GRANT ALL PRIVILEGES ON *.* TO 'oozie'@'hadoop79'; 
FLUSH PRIVILEGES;





安装ambari-agent(所有节点执行)
yum -y install ambari-agent
systemctl start ambari-agent


启动ambari服务
#启动
ambari-server start
#查看状态
#ambari-server status
#停止
ambari-server stop








http://10.99.67.79:8080/#/main/dashboard/metrics (admin,admin)
http://10.99.67.79:8080/#/main/services/HDFS/summary
http://10.99.67.79:50070/dfshealth.html#tab-overview
http://10.99.67.79:3000/ (admin,admin)

http://10.99.67.79:9000/#/legacy-ui-disabled
 Notification!
The SmartSense legacy UI previously available on port 9000 is currently disabled. Please use the Ambari view for accessing SmartSense.

 

 

 

 

 

 

 

 

 

ambari-2.7.5 + HDP-3.1.5 其他安装资料


https://www.modb.pro/db/179147
https://zhuanlan.zhihu.com/p/523062467

 

标签：ambari,HDP,Ambari2.7,HDP3.1,server,部署,yum,mysql,rpm
From： https://blog.51cto.com/lenglingx/6393404

相关文章

  • 百度飞桨(PaddlePaddle) - PP-OCRv3 文字检测识别系统 基于 Paddle Serving快速使用（服
    目录安装Docker安装PaddleOCR安装准备PaddleServing的运行环境，模型转换PaddleServingpipeline部署重启测试百度飞桨(PaddlePaddle)-PP-OCRv3文字检测识别系统预测部署简介与总览百度飞桨(PaddlePaddle)-PP-OCRv3文字检测识别系统PaddleInference模型推理（离线部署）......
  • docker部署PaddleOCR流程
    先安装docker，输入docker-v后即可开始部署paddleocr1.容器环境使用ubuntu16.04dockerpullubuntu:16.042.启动容器使用-itd可以在后台运行正常-it就可以dockerrun-itd--name="paddleocr"ubuntu:16.04bash3.修改apt源（这里设置为清华源还不错）sed-i's/archive.ubun......
  • 搭建jenkins实现自动化部署
    一、安装jenkins1、添加yumrepos，然后安装12sudo wget-O /etc/yum.repos.d/jenkins.repohttps://pkg.jenkins.io/redhat/jenkins.reposudo rpm--import https://pkg.jenkins.io/redhat/jenkins.io.key注：如果上边的执行成功就不用再执行这两......
  • ES部署+Kibana部署+IK分词器(单节点)
    一、Elasticsearch7.9.1官网下载地址：ElasticsearchES7.9.1，内置了JDK，我是centos系统，下载LINUXX86_64下载、解压、放到/usr/es/下二、配置cd/usr/es&&llvi/elasticsearch/config/elasticsearch.yml#打开节点名称node.name:node-1#这里的node-1为node-name配置的值clus......
  • 3、实战案例：部署基于JAVA的博客系统JPress
    官方网站：http://www.jpress.io/安装包下载第一步：[root@ubuntu2004]#mkdir/data/jpress/-p创建网站数据存放的目录，ROOT可以不建把下载好的包拉进/data/jpress/目录，并改名为ROOT.war，它会自动解压成一个ROOT文件夹[root@ubuntu2004jpress]#rz-Erzwaitingtoreceive.[root@......
  • docker-compose一键部署jdk+tomcat+jmeter+ant+jenkins测试环境
    1.编写DokcerfileFROMcentos:7MAINTAINERhuangkai<[email protected]>RUNyum-yinstallvim\&&yum-yinstallnet-tools\&&yuminstalldejavu-sans-fontsfontconfig-yADDjdk-11_linux-x64_bin.tar.gz/usr/local/ADDapache-tomcat-9.0......
  • flink1.14.5集群（flink on yarn）部署1
    先安装hadoop,yarn,zookeeper 配置环境变量vim/etc/profile(注意新增了HADOOP_CLASSPATH变量)exportHADOOP_CLASSPATH=`/home/opt/hadoop-2.9.2/bin/hadoopclasspath`exportFLINK_HOME=/home/opt/flink-1.14.5exportPATH=$PATH:$FLINK_HOME/binsource/etc/profile......
  • sonarqube部署安装
    镜像准备postgres:11.6sonarqube:9.0.0-communitybusybox:1.27.1安装postgressonarqube7.9及以上使用postgres为数据库使用 PVCapiVersion:v1kind:PersistentVolumeClaimmetadata:name:sonar-mysqlnamespace:demospec:accessModes:-ReadWriteMan......
  • Inofity+Rsync实时同步服务部署
    第一步：部署rsync服务1.1服务端：1.检查rsync是否安装[root@rsync-backup~]#rpm-qarsyncrsync-3.1.2-12.el7_9.x86_642.对rsync服务进行配置[root@rsync-backup~]#cat/etc/rsyncd.confuid=rsyncgid=rsyncfakesuper=yesusechroot=nomaxconnections=......
  • Razor Pages本地IIS服务器部署流程及部分问题解决方法
     记录一下自己在本地IIS服务器部署的基本流程：添加IIS服务器控制面板>>程序和功能 启用或关闭windows功能>>勾选相关功能   网站部署将项目发布（publish）至本地文件夹：在包含.sln文件的目录下打开终端，输入dotnetpublish-cdebug--no-self-contained......

赞助商

阅读排行

网站主页关于我们联系我们网站地图

本网站内容转载自其他媒体，侵权联系[admin##ips99.com]。

Copyright © 2020-2023 IPS99 版权所有 IPS99