1.下载
- winutils:https://github.com/s911415/apache-hadoop-3.1.0-winutils,替换bin文件
- hadoop:3.2.0 tar 解压
2.环境变量
- hadoop version验证
问题:Error:java_home is incorrectly set
解决:由于jdk在C盘引起:hadoop-env.cmd 加入 set JAVA_HOME=C:\PROGRA~1\Java\jdk1.8.0_112
3.配置文件
- core-site.xml
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://localhost:900</value>
</property>
</configuration>
- hadoop-env.sh
export JAVA_HOME=C:\Program Files\Java\jdk1.8.0_112\bin
- hdfs-site.xml
在hadoop-3.2.0目录下新键data目录,再新建namenode和datanode目录
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:///D:/BigData/Hadoop/hadoop-3.2.0/data/namenode</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:///D:/BigData/Hadoop/hadoop-3.2.0/data/datanode</value>
</property>
</configuration>
- mapred-site.xml
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
- yarn-site.xml
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.nodemanager.auxservices.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
</configuration>
4.启动
-
bin目录下hdfs namenode –format
没报错说明配置没问题 -
sbin下
start-dfs.cmd
start-yarn.cmd
访问http://localhost:9870/