字数
481 字
阅读时间
3 分钟
前置步骤
安装配置Hadoop
创建资源目录
sudo mkdir /opt/softwares
sudo mkdir /opt/modules
sudo chown -R hadoop:hadoop /opt
sudo chmod -R 755 /opt
cd /opt/softwares上传资源
sudo rz![[Pasted image 20251117144355.png]] 解压资源
sudo tar -zxvf /opt/softwares/hadoop-2.7.2.tar.gz -C /opt/modules/
sudo tar -zxvf /opt/softwares/jdk-8u25-linux-x64.tar.gz -C /opt/modules/添加环境变量
卸载原本的Java环境
查看已安装的Java
rpm -qa | grep -i java常见的输出可能是
java-1.8.0-openjdk-1.8.0.xxx
java-1.8.0-openjdk-headless-1.8.0.xxx根据查找到的包名逐个卸载
sudo yum remove -y java-1.8.0-openjdk
sudo yum remove -y java-1.8.0-openjdk-headless验证是否卸载成功
java -version如果提示如下便是卸载成功
bash: java: command not found添加环境变量
创建hadoop-env.sh文件
sudo vi /etc/profile.d/hadoop-env.sh写入以下内容
export JAVA_HOME=/opt/modules/jdk1.8.0_25
export HADOOP_HOME=/opt/modules/hadoop-2.7.2
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin修改环境变量文件
sudo chmod +x /etc/profile.d/hadoop-env.sh立即生效
source /etc/profile.d/hadoop-env.sh修改Hadoop配置文件
将Java环境路径注入Hadoop安装脚本
sudo echo "export JAVA_HOME=/opt/modules/jdk1.8.0_25/" >> /opt/modules/hadoop-2.7.2/etc/hadoop/hadoop-env.sh
sudo echo "export JAVA_HOME=/opt/modules/jdk1.8.0_25/" >> /opt/modules/hadoop-2.7.2/etc/hadoop/mapred-env.sh
sudo echo "export JAVA_HOME=/opt/modules/jdk1.8.0_25/" >> /opt/modules/hadoop-2.7.2/etc/hadoop/yarn-env.shcore-site.xml
vi /opt/modules/hadoop-2.7.2/etc/hadoop/core-site.xml配置文件如下：
<configuration>
<property>
<name>fs.defaultFS</name>
<value>hdfs://gyy1:9000</value>
</property>
<property>
<name>hadoop.tmp.dir</name>
<value>file:/opt/modules/hadoop-2.7.2/tmp</value>
</property>
</configuration>hdfs-site.xml
vi /opt/modules/hadoop-2.7.2/etc/hadoop/hdfs-site.xml配置文件如下：
<configuration>
<property>
<name>dfs.replication</name>
<value>3</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:/opt/modules/hadoop-2.7.2/tmp/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:/opt/modules/hadoop-2.7.2/tmp/dfs/data</value>
</property>
</configuration>mapred-site.xml
mv /opt/modules/hadoop-2.7.2/etc/hadoop/mapred-site.xml.template /opt/modules/hadoop-2.7.2/etc/hadoop/mapred-site.xml
vi /opt/modules/hadoop-2.7.2/etc/hadoop/mapred-site.xml配置文件如下
<configuration>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>yarn-site.xml
vi /opt/modules/hadoop-2.7.2/etc/hadoop/yarn-site.xml配置文件如下
<configuration>
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
<property>
<name>yarn.resourcemanager.hostname</name>
<value>gyy1</value>
</property>
</configuration>slaves
vi /opt/modules/hadoop-2.7.2/etc/hadoop/slaves配置文件如下
gyy1
gyy2
gyy3在配置完成Hadoop之后，克隆两台虚拟机并03.配置SSH互信
格式化并启动集群
在主机1上运行Hadoop格式化集群
hadoop namenode -format启动Hadoop集群
start-all.sh警告
在关闭主机前，需要在主机1上运行stop-all.sh关闭所有集群，否则集群下次运行会报错
贡献者
lishu620